The rapid evolution of transformers and large language models, culminating in ChatGPT, fundamentally disrupted natural language processing, triggering debates about understanding, scale, and the field's identity.
- The BERT transformer model launched a benchmark boom and rapid progress in NLP.
- Scaling became the dominant approach after GPT-3 demonstrated powerful capabilities that surprised many researchers.
- The “understanding wars” split the field into camps debating whether LLMs truly understand or merely mimic language.
- ChatGPT’s release in November 2022 erased entire research directions and forced the field to reorient.
- The influx of corporate funding and public attention blurred boundaries between NLP and AI, raising questions about the field’s future as a science.
Prologue: Before the Flood
In 2017, the transformer architecture was introduced in the paper “Attention Is All You Need,” but many researchers initially dismissed it as a hack. Google’s BERT (2018) soon shattered performance records, triggering a “BERTology” boom and a race to create harder benchmarks. This period saw the rise of scaling as a key driver of progress.
I. The Wars of the Roses (2020–22)
The release of GPT-3 in 2020 caused an existential crisis for many researchers as it could perform tasks that had taken years in moments, but its corporate secrecy and scale sparked the “understanding wars.” The paper “On the Dangers of Stochastic Parrots” injected moral urgency and divided the community into pro- and anti-LLM factions. A field-wide survey in 2022 confirmed that NLP was in crisis.
II. Chixculub (November 2022 through 2023)
ChatGPT’s launch in November 2022 wiped out entire research categories, leading to dread at the EMNLP conference. Graduate students formed support groups, and professors faced new attention from media and government. The field was flooded with papers on prompting and evaluation, transforming NLP into AI.
III. Mutatis Mutandis (2024–25) and Epilogue
Researchers became “LLM-ologists,” studying proprietary systems with limited access, while corporate interests and hype distorted scientific discourse. Some argued that scaling and corporate influence undermined reproducibility. Opinions differ on whether LLMs represent a paradigm shift: some see a dead end, others a fundamental reorientation, but all agree the field has been permanently altered.
Read this at any depth.
Install Depth and pick your level — Glance for a sentence, Summary for the gist, Read for the full take. Free daily quota, no signup needed.
Add to Chrome