Arcs Directions
Back to Timeline
2017-12

Capability Breakthrough

Event Summary

Attention Is All You Need: the 2017 paper that introduced the Transformer. Cited 250K+ times, it founded GPT, BERT, AlphaFold, and all modern AI.

Impact Assessment

  • Capability Leap +3 · Long-term

    Replaced RNNs and CNNs as the dominant sequence-processing architecture across NLP, vision, speech, and biology. The self-attention mechanism proved so general and scalable that it became the universal compute substrate for modern AI—every major AI system since 2018 (BERT, GPT series, Claude, Gemini, AlphaFold) uses Transformers at its core.

    Affected Groups: all AI researchers, NLP researchers, computer vision researchers, computational biologists

  • Economic Disruption +3 · Long-term

    Enabled the scaling laws that led to GPT-3, ChatGPT, and all subsequent large language models. The authors collectively founded or joined companies now worth tens of billions (Cohere, Character.AI, Essential AI). The architecture's parallelizability made GPU/TPU training efficient at unprecedented scale, directly shaping the modern AI hardware market.

    Affected Groups: tech industry, investors, hardware manufacturers, startups

  • Paradigm Shift +3 · Long-term

    'Attention Is All You Need' became a scientific meme. Its audacious title captured a truth that proved deeper than the authors knew: attention was sufficient not just for translation, but for vision, protein folding, reasoning, and generation. The paper marks the clearest 'before and after' line in modern AI research methodology.

    Affected Groups: entire AI field, researchers, engineers

Consensus & Sources

Significance L3
Category Capability Breakthrough
Consensus Broad Consensus
Impact Index 8/10

Related Events

← Origins
2012-09-30 L3

AlexNet wins ImageNet and ignites the deep learning revolution

Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton's AlexNet crushed the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, achieving a top-5 error rate of 15.3%—over 10 percentage points better than the runner-up's 26.2%. The key innovation: training a deep convolutional neural network on two NVIDIA GTX 580 GPUs. This single result convinced the computer vision community that deep learning on GPUs was the future, ending decades of hand-engineered feature methods and launching the modern AI era.

2014-06 L1

Ian Goodfellow invents Generative Adversarial Networks (GANs)

Ian Goodfellow published 'Generative Adversarial Nets,' introducing a novel framework where two neural networks—a generator and a discriminator—compete against each other in a zero-sum game. The generator learns to produce increasingly realistic samples while the discriminator learns to distinguish real from fake. GANs became the dominant paradigm for generative AI until supplanted by diffusion models in the mid-2020s, fundamentally changing how the field thought about unsupervised learning.

2014-09-01 L2

Attention mechanism proposed — the foundation of modern AI architectures

Bahdanau, Cho, and Bengio publish 'Neural Machine Translation by Jointly Learning to Align and Translate', introducing the attention mechanism that allows neural networks to dynamically focus on relevant parts of input when generating output. This innovation directly leads to the Transformer architecture.

2015-11-09 L2

Google open-sources TensorFlow — deep learning becomes accessible

Google releases TensorFlow as an open-source deep learning framework under Apache 2.0 license. It quickly becomes the most widely adopted machine learning library, powering research and production systems across industries.

2015-12-10 L2

ResNet wins ImageNet — residual connections enable ultra-deep networks

Kaiming He et al. publish Deep Residual Learning for Image Recognition, introducing residual connections that allow training of neural networks with over 150 layers. ResNet achieves 3.57% top-5 error on ImageNet, surpassing human-level performance for the first time on this benchmark.

2016-03-15 L3

AlphaGo defeats Lee Sedol, a milestone once believed decades away

DeepMind's AlphaGo defeated 18-time world Go champion Lee Sedol 4–1 in a five-game match held in Seoul, South Korea, from March 9–15, 2016. The match was watched by over 200 million people worldwide, mostly in East Asia. Lee Sedol's sole victory in Game 4 became legendary—his 'God's Touch' move 78 forced a rare AlphaGo miscalculation. But the overall result was decisive: Go, long considered the grand challenge of AI due to its astronomical search space, had fallen decades earlier than most experts predicted.

→ Successors
2018-10-11 L1

Google releases BERT and transforms NLP with bidirectional pre-training

Google AI published 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,' introducing a method that pre-trained a Transformer model bidirectionally on a large text corpus using masked language modeling. BERT achieved state-of-the-art results on 11 NLP benchmarks within months of release, and its open-source model and pre-trained weights made transfer learning in NLP accessible to anyone, launching the 'BERT era' of NLP.

2020-06-11 L3

OpenAI releases GPT-3, demonstrating that scaling language models unlocks emergent capabilities

In June 2020, OpenAI published GPT-3 (Generative Pre-trained Transformer 3), a 175-billion-parameter language model that demonstrated remarkable few-shot learning—the ability to perform novel tasks from just a handful of examples without fine-tuning. GPT-3 could write essays, generate code, translate languages, and answer questions at a quality that often blurred the line between human and machine output. The paper 'Language Models are Few-Shot Learners' (Brown et al.) validated the scaling hypothesis: larger models trained on more data get qualitatively better, not just incrementally better.

2022-11-30 L3

OpenAI launches ChatGPT, reaching 100 million users in two months

OpenAI released ChatGPT, a conversational AI product based on GPT-3.5, free to the public on November 30, 2022. It reached 1 million users in 5 days and 100 million in 2 months, becoming the fastest-growing consumer application in history. The launch marked the moment AI became a mainstream conversation—not just for researchers and tech enthusiasts, but for policymakers, teachers, artists, and the general public worldwide.

2024-02-15 L2

OpenAI previews Sora, the first text-to-video model that looked like it was working

On February 15, 2024, OpenAI unveiled Sora, a text-to-video generation model that produced strikingly coherent, photorealistic video clips up to one minute long from text prompts. Unlike previous text-to-video efforts that produced glitchy, short, and disjointed output, Sora's results demonstrated object permanence, physical consistency, and cinematic quality. The announcement was a 'GPT-1 moment for video'—the first time video generation seemed to genuinely work.

2024-09-12 L2

OpenAI releases o1, the first reasoning model that 'thinks before it answers'

On September 12, 2024, OpenAI released o1 (codenamed 'Strawberry'), a large language model trained to spend more time 'thinking' before generating responses — using a chain-of-thought reasoning process internally. o1 scored in the 89th percentile on the International Mathematics Olympiad qualifier and reached PhD-level accuracy on physics, chemistry, and biology benchmarks. It demonstrated that scaling compute at inference time — not just training time — could unlock new capabilities, opening a second axis of AI progress.

2024-10-09 L2

AlphaFold developers win Nobel Prize in Chemistry, the first Nobel awarded for AI-driven scientific discovery

On October 9, 2024, the Nobel Prize in Chemistry was awarded to Demis Hassabis and John Jumper of Google DeepMind for AlphaFold, an AI system that predicts protein structures from amino acid sequences, and to David Baker for computational protein design. AlphaFold has predicted the 3D structure of virtually all 200 million known proteins, serving over 3 million researchers in 190+ countries. The award marked the first time a Nobel Prize recognized an AI system as a primary instrument of scientific discovery, legitimizing AI's role in fundamental science at the highest level.

Local Lineage "Attention Is All Yo... 2017-12 · L3 Attention mechanis... 2014-09-01 · L2 Google open-source... 2015-11-09 · L2 ResNet wins ImageN... 2015-12-10 · L2 AlphaGo defeats Le... 2016-03-15 · L3 Google releases BE... 2018-10-11 · L1 OpenAI releases GP... 2020-06-11 · L3 OpenAI launches Ch... 2022-11-30 · L3 OpenAI previews So... 2024-02-15 · L2