返回时间轴
2017-12

"Attention Is All You Need" — the Transformer architecture is born

Capability Breakthrough

事件摘要

Eight Google researchers published 'Attention Is All You Need' at NeurIPS 2017, introducing the Transformer architecture. By dispensing with recurrence and convolution entirely and relying solely on self-attention mechanisms, the Transformer achieved state-of-the-art translation results while training significantly faster. The paper has been cited over 250,000 times, and the Transformer became the foundation of every major AI system since—from GPT and BERT to AlphaFold and modern speech models.

影响评估

  • Capability Leap +3 · Long-term

    Replaced RNNs and CNNs as the dominant sequence-processing architecture across NLP, vision, speech, and biology. The self-attention mechanism proved so general and scalable that it became the universal compute substrate for modern AI—every major AI system since 2018 (BERT, GPT series, Claude, Gemini, AlphaFold) uses Transformers at its core.

    Affected Groups: all AI researchers, NLP researchers, computer vision researchers, computational biologists

  • Economic Disruption +3 · Long-term

    Enabled the scaling laws that led to GPT-3, ChatGPT, and all subsequent large language models. The authors collectively founded or joined companies now worth tens of billions (Cohere, Character.AI, Essential AI). The architecture's parallelizability made GPU/TPU training efficient at unprecedented scale, directly shaping the modern AI hardware market.

    Affected Groups: tech industry, investors, hardware manufacturers, startups

  • Paradigm Shift +3 · Long-term

    'Attention Is All You Need' became a scientific meme. Its audacious title captured a truth that proved deeper than the authors knew: attention was sufficient not just for translation, but for vision, protein folding, reasoning, and generation. The paper marks the clearest 'before and after' line in modern AI research methodology.

    Affected Groups: entire AI field, researchers, engineers

共识度与来源

重要度 L3
分类 Capability Breakthrough
共识度 Broad Consensus
影响指数 10/10

关联事件

← Origins
2012-09-30 L3

AlexNet wins ImageNet, igniting the deep learning revolution

Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton's AlexNet crushed the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, achieving a top-5 error rate of 15.3%—over 10 percentage points better than the runner-up's 26.2%. The key innovation: training a deep convolutional neural network on two NVIDIA GTX 580 GPUs. This single result convinced the computer vision community that deep learning on GPUs was the future, ending decades of hand-engineered feature methods and launching the modern AI era.

2014-06 L1

Ian Goodfellow invents Generative Adversarial Networks (GANs)

Ian Goodfellow published 'Generative Adversarial Nets,' introducing a novel framework where two neural networks—a generator and a discriminator—compete against each other in a zero-sum game. The generator learns to produce increasingly realistic samples while the discriminator learns to distinguish real from fake. GANs became the dominant paradigm for generative AI until supplanted by diffusion models in the mid-2020s, fundamentally changing how the field thought about unsupervised learning.

2014-09-01 L2

Attention mechanism proposed — the foundation of modern AI architectures

Bahdanau, Cho, and Bengio publish 'Neural Machine Translation by Jointly Learning to Align and Translate', introducing the attention mechanism that allows neural networks to dynamically focus on relevant parts of input when generating output. This innovation directly leads to the Transformer architecture.

2015-11-09 L2

Google open-sources TensorFlow — deep learning becomes accessible

Google releases TensorFlow as an open-source deep learning framework under Apache 2.0 license. It quickly becomes the most widely adopted machine learning library, powering research and production systems across industries.

2015-12-10 L2

ResNet wins ImageNet — residual connections enable ultra-deep networks

Kaiming He et al. publish Deep Residual Learning for Image Recognition, introducing residual connections that allow training of neural networks with over 150 layers. ResNet achieves 3.57% top-5 error on ImageNet, surpassing human-level performance for the first time on this benchmark.

2016-03-15 L3

AlphaGo defeats Lee Sedol, shattering the belief that Go was decades beyond AI's reach

DeepMind's AlphaGo defeated 18-time world Go champion Lee Sedol 4–1 in a five-game match held in Seoul, South Korea, from March 9–15, 2016. The match was watched by over 200 million people worldwide, mostly in East Asia. Lee Sedol's sole victory in Game 4 became legendary—his 'God's Touch' move 78 forced a rare AlphaGo miscalculation. But the overall result was decisive: Go, long considered the grand challenge of AI due to its astronomical search space, had fallen decades earlier than most experts predicted.

→ Successors
2018-10-11 L1

Google releases BERT, transforming NLP with bidirectional pre-training

Google AI published 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,' introducing a method that pre-trained a Transformer model bidirectionally on a large text corpus using masked language modeling. BERT achieved state-of-the-art results on 11 NLP benchmarks within months of release, and its open-source model and pre-trained weights made transfer learning in NLP accessible to anyone, launching the 'BERT era' of NLP.

2020-06-11 L3

OpenAI releases GPT-3, proving that scaling language models unlocks emergent capabilities

In June 2020, OpenAI published GPT-3 (Generative Pre-trained Transformer 3), a 175-billion-parameter language model that demonstrated remarkable few-shot learning—the ability to perform novel tasks from just a handful of examples without fine-tuning. GPT-3 could write essays, generate code, translate languages, and answer questions at a quality that often blurred the line between human and machine output. The paper 'Language Models are Few-Shot Learners' (Brown et al.) validated the scaling hypothesis: larger models trained on more data get qualitatively better, not just incrementally better.

2022-11-30 L3

OpenAI launches ChatGPT, bringing AI to 100 million users in two months

OpenAI released ChatGPT, a conversational AI product based on GPT-3.5, free to the public on November 30, 2022. It reached 1 million users in 5 days and 100 million in 2 months, becoming the fastest-growing consumer application in history. The launch marked the moment AI became a mainstream conversation—not just for researchers and tech enthusiasts, but for policymakers, teachers, artists, and the general public worldwide.

2024-02-15 L2

OpenAI previews Sora, the first text-to-video model that looked like it was working

On February 15, 2024, OpenAI unveiled Sora, a text-to-video generation model that produced strikingly coherent, photorealistic video clips up to one minute long from text prompts. Unlike previous text-to-video efforts that produced glitchy, short, and disjointed output, Sora's results demonstrated object permanence, physical consistency, and cinematic quality. The announcement was a 'GPT-1 moment for video'—the first time video generation seemed to genuinely work.

2024-09-12 L2

OpenAI releases o1, the first reasoning model that 'thinks before it answers'

On September 12, 2024, OpenAI released o1 (codenamed 'Strawberry'), a large language model trained to spend more time 'thinking' before generating responses — using a chain-of-thought reasoning process internally. o1 scored in the 89th percentile on the International Mathematics Olympiad qualifier and reached PhD-level accuracy on physics, chemistry, and biology benchmarks. It demonstrated that scaling compute at inference time — not just training time — could unlock new capabilities, opening a second axis of AI progress.

2024-10-09 L2

AlphaFold developers win Nobel Prize in Chemistry, the first Nobel awarded for AI-driven scientific discovery

On October 9, 2024, the Nobel Prize in Chemistry was awarded to Demis Hassabis and John Jumper of Google DeepMind for AlphaFold, an AI system that predicts protein structures from amino acid sequences, and to David Baker for computational protein design. AlphaFold has predicted the 3D structure of virtually all 200 million known proteins, serving over 3 million researchers in 190+ countries. The award marked the first time a Nobel Prize recognized an AI system as a primary instrument of scientific discovery, legitimizing AI's role in fundamental science at the highest level.

Local Lineage "Attention Is All Yo... 2017-12 · L3 Attention mechanis... 2014-09-01 · L2 Google open-source... 2015-11-09 · L2 ResNet wins ImageN... 2015-12-10 · L2 AlphaGo defeats Le... 2016-03-15 · L3 Google releases BE... 2018-10-11 · L1 OpenAI releases GP... 2020-06-11 · L3 OpenAI launches Ch... 2022-11-30 · L3 OpenAI previews So... 2024-02-15 · L2