返回时间轴
2014-09-01

Attention mechanism proposed — the foundation of modern AI architectures

Capability Breakthrough Products & Tools

事件摘要

Bahdanau, Cho, and Bengio publish 'Neural Machine Translation by Jointly Learning to Align and Translate', introducing the attention mechanism that allows neural networks to dynamically focus on relevant parts of input when generating output. This innovation directly leads to the Transformer architecture.

影响评估

  • Paradigm Shift +2 · Long-term

    Attention is the core innovation behind Transformer and all subsequent large language models.

    Affected Groups: ai researchers, nlp practitioners, deep learning community

  • Capability Leap +2 · Medium-term

    Enabled neural machine translation to surpass traditional statistical methods, especially on long sentences.

    Affected Groups: nlp researchers, translation services, language industry

  • Access Democratization +1 · Long-term

    Attention mechanisms became the foundation of billions of daily AI interactions through search, translation, and chatbots.

    Affected Groups: general public, developers

共识度与来源

重要度 L2
分类 Capability Breakthrough / Products & Tools
共识度 Broad Consensus
影响指数 10/10

关联事件

→ Successors
2017-12 L3

"Attention Is All You Need" — the Transformer architecture is born

Eight Google researchers published 'Attention Is All You Need' at NeurIPS 2017, introducing the Transformer architecture. By dispensing with recurrence and convolution entirely and relying solely on self-attention mechanisms, the Transformer achieved state-of-the-art translation results while training significantly faster. The paper has been cited over 250,000 times, and the Transformer became the foundation of every major AI system since—from GPT and BERT to AlphaFold and modern speech models.

2018-10-11 L1

Google releases BERT, transforming NLP with bidirectional pre-training

Google AI published 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,' introducing a method that pre-trained a Transformer model bidirectionally on a large text corpus using masked language modeling. BERT achieved state-of-the-art results on 11 NLP benchmarks within months of release, and its open-source model and pre-trained weights made transfer learning in NLP accessible to anyone, launching the 'BERT era' of NLP.

2019-02-14 L1

OpenAI says GPT-2 is 'too dangerous to release' — the first mainstream AI safety debate goes global

On February 14, 2019, OpenAI announced GPT-2, a language model capable of generating remarkably coherent text. But instead of releasing it fully, OpenAI withheld the complete model, citing 'concerns about malicious applications.' The decision sparked an intense global debate: was OpenAI responsibly cautious or grandstanding for attention? The staged rollout — releasing progressively larger versions over months — became a template for AI safety disclosure. It also marked the first time ordinary people heard the argument that AI could be too dangerous to release, permanently changing how the public thought about AI risk.

Local Lineage Attention mechanism ... 2014-09-01 · L2 "Attention Is All ... 2017-12 · L3 Google releases BE... 2018-10-11 · L1 OpenAI says GPT-2 ... 2019-02-14 · L1