Arcs Directions
Back to Timeline
2015-12-10

ResNet wins ImageNet — residual connections enable ultra-deep networks

Capability Breakthrough

Event Summary

ResNet (2015) won ImageNet with residual connections that enabled 152-layer networks. How skip connections solved the degradation problem and enabled deep learning at scale.

Impact Assessment

  • Paradigm Shift +2 · Long-term

    Residual connections became a universal building block in nearly all subsequent neural network architectures.

    Affected Groups: ai researchers, machine learning engineers

  • Capability Leap +2 · Long-term

    Enabled training of networks 8× deeper than previously possible, achieving superhuman ImageNet accuracy.

    Affected Groups: computer vision researchers, deep learning practitioners

  • Access Democratization +1 · Long-term

    The simplicity of residual connections made deep networks easier to train, lowering the barrier for practitioners.

    Affected Groups: machine learning practitioners, students

Consensus & Sources

Significance L2
Category Capability Breakthrough
Consensus Broad Consensus
Impact Index 6/10

Related Events

← Origins
2012-09-30 L3

AlexNet wins ImageNet and ignites the deep learning revolution

Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton's AlexNet crushed the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, achieving a top-5 error rate of 15.3%—over 10 percentage points better than the runner-up's 26.2%. The key innovation: training a deep convolutional neural network on two NVIDIA GTX 580 GPUs. This single result convinced the computer vision community that deep learning on GPUs was the future, ending decades of hand-engineered feature methods and launching the modern AI era.

2013-12-19 L1

DeepMind shows DQN playing Atari games from raw pixels — deep reinforcement learning's first breakthrough

Google DeepMind published 'Playing Atari with Deep Reinforcement Learning,' introducing the Deep Q-Network (DQN) — the first system to learn to play multiple Atari 2600 games directly from raw pixels using only the game score as feedback. The same network architecture, with no game-specific tuning, achieved human-level or better performance on 6 out of 7 games. This was the first successful integration of deep learning with reinforcement learning.

2015-11-09 L2

Google open-sources TensorFlow — deep learning becomes accessible

Google releases TensorFlow as an open-source deep learning framework under Apache 2.0 license. It quickly becomes the most widely adopted machine learning library, powering research and production systems across industries.

→ Successors
2017-12 L3

"Attention Is All You Need" — the Transformer architecture is born

Eight Google researchers published 'Attention Is All You Need' at NeurIPS 2017, introducing the Transformer architecture. By dispensing with recurrence and convolution entirely and relying solely on self-attention mechanisms, the Transformer achieved state-of-the-art translation results while training significantly faster. The paper has been cited over 250,000 times, and the Transformer became the foundation of every major AI system since—from GPT and BERT to AlphaFold and modern speech models.

Local Lineage ResNet wins ImageNet... 2015-12-10 · L2 AlexNet wins Image... 2012-09-30 · L3 DeepMind shows DQN... 2013-12-19 · L1 Google open-source... 2015-11-09 · L2 "Attention Is All ... 2017-12 · L3