ImageNet Classification with Deep Convolutional Neural Networks
Source: https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html ↗
The hinge of the modern era.
In 2012 AlexNet — a deep convolutional network trained on 1.2 million ImageNet images across 1,000 classes — won the field's benchmark competition by a margin that settled the argument: networks that learn their own features, given enough data and GPU compute, beat decades of hand-engineered vision.
The specifics are dated and it is about images, not language, but the recipe it proved — big network + big data + GPUs, made trainable by ReLU, dropout and augmentation — is the one every later model, LLMs included, inherits.
It is also the summit of the supervised era, the perfect backdrop for understanding why the later turn to self-supervised next-token prediction mattered so much.
Central argument
Krizhevsky, Sutskever and Hinton train a deep convolutional neural network to classify the 1.2 million images of ImageNet into 1,000 categories, and win ILSVRC-2012 by a dramatic margin. The paper's contributions are as much engineering as conceptual: ReLU activations for fast training, training across two GPUs, local response normalization, aggressive data augmentation, and dropout to fight overfitting in a 60-million-parameter model.
Critique
It is a 2012 supervised computer-vision paper: convolutional, label-hungry, and pre-transformer. Nothing in it is about language or generation, and its specific architecture has long been superseded. Its importance is historical and paradigmatic — it proved the approach — not a method you would reach for today.
Why it matters for product
For anyone tracing where modern AI came from, this is the hinge. It established the recipe — big network, big data, GPUs — that everything since, including LLMs, is a variation on, and it marks the peak of the supervised era just before the field pivoted to learning from unlabeled text. Reading it shows how recent, and how empirical, the whole revolution actually is.
- The Rise of AI Search: Implications for Information Markets and Human Judgement at Scale — Sinan Aral, Haiwen Li & R. Zuo
- Foundations of Large Language Models — Tong Xiao & Jingbo Zhu
- Hands-On Large Language Models — Jay Alammar & Maarten Grootendorst
- Build a Large Language Model (From Scratch) — Sebastian Raschka
- Understanding Deep Learning — Simon J.D. Prince
- Deep Learning with Python — François Chollet