LeNetLeNet

Long before deep learning became a buzzword, Yann LeCun and colleagues at AT&T Bell Labs built a convolutional neural network to solve a stubbornly practical problem: reading handwritten numbers. Their 1998 paper, 'Gradient-Based Learning Applied to Document Recognition,' introduced LeNet-5, a design that still shapes computer vision today.

The key idea was the convolution. Instead of connecting every pixel to every neuron, LeNet slid small learnable filters across the image, detecting edges and strokes wherever they appeared. This made the network far smaller, faster to train, and naturally tolerant of digits shifted a few pixels in any direction.

LeNet stacked these convolutions with pooling layers that shrank the image while keeping what mattered, then finished with a few fully connected layers to make the final decision. Trained by backpropagation on the MNIST dataset of handwritten digits, it reached accuracy good enough to deploy commercially.

And deploy it did. By the end of the 1990s, systems descended from LeNet were reading a large share of the checks processed in the United States, one of the first real-world triumphs of neural networks.

Then progress stalled. Computers were too slow and datasets too small for deeper networks, and the field cooled for nearly a decade. LeNet's blueprint would not reach its full potential until GPUs and ImageNet arrived to power much larger descendants.