CUDACUDA

By the mid-2000s, graphics cards had quietly become the most powerful parallel processors consumers could buy, built to shade millions of pixels simultaneously for video games. A few researchers had begun contorting scientific computations into fake graphics operations just to tap that power, a practice that required expressing math problems in terms of textures and pixel shaders.

NVIDIA decided to make the hack official. In November 2006 it announced CUDA, a platform that let programmers write ordinary C code that ran across thousands of GPU threads, launching alongside the GeForce 8800. The effort grew out of work by Ian Buck, whose Stanford project Brook had pioneered stream programming on graphics hardware before NVIDIA hired him.

Inside and outside the company, CUDA looked like a niche bet. It consumed silicon area and engineering budget in service of a scientific computing market that barely existed, while the revenue still came from gamers. NVIDIA shipped CUDA capability in every consumer card anyway, which meant any graduate student with a gaming PC owned a supercomputer.

Machine learning researchers noticed. In 2009, Andrew Ng's group at Stanford showed that GPUs could accelerate the training of large neural networks by orders of magnitude. Then in 2012, Alex Krizhevsky trained AlexNet on two consumer GeForce GTX 580 cards and crushed the ImageNet competition, the result that ignited the deep learning era.

Deep learning's appetite for matrix multiplication mapped almost perfectly onto what GPUs already did, and CUDA was the only mature bridge between the two. The years of libraries, tooling, and developer habits built on CUDA hardened into one of the deepest moats in technology.

The gaming company became the arms dealer of the AI boom. Every major model of the deep learning era was trained on hardware descended from that 2006 decision, and NVIDIA rode the demand to become one of the most valuable companies in the world.