DQNDQN

In 2013, researchers at a then little-known company called DeepMind published 'Playing Atari with Deep Reinforcement Learning.' Their system, the Deep Q-Network or DQN, learned to play classic arcade games using only the raw screen pixels and the game score as feedback.

DQN combined two older ideas: Q-learning, a method from reinforcement learning for estimating the value of actions, and deep convolutional networks for seeing. The network watched the screen and gradually learned which button presses tended to lead to higher scores over time.

What made the result stunning was its generality. The same architecture, with no game-specific tuning, learned to play Breakout, Pong, Space Invaders and more, in several cases surpassing expert human players.

Getting it to work required clever tricks, most notably experience replay, which stored past moments and replayed them in random order to keep training stable. Without it, the network chased its own tail and diverged.

DQN was the demonstration that convinced many skeptics that deep reinforcement learning could scale. It helped make DeepMind's reputation and set the stage for later triumphs like AlphaGo.