Bayes' TheoremBayes' Theorem

Thomas Bayes was a Presbyterian minister in Tunbridge Wells, England, who dabbled in mathematics between sermons. When he died in 1761, he left behind an unpublished manuscript on probability. His friend Richard Price found it, recognized its significance, and read it to the Royal Society. It appeared in 1763 as 'An Essay towards solving a Problem in the Doctrine of Chances'.

The idea is deceptively simple: beliefs should be updated in proportion to evidence. Bayes' theorem gives an exact recipe for revising the probability of a hypothesis when new data arrives, combining what was believed before (the prior) with how well the hypothesis explains the observations (the likelihood).

The theorem languished in relative obscurity until Pierre-Simon Laplace independently rediscovered and generalized it in the 1770s, using it for everything from estimating the mass of Saturn to demographic questions. For much of the twentieth century, Bayesian methods were dismissed by mainstream statisticians as too subjective, since they required an explicit prior belief.

Computers changed everything. Techniques like Markov chain Monte Carlo, which became practical in the 1990s, made it possible to compute Bayesian answers for problems with thousands of unknowns. Suddenly the 'subjective' approach was the workhorse behind spam filters, medical diagnostics, and code-breaking-style inference at scale.

Modern machine learning is saturated with Bayesian thinking, even where the name never appears. Regularization can be read as a prior, model uncertainty estimates lean on posterior distributions, and probabilistic programming languages let engineers write priors directly into code.

The minister's unpublished note turned out to be a general theory of learning from experience. Every system that starts with an assumption and refines it as data streams in is, in spirit, running Bayes' rule.