Classical probability goes from causes to effects: knowing the coin is fair, what is the chance of heads? Often the problem is the reverse. We see the effects and want to know about the cause: after seeing certain results, what can we say about the coin? The first to treat this probability of causes mathematically was the English clergyman Thomas Bayes, in a paper published after his death, in 1763. Laplace arrived at the same ideas independently in 1774, without knowing of Bayes, and it was he who developed and applied them.

The unknown urn

Imagine an urn with white and black balls in an unknown proportion $p$. We know nothing about $p$: any value between 0 and 1 seems equally plausible. We draw $n$ balls (replacing each one) and all come out white. What is the chance the next one is also white?

If the proportion were $p$, the chance of $n$ whites in a row would be $p^n$. Values of $p$ that better explain what we saw gain more weight. Weighting each $p$ by $p^n$, the chance that the next ball is white is the average of $p$ with these weights:

$$P(\text{next white}) = \frac{\int_0^1 p \cdot p^n\,dp}{\int_0^1 p^n\,dp} = \frac{\frac{1}{n+2}}{\frac{1}{n+1}} = \frac{n+1}{n+2}.$$

This is Laplace’s rule of succession. With no observations ($n = 0$), it gives $\frac{1}{2}$. After one white, $\frac{2}{3}$. After ten, $\frac{11}{12}$. More generally, if $s$ whites came out in $n$ draws, the chance that the next is white is $\frac{s+1}{n+2}$.

The figure shows the weight curve, $(n+1)p^n$, as the whites pile up. It concentrates near $p = 1$, but never reaches certainty:

The sunrise

In the Philosophical Essay of 1814 Laplace applied the rule to an example that became famous. Suppose recorded history covers 5000 years, or $1\,826\,213$ days, and that the Sun has risen on every one of them. By the rule, the chance that it rises tomorrow is

$$\frac{1\,826\,214}{1\,826\,215},$$

and Laplace said one could bet $1\,826\,214$ to 1 that the Sun will rise tomorrow.

But on the same page he added a caveat that many forget: this number holds for someone who knows only the sequence of sunrises. Whoever knows the laws that govern days and seasons, he wrote, knows that nothing in the present can stop the Sun’s motion, and for that person the probability is “incomparably greater”. The rule of succession is what remains when nothing else is known.

Exercise: the weights add up to 1

Show that $\int_0^1 p^n\,dp = \frac{1}{n+1}$ and conclude that the curve $(n+1)p^n$ in the figure has area 1 and mean $\frac{n+1}{n+2}$.

An antiderivative of $p^n$ is $\frac{p^{n+1}}{n+1}$, which equals $\frac{1}{n+1}$ at $p = 1$ and 0 at $p = 0$. Multiplying by $n+1$, the area becomes 1. The mean is $\int_0^1 p\,(n+1)p^n\,dp = (n+1)\cdot\frac{1}{n+2} = \frac{n+1}{n+2}$: exactly the rule of succession.

Exercise: nine out of ten, ninety out of a hundred

One player made 9 of 10 free throws. Another made 90 of 100. By the rule of succession, what is the chance that each one makes the next?

First: $\frac{9+1}{10+2} = \frac{10}{12} \approx 0.83$. Second: $\frac{90+1}{100+2} = \frac{91}{102} \approx 0.89$. The observed proportions are the same, 90%, but the rule trusts more in whoever has more data: with few observations, it pulls the estimate towards $\frac{1}{2}$.

Sources: Pierre-Simon Laplace, “Mémoire sur la probabilité des causes par les événements”, Mémoires de mathématique et de physique présentés à l’Académie 6 (1774) · Pierre-Simon Laplace, Essai philosophique sur les probabilités (1814) · Thomas Bayes, “An Essay towards Solving a Problem in the Doctrine of Chances”, Philosophical Transactions 53 (1763) · Stephen M. Stigler, “Laplace’s 1774 Memoir on Inverse Probability”, Statistical Science 1 (1986) · Sharon Bertsch McGrayne, The Theory That Would Not Die (2011).

All topics on Pierre-Simon Laplace · ← The central limit theorem · The Laplace transform →

Image credits.