Laplace 1814
In 1814, Pierre-Simon Laplace published An Philosophical Essay on Probabilities Laplace1814 Hoang20FR. This essay introduced the core philosophical ideas of Bayesianism (which is arguably a misnomer, and should thus be called Laplacianism), as a follow-up of Laplace's 1774 Memoir on the Probability of Causes given Events.
Contents
The structure of the Essay
The essay starts with an introduction on the importance of probability, even in a deterministic universe. Thereby, it adopts a resolutely Bayesian interpretation of probabilities, as according to Laplace, probabilities are tools to describe our knowledge and our ignorance. As a result, for Laplace, probabilities are necessarily subjective.
Laplace then goes on laying the foundations of probability theory by introducing 10 principles, and discusses briefly some mathematical methods to compute probabilities. Afterwards, the Essay presents applications in history, justice, birth, death, marriage and decision-making. It also discusses humans' cognitive biases and "pragmatic Bayesianism", i.e. ways to approximate the rules of probability.
Laplace's philosophy of probability
On Probability
From the outset, Laplace stresses the role of probability. He wrote (all subsequent quotes are translated by Lê).
[The] most important questions in life are in fact, for most of them, only problems of probability.
Yet, somewhat surprisingly, in this book, Laplace defends the determinism of the universe.
An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.
This would seem self-contradictory for a naive interpretation of probabilities. But Laplace argues that there is a difference between the determinism of atoms and that of planets, wherein lies the role of probabilities.
The trajectory followed by a simple air molecule or by vapor, is ruled in a manner as certain as that of planetary orbits: the only difference between them lies in our ignorance.
In other words, probabilities are the essential tools, not to describe the universe itself, but to describe our knowledge and our ignorance of the universe.
The probability is relative in part to this ignorance, and in part to our knowledge [...] In this state of indecision, it is impossible for us to declare anything with certainty.
The quote above is the philosophical foundation of Bayesianism. In this philosophy of knowledge, probabilities are tools to describe the extent of our knowledge; or, equivalently, of our ignorance. It matters little (if at all) whether these probabilities are "real". This is not their purpose. Their purpose is for us to describe what we know and what we don't.
As a result, there should be no surprise in the fact that Laplacian probabilities are subjective. Of course they are, since they describe the speaker's state of knowledge and ignorance.
The theory of probabilities lies in considerations so subtle, that it is not surprising that, given the same data, two people find different results, especially on very complex matters.
This complexity motivates epistemological modesty. In fact, Laplace calls for what may be called intellectual honesty Galef19.
Let us enlighten those that we deem to be insufficiently instructed; but before this, let us examine severely our own opinions, and weigh with impartiality, their respective probabilities.
Laplace underlines what he argues to be his contemporary historians' overconfidence.
Historians do not seem to pay enough attention to the degradation of the probability of facts [...] many historical events, claimed to be certain, would be at least doubtful, if they were subject to this challenge.
He also seems to argue against the universality of deontological principles, based on a medium/long-term consequentialist argument. Interestingly, this argument echoes what, centuries later, would be known as chaos theory.
In moral sciences [...] the chance of error increases with their number, and eventually surpasses the chance of truth, in consequences very distant from this principle.
On General Principles of the Calculus of Probability
Afterwards, in the book, Laplace introduces the principles of probability. Most of them would be unsurprising to anyone studying probability theory. But this particular quote is particularly important, as it introduces what's arguably the most fundamental (and controversial) rule of probability, namely what is known as Bayes rule (historically though, Thomas Bayes only introduced Bayes rule in a special case and did not even publish his findings; there are good reasons to think that Bayes wasn't much of a Bayesianism, and that Laplace is the actual father of Bayes rule and Bayesianism... McGrayne12).
The probability [of a cause] is thus a fraction whose numerator is the probability of the event resulting from the cause, and whose denominator is the sum of similar probabilities relative to all causes; if the different causes considered a priori were not equally probable, then instead of the probability of the event result from each cause, one would need to invoke the product of such probabilities by those of the events themselves.
Evidenty, calling [math]T[/math] the cause, [math]D[/math] the event and [math]A[/math]'s the alternatives to [math]T[/math], this sentence describes the equation [math]\mathbb P[T|D] = \frac{\mathbb P[D|T] \mathbb P[T]}{\mathbb P[D|T] \mathbb P[T] + \sum_{A \neq T} \mathbb P[D|A] \mathbb P[A]}[/math]. This is Bayes rule.
On Illusions in the Estimation of Probabilities
Laplace is highly critical of humans' poor intuitive estimations of probabilities, and calls for a greater use of computations.
The mind has its illusions, like visual perception; and just like the haptic perception can rectify the latter, thought and computation can rectify the former.
He then describes what might be called the familiarity bias Veritasium16.
The events that we witness have on our judgments an influence that often misleads us in our appreciation of the causes they depend on. The lively impression we have barely allows us to notice the contrary events that others observed. We cannot give too much care to protect ourselves against this illusion, which is one of the main sources of our mistakes.
Laplace then discusses what we would now call the gambler's fallacy.
When at the lottery of France, a numero has not come out for a long time; the crowd hastens to bet on it. The crowd considers that, given that the number has not come out for a long time, it must be more likely to be drawn than others at the next lottery. An error so common seems to be an illusion to me.
He calls more generally for deferring our judgments to probabilistic computations, when such computations can be done reliably. And even when we cannot, he argues that we still should be skeptical of our intuitive judgments.
One of the great advantages of the computation of probabilities is to teach us to challenge our first impressions. As we recognize that they mislead us often, when we can subject them to computation; we must then conclude that on other matters, we should accept them only with an extreme skepticism.
Later, he describes an "illusion" of Leibniz, based on the equality [math]\sum (-x)^n = \frac{1}{1+x}[/math]. By setting [math]x=1[/math], Leibniz concluded that [math]1-1+1-1+... = 1/2[/math]. But then, by grouping terms [math]1-1[/math], this can be rewritten as [math](1-1)+(1-1)+... = 0+0+... = 1/2[/math]. Leibniz would go on concluding that this shows that the Supreme Being can emerge out of nothingness.
This idea pleased Leibniz so much that he shared it with Jesuit Grimaldi, president of the court of mathematics in China, in the hope that this symbol of Creation would convert to Christianism the emperor of that time who particularly enjoyed science. I only report this to show to which extent the childhood prejudices can mislead the greatest man.
Laplace arguably highlighted humans' tendency towards confirmation bias.
Conclusion
After presenting numerous applications of the rules of probability to diverse fields, Laplace eventually concludes his Essay by stressing the importance of probabilities. First he argues that it is the crux of thinking correctly.
The theory of probabilities is basically just common sense reduced to computation; it makes one appreciate with exactness that which accurate minds feel with a sort of instinct, often without being able to account for it.
He then stresses the beauty of the theory and the formidable range of its applications.
If we consider the analytical methods that the theory [of probabilities] gave rise to, the truth of the principles it relies on, the subtle logic that demands its application to solving problems, the public utility goods that is built upon it, and the extensions it has received and can still receive, given its application to the most important questions of natural philosophy and political economics; if we then observe that even in things that cannot be reduced to computation, probability theory allows the most reliable insights to guide us in our judgment, and that it teaches us to steer away from the illusions that often mislead us; we shall see that there is no science more worthy of our meditations, and whose results are more useful.
