Laplace 1814
In 1814, Pierre-Simon Laplace published An Philosophical Essay on Probabilities Laplace1814 Hoang20FR. This essay introduced the core philosophical ideas of Bayesianism (which is arguably a misnomer, and should thus be called Laplacianism), as a follow-up of Laplace's 1774 Memoir on the Probability of Causes given Events.
The structure of the Essay
The essay starts with an introduction on the importance of probability, even in a deterministic universe. Thereby, it adopts a resolutely Bayesian interpretation of probabilities, as according to Laplace, probabilities are tools to describe our knowledge and our ignorance. As a result, for Laplace, probabilities are necessarily subjective.
Laplace then goes on laying the foundations of probability theory by introducing 10 principles, and discusses briefly some mathematical methods to compute probabilities. Afterwards, the Essay presents applications in history, justice, birth, death, marriage and decision-making. It also discusses humans' cognitive biases and "pragmatic Bayesianism", i.e. ways to approximate the rules of probability.
Laplace's philosophy of probability
From the outset, Laplace stresses the role of probability. He wrote (all subsequent quotes are translated by Lê).
[The] most important questions in life are in fact, for most of them, only problems of probability.
Yet, somewhat surprisingly, in this book, Laplace defends the determinism of the universe.
An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.
This would seem self-contradictory for a naive interpretation of probabilities. But Laplace argues that there is a difference between the determinism of atoms and that of planets, wherein lies the role of probabilities.
The trajectory followed by a simple air molecule or by vapor, is ruled in a manner as certain as that of planetary orbits: the only difference between them lies in our ignorance.
In other words, probabilities are the essential tools, not to describe the universe itself, but to describe our knowledge and our ignorance of the universe.
The probability is relative in part to this ignorance, and in part to our knowledge [...] In this state of indecision, it is impossible for us to declare anything with certainty.
The quote above is the philosophical foundation of Bayesianism. In this philosophy of knowledge, probabilities are tools to describe the extent of our knowledge; or, equivalently, of our ignorance. It matters little (if at all) whether these probabilities are "real". This is not their purpose. Their purpose is for us to describe what we know and what we don't.
As a result, there should be no surprise in the fact that Laplacian probabilities are subjective. Of course they are, since they describe the speaker's state of knowledge and ignorance.
The theory of probabilities lies in considerations so subtle, that it is not surprising that, given the same data, two people find different results, especially on very complex matters.
This complexity motivates epistemological modesty. In fact, Laplace calls for what may be called intellectual honesty Galef19.
Let us enlighten those that we deem to be insufficiently instructed; but before this, let us examine severely our own opinions, and weigh with impartiality, their respective probabilities.
Laplace underlines what he argues to be his contemporary historians' overconfidence.
Historians do not seem to pay enough attention to the degradation of the probability of facts [...] many historical events, claimed to be certain, would be at least doubtful, if they were subject to this challenge.
He also seems to argue against the universality of deontological principles, based on a medium/long-term consequentialist argument. Interestingly, this argument echoes what, centuries later, would be known as chaos theory.
In moral sciences [...] the chance of error increases with their number, and eventually surpasses the chance of truth, in consequences very distant from this principle.
Afterwards, in the book, Laplace introduces the principles of probability. Most of them would be unsurprising to anyone studying probability theory. But this particular quote is particularly important, as it introduces what's arguably the most fundamental (and controversial) rule of probability, namely what is known as Bayes rule (historically though, Thomas Bayes only introduced Bayes rule in a special case and did not even publish his findings; there are good reasons to think that Bayes wasn't much of a Bayesianism, and that Laplace is the actual father of Bayes rule and Bayesianism... McGrayne12).
The probability [of a cause] is thus a fraction whose numerator is the probability of the event resulting from the cause, and whose denominator is the sum of similar probabilities relative to all causes; if the different causes considered a priori were not equally probable, then instead of the probability of the event result from each cause, one would need to invoke the product of such probabilities by those of the events themselves.
Evidenty, calling [math]T[/math] the cause, [math]D[/math] the event and [math]A[/math]'s the alternatives to [math]T[/math], this sentence describes the equation [math]\mathbb P[T|D] = \frac{\mathbb P[D|T] \mathbb P[T]}{\mathbb P[D|T] \mathbb P[T] + \sum_{A \neq T} \mathbb P[D|A] \mathbb P[A]}[/math]. This is Bayes rule.
After presenting numerous applications of the rules of probability to diverse fields, Laplace eventually concludes his Essay by stressing the importance of probabilities. First he argues that it is the crux of thinking correctly.
The theory of probabilities is basically just common sense reduced to computation; it makes one appreciate with exactness that which accurate minds feel with a sort of instinct, often without being able to account for it.
He then stresses the beauty of the theory and the formidable range of its applications.
If we consider the analytical methods that the theory [of probabilities] gave rise to, the truth of the principles it relies on, the subtle logic that demands its application to solving problems, the public utility goods that build upon it, and the extensions it has received and can still receive, given its application to the most important questions of natural philosophy and political economics; if we then observe that even in things that cannot be reduced to computation, probability theory allows the most reliable insights to guide us in our judgment, and that it teaches us to stear away from the illusions that often mislead us; we shall see that there is no science more worthy of our meditations, and whose results are more useful.
