On Probabilities

1.0 | Introduction

People tend to avoid making absolute statements about reality—excluding the mind, such as statements about one’s emotions. An absolute statement is one that is definitively true, without exception. These are rare because proving such claims is difficult. As a result, people often turn to weaker statements involving probabilities. For instance, the claim “chairs exist in reality independently of the mind” becomes “chairs probably exist in reality independently of the mind.”

Let’s examine this statement. What function does the word “probably” serve? Is it merely a reflection of past frequencies? Does it inform us about whether chairs truly exist independently of perception? Some might argue it reflects our level of certainty—a psychological state. In that case, the probability says more about our minds than about the object itself.

2.0 | When are Probabilities Meaningful?

These questions highlight the technical challenges surrounding the nature of probabilities. I’ll introduce five axioms—unjustified assumptions—that make probabilities practically useful. But first, here’s what I expect probabilities to do.

Probabilities should inform us about reality—not just reflect our mental states. They should take perceptual input and help us infer something about what lies beyond perception. For this to work, the following assumptions must hold:

1. There exists a mind with sensory input.
   
2. There exists a reality beyond the mind.
   
3. The transmission from reality to the mind is reliable.
   
4. Reality is not maximally complex (i.e., not incompressible).
   
5. The frequency of an event in the past is approximately the same as in the future.

Axioms 4 and 5 are closely related but distinct. Axiom 4 assumes reality contains patterns and is not globally random. This isn’t something we can currently prove. We may inhabit a region of the universe rich in patterns, even if the larger structure is not. Imagine landing at position 17,387,594,880 in the digits of $\pi$. There, you see the sequence 0123456789. If that’s all you ever saw, you’d reasonably predict “10” next—but you’d be wrong.

So even if our lived experience is pattern-rich, that may be a local illusion. Axiom 5 adds that what happened frequently in the past will likely happen frequently in the future. Exceptions may occur but should be limited. For general probabilistic reasoning, this axiom must be accepted. Think of the probability of heads when you flip a coin—it does not change through time.

With these assumptions, we can begin to make meaningful probabilistic statements about reality. For example: “The theory of relativity is probably a close approximation of reality.” This assumes reliable transmission (axioms 1–3), pattern richness (4), and temporal stability (5). If any of these fail, the theory rests on shaky ground. Scientific experiments and error margins show how close a theory is to reality. Smaller errors suggest higher accuracy—but not certainty. Even consistent success in X experiments doesn’t guarantee success in X+1. Universal claims require infinite validation, which is impossible.

3.0 | What are Probabilities? (in mathematics)

The modern foundation of probability theory comes from Andrey Kolmogorov (1933), who proposed these axioms:

1. Non-Negativity: If a set A is any event, then, $0$ $\le$ $P(A)$ $\le$ $1$.
2. Normalization: If $\mathrm{\Omega }$ is the sample space, then $P(\mathrm{\Omega })=1$.
3. Finite Additivity: If A and B are mutually exclusive events, then $P(A\cup B)=P(A)+P(B)$.

Non-Negativity means 0 is impossibility and 1 is certainty. Normalization ensures that probabilities add up to 1, typically by dividing frequency counts by their total. For infinite sets, we often need a “normalization constant” to ensure the distribution integrates (or sums) to 1.

Finite Additivity applies to finite sample spaces. For infinite sets, a more advanced tool—$\sigma$-additivity—is needed, which handles countably infinite unions of disjoint sets.

4.0 | Interpertation of Probability

With the formal basis in place, how should we interpret probabilities? Among many philosophical views, two dominate: frequentist and Bayesian.

The frequentist view sees probability as the long-run frequency of an event. If you flip a coin many times, about half the results will be heads. The more you flip, the closer the proportion of heads gets to 0.5. So, the probability of heads is 0.5—an objective feature of the coin.

The Bayesian view defines probability as a degree of belief, which updates as new evidence arrives. This is encoded by Bayes’ theorem:

$\underset{\text{posterior}}{\underbrace{p(\theta \mid y)}}=\frac{\stackrel{\text{likelihood (rescaling factor)}}{\overbrace{p(y\mid \theta )}}\times \stackrel{\text{prior (initial belief)}}{\overbrace{p(\theta )}}}{\underset{\text{evidence (normalizing constant)}}{\underbrace{p(y)}}}$,

In discrete cases, probabilities must be between 0 and 1 and sum to 1. In continuous cases, they must be non-negative and integrate to 1. The likelihood adjusts relative weights of different hypotheses, and the posterior reflects your updated beliefs.

A key strength of Bayesian probability is that it works even without repeated trials. Say you’ve never flipped a coin but want to estimate the chance of heads. You assign a prior to the unknown bias $\theta$. To stay neutral, you might use $\text{Beta}(1,1)$, which is uniform across [0, 1]. This prior has maximum entropy—it makes minimal assumptions. Then, the expected probability of heads is $0.5$. So you can say, “There’s a 50% chance,” even with no data. Bayesians can also update their beliefs as new data arrives, refining the model over time.

The frequentist interpertation of probabilities is still useful, especially in evaluating models. Techniques like CBPE (Confidence-Based Performance Estimation) let you assess models without needing labels—if the model is well-calibrated, meaning predicted probabilities match actual outcomes. For example, if a model assigns a score of 0.8, about 80% of those cases should be positive, given enough data. A prime example of a well calibrated model is logistic-regression. Nonetheless, Bayesian methods do offer flexible alternatives for similar tasks.

5.0 | Final Remarks

Probability theory bridges the gap between our limited knowledge and the world beyond our minds. It helps us make decisions under uncertainty, provided we accept some base assumptions. Whether seen as an objective regularity (frequentist) or a belief-update mechanism (Bayesian), probabilities become meaningful only within a framework of perception, signal reliability, and repeatable structure.

The frequentist and Bayesian views offer different perspectives on probabilities. One leans on repetition; the other on reasoned belief revision. But both rest on foundational assumptions. If the world lacked structure or consistency, probabilistic reasoning would lose meaning. Still, in a pattern-rich and stable reality, probabilities give us a reliable compass—even when the full map remains unseen.