$\sigma$-Algebras

A card, face down

A card is drawn from a shuffled deck and laid face down on the table. What can be known about it?

Write $\Omega$ for the set of all 52 cards that could have been drawn, and $\omega$ for the card that actually was. Probability never tells you which $\omega$ it is. It tells you how to reason while you don’t know.

A yes-or-no question about the card amounts to the set of cards for which the answer is yes. “Is it red?” is the set of 26 red cards, and “Is it the ace of spades?” is a set with one card in it. So a question is a subset of $\Omega$, and from here on the two words mean the same thing. Mathematicians call such subsets events.

Primer Sets in one minute

• A set is a collection of things, called its elements. $\omega \in A$ means that $\omega$ is one of them.
• $A \subseteq \Omega$ means every element of $A$ is also in $\Omega$, so $A$ is a subset of $\Omega$.
• The complement $A^{c}$ is everything in $\Omega$ that is not in $A$. As a question it reads “not $A$?”.
• The union $A \cup B$ collects everything in $A$, in $B$, or in both. As a question it reads “$A$ or $B$?”.
• The intersection $A \cap B$ collects everything in both. As a question it reads “$A$ and $B$?”.
• The empty set $\varnothing$ contains nothing. It is the question whose answer is always no, just as $\Omega$ is the question whose answer is always yes.

Four people, four kinds of knowledge

Four people are in the room, and each of them sees the card differently.

• The colour-watcher sits across the room. She can tell red from black, and nothing more.
• The suit-watcher sees the suit symbol, but a coin covers the number.
• The rank-watcher sees the number, but the suit symbol is smudged.
• The dealer sees everything.

Ask each of them “Is it a heart?”. The suit-watcher and the dealer can answer. The colour-watcher cannot: she sees red, and red could mean hearts or diamonds. The rank-watcher cannot answer either.

So the useful way to describe what someone knows is the complete list of questions she can answer. A single fact such as “the card is red” is one answer from that list. The list itself is the same whichever card was drawn, and it is what we will mean by her knowledge.

Each watcher has groups of cards she cannot tell apart. The colour-watcher has two such groups, the red cards and the black cards. The suit-watcher has four, the rank-watcher has thirteen, and the dealer has fifty-two groups of one card each. Call these groups atoms. A question can be answered exactly when it never splits an atom, that is, when its yes-set is made of whole atoms.

Atoms work like the pixels of a photograph. A low-resolution photo can show any shape built from whole pixels, and nothing finer.

$$\#\{\text{answerable questions}\} = 2^{k}$$

Symbols

$\#$ counts the questions on a watcher’s list, and $k$ is the number of her atoms.

Read it as

“A watcher with $k$ atoms can answer exactly $2^{k}$ questions.”

Intuition

An answerable question is a choice, atom by atom, of yes or no, with two choices for each of the $k$ atoms. The four watchers can answer $2^{2} = 4$, $2^{4} = 16$, $2^{13} = 8{,}192$ and $2^{52} \approx 4.5 \times 10^{15}$ questions.

The colour-watcher’s four questions are worth writing out, because the trivial ones count too: “Is it red?”, “Is it black?”, “Is it one of the 52 cards?”, which is always yes, and “Is it none of them?”, which is always no. As sets, her list is $\{\text{red}, \text{black}, \Omega, \varnothing\}$.

Who is looking

Colour-watcher Suit-watcher Rank-watcher Dealer

The question

Is it red? Is it a heart? Is it an ace? Is it a face card? Is it a red ace? Is it the queen of spades? Is it one of the 52 cards?

A

2

3

4

5

6

7

8

9

10

J

Q

K

A

2

3

4

5

6

7

8

9

10

J

Q

K

A

2

3

4

5

6

7

8

9

10

J

Q

K

A

2

3

4

5

6

7

8

9

10

J

Q

K

The colour-watcher has 2 atoms and can answer $2^{2} = 4$ questions. “Is it red?” is answerable: every atom is entirely yes or entirely no.“Is it a heart?” is not answerable: it splits one atom, the red cards.“Is it an ace?” is not answerable: it splits every atom.“Is it a face card?” is not answerable: it splits every atom.“Is it a red ace?” is not answerable: it splits one atom, the red cards.“Is it the queen of spades?” is not answerable: it splits one atom, the black cards.“Is it one of the 52 cards?” is answerable: every atom is entirely yes or entirely no. The suit-watcher has 4 atoms and can answer $2^{4} = 16$ questions. “Is it red?” is answerable: every atom is entirely yes or entirely no.“Is it a heart?” is answerable: every atom is entirely yes or entirely no.“Is it an ace?” is not answerable: it splits every atom.“Is it a face card?” is not answerable: it splits every atom.“Is it a red ace?” is not answerable: it splits 2 atoms, the hearts among them.“Is it the queen of spades?” is not answerable: it splits one atom, the spades.“Is it one of the 52 cards?” is answerable: every atom is entirely yes or entirely no. The rank-watcher has 13 atoms and can answer $2^{13} = 8{,}192$ questions. “Is it red?” is not answerable: it splits every atom.“Is it a heart?” is not answerable: it splits every atom.“Is it an ace?” is answerable: every atom is entirely yes or entirely no.“Is it a face card?” is answerable: every atom is entirely yes or entirely no.“Is it a red ace?” is not answerable: it splits one atom, the aces.“Is it the queen of spades?” is not answerable: it splits one atom, the queens.“Is it one of the 52 cards?” is answerable: every atom is entirely yes or entirely no. The dealer has 52 atoms and can answer $2^{52} \approx 4.5 \times 10^{15}$ questions. “Is it red?” is answerable: every atom is entirely yes or entirely no.“Is it a heart?” is answerable: every atom is entirely yes or entirely no.“Is it an ace?” is answerable: every atom is entirely yes or entirely no.“Is it a face card?” is answerable: every atom is entirely yes or entirely no.“Is it a red ace?” is answerable: every atom is entirely yes or entirely no.“Is it the queen of spades?” is answerable: every atom is entirely yes or entirely no.“Is it one of the 52 cards?” is answerable: every atom is entirely yes or entirely no.

The rules that answering forces

Take any watcher, and any two questions $A$ and $B$ on her list. Three things follow, whoever she is.

1. She can answer “Is it one of the 52 cards?”, since the answer is always yes. So $\Omega$ is on her list.
2. If she can answer “$A$?”, she can answer “not $A$?”, which is the same knowledge read the other way round. So $A^{c}$ is on her list.
3. If she can answer “$A$?” and “$B$?”, she can answer “$A$ or $B$?”. So $A \cup B$ is on her list.

She never needs a separate rule for “and”, because “$A$ and $B$” means “not (not $A$, or not $B$)”:

$$A \cap B = \big(A^{c} \cup B^{c}\big)^{c}$$

Symbols

$A^{c}$ is “not $A$”, $\cup$ is “or” and $\cap$ is “and”.

Read it as

“Both happen exactly when it is false that at least one of them fails.”

Intuition

Negation turns “or” into “and”, so the second and third rules already give her “and” for free.

These rules come straight from what “can answer” means, with nothing added. A collection of questions that obeys them is called an algebra of sets. On a deck of 52 cards this is the whole story. Probability theory needs one more letter, the σ, and to see why we need a hidden thing with infinitely many possibilities.

The theory Why finite knowledge always comes in atoms

Counting questions by counting atoms always works, and the reason is a small theorem. Take any list of questions about a finite $\Omega$ that obeys the three rules, and call two cards indistinguishable if every question on the list contains both or neither. The groups of mutually indistinguishable cards are the atoms.

The atom of a card $\omega$ is the intersection of all the questions on the list that contain $\omega$. (A card lies in that intersection exactly when no question on the list separates it from $\omega$, using complements.) There are finitely many such questions, so by the rules the intersection is itself on the list. Any answerable question containing $\omega$ therefore contains its whole atom. So the answerable questions are exactly the unions of atoms, and there are $2^{k}$ of them.

On an infinite $\Omega$ the argument can fail, because the intersection may involve infinitely many questions.

A number with infinitely many digits

Now the hidden thing is a number $U$ between 0 and 1, made by rolling a fair ten-sided die forever. The first roll is the first decimal digit, the second roll is the second digit, and so on, giving something like $U = 0.31415\ldots$

Think of $U$ as an address that never ends. The first digit names the district, the second the street, the third the building, the fourth the floor, and the levels keep going. Each digit narrows the location tenfold.

Someone who has read only the first digit, say a 3, knows that $U$ lies in $[0.3, 0.4)$. Her atoms are the ten intervals $[0, 0.1), [0.1, 0.2), \ldots, [0.9, 1)$, so she can answer $2^{10} = 1{,}024$ questions. “Is $U$ below 0.5?” is one of them, because its yes-set is five whole atoms. “Is $U$ below 0.55?” is not, because it splits the atom $[0.5, 0.6)$. After $n$ digits she has $10^{n}$ atoms, each an interval of width $10^{-n}$.

Primer Probability for the random number

Each digit is equally likely to be any of 0 to 9, independently of the others. The chance that $U$ begins with 0.31 is therefore $\tfrac{1}{10} \times \tfrac{1}{10} = \tfrac{1}{100}$, which is exactly the width of the interval $[0.31, 0.32)$. In general, the probability that $U$ lands in an interval equals the interval’s length.

Only two properties of a probability $\mathbb{P}$ are used. The sure question has probability 1, and when two questions cannot both be answered yes, their probabilities add.

Now ask a question of a new kind: will a 7 ever appear?

Nobody who has read finitely many digits can be sure the answer is no, since the next digit might be a 7. Yet the question is perfectly meaningful, and it has an answer. Its yes-set is built from simple digit questions joined by an “or” that never stops:

$$\{\text{a 7 appears}\} = \bigcup_{n=1}^{\infty} \{d_n = 7\}$$

Symbols

$d_n$ is the $n$-th digit of $U$, and $\{d_n = 7\}$ is the question “is the $n$-th digit a 7?”. The large $\bigcup$ joins them all: $\{d_1 = 7\} \cup \{d_2 = 7\} \cup \{d_3 = 7\} \cup \cdots$.

Read it as

“A 7 appears exactly when the first digit is a 7, or the second is, or the third is, and so on forever.”

Intuition

“Ever” is an “or” that never ends. Its terms can still be counted off one at a time, and each of them eventually gets its turn.

Primer What “countable” means

An infinite set is countable if its elements can be lined up as a first, a second, a third and so on, so that each one eventually gets its turn. Think of an endless queue in which everyone holds a ticket number.

The whole numbers are countable. So are the fractions between 0 and 1: list them by denominator, as $0, \tfrac{1}{2}, \tfrac{1}{3}, \tfrac{2}{3}, \tfrac{1}{4}, \tfrac{3}{4}, \tfrac{1}{5}, \ldots$, and every fraction turns up at some finite position.

The numbers in $[0,1)$ are not countable, and digits show why (Cantor’s diagonal argument, 1891). Given any list of them, build a number whose $n$-th digit is 5, unless the $n$-th digit of the $n$-th number on the list is 5, in which case use 6. The new number differs from every number on the list somewhere, so the list missed it. The interval is a crowd too large to be issued tickets.

The probability that a 7 appears comes from adding infinitely many pieces. Split the question according to where the first 7 occurs:

$$\mathbb{P}(\text{a 7 appears}) = \sum_{n=1}^{\infty} \Big(\tfrac{9}{10}\Big)^{n-1} \cdot \tfrac{1}{10} = 1$$

Symbols

$\mathbb{P}$ is probability. The $n$-th term is the chance that the first 7 is the $n$-th digit.

Read it as

“The probability that a 7 appears is the sum, over every position, of the chance that the first 7 is there.”

Intuition

For the first 7 to sit at position $n$, the $n-1$ digits before it must avoid 7, each with chance $\tfrac{9}{10}$, and digit $n$ must be 7, with chance $\tfrac{1}{10}$. The first 7 can only be in one place, so the pieces never overlap and their probabilities add. The geometric series sums to exactly 1. A 7 is certain to turn up, even though at no finite moment can you be sure of it.

Will a gambler ever go broke? Will a share price ever touch 100? Questions of this kind are everywhere. Some are subtler. “Do the digits of $U$ eventually repeat in a cycle?” is secretly “Is $U$ a fraction?”, and it too is built from digit questions, using infinitely many “or”s and “and”s.

The theory Fractions as a digit question

A number is a fraction exactly when its digits are eventually periodic: from some position $N$ on, every digit equals the digit $p$ places later, for some period $p$. For example $\tfrac{1}{7} = 0.142857\,142857\ldots$ has period 6 from the start, and $\tfrac{1}{4} = 0.25000\ldots$ has period 1 from the third digit on. In symbols:

$$\{U \text{ is a fraction}\} = \bigcup_{N=1}^{\infty} \, \bigcup_{p=1}^{\infty} \, \bigcap_{n=N}^{\infty} \{d_n = d_{n+p}\}$$

Symbols

$N$ is where the repetition starts and $p$ is its period. Each $\{d_n = d_{n+p}\}$ is a question about two digits.

Read it as

“There is a starting position and a period such that, from that position on, every digit equals the digit $p$ places after it.”

Intuition

“There is” becomes $\bigcup$ and “for every” becomes $\bigcap$. Two countable “or”s wrapped around a countable “and” turn a question about infinitely many digits into a legitimate event.

Its probability is 0. Each fixed pair $(N, p)$ needs infinitely many coincidences, each with chance $\tfrac{1}{10}$, so it has probability 0. The pairs $(N, p)$ can be listed, so only countably many zeros are added.

The theory Borel’s normal numbers, and an open problem

In 1909 Émile Borel asked whether, in the long run, each digit makes up a tenth of the digits of $U$. For the digit 7 the question is

$$\Big\{ \lim_{n \to \infty} \frac{\#\{k \le n : d_k = 7\}}{n} = \frac{1}{10} \Big\}$$

Symbols

$\#\{k \le n : d_k = 7\}$ counts the 7s among the first $n$ digits, so dividing by $n$ gives their frequency so far.

Read it as

“The fraction of 7s among the first $n$ digits tends to one tenth.”

Intuition

A limit says “for every tolerance, from some point on, always within it”, which is again countably many “and”s and “or”s of digit questions. So this is an event, and it has a probability.

Borel proved that the probability is 1. In fact, with probability 1 every digit and every block of digits turns up with its fair long-run frequency, in every base. Such numbers are called normal. To state this at all, Borel had to let probabilities add over countably many pieces, as in the sum above.

So almost every number is normal, yet no famous constant has been proved normal. Whether $\pi$ or $\sqrt{2}$ is normal is an open problem. The normal numbers anyone can write down were built on purpose, such as Champernowne’s $0.123456789101112\ldots$ (1933), which is normal in base 10.

Why countably many is exactly enough

So the rules have to allow infinitely many “or”s. How many, though? Finitely many is too few, since it cannot ask whether a 7 will ever appear. Could we allow any number at all?

Look at a single point. “Is $U$ exactly $\tfrac{1}{3}$?” asks whether every digit is a 3, an “and” that never ends. The first $n$ digits are all 3 with chance $10^{-n}$, so

$$\mathbb{P}\big(U = \tfrac{1}{3}\big) \le 10^{-n} \quad \text{for every } n$$

Symbols

$10^{-n}$ is the chance that the first $n$ digits are all 3.

Read it as

“The probability that $U$ is exactly one third is at most $10^{-n}$, however large $n$ is, so it is zero.”

Intuition

Hitting one exact number needs infinitely many lucky digits in a row. The same holds for every number, so each single point has probability zero.

But $[0,1)$ is the union of all its single points, and its probability is 1. If probability could add over any number of pieces, the probability of the interval would be a sum of zeros, one for each point, which is 0 and not 1. So adding has to stop somewhere, and it stops at countable. Countably many zeros add up to zero without any contradiction. The fractions in $[0,1)$ can be listed and each has probability zero, so the chance that $U$ is a fraction is $0 + 0 + 0 + \cdots = 0$. The points of the interval cannot be listed, and that is exactly why adding them up produces nonsense.

Probability adds over countably many pieces and no further, and the list of answerable questions is built to match: it is closed under countably many “or”s. That is the σ, from the German Summe, a sum or union. Probability can serve an endless queue, provided everyone holds a ticket. It cannot serve a crowd too large to number.

The definition

A σ-algebra on $\Omega$ is a collection $\mathcal{F}$ of subsets of $\Omega$ with three properties:

$$\begin{aligned} &\text{(S1)} \quad \Omega \in \mathcal{F} \\ &\text{(S2)} \quad A \in \mathcal{F} \;\Longrightarrow\; A^{c} \in \mathcal{F} \\ &\text{(S3)} \quad A_1, A_2, \ldots \in \mathcal{F} \;\Longrightarrow\; \bigcup_{n=1}^{\infty} A_n \in \mathcal{F} \end{aligned}$$

Symbols

$\mathcal{F}$ is a set whose members are themselves sets: the list of answerable questions. $A \in \mathcal{F}$ says “$A$ is answerable”. $A_1, A_2, \ldots$ is any list of answerable questions that can be counted off.

Read it as

“The sure question is answerable. The negation of an answerable question is answerable. For any countable list of answerable questions, ‘is at least one of these yes?’ is answerable.”

Intuition

(S1) and (S2) are the watchers’ first two rules, unchanged. (S3) is their “$A$ or $B$?”, stretched to countably many questions. That is the stretch “will a 7 ever appear?” required, and it goes exactly as far as probability’s addition reaches.

A σ-algebra describes what can be known. The colour-watcher’s four questions form one. So does the list you hold after reading $n$ digits of $U$, which contains $2^{10^{n}}$ questions. Put a probability $\mathbb{P}$ on the questions in $\mathcal{F}$ and you have a probability space, written $(\Omega, \mathcal{F}, \mathbb{P})$. The σ-algebra comes before $\mathbb{P}$, because it decides which questions $\mathbb{P}$ is allowed to answer.

Why not let every question in?

On the deck, the dealer’s σ-algebra contains every subset of $\Omega$, so every question has both an answer and a probability. Why not do the same for $U$, and let every subset of $[0,1)$ in?

Because it cannot be done. In 1905 Giuseppe Vitali found a set of numbers that cannot be given any probability, if probabilities are to add over countably many pieces and to stay the same when a set is shifted.

Picture a cake that can be cut into countably many pieces, all exact copies of one another, each turned to a different position. If one piece weighed $w$, the cake would weigh $w + w + w + \cdots$. That is nothing if $w = 0$ and infinitely much if $w > 0$, but never one kilogram. Such a piece cannot be weighed at all.

Vitali’s set $V$ is such a piece. Bend $[0,1)$ into a circle, so that 1 meets 0. The set $V$ can be built so that turning it by every fraction of a full turn gives copies that never overlap and together cover the circle exactly. There are countably many fractions, so there are countably many copies. Turning a set does not change its probability, because $U$ is spread evenly round the circle. So

$$1 = \mathbb{P}\big([0,1)\big) = \sum_{q} \mathbb{P}(V \oplus q) = \sum_{q} \mathbb{P}(V)$$

Symbols

$q$ runs over the fractions in $[0,1)$. $V \oplus q$ is $V$ turned by $q$ round the circle, and each copy has the same probability as $V$.

Read it as

“The probability of the whole circle, which is 1, equals the sum of the copies’ probabilities, and every copy has probability $\mathbb{P}(V)$.”

Intuition

A countable sum of one number, repeated, is 0 if the number is 0 and infinite otherwise. It is never 1, so no value of $\mathbb{P}(V)$ works. The only way out is to accept that “Is $U$ in $V$?” is a question with no probability, which means $V$ is not an event.

So we must choose which questions to allow, and the choice must obey the three rules. On a finite deck we could afford every question. On the continuum we cannot.

The same picture explains why the rules stop at countably many “or”s. Every set is a union of single points, and “Is $U$ exactly $\tfrac{1}{3}$?” turned out to be a legitimate question, an endless “and” of digit questions; the same goes for every other number. Rules allowing any number of “or”s would therefore let every subset in, Vitali’s set included.

The theory How Vitali builds his set, and whether it is real

Call two numbers in $[0,1)$ cousins if they differ by a fraction. Cousinhood splits $[0,1)$ into families. Each family is countable, so there must be uncountably many families. From each family choose one member, and let $V$ be the set of chosen members.

The copies never overlap. If $V \oplus q$ and $V \oplus q'$ shared a point, two chosen members would differ by a fraction, which makes them cousins. Only one member was chosen from each family, so they are the same member, and then $q = q'$.

The copies cover the circle. Any $x$ is a cousin of the member $v$ chosen from its family, so $x = v \oplus q$, where $q$ is the fraction $x - v$ taken round the circle.

The word “choose” carries the weight. Picking one member from each of uncountably many families uses the Axiom of Choice, and nobody can name the members. That cannot be avoided. Solovay (1970) showed that if the Axiom of Choice is weakened to a form called dependent choice, it is consistent that every set of numbers has a probability, provided a certain assumption about large infinities is itself consistent. So you will never meet a set like $V$ in a calculation; it exists because the Axiom of Choice says it does.

Could we keep every set by asking probabilities to add over finitely many pieces only? On the line and in the plane this works (Banach, 1923). In three dimensions it fails dramatically: a solid ball can be cut into finitely many pieces and reassembled into two balls, each the size of the original (Banach and Tarski, 1924).

Everything the digits can tell you

Which σ-algebra, then, should we use for $U$? The natural choice is every question that can be worked out from the digits. Start with the simple questions “is the $n$-th digit $k$?”, for every position $n$ and every digit $k$, and close them up under the three rules. What you get is the smallest σ-algebra that contains every digit question.

In general, the smallest σ-algebra containing a starting collection of questions $\mathcal{C}$ is written $\sigma(\mathcal{C})$ and called the σ-algebra generated by $\mathcal{C}$. It holds everything you can deduce from the questions in $\mathcal{C}$ using the three rules.

A detective’s casebook works the same way. She starts with a few clues, and the casebook holds every conclusion that follows from them by logic, including conclusions that need an endless but countable chain of steps, such as “a 7 appears”.

For $U$, this σ-algebra has a famous name: the Borel σ-algebra on $[0,1)$. Knowing the first $n$ digits is the same as knowing which interval of width $10^{-n}$ contains $U$, so it is also the σ-algebra generated by intervals, which is how textbooks usually introduce it. It contains every set you are likely to meet, including intervals, single points, the fractions and “a 7 appears”. It does not contain Vitali’s set.

Here the pixel picture bends. To someone who has read every digit, each single number is its own atom, because she can tell any two numbers apart. On the deck, a watcher who could tell every pair of cards apart could answer every question. Here she cannot. She knows $U$ exactly, and yet “Is $U$ in $V$?” still has no probability. On the continuum, “answerable” has to mean two things at once: settled by what you see, and able to carry a probability. A σ-algebra keeps track of both.

The theory Two theorems behind “generated”

Why a smallest one exists. $\sigma(\mathcal{C})$ is defined from above, as the intersection of every σ-algebra that contains $\mathcal{C}$:

$$\sigma(\mathcal{C}) = \bigcap \big\{ \mathcal{F} : \mathcal{F} \text{ a } \sigma\text{-algebra}, \ \mathcal{C} \subseteq \mathcal{F} \big\}$$

Symbols

The large $\bigcap$ keeps only the questions that lie on every σ-algebra containing $\mathcal{C}$.

Read it as

“σ of $\mathcal{C}$ consists of the questions that every σ-algebra containing $\mathcal{C}$ is forced to include.”

Intuition

Each of the three rules survives the intersection, so the result is a σ-algebra. It sits inside every other candidate, so it is the smallest. For the digit questions, building it from below instead, by applying the rules again and again, does not finish after countably many rounds. It settles only after uncountably many, which is why the definition works from above.

Why the digit probabilities decide everything. Suppose two probabilities agree on every question of the form “do the first $n$ digits read $d_1 d_2 \ldots d_n$?”. Then they agree on every set in the Borel σ-algebra. The tool is Dynkin’s π–λ theorem. The questions on which two probabilities agree always include $\Omega$, are closed under “not”, and are closed under “or” across non-overlapping questions, because probabilities add. That alone is not enough, since agreement can fail for “and”. But if the starting questions are themselves closed under “and”, agreement spreads to the whole σ-algebra they generate. Digit-prefix questions are closed under “and”, because two of them are either nested or disjoint.

This is why a distribution on the line is pinned down by the probabilities of intervals, so a CDF tells you everything. The condition on “and” matters. A pair of fair coins tossed independently and a pair glued together so that they always match agree on “the first is heads” and on “the second is heads”, but not on “both are heads”. That the uniform probabilities of the digit prefixes extend to the whole σ-algebra in the first place is the content of Carathéodory’s extension theorem.

Knowledge that grows

Read $U$ one digit at a time. What you know after $n$ digits is a σ-algebra, call it $\mathcal{F}_n$, with $10^{n}$ atoms. Each new digit splits every atom into ten, so any question you could answer before, you can still answer:

$$\mathcal{F}_1 \subseteq \mathcal{F}_2 \subseteq \mathcal{F}_3 \subseteq \cdots$$

Symbols

$\mathcal{F}_n$ is the list of questions answerable after $n$ digits, and $\subseteq$ says every question on the earlier list is also on the later one.

Read it as

“What you can answer after one digit you can still answer after two, and so on.”

Intuition

Knowledge accumulates and is never lost. A growing chain of σ-algebras like this is called a filtration, and it is how probability writes down the passage of time.

A number computed from $U$, such as its first digit or $U$ cut off after two decimals, is called a random variable. A random variable $X$ is knowable from $\mathcal{F}_n$ when every question about it is answerable from $\mathcal{F}_n$. It is enough to check the threshold questions “is $X$ at most $x$?”, because every other question about $X$ can be built from those with the three rules. So $X$ is called $\mathcal{F}_n$-measurable when

$$\{X \le x\} \in \mathcal{F}_n \quad \text{for every number } x$$

Symbols

$\{X \le x\}$ is the question “is $X$ at most $x$?”. Measurable is the official word for “knowable from this σ-algebra”.

Read it as

“For every $x$, the question ‘is $X$ at most $x$?’ can be answered after reading $n$ digits.”

Intuition

Measurable means computable from what you know. The first digit is $\mathcal{F}_1$-measurable, and $U$ cut off after two decimals is $\mathcal{F}_2$-measurable. $U$ itself is not $\mathcal{F}_n$-measurable for any $n$; you only learn it in the limit. The card table works the same way: the colour is measurable for the suit-watcher, and the number is not.

You can also make a best guess. After reading 0.31, $U$ lies somewhere in $[0.31, 0.32)$, with every position equally likely, so the best guess is the middle, 0.315. That guess is the conditional expectation of $U$ given $\mathcal{F}_2$, written $\mathbb{E}[U \mid \mathcal{F}_2]$.

Primer Conditional expectation, briefly

$\mathbb{E}[X]$ is the average value of $X$, weighted by probability. Given a single question $A$ with $\mathbb{P}(A) > 0$, $\mathbb{E}[X \mid A]$ is the average of $X$ over the outcomes in $A$ alone. Given a σ-algebra $\mathcal{G}$, the guess has to adapt to whatever $\mathcal{G}$ reveals, so $\mathbb{E}[X \mid \mathcal{G}]$ is a rule with one guess per atom:

$$\mathbb{E}[X \mid \mathcal{G}](\omega) = \mathbb{E}[X \mid G_\omega]$$

Symbols

$\omega$ is the true outcome, and $G_\omega$ is the atom of $\mathcal{G}$ that contains it: every outcome the observer cannot tell apart from $\omega$.

Read it as

“At the true outcome, the best guess given $\mathcal{G}$ is the average of $X$ over the atom you are in.”

Intuition

The guess depends on which atom you are in, so it is itself a random variable, and it is $\mathcal{G}$-measurable by construction. Among all guesses that use only $\mathcal{G}$’s information, it has the smallest mean squared error.

After $n$ digits of $U$ this gives the midpoint of your interval, its left end plus half its width:

$$\mathbb{E}[U \mid \mathcal{F}_n] = 0.d_1 d_2 \ldots d_n + \tfrac{1}{2} \cdot 10^{-n}$$

For σ-algebras whose atoms have probability zero, such as the Borel σ-algebra itself, the definition is phrased through averages over every event in $\mathcal{G}$ instead of over atoms. The idea is unchanged.

Digits read

0 1 2 3 4 5 6

What you know

U is in [0, 1)

Best guess

0.5, within 0.5

Answerable questions

$2^{1} = 2$

What you know

U is in [0.3, 0.4)

Best guess

0.35, within 0.05

Answerable questions

$2^{10} = 1{,}024$

What you know

U is in [0.31, 0.32)

Best guess

0.315, within 0.005

Answerable questions

$2^{100} \approx 1.27 \times 10^{30}$

What you know

U is in [0.314, 0.315)

Best guess

0.3145, within 0.0005

Answerable questions

$2^{1{,}000} \approx 1.07 \times 10^{301}$

What you know

U is in [0.3141, 0.3142)

Best guess

0.31415, within 0.00005

Answerable questions

$2^{10{,}000} \approx 2.00 \times 10^{3{,}010}$

What you know

U is in [0.31415, 0.31416)

Best guess

0.314155, within 0.000005

Answerable questions

$2^{100{,}000} \approx 9.99 \times 10^{30{,}102}$

What you know

U is in [0.314159, 0.314160)

Best guess

0.3141595, within 0.0000005

Answerable questions

$2^{1{,}000{,}000} \approx 9.90 \times 10^{301{,}029}$

Before any digits: U is in [0, 1)

0123456789

After 1 digit: U is in [0.3, 0.4)

0123456789

After 2 digits: U is in [0.31, 0.32)

0123456789

After 3 digits: U is in [0.314, 0.315)

0123456789

After 4 digits: U is in [0.3141, 0.3142)

0123456789

After 5 digits: U is in [0.31415, 0.31416)

0123456789

After 6 digits: U is in [0.314159, 0.314160)

0123456789

As $n$ grows, the guess closes in on the truth, and it is never more than $\tfrac{1}{2} \cdot 10^{-n}$ away. The chain $\mathcal{F}_1 \subseteq \mathcal{F}_2 \subseteq \cdots$ climbs towards the Borel σ-algebra, and the best guesses climb towards $U$ itself.

The theory Martingales: you cannot predict your own revisions

Write $M_n = \mathbb{E}[U \mid \mathcal{F}_n]$ for your guess after $n$ digits. Before reading the next digit, what do you expect your next guess to be? The ten possible next intervals are equally likely, and their midpoints average to your current midpoint, so

$$\mathbb{E}[M_{n+1} \mid \mathcal{F}_n] = M_n$$

Symbols

$M_{n+1}$ is the next guess, seen from now, and conditioning on $\mathcal{F}_n$ means using only what you know now.

Read it as

“Your expected next guess, given what you know now, is your current guess.”

Intuition

If you could predict which way you would revise, you would revise now. A sequence with this property, relative to a filtration, is called a martingale.

Paul Lévy’s upward theorem says that for any random variable $X$ with a finite average, $\mathbb{E}[X \mid \mathcal{F}_n]$ converges, with probability 1, to the best guess given everything the filtration eventually reveals. Here that is every digit, so the guesses converge to $U$ itself, as the figure shows.

Swap digits for daily closing prices and nothing changes. $\mathcal{F}_t$ is everything the market has revealed by day $t$. A trading rule is legitimate only if each day’s position is $\mathcal{F}_t$-measurable, which is the exact meaning of “no peeking at tomorrow”, and a backtest that breaks this has look-ahead bias. The price of an option on day $t$ is a conditional expectation given $\mathcal{F}_t$: a best guess of its discounted payoff, made with a particular pricing probability.

The fine print Where “information” stops being a perfect metaphor

Reading a σ-algebra as information is exact when there are finitely many atoms. In general it holds only up to events of probability zero, and Billingsley’s Probability and Measure uses an example that makes the point. On $[0,1)$, take the σ-algebra $\mathcal{G}$ of sets that are countable or whose complement is countable. It contains every single point, so in one sense it tells you $U$ exactly. Yet each of its sets has probability 0 or 1, so the best guess it supports is the same as knowing nothing: $\mathbb{E}[U \mid \mathcal{G}] = \tfrac{1}{2}$. Information that only settles questions whose probability is already 0 or 1 is, to probability theory, no information at all.

The idea in one breath

What someone knows is a list of answerable questions. Answering puts the sure question on that list and keeps it closed under “not” and “or”. Infinity stretches “or” to countably many questions, and probability, which can only add countably many pieces, stops it there. Infinity also rules out letting every question in, because some sets cannot carry a probability. A σ-algebra is what remains: the questions that can be both settled and weighed.

Check yourself

1. The colour-watcher and the rank-watcher pool what they see. How many questions can they answer together? Can they answer “Is it the queen of hearts?”

   Answer

   Together they know colour and rank, so their atoms are pairs such as “red queen”: $2 \times 13 = 26$ atoms and $2^{26} \approx 6.7 \times 10^{7}$ questions. “Is it the queen of hearts?” splits the atom “red queen”, which also holds the queen of diamonds, so they cannot answer it. Pooled knowledge is the σ-algebra generated by both lists together.

2. Call a question about $U$ finitely settled if it can always be answered after reading some fixed number of digits. Do the finitely settled questions form a σ-algebra?

   Answer

   They satisfy (S1), (S2) and the finite version of (S3), so they form an algebra of sets. They fail (S3) itself. Each $\{d_n = 7\}$ is finitely settled, but their union, “a 7 appears”, is not: whatever fixed number of digits you read, if none of them is a 7 you still cannot answer. This gap is exactly the one the σ closes.

3. Is “$U$ rounded to the nearest hundredth” $\mathcal{F}_2$-measurable? What about “$U$ cut off after two decimals”?

   Answer

   Cutting off gives $0.d_1 d_2$, which you know after two digits, so it is $\mathcal{F}_2$-measurable. Rounding is not. The numbers 0.314 and 0.315 lie in the same atom $[0.31, 0.32)$ of $\mathcal{F}_2$, but the first rounds to 0.31 and the second to 0.32. Rounding needs the third digit, so it is $\mathcal{F}_3$-measurable.

To go further: Billingsley’s Probability and Measure, sections 1 and 2, builds probability from the binary digits of a random number, with full proofs. Williams’s Probability with Martingales, chapters 1 to 3, is a short, rigorous treatment of σ-algebras as information.