Exchangeability and de Finetti
Alexandre Bouchard-Côté
De Finetti: intuition and motivation
- We have invoked the “exchangeable/iid” setup several times already
- De Finetti theorem will give us more insight in this class of models
- Motivation: often, may not want to rely on the order of the rows in the spreadsheet given to you. So we may want to shuffle it to be safe.
- In such circustance, de Finetti motivates the existence of a well-specified model with the graphical model on the right
- I.e. that there exists a prior such that the data is iid given that prior
- This motivation gives you an idea of the theorem, but is not actually 100% exactly what de Finetti says, let’s formalize the theorem!
Exchangeable random variables
- Recall: notion of equality in distribution \(X_1 {\stackrel{\scriptscriptstyle d}{=}}X_2\)
- …this means the distribution of \(X_1\) is equal to the distribution of \(X_2\)
- …concretely: draw the marginal PMF or density of each random variable, check if they match
Poll: Are there random variables where \(X_1 {\stackrel{\scriptscriptstyle d}{=}}X_2\) but \(X_1 \neq X_2\)?
- This cannot happen \(X_1 {\stackrel{\scriptscriptstyle d}{=}}X_2\) if and only if \(X_1 = X_2\)
- You flip a fair coin and say what is on the top \(X_1\). Your friend flip another coin \(X_2\) with bias 1/3
- You flip a fair coin and say what is on the top \(X_1\). Your friend say what is on the bottom \(X_2\)
- You flip a fair coin and say what is on the top \(X_1\). Your friend say what is on the top, \(X_2\)
- Two random variables are exchangeable if \((X_1, X_2) {\stackrel{\scriptscriptstyle d}{=}}(X_2, X_1)\)
- Example: indicator variable (i.e. Bernoulli random variables).
Poll: Develop a criterion for checking from the joint density if it’s exchangeable.
- It should be invariant to 90 degree rotation around the origin (clockwise)
- It should be invariant to 90 degree rotation around the origin (counter-clockwise)
- It should be symmetric with respect to the line \(y = -x\)
- It should be symmetric with respect to the line \(y = x\)
Examples of exchangeable random variables
- Using our criterion (symmetry along the line \(y = x\)), we obtain that the indicator on a square, a circle, and checkers board centered at zero all lead to exchangeable random variables.
Poll: Which one is iid (in any)
- Only the square is iid
- Only the circle is iid
- Only the checker board is iid
- All of them
- None of them
- Note: exchangeability implies identical distributions
- Note: this notion is closely related to reversibility
- Generalization: \(n\) variables are exchangeable if for all permutations \(\sigma : \{1, 2, \dots, n\} \to \{1, 2, \dots, n\} \in S_n\), \((X_1, X_2, \dots, X_n) {\stackrel{\scriptscriptstyle d}{=}}(X_{\sigma(1)}, X_{\sigma(2)}, \dots, X_{\sigma(n)})\)
- Infinite exchangeable: an infinite sequence of random variable \((X_1, X_2, \dots)\) is exchangeable if all finite subsets are exchangeable.
Optional readings
- More background on exchangeability and de Finetti theorems:
- State of the art work developed by UBC faculty: