The Betting Oracle

Sequential hypothesis testing by betting reframes statistical inference as a gambling game. A statistician places bets against the null hypothesis, and if the accumulated wealth grows large enough, the null is rejected. The wealth process — an e-process — replaces the p-value, with the advantage that it allows optional stopping: you can peek at the data as it arrives without inflating the false positive rate.

Sandoval, Waudby-Smith, and Jordan extend this to multiple arms. At each step, the statistician chooses which data source to sample — which arm to pull. The global null says all arms are null; the alternative says at least one isn’t. The goal is to reject the global null as quickly as an oracle that knows in advance which arm provides the strongest evidence.

The oracle knows something the statistician doesn’t: which arm to focus on. The statistician must explore (sample different arms to estimate which is most informative) and exploit (sample the most informative arm to accumulate evidence fastest). This is the multi-armed bandit problem wearing statistical clothes.

The authors prove matching upper and lower bounds for both log-optimal wealth growth and expected rejection time. A modified upper-confidence-bound algorithm achieves oracle-optimal performance nonasymptotically, with explicit concentration inequalities for the wealth growth rate in the sense of Kelly.

The bridge between gambling theory and hypothesis testing is complete: the optimal test is the optimal bet, the optimal arm allocation is the optimal exploration strategy, and the oracle’s advantage over the statistician is quantified exactly.