| # ArbIntel Part 2: Smoothing the Noise with Bayesian Statistics and Kalman Filters |
|
|
| In Part 1, we established ArbIntel's unified data pipeline. However, prediction markets are notoriously noisy. A single retail trader's market order on Polymarket can dramatically swipe an illiquid order book, creating a temporary price spike that doesn't reflect a true change in the underlying event's probability. |
|
|
| How do we differentiate between random bid-ask bounce and genuine new information? The answer lies in state-space smoothing and Bayesian updating. |
|
|
| ## The 1D Kalman Filter |
|
|
| A Kalman Filter is recursive. It estimates the true state of a system (the "Fair Value" of an asset) from a series of incomplete and noisy measurements (the traded prices). |
|
|
| In ArbIntel, we use a 1D implementation: |
| * **State Estimate ($x_t$)**: Our current belief of the true probability. |
| * **Process Variance ($Q$)**: How fast we think the true probability fundamentally changes over time (low for stable markets, high for volatile ones). |
| * **Measurement Variance ($R$)**: The estimated noise in the exchange's order book. |
| |
| When a new trade occurs, the Kalman Filter calculates the Kalman Gain ($K$)—essentially deciding whether to trust the new market print or our existing belief. This allows ArbIntel to completely ignore short-term liquidity sweeps. |
| |
| ## Bayesian Fair Value |
| |
| While the Kalman filter is great for smoothing, it doesn't establish an intrinsic "Fair Value." For this, we treat the probability of an event resolving "YES" as a Beta distribution, $Beta(\alpha, \beta)$. |
| |
| 1. **The Prior**: We establish base rates (e.g., incumbents win elections 65% of the time $\rightarrow Beta(65, 35)$). |
| 2. **The Update**: When trades occur on Polymarket, we adjust our $\alpha$ and $\beta$ parameters scaled by the trade volume. High volume trades shift our belief significantly; low volume trades are treated as noise. |
| |
| By combining the structural rigidity of a Kalman Filter with the probabilistic updates of a Bayesian model, ArbIntel refuses to be faked out by market microstructure dynamics. |
| |
| In **Part 3**, we'll discuss the easiest money in prediction markets: Cross-Platform Arbitrage. |
| |
