eagL / backtesting

Backtesting prediction markets without fooling yourself

Almost every prediction-market backtest overstates edge, and the failure modes are the same three every time: random splits, unlimited parameter sweeps, and statistics computed on samples too small to mean anything. eagL builds the defenses in.

Chronological holdouts, never random ones

A random train-test split leaks the future into the past: the model tunes on Tuesday afternoon and gets tested on Tuesday morning. eagL splits time in order, 60% early for tuning and 40% late held out. The held-out window is touched exactly once, and the app shows you both numbers plus the drop between them. The drop is the honest statistic. A strategy that earns 8% in sample and 1% out of sample is a 1% strategy with good marketing.

Capped sweeps

Run enough parameter combinations and one of them will look brilliant by luck alone. eagL caps the sweep at a few hundred combinations, because past that point you are selecting for noise. The cap is a limit that only ever tightens; there is no setting to search harder until the backtest flatters you.

Minimum samples or no statistics at all

Before eagL publishes a figure, it requires 100 detections, 500 quote frames, 50 paper fills, 100 settled outcomes and seven days of watching. Below any of those floors it throws the number away rather than printing it with a footnote, because a footnote never survives a screenshot.

Scoring forecasts, not just trades

Trade PnL is a noisy grade for a probability model. eagL also scores every resolved forecast directly with Brier score, log loss and a ten-bin calibration curve, so you can see whether events it called 70% happened about 70% of the time. The model behind those forecasts is described on the crypto markets page.

Executable, not theoretical

The final trap is backtesting against screen prices. eagL charges each venue's fee formula at the projected fill VWAP and replays quotes forward to check both legs were reachable, the same accounting covered on the arbitrage page. The complete regime sits on the methodology page.