crypto-quant-eval

Institutional-grade evaluation harness for crypto quant trading strategies, plus systematic strategy sweeps (hourly single-asset and daily multi-asset). The companion dataset lives at Avi-L/crypto-quant-eval-data.

What this eval measures

A strategy "passes" only if it survives all of:

Check	Where	Rule
No lookahead (purge + embargo)	crypto_eval/cv.py	label window [t, t+h] must never overlap a test block; embargo >= horizon (8h = one funding interval) enforced
Leakage audit	crypto_eval/cv.py	independent per-segment audit of every CPCV fold; run fails loudly on violation
Fee-aware PnL	crypto_eval/costs.py	5 bps taker / 2 bps maker (Binance VIP0), funding accrued at 8h ticks
Slippage	crypto_eval/costs.py	square-root impact k·σ_daily·√(Q/ADV), σ_daily = realized daily vol from the data (317 bps on this dataset), 2 bps floor
Deflated Sharpe (DSR)	crypto_eval/stats.py	accept only if DSR > 0.95
Probability of Backtest Overfitting (PBO)	crypto_eval/stats.py	accept only if PBO <= 0.5
Min Track Record Length	crypto_eval/stats.py	how long the track record must be before SR is significant

Splitting uses CPCV (N=6 blocks, K=2 held out → 15 combinations) via purgedcv 0.1.11, following López de Prado, Advances in Financial Machine Learning (Wiley 2018), ch. 7 (purged K-fold + embargo) and ch. 11–12 (DSR, PBO, MinBTL).

Quickstart

pip install -r requirements.txt
python run_eval.py --data /path/to/curated --symbol BTCUSDT --freq 1h \
    --strategy funding_carry

--data is a directory containing the dataset repo's parquet files (bars_BTCUSDT_1h.parquet, funding_BTCUSDT.parquet, manifest.json). Strategies are functions (bars, funding) -> positions; add yours to crypto_eval/strategies.py. Decision-time discipline is enforced structurally: a bar starting at t closes at t+freq, so signals use shift_bars (last closed bar at t-1); the engine and audit catch peeking.

Strategy sweep v2 — daily multi-asset panel (2026-10-25): PASS

66 configurations (panel TSMOM/Donchian × 24+8, XS momentum ± trend filter × 18, single-asset BTC/ETH long-only trend × 16) through the full gates on daily data 2014-01 → 2026-10, 4,526 days × 8 majors (coinmetrics VWAP index, 5bps/2bps fees, per-asset ADV impact, 0.1095 APR funding drag).

Winner: Donchian 20d long-only on ETH — the first config to clear every gate:

Gate	Value	Required
Deflated Sharpe (66 trials)	0.9981	> 0.95	✅
PBO (CSCV, 8 splits)	0.257	≤ 0.5	✅
Leakage audit (15 CPCV combos)	OK	no leakage	✅
OOS mean SR (CPCV, purge+embargo 5d)	1.48	—	15/15 combos positive
Full-sample SR / MDD	1.56 / −61%	—	skew +1.28, kurt 18

Era analysis (mandatory honesty check): the ETH winner's edge concentrates pre-2022 (2023-26 era SR 0.50). The era-stable picks are BTC long-only trend: Donchian 200d (recent-era SR 0.92, full 0.93), TSMOM 50d (recent 0.89), Donchian 20d (recent 0.87) — lower full-sample SR (~0.9) but no era decay. Engine fix found during this sweep: the portfolio divisor previously granted implicit early-era leverage; corrected (unlisted capital = cash), which lowered all panel numbers — the PASS survived the fix.

Read: long-only trend following on daily crypto is a statistically significant, era-stable (BTC) edge after full institutional costs — with −60/−80% drawdowns inherent to the unlevered profile and an edge magnitude that decays as the market matures.

Strategy sweep v1 — hourly single-asset (2026-10-25): FAIL

Run on real curated data: BTCUSDT 1h, 59,654 bars (2020-01 → 2026-10), 5,295 funding ticks, $1M initial capital, CPCV N=6/K=2, horizon=embargo=8h.

A 136-configuration grid (TSMOM × lookback/long-only/vol-target/band, Donchian, funding carry, funding-extreme events, mean-reversion controls) was run through the full harness on real data — every config a counted trial. Full 136-config ranking in results/sweep_report_v1.json; runner: sweep.py.

VERDICT: FAIL — no configuration passes the institutional gate.

Rank	Config	OOS mean SR (CPCV)	Combos positive	Net PnL	Gates
1	Donchian 55d long-only	0.41	12/15	+$61,557	DSR≈0 ✗, PBO 0.63 ✗
2	TSMOM 65d long-only	0.35	11/15	+$54,597	DSR≈0 ✗
3	TSMOM 200d long-only + vol-target 30% + 20% band	0.39	11/15	+$35,178	DSR≈0 ✗

The best strategies are modestly positive OOS (SR ~0.3-0.4, essentially long-bias trend capture in a 2020-2026 bull market) but cannot clear the multiple-testing bar: after 136 counted trials, DSR ≈ 0, and PBO 0.63 says config selection inside this grid is worse than a coin flip. Mean-reversion and short-horizon momentum controls are strongly negative, as the literature predicts. Funding carry is positive (+$22k) but individually insignificant (PSR 0.53).

That is the honest institutional answer: a deployable edge is not found by grid search on one asset's 1h bars. The harness is doing its job.

Demo-strategy validation results (costs corrected 2026-10-25)

NOTE: the original 2026-10-23 run used a mis-scaled slippage model (missing the σ_daily term, impact ~50× overstated). All results below use the corrected model. The correction is reported openly because it changed the carry verdict.

Strategy	Net PnL	Fees	Slippage	Funding	Mean OOS SR	PBO	Verdict
funding_carry	+$22,242	$2,856	$3,016	+$5,940	+0.17	0.100	REJECT (PSR 0.53)
ma_crossover	−$62,096	$29,208	$47,872	−$3,469	−0.21	0.057	REJECT

The MA crossover remains a cost-loser. The carry trade flips positive once slippage is realistic, but PSR 0.53 is nowhere near significance.

Note on the DSR column: per-combination values are PSR (n_trials=1 within a combination); DSR applies when multiple strategy configurations are compared — eval_trial_set computes it with the across-trial SR variance.

Validation performed

• Synthetic-data smoke: 20k GBM bars, both strategies, numeric DSR/PBO (synth_gen.py)
• Leakage test: --embargo 0 with horizon 8 raises ValueError (guard in make_cpcv)
• Sanity: zero-position strategy produces exactly 0.0 net PnL; any turnover produces strictly positive fees + slippage
• No-lookahead audit passes on all 15 CPCV combinations, both real and synthetic data

Known gaps

• ETH enters via funding only; ETH spot 1m (HaianCao/Binance-Public-OHLCV, 34.6 GB) was not ingested
• On-chain metrics are daily (coinmetrics-community), no usable HF on-chain repo exists
• PBO uses n_splits=8 (70 combos) rather than 16 (12,870) — the latter is prohibitively slow on CPU with purge/embargo enabled
• Cross-venue funding divergence (Chainticks/funding-divergence) is verified as a source but not yet wired into the harness

References

• López de Prado, Advances in Financial Machine Learning, Wiley 2018 (ch. 7, 11, 12)
• purgedcv — MIT-licensed implementation of the splitting and statistics used here