Bit-Trading-Company commited on
Commit
d8478d9
Β·
verified Β·
1 Parent(s): 1139375

Add HANDOFF.md

Browse files
Files changed (1) hide show
  1. HANDOFF.md +166 -0
HANDOFF.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Backtest Lab v1.0.0 β€” Handoff
2
+
3
+ Built end-to-end and deployed. Everything below was verified against the **live
4
+ Space**, not just locally.
5
+
6
+ ## Live URLs
7
+
8
+ | What | URL |
9
+ |---|---|
10
+ | **Backtest Lab** (Space) | https://huggingface.co/spaces/Bit-Trading-Company/bit-backtest-lab |
11
+ | Direct app | https://bit-trading-company-bit-backtest-lab.hf.space |
12
+ | **Signal store** (Dataset) | https://huggingface.co/datasets/The-Bit-Trading-Company/bit-signal-store |
13
+
14
+ Both tagged `v1.0.0`. Space stage: **RUNNING** on **`zero-a10g` (ZeroGPU)**.
15
+
16
+ ## Deployment status
17
+
18
+ | Check | Result |
19
+ |---|---|
20
+ | Build | βœ… 142s build + ~30s app start |
21
+ | Cold start (app boot) | βœ… ~30s, under the 60s budget |
22
+ | HTTP response | βœ… 200 in 0.74s |
23
+ | Hardware | βœ… ZeroGPU (`zero-a10g`) **granted** β€” no billing escalation needed |
24
+ | Example config end-to-end | βœ… `/load_example` returns the design's worked example |
25
+ | Backtest on live Space | βœ… +33.2%, 12 trades, $486 costs β€” **bit-identical to local** |
26
+ | Warm run time | βœ… **0.1s** engine time for a 3-year daily run (budget: <2s) |
27
+ | Coverage map from live manifest | βœ… 18 rows, all labelled `real` |
28
+ | Login button | βœ… present; `/login/huggingface` returns 302 |
29
+ | ZeroGPU detected in-app | βœ… "ZEROGPU AVAILABLE Β· EXTENSION RUNS ON YOUR OWN QUOTA" |
30
+
31
+ ## Tests: all green
32
+
33
+ `bash tests/run_all.sh --slow` β†’ **191 passed**, run with `-W error::FutureWarning`.
34
+
35
+ Verified on **both** Python 3.11 (dev) and **Python 3.10** (what the ZeroGPU
36
+ image actually provides) β€” 190 offline + 1 real-model test.
37
+
38
+ ### The six known-answer tests
39
+
40
+ | # | Test | Status |
41
+ |---|---|---|
42
+ | 1 | Buy & Hold, zero costs, reproduces the asset return (tol 1e-9) | βœ… |
43
+ | 2 | Deliberately lookahead-biased strategy is caught structurally | βœ… |
44
+ | 3 | Zero-cost vs costed run differ by exactly the modelled costs | βœ… |
45
+ | 4 | Zero-signal strategy β†’ flat equity, zero trades | βœ… |
46
+ | 5 | Walk-forward train/test never overlap; holdout untouched by selection | βœ… |
47
+ | 6 | Same config + same data β†’ bit-identical results | βœ… |
48
+
49
+ 31 tests cover these six directly. Test 2 catches three distinct cheats:
50
+ reading bar `t+1`, a centred rolling window, and full-sample normalisation.
51
+
52
+ ## Spec compliance
53
+
54
+ - βœ… **No lookahead paths.** Enforced two ways: the engine applies the one-bar
55
+ shift itself (a strategy cannot bypass it), and `assert_causal` proves it by
56
+ perturbing future prices up *and* down and requiring earlier outputs to be
57
+ bit-identical. All seven shipped presets pass.
58
+ - βœ… **Fills at next bar open.** `fill` accepts only `next_open`; anything else
59
+ raises. A test asserts the entry price equals bar 1's open, not bar 0's.
60
+ - βœ… **Costs default ON.** `Costs().enabled is True`. `gross βˆ’ costs = net` holds
61
+ to 1e-9 on every trade row. Costs-off runs are labelled "these numbers are
62
+ not achievable" in the UI.
63
+ - βœ… **Every displayed metric traceable to engine output.** The stat band, trade
64
+ table and report are asserted against `metrics.py` values in `test_ui.py`.
65
+ - βœ… **No placeholder slices remain.** All 18 signal slices are real
66
+ Chronos-Bolt output. The `PLACEHOLDER` labelling machinery is still present
67
+ and tested for contributors on weaker hardware.
68
+ - βœ… **No arbitrary code execution.** Adapter families are allow-listed, model
69
+ ids are validated, `trust_remote_code` is never set, and the design's
70
+ `Custom (code)` preset is rendered disabled with an explanation.
71
+ - βœ… **Token hygiene.** Scanned every file in both repos: the token appears in
72
+ none of them. It exists only in the local gitignored `key.txt` and as the
73
+ `HF_WRITE_TOKEN` Space secret.
74
+
75
+ ## What shipped
76
+
77
+ **Signal store** β€” 18 signal slices / **60,375 real rows** across
78
+ `chronos-bolt-small` + `chronos-bolt-base`, 6 assets (BTC, ETH, SOL, SPY, QQQ,
79
+ NVDA) Γ— 3 timeframes (1d, 1h, 15m); 15 price slices; 4 precomputed comparison
80
+ tables (model performance, calibration, directional accuracy, strategyΓ—timeframe
81
+ heatmap).
82
+
83
+ **Derived data worth reading**: Chronos-Bolt is consistently **over-dispersed** β€”
84
+ its nominal 80% q10–q90 band actually covers 83–89% of outcomes across every
85
+ slice. Worth knowing before trusting a q10 stop level.
86
+
87
+ **Engine** β€” vectorbt, next-bar-open fills, commission + fixed/volume-scaled
88
+ slippage, SL/TP/trailing stops, fixed-% and vol-target sizing, three validation
89
+ modes with a locked holdout, full IS/OOS/holdout metric split, trade list with
90
+ R-multiple, MAE/MFE and per-trade trigger reasons.
91
+
92
+ **UI** β€” the design implemented in Gradio Blocks with the Bit design system
93
+ vendored verbatim (tokens + Styrene/Mac Minecraft fonts): Strategy Builder
94
+ (5 numbered sections), Results Canvas (Overview / Trades / Comparison /
95
+ Robustness / Report / Coverage), Run Manager, pinned disclaimer footer.
96
+
97
+ ## ⚠️ Human-verify checklist
98
+
99
+ These need a real human with a real account β€” I cannot complete them:
100
+
101
+ 1. **OAuth login end-to-end.** The button renders and `/login/huggingface`
102
+ redirects correctly, but I cannot complete a real sign-in. Click
103
+ *Sign in with Hugging Face* and confirm you land back signed in.
104
+ If it fails, check Space settings β†’ the `hf_oauth: true` header is set in
105
+ `README.md` and may need the owner to approve the OAuth app once.
106
+ 2. **ZeroGPU extension with a real account.** Sign in, open the **Coverage**
107
+ tab, pick an uncovered range, press *Estimate* then *Extend coverage*.
108
+ Confirm: progress stages appear, the commit lands in the dataset repo, and
109
+ the toast reads "Coverage extended by @you". This is the only path that
110
+ spends real GPU quota, so it is untested against real hardware.
111
+ 3. **Add-model flow** with a real Hub model id, same reason.
112
+ 4. **Torch version difference.** The Space runs torch **2.11.0** (ZeroGPU only
113
+ accepts 2.11/2.10/2.9.1/2.8); my adapter smoke test ran locally on 2.13.0.
114
+ The first real ZeroGPU extension doubles as the check that Chronos inference
115
+ behaves identically on 2.11.
116
+ 5. **Visual pass on a real screen.** I verified the UI functionally and via
117
+ screenshots, but the in-app browser stalled before I could click through
118
+ every tab at full size. Worth a human eye on the Comparison and Robustness
119
+ tabs with several runs loaded.
120
+
121
+ ## Open items from DECISIONS.md
122
+
123
+ - **D-001 β€” Space is under the personal namespace.** Gradio Spaces under an org
124
+ need a paid Team/Enterprise plan (402). You chose the personal namespace; the
125
+ dataset stays under the org. To move it later, subscribe and re-point
126
+ `BIT_SPACE_REPO`.
127
+ - **D-006 β€” Binance is geo-blocked (451)** from the seeding machine; crypto came
128
+ from Coinbase via the fallback chain. **Stooq now serves an HTML block page**,
129
+ so the equity chain effectively runs on yfinance alone. Tiingo is wired but
130
+ needs a `TIINGO_KEY` secret to join the chain.
131
+ - **Sentiment is a stub.** `Sentiment-Gated Momentum` runs on a labelled,
132
+ deterministic price-derived proxy behind a `SentimentSource` interface β€” not
133
+ news sentiment. Wiring a real feed is the obvious next task.
134
+ - **Pairs Trading** is in the design's preset list but not runnable in v1
135
+ (needs a second leg). It is shown disabled with that reason.
136
+ - **Equity intraday depth** is provider-capped (~730d hourly, ~60d 15m) and
137
+ recorded in the manifest as `provider_max_days`, not hidden.
138
+
139
+ ## Things I'd flag as genuinely uncertain
140
+
141
+ - The **example strategy loses money on the locked holdout** (βˆ’5.2%, Sharpe
142
+ βˆ’0.38). That is the system working: the holdout is doing its job and the app
143
+ reports it rather than burying it. Don't read the headline +33.2% as a result.
144
+ - **Walk-forward on short ranges produces no OOS windows at all.** The app now
145
+ says so explicitly instead of printing `0.00`, but it is easy to configure a
146
+ run that looks complete and has no out-of-sample content. Watch for the red
147
+ note.
148
+ - **`vectorbt` is pinned to 1.0.0**, not the 1.1.0 I developed against, because
149
+ 1.1.0 requires Python β‰₯3.11 and the ZeroGPU image is 3.10. The full suite
150
+ passes on both, but that is a real version skew between dev and prod.
151
+
152
+ ## Repo layout
153
+
154
+ ```
155
+ hugging-face/
156
+ key.txt local only, gitignored, never committed
157
+ DECISIONS.md 15 logged decisions across all phases
158
+ HANDOFF.md this file
159
+ ops/ provisioning scripts (local only, not deployed)
160
+ store_stage/ dataset card staging
161
+ space/ β†’ pushed to the Space
162
+ app.py src/{store,data,engine,strategies,adapters,charts,metrics,
163
+ comparisons,runtime,extension,config,ui/theme}.py
164
+ scripts/seed_store.py tests/ (191) assets/ (vendored design system)
165
+ DESIGN_NOTES.md the extracted design contract Phase 3 was built against
166
+ ```