Spaces:
Running on Zero
Running on Zero
Add HANDOFF.md
Browse files- HANDOFF.md +166 -0
HANDOFF.md
ADDED
|
@@ -0,0 +1,166 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Backtest Lab v1.0.0 β Handoff
|
| 2 |
+
|
| 3 |
+
Built end-to-end and deployed. Everything below was verified against the **live
|
| 4 |
+
Space**, not just locally.
|
| 5 |
+
|
| 6 |
+
## Live URLs
|
| 7 |
+
|
| 8 |
+
| What | URL |
|
| 9 |
+
|---|---|
|
| 10 |
+
| **Backtest Lab** (Space) | https://huggingface.co/spaces/Bit-Trading-Company/bit-backtest-lab |
|
| 11 |
+
| Direct app | https://bit-trading-company-bit-backtest-lab.hf.space |
|
| 12 |
+
| **Signal store** (Dataset) | https://huggingface.co/datasets/The-Bit-Trading-Company/bit-signal-store |
|
| 13 |
+
|
| 14 |
+
Both tagged `v1.0.0`. Space stage: **RUNNING** on **`zero-a10g` (ZeroGPU)**.
|
| 15 |
+
|
| 16 |
+
## Deployment status
|
| 17 |
+
|
| 18 |
+
| Check | Result |
|
| 19 |
+
|---|---|
|
| 20 |
+
| Build | β
142s build + ~30s app start |
|
| 21 |
+
| Cold start (app boot) | β
~30s, under the 60s budget |
|
| 22 |
+
| HTTP response | β
200 in 0.74s |
|
| 23 |
+
| Hardware | β
ZeroGPU (`zero-a10g`) **granted** β no billing escalation needed |
|
| 24 |
+
| Example config end-to-end | β
`/load_example` returns the design's worked example |
|
| 25 |
+
| Backtest on live Space | β
+33.2%, 12 trades, $486 costs β **bit-identical to local** |
|
| 26 |
+
| Warm run time | β
**0.1s** engine time for a 3-year daily run (budget: <2s) |
|
| 27 |
+
| Coverage map from live manifest | β
18 rows, all labelled `real` |
|
| 28 |
+
| Login button | β
present; `/login/huggingface` returns 302 |
|
| 29 |
+
| ZeroGPU detected in-app | β
"ZEROGPU AVAILABLE Β· EXTENSION RUNS ON YOUR OWN QUOTA" |
|
| 30 |
+
|
| 31 |
+
## Tests: all green
|
| 32 |
+
|
| 33 |
+
`bash tests/run_all.sh --slow` β **191 passed**, run with `-W error::FutureWarning`.
|
| 34 |
+
|
| 35 |
+
Verified on **both** Python 3.11 (dev) and **Python 3.10** (what the ZeroGPU
|
| 36 |
+
image actually provides) β 190 offline + 1 real-model test.
|
| 37 |
+
|
| 38 |
+
### The six known-answer tests
|
| 39 |
+
|
| 40 |
+
| # | Test | Status |
|
| 41 |
+
|---|---|---|
|
| 42 |
+
| 1 | Buy & Hold, zero costs, reproduces the asset return (tol 1e-9) | β
|
|
| 43 |
+
| 2 | Deliberately lookahead-biased strategy is caught structurally | β
|
|
| 44 |
+
| 3 | Zero-cost vs costed run differ by exactly the modelled costs | β
|
|
| 45 |
+
| 4 | Zero-signal strategy β flat equity, zero trades | β
|
|
| 46 |
+
| 5 | Walk-forward train/test never overlap; holdout untouched by selection | β
|
|
| 47 |
+
| 6 | Same config + same data β bit-identical results | β
|
|
| 48 |
+
|
| 49 |
+
31 tests cover these six directly. Test 2 catches three distinct cheats:
|
| 50 |
+
reading bar `t+1`, a centred rolling window, and full-sample normalisation.
|
| 51 |
+
|
| 52 |
+
## Spec compliance
|
| 53 |
+
|
| 54 |
+
- β
**No lookahead paths.** Enforced two ways: the engine applies the one-bar
|
| 55 |
+
shift itself (a strategy cannot bypass it), and `assert_causal` proves it by
|
| 56 |
+
perturbing future prices up *and* down and requiring earlier outputs to be
|
| 57 |
+
bit-identical. All seven shipped presets pass.
|
| 58 |
+
- β
**Fills at next bar open.** `fill` accepts only `next_open`; anything else
|
| 59 |
+
raises. A test asserts the entry price equals bar 1's open, not bar 0's.
|
| 60 |
+
- β
**Costs default ON.** `Costs().enabled is True`. `gross β costs = net` holds
|
| 61 |
+
to 1e-9 on every trade row. Costs-off runs are labelled "these numbers are
|
| 62 |
+
not achievable" in the UI.
|
| 63 |
+
- β
**Every displayed metric traceable to engine output.** The stat band, trade
|
| 64 |
+
table and report are asserted against `metrics.py` values in `test_ui.py`.
|
| 65 |
+
- β
**No placeholder slices remain.** All 18 signal slices are real
|
| 66 |
+
Chronos-Bolt output. The `PLACEHOLDER` labelling machinery is still present
|
| 67 |
+
and tested for contributors on weaker hardware.
|
| 68 |
+
- β
**No arbitrary code execution.** Adapter families are allow-listed, model
|
| 69 |
+
ids are validated, `trust_remote_code` is never set, and the design's
|
| 70 |
+
`Custom (code)` preset is rendered disabled with an explanation.
|
| 71 |
+
- β
**Token hygiene.** Scanned every file in both repos: the token appears in
|
| 72 |
+
none of them. It exists only in the local gitignored `key.txt` and as the
|
| 73 |
+
`HF_WRITE_TOKEN` Space secret.
|
| 74 |
+
|
| 75 |
+
## What shipped
|
| 76 |
+
|
| 77 |
+
**Signal store** β 18 signal slices / **60,375 real rows** across
|
| 78 |
+
`chronos-bolt-small` + `chronos-bolt-base`, 6 assets (BTC, ETH, SOL, SPY, QQQ,
|
| 79 |
+
NVDA) Γ 3 timeframes (1d, 1h, 15m); 15 price slices; 4 precomputed comparison
|
| 80 |
+
tables (model performance, calibration, directional accuracy, strategyΓtimeframe
|
| 81 |
+
heatmap).
|
| 82 |
+
|
| 83 |
+
**Derived data worth reading**: Chronos-Bolt is consistently **over-dispersed** β
|
| 84 |
+
its nominal 80% q10βq90 band actually covers 83β89% of outcomes across every
|
| 85 |
+
slice. Worth knowing before trusting a q10 stop level.
|
| 86 |
+
|
| 87 |
+
**Engine** β vectorbt, next-bar-open fills, commission + fixed/volume-scaled
|
| 88 |
+
slippage, SL/TP/trailing stops, fixed-% and vol-target sizing, three validation
|
| 89 |
+
modes with a locked holdout, full IS/OOS/holdout metric split, trade list with
|
| 90 |
+
R-multiple, MAE/MFE and per-trade trigger reasons.
|
| 91 |
+
|
| 92 |
+
**UI** β the design implemented in Gradio Blocks with the Bit design system
|
| 93 |
+
vendored verbatim (tokens + Styrene/Mac Minecraft fonts): Strategy Builder
|
| 94 |
+
(5 numbered sections), Results Canvas (Overview / Trades / Comparison /
|
| 95 |
+
Robustness / Report / Coverage), Run Manager, pinned disclaimer footer.
|
| 96 |
+
|
| 97 |
+
## β οΈ Human-verify checklist
|
| 98 |
+
|
| 99 |
+
These need a real human with a real account β I cannot complete them:
|
| 100 |
+
|
| 101 |
+
1. **OAuth login end-to-end.** The button renders and `/login/huggingface`
|
| 102 |
+
redirects correctly, but I cannot complete a real sign-in. Click
|
| 103 |
+
*Sign in with Hugging Face* and confirm you land back signed in.
|
| 104 |
+
If it fails, check Space settings β the `hf_oauth: true` header is set in
|
| 105 |
+
`README.md` and may need the owner to approve the OAuth app once.
|
| 106 |
+
2. **ZeroGPU extension with a real account.** Sign in, open the **Coverage**
|
| 107 |
+
tab, pick an uncovered range, press *Estimate* then *Extend coverage*.
|
| 108 |
+
Confirm: progress stages appear, the commit lands in the dataset repo, and
|
| 109 |
+
the toast reads "Coverage extended by @you". This is the only path that
|
| 110 |
+
spends real GPU quota, so it is untested against real hardware.
|
| 111 |
+
3. **Add-model flow** with a real Hub model id, same reason.
|
| 112 |
+
4. **Torch version difference.** The Space runs torch **2.11.0** (ZeroGPU only
|
| 113 |
+
accepts 2.11/2.10/2.9.1/2.8); my adapter smoke test ran locally on 2.13.0.
|
| 114 |
+
The first real ZeroGPU extension doubles as the check that Chronos inference
|
| 115 |
+
behaves identically on 2.11.
|
| 116 |
+
5. **Visual pass on a real screen.** I verified the UI functionally and via
|
| 117 |
+
screenshots, but the in-app browser stalled before I could click through
|
| 118 |
+
every tab at full size. Worth a human eye on the Comparison and Robustness
|
| 119 |
+
tabs with several runs loaded.
|
| 120 |
+
|
| 121 |
+
## Open items from DECISIONS.md
|
| 122 |
+
|
| 123 |
+
- **D-001 β Space is under the personal namespace.** Gradio Spaces under an org
|
| 124 |
+
need a paid Team/Enterprise plan (402). You chose the personal namespace; the
|
| 125 |
+
dataset stays under the org. To move it later, subscribe and re-point
|
| 126 |
+
`BIT_SPACE_REPO`.
|
| 127 |
+
- **D-006 β Binance is geo-blocked (451)** from the seeding machine; crypto came
|
| 128 |
+
from Coinbase via the fallback chain. **Stooq now serves an HTML block page**,
|
| 129 |
+
so the equity chain effectively runs on yfinance alone. Tiingo is wired but
|
| 130 |
+
needs a `TIINGO_KEY` secret to join the chain.
|
| 131 |
+
- **Sentiment is a stub.** `Sentiment-Gated Momentum` runs on a labelled,
|
| 132 |
+
deterministic price-derived proxy behind a `SentimentSource` interface β not
|
| 133 |
+
news sentiment. Wiring a real feed is the obvious next task.
|
| 134 |
+
- **Pairs Trading** is in the design's preset list but not runnable in v1
|
| 135 |
+
(needs a second leg). It is shown disabled with that reason.
|
| 136 |
+
- **Equity intraday depth** is provider-capped (~730d hourly, ~60d 15m) and
|
| 137 |
+
recorded in the manifest as `provider_max_days`, not hidden.
|
| 138 |
+
|
| 139 |
+
## Things I'd flag as genuinely uncertain
|
| 140 |
+
|
| 141 |
+
- The **example strategy loses money on the locked holdout** (β5.2%, Sharpe
|
| 142 |
+
β0.38). That is the system working: the holdout is doing its job and the app
|
| 143 |
+
reports it rather than burying it. Don't read the headline +33.2% as a result.
|
| 144 |
+
- **Walk-forward on short ranges produces no OOS windows at all.** The app now
|
| 145 |
+
says so explicitly instead of printing `0.00`, but it is easy to configure a
|
| 146 |
+
run that looks complete and has no out-of-sample content. Watch for the red
|
| 147 |
+
note.
|
| 148 |
+
- **`vectorbt` is pinned to 1.0.0**, not the 1.1.0 I developed against, because
|
| 149 |
+
1.1.0 requires Python β₯3.11 and the ZeroGPU image is 3.10. The full suite
|
| 150 |
+
passes on both, but that is a real version skew between dev and prod.
|
| 151 |
+
|
| 152 |
+
## Repo layout
|
| 153 |
+
|
| 154 |
+
```
|
| 155 |
+
hugging-face/
|
| 156 |
+
key.txt local only, gitignored, never committed
|
| 157 |
+
DECISIONS.md 15 logged decisions across all phases
|
| 158 |
+
HANDOFF.md this file
|
| 159 |
+
ops/ provisioning scripts (local only, not deployed)
|
| 160 |
+
store_stage/ dataset card staging
|
| 161 |
+
space/ β pushed to the Space
|
| 162 |
+
app.py src/{store,data,engine,strategies,adapters,charts,metrics,
|
| 163 |
+
comparisons,runtime,extension,config,ui/theme}.py
|
| 164 |
+
scripts/seed_store.py tests/ (191) assets/ (vendored design system)
|
| 165 |
+
DESIGN_NOTES.md the extracted design contract Phase 3 was built against
|
| 166 |
+
```
|