bit-backtest-lab / DECISIONS.md
Bit-Trading-Company's picture
Add DECISIONS.md
39c225c verified
|
Raw
History Blame Contribute Delete
8.86 kB
# DECISIONS
Running log of choices the build spec left open, and of anything that failed.
Newest phase last.
---
## Phase βˆ’1 β€” Provisioning
### D-001 β€” The Space lives under the personal namespace, not the org β€” **decided by the human**
`create_repo(..., repo_type="space", space_sdk="gradio")` under
`The-Bit-Trading-Company` returns **402 Payment Required**:
> Static Spaces are free for everyone, but hosting Gradio and Docker Spaces on
> free cpu-basic requires a Team or Enterprise plan for organization
> The-Bit-Trading-Company.
Verified this is a plan limit, not a token-scope problem: the token has
`repo.write` on the org, the dataset repo created fine under the org, and a
*static* Space probe under the org succeeded. The org has no paid plan; the
owner's personal account is PRO.
This hit the "anything requiring payment" flag, so it went to the human, who
chose the personal namespace.
| Repo | Location |
|---|---|
| Signal store (dataset) | `The-Bit-Trading-Company/bit-signal-store` |
| Backtest Lab (Space) | `Bit-Trading-Company/bit-backtest-lab` |
The shared data asset keeps the company namespace; only the app URL changes.
The personal account being PRO means ZeroGPU is still available for Phase 4, so
no capability is lost. Moving the Space to the org later is a rename, and
`src/config.py` reads both ids from env (`BIT_SPACE_REPO`, `BIT_STORE_REPO`).
### D-002 β€” Python 3.11 toolchain
The machine had only Python 3.14, which numba (and therefore vectorbt) does not
support. Used `uv` to install 3.11.15 into a local `.venv`. Final resolved
stack: vectorbt 1.1.0, numpy 2.4.6, pandas 2.3.3, numba 0.67.0, pyarrow 25.0.1,
plotly 6.9.0, gradio 5.49.1. `requirements.txt` pins these for the Space, and
the Space is pinned to `python_version: "3.11"` to match.
### D-003 β€” Writes are one atomic commit, not "manifest last"
The spec asks for `CommitScheduler` with "atomic manifest update last". A single
`create_commit` carrying the parquet slices *and* the manifest is strictly
stronger: readers can never observe a manifest referencing a slice that has not
landed yet. `SignalStore.flush()` does that, ordering the manifest as the final
operation within the commit. `SignalStore.attach_scheduler()` still provides
`CommitScheduler`-based background batching for the running Space, pointed at
the same local mirror, so the two paths cannot disagree about what is on disk.
### D-004 β€” `Custom (code)` preset is present but inert
The design's preset list ends with `Custom (code)`, backed by a code editor. The
build spec says "No arbitrary code execution from user input anywhere", which is
the stronger constraint. The control is rendered, disabled, with an explanation,
rather than silently dropped from the design β€” see `DESIGN_NOTES.md`.
### D-005 β€” Design source recovered from local files
`claude.ai/design/p/00cbd10f…` returned 403 to every unauthenticated fetch and
no Chrome instance was connected, so the design was unreachable at first. The
human supplied the export locally. The Bit design system tokens and fonts are
vendored verbatim into `space/assets/`; `space/DESIGN_NOTES.md` records the
extracted layout, copy and component inventory that Phase 3 is built against.
### Provisioning results
| Action | Result |
|---|---|
| `whoami` with token | OK β€” user `Bit-Trading-Company`, PRO, admin of the org |
| Token scope | fine-grained; `repo.write` on org + user |
| Create dataset `The-Bit-Trading-Company/bit-signal-store` | **created**, public |
| Create Space under org | **failed, 402** β€” see D-001 |
| Create Space `Bit-Trading-Company/bit-backtest-lab` | **created**, gradio, cpu-basic, public |
| Space secret `HF_WRITE_TOKEN` | **set** via API; value never logged or committed |
| Signal store init | **committed** β€” `manifest.json` (schema_version 1), `signals/ prices/ comparisons/ runs/`, dataset card |
| Hardware `zero-a10g` | not yet requested β€” deferred to Phase 4 per spec |
Token hygiene: the value lives only in the local `key.txt` (gitignored at the
repo root) and in the Space secret. It appears in no source file, no committed
config, and no log line. `.env.example` documents variable names only.
---
## Phase 0 β€” Data layer
### D-006 β€” Providers are config; both primaries failed and the chain absorbed it
Two provider-level blocks appeared during seeding, and both were handled by the
fallback chain rather than by editing code:
- **Binance returns HTTP 451** ("restricted location") from the seeding machine.
Crypto OHLCV therefore came from **Coinbase**, the configured fallback.
- **Stooq now serves an HTML block page** instead of CSV. `data.py` detects the
missing CSV header and advances the chain, so equities ran on **yfinance**.
Neither was silently swallowed: every fallback is recorded in the fetch result's
notes, and `source` is stored per price row. Recorded here rather than escalated
because the spec's flag-to-human bar is provider blocking that persists *after
backoff and fallback exhaustion* β€” the fallbacks succeeded.
### D-007 β€” Equity gap counts are calendar gaps, not data faults
Daily equity series report ~171 "gaps" over three years. Those are weekends and
holidays. The gap report measures distance from a continuous bar grid and is
surfaced as information; it never fails validation.
## Phase 1 β€” Engine
### D-008 β€” Causality is checked by perturbation, in both directions
The spec asks for a structural lookahead assertion. `assert_causal` scales the
tail of the price frame and requires every earlier output to be bit-identical.
Scaling **up only** proved insufficient: a boolean comparison that is already
`True` can survive an upward bump unchanged, and a peeking strategy slipped
through on one seed. The check now perturbs up *and* down at four probe points,
which catches obvious lookahead, centred rolling windows, and full-sample
normalisation.
### D-009 β€” R-multiple when no stop is configured
R is P&L over initial risk, and initial risk is the stop distance. With no stop
there is no such distance, so the engine falls back to
`risk_per_trade_pct` (default 2% of entry notional). It is a documented config
field rather than a hidden constant.
### D-010 β€” Costs are one atomic identity on every trade row
Slippage is embedded in the fill price rather than booked as a fee, so it is
reconstructed from the *unslipped* reference price (direction-aware: a buy fills
high, a sell fills low). This makes `gross βˆ’ costs = net` exact to 1e-9 on every
row, which is what known-answer test 3 asserts.
### D-011 β€” The holdout applies to every validation mode
Originally only `mode="holdout"` reserved one. Walk-forward could then roll its
windows across the whole period. Since the design shows rolling windows and an
"OOS holdout LAST 6MO" side by side, `holdout_months > 0` now reserves a holdout
in every mode except `none`.
### D-012 β€” An empty segment renders as "β€”", never 0.00
A short range under walk-forward can produce no out-of-sample window at all. The
stat band was reporting `OOS Sharpe 0.00` for that case, which asserts something
false. Segments with zero bars now render an em dash, the plan carries an
explicit note explaining why no OOS exists, and the overfit verdict counts it as
a failed check.
## Phase 2 β€” Adapters & seed
### D-013 β€” The v1 seed contains no placeholder slices
The spec allows clearly-labelled synthetic placeholders where GPU capacity is
short. Batched Chronos-Bolt inference on local MPS ran at roughly **1 ms/step**,
so the entire v1 seed was generated for real: **18 slices, 60,375 rows**, across
`chronos-bolt-small` and `chronos-bolt-base`, 6 assets, 3 timeframes.
`PlaceholderAdapter` and its `PLACEHOLDER` labelling remain in the codebase and
are still tested, because contributors on weaker hardware may need them.
### D-014 β€” Dedup compares the *producible* range, not the requested one
A forecast needs a full trailing context window, so the first producible
timestamp sits `context_len` bars after the start of a price slice. Comparing
the user's typed range against manifest coverage therefore reported
already-covered slices as uncovered and would have paid for the same inference
twice. Both `scripts/seed_store.py` and `extension.estimate()` now compare the
range the request would actually produce. This was caught by the Phase 4 dedup
test, not by inspection.
## Phase 5 β€” Ship
### D-015 β€” torch pinned to 2.11.0 for ZeroGPU
The first deploy landed in `CONFIG_ERROR`: ZeroGPU accepts only torch 2.11.0,
2.10.0, 2.9.1 or 2.8.0, and the build was pinned to the locally-tested 2.13.0.
Pinned to **2.11.0**. The local test environment still runs 2.13.0, so the
adapter code path is exercised on a newer torch than the Space uses β€” noted in
HANDOFF.md as a residual difference worth a post-deploy check.