bit-backtest-lab / DECISIONS.md
Bit-Trading-Company's picture
Add DECISIONS.md
39c225c verified
|
Raw
History Blame Contribute Delete
8.86 kB

A newer version of the Gradio SDK is available: 6.25.0

Upgrade

DECISIONS

Running log of choices the build spec left open, and of anything that failed. Newest phase last.


Phase βˆ’1 β€” Provisioning

D-001 β€” The Space lives under the personal namespace, not the org β€” decided by the human

create_repo(..., repo_type="space", space_sdk="gradio") under The-Bit-Trading-Company returns 402 Payment Required:

Static Spaces are free for everyone, but hosting Gradio and Docker Spaces on free cpu-basic requires a Team or Enterprise plan for organization The-Bit-Trading-Company.

Verified this is a plan limit, not a token-scope problem: the token has repo.write on the org, the dataset repo created fine under the org, and a static Space probe under the org succeeded. The org has no paid plan; the owner's personal account is PRO.

This hit the "anything requiring payment" flag, so it went to the human, who chose the personal namespace.

Repo Location
Signal store (dataset) The-Bit-Trading-Company/bit-signal-store
Backtest Lab (Space) Bit-Trading-Company/bit-backtest-lab

The shared data asset keeps the company namespace; only the app URL changes. The personal account being PRO means ZeroGPU is still available for Phase 4, so no capability is lost. Moving the Space to the org later is a rename, and src/config.py reads both ids from env (BIT_SPACE_REPO, BIT_STORE_REPO).

D-002 β€” Python 3.11 toolchain

The machine had only Python 3.14, which numba (and therefore vectorbt) does not support. Used uv to install 3.11.15 into a local .venv. Final resolved stack: vectorbt 1.1.0, numpy 2.4.6, pandas 2.3.3, numba 0.67.0, pyarrow 25.0.1, plotly 6.9.0, gradio 5.49.1. requirements.txt pins these for the Space, and the Space is pinned to python_version: "3.11" to match.

D-003 β€” Writes are one atomic commit, not "manifest last"

The spec asks for CommitScheduler with "atomic manifest update last". A single create_commit carrying the parquet slices and the manifest is strictly stronger: readers can never observe a manifest referencing a slice that has not landed yet. SignalStore.flush() does that, ordering the manifest as the final operation within the commit. SignalStore.attach_scheduler() still provides CommitScheduler-based background batching for the running Space, pointed at the same local mirror, so the two paths cannot disagree about what is on disk.

D-004 β€” Custom (code) preset is present but inert

The design's preset list ends with Custom (code), backed by a code editor. The build spec says "No arbitrary code execution from user input anywhere", which is the stronger constraint. The control is rendered, disabled, with an explanation, rather than silently dropped from the design β€” see DESIGN_NOTES.md.

D-005 β€” Design source recovered from local files

claude.ai/design/p/00cbd10f… returned 403 to every unauthenticated fetch and no Chrome instance was connected, so the design was unreachable at first. The human supplied the export locally. The Bit design system tokens and fonts are vendored verbatim into space/assets/; space/DESIGN_NOTES.md records the extracted layout, copy and component inventory that Phase 3 is built against.

Provisioning results

Action Result
whoami with token OK β€” user Bit-Trading-Company, PRO, admin of the org
Token scope fine-grained; repo.write on org + user
Create dataset The-Bit-Trading-Company/bit-signal-store created, public
Create Space under org failed, 402 β€” see D-001
Create Space Bit-Trading-Company/bit-backtest-lab created, gradio, cpu-basic, public
Space secret HF_WRITE_TOKEN set via API; value never logged or committed
Signal store init committed β€” manifest.json (schema_version 1), signals/ prices/ comparisons/ runs/, dataset card
Hardware zero-a10g not yet requested β€” deferred to Phase 4 per spec

Token hygiene: the value lives only in the local key.txt (gitignored at the repo root) and in the Space secret. It appears in no source file, no committed config, and no log line. .env.example documents variable names only.


Phase 0 β€” Data layer

D-006 β€” Providers are config; both primaries failed and the chain absorbed it

Two provider-level blocks appeared during seeding, and both were handled by the fallback chain rather than by editing code:

  • Binance returns HTTP 451 ("restricted location") from the seeding machine. Crypto OHLCV therefore came from Coinbase, the configured fallback.
  • Stooq now serves an HTML block page instead of CSV. data.py detects the missing CSV header and advances the chain, so equities ran on yfinance.

Neither was silently swallowed: every fallback is recorded in the fetch result's notes, and source is stored per price row. Recorded here rather than escalated because the spec's flag-to-human bar is provider blocking that persists after backoff and fallback exhaustion β€” the fallbacks succeeded.

D-007 β€” Equity gap counts are calendar gaps, not data faults

Daily equity series report ~171 "gaps" over three years. Those are weekends and holidays. The gap report measures distance from a continuous bar grid and is surfaced as information; it never fails validation.

Phase 1 β€” Engine

D-008 β€” Causality is checked by perturbation, in both directions

The spec asks for a structural lookahead assertion. assert_causal scales the tail of the price frame and requires every earlier output to be bit-identical. Scaling up only proved insufficient: a boolean comparison that is already True can survive an upward bump unchanged, and a peeking strategy slipped through on one seed. The check now perturbs up and down at four probe points, which catches obvious lookahead, centred rolling windows, and full-sample normalisation.

D-009 β€” R-multiple when no stop is configured

R is P&L over initial risk, and initial risk is the stop distance. With no stop there is no such distance, so the engine falls back to risk_per_trade_pct (default 2% of entry notional). It is a documented config field rather than a hidden constant.

D-010 β€” Costs are one atomic identity on every trade row

Slippage is embedded in the fill price rather than booked as a fee, so it is reconstructed from the unslipped reference price (direction-aware: a buy fills high, a sell fills low). This makes gross βˆ’ costs = net exact to 1e-9 on every row, which is what known-answer test 3 asserts.

D-011 β€” The holdout applies to every validation mode

Originally only mode="holdout" reserved one. Walk-forward could then roll its windows across the whole period. Since the design shows rolling windows and an "OOS holdout LAST 6MO" side by side, holdout_months > 0 now reserves a holdout in every mode except none.

D-012 β€” An empty segment renders as "β€”", never 0.00

A short range under walk-forward can produce no out-of-sample window at all. The stat band was reporting OOS Sharpe 0.00 for that case, which asserts something false. Segments with zero bars now render an em dash, the plan carries an explicit note explaining why no OOS exists, and the overfit verdict counts it as a failed check.

Phase 2 β€” Adapters & seed

D-013 β€” The v1 seed contains no placeholder slices

The spec allows clearly-labelled synthetic placeholders where GPU capacity is short. Batched Chronos-Bolt inference on local MPS ran at roughly 1 ms/step, so the entire v1 seed was generated for real: 18 slices, 60,375 rows, across chronos-bolt-small and chronos-bolt-base, 6 assets, 3 timeframes. PlaceholderAdapter and its PLACEHOLDER labelling remain in the codebase and are still tested, because contributors on weaker hardware may need them.

D-014 β€” Dedup compares the producible range, not the requested one

A forecast needs a full trailing context window, so the first producible timestamp sits context_len bars after the start of a price slice. Comparing the user's typed range against manifest coverage therefore reported already-covered slices as uncovered and would have paid for the same inference twice. Both scripts/seed_store.py and extension.estimate() now compare the range the request would actually produce. This was caught by the Phase 4 dedup test, not by inspection.

Phase 5 β€” Ship

D-015 β€” torch pinned to 2.11.0 for ZeroGPU

The first deploy landed in CONFIG_ERROR: ZeroGPU accepts only torch 2.11.0, 2.10.0, 2.9.1 or 2.8.0, and the build was pinned to the locally-tested 2.13.0. Pinned to 2.11.0. The local test environment still runs 2.13.0, so the adapter code path is exercised on a newer torch than the Space uses β€” noted in HANDOFF.md as a residual difference worth a post-deploy check.