SPEC.md

# hack-x — Environment & Training Spec

A self-improving crypto-trading **coding agent** trained with RL on Prime Lab.
tradewatch's soft reflection loop (events → MEMORY.md prompt rules) becomes a real
gradient loop: rollout → backtest verifier → GRPO → **LoRA adapter** → deploy → live demo.

---

## 1. The one-line thesis
> Today tradewatch's trading discipline lives as *prompt text* (MEMORY.md). We make it
> live as *adapter weights* by rewarding disciplined, profitable behavior on
> out-of-sample replayed market history. The proof: the trained adapter stays
> disciplined with MEMORY.md **removed from the prompt**.

## 1b. tradewatch as seed (pull all of it — but know its limits)
Mined both DB (4,875 events) and journal (708 rows). What's there:
- **618 `analysis_decision` rows** with 18 market features each (price_usd, liquidity_usd,
  volume_24h/1h, buys/sells 1h+24h, buy_sell_24h, vol/liq, price_change 5m→24h, age_hours,
  fdv, market_cap) — this IS the agent's feature schema + seed/format examples.
- Real verdicts + entry/target/stop + reasoning over 9 tokens (action-surface-B examples).
- **9 confirmed pool addresses** → guaranteed-good fetch universe seed.
- Trade outcomes (exits/PnL/skips) → rubric calibration.
- **NOT present:** any stored OHLCV time series. `ohlcv_1h` was fetched live, never persisted.
  Every market row is a point-in-time snapshot, not a replayable tape.
- **Therefore:** tradewatch = seed of *decisions/features/pools* (the brains + universe);
  GeckoTerminal fetch = the *replay price tape*. Both required; neither substitutes.

## 1c. The three loops are independent (demo ≠ batch_size)
- **Training loop:** batch_size=128 parallel replay-rollouts/step — a GRPO requirement only.
- **Demo loop:** 1 (or few) LIVE paper session in tradewatch, trained adapter via
  base:adapter_id. As small as we want. 128 was never the demo.
- **Recursive loop (stretch):** reflect → checkpoint_id warm-start → retrain.

## 2. Data (replay substrate) — crypto
- **Source:** GeckoTerminal free OHLCV — `/networks/base/pools/{pool}/ohlcv/{tf}?aggregate=N&limit=L`
  (already used in `scanner/base_scanner.py:843`). No key.
- **Universe:** start from the 9 journal tokens (NOCK, VIRTUAL, DEUS, CTR, Surplus,
  GITLAWB, VVV, DEGEN, PITCH) + expand via trending-pool discovery to ~30–50 Base pools.
- **Bars & DEPTH CEILING (probed live 2026-05-30):** free OHLCV caps at **~1000 bars/pool,
  `before_timestamp` pagination does NOT walk back further.** So:
  - hourly (`tf=hour`): ~1000 bars ≈ **41 days**
  - daily (`tf=day`): ~181 bars ≈ **6 months**
  - **Implication:** shallow history → depth can't supply many decorrelated time windows.
    **Breadth carries GRPO variance: tasks = symbol × window, universe = 30–50 pools.**
- **Storage:** `data/ohlcv/<symbol>.parquet` (ts, o, h, l, c, v). Reproducible, no rate limits at train time.
- **Splits (anti-leakage, structural):**
  - per-symbol time split: train window | OOS window (chronological, no overlap)
  - held-out OOS **symbols** (never seen in training) for generalization check
  - with shallow per-symbol history, the symbol-holdout split does the heavy lifting
- **Crypto-hardening (because reward is gameable here):**
  - slippage + fee model on every fill (fixed bps + size-vs-liquidity impact)
  - min-liquidity / min-volume gate per bar (illiquid bars can't be traded)
  - survivorship caveat noted; mitigate by including tokens that later died

## 3. Action surface — support BOTH (decide after baseline rollouts)
The backtester accepts a uniform `target_positions` dict; two agent framings feed it:
- **A. Strategy-code:** agent writes `strategy(features_window) -> target_positions`,
  executed causally over the OOS window. Strongest "coding agent" framing.
- **B. Structured decisions:** agent emits per-bar JSON (verdict/entry/target/stop),
  reusing tradewatch's schema. Closer to existing code.
Both reduce to the same backtest call → same rubric. We prototype both, pick via eval.

## 4. Backtester (port `agents/paper_ledger.py`)
- `ReplayFeed`: serves bars **only up to t** — lookahead impossible by construction
  (the agent/strategy literally never receives `bars[t+1:]`).
- Engine: step bar-by-bar, apply target_positions via slippage/fee model, mark-to-market,
  track equity, cash, exposure, per-trade R:R, drawdown.
- Metrics: Sharpe (primary), CAGR/return, max drawdown, equity/cash ratio, turnover, win rate.
- Benchmarks computed on same window: **buy-and-hold**, MA-crossover, z-score mean-reversion.
- Invariants (unit-tested): causality (no future leak), determinism (seed → identical run).

## 5. verifiers environment (v0 stable API)
Class: **`StatefulToolEnv`** (per-rollout state + stateful tools).

```python
def load_environment(
    symbols: list[str] | str = "train",   # universe or split name
    split: str = "train",                  # train | oos | oos_symbols
    objective: str = "sharpe",             # sharpe | return | min_drawdown
    max_turns: int = 8,
    n_windows: int = 4,                     # OOS windows averaged per reward
    seed: int = 0,
) -> vf.Environment: ...
```

- `setup_state(state)`: pick task (symbol(s) × window × objective), build fresh ReplayFeed + ledger.
- **Tools** (stateful, NOT per-rollout sandbox — global/in-proc exec to save credits):
  - `get_features(lookback)` → indicators up to current bar (RSI, MACD, MAs, z-score,
    BB, vol, buy/sell ratio, liquidity) — reuses scanner feature logic
  - `run_backtest(strategy_or_decisions)` → in-sample metrics (the agent's feedback loop)
  - `read_metrics()` → current equity/DD/Sharpe
- `@vf.stop`: agent submits final strategy, or max_turns.
- **Reward (Rubric, weighted) — computed on OOS, aggregated over n_windows × basket:**

  | reward fn | weight | source |
  |---|---|---|
  | `r_sharpe` (normalized OOS Sharpe) | 0.40 | spine |
  | `r_beats_benchmark` (vs buy-and-hold) | 0.20 | anti-overfit |
  | `r_drawdown` (penalty for deep DD) | 0.15 | MEMORY: protect capital |
  | `r_rr_discipline` (R:R≥2 compliance) | 0.10 | trading_agent._validate |
  | `r_exposure` (sane equity/cash, no all-in) | 0.10 | MEMORY: sizing |
  | `r_cost` (turnover/fee penalty) | 0.05 | realism |
  | **HARD GATES → reward 0** | — | invalid code, lookahead, NaN equity, illiquid trades |

  Outcome terms (sharpe + beats_benchmark) must dominate so the model can't fake
  discipline while losing money.

## 6. Training (Prime Hosted, FREE for Laguna)
```toml
model = "poolside/Laguna-XS.2"   # validate first on Qwen/Qwen3-4B-Instruct-2507
max_steps = 50                    # small validation; scale after curve moves
batch_size = 128                  # = parallel rollouts (the "128 sessions")
rollouts_per_example = 8          # GRPO group — needs decorrelated windows
learning_rate = 1e-4
lora_alpha = 16
[sampling]
max_tokens = 512
enable_thinking = false           # docs: start non-reasoning for agentic tasks
[[env]]
id = "<user>/stock-strategy-env"
[eval]
eval_base_model = true            # base-vs-trained comparison built in
```
Pipeline: `prime eval run` (sanity, baseline 10–80%) → small Qwen RL → Laguna RL →
download LoRA → `prime deployments create` → `base:adapter_id`.

## 7. Closing the loop
- Deploy adapter → point tradewatch `HybrieClient` at `api.pinference.ai` → **live paper demo**.
- **Ablation (money shot):** trained adapter, MEMORY.md stripped from prompt → discipline holds.
- **Recursive (stretch):** `checkpoint_id` warm-start → reflect on failures →
  adjust rubric weights / add tasks / `[buffer]` difficulty filtering → retrain.

## 8. Non-negotiable risks
1. Leakage → structural causal feed (not a detector).
2. Reward variance → aggregate over basket × windows (or GRPO has no signal).
3. Reward hacking → OOS + must-beat-benchmark + hard lookahead gate.
4. Crypto noise → slippage/fee/liquidity model on every fill.