CLAUDE.md

# CLAUDE.md

Guidance for Claude Code (claude.ai/code) when working in this repository.

## What this is

ControlAI is an offline control-systems engineering assistant. A locally held model
(`mlx-community/Qwen3-14B-4bit` by default, via MLX) answers control questions, and every number it
states comes from a deterministic solver — SciPy/LAPACK/CVXPY behind a validated tool registry —
never from the model's own arithmetic. A BM25 + dense hybrid retriever over a local control-theory
corpus grounds conceptual answers. A FastAPI + vanilla-JS console and a terminal CLI are the two
front ends. Nothing leaves the machine.

Apple Silicon only for local use. There is no GGUF or Ollama path — an earlier version carried
both plus CUDA and a Spaces deployment, and maintaining four backends for one machine was most of
the complexity in the codebase.

One CUDA path came back, narrowly, for the public demo Space: see **Deployment** below. It is a
second `Engine` implementation behind the same contract, selected by one env var. It does not touch
the agent loop, the registry, or any tool.

## Commands

```bash
./run.sh                      # web console at http://127.0.0.1:8000
./run.sh --cli                # interactive terminal chat
./run.sh --cli "design an LQR for A=[[0,1],[-2,-3]], B=[[0],[1]], Q=eye(2), R=1"
./run.sh --fetch-index        # download the prebuilt index from the private Hub dataset repo
./run.sh --build-index        # rebuild the dense retrieval index (~45 min, checkpointed/resumable)
./run.sh --ingest-corpus      # merge data/processed/ chunks into the index
./run.sh --calibrate          # re-measure MIN_COSINE against the current corpus

python -m unittest discover tests -v          # unittest, not pytest -- pytest is not installed
python -m unittest tests.test_agent_tools     # deterministic numerics
python -m unittest tests.test_agent_core      # parsing, stream gating, truncation

pip install -r requirements.txt               # runtime
pip install -r requirements-training.txt      # dataset generation / ground truth
pip install -r requirements-corpus.txt        # corpus crawling and extraction
```

Environment: `CONTROLAI_MODEL`, `CONTROLAI_ADAPTER`, `CONTROLAI_THINKING` (`off|auto|on`),
`CONTROLAI_THINK_BUDGET`, `CONTROLAI_EMBED_MODEL`.

There is no lint or format command configured — don't invent one.

## Architecture

### Request flow
`web/` (SSE console) or `cli.py` → `app.py` (`/api/chat`, `/api/chat/stream`) →
`ControlAgent` (`controlai_agent/agent.py`) → `LocalEngine` (`controlai_agent/engine.py`, MLX) and
`registry.execute` (`controlai_agent/registry.py`) → tools in `controlai_agent/tools/*.py`.

### `engine.py` — inference
One persistent KV cache lives for the process. `_align_cache` finds the longest prefix of the
incoming prompt that the cache already holds, trims to that point, and feeds only the remainder, so
the fixed system-prompt-plus-tool-schema prefix (~8k tokens) is prefilled once at startup by
`prewarm()` rather than once per tool step. Generated tokens are tracked in the cache too, which is
what makes continuing a tool-calling turn nearly free. Measured effect: first-token latency fell
from ~6.4 s per step to ~0.7 s.

`stream()` yields tokens as they are produced. `think_budget` caps tokens spent inside a `<think>`
block and closes it by hand on overrun, bounding worst-case latency on a reasoning model.

Sampling uses `presence_penalty`, not a flat `repetition_penalty`. A blanket repetition penalty
punishes the repeated structural tokens that matrices and JSON are made of (`[`, `0`, `,`) exactly
when the model is emitting a tool call.

### `agent.py` — the loop
A short tool-calling loop (`MAX_TOOL_STEPS = 2`, `MAX_CALLS_PER_TOOL = 2`) that streams throughout.
`_StreamGate` releases text as soon as it cannot be the start of `<tool_call>`, so prose appears
immediately while a tool call never leaks into the chat. Retrieved passages are attached to the
*user* turn, never spliced into the system prompt, because that keeps the cached prefix
byte-identical across questions.

**There is deliberately no parameter-provenance check.** An earlier version refused any matrix it
could not trace back to the user's message. It blocked the single most useful thing the assistant
does — working an example that the user asked for — so it was removed. Schema validation and
sandboxed execution in `registry.execute` are the real guarantees; `toolcall.degenerate_reason`
catches genuine decoding loops with thresholds set well above any hand-written matrix.

### `prompts.py`
506 tokens, down from 2,587. States what to do rather than what not to do. The previous prompt's
"never invent a parameter" clause taught the model to refuse worked examples; the current one
explicitly authorises choosing an illustrative system when the user asks for a demonstration.

### Tool registry (`registry.py` + `controlai_agent/tools/`)
Tools are plain functions decorated with `@registry.register(name, description, parameters_schema)`
across `linear.py`, `frequency.py`, `synthesis.py`, `estimation.py`, `nonlinear.py`, `robust.py`,
`allocation.py`, `simulation.py`, `matrix_ops.py`, `plotting.py`, `python_executor.py`, `rag.py`.
`registry.execute()` coerces stringified arrays, validates against JSON Schema, runs the function,
and rounds every float to 6 significant figures. `controlai_agent/verifier.py` cross-checks results
with independent invariants — Riccati residual, pole-placement error, CBF forward invariance.

**Prefer putting a fact in a tool result over instructing the model to derive it.** Two observed
failures were fixed that way rather than by prompt wording, and it is the pattern to reach for
first:
- `continuous_lqr`, `discrete_lqr` and `place_state_feedback` return `closed_loop_A`. Given a
  correct gain, the model wrote $A-BK$ for a double integrator as `[[-6,1],[-5,-6]]` instead of
  `[[0,1],[-6,-5]]` — right gain, wrong write-up.
- `bode_analysis` returns the gain/phase margins as well. Asked for a phase margin the model
  reached for it instead of `stability_margins`, got back sampled curves, and correctly but
  uselessly concluded the margin "cannot be determined".

### Retrieval (`controlai_rag/`)

Three data bugs were found here and are worth knowing about, because each one silently degraded
retrieval rather than failing:
- **Chunk ids were not unique.** `chunk_document` is called once per page and restarted its counter
  each time, so every page's first chunk was `<file>_c0000` — 154 distinct ids across 9,976 chunks.
  Anything keyed on chunk_id resolved to the wrong row. Fixed in `chunker.py` (ids are now
  page-qualified); `scripts/repair_chunk_ids.py` migrated the existing index in place.
- **Embeddings were pooled without an EOS token.** Qwen3-Embedding pools the last position and was
  trained with `<|endoftext|>` there. Omitting it dropped the margin between relevant and irrelevant
  passages from +0.32 to +0.15. Note the checkpoint's own `eos_token_id` is `<|im_end|>`, which is
  the wrong token here — `controlai_rag/embeddings.py` pins the right one.
- **71.5% of the Nise textbook was mojibake.** Its PDF has a broken symbol-font ToUnicode map, so
  every extractor returns `L½ f ðtÞ/C138 ¼FðsÞ` for `L[f(t)] = F(s)`. `controlai_rag/textfix.py`
  reverses the substitution (it is deterministic); `scripts/repair_corpus_text.py` migrated the
  index. `document_loader` now applies it on ingest.

`document_loader.py` → `chunker.py` → `index.py` (BM25, persisted to `data/rag_index/`) and
`embeddings.py` (Qwen3-Embedding-0.6B under MLX, last-token pooling, instruction-prefixed queries).

**The index holds 80,370 chunks.** 9,976 come from `data/user_docs/` (course notes, Nise, Ogata);
the other 70,394 were bridged in from `data/processed/*_chunks/` by
`scripts/ingest_processed_corpus.py`. That bridge did not exist before: the `scripts/` pipeline
(`raw → extracted → processed`) fed *training dataset generation* only, while `ControlRAGIndex` read
`user_docs` alone. Retrieval was therefore seeing 12% of the corpus, and none of the canonical
texts — Doyle/Francis/Tannenbaum, Åström & Murray, Rawlings/Mayne/Diehl, Sontag, Liberzon, Boyd,
Söderström & Stoica — which were downloaded, extracted and chunked on disk, unread. Re-run the
bridge after adding anything to `data/processed/`.

**The index is not in git.** `chunks.json` (187MB), `embeddings.npz` (144MB) and `bm25.pkl`
(124MB) are each past GitHub's 100MB per-file limit, and `chunks.json` holds the full extracted text
of 666 documents including commercial textbooks (Nise, Ogata) — fine to keep locally, wrong to
redistribute. They live in the **private** Hub dataset repo `atakankahya/controlai-rag-index`;
`controlai_rag/fetch_index.py` (`./run.sh --fetch-index`) pulls them with your HF token. It copies
out of the Hub cache rather than symlinking, because the index is mutated in place by uploads and by
`scripts/repair_*.py`. A public clone has no index and builds its own from `data/user_docs/`.

`build()` is checkpointed every 4,000 chunks to `embeddings.partial.npz` and resumes from it. An
80k-chunk build takes ~45 minutes, and writing only at the end meant one interruption discarded all
of it (observed at 72,008 of 80,370).
`retriever.py` fuses the two rankings with reciprocal rank fusion and gates on cosine similarity
(`MIN_COSINE = 0.62`). That number is a property of the model *and* the corpus, so re-measure it
with `./run.sh --calibrate` whenever the corpus changes size materially — it moved when the index
grew 8x. Current margins: in-domain worst best-match 0.678, off-domain best 0.571. Lexical candidates are scored densely too — without that, a chunk BM25 ranked first but
that fell outside the dense top-K was dropped for having no score rather than for being irrelevant.
`_is_low_value` discards back-of-book index pages, which match almost any control query because they
contain every term in the field, while explaining nothing.

The gate matters. BM25 scores are unbounded and corpus-relative, so the old threshold of 2.5 passed
essentially everything: a question about the Bode sensitivity integral retrieved Routh-Hurwitz
tables at score 19.7 and injected them as authoritative context. Returning nothing is a valid and
frequent outcome — the model then answers from its own knowledge.

`display_source_name()` maps raw indexed filenames (which carry owner initials, course codes, scan
artifacts) to clean citable labels. Never let a raw filename reach an answer.

Uploads through `/api/upload` are embedded immediately via `HybridRetriever.add_chunks`, or they
would have no vector and the cosine gate would make them permanently unreachable.

### `app.py` — serving
Local-only FastAPI. MLX keeps its compute stream in thread-local state, so every inference call
goes through one dedicated worker thread for the process lifetime; a lock serialises turns because
they all mutate the same KV cache. `_to_wire_events` translates the agent's event vocabulary
(`text`/`thinking`/…) into what `web/app.js` consumes (`token`/`thought`/…).

### Model artifacts
`adapters/` and `models/` hold earlier fine-tuning output. **They are gitignored — not backed up by
git.** The LoRA adapters are not loaded by default: `behavior_v1` emits a spurious empty
`<tool_call></tool_call>` as its first output on essentially every prompt, so no tool ever runs, and
it refuses fully-specified problems; `sft_v2` produces empty output when no tools are exposed and
string-typed numbers when they are. Both were measured against the base model, which routes and
formats correctly. `CONTROLAI_ADAPTER=<path>` loads one anyway for A/B work.

`configs/*.yaml` are MLX-LoRA training configs and `scripts/` holds the offline pipeline (corpus
discovery → extraction → dataset generation → training → evaluation). That pipeline is independent
of the serving path and uses `requirements-training.txt`/`requirements-corpus.txt`.

### Deployment (`app_space.py`, `engine_torch.py`, `requirements-space.txt`)
The demo Space is a **Gradio app on ZeroGPU**, not the FastAPI console. That is not a preference:
**ZeroGPU assumes the Gradio app is the Space.** It schedules GPU workers by forking the server
process, and its startup validation looks for a `@spaces.GPU` function wired to a Gradio event
handler. Keeping FastAPI on the public port with Gradio as a hidden side-car was tried at length and
does not work — see the list below. `app_space.py` therefore gives Gradio the port and drives
`ControlAgent` from inside `@spaces.GPU`. The `web/` console is lost on the Space only; the agent,
the 29 solvers, the verifier and the 80,370-chunk retriever are all the same.

Seven ZeroGPU constraints, each found the hard way. Read these before changing anything here:
1. **No upper bounds in `requirements-space.txt`.** The platform appends its own
   `gradio[oauth,mcp]`, `spaces`, `uvicorn` and a `torch` ceiling; one extra constraint can make the
   resolve impossible. `transformers<4.56` did, because gradio 6 needs `huggingface-hub>=1.16` and
   every `transformers<4.56` needs `<1.0`. **And do not pin torch down to CUDA 12** — ZeroGPU is
   backed by Blackwell (sm_120), which needs CUDA 12.8+; an older wheel has no kernels for it.
2. **Build the model at import scope.** ZeroGPU patches torch during the entry module's import and
   only intercepts CUDA inside that window. Building it in a lifespan or a request reaches real CUDA
   init and raises `Low-level CUDA init (torch._C._cuda_init) reached`.
3. **No `device_map`, no bitsandbytes.** `device_map` routes transformers through
   `caching_allocator_warmup`'s direct `torch.empty(..., device="cuda")`, which trips the same
   guard; bitsandbytes requires `device_map`, so 4-bit is unavailable and the model must fit in bf16.
4. **Inference must be inside `@spaces.GPU`.** Otherwise the packed tensors are never materialised
   and forward passes return fluent multilingual noise at ~15 min/turn — up, responsive, and
   confidently wrong. Wrap the whole *turn*: a turn is several generations sharing one KV cache.
5. **Never pass the agent as an argument** to a `@spaces.GPU` function; reach it through a module
   global. ZeroGPU marshals arguments across a process boundary and tries to share CUDA tensors,
   hanging with no output.
6. **The embedder is a second model** with the same import-window problem. It loads lazily on the
   first query, so `_build()` embeds one throwaway string to force it in. Without that the Space
   starts fine and every answer carries `[agent] retrieval failed`.
7. **Gradio must own the public port.** With all of the above fixed but FastAPI on the port, the GPU
   was scheduled and acquired and ZeroGPU's own forked worker still died in `torch.init()`, while
   the platform probed the public port for `/api/predict` and got 404. A no-op `@spaces.GPU`
   function failed identically, so it was not the model or the payload.

`HF_TOKEN` must be a Space secret with read access to the private index dataset repo, or retrieval
is silently disabled.

`engine_api.py` is an unused-but-working alternative: `CONTROLAI_BACKEND=api` runs generation over
Inference Providers with everything else local. It needs a token carrying *Make calls to Inference
Providers*. Kept for anyone who wants a demo without a GPU at all.

### Benchmark (`benchmarks/`)
`controlbench_v1.jsonl` is the eval set; `SCOPE.md` defines the taxonomy and `README.md` the
data-hygiene rules (never let benchmark prompts leak into training data; split by `family`, not by
individual question). `scripts/eval_answer_quality.py` is the fast qualitative check across 18
domain cases.