docs/features/memory_embedder.md

# F4 — Memory / Embedder

> Parent: [../MASTER_PLAN.md](../MASTER_PLAN.md)
> Status: **planning** (M0+M1+M2 done — 2026-05-22)
> Owner: TBD
> Reference: `/shared/dev/claude-hooks` (the embedder lifecycle is
> mirrored from claude-hooks' Python implementation)

## Problem

opencoti needs persistent memory on day-one — well before F3
(`opencoti-server`, the Go companion daemon) is ready. Memory
needs an embedder, and the embedder needs to work without
requiring the user to install a separate runtime. The same
constraint that drove F2 (bundle llamafile so opencoti runs out
of the box) drives F4: ship a prebuilt embedder llamafile, manage
it with a small TypeScript launcher, default to CPU, allow
opt-in GPU.

There's a second, sharper problem the user surfaced from
real-world claude-hooks operation: the canonical embedder
(`qwen3-embedding-0.6b-16k.llamafile`, ~1.5 GB on disk for a
700 MB model) allocates **3.32 GB RSS** at idle. The bottleneck
is *per-slot context buffer* allocation, not the autoregressive
KV cache. The embedder server runs `--parallel 2 --ctx-size
16384` and pre-allocates the full 16K context for both slots up
front. Live workload shows most requests at 7–56 tokens with
only 1 slot active most of the time — so the eager allocation
pays full price for capacity that is almost never used.

F4 ships the embedder, *and* ships a lazy-slot-context patch
against the vendored llamafile/llama.cpp tree so the embedder's
RSS scales with actual demand instead of `slot_count × n_ctx_slot`.

## Goals

- **G1.** TypeScript package `@opencoti/embedder` that owns the
  embedder lifecycle: spawn, health-probe, idle-reap, GPU/CPU
  mode, `embed(text)` / `embedBatch(texts)`.
- **G2.** First-run setup downloads the prebuilt asset
  (`qwen3-embedding-0.6b-16k.llamafile`, dim 1024, default port
  **47092** in opencoti's reserved 47000-48000 range) into the
  opencoti data dir. Resume + SHA256 verify.
- **G2a.** **Daemon-launch policy** (project-wide convention).
  Any opencoti or opencoti-server daemon that opens a listening
  port MUST: (1) pick its default port from the **47000-48000
  range** — opencoti must never collide with claude-hooks'
  38000-39000 / 18790-18811 ranges so the two systems coexist
  on one host; (2) ask the user at first-run setup where to
  bind — `127.0.0.1` (default), all interfaces, or a specific
  IP — and (3) propose the default port but allow the user to
  override it before persisting the choice. The setup flow
  validates that the chosen port is free and within range
  before writing the config.
- **G3.** GPU mode = auto-detect with CPU fallback; default = CPU.
  Reuses F1 M5.7's `gpu-detect` chain (nvidia → amd → vulkan).
- **G4.** Composite fallback chain: an `EmbedderClient`
  interface + a `CompositeEmbedder` that tries primary then
  fallback on `EmbedderError`, with a dim-match assertion to
  prevent vector-space corruption.
- **G5.** Lazy-slot-context patch (`0001-lazy-slot-context.patch`)
  against the vendored llamafile so RSS scales with real demand:
  defer per-slot allocation, grow on demand, **shrink on idle**.

## Non-goals

- Replacing claude-hooks for users who already run it on the
  same host. opencoti's embedder is independent and uses its own
  port + data dir.
- Bundling weights. The model GGUF is inside the .llamafile
  asset; we don't ship a separate GGUF download.
- A full vector store + recall pipeline. F4 M5 wires recall +
  store as a thin seam; the real vector-store ownership lives
  in F3 (opencoti-server) when it ships.

## Default asset

- **Repo:** [`mann1x/claude-hooks`](https://github.com/mann1x/claude-hooks)
  releases — **transitional pin, two-stage migration**:
  - **Stage 1 (immediate)**: mirror the byte-identical asset to
    a `mann1x/opencoti` release (proposed tag:
    `embedder-v1.4.0`) and flip just the `repo` field in
    `vendors/pin/embedder.txt`. Same SHA, no rebuild —
    decouples opencoti's release stream from claude-hooks'.
  - **Stage 2 (after F4 M3 lazy-context patch ships)**: rebuild
    the embedder composite-llamafile to include the lazy-
    slot-context patch. The new binary has a different SHA;
    `tag` + `sha256` in the pin file both bump (proposed tag:
    `embedder-v1.5.0`). F4 M6 (release bundling) owns the
    apply-patches → make → zipalign-with-GGUF-and-args
    pipeline. Without Stage 2, the embedder still benefits
    from the lazy-context patch only when run against a
    user-supplied locally-built embedder binary — not the
    prebuilt asset.
- **Tag:** `v1.4.0`
- **Asset:** `qwen3-embedding-0.6b-16k.llamafile`
- **SHA256:** `414f616689aaba44f6982b474918e8549ad764c03b2f82f6b56a5d6e582ef59b`
- **Pinned in:** [`vendors/pin/embedder.txt`](../../vendors/pin/embedder.txt)
- **Default port:** **47092** (opencoti's reserved 47000-48000
  range — see G2a).
- **Endpoint:** `POST /embedding` (llama.cpp server style, not
  OpenAI-compat). Request: `{"content": "<text>"}` or batch
  `{"content": ["t1", "t2"]}`. Response: `{"embedding": [...]}`
  (single) / `[{"embedding": [...]}, ...]` (batch).

## Design sketch

### Layout

```
packages/opencoti-embedder/           # NEW
├── package.json
├── tsconfig.json
├── src/
│   ├── index.ts
│   ├── client.ts                     # EmbedderClient interface + types
│   ├── llamafile-embedder.ts         # POST /embedding HTTP client
│   ├── composite.ts                  # primary → fallback chain
│   ├── manager.ts                    # process spawn / health / idle reap
│   ├── gpu-mode.ts                   # CPU | auto resolution
│   ├── download.ts                   # asset fetch + SHA256 verify
│   └── config.ts                     # Zod schema for embedder config
├── script/
│   └── smoke.ts
└── test/

vendors/
├── pin/
│   └── embedder.txt                  # NEW (M0)
└── patches/
    └── llamafile/
        └── 0001-lazy-slot-context.patch   # NEW (M3)
```

### Reused, not rebuilt

- `@opencoti/tiers/registry/gpu-detect` — nvidia/amd/vulkan probe
  chain (F1 M5.7).
- `@opencoti/tiers/registry/hf-download` — resume + SHA256 verify
  download. To be generalized into a shared
  `@opencoti/tiers/util/download` helper consumed by both HF
  downloads and GitHub Release downloads.

## Milestones

### M0 — Vendoring decision + asset URL pin *(done — 2026-05-22)*

- `vendors/pin/embedder.txt` records repo / tag / asset / SHA256.
- Decision recorded: pin claude-hooks v1.4.0 directly until
  opencoti has its own release stream. Switch the pin URL to
  `mann1x/opencoti` when that exists.
- This feature plan + F5 plan + MASTER_PLAN.md updated together
  in the same commit.

### M1 — Embedder package, CPU mode, smoke green *(done — 2026-05-22)*

- `@opencoti/embedder` shipped with client interface,
  llamafile-embedder HTTP impl (POST /embedding, response-shape
  tolerant for A/B/C/D variants seen in the wild),
  download.ts (resume + SHA verify, builds the canonical
  github-release URL from the pin), manager.ts (spawn +
  `/health` poll + LRU idle reap @ 5 min default + idempotent
  stop), config.ts (Zod strict schema, defaults claude-hooks-
  parity), gpu-mode.ts (CPU default + auto-resolution),
  composite.ts (primary→fallback chain with dim-match guard).
- Shared `downloadWithResume` helper extracted to
  `@opencoti/tiers/util/download`; `hf-download.ts` refactored
  to consume it. New subpath exports:
  `@opencoti/tiers/util/download`,
  `@opencoti/tiers/registry/gpu-detect`,
  `@opencoti/tiers/registry/hf-download`.
- `script/smoke.ts` downloads the asset to `<repo>/.opencoti/embedder/`
  (skips on SHA match), spawns CPU-mode, embeds "hello world",
  asserts dim=1024 + L2-norm. Live smoke run is opt-in (asset
  is 1.5 GB).
- 52 unit tests across 7 files. Hook footprint unchanged
  (still 4 `opencoti-hook:` markers).
- **Out of scope (deferred to M5)**: opencode session
  injection, recall-into-prompt, vector store.

### M2 — Baseline measurement *(done — 2026-05-22)*

Ships two scripts and three fixture files:

- `script/generate-corpus.ts` — deterministic 48-prompt corpus
  generator spanning 8-3000 approx_tokens. Buckets: 10 short
  (8-32 tok), 11 medium (33-128), 19 medium-long (129-512), 6
  long (513-2048), 2 very-long (2049-8192). Original draft had
  5 in the very-long bucket; 2 mega prompts (60+ LOREM repeats)
  aborted past the 60s embedder request-timeout during the live
  baseline run, so they were dropped from the generator.
- `script/measure-baseline.ts` — boots the embedder via the M1
  manager (or attaches to a running instance with `--use-running
  <url> --pid <int>`), captures RSS from `/proc/<pid>/status`,
  extracts allocation lines from the embedder startup log,
  embeds the corpus, writes the fixture.

Fixtures (in `test/fixtures/`):
- `quality-prompts.json` — 48 prompts, deterministic.
- `quality-baseline.json` — 48 vectors (dim 1024) M3 must
  reproduce to cosine ≥ 0.999.
- `baseline-meta.json` — RSS + allocation breakdown +
  startup-log excerpt.

**Live measurement (2026-05-22, attached to claude-hooks
embedder, parallel=4 default, ctx=16384, CPU mode)**:

| Component | Size |
| --- | --- |
| Model weights (CPU_Mapped) | 603.87 MiB |
| KV cache (all slots) | **1792.00 MiB** |
| Compute buffer | 330.24 MiB |
| Output buffer | 2.33 MiB |
| RSS idle (process VmRSS) | **2.52 GB** |
| RSS peak (after 48-prompt run) | **3.81 GB** |
| Avg latency per prompt | 5352 ms |
| Total corpus embed time | 256.9 s |

Startup-log key lines (captured in baseline-meta.json):
`llama_context: n_ctx = 16384`,
`llama_kv_cache: CPU KV buffer size = 1792.00 MiB`,
`sched_reserve: CPU compute buffer size = 330.24 MiB`,
`server_main: embeddings enabled with n_batch (2048) >
n_ubatch (512) → setting n_batch = n_ubatch = 512 to avoid
assertion failure`, `slot load_model: id 0/1/2/3 | new slot,
n_ctx = 16384`. The binary defaults to parallel=4 with no
`--parallel` override.

The earlier user observation of **3.32 GB RSS** came from a
2-slot config explicitly setting `--parallel 2 --ctx-size
16384`. The M3 patch targets that production-shape config:

- Pre-patch per-slot KV allocation = `n_ctx_slot × 2 (K+V) ×
  layers × per-token-dim` ≈ 448 MiB / slot in CPU mode,
  allocated eagerly for every slot at startup.
- M3 target: per-slot allocation deferred to first prompt,
  sized to `--slot-initial-ctx 4096` (~112 MiB), grown on
  demand up to 16384, shrunk back to 4096 after
  `--slot-shrink-idle-ms`. Combined with 2 slots + model +
  compute, idle RSS target lands near 1.0 GB.

**Two corpus prompts intentionally dropped**: `lorem-mega-00`
and `lorem-mega-01` (60+ LOREM repeats) aborted past the
embedder's 60s default request-timeout on the CPU-mode
production instance — n_ubatch is forced to 512 in embedding
mode, so 9000+ token requests need ~18 forward passes that
exceed 60s. Not an M3 concern (the patch addresses
*allocation*, not *throughput*).

### M3 — Lazy-slot-context patch

> **Architecture reference:** [ADR 0001 — Lazy slot-context
> allocation](../decisions/0001-lazy-slot-context.md). The ADR
> documents the llama.cpp KV-cache layout, the three+1 orthogonal
> axes of "lazy," the four-phase rollout (defer / grow / shrink /
> per-stream split), the CLI surface, and the F5 milestone
> hand-offs that depend on the allocator hooks landed here.
> Implementation is multi-session per user directive B2.
>
> **Phase status (2026-05-23):**
> - **Phase 1 — deferred zero-fill: shipped** as
>   `vendors/patches/llamafile/0001-lazy-slot-context-defer.patch`.
>   Patch applies cleanly; build succeeds on x86_64 + aarch64;
>   smoke validation on Qwen2-Math-1.5B confirms the
>   `KV cache zero-fill deferred until first batch` startup log
>   line and the matching `ensure_cleared: ... (deferred)`
>   first-batch log line. Bench against M2 corpus + RSS table
>   for the embedder workload pending (Phase 1 alone is a
>   cold-start win; the headline warm-state RSS savings come
>   in Phases 2-3).
> - **Phase 2 — grow on demand + partial zero-fill: shipped**
>   as `vendors/patches/llamafile/0002-lazy-slot-context-grow.patch`.
>   Adds `--slot-initial-ctx N` (default 0 = Phase 1 behavior) that
>   caps initial KV-cell allocation per sequence; find_slot honors
>   the soft cap; prepare() grows on overflow up to kv_size_max via
>   next-pow2 geometric ramp; ensure_cleared partial-memsets only
>   the [n_cells_cleared, target) range via `ggml_backend_tensor_memset`.
>   With `--slot-initial-ctx 4096` for the production embedder,
>   warm-state RSS scales with actual workload instead of the
>   `slot_count × n_ctx_slot` product. Bench numbers vs M2 corpus
>   pending.
> - **Phase 3 — shrink on idle: shipped** as
>   `vendors/patches/llamafile/0005-lazy-slot-context-shrink.patch`.
>   Adds `--slot-shrink-idle-ms N` (default 30000); per-stream
>   idle timer sweeps at `init_batch` and reclaims buffers back
>   to `--slot-initial-ctx` after the timeout. Bench delta:
>   **391.7 MB returned to kernel** on the CPU embedder after 32 s
>   idle (1.68 GB peak → 1.30 GB post-shrink, cosine 0.999797 vs
>   M2 baseline).
>   **CPU-only — the shrink is gated on
>   `ggml_backend_buffer_is_host()` inside
>   `opencoti_decommit_layer_range()` because
>   `madvise(MADV_DONTNEED)` is a no-op against device-backed
>   pages.** On GPU (`-ngl > 0`) the `shrink_if_idle` log marker
>   still fires and the soft-cap bookkeeping rewinds, but the
>   CUDA/ROCm/Vulkan backend buffer is not freed and VRAM stays
>   at the peak the workload reached. The GPU-shrink path is
>   tracked at **F5 M2 (HeadInfer)**, not Phase 4.
> - **Phase 4 — per-stream tensor split: DEFERRED** (task #109,
>   target F5 M2). 2026-05-23 user decision after planning-session
>   exploration showed Phase 4 is a deep refactor (per-stream
>   tensors + per-stream backend buffers + reworked
>   `cpy_k`/`cpy_v` scatter ops that currently rely on
>   `ggml_reshape_2d(k, n_embd, kv_size*n_stream)` to flatten all
>   streams into a single `ggml_set_rows`) and only adds value in
>   non-unified mode. The embedder ships unified by default and
>   already gets all Phase 1-3 wins with the contiguous tensor
>   intact. Phase 4's real beneficiary is F5 (HeadInfer's per-head
>   GPU/CPU split + PolyKV's shared pool). Full implementation
>   map persisted in
>   [docs/decisions/0001-lazy-slot-context.md §Phase 4 — implementation map](../decisions/0001-lazy-slot-context.md)
>   for the F5 owner to pick up cold. **Not the GPU-shrink fix —
>   that's F5 M2 (HeadInfer), independently of Phase 4.**

- `vendors/patches/llamafile/0001-lazy-slot-context.patch`.
- Behavior:
  - Defer per-slot KV/compute-buffer allocation until slot's
    first real prompt.
  - Initial allocation rounded up to `--slot-initial-ctx`
    (default 4096).
  - Grow on demand (next power-of-two), cap at `n_ctx_slot`.
  - **Shrink on idle** (`--slot-shrink-idle-ms`, default
    30000): free buffers and return to initial-ctx after the
    timeout, otherwise the first 16K request permanently sticks
    the slot at 16K and the lazy-allocation win evaporates.
- Bench (`perf/llamafile/embedder-rss.bench.ts`): RSS at idle /
  after 8-token request / after 16K request / after idle
  timeout + 8-token request. Quality: cosine ≥ 0.999 vs M2's
  baseline.
- Header: `Milestone: F4 M3`, `Upstreaming: TBD — propose to
  llamafile + llama.cpp upstream once bench numbers are public`.

### M4 — Auto-GPU mode + composite fallback

- `gpu-mode.ts` adds `"auto"`: probe nvidia → amd → vulkan; if
  VRAM ≥ 1.6 GB, launch with `--gpu auto` + n-gpu-layers fully
  offloaded; otherwise silent CPU fallback.
- `CompositeEmbedder` ships: primary (local llamafile) +
  fallback (cloud-route delegate, when Tier 1 is configured).
  Dim-mismatch assertion lives here.

### M5 — Opencode integration: recall + storage seam *(2026-05-23 — shipped, rescoped mid-milestone)*

**Original scope:** one synthetic tools pair + one surgical session-end hook +
setup-flow listen-address question. **Actual scope after the user rescope
(verbatim): "memory store and recall is going to be another crucial point,
we support sqlite + sqlite_vec as base memory backend, managed by
opencoti-server. opencoti-server will manage optional sharing on the
network. the pgvector db is an add on memory connector. both works in
parallel. memory in opencoti can be managed per-session. user can manage
which memories to use and list, delete, connect to the memories via the
harness menu. shared memory for all sessions, specific memories can be
added and or connected. for each session the user can decide which one
to read/write/rw"**

Path taken (after user decision): **in-process M5 now, refactor to
opencoti-server later**. The opencoti-server piece is queued for F3
once F4 closes. The data model and ACL contract land here so they
survive that refactor unchanged.

What M5 actually shipped:

- **M5-A — `@opencoti/memory` package.** `SqliteVecStore` over
  bun:sqlite + sqlite-vec 0.1.9 vec0 virtual table. Embedding dim
  locked at DB creation (default 1024). Schema: `collections`,
  `memories` (with `content_hash` UNIQUE for idempotent insert),
  `memory_acl` (per-session override), `vec_memories` (vec0
  virtual table with FLOAT[dim]). Default DB path
  `~/.opencoti/memory/state.db`. ACL model: explicit overrides win;
  defaults are "rw" for global collections, "rw" for the owning
  session of a session-private collection, "none" otherwise. The
  resolver lives in pure code (`resolveAccessMode`, `canRead`,
  `canWrite`) and is exhaustively tested.
- **M5-B — synthetic tools.** `__memory_recall(query, k?, collections?)`,
  `__memory_store(content, collection?)`, `__memory_list()`. All
  three are ACL-aware (sessionID is the actor) and best-effort —
  embedder failure returns `{ok: false, error: "embedder_unavailable"}`
  rather than throwing. `ensureSessionCollection` is idempotent.
- **M5-C — runtime wire-up via dynamic-import bridge.**
  `@opencoti/tiers/src/memory-bridge.ts` resolves
  `@opencoti/memory` + `@opencoti/embedder` via dynamic import to
  break the tiers→memory→embedder→tiers cycle. Wired into
  `runtime.ts:maybeRoute` via `mergeTools` (no new surgical
  hook — extends an existing additive seam). The default
  embedder is `createDefaultEmbedFn` (loopback LlamafileEmbedder
  on 47092). Workspace dep `@opencoti/memory` added on
  `packages/opencode` to make the dynamic import resolvable;
  registered in UPSTREAM_SYNC.md as `memory-tools-resolvability`
  (dep).
- **M5-D2 — `@opencoti/memory-plugin` (opencode SERVER plugin).**
  Two hooks: `experimental.chat.system.transform` appends a
  system-prompt suffix advertising the memory tool surface +
  listing the session's accessible collections;
  `event` listens for `session.idle` and, when
  `auto_ingest: true` (default FALSE), ensures the session-private
  collection. Defaults reflect the rescoped vision: advertise=true
  (the point of loading the plugin), auto_ingest=false (the agent
  already has `__memory_store`).
- **M5-D1 — `@opencoti/tui-memory` (opencode TUI plugin).** One
  command `opencoti.memory` opens a DialogSelect listing every
  collection (global + session-private), with per-session
  resolved-mode annotations when in a session route. Drill-down
  per collection offers: ACL toggle r/w/rw/none (session-only),
  DialogConfirm-guarded delete, Create flow
  (DialogPrompt name → scope DialogSelect). Lazy store open,
  closed cleanly on `api.lifecycle.onDispose`.

**Default-plugin auto-wiring (2026-05-23 follow-up to user
directive: "opencoti will need to wire them by default"):**
The opencoti plugins remain opt-in for upstream-opencode users
(they stay out of upstream's plugin list unless explicitly
declared), but opencoti's distribution auto-wires them whenever a
project has any `opencoti.*` config section. Implementation in
`@opencoti/tiers/default-plugins`: a pure helper
`applyDefaultPlugins(cfg)` gap-fills `@opencoti/memory-plugin`,
`@opencoti/tui-memory`, `@opencoti/tui-tiers` into the final
`plugin_origins` list — user-declared entries (including their
options) are preserved untouched. Wired via two surgical hooks
in `packages/opencode/src/config/config.ts` (import + call-site,
registered as `opencoti-default-plugins`). Net change vs M5
substance: TWO new opencoti-hook markers (still tagged + in
registry). With opencoti disabled (no `cfg.opencoti` block) the
helper is a no-op and the byte-output is identical to upstream.

What M5 deliberately did NOT ship:

- The session-end **surgical** hook (the original plan). Replaced by
  the optional `event: "session.idle"` listener in `@opencoti/memory-plugin`
  with `auto_ingest: false` default. Net new `opencoti-hook:`
  markers from M5 substance + follow-up: TWO (the
  `opencoti-default-plugins` import + call-site). The
  `memory-tools-resolvability` dep is a registry-only entry (no
  source marker — JSON has no comments).
- The pgvector add-on connector. Deferred — the in-process backend
  is sqlite-vec only for now. Pgvector becomes a parallel
  `MemoryStore` implementation once opencoti-server (F3) lands.
- The network-sharing layer. That's opencoti-server's job (F3).
- The "Clear ACL override" UI option in the TUI panel. The
  `MemoryStore` interface does NOT currently expose a clear-ACL
  method — adding it would require a new schema/interface revision
  (deferred to a future minor bump).
- **M5-E — setup-flow embedder daemon listen address question
  (shipped 2026-05-23).** `@opencoti/tiers/cli/setup-flow`
  extended with an embedder listen-address prompt in
  `finalizePatch`. Steps: (1) DialogSelect over
  loopback / all-interfaces / specific-IP / skip; (2) if
  specific-IP, prompt for the IP with an IPv4 validator;
  (3) text prompt for the port, validated against the opencoti
  47000-48000 range; (4) port-free check via injected
  `isPortAvailable(host, port)` (live: TCP bind probe in
  setup.ts) — busy port logs a warn and asks the user whether to
  proceed anyway; "no" recursively re-asks the bind block. The
  chosen `{host, port}` lands in
  `~/.config/opencode/opencode.jsonc` under
  `opencoti.embedder.host` + `.port` via the jsonc-parser
  modify-then-applyEdits path (comments preserved). A new
  `embedderConfigured` sentinel (true when BOTH host AND port
  already present in the user's config) gates the prompt on
  re-runs. 16 new tests across the flow + the `validatePort` and
  `isValidIPv4` pure helpers.

### M6 — Production llamafile build pipeline *(2026-05-23 — promoted ahead of M5)*

Driver: user directive 2026-05-23 — the post-F4 M4 state (prebuilt
claude-hooks asset + manual zipalign of ggml-cuda.so + cross-repo
pin dependency) is "patchwork […] not acceptable for production".
M6 was originally scoped to the embedder composite alone; the
expanded M6 below covers all three backends (CUDA / ROCm / Vulkan)
and the local-asset modes the embedder downloader needs to
consume an opencoti-built artifact.

M6 lands before M5 (opencode session-end hook) because there's no
point storing memories on the back of a launcher binary that
doesn't engage the M3 lazy-context patch.

**What M6 ships:**

1. **Multi-backend DSO builders in
   `packages/opencoti-llamafile/script/build-pipeline.ts`:**
   `runCuda` / `runRocm` / `runVulkan` / `runAllBackends`,
   driven by a shared `buildBackend(spec)` helper.
   - Each backend has a toolchain probe: `OPENCOTI_CUDA_PATH/bin/nvcc`
     (default `/usr/local/cuda-12.6` — CUDA 13.x is known-broken
     with `cuda.sh`), `OPENCOTI_ROCM_PATH/bin/hipcc` (default
     `/opt/rocm`), `OPENCOTI_VULKAN_SDK` env / system `glslc`.
   - Soft-fail per backend: a host without ROCm produces a clean
     CUDA-only build instead of failing the pipeline.
   - Each built DSO is staged at
     `vendors/dist/llamafile/<backend>/<ver>/ggml-<backend>.so`
     AND mirrored to `~/.llamafile/v/<ver>/` so devs running the
     thin binary directly still get GPU support locally.

2. **`package` subcommand in the same script:**
   Zipaligns the thin patched binary with every staged DSO via
   the vendored `vendors/sources/llamafile/o/third_party/zipalign/
   zipalign -j0` invocation. Always emits
   `dist/llamafile/opencoti-llamafile-<ver>-<arch>.llamafile`
   plus a sibling `MANIFEST.json` (artifact SHA + per-DSO SHA +
   patches list).
   - With `--with-model PATH` ALSO emits
     `dist/llamafile/opencoti-embedder-<ver>-<arch>.llamafile`
     by copying the bare and zipalign-embedding the GGUF + a
     `.args` file generated from the F2 M3 canonical embedder
     args contract (`-m /zip/<basename> --server --embedding
     --pooling last --ctx-size 16384 --parallel 2
     --slot-initial-ctx 4096 --slot-shrink-idle-ms 30000`).
   - `--with-args PATH` overrides the generated `.args`.
   - `--cpu-only` produces a binary with NO DSO embedded (for CI
     smoke on hosts without GPU).

3. **Pure helpers in
   `packages/opencoti-llamafile/src/build-pipeline.ts`** (kept
   side-effect-free per F2 M3 convention): `Backend` type +
   `ALL_BACKENDS`, `dsoFilename`, `stagingDir`, `artifactDir`,
   `readBackendArtifacts`, `sha256OfFile`, `composeEmbedderArgs`,
   `parseBuildArgv`, `ArtifactManifest` type. 19 new unit tests
   under `test/build-pipeline.test.ts`.

4. **`@opencoti/embedder` downloader local-asset modes
   (`src/download.ts`):**
   - `file:` prefix on the pin's repo field — bytes come from
     disk; SHA still verified.
   - `OPENCOTI_EMBEDDER_LOCAL=<abs-path>` env override —
     bypasses pin path entirely; SHA still verified.
   - Both routed through a shared `localCopyPath` helper; the
     HTTPS+resume path is unchanged. 12 new tests in
     `test/download.test.ts`.

5. **Hook footprint unchanged.** M6 is pure build orchestration +
   downloader extension; no surgical hooks in upstream opencode.

**Deferred to F4 M7 (release bundling):**

- GitHub Releases publish pipeline. Producing artifacts under
  `dist/llamafile/` is M6; uploading them as `mann1x/opencoti`
  release assets and flipping `vendors/pin/embedder.txt` to
  `mann1x/opencoti` (Stage 2 of the migration the existing pin
  file header describes) is M7. The pin migration is a one-line
  change once the release exists.
- Metal / macOS backend. `vendors/sources/llamafile/llamafile/
  metal.c` is macOS-only; M6 ships Linux-only since the build
  host is solidPC. Needs a Darwin CI runner.
- aarch64 / multi-arch DSO orchestration. cosmocc APE is dual-
  arch by design, but ggml-*.so are arch-specific. The `<arch>`
  token in the output filename is anticipatory; M6 ships x86_64
  only.

### M7 — Release bundling + dev-cut workflow alignment with claude-hooks

The build pipeline (M6) produces `dist/llamafile/*.llamafile` artifacts.
M7 wraps that with the **commit-the-contract, gitignore-the-bytes**
release scheme claude-hooks uses, plus a `release-cut` subcommand
that prepares (but does NOT auto-publish) a `mann1x/opencoti`
GitHub Release.

**What's tracked (committed):**

- `vendors/llamafile/README.md` — audit-trail doc describing the
  pinned upstream + the contract layout. Mirrors claude-hooks's
  `vendor/llamafile/README.md`.
- `vendors/llamafile/LICENSE.upstream` — single-file copy of the
  four upstream LICENSE files (Mozilla-Ocho/llamafile Apache-2.0 +
  llama.cpp/whisper.cpp/stable-diffusion.cpp MIT × 3) at the
  pinned commit. Survives independently of the submodule.
- `vendors/llamafile/SHA256SUMS.composite` — the contract.
  `sha256sum`-compatible lines listing every released composite
  artifact. Refreshed by `build:llamafile:release-cut`. Verified
  by the embedder downloader at install time.

**What's gitignored (reproducible from source):**

- `vendors/dist/llamafile/<backend>/<ver>/ggml-*.so` — per-backend
  staging DSOs.
- `dist/llamafile/*.llamafile` — final composite artifacts.
- `dist/llamafile/*.MANIFEST.json` — per-artifact manifests with
  embedded-backend metadata.

**Release-cut workflow** (matches claude-hooks's
`make -C vendor/llamafile/dist`):

1. `bun run build:llamafile` — patched binary.
2. `bun run build:llamafile:all-backends` — every available DSO.
3. `bun run build:llamafile:package -- --with-model PATH` — bare
   + composite artifacts.
4. `bun run build:llamafile:release-cut` — refreshes the SHA
   contract; prints the suggested `gh release create` invocation
   (the actual publish is a user-confirmation step, not
   automated).
5. User executes `gh release create` + `gh release upload`, then
   commits the refreshed `SHA256SUMS.composite` + the bumped
   `vendors/pin/embedder.txt`.

**Downloader behavior:** when `shaContractText` is provided to
`downloadEmbedder`, the pin's SHA is cross-checked against the
contract before any download or local-copy. Disagreement is a
hard error with a "redownload or rebuild" breadcrumb (matches
claude-hooks's `install.py`). An asset not present in the
contract is a soft warning, not an error — supports the
transitional state where the pin still references claude-hooks
pre-flip.

**What's NOT in M7:**

- Actual first `mann1x/opencoti` GitHub Release publish — that's
  the user-confirmed external step the scaffolding enables.
- Pin flip from `mann1x/claude-hooks` → `mann1x/opencoti` — only
  meaningful after the first release exists.
- macOS / aarch64 — same constraints as M6.

**Critical files (M6):**

- `packages/opencoti-llamafile/script/build-pipeline.ts` — new
  subcommands.
- `packages/opencoti-llamafile/src/build-pipeline.ts` — pure
  helpers + types.
- `packages/opencoti-llamafile/test/build-pipeline.test.ts` —
  helper tests (19 new, 46 total).
- `packages/opencoti-embedder/src/download.ts` — local-asset
  modes.
- `packages/opencoti-embedder/test/download.test.ts` — local-
  asset tests (12 new, 13 total for download).
- `docs/features/llamafile_build.md` — toolchain + pipeline
  documentation.
- Root `package.json` — new `build:llamafile:{rocm,vulkan,
  all-backends,package}` scripts.
- `.gitignore` — `vendors/dist/`, `/dist/llamafile/`.

## Surgical-hook footprint

- M1–M4, M6: zero new hooks.
- M5: exactly one new hook (session-end memory ingest).

## Open questions

- **Release stream ownership.** Decision (M0): pin claude-hooks
  v1.4.0. When opencoti's own release cadence starts, switch
  the pin URL to `mann1x/opencoti`. Same SHA + same asset on
  disk; just a different `repo` field.
- **Composite-llamafile build.** F4 consumes the prebuilt
  asset. If F4 M3's lazy-context patch lands and we want it in
  the embedder binary too, we need to build our own
  composite-llamafile (vendored llamafile + Qwen3 GGUF +
  `.args`). That's potentially a small new milestone (F4 M6)
  that overlaps with F2 M6 (release artifact bundling).
- **Idle-reap vs always-on for the manager.** 5-min idle reap
  matches claude-hooks. For opencoti, sessions are typically
  longer; revisit if cold-start latency turns out to be an
  issue in production use.
- **Sqlite-vec dep choice.** The Bun-native binding vs
  `better-sqlite3` + the `sqlite-vec` extension. Decide at M5
  based on which builds cleanly across Linux/macOS/Windows.

## Risks

- **F4 M3 patch surface area.** Slot-init touches code that F5's
  ReST-KV (M1) and HeadInfer (M2) also modify. Ordering: F4 M3
  lands first as patch `0001-`; F5's start at `0010-`. On a
  later upstream pin bump, M3 needs to rebase first.
- **Asset hosting.** While we pin claude-hooks' release, the
  embedder's lifecycle is tied to claude-hooks' tagging
  cadence. Mitigation: mirror to opencoti's own release once
  it's set up.
- **`@opencoti/tiers` ↔ `@opencoti/embedder` boundary.** The
  embedder depends on tiers (for GPU detect + download helper),
  and tiers dynamic-imports `@opencoti/llamafile`. No cycle
  introduced (embedder is the new edge of the DAG), but confirm
  with `bun --cwd packages/opencoti-embedder typecheck` after
  the shared-download refactor.

## M8 — opencoti-server owns the embedder process (daemon-side lifecycle)

> **Status: in progress (2026-07-18).** Local task **#815** / STATE_SUMMARY
> "OPEN — Distribution & provisioning rebuild". Opened after the 3-way
> distribution audit found the Go daemon's embedder is **client-only**.

### Why

M0–M7 built the embedder lifecycle (spawn / `/health` / idle-reap /
GPU-auto+CPU-fallback / download) in the **TypeScript** `@opencoti/embedder`
package — and even there it is reachable only from scripts; the opencode runtime
is itself a bare HTTP client to `:47092` (`default-embed-fn.ts`). The **Go daemon
`opencoti-server`** (SP-M1 #768) only ever gained an HTTP *client*
(`internal/embed/embed.go`): an empty `--embedder-url` yields a **disabled**
embedder with **no fallback and no process management**. That is the gap: the
daemon must own the embedder the way `claude-hooks-daemon` does
(`embedding_manager.py`) — demand-spawn the bundled `qwen3-embedding-0.6b`
llamafile, `/health`-gate, idle-reap + demand-respawn, GPU-auto/CPU-fallback,
download-if-missing, and **default to the bundled embedder when no external
endpoint is configured**.

### Architecture — reuse the supervisor module, add an embedder controller

The `opencoti/llamafile-supervisor/` Go module's package doc **explicitly names
"opencoti-server in prod — a future consumer that imports this package and
registers its own controller"** (`supervisor/controller.go:1-12`). So:

- **`internal/embedsvc` (new).** An `supervisor.EngineController` for the
  embedder: `Start` spawns the composite `.llamafile` via `sh <bin> <args>`
  (Cosmopolitan APE), `Stop` SIGTERMs + waits for exit, `HealthURL`/`UpstreamURL`
  point at the child. **Args ported verbatim from `manager.ts buildEmbedderArgs`**
  (`--server --embedding --host --port --parallel --ctx-size --pooling` +
  GPU/CPU flags + lazy-slot flags). **GPU auto→CPU fallback** ported from
  `gpu-mode.ts` (probe VRAM ≥ min → `--gpu auto -ngl 99`, else `--gpu disable`;
  on a GPU spawn/health failure, flip a sticky CPU flag and relaunch — silent,
  matching claude-hooks).
- **`supervisor.Supervisor`** provides, unchanged, the single-flight
  `ensureRunning`, `waitReady` (`/health` 200), idle-reap + `StartReaper`, and the
  VRAM precheck (reuse `supervisor.VRAMProber`). The daemon runs
  `Supervisor.Handler()` on a **loopback listener** (a free port in 47000–48000)
  and points its own `internal/embed` client at that port — so a
  `/v1/memory/search` query embeds through the supervisor, which **demand-spawns
  the child on the first request and reaps it on idle**.
- **Download/install** (`internal/embedsvc` or `internal/embed`): read
  `vendors/pin/embedder.txt`, resolve a local path under `~/.opencoti/embedder/`,
  and **download-if-missing** (resume + SHA256 verify) — porting `download.ts`.
  Source order: `OPENCOTI_EMBEDDER_LOCAL` / `file:` override → the pin repo. The
  pin now has a **public HF mirror** at
  `ManniX-ITA/opencoti-llamafile → embedder/qwen3-embedding-0.6b-16k.llamafile`
  (uploaded 2026-07-18) so the anonymous path works while `mann1x/opencoti` is
  private. Download is **lazy** — only on the first embed that finds the binary
  absent — so daemon boot (and the whole server test suite) stays byte-identical.

### Opt-in / default model

- **Default-to-bundled is the point** (user directive): when **no
  `--embedder-url`** is set and `--embedder-autostart` (default **true**) is on,
  the daemon stands the bundled embedder up behind the supervisor and embeds
  server-side. Everything is **lazy** (no spawn, no download at boot) → boot
  byte-identical; the child only appears on the first real embed.
- **External endpoint wins:** a non-empty `--embedder-url` short-circuits
  autostart (explicit URL → direct client, exactly as SP-M1 today).
- **Escape hatch:** `--embedder-autostart=false` + no URL = embedder disabled
  (today's behaviour, for minimal/test deployments).

### Phases

| Phase | Scope | State |
|-------|-------|-------|
| P0 | M8 plan (this) + `opencoti.server.embedder` config schema + cerebrum | in progress |
| P1 | `internal/embedsvc` spawn-child controller (args/gpu ported, spawn/stop/health) + go.mod require/replace the supervisor module + unit tests | **done** — P1a `args.go` (arg/GPU builder ported from `manager.ts`/`gpu-mode.ts`); P1b `controller.go` implements `supervisor.EngineController` (Start/Stop/HealthURL/UpstreamURL/Mode) with `sh <bin>` APE spawn, SIGTERM+wait stop, and **GPU-auto→CPU sticky fallback** on spawn/exit/health-timeout; injectable spawn/VRAM/health seams. 76% cov, `-race`+`vet` clean, `go build ./...` green |
| P2 | download/install + pin verify (port `download.ts`; `~/.opencoti/embedder/`; HF-mirror/`file:`/`LOCAL` sources) + tests | **done** — `download.go`: `EnsureBinary` (LocalPath override → cached → download), **HF-native** `DefaultPin`/`Pin.URL()` (claude-hooks GitHub release purged per 2026-07-22 directive), resume (`.part` + Range) + SHA256 verify + atomic rename, `~/.opencoti/embedder/`. 8 tests (pin-is-HF-only, local-override±SHA, cached, fresh-download, SHA-mismatch-cleans-part, resume-from-partial); 75% cov, `-race`+`vet` clean, build green |
| P3a | `embedsvc.Service` glue: Controller + `supervisor.Supervisor` + loopback front listener; `EmbeddingsURL()`/`Front()`/`Mode()`/`Close()`; lazy `WithBinResolver` (download deferred to first request). Integration test (real in-process child, demand-spawn+proxy through the front, lazy-resolve-once). 78% cov, `-race`+`vet`+build green | **done** |
| P3b | main.go wiring: `buildEmbedder` default-to-bundled path (build+Start the Service, embed URL → `Service.EmbeddingsURL()`, Close on shutdown), `--embedder-autostart`/`--embedder-bin`/`--embedder-gpu`/`--embedder-idle` flags + `opencoti.embedder.*` bridge, `embedder_ok`/`embedder_mode` healthz, Windows baking | **done** — `buildEmbedderStack` (external / bundled-lazy / disabled); `--embedder-autostart`(default true)/`--embedder-bin`/`--embedder-gpu`/`--embedder-idle`/`--embedder-port` flags + Windows install baking; `Options.EmbedderMode` + `embedder_ok`/`embedder_mode` healthz + test. **Live smoke (3 modes):** autostart-off→`disabled` (byte-identical), autostart-on→`bundled-stopped` (lazy, no spawn/download at boot), url-set→`external`. server suite green. TS `opencoti.embedder.*`→flag bridge deferred to the wizard (F1 M11). |
| P4 | integration tests (fake embedder child: demand-spawn, reap, GPU→CPU fallback, download SHA), byte-identical-when-disabled, full gate; docs + `.wolf` + memory | **done** — **native suites green** (`embedsvc`: args/controller/download/service tables incl. fake-child demand-spawn+proxy, lazy-resolve-once, GPU→CPU sticky fallback, HF-pin-only, SHA-mismatch-cleans-`.part`, resume-from-partial; `server`: `embedder_ok`/`embedder_mode` disabled/external/bundled). **Cross-compile clean** (windows/amd64, linux/arm64). **Live smoke on solidPC (GPU host, real HF download):** daemon boots `bundled-stopped` (lazy — no spawn/download at boot, byte-identical); first `POST /v1/embeddings` demand-downloads the pin from HF (1.5 GB, SHA `414f6166…2ef59b` verified == `DefaultPin`, `.part` gone, 1 517 042 662 B) → demand-spawns → **HTTP 200, 1024-dim vector**, healthz flips `bundled-stopped`→`bundled-cpu`. **Idle-reap + cached respawn:** after idle (90 s) the reaper flips healthz back to `bundled-stopped` (~40 s); a second embed respawns from the **cached** binary (no re-download, ~1 s) → HTTP 200, 1024-dim, `bundled-cpu`. |

### Constraints

Additive: new Go in `opencoti/server/internal/embedsvc` + a cross-module import of
the already-present `opencoti/llamafile-supervisor` (require + local `replace`);
new `opencoti.server.embedder` config rides the existing
`opencoti-server-autostart` schema hook (no new marker). No upstream source edits.
Byte-identical when `--embedder-url` is set or `--embedder-autostart=false`. No
`meta.schema_version` bump. Go tests `-tags sqlite_fts5`.