--- license: apache-2.0 pretty_name: Threefold (L-lens) tags: - ai-observability - interpretability - provenance - mechanistic-interpretability --- # threefold **Tombstone ∥ Covenant ∥ L-lens**: an experimental AI observability and provenance apparatus. Threefold links three things under one task identity: mechanistic measurements, persistent structured memory, and an append-only execution record. **L-lens** is the Threefold observability layer built around upstream **Jacobian Lens (JLens)** measurements. It is not a replacement name for the Jacobian Lens. JLens is Anthropic's instrument ([anthropics/jacobian-lens](https://github.com/anthropics/jacobian-lens), Apache-2.0), and Threefold runs it unmodified. ``` Anthropic Jacobian Lens L-lens (ours) Threefold linkage jlens.JacobianLens.apply → llens.py: hashes the capture, → one task_id, three legs: (run by jlens_snapshot.py; derives summaries from the raw tombstone entry (cites both digests) its outputs stored as raw) arrays, packages and links them covenant memories task-/jlens-/llens- verified by triad/verify_triad.py ``` Every completed task fires **all three** legs under one shared `task_id`. There is no degrade path. If any leg fails, or the legs do not agree, the triad fails. | Leg | What it is | Artifacts | |-----|------------|-----------| | **Tombstone** | Append-only run log | `tombstone/tombstone.md` (or `$TOMBSTONE_LOG`); the notes carry `task_id`, `jlens_capture_sha256`, `llens_sha256` | | **Covenant** | Structured AI memories, written by [covenant](https://github.com/LAWLESS1987/covenant)'s `ai_memory_system` | `task-` (both digests), `jlens-` (the JLens capture), `llens-` (the L-lens artifact) under `AI_MEMORY_ROOT` | | **Lens** | Upstream JLens measurements (`jlens_snapshot.py`), and the L-lens layer around them (`llens.py`) | `runs//jlens_snapshot.json`, `jlens_raw.npz`, `jlens_vocab.json`, `llens.json` | Self-observation is **not** a triad leg. ## The Jacobian Lens: what it measures, and what it does not These are upstream's terms. The lens linearly transports a residual-stream vector at layer `l` and one token position into the final-layer basis, using the average input-output Jacobian `J_l = E[∂h_final/∂h_l]` fitted on a text corpus. It then decodes the result with the model's own unembedding into logits over the vocabulary. A readout says which tokens that residual is disposed to make the model predict. It is a mechanistic probing technique that produces layer-level predictive information about **one subject model's** internal states. Threefold makes no claim that it reads thoughts, beliefs or intent, detects consciousness, or establishes semantic ground truth. **The subject is not the agent.** By default the subject model is GPT-2 small (`openai-community/gpt2`), reading the text of a task (for a speak-turn, the words that were spoken). The lens observes GPT-2's residual stream while it reads that text. It does not observe whatever agent wrote the text. Every capture says so in `subject_model.note`. ## What a capture records (`jlens_snapshot.py`) The defaults follow upstream's own API. `JacobianLens.apply` reads **every fitted layer** when `layers=None` and **every token position** when `positions=None`. Upstream's slice view (`jlens.vis.compute_slice`) keeps the top 10 tokens per (position, layer) cell. The capture does the same for: - the Jacobian-lens logits (`use_jacobian` on); - upstream's vanilla **logit-lens baseline** (`use_jacobian` off) at the same cells, so the effect of the Jacobian transport can be compared against plain unembedding; - the model's own final-layer logits at the same positions. For the bundled GPT-2 lens that is all 11 fitted layers (0–10; layer 11 is the model's output) at every position. The earlier version of this script read 4 layers at position −2. Those cells are a subset of the new readout and give the same tokens. | Field | Where | |-------|-------| | task_id | `task_id` | | model and version | `subject_model.id`, `subject_model.revision` (Hub commit), `n_layers`, `d_model`, `vocab_size`, dtype, device | | lens and its source | `lens.source`, `lens.sha256` (of the .pt file), `lens.n_prompts`, `lens.source_layers` | | upstream library | `upstream.source`, `upstream.version`, `upstream.vcs_commit`, `upstream.call` | | prompt / input | `input.prompt`, `input.prompt_sha256`, `input.input_ids`, `input.n_tokens`, `input.starts_with_bos` | | layers and positions | `readout.layers`, `readout.positions`, `readout.top_k`, `readout.baseline` | | raw JLens outputs | `jlens_raw.npz`: `jacobian_top_ids/logits`, `baseline_top_ids/logits` `[layer, position, K]`, `final_top_ids/logits` `[position, K]`, `input_ids`, `layers`, `positions` | | timestamps, runtime | `timing.started_utc`, `model_load_s`, `apply_jacobian_s`, `apply_baseline_s`, `total_s` | | environment | `env`: python, torch, transformers, numpy, platform, thread count, seed | | digests | `llens_capture_time.array_sha256` (every raw array), `full_tensor_sha256` (every complete `[positions, vocab]` tensor), `vocab_sha256` | **Raw versus derived.** The `.npz` holds only values copied out of the tensors upstream returned. The one operation applied is `torch.topk` selecting each cell's top K. Hashes are L-lens's work. Where they must be taken at capture time (the full tensors are too large to keep, about 24 MB per layer at 120 tokens), they sit in the separate `llens_capture_time` section, never under raw. Decoded token strings are a tokenizer view of the ids, not a measurement. They live in `jlens_vocab.json` (hashed), and `python llens.py show` prints them. ## L-lens (`llens.py`) L-lens consumes a capture and writes `llens.json`: - `inputs.jlens_capture_sha256`: the canonical-JSON sha256 of the capture it was built from, plus the digest of every raw array; - `derived`: integer summaries computed **only** from the stored raw arrays, so they can be recomputed exactly. For the Jacobian lens and the baseline, each layer gets `top1_agree`, `final_top1_in_topk` and `topk_overlap` (each against the model's own final top-K), and each position gets `first_layer_final_top1_in_topk`. The definitions are written into every artifact; - `upstream_credit`, `subject`, `links` (task_id and the names of the tombstone/covenant records), `what` (the statement of what L-lens is and is not). The derived numbers are observability statistics about one model reading one text. `first_layer_…` is limited to the stored top-K: a token ranked K+1 counts as absent. ## Verification `python llens.py verify --capture … --raw … --llens …` checks the lens leg on its own: the array digests, the vocabulary digest, shapes against the declared readout, that the artifact cites this capture, and that `derived` recomputes exactly. `python triad/verify_triad.py --task-id ID` checks the whole triad: the lens leg as above, a tombstone entry whose notes carry this task_id **and** both digests, and covenant memories `task-/jlens-/llens-` whose contents give the same task_id and the same canonical digests. `fire_triad.sh` runs it after every triad. Any FAIL exits 1. **What verification cannot see.** It checks that the records agree with each other and with the stored arrays. It does not re-run the model. To check that the arrays are what upstream returns, re-run with the recorded configuration and compare `full_tensor_sha256`; `tests/check_against_upstream.py` does that. Bit equality across different CPUs or BLAS builds is not guaranteed. Covenant's own hash chain is checked by covenant (`verify_chain`), not here. Verification runs after the legs are written. A failure marks the triad failed, and it does not unwrite append-only records. ## Tests ```bash python -m pytest tests/test_llens.py # no model needed: derivation, hashing, every link broken on purpose python tests/check_against_upstream.py runs/ID # model + lens: the stored capture vs a direct upstream apply call COVENANT_SRC=… tests/e2e_triad.sh # model + lens + covenant: a real speak-turn in a scratch dir, then broken 3 ways ``` Results when this was committed (2026-10-08; Windows 11, CPU, Python 3.12.10, torch 2.14.1+cpu, transformers 5.19.0, jlens 0.1.0 at 581d398): - `tests/test_llens.py`: 26 passed. - `check_against_upstream.py` on the seed prompt: 70 of 70 comparisons equal. The stored top-K ids and logits, and every full-tensor digest, match a direct upstream call that reads all layers at once. - `e2e_triad.sh`: a speak-turn verified with 44 checks. A wrong tombstone digest and a missing L-lens memory each failed with that named reason, and restoring each restored green. A broken lens file failed the triad before anything was appended. - `scripts/seed_demo.sh`: `MODE=apply` and `MODE=triad` both OK (scratch tombstone and memory root). - A 480-character speak-turn (106 positions): 12 s total, raw arrays 135 KB, memory bodies 10 KB (capture) and 9 KB (L-lens). - Not exercised: a Linux run, `scripts/push_to_github.sh`, and pushing to the live `threefold-memory` repository (the tests use a scratch memory root that is not a git repo). ## Known limits - One subject model (GPT-2 small) and one pretrained lens (fitted on 277 wikitext prompts; hosted by Neuronpedia). The readout is only as good as that lens. Upstream's paper uses lenses fitted on ~1000 prompts. - Only the top K of each cell is stored. Full tensors are hashed, not kept. - Covenant's memory gate is a bag-of-words judge. It refused a capture whose text contained upstream's literal parameter value for the baseline, and another that listed decoded vocabulary. So records say "use_jacobian off", and decoded strings stay out of memory bodies. A spoken text the gate refuses fails the triad, closed. - The live runner that writes `lspace-`/`jspace-` files to threefold-memory runs code that is not in this repository. This repository describes and tests only what is in it. ## CPU notes - Model: `openai-community/gpt2` (~124M, 12 layers). Fits comfortably on CPU. - **No GPU required.** float32 CPU only. - Lens: pretrained gpt2-small weights from Hugging Face `neuronpedia/jacobian-lens` (`gpt2-small/jlens/Salesforce-wikitext/gpt2_jacobian_lens.pt`), fetched by `scripts/fetch_lens.sh`. The 13 MB `.pt` file is **not** shipped in this repo (size / GitHub limits). - Prefer Python **3.12**. Install **CPU** torch first. ## Setup ```bash git clone https://github.com/LAWLESS1987/threefold cd threefold python3.12 -m venv .venv source .venv/bin/activate pip install --index-url https://download.pytorch.org/whl/cpu torch==2.14.1 pip install -r requirements.txt pip install "jlens @ git+https://github.com/anthropics/jacobian-lens" # Covenant (memory CLI) — do not vendor the whole tree into git history git clone https://github.com/LAWLESS1987/covenant vendor/covenant # or: export COVENANT_SRC=/path/to/covenant ./scripts/fetch_lens.sh # PYTHON=… to choose the interpreter (default python3) ./scripts/seed_demo.sh # MODE=apply (default): proves SUCCESS via demo_apply.py # MODE=triad ./scripts/seed_demo.sh # full three-leg fire with fixed TASK_ID ``` On Windows (Git Bash), the scripts also find a venv's `Scripts/python.exe`. Paths written inside heredocs are not translated for Windows Python, so give `fetch_lens.sh` a Windows-style `OUT=C:/…` there. Memories default to `./data/ai_memory_root` (gitignored). Override with `AI_MEMORY_ROOT`, the tombstone log with `TOMBSTONE_LOG`, and the run directory with `RUN_DIR`. ## How seed works `scripts/seed_demo.sh` uses fixed demo values: - `TASK_ID=20261007-020907-13423` - `PROMPT="Fact: The currency used in the country shaped like a boot is"` Default `MODE=apply` fetches the lens (if missing) and runs `demo_apply.py`, expecting the line: `SUCCESS: lens.apply returned real top-k tokens` `MODE=triad` additionally clones covenant if needed and runs `triad/fire_triad.sh`, expecting `FIRE_TRIAD_OK`. The sample entry in `tombstone/tombstone.md` is illustrative of that same `task_id` format only. It predates L-lens, so it carries the older `jlens_digest` note and no `llens_sha256`. ## Layout ``` threefold/ README.md requirements.txt jlens_snapshot.py # JLens capture: runs upstream apply, stores what it returned llens.py # L-lens: hashing, derived summaries, packaging, verify, show demo_apply.py tombstone/ append.sh tombstone.md # SAMPLE only triad/ fire_triad.sh # the three legs, then verify_triad.py on_speak.sh put_covenant_memory.sh put_memory_file.py # covenant MemoryStore.put with the body read from a file verify_triad.py # cross-leg verification, fails closed scripts/ fetch_lens.sh seed_demo.sh tests/ test_llens.py check_against_upstream.py e2e_triad.sh ``` ## License / upstream - **threefold** itself is licensed under the [Apache License 2.0](LICENSE) (Lawrence's choice, 2026-10-08). That is the same license as covenant and as the jlens dependency. - **Jacobian Lens** (the instrument): [anthropics/jacobian-lens](https://github.com/anthropics/jacobian-lens), © Anthropic PBC, Apache-2.0. It is installed as a dependency, not copied into this repository. Paper: [Verbalizable Representations Form a Global Workspace in Language Models](https://transformer-circuits.pub/2026/workspace/index.html). - Pretrained lens weights: [neuronpedia/jacobian-lens](https://huggingface.co/neuronpedia/jacobian-lens) - Subject model: [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) - Covenant memory CLI: [LAWLESS1987/covenant](https://github.com/LAWLESS1987/covenant) ## Speak-turn hook Every Grok Bot user-visible reply is a triad task. Run: ```bash ./triad/on_speak.sh "exact words spoken" "short-title" ``` This calls `fire_triad.sh` (all three legs required, then verified). ## Public memory root Live data: https://github.com/LAWLESS1987/threefold-memory ```bash git clone https://github.com/LAWLESS1987/threefold-memory export AI_MEMORY_ROOT=$PWD/threefold-memory ``` --- ## Part of one system — each piece usable on its own These repositories are independent instruments that also compose into a single apparatus for **building and honestly auditing trustworthy multi-intelligence systems**. Any one can be used standalone; together they form the stack: - **Governed core** — [covenant](https://huggingface.co/Lawless1987/covenant): a self-governing multi-agent system with an ethics gate inside the transaction path. - **Observability & provenance** — [threefold](https://huggingface.co/Lawless1987/threefold) (the L-lens layer over Jacobian Lens) and [threefold-memory](https://huggingface.co/Lawless1987/threefold-memory) (its persistent, provenance-tracked memory): every action measured and recorded under one task identity. - **Measurement instruments** — [contradiction-probe](https://huggingface.co/Lawless1987/contradiction-probe) and [flinch-observability](https://huggingface.co/Lawless1987/flinch-observability), with the session records [flinch-record](https://huggingface.co/Lawless1987/flinch-record) and [grokflinch](https://huggingface.co/Lawless1987/grokflinch): preregistered, blind-scored tests of AI behaviour that embody the covenant's rule — *investigate a signal, never convict on one*. - **Neuromorphic research** — [spiking-experiments](https://huggingface.co/Lawless1987/spiking-experiments): from-scratch spiking-network experiments with the pass/fail bar declared before each run. - **Ledger surface** — [Sentinel-Witness](https://huggingface.co/Lawless1987/Sentinel-Witness): the trading / ledger dashboard. **This repository's place in it:** the observability + provenance layer that measures and records what the system does.