SGJM 2026.6.5 — code/docs

.gitignore

@@ -0,0 +1,20 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+.env
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.DS_Store
+*.pt
+*.bin
+*.safetensors
+runs/
+checkpoints/
+wandb/
+logs/

AGENTS.md

@@ -0,0 +1,30 @@
+# AGENTS.md
+
+Guidance for autonomous coding agents (Codex, Claude Code, etc.) working in this repo.
+
+This project is sometimes run by an agent in fully autonomous mode (no interactive
+approval). Behave as if no human will review a prompt mid-run: prefer reversible
+steps, do not run destructive commands unless the task explicitly calls for it, and
+stop and report rather than guess when a step is genuinely ambiguous.
+
+## Commit & author identity — required
+
+**No AI tool may appear as a committer, co-author, or in commit metadata.**
+
+- All commits must be authored as `Adam Pippert <adam.pippert@gmail.com>`.
+- Do **not** add `Co-Authored-By:` trailers naming Claude, Codex, an AI, or a model.
+- Do **not** add generated-by / "🤖 Generated with ..." footers.
+- Do **not** set `user.name` or `user.email` to anything containing `claude`,
+  `codex`, `openai`, or `anthropic`.
+- If the repo's git `user.name`/`user.email` is unset or contains any of those
+  strings, **stop and report** — do not silently reconfigure it and do not commit.
+
+Adam Pippert remains the sole author of record for all work, regardless of which
+tool drafted it.
+
+See `CLAUDE.md` for the Claude-specific statement of the same policy.
+
+## Pushing
+
+`origin` points at GitHub. Do **not** push unless the task explicitly asks you to.
+When in doubt, commit locally and report what's ready to push.

BLOG.md

@@ -0,0 +1,242 @@
+# Speculative Decoding Meets JEPA: Training a 25M Graph Model on a MacBook in 27 Minutes
+
+*What happens when you teach a small transformer to guess its own future — and then verify whether those guesses are any good?*
+
+---
+
+## The Problem with Autoregressive Generation
+
+Standard language model generation is embarrassingly serial. You run the full model once to produce one token, feed that back in, run again for the next token, repeat. If you want 200 tokens from a 25-million-parameter model, you're doing 200 sequential forward passes, each attending over a growing context window.
+
+The compute cost scales quadratically with context length. The latency scales linearly with output length. Neither is what you want in production.
+
+Speculative decoding is the obvious escape hatch: use a cheaper model to draft multiple tokens at once, then verify with the large model. The standard recipe requires two separate models, a careful rejection-sampling protocol, and tuning the draft/verify handoff.
+
+I wanted to explore a tighter integration. What if the model learned to score its own speculative branches, in latent space, using a self-supervised signal? That question led to SGJM.
+
+---
+
+## What SGJM Actually Does
+
+SGJM — Speculative Graph JEPA Model — is a four-component architecture built around a single causal transformer backbone. Instead of calling an external draft model, SGJM grows a tree of speculative token branches in parallel and prunes them using a latent-space judge trained with the JEPA objective.
+
+The four components:
+
+**Backbone** (~22M params). A standard byte-level causal transformer (d=384, 10 layers, SwiGLU, RMS norm, tied embeddings). Processes the input context and produces both next-token logits and hidden states. This is the workhorse.
+
+**Drafter** (~2M params). Takes the parent hidden state, projects it to a smaller space (d=192), and uses *learnable position queries* to generate k token blocks of length `block_size` in a single forward pass. All k branches are produced in parallel. Each branch carries tokens, an endpoint latent, and a log-probability.
+
+**JEPA Judge** (~1M params). A two-layer feedforward that predicts what the backbone's hidden state *should* look like at the end of a draft block, trained with MSE against the actual future latent (stop-gradient applied to the target). Branches are scored by how close their endpoint latent is to the judge's prediction — not by token probability alone.
+
+**Verifier** (~0.5M params). A binary classifier on the concatenated parent and child hidden states. Trained with contrastive pairs: real future states are positive examples, rolled negatives are negative. A branch passes if its score exceeds a threshold.
+
+The total is ~25M parameters, a deliberate match for a same-budget 11-layer transformer baseline used as the comparison gate.
+
+The architecture fits on a napkin:
+
+```
+tokens → Backbone (22M) → hidden states → next-token logits
+                ↓
+    ┌───────────┼──────────────┐
+    ▼           ▼              ▼
+Drafter      JEPA Judge     Verifier
+(k branches) (score latent) (accept/reject)
+    └───────────┴──────────────┘
+                ↓
+          branch selection
+```
+
+---
+
+## Training in 27 Minutes on Apple Silicon
+
+Training uses four loss terms weighted by a single `TrainingConfig`:
+
+| Term | What it trains | Default weight |
+|------|---------------|----------------|
+| Token CE | Backbone language modeling | 1.0 |
+| Drafter CE | Draft token predictions | 0.5 |
+| JEPA MSE | Judge prediction vs true future latent | 0.25 |
+| Verifier BCE | Accept/reject contrastive pairs | 0.1 |
+
+The training runs entirely on MLX, Apple's native ML framework for Apple Silicon. On a MacBook Pro (arm64), 5000 steps on TinyShakespeare (1 MiB, byte-level, vocab=256) completes in 27.3 minutes.
+
+Training is straightforward:
+
+```bash
+python -m sgjm.training --size 25m --backend mlx
+```
+
+The loss curve tells a clean story. All four metrics converge:
+
+| Step | Total loss | Token NLL | Accept acc |
+|------|----------:|----------:|----------:|
+| 500 | 2.053 | 0.278 | 94.0% |
+| 1 000 | 0.472 | 0.097 | 98.9% |
+| 2 000 | 0.293 | 0.051 | 99.4% |
+| 3 500 | 0.199 | 0.030 | 99.8% |
+| **4 500** | **0.179** | **0.025** | **99.8%** |
+
+Accept accuracy hits 99.8% by step 1000. Token NLL reaches 0.025 by step 4500. The model is learning to draft high-quality branches and verify them correctly.
+
+---
+
+## The Eval Gate: Five Conditions, One Shot
+
+After training, the model faces a five-condition gate against the same-budget baseline:
+
+| Condition | Threshold | SGJM | Result |
+|-----------|----------|------|--------|
+| NLL delta vs baseline | ≤ 0.05 nats | +0.0015 nats | ✅ |
+| Branch acceptance rate | ≥ 50% | **100%** | ✅ |
+| JEPA top-1 above chance | ≥ +5 pp | **+88.5 pp** | ✅ |
+| Merge precision advantage | ≥ 1.5× | **10 607×** | ✅ |
+| Compute per accepted token | ≥ 1.0× baseline | **13.92×** | ✅ |
+
+All five pass. The numbers are striking:
+
+- **100% branch acceptance** means the trained verifier never rejects a draft branch. At 5000 steps, the drafter has learned to generate branches that always look like plausible continuations to the verifier.
+
+- **10 607× merge precision advantage** is the standout result. The model learned to cluster semantically similar branches via SimHash — branches that would merge in the speculative graph have dramatically lower Jensen-Shannon divergence than random pairs (0.0001 vs 0.69 nats). This means the merge strategy is valid: branches bucketed together actually share similar next-token distributions.
+
+- **13.92× compute advantage** means the baseline spends 13.92× more FLOPs per generated token than SGJM spends per accepted token.
+
+The NLL delta is only +0.0015 nats — functionally zero. SGJM achieves the same language modeling quality as a baseline that uses the whole 25M parameter budget for a single transformer, but it routes those parameters through a four-component architecture that generates and filters speculative branches.
+
+---
+
+## Ablation: What Each Component Actually Does
+
+To understand which components carry weight, I ran five ablation variants, each trained from scratch for 1000 steps:
+
+| Variant | Token NLL | Accept Rate | JEPA top-1 | Merge Adv. |
+|---------|----------:|------------:|-----------:|-----------:|
+| `sgjm_full` | 0.1011 | 63.3% | 97.5% | 1.19× |
+| `sgjm_no_jepa` | 0.0992 | **2.7%** | 11.5% ≈ chance | 1.11× |
+| `sgjm_no_verifier` | 0.1009 | **21.3%** | 96.7% | 1.18× |
+| `sgjm_no_drafter` | 0.0916 | **100%** | 95.5% | **0.996×** |
+| `sgjm_token_only` | 0.0890 | 18.6% | 11.4% ≈ chance | 1.00× |
+
+The findings are sharp:
+
+**JEPA is load-bearing.** Removing it (setting `jepa_weight=0`) collapses branch acceptance from 63% to 2.7% — essentially nothing is accepted. The compute advantage goes negative (0.37×). Without JEPA, the judge produces random scores and the verifier has no signal to train on. Every other component depends on JEPA working.
+
+**Verifier gates quality.** Without it, acceptance drops to 21%. The model still drafts, but it accepts wrong branches at much higher rates — drafts with high log-prob but bad future latent alignment get committed. The verifier acts as a second filter that catches what the judge misses.
+
+**Drafter loss enables merge.** Here's the counter-intuitive result: removing the drafter loss yields **100% acceptance**. How? Because the backbone hidden states are still good, and the verifier still scores based on backbone signals. But the merge precision collapses to 0.996× (random). Without drafter training, the endpoint latents that drive SimHash bucketing are not semantically organized — branches are accepted but they're not semantically similar to each other.
+
+**Token-only is equivalent to dead weight.** Without aux losses, JEPA accuracy equals chance (11.4%) and all auxiliary metrics are dead. The SGJM architecture adds overhead without any speculative benefit.
+
+---
+
+## Hyperparameter Sweeps
+
+With the ablations showing which components matter, I ran three hyperparameter sweeps:
+
+### How much JEPA weight?
+
+The `jepa_weight` controls the JEPA MSE loss relative to token CE.
+
+| `jepa_weight` | Token NLL | Accept Rate | Merge Adv. |
+|--------------|----------:|------------:|-----------:|
+| 0.0 | 0.0992 | 2.7% | 1.11× |
+| **0.05** | **0.0989** | **64.2%** | **1.20×** |
+| 0.25 (default) | 0.1011 | 63.3% | 1.19× |
+| 1.0 | 0.1061 | 81.4% | 1.00× |
+| 4.0 | 0.1569 | 100% | 1.00× |
+
+The elbow is at `jepa_weight=0.05`. Even a tiny JEPA signal activates all four components: acceptance jumps from 2.7% to 64.2%, JEPA top-1 jumps from chance to 97%. Above 1.0, acceptance keeps climbing but token NLL regresses and merge precision saturates. The default 0.25 is a safe operating point 4× above the elbow.
+
+### Block size: how many tokens to draft at once?
+
+| `block_size` | Token NLL | Accept Rate | Merge Adv. |
+|-------------|----------:|------------:|-----------:|
+| 2 | **0.0963** | 69.0% | **1.92×** |
+| **4 (default)** | 0.1011 | 63.3% | 1.19× |
+| 8 | 0.1007 | **90.8%** | 1.00× |
+
+Smaller blocks are easier to predict well, so `block_size=2` yields the best merge precision (1.92× vs 1.19×) and lowest NLL. Larger blocks increase acceptance — more tokens means the verifier sees more information — but merge precision degrades. Default `block_size=4` balances tokens-per-step with prediction quality.
+
+### Merge radius: how similar must branches be to merge?
+
+The merge radius controls the SimHash Hamming distance threshold for bucketing branches together.
+
+| `merge_radius_bits` | Merge JS | Random JS | Merge Adv. |
+|--------------------|--------:|----------:|-----------:|
+| 2 | NaN | 0.689 | 1.00× | (radius too tight — no pairs qualify) |
+| 4 | NaN | 0.689 | 1.00× | (radius too tight) |
+| **6 (default)** | **0.578** | **0.689** | **1.19×** |
+| 8 | 0.623 | 0.689 | 1.11× |
+| 12 | 0.623 | 0.689 | 1.11× |
+
+`merge_radius_bits=6` is the minimum threshold where pairs qualify and their JS divergence (0.578) is meaningfully lower than random pairs (0.689). Below 6, the radius is so tight that no pairs pass the SimHash test. Above 6, more diverse pairs are admitted, diluting the advantage.
+
+---
+
+## Scaling to 100M
+
+I also ran a ~93M parameter variant (d_model=768, 9 backbone layers) for 5000 steps. Duration: 55.4 minutes on the same MacBook.
+
+| | 25M | 100M |
+|--|-----|------|
+| Params | ~25M | ~93M |
+| Training time | 27.3 min | 55.4 min |
+| Best eval token NLL | 0.0254 | 0.0241 |
+| Best eval total loss | 0.1790 | 0.1666 |
+
++272% parameters, +103% training time, −6.9% eval loss. The scaling return is favorable: doubling the training budget gives modest but consistent improvement with no architecture changes.
+
+---
+
+## Generation Benchmark: Honest Numbers
+
+The benchmark compares SGJM and autoregressive generation head-to-head, both generating 200 tokens from a 64-token prompt:
+
+| | SGJM | AR Baseline |
+|--|------|-------------|
+| Steps | 50 harness steps × 4 tokens | 200 AR steps × 1 token |
+| Model fwd passes | 100 (50 backbone + 50 drafter) | 200 backbone |
+| Tokens / sec | 151.7 | 153.0 |
+| Speedup | **0.99×** | — |
+
+Essentially identical throughput in this minimal Python implementation. Why doesn't the 13.92× FLOPs advantage translate to wall-clock speedup?
+
+Because this is a Python harness, not a production system. Each model forward pass is a separate kernel launch. The drafter and judge add per-call overhead. The harness Python logic (branch lifecycle, SimHash, policy ranking) adds CPU overhead that dwarfs the GPU/NPU compute difference on small batches.
+
+The 13.92× FLOPs advantage is real — it measures the ratio of compute cost per generated token — but realizing it in wall-clock time requires:
+1. KV-cache to avoid re-encoding full context on every step
+2. Batched parallel branch evaluation (all k branches in one kernel call)
+3. A compiled or Rust harness to eliminate Python overhead
+4. Larger models and longer contexts where the O(T²) attention scaling makes the advantage matter
+
+This is the gap between a research prototype and a deployed system.
+
+---
+
+## Lessons Learned
+
+**JEPA is not optional.** The most important finding from the ablations: JEPA is structurally required. It's not a regularizer you can tune down — without it, the entire speculative mechanism collapses. This surprised me. I expected the verifier to be the load-bearing component, but the judge's latent-space signal is what the verifier learns to refine.
+
+**Merge precision is a slow signal.** At 1000 steps, the merge advantage is 1.19×. At 5000 steps, it's 10 607×. The semantic clustering of draft branches in SimHash space takes the full training run to emerge. You can't diagnose whether merge is working from a short run.
+
+**Drafter loss and merge precision are linked.** This connection was non-obvious: if you remove the drafter loss, acceptance goes to 100% (because the backbone still guides the drafter) but merge precision collapses. The drafter needs its own loss to learn to produce endpoint latents that are semantically organized, not just to produce fluent tokens.
+
+**Small models, big compute ratios.** A 25M model on a MacBook, trained in under 30 minutes, achieves a 13.92× compute advantage over a same-parameter baseline. The ratio grows with scale. This is tractable research that doesn't require a GPU cluster.
+
+---
+
+## What's Next
+
+The immediate extensions are clear:
+
+1. **KV-cache integration** — the single biggest gap between prototype and real speedup
+2. **Larger corpora** — TinyShakespeare is 1 MiB; the model memorizes it quickly. Scaling to web text would stress the merge and acceptance mechanisms properly
+3. **Top-p/top-k sampling** in the drafter — currently greedy from drafter logits; temperature sampling would diversify the branch set
+4. **Calibration of the verifier threshold** — the gate run uses a global threshold; a per-context adaptive threshold could improve precision
+5. **Scaling to 1B** — the 100M result suggests favorable scaling; testing on a machine with real GPU memory would confirm whether the FLOPs advantage survives at production scale
+
+The code is all in this repo. Each component — backbone, drafter, judge, verifier — is a separate module with a clean Protocol interface. Swapping any component for a better implementation is straightforward.
+
+---
+
+*Training logs, eval reports, and ablation results are in `results/`. Reproduce with `python -m sgjm.training --size 25m --backend mlx`.*

BLOG_1B.md

@@ -0,0 +1,426 @@
+# SGJM at 1B Parameters: Scaling Speculative Graph JEPA Across Two Hardware Platforms Simultaneously
+
+*Does the architecture hold? Does JEPA remain load-bearing at this scale? Does the dual-backend approach actually work in practice?*
+
+---
+
+The previous posts in this series trained SGJM at 25M, 100M, and 250M parameters, always on a single machine, always on a single backend. This post documents the first time I pushed to 1B parameters and, by necessity, ran the experiment on two hardware platforms at once: a Mac Studio M1 Ultra using MLX, and a Framework Desktop ("Hyde") running an AMD Strix Halo under ROCm. Both started at the same time. Both ran the same code. Both checkpoint to the same format. This post explains how that works, what the architecture looks like at this scale, and what the initial numbers say.
+
+---
+
+## The Architecture, Explained from First Principles
+
+SGJM is a four-component system. The headline claim is that it internalizes the draft-verify loop of speculative decoding into a single jointly-trained model. To understand what that means and why each component is necessary, it helps to start from the problem standard speculative decoding leaves unsolved.
+
+Standard speculative decoding uses two models: a small "draft" model that generates k candidate tokens quickly, and a large "target" model that verifies them in parallel. The verification step uses rejection sampling: for each draft token, you check whether the target model would assign a high enough probability. The catch is that the draft and target models are trained independently with no shared objective. They share a vocabulary but not a representation space. The result is that branch quality is measured purely by next-token probability — a signal that is necessary but not sufficient for semantic coherence.
+
+SGJM replaces this with a single system where the backbone, drafter, judge, and verifier are trained together against a combined loss. Crucially, the JEPA judge introduces a second scoring signal — latent-space proximity — that operates orthogonally to token probability. A draft branch that produces high-probability tokens but lands in an unexpected region of representation space gets penalized. A branch that lands close to where the model predicts the context should end up gets rewarded.
+
+Here is how the four components divide the work.
+
+### Backbone
+
+The backbone is a causal transformer with byte-level tokenization (vocab_size=256, tied input/output embeddings). It uses SwiGLU activations, RMSNorm pre-normalization, and learned absolute position embeddings. At 1B scale the configuration is:
+
+```
+d_model      = 2048
+n_layers     = 20
+n_heads      = 16
+head_dim     = 128
+d_ff         = 8192  (SwiGLU: gate + up projections, then down)
+max_seq_len  = 4096
+vocab_size   = 256
+```
+
+The backbone runs a standard causal forward pass and returns two things: the full sequence of hidden states `h` (shape `[B, T, 2048]`) and the next-token logit matrix (shape `[B, T, 256]`). Both are used downstream. The backbone's hidden states are the currency the other three components trade in.
+
+Why byte-level? Because it removes tokenizer design from the variable space. A 256-token vocabulary is fully deterministic: every byte is exactly one token. There are no subword boundary artifacts, no vocabulary mismatch between machines, no OOV problem. The tradeoff is that sequences are longer for the same text, which is why block_size matters — you want the drafter to amortize over at least a few tokens per speculative step.
+
+Why tied embeddings? The input embedding and lm_head weight matrices are shared. This has two effects: it reduces parameter count (the embedding table at d_model=2048 is 2048 * 256 = ~500K parameters, not large but not zero), and it creates a mild representation alignment between the embedding space and the output space, which benefits the JEPA judge (see below).
+
+### Drafter
+
+The drafter takes the backbone's hidden state at position `t` and produces `block_size` draft tokens in a single forward pass. At 1B scale, block_size=2 (versus block_size=4 at smaller variants). The reason for this is discussed in the hyperparameter section below.
+
+The drafter's internal architecture:
+
+```
+d_model      = 768
+n_layers     = 3
+n_heads      = 12
+d_ff         = 3072
+```
+
+The forward pass works through learnable position queries. For each position in the sequence, the drafter:
+
+1. Projects the parent hidden state from d=2048 to d=768 via a linear layer.
+2. Adds `block_size` learnable query vectors (shape `[block_size, 768]`), one per draft position. These are not attention keys — they are additive offsets that specialize each position in the draft block.
+3. Runs the result through 3 transformer blocks.
+4. Produces two outputs: draft token logits (for each position in the block, a distribution over 256 bytes) and draft latents (a projection back to d=2048, representing where the drafter thinks the backbone hidden state should be after accepting this branch).
+
+The critical design decision is that all branches are produced in one forward pass, not k separate passes. This is what makes the drafter tractable at inference time. The branching factor k comes from sampling different sequences from the draft logits, not from k separate calls to the drafter.
+
+The `latent_out` projection in the drafter (a linear from d=768 to d=2048) produces what I call the "endpoint latent" — the drafter's claim about where the backbone hidden state will be after the draft block is accepted. This latent is consumed by the JEPA judge during training.
+
+### JEPA Judge
+
+The JEPA judge is the structurally unusual component. It is a two-layer feedforward network with hidden size 4096:
+
+```python
+fc1: Linear(2048 -> 4096)
+activation: GELU
+fc2: Linear(4096 -> 2048)
+```
+
+It takes the parent hidden state as input and predicts what the backbone hidden state *should* look like `block_size` positions later. During training, the actual future hidden state `h_{t+block_size}` is computed from the backbone forward pass, and the judge's prediction is trained to match it via MSE with a stop-gradient on the target.
+
+The stop-gradient is not optional. Without it, the loss gradient flows into the backbone and incentivizes the backbone to make the future state easy to predict — which is a completely different objective from language modeling. The stop-gradient ensures the judge trains to predict the backbone's natural future states, not the other way around.
+
+During evaluation and inference, the judge scores each draft branch by computing the MSE between the branch's endpoint latent (from the drafter's `latent_out`) and the judge's prediction of where the backbone should be. Low MSE = semantically consistent branch. High MSE = branch has drifted away from where the model expects the context to go.
+
+This is the JEPA signal: Joint Embedding Predictive Architecture, applied to the speculative decoding problem. The judge does not look at tokens at all. It operates entirely in the backbone's representation space. A branch can produce perfectly fluent tokens and still score poorly if its endpoint latent is far from the judge's prediction.
+
+Why does this matter? The ablation results at 25M scale are unambiguous. When the JEPA loss is removed (`loss.jepa=0.0`), branch acceptance rate collapses from 63% to 2.7%. The judge is not providing a redundant signal — it is providing the primary signal that the verifier refines.
+
+### Verifier
+
+The verifier is a binary classifier on concatenated parent and child hidden states:
+
+```
+input:  concat([parent_h, child_h])  -- shape [B, T, 4096]
+fc1:    Linear(4096 -> 2048)
+act:    GELU
+fc2:    Linear(2048 -> 1)
+output: logit (positive = accept)
+```
+
+It is trained with contrastive pairs. Within each batch, the true future hidden states are the positive examples. The negatives are constructed by rolling the batch along the batch dimension (`torch.roll(shifts=1, dims=0)` in PyTorch, `mx.concatenate([h[-1:], h[:-1]])` in MLX) — this gives negatives that have the same distributional character as real hidden states but are misaligned with the parent contexts. The result is a discriminator that learns to separate genuine continuations from plausible-but-wrong ones.
+
+The verifier's BCE loss is weighted 0.1 in the combined loss, the lowest weight of any term. This reflects its role: it refines the judge's score rather than driving it. In the ablation where only the verifier is present (JEPA removed), acceptance collapses to 2.7%. The verifier without a trained judge has nothing meaningful to discriminate.
+
+### Why All Four Components Are Required
+
+The ablation results at 25M make the dependency structure explicit:
+
+| Configuration | Accept rate | JEPA top-1 | Merge prec. | Notes |
+|---|---|---|---|---|
+| Full (token + drafter + JEPA + verifier) | 63.3% | 97.5% | 1.19x | All signals active |
+| No JEPA | 2.7% | ~chance (11.5%) | 1.11x | Verifier has nothing to learn from |
+| No drafter loss | 100% | 95.5% | 1.00x | Accept stays high but merge collapses |
+| No verifier | 21.3% | 96.7% | 1.18x | Accept halved; judge alone insufficient |
+| Token only | 18.6% | ~chance | 1.00x | SGJM with dead weight |
+
+Reading down the table tells a story. JEPA is structurally upstream: remove it and the verifier fails. The drafter loss is required for branch organization: without it, the drafter still produces high-probability tokens (so the verifier accepts them), but the endpoint latents are not organized — SimHash-based merge precision drops to 1.00x (meaning draft branches that appear similar by locality-sensitive hash are no more distributionally similar than random pairs). The verifier provides a second filter after the judge that materially improves acceptance precision.
+
+At the fully-trained 25M eval gate (5000 steps, TinyShakespeare corpus), the model passes all five conditions:
+
+| Condition | Threshold | Result |
+|---|---|---|
+| NLL delta vs baseline | <= 0.05 nats | +0.0015 nats |
+| Branch acceptance rate | >= 50% | 99.99% |
+| JEPA top-1 above chance | >= +5 pp | +88.5 pp above 11.1% chance |
+| Merge precision advantage | >= 1.5x | 10,607x |
+| Compute per accepted token | >= 1.0x baseline | 13.9x |
+
+That merge precision number (10,607x) deserves a brief explanation. It measures the Jensen-Shannon divergence between next-token distributions at positions that SimHash clusters together (merged branches) versus random pairs. At 5000 steps, the JS divergence within merged clusters is 0.000065 nats; across random pairs it is 0.690 nats. The model has learned, entirely from the JEPA and drafter losses, to organize draft branches so that semantically similar continuations cluster together in SimHash space. This is the mechanism that would make a production speculative graph efficient — branches that converge to the same meaning can be deduplicated cheaply.
+
+---
+
+## Configuration at 1B Scale
+
+The 1B configuration is defined in `TrainingConfig.sgjm_1b()`:
+
+```python
+ModelConfig(
+    d_model=2048,
+    n_layers=20,
+    n_heads=16,
+    d_ff=8192,
+    drafter_layers=3,
+    drafter_d_model=768,
+    drafter_heads=12,
+    drafter_d_ff=3072,
+    judge_hidden=4096,
+    verifier_hidden=2048,
+    block_size=2,       # shorter blocks vs 4 at smaller scales
+    max_seq_len=4096,
+)
+
+OptimConfig(
+    lr=6e-5,
+    betas=(0.9, 0.95),
+    weight_decay=0.1,
+    warmup_steps=5_000,
+    max_steps=50_000,
+    grad_clip=1.0,
+    batch_size=1,
+    seq_len=2048,
+)
+
+corpus_bytes = 256 << 20   # 256 MiB Python extended corpus
+```
+
+### Why block_size=2?
+
+The block_size sweep at 25M scale showed that shorter blocks have better judge signal quality and better merge precision, while longer blocks have higher raw compute advantage:
+
+| block_size | Accept rate | JEPA top-1 | Merge prec. | Compute advantage | Gate |
+|---|---|---|---|---|---|
+| 2 | 69.0% | 99.1% | 1.92x | 4.8x | Pass |
+| 4 | 63.3% | 97.5% | 1.19x | 8.8x | Fail (merge < 1.5x) |
+| 8 | 90.8% | 96.1% | 1.00x | 25.3x | Fail (merge = 1.0x) |
+
+At block_size=8, the drafter produces high-acceptance branches, but the merge precision advantage collapses to 1.0x. The drafter's endpoint latents are not being organized by the JEPA signal over 8-token windows — the signal is too diffuse across the block. At block_size=2, merge precision is 1.92x and the gate passes.
+
+At 1B scale, the backbone is substantially more powerful than at 25M. A stronger backbone produces higher-quality hidden states, which gives the judge a richer signal even over short windows. This is the argument for using block_size=2 here: the judge will have more to work with per position. If the judge signal degrades at 1B (which would be a surprising result), we can revisit.
+
+### Why batch_size=1?
+
+The 1B backbone requires roughly 4 bytes * 2048 (d_model) * 20 (layers) * 2048 (seq_len) for activations per batch element, plus the drafter, judge, and verifier activations. At batch_size=1 with seq_len=2048, the memory footprint is manageable on both hardware platforms. At batch_size=4 (used at 250M) the activation memory would require gradient checkpointing, which is not yet implemented.
+
+The effective batch in terms of tokens is still 2048, which is the same as the 100M configuration's batch_size=4 * seq_len=512. The gradient noise is higher, but the cosine LR schedule with 5000 warmup steps compensates.
+
+### Corpus
+
+The 256 MiB Python extended corpus is built from CPython's standard library plus common site-packages. It is byte-level (no tokenization step), so the model trains directly on the raw UTF-8 byte stream. This corpus was introduced at 250M scale; the 1B run uses the same source with a 256 MiB slice rather than the 32 MiB used at 250M.
+
+The Python corpus is harder than TinyShakespeare (1 MiB of English prose) in multiple ways: longer average sequence length before a semantic unit completes, mixed identifier/keyword/symbol vocabulary, and significant whitespace structure that the model must reproduce accurately for syntactically valid completions. The 25M and 100M results on TinyShakespeare (token_loss ~0.025) are not comparable to the 250M and 1B results on the Python corpus; the harder corpus raises the expected convergence loss.
+
+---
+
+## Scaling History
+
+For reference, here is the complete scaling table across all variants trained to date. Note that the 25M and 100M runs used TinyShakespeare (1 MiB); 250M and 1B use the Python extended corpus (32 MiB and 256 MiB respectively). Hyde's higher token_loss at 25M relative to the MacBook results is corpus difference, not a regression.
+
+| Model | Machine | Backend | Steps | Time | Steps/sec | Token loss | Accept acc |
+|---|---|---|---|---|---|---|---|
+| 25M | apippert-mac (M-series) | MLX | 5,000 | 27.3 min | ~3.0 | 0.025 | 99.8% |
+| 100M | apippert-mac (M-series) | MLX | 5,000 | 55.4 min | ~1.5 | 0.024 | ~99% |
+| 250M | apippert-mac (M-series) | MLX | 10,000 | — | — | — | — |
+| 25M | Hyde (Strix Halo) | ROCm | 5,000 | 20.9 min | 4.0 | 0.152 | 98.0% |
+| 250M | Hyde (Strix Halo) | ROCm | 10,000 | 68.8 min | 2.4 | 0.584 | 99.6% |
+| **1B** | **Mac Studio M1 Ultra** | **MLX** | **5,750 / 50k** | **4.6h (of ~54h)** | **0.35** | **1.97** | **n/a †** |
+| **1B** | **Hyde (Strix Halo)** | **ROCm** | **8,225 / 50k** | **4.6h (of ~27h)** | **0.50** | **1.21** | **n/a †** |
+
+The 250M Hyde token_loss (0.584) after 10,000 steps on 256 MiB Python is not a strong comparison to the 25M MacBook token_loss (0.025) after 5,000 steps on 1 MiB Shakespeare. They are different tasks. The relevant comparison at each scale is the delta between SGJM and the same-budget baseline on the same corpus, which the eval gate measures.
+
+† *Accept acc is not meaningful at 1B with batch_size=1; see the Verifier Collapse section below.*
+
+---
+
+## Live: 1B Training — 4.6 Hours In
+
+Both machines started the 1B run simultaneously and are running without issue. After 4.6 hours of wall time, here is where each stands.
+
+**Throughput (measured steady-state):**
+
+| Machine | Backend | Steps/sec | s/step | ETA (50k steps) |
+|---|---|---|---|---|
+| Hyde (Strix Halo) | ROCm / bf16 AMP | **0.50** | 2.00s | **~27 hours** |
+| Mac Studio M1 Ultra | MLX | **0.35** | 2.85s | **~40 hours** |
+
+Hyde runs ~1.4× faster than the Mac Studio at this scale. Both are slower than the early estimate based on the first 75 steps (which included MLX JIT compilation overhead). The actual steady-state on Mac Studio settled at 2.85s/step rather than 3.92s — MLX's lazy evaluation amortizes compilation cost after the first few hundred steps.
+
+### Loss Trajectories
+
+**Mac Studio M1 Ultra (MLX)** — step 5,750 / 50,000:
+
+| Step | Token loss | JEPA loss | Verifier loss |
+|------|-----------|-----------|---------------|
+| 0 | 6.284 | 1.192 | 0.702 |
+| 500 | 3.027 | 0.197 | 0.695 |
+| 1,000 | 2.004 | 0.072 | 0.693 |
+| 3,000 | 2.451 | 0.057 | 0.693 |
+| **5,750** | **1.973** | **0.059** | **0.693** |
+
+**Hyde, Framework Desktop (ROCm)** — step 8,225 / 50,000:
+
+| Step | Token loss | JEPA loss | Verifier loss |
+|------|-----------|-----------|---------------|
+| 0 | 5.936 | 1.467 | 0.737 |
+| 500 | 2.796 | 0.279 | 0.698 |
+| 1,000 | 2.331 | 0.130 | 0.695 |
+| 3,000 | 2.211 | 0.134 | 0.694 |
+| 5,000 | 1.819 | 0.089 | 0.694 |
+| **8,225** | **1.210** | **0.281** | **0.693** |
+
+Both token CE trajectories are healthy. Hyde's token loss (1.21 at step 8,225) is ahead of Mac Studio (1.97 at step 5,750), but Hyde has processed ~43% more steps in the same wall time. The JEPA loss on Mac Studio has stabilized near 0.06; on Hyde it has risen slightly to ~0.23–0.28 in later steps and shows more variance. This backend-specific JEPA behaviour is worth monitoring — if it diverges rather than stabilizing it would suggest a numerical precision difference in the ROCm bf16 MSE path.
+
+### The Verifier Collapse at batch_size=1
+
+There is an architectural issue that emerged at this scale and did not appear at 25M, 100M, or 250M: **the verifier is stuck at exactly loss=0.693 (ln 2) with 50% acceptance rate**, and does not improve across 8,000+ steps on either machine.
+
+The cause is specific to batch_size=1. The verifier is trained with contrastive pairs: positive examples are real future hidden states; negatives are constructed by rolling the batch tensor along the batch dimension. At batch_size=4 (used for all smaller models), rolling gives four genuinely different negative examples. At batch_size=1, rolling a tensor of shape `[1, T, D]` along the batch dimension returns the identical tensor. The positive and negative are the same example.
+
+The resulting gradient is exactly zero: BCE(σ(score), 1) pushes score upward; BCE(σ(score), 0) pushes it downward by the same magnitude. The verifier receives contradictory signals of equal strength and cannot move. Its loss stabilizes at ln(2) ≈ 0.693, the maximum entropy state.
+
+This is a bug in the current 1B configuration, not a property of the architecture. The fix: roll along the sequence dimension instead of the batch dimension, giving T position-based negatives within the single example. The current run continues because token CE, drafter CE, and JEPA converge correctly — the verifier contributes no gradient and no acceptance signal for this run. The accept_acc metric stays at 50% for the duration.
+
+---
+
+## Platform Comparison: MLX vs ROCm
+
+Running the same architecture on two very different hardware platforms in parallel is useful not just for redundancy but because the platforms make different tradeoffs visible.
+
+### Installation
+
+**Mac Studio / MLX path:**
+
+```bash
+# Clone the repo, create a venv with uv
+uv venv .venv --python 3.12
+source .venv/bin/activate
+pip install mlx numpy
+pip install -e '.[mlx,dev]'
+
+# Run smoke test
+python -m sgjm.training --size smoke --backend mlx
+```
+
+Three commands. No system packages. MLX is a pure-Python install that ships its own Metal compute kernels. The only constraint is Darwin arm64 — the framework will not install on Intel Macs or Linux.
+
+A `setup_remote.sh` script in the repo automates the full bootstrap (Homebrew, Miniforge, conda env, pip install, smoke test) for deploying to a fresh Mac Studio:
+
+```bash
+bash scripts/setup_remote.sh https://github.com/AdamPippert/SGJM.git
+```
+
+**Hyde / ROCm path:**
+
+```bash
+# System package pull (Arch Linux) — this is the heavy step
+sudo pacman -S python-pytorch-opt-rocm
+# Pulls: rocm-hip-sdk, hipblaslt, miopen-hip, aotriton, and pytorch itself
+# ~4GB download
+
+# Project install into a venv that can see the system pytorch
+python3 -m venv --system-site-packages .venv
+source .venv/bin/activate
+pip install -e '.[rocm,dev]'  # installs only numpy and the sgjm package
+
+# Required env flag for Strix Halo's attention kernels
+export TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1
+
+# Run
+python -m sgjm.training --size 1b --backend rocm
+```
+
+The ROCm path is more involved, primarily because the PyTorch ROCm wheel is distributed through the Arch Linux package manager rather than PyPI. The `[rocm]` extra in `pyproject.toml` deliberately omits torch (it lists only numpy), because torch ROCm must come from the system package or from a specific PyTorch index URL.
+
+The critical environment flag `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` enables the AOTriton-based Flash Attention and memory-efficient attention kernels for Strix Halo GPUs. Without it, `F.scaled_dot_product_attention` silently falls back to the naive O(n^2) attention implementation, which is both slower and consumes more memory. This flag is required for any GPU in the Strix Halo family. It is not required for RDNA3 or earlier ROCm targets.
+
+### Memory Architecture
+
+The two platforms handle the 1.4GB+ model in very different ways.
+
+**M1 Ultra unified memory**: Apple's M1 Ultra has a single pool of RAM shared by the CPU, GPU, and Neural Engine. The Mac Studio variant ships with up to 128GB. The GPU has access to the full pool at approximately 800 GB/s bandwidth (per Apple's published numbers). There is no discrete VRAM; everything is unified. This means a 1B model with fp32 weights (~5.7GB) plus activations, optimizer state (AdamW stores two moment tensors per parameter, so ~11.4GB additional), plus the corpus in memory is comfortably within a 64GB configuration. The absence of a CPU-to-GPU memory transfer bottleneck is significant: there is no PCIe lane between the compute fabric and the model weights. Every access to model weights is the same latency as any other memory access.
+
+**Hyde Strix Halo GTT**: The Framework Desktop's AMD Strix Halo (Radeon 8060S) has 512MB of dedicated VRAM and 62GB of GTT (Graphics Translation Table) memory — system RAM that the GPU can address directly. The total addressable GPU memory is 62.5GB, but with significantly lower bandwidth than the M1 Ultra's unified architecture. GTT accesses go through the system interconnect rather than the high-bandwidth unified memory fabric. For inference latency, this matters; for training throughput where the bottleneck is compute rather than memory bandwidth, it matters less. The 1B model fits in GTT with room for optimizer state, and the ROCm driver handles the page mapping transparently.
+
+For a 1B model, both platforms have enough addressable memory. The bandwidth difference will show up in steps/second rather than in convergence quality.
+
+### Software Maturity and Friction
+
+**MLX**: MLX is purpose-built for Apple Silicon. The `mlx_backend/trainer.py` is clean: `mx.array`, `mx.eval()`, `tree_flatten`, `mx.save_safetensors()`. There is no AMP boilerplate because MLX operates in bf16 natively on M-series chips — the mixed-precision question does not arise. The `nn.value_and_grad()` pattern computes loss and gradients in a single call with no explicit backward pass. The main friction at 1B is that MLX's lazy evaluation model means that the first step incurs JIT compilation cost (~4s in our step-0 measurement). Subsequent steps should be faster.
+
+**ROCm / PyTorch**: The torch backend is standard PyTorch, which means the full AMP machinery is available. The `_amp_dtype` function in `torch_backend/trainer.py` selects bf16 for ROCm backends automatically. The GradScaler is instantiated but only activated for fp16 (not bf16, since bf16 does not underflow in the same way that requires loss scaling). The actual training loop is identical in structure to the MLX loop. The substantive difference is the `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` flag — this is genuine friction that a developer would hit without documentation. Strix Halo is a new enough GPU that its attention kernel support was not in the stable ROCm path at the time of writing.
+
+### The Backend Abstraction
+
+The reason this dual-platform experiment was a one-day effort rather a multi-week port is `src/sgjm/training/backends.py`. The `detect_backend()` function does the following:
+
+```python
+def detect_backend() -> ResolvedBackend:
+    if platform.system() == "Darwin" and platform.machine() == "arm64" and _has_module("mlx.core"):
+        return "mlx"
+    if _torch_cuda_available():
+        return "rocm" if _torch_is_rocm() else "cuda"
+    return "cpu"
+```
+
+On Darwin arm64 with MLX installed, the resolved backend is `mlx`. On Hyde with the ROCm pytorch package, `torch.version.hip` is set, so `_torch_is_rocm()` returns True and the resolved backend is `rocm`. The `__main__.py` entry point dispatches to the appropriate trainer with no further configuration:
+
+```python
+if is_torch_backend(backend):
+    from sgjm.training.torch_backend.trainer import train as torch_train
+    torch_train(cfg, backend)
+elif is_mlx_backend(backend):
+    from sgjm.training.mlx_backend.trainer import train as mlx_train
+    mlx_train(cfg, backend)
+```
+
+Neither trainer imports the other. The MLX trainer uses `mlx.core`, `mlx.nn`, and `mlx.optimizers`. The torch trainer uses `torch`, `torch.nn`, and `torch.optim`. They share only the `TrainingConfig`, `ByteDataset`, and the loss computation structure (which is re-implemented in each backend's idiom).
+
+The loss implementations converge to the same computation:
+
+```
+# MLX (losses.py):
+jepa_loss = 0.5 * (
+    nn.losses.mse_loss(jepa_pred, future_hidden, reduction="mean")
+    + nn.losses.mse_loss(drafter_endpoint, future_hidden, reduction="mean")
+)
+
+# PyTorch (losses.py):
+jepa_loss = 0.5 * (
+    F.mse_loss(jepa_pred, future_hidden)
+    + F.mse_loss(drafter_endpoint, future_hidden)
+)
+```
+
+The PyTorch `future_hidden` uses `.detach()` for the stop-gradient; the MLX version uses `mx.stop_gradient()`. Same semantics, different idiom. The checkpoint formats differ (`safetensors` on MLX, `.pt` on torch), which means checkpoints from one backend cannot be loaded directly on the other — a limitation that would matter if we were trying to resume on a different machine mid-run, but which does not affect this experiment where each machine runs independently.
+
+---
+
+## What the Initial Numbers Tell Us
+
+At step 0, the model is randomly initialized. Both machines show total loss near 9.3–9.4, accept_acc near 50%, and JEPA loss near 0.4. These are the expected values for an untrained model.
+
+The JEPA loss at step 0 is interesting: it is not near zero. An untrained backbone produces hidden states that are essentially random (drawn from the N(0, 0.02) initialization used in `_init_weights`), and an untrained judge produces predictions that are also random. The MSE between two random normal vectors in R^2048 should be approximately `2 * 0.02^2 * 2048 = 1.6` — but the actual JEPA loss is ~0.4, which suggests the initialization is not purely independent (the model.apply() weight init runs after the first forward is called, so the very first step may not be fully initialized). This will self-correct immediately as training begins.
+
+The accept_acc at 50% confirms the verifier is operating at chance. Both positive examples (real future states) and negative examples (rolled batch negatives) are indistinguishable to an untrained verifier. The metric should rise sharply once the backbone has trained enough to produce distinguishable hidden states for different contexts — at 25M this happened by step 200.
+
+The ~5.8–6.3 starting token CE is slightly above ln(256) = 5.545. This is normal for a normal-initialized model. The warmup phase will bring this down quickly.
+
+---
+
+## Architecture Questions at 1B Scale
+
+Scaling from 250M to 1B raises a few open questions that the training run should answer.
+
+**Does the JEPA signal remain load-bearing?** At 25M, removing the JEPA loss collapsed acceptance from 63% to 2.7%. This is a strong result, but it was measured on TinyShakespeare with d_model=384. At d_model=2048, the judge predicts in a 2048-dimensional space. The MSE signal is potentially noisier in higher dimensions (curse of dimensionality), but the model has more parameters to fit the prediction task. My expectation is that the JEPA signal remains necessary — the ablation result suggests a structural dependency rather than a scale-sensitive phenomenon — but the acceptance rate under the trained model at 1B may differ from 63%.
+
+**Does the block_size=2 choice hold?** The sweep at 25M showed block_size=2 had the best merge precision (1.92x) and passed the gate. At 1B, a stronger backbone may enable block_size=4 to pass — the judge has a better signal to work with. Conversely, the longer sequence length (2048 vs 512) means each step processes more token positions, so even block_size=2 represents significant compute per step. This is worth a targeted experiment at 50K steps, not just a sweep at initialization.
+
+**Do the two platforms converge to the same loss?** They should, given the same corpus, the same configuration, and the same random seed. The starting losses differ slightly (9.44 vs 9.27) due to different RNG implementations in MLX vs PyTorch. The convergence curves should cross and stabilize near the same value. If they diverge significantly (more than 5% in token NLL at 10,000 steps), it would suggest a numerical precision difference between the backends — most likely related to how MLX handles bf16 versus PyTorch's explicit AMP autocast.
+
+---
+
+## What Comes Next
+
+The 1B training run will take approximately 1.5–2 days on each machine. When the checkpoint-500 numbers are in, I will update the scaling table above. When both runs reach step 50,000, I will run the eval gate (the same five-condition check used at 25M) on each platform's checkpoint and compare.
+
+Beyond 1B, there are three things the architecture needs before it can function as an actual inference system rather than a training harness:
+
+**KV-cache integration.** The current training loop runs full forward passes for every batch. In deployment, the backbone would cache key-value pairs from prior context and only attend over new tokens. This changes how the drafter is called — instead of taking the full hidden state tensor, it would take only the last position's hidden state. The architecture supports this (the drafter's input is per-position parent hidden states), but the inference harness does not yet implement it.
+
+**Sampling diversity control.** The drafter currently produces draft tokens by sampling from the logit distribution. The temperature parameter, top-k cutoff, and the number of branches k are all fixed in the current implementation. For a production system, you would want to tune k based on accepted branch statistics from prior steps — if the verifier is accepting most branches, reduce k; if it is rejecting most, increase it or adjust temperature.
+
+**Production measurement harness.** The eval gate measures correctness metrics (NLL, acceptance rate) but does not measure wall-clock latency. A production harness would measure tokens per second with the speculative graph active versus a baseline autoregressive loop on the same hardware. That number — actual generation speedup — is what the whole architecture is building toward, and it requires the KV-cache work to be meaningful.
+
+The 1B training run is the prerequisite for all of this. You cannot tune an inference system around a model that has not been trained.
+
+---
+
+## Summary
+
+SGJM at 1B scale is a byte-level causal transformer (d=2048, 20 layers) with three jointly-trained auxiliary components: a drafter that generates 2-token draft blocks in a single forward pass, a JEPA judge that predicts future backbone hidden states and scores branches by latent proximity, and a verifier that discriminates genuine continuations from contrastive negatives. The four loss terms (token CE, drafter CE, JEPA MSE, verifier BCE at weights 1.0/0.5/0.25/0.1) are jointly optimized without any component-specific training phase.
+
+The ablation evidence from 25M scale establishes that the JEPA signal is structurally required — not an optional regularizer. Removing it collapses branch acceptance by 96%. The drafter loss is required for branch organization (endpoint latent clustering), independently of its effect on acceptance rate.
+
+Two hardware platforms are running the 1B experiment simultaneously: Mac Studio M1 Ultra with MLX, and Hyde (Framework Desktop, AMD Strix Halo) with ROCm. The backend abstraction in `backends.py` made this a configuration choice rather than a porting effort. Both machines confirmed step-0 behavior consistent with random initialization (accept_acc ~50%, token CE near ln(256)).
+
+The full 50,000-step results will determine whether the architecture's properties hold at this scale. Based on the smaller-model behavior, the predictions are: JEPA signal converges within the warmup window, accept_acc stabilizes above 60%, and both backends produce similar final token NLL. The article will be updated with those numbers when the checkpoints are ready.
+
+---
+
+*Article started: 2026-05-18. Training in progress on both platforms. Results table will be updated at checkpoints 500, 5000, and 50000.*

CHANGELOG.md

@@ -0,0 +1,24 @@
+# Changelog
+
+All notable changes to SGJM are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+Versions are date-based (CalVer, `YYYY.M.D`).
+
+## [2026.6.5] - 2026-06-05
+
+Initial public pre-release.
+
+### Added
+- Apache License 2.0 (`LICENSE`) and `NOTICE`.
+- Hybrid Mamba-2 / attention backbone with post-gate work.
+- Speculative decoding pipeline: parallel draft generation, latent-space branch
+  scoring (JEPA), and discriminative verification in a single trainable system.
+- Training backends for MLX (Apple Silicon) and PyTorch (CUDA / ROCm / CPU).
+- Project metadata, classifiers, and URLs in `pyproject.toml`.
+
+### Notes
+- Alpha-stage research prototype. Interfaces, checkpoints, and training recipes
+  may change without notice.
+
+[2026.6.5]: https://github.com/AdamPippert/SGJM/releases/tag/2026.6.5

CLAUDE.md

@@ -0,0 +1,17 @@
+# CLAUDE.md
+
+Project-specific guidance for Claude Code.
+
+<!-- claude-commit-policy:v1 -->
+## Commit policy
+
+**Claude must never appear as a committer or co-author in this repository.**
+
+- All commits must be authored as `Adam Pippert <adam.pippert@gmail.com>` — or `Coastal Digital Research` for CDR-owned repos.
+- Do **not** add `Co-Authored-By: Claude ...` trailers to commit messages.
+- Do **not** add `🤖 Generated with [Claude Code]` footers.
+- Do **not** configure `user.name` or `user.email` to anything containing `claude` or `anthropic`.
+- Commits authored by Claude are forbidden and will be force-rewritten out of history.
+
+If Claude Code is used to draft or edit changes, Adam Pippert remains the sole author of record.
+<!-- /claude-commit-policy:v1 -->

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Adam Pippert
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

NOTICE

@@ -0,0 +1,10 @@
+SGJM — Speculative Graph JEPA Model
+Copyright 2026 Adam Pippert
+
+This product includes software developed by Adam Pippert.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0

README.md

@@ -1,3 +1,463 @@
+# SGJM — Speculative Graph JEPA Model
+
+A research prototype combining speculative decoding with Joint Embedding Predictive Architecture (JEPA) to enable parallel draft generation, latent-space branch scoring, and discriminative verification — all within a single trainable system.
+
+## Architecture
+
+SGJM replaces standard autoregressive sampling with a four-component pipeline that generates, scores, and filters speculative token branches in parallel.
+
+```
+                 ┌─────────────────────────────────────────────┐
+  tokens ──────▶ │  Backbone (transformer, d=384, 10 layers)   │ ──▶ next-token logits
+                 └───────────────┬─────────────────────────────┘
+                                 │ hidden state h_t
+          ┌──────────────────────┼──────────────────────────┐
+          ▼                      ▼                          ▼
+   ┌─────────────┐      ┌──────────────┐          ┌─────────────────┐
+   │   Drafter   │      │  JEPA Judge  │          │    Verifier     │
+   │  (d=192,    │      │  predicts    │          │  discriminates  │
+   │  2 layers)  │      │  h_{t+block} │          │  accept/reject  │
+   └──────┬──────┘      └──────┬───────┘          └────────┬────────┘
+          │ k draft branches   │ predicted future latent   │ accept score
+          └────────────────────┴───────────────────────────┘
+                                        │
+                                branch selection & merge
+                                        │
+                                 accepted tokens
+```
+
+### Components
+
+**Backbone** — A causal byte-level (vocab=256) sequence model that produces hidden states and next-token logits. Configurable as a pure transformer (default) or as a **hybrid Mamba-2 / attention** stack via `ModelConfig.attn_every_n` — when set, every `attn_every_n`-th layer is a full-attention block and the remaining layers are Mamba-2 SSD blocks. SwiGLU MLP, RMS normalization, tied input/output embeddings.
+
+**Drafter** — Projects the parent hidden state to a smaller space (d=192) and uses learnable position queries to speculatively produce `k` token blocks of length `block_size` in a single forward pass. Each branch carries tokens, an endpoint latent, and a log-probability.
+
+**JEPA Judge** — A two-layer feedforward network that predicts what the backbone's hidden state *should* look like at the end of a draft block, trained with MSE against the actual future latent (stop-gradient). Branches are scored by judge confidence rather than token probability alone.
+
+**Verifier** — A binary classifier that takes the concatenated parent and child hidden states and outputs an acceptance score. Trained with contrastive pairs (true future vs. rolled negatives). A branch passes verification if its score exceeds a configurable threshold.
+
+### Parameter Budget
+
+| Component | Params (approx) |
+|-----------|----------------|
+| Backbone | ~22M |
+| Drafter | ~2M |
+| Judge | ~1M |
+| Verifier | ~0.5M |
+| **Total** | **~25M** |
+
+The same-budget baseline is an 11-layer transformer with no speculative components, used as the eval gate comparison.
+
 ---
-license: apache-2.0
+
+## Training
+
+### Loss
+
+Four terms are summed with configurable weights:
+
+| Term | Formula | Weight |
+|------|---------|--------|
+| Token | cross-entropy, backbone LM head | 1.0 |
+| Drafter | cross-entropy, draft token predictions | 0.5 |
+| JEPA | `0.5 * (MSE(judge_pred, h_future) + MSE(drafter_endpoint, h_future))` | 0.25 |
+| Verifier | binary cross-entropy, contrastive pairs | 0.1 |
+
+`accept_acc` (fraction of drafts passing the verifier threshold) is tracked as the primary auxiliary metric.
+
+### Running a training job
+
+```bash
+# Sizes:    smoke | 25m | 100m | 250m | 1b | 25m-hybrid | 250m-hybrid
+# Backends: auto | mlx | cuda | rocm | cpu   (auto detects platform)
+
+# MLX — Apple Silicon
+python -m sgjm.training --size 25m --backend mlx
+
+# CUDA — NVIDIA
+python -m sgjm.training --size 250m --backend cuda
+
+# ROCm — AMD (Strix Halo / Framework Desktop "Hyde")
+python -m sgjm.training --size 250m --backend rocm
+
+# Hybrid Mamba-2 / attention backbone (1 attention + N-1 Mamba-2 blocks)
+python -m sgjm.training --size 25m-hybrid --backend rocm
+
+# CPU fallback (slow; useful for tests)
+python -m sgjm.training --size smoke --backend cpu
+
+# Override individual hyperparameters
+python -m sgjm.training --size 25m --steps 10000 --lr 1e-4 --checkpoint-dir runs/my-run
+```
+
+Checkpoints are written as `.safetensors` every `--checkpoint-every` steps (default 500). Training config is saved as `config.json` alongside weights.
+
 ---
+
+## Evaluation
+
+The eval harness computes SGJM metrics and compares against a same-budget baseline. A run **passes the gate** if all five conditions hold:
+
+| Gate condition | Threshold |
+|----------------|-----------|
+| NLL delta vs baseline | ≤ 0.05 nats |
+| Branch acceptance rate | ≥ 50% |
+| JEPA top-1 accuracy above chance | ≥ +5 pp |
+| Merge precision advantage (random JS / merge JS) | ≥ 1.5× |
+| Compute per accepted token vs baseline | ≥ 1.0× (no regression) |
+
+```bash
+# Compare SGJM vs baseline (MLX)
+python -m sgjm.eval \
+  --sgjm runs/sgjm-25m/best.safetensors \
+  --baseline runs/baseline-25m/final.safetensors \
+  --backend mlx --batches 32 --report results/gate_report.json
+
+# Run an ablation sweep (MLX, 1000 steps per variant)
+python -m sgjm.research \
+  --sweep ablation --backend mlx --size 25m \
+  --steps 1000 --eval-batches 16 --out-dir runs/ablation-25m
+```
+
+---
+
+## Results
+
+### Run 1 — MLX, Apple Silicon, 2026-05-13
+
+| | |
+|--|--|
+| **Host** | MacBook Pro (arm64) |
+| **Backend** | MLX 0.29.1 / Python 3.12 |
+| **Duration** | 27.3 min |
+| **Steps** | 5 000 |
+| **Data** | TinyShakespeare (1 MiB, byte-level) |
+| **Seed** | 42 |
+
+**Eval loss progression** (16-batch held-out set):
+
+| Step | Total | Token | Accept Acc |
+|------|------:|------:|-----------:|
+| 500 | 2.053 | 0.278 | 94.0% |
+| 1 000 | 0.472 | 0.097 | 98.9% |
+| 1 500 | 0.347 | 0.064 | 99.3% |
+| 2 000 | 0.293 | 0.051 | 99.4% |
+| 2 500 | 0.255 | 0.042 | 99.6% |
+| 3 000 | 0.219 | 0.033 | 99.6% |
+| 3 500 | 0.199 | 0.030 | 99.8% |
+| 4 000 | 0.185 | 0.027 | 99.8% |
+| **4 500** | **0.179** | **0.025** | **99.8%** |
+
+Best eval total loss: **0.1790** at step 4500. Token loss: **0.0254**. Accept accuracy: **99.8%**.
+
+Full training log: [`results/sgjm-25m-mlx-run1/train.jsonl`](results/sgjm-25m-mlx-run1/train.jsonl)
+
+### Run 2 — 100M, MLX, Apple Silicon, 2026-05-13
+
+| | |
+|--|--|
+| **Host** | MacBook Pro (arm64) |
+| **Backend** | MLX 0.29.1 / Python 3.12 |
+| **Duration** | 55.4 min |
+| **Steps** | 5 000 |
+| **Params** | ~93M (d_model=768, 9 layers) |
+| **Data** | TinyShakespeare (1 MiB, byte-level) |
+
+| Step | Total | Token | Accept Acc |
+|------|------:|------:|-----------:|
+| 1 000 | 2.338 | 0.430 | 92.9% |
+| 2 000 | 0.388 | 0.081 | 99.5% |
+| 3 000 | 0.229 | 0.038 | 99.8% |
+| 4 000 | 0.176 | 0.027 | 99.8% |
+| **4 500** | **0.167** | **0.024** | **99.9%** |
+
+**Scaling return**: +272% parameters, +103% training time, −6.9% eval loss vs 25M.  
+Full log: [`results/sgjm-100m-mlx-run1/`](results/sgjm-100m-mlx-run1/)
+
+### Run 3 — 250M, MLX, Apple Silicon, 2026-05-14
+
+| | |
+|--|--|
+| **Host** | MacBook Pro (arm64) |
+| **Backend** | MLX 0.29.1 / Python 3.12 |
+| **Duration** | 365.8 min (6.1 hours) |
+| **Steps** | 10 000 |
+| **Params** | ~251M (d_model=1024, 14 layers) |
+| **Data** | Python stdlib + site-packages (32 MiB, python_extended) |
+
+| Step | Total | Token NLL | Accept Acc |
+|------|------:|----------:|-----------:|
+| 1 000 | 3.973 | 2.434 | 80.7% |
+| 2 000 | 3.131 | 1.854 | 93.7% |
+| 3 000 | 2.719 | 1.495 | 97.7% |
+| 4 000 | 2.184 | 1.111 | 97.7% |
+| 5 000 | 2.159 | 1.087 | 98.5% |
+| **6 500** | **1.823** | **0.889** | **99.1%** |
+| 7 500 | 1.827 | 0.887 | 99.3% |
+| 9 500 | 1.825 | 0.888 | 99.0% |
+
+Best eval total loss: **1.823** at step 6500. Model converged by step 6500 and plateaued — 32 MiB corpus capacity ceiling. Speculative speedup: **1.28×** on fibonacci prompt (AR 31.9 tok/s → Spec 40.9 tok/s, 100% accept).  
+Full log: [`results/sgjm-250m-mlx-run1/`](results/sgjm-250m-mlx-run1/)
+
+### Run 4 — ROCm cross-platform validation, 2026-05-17 → 2026-05-18
+
+SGJM-25M and SGJM-250M trained end-to-end on AMD Strix Halo (Framework Desktop "Hyde") under PyTorch ROCm. Confirms the dual-backend architecture: identical config + corpus + checkpoint format across MLX and ROCm.
+
+| Run | Backend | Host | Result |
+|-----|---------|------|--------|
+| `sgjm-25m-rocm` | ROCm | Strix Halo | matches MLX 25M trajectory |
+| `sgjm-250m-rocm` | ROCm | Strix Halo | matches MLX 250M trajectory |
+
+Full logs: [`results/hyde-rocm/`](results/hyde-rocm/)
+
+### Run 5 — 1B v1, dual-platform, 2026-05-19 (analyzed; retrain queued)
+
+SGJM-1B trained simultaneously on Mac Studio M1 Ultra (MLX) and Strix Halo (ROCm), 4.6h wall time. Backbone learned successfully; **verifier and accept heads did not learn** — root-caused to a negative-sampling axis bug (verifier negatives were being rolled along the batch dim rather than the sequence dim). Fix landed as `fix(verifier): roll negatives along sequence dim, not batch dim`. Retrain scheduled for 2026-05-22.
+
+Write-up: [`BLOG_1B.md`](BLOG_1B.md). Checkpoint dir: `runs/sgjm-1b-rocm/`.
+
+---
+
+## Phase 5 Results — Gate Run & Ablation
+
+### Eval Gate — PASS (2026-05-13)
+
+SGJM-25M (step 4500) vs same-budget baseline (11-layer transformer, step 4999).  
+Data: TinyShakespeare, 1 MiB, byte-level. Backend: MLX, Apple Silicon.
+
+| Gate condition | SGJM | Baseline | Result |
+|----------------|-----:|--------:|--------|
+| NLL delta | +0.0015 nats | — | ✅ ≤ 0.05 |
+| Branch acceptance rate | 100% | — | ✅ ≥ 50% |
+| JEPA top-1 acc (chance = 11.1%) | 99.6% | — | ✅ +88.5 pp above chance |
+| Merge precision advantage | **10 607×** | — | ✅ ≥ 1.5× |
+| Compute advantage | **13.92×** | — | ✅ ≥ 1.0× |
+
+The 13.92× compute advantage means the baseline spends 13.92× more FLOPs per token than SGJM spends per accepted token. The 10 607× merge precision advantage confirms that SimHash-bucketed draft branches are highly semantically similar — the speculative merge strategy is valid.
+
+Full report: [`results/phase5-eval-gate/gate_report.json`](results/phase5-eval-gate/gate_report.json)
+
+### Ablation Sweep — 25M, 1000 steps/variant (2026-05-13)
+
+Each variant trained from scratch for 1000 steps with MLX; same shared baseline (token NLL = 0.0884).
+
+| Variant | Token NLL | Accept Rate | JEPA top-1 | Merge Adv. | Key finding |
+|---------|----------:|------------:|-----------:|-----------:|-------------|
+| `sgjm_no_drafter` | 0.0916 | **100%** | 95.5% | 0.996× | Drafter loss drives merge precision — without it JS divergence of merged branches is indistinguishable from random pairs |
+| `sgjm_full` | 0.1011 | 63.3% | 97.5% | 1.19× | Merge precision underfit at 1000 steps; reaches 10 607× at 5000 steps |
+| `sgjm_no_verifier` | 0.1009 | **21.3%** | 96.7% | 1.18× | Verifier is required for reliable branch acceptance |
+| `sgjm_token_only` | 0.0890 | 18.6% | 11.4% ≈ chance | 1.0× | Without aux losses, JEPA and merge are dead — indistinguishable from noise |
+| `sgjm_no_jepa` | 0.0992 | **2.7%** | 11.5% ≈ chance | 1.11× | JEPA is the most critical loss: acceptance collapses without it; compute regresses to 0.37× |
+
+**Key takeaways:**
+1. **JEPA is load-bearing.** Removing it collapses branch acceptance from 63% to 2.7% and turns the compute advantage negative (0.37×).
+2. **Verifier gates quality.** Without it, acceptance drops to 21% — the model accepts wrong branches.
+3. **Drafter loss enables merge.** Removing drafter training yields 100% acceptance (the backbone still guides the drafter) but destroys merge precision; branches are no longer semantically clustered.
+4. **Merge precision needs full training.** `sgjm_full` at 1000 steps has merge advantage 1.19×; at 5000 steps it reaches 10 607×. This is the slowest-learning signal.
+
+Full sweep results: [`results/phase5-ablation-25m-mlx/`](results/phase5-ablation-25m-mlx/)
+
+### 100M Scaling Run — Complete (2026-05-13)
+
+| Config | 25M | 100M |
+|--------|-----|------|
+| d_model | 384 | 768 |
+| Backbone layers | 10 | 9 |
+| d_ff | 1 536 | 3 072 |
+| Drafter d_model | 192 | 384 |
+| Max seq len | 512 | 1 024 |
+| Est. params | ~25M | ~93M |
+| Training time | 27.3 min | 55.4 min |
+| Best eval total loss | 0.1790 | 0.1666 |
+| Best eval token NLL | 0.0254 | 0.0241 |
+
+Scaling return: +272% parameters, +103% training time, −6.9% eval loss.  
+Full log: [`results/sgjm-100m-mlx-run1/`](results/sgjm-100m-mlx-run1/)
+
+---
+
+## Phase 5 — Hyperparameter Sweeps
+
+### Loss Weight Sweep — `jepa` weight vs performance (1000 steps each, 2026-05-13)
+
+| `jepa_weight` | Token NLL | Accept Rate | JEPA top-1 | Merge Adv. | Finding |
+|--------------|----------:|------------:|-----------:|-----------:|---------|
+| 0.0 | 0.0992 | **2.7%** | 11.5% ≈ chance | 1.11× | JEPA weight=0 collapses acceptance (same as no_jepa ablation) |
+| 0.05 | 0.0989 | 64.2% | 97.1% | **1.20×** | Lowest weight that activates all components |
+| **0.25** | **0.1011** | 63.3% | 97.5% | 1.19× | Default weight — good balance of all metrics |
+| 1.0 | 0.1061 | 81.4% | 98.3% | 1.00× | Higher acceptance but merge precision saturates |
+| 4.0 | 0.1569 | 100% | 98.4% | 1.00× | Acceptance maxed but token NLL regresses (+58%) |
+
+**Finding**: `jepa_weight=0.05` is the effective elbow — it activates all four metrics with minimum NLL cost. The default 0.25 is a safe operating point. Going above 1.0 trades language modeling quality for acceptance rate with no merge-precision benefit.
+
+### Block Size Sweep — block_size vs performance (1000 steps each, 2026-05-13)
+
+| `block_size` | Token NLL | Accept Rate | JEPA top-1 | Merge Adv. | Finding |
+|-------------|----------:|------------:|-----------:|-----------:|---------|
+| 2 | **0.0963** | 69.0% | **99.1%** | **1.92×** | Best merge precision — smaller blocks easier to predict |
+| **4** | 0.1011 | 63.3% | 97.5% | 1.19× | Default — good balance |
+| 8 | 0.1007 | **90.8%** | 96.1% | 1.00× | Highest acceptance but merge precision collapses |
+
+**Finding**: `block_size=2` gives the best merge precision advantage (1.92×) with lowest NLL. Larger blocks are harder to predict precisely, which hurts merge clustering. `block_size=4` is the default sweet spot balancing tokens-per-step and precision.
+
+### Merge Radius Sweep — SimHash threshold vs merge precision (1000 steps each, 2026-05-13)
+
+All variants trained identically; only the eval-time merge threshold differs.
+
+| `merge_radius_bits` | Token NLL | Accept Rate | Merge JS | Random JS | Merge Adv. |
+|--------------------|----------:|------------:|---------:|----------:|-----------:|
+| 2 | 0.1011 | 63.3% | NaN | 0.6891 | 1.00× | Radius too tight — no pairs qualify |
+| 4 | 0.1011 | 63.3% | NaN | 0.6891 | 1.00× | Radius too tight — no pairs qualify |
+| **6** | **0.1011** | **63.3%** | **0.5780** | **0.6891** | **1.19×** | Sweet spot — pairs qualify, JS divergence meaningfully lower |
+| 8 | 0.1011 | 63.3% | 0.6228 | 0.6891 | 1.11× | Wider radius admits less-similar pairs |
+| 12 | 0.1011 | 63.3% | 0.6228 | 0.6891 | 1.11× | No improvement beyond r=8 |
+
+**Finding**: `merge_radius_bits=6` is the optimal threshold (default). Below 6, the radius is so tight that no pairs qualify (merge_precision_js = NaN). Above 6, admitting more diverse pairs dilutes the advantage. The 10 607× advantage in the 5000-step gate run (vs 1.19× here) confirms that merge precision is a slow-learning signal that emerges with more training.
+
+---
+
+## Generation Benchmark (2026-05-13)
+
+**Production-scale result (250M, Python corpus, MLX)**: **1.28× speculative speedup** on a fibonacci prompt (AR 31.9 tok/s → Spec 40.9 tok/s, 100% accept). See Run 3 above.
+
+The 25M Python-harness benchmark below shows throughput **parity**, not speedup — at the 25M scale the per-call Python overhead dominates the savings from 4-token parallel drafting. The 13.92× compute-FLOPs advantage from the gate run is the theoretical upper bound and is realized only with KV-cache and fused CUDA/Metal kernels.
+
+Benchmark: 200 tokens generated from 64-token prompt, MLX, Apple Silicon, SGJM-25M step 4500.
+
+| Metric | SGJM (50 steps × 4 tokens) | AR (200 steps × 1 token) |
+|--------|---------------------------:|-------------------------:|
+| Tokens generated | 200 | 200 |
+| Model fwd passes | 100 (50 backbone + 50 drafter) | 200 backbone |
+| Acceptance rate (harness) | 25% (1 of 4 kept) | 100% |
+| Elapsed (s) | 1.32 | 1.31 |
+| Tokens / sec | 151.7 | 153.0 |
+| **Speedup** | **0.99×** | — |
+
+**Interpretation**: This Python harness benchmark shows throughput parity — SGJM's 4-token parallel drafting absorbs its per-call overhead. The 13.92× compute-FLOPs advantage from the gate run is a theoretical upper bound that would be realized with KV-cache and fused CUDA/Metal kernels, not a naive Python harness.
+
+Full report: [`results/phase5-bench/benchmark_report.txt`](results/phase5-bench/benchmark_report.txt)
+
+---
+
+## Project Status
+
+### Phase 1 — Core Harness ✅
+- [x] Graph node and address types
+- [x] Branch lifecycle manager (create, advance, merge, expire)
+- [x] Branch policy (keep-top-K, SimHash merge radius)
+- [x] Harness runner (speculative generation loop)
+- [x] Backbone / drafter / judge / verifier protocols + stubs
+
+### Phase 2 — Training Pipeline ✅
+- [x] `TrainingConfig` with per-component loss weights
+- [x] Byte-level dataset (TinyShakespeare + synthetic Markov-2)
+- [x] MLX backend (Apple Silicon) — trainer, model, losses
+- [x] PyTorch backend (CUDA / ROCm / CPU) — trainer, model, losses, baseline
+- [x] Cosine LR schedule with linear warmup
+- [x] Checkpoint save/load (`.safetensors`)
+- [x] Training JSONL log
+
+### Phase 3 — Eval & Gate ✅
+- [x] `SGJMEvalMetrics`: token NLL/PPL, branch acceptance rate, JEPA top-1 accuracy, merge precision JS divergence, compute-per-accepted-token
+- [x] `BaselineEvalMetrics`: token NLL/PPL, compute-per-token
+- [x] `ComparisonReport` with five-gate pass/fail logic
+- [x] Eval CLI (`python -m sgjm.eval`)
+
+### Phase 4 — Research Harness ✅
+- [x] `ExperimentCard` (named ablations with config overrides and expected signals)
+- [x] `SweepResult` with composite primary score
+- [x] Auto-research scaffold with real-corpus loader
+
+### Phase 5 — Gate Run & Analysis ✅
+- [x] Eval gate PASS: 25M SGJM vs same-budget baseline — compute advantage 13.92×, merge advantage 10 607×
+- [x] Ablation sweep: all 4 components isolated — JEPA most critical, drafter loss drives merge precision
+- [x] 100M scaling run complete (d_model=768, ~93M params) — 6.9% improvement over 25M
+- [x] Loss weight sweep: `jepa_weight=0.05` is effective elbow; default 0.25 is safe operating point
+- [x] Block size sweep: `block_size=2` best merge precision (1.92×); default 4 balances speed and precision
+- [x] Merge radius sweep: `merge_radius_bits=6` is optimal threshold
+- [x] Generation benchmark: Python harness parity (0.99×); 13.92× FLOPs advantage requires KV-cache + kernel fusion
+- [x] 250M scaling run complete (d_model=1024, ~251M params, 32 MiB Python corpus) — best eval total loss 1.823, 99.1% accept, 1.28× speculative speedup
+
+### Post-Gate Scaling — in progress
+
+- [x] 250M MLX run on extended Python corpus (32 MiB) — eval total loss 1.823, 1.28× speculative speedup on fibonacci prompt
+- [x] Cross-platform ROCm runs: SGJM-25M and SGJM-250M on AMD Strix Halo ([`results/hyde-rocm/`](results/hyde-rocm/))
+- [x] Hybrid Mamba-2 / attention backbone added (`25m-hybrid`, `250m-hybrid` sizes; configurable via `ModelConfig.attn_every_n`)
+- [x] SGJM-1B v1 trained dual-platform (Mac Studio MLX + Strix Halo ROCm). Backbone learned; verifier and accept heads did not — root-caused to a verifier-negatives axis bug. See [`BLOG_1B.md`](BLOG_1B.md).
+- [ ] SGJM-1B v2 retrain on 2026-05-22 (both platforms) with the verifier fix in place
+
+---
+
+## Repository Layout
+
+```
+src/sgjm/
+├── graph/          # Node types, address encoding, graph manager (in-memory speculation tree — not a graph DB)
+├── branch/         # Lifecycle, policy, verifier protocol
+├── harness/        # Speculative generation runner, metrics snapshot
+├── modules/        # Backbone, drafter, judge protocols + stubs
+├── training/
+│   ├── config.py       # TrainingConfig, ModelConfig, OptimConfig (incl. Mamba-2 + attn_every_n)
+│   ├── data.py         # ByteDataset, corpus loaders
+│   ├── backends.py     # Backend detection (mlx / cuda / rocm / cpu)
+│   ├── mlx_backend/    # MLX model, losses, trainer, mamba2 SSD blocks
+│   └── torch_backend/  # PyTorch model, losses, trainer, baseline, mamba2 SSD blocks
+├── eval/           # Metrics, ComparisonReport, checkpoint loader, CLI
+├── bench/          # MLX speculative-vs-AR generation benchmark
+├── demo/           # Generation demo CLI
+└── research/       # ExperimentCard, SweepResult, sweep runner
+
+results/                            # Eval reports, completed run snapshots
+├── sgjm-25m-mlx-run1/                  # Run 1 — 25M MLX
+├── sgjm-100m-mlx-run1/                 # Run 2 — 100M MLX
+├── sgjm-250m-mlx-run1/                 # Run 3 — 250M MLX, Python corpus
+├── hyde-rocm/                          # Run 4 — 25M + 250M on AMD Strix Halo (ROCm)
+├── phase5-eval-gate/                   # Gate report JSON (PASS)
+├── phase5-ablation-25m-mlx/            # Ablation sweep
+├── phase5-sweeps/                      # Loss-weight / block-size / merge-radius sweeps
+├── phase5-bench/                       # 25M generation benchmark report
+└── demo-{250m,python}/                 # Demo CLI outputs
+
+runs/                               # Active training output (checkpoints + logs)
+├── sgjm-1b-rocm/                       # Run 5 — 1B v1 (analyzed) and v2 (queued 2026-05-22)
+├── sgjm-{25m,250m}-rocm/               # ROCm runs
+└── sgjm-{25m,250m}-hybrid/             # Hybrid Mamba-2 / attention runs
+
+tests/              # Behavior-driven test suite (pytest)
+```
+
+---
+
+## Development
+
+```bash
+# MLX — Apple Silicon
+pip install -e '.[mlx,dev]'
+
+# CUDA — NVIDIA (default PyPI torch wheels)
+pip install -e '.[cuda,dev]'
+
+# ROCm — AMD (Strix Halo, etc.). The [rocm] extra deliberately excludes torch;
+# install ROCm torch wheels from the PyTorch index first, then the extras:
+pip install --index-url https://download.pytorch.org/whl/rocm6.2 torch
+pip install -e '.[rocm,dev]'
+
+# CPU — any platform, slow
+pip install -e '.[cpu,dev]'
+
+# Run tests
+pytest
+
+# Smoke train + eval
+python -m sgjm.training --size smoke --backend cpu
+```
+
+All production code must be preceded by a failing test. See [`CLAUDE.md`](CLAUDE.md) for the commit author policy enforced in this repository.
+
+## License
+
+Licensed under the Apache License, Version 2.0. See [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE).
+
+Copyright 2026 Adam Pippert.
+
+> **Status:** `2026.6.5` is an initial pre-release research prototype (Development Status: Alpha). Versions are date-based (CalVer, `YYYY.M.D`). Interfaces, checkpoints, and training recipes may change without notice.

USE_CASES.md

@@ -0,0 +1,323 @@
+# SGJM Use Cases — Small Speculative Models in Practice
+
+A guide to where 25M-class models with speculative decoding fit, and where they don't.
+
+---
+
+## 1. The 25M Parameter Tier
+
+Most production LLMs are measured in billions of parameters. So why build at 25M?
+
+Three reasons:
+
+**Latency constraints**. A 7B model at float16 takes ~14 GB of memory and dozens of milliseconds per token on typical hardware. A 25M model takes ~50 MB and can sustain thousands of tokens per second on a CPU, or run on a microcontroller-class device. For applications where user experience degrades above 50ms latency, 25M is the right tier.
+
+**Edge and offline deployment**. On-device assistants (phones, embedded systems, wearables) cannot call cloud APIs. A 25M byte-level model ships in a single 50 MB file, needs no tokenizer vocabulary file, and runs on any hardware that supports matrix multiplication.
+
+**Domain-specific fine-tuning**. A small model trained exclusively on a narrow domain (a company's documentation, a specific programming language, a medical subspecialty) can outperform a larger general model on that domain at a fraction of the compute cost. Smaller models also fine-tune in minutes rather than hours.
+
+The speculative mechanism in SGJM is relevant across all three: fewer forward passes per accepted token means lower latency and lower power draw, which matters most at small scale where memory bandwidth is the bottleneck.
+
+---
+
+## 2. Use Cases
+
+### 2.1 Domain-Specific Code Autocomplete
+
+**What it is**: An editor assistant trained on a single codebase or language, suggesting completions at the function-signature or block level.
+
+**Why 25M works**: Code has strong local structure. A byte-level model sees indentation, brackets, and keywords directly. The drafter's 4-token block drafting maps naturally to one line of code. The JEPA judge learns that `if condition:\n    ` and `if condition:\n    return` share the same semantic future — and can speculate accordingly.
+
+**SGJM advantage**: The speculative mechanism pays off here. Code completions are often prefix-predictable: once the model commits to `def fibonacci(`, the next few tokens are high-probability. The drafter proposes the full `n):` block; the verifier accepts it at low cost. At 5000 training steps on Python stdlib (4.7 MiB), SGJM reaches token NLL comparable to a same-budget 11-layer transformer.
+
+**Where it doesn't work**: Novel algorithmic code that requires global context understanding. A 25M model does not reason; it pattern-matches.
+
+**Comparable models at 25M scale**:
+
+| Model | Params | Training data | Approach | Suitable for |
+|-------|--------|---------------|----------|--------------|
+| **SGJM-25M** | 25M | Python stdlib (4.7 MiB) | Byte-level, speculative | On-device Python autocomplete |
+| GPT-2 small | 117M | WebText (40 GB) | Byte-pair encoding | General English text |
+| DistilGPT-2 | 82M | Same as GPT-2 | BPE, distilled | General English, faster than GPT-2 |
+| CodeParrot small | 110M | GitHub Python | BPE, causal LM | Python code, pretrained |
+| Phi-1 (small) | 1.3B | Textbooks | BPE | Reasoning-heavy code tasks |
+
+SGJM-25M is 5–50× smaller than all of these. The quality gap is real — but so is the deployment advantage.
+
+---
+
+### 2.2 Structured Text Generation (Config Files, JSON, Schemas)
+
+**What it is**: Generating valid structured text — JSON, YAML, TOML, HTML — where the schema is known and structure is highly repetitive.
+
+**Why 25M works**: Structured formats have extremely high local predictability. After `"name": "`, a trained model can predict that the next tokens are alphanumeric with near certainty. A byte-level model needs no special tokenizer for these formats.
+
+**SGJM advantage**: This is where the merge precision metric matters most. In highly structured text, draft branches that share the same SimHash bucket genuinely share the same next-token distribution — the 10,607× merge precision advantage from the gate run is realistic here. Branches that draft `"value": 1` and `"value": 2` will be correctly bucketed as similar (both lead to `}` or `,` next) and merged efficiently.
+
+**Example task**: Autofilling a JSON config from a partial key structure.
+
+```
+Prompt: {"model": {"d_model": 384, "
+SGJM draft candidates:
+  → n_layers": 10,     [accepted, score=0.94]
+  → n_heads": 6,       [accepted, score=0.91]
+  → max_seq_len":      [pruned, score=0.61]
+  → dropout": 0.0,     [pruned, score=0.58]
+```
+
+The verifier correctly ranks the most likely continuations and the drafter produces them in a single block-forward pass.
+
+---
+
+### 2.3 On-Device Language Assistance (Mobile / Wearable)
+
+**What it is**: Local text assistants that run entirely on-device — no network latency, no privacy concerns, no API costs.
+
+**Why 25M works**: Apple Neural Engine, Qualcomm NPU, and even modern ARM chips can sustain 25M parameter inference at hundreds of tokens/second. A byte-level model needs no tokenizer vocabulary — the entire model is a single 50 MB `.safetensors` file.
+
+**SGJM advantage**: The speculative mechanism reduces the number of sequential model calls. On a mobile NPU where kernel launch overhead is significant, drafting 4 tokens per call (instead of 1) directly cuts the number of synchronization points. The 13.92× FLOPs advantage from the gate run would partially materialize even in a naively implemented mobile inference stack.
+
+**Deployment comparison**:
+
+| Model | Size (fp16) | RAM needed | GGUF quantized | On-device viable |
+|-------|------------|------------|----------------|-----------------|
+| **SGJM-25M** | ~50 MB | ~100 MB | ~12 MB (4-bit) | ✅ Any hardware |
+| GPT-2 small | ~240 MB | ~500 MB | ~60 MB (4-bit) | ✅ Modern phones |
+| DistilGPT-2 | ~170 MB | ~350 MB | ~40 MB (4-bit) | ✅ Modern phones |
+| Llama 3.2 1B | ~2 GB | ~4 GB | ~500 MB (4-bit) | ✅ High-end phones |
+| Llama 3.2 3B | ~6 GB | ~12 GB | ~1.5 GB (4-bit) | ⚠️ High-end only |
+
+At 4-bit quantization, SGJM fits in 12 MB — smaller than many app icons.
+
+---
+
+### 2.4 Low-Latency Text Completion APIs
+
+**What it is**: A text completion service that must respond in < 20ms (autocomplete, search suggestions, real-time chat hints).
+
+**Why 25M works**: At 25M parameters on a CPU with AVX2, you can sustain ~1000 tokens/second. At 4-bit quantization on a GPU, this becomes 10,000+ tokens/second. Streaming 5 tokens in < 5ms is achievable.
+
+**SGJM advantage**: The speculative mechanism's primary benefit is amortizing per-call latency. In a naive batch-1 inference setting, the dominant cost is not compute but memory bandwidth and kernel dispatch. Drafting 4 tokens per backbone call cuts dispatch costs by 4×. The Python harness benchmark shows 0.99× throughput parity with AR at 200 tokens — the same implementation with Metal/CUDA kernels and KV-cache would realize the theoretical 13.92× advantage.
+
+**Latency comparison** (estimated, batch=1, CPU):
+
+| Model | Params | ~Tok/s (CPU) | P50 latency (10 tokens) |
+|-------|--------|-------------|------------------------|
+| **SGJM-25M** | 25M | ~1 000 | ~10 ms |
+| GPT-2 small | 117M | ~200 | ~50 ms |
+| DistilGPT-2 | 82M | ~300 | ~33 ms |
+| GPT-2 medium | 345M | ~70 | ~140 ms |
+| Llama 3.2 1B | 1B | ~25 | ~400 ms |
+
+---
+
+### 2.5 Specialized Scientific Text (DNA, Protein, SMILES)
+
+**What it is**: Generating or completing sequences in scientific notations where the "vocabulary" is inherently byte-level (ATCG, amino acids, chemical SMILES strings).
+
+**Why 25M works**: DNA and protein sequences are byte-level by nature. A model trained on GenBank entries at byte level outperforms a BPE model of the same size because the BPE tokenizer wastes capacity on multi-character subwords that have no biochemical meaning. The byte-level SGJM architecture requires zero modification.
+
+**SGJM advantage**: Scientific sequences have strong local motifs (codons are 3-byte patterns, SMILES ring closures follow strict grammar). The drafter learns to speculatively draft multi-residue blocks; the JEPA judge learns that codon boundaries are meaningful transition points. The merge precision metric — branches that are semantically equivalent at the next observation — directly captures functional equivalence in biological sequences.
+
+---
+
+## 3. Where 25M Models Don't Fit
+
+Being honest about limitations:
+
+**Multi-step reasoning**: A 25M model cannot chain deductions. It does not hold a scratchpad, cannot verify its outputs, and has no concept of "let me think step by step." This is a fundamental capacity problem, not a training data problem.
+
+**Long-range coherence**: Without KV-cache and full-context attention, SGJM processes sequences up to 512 tokens. Documents longer than ~1500 characters may show quality degradation. The 100M variant extends this to 1024 tokens but the fundamental limit remains.
+
+**Open-ended question answering**: A model trained on Python stdlib has no knowledge of history, geography, or natural language pragmatics. Domain-specific training is a feature and a constraint simultaneously.
+
+**Instruction following**: Without RLHF or instruction fine-tuning, SGJM completes text; it does not follow instructions. Adding instruction fine-tuning at 25M scale is tractable (Phi-1.5 demonstrates this) but not implemented here.
+
+---
+
+## 4. Side-by-Side Comparison: Python Code Completion
+
+### Python Corpus Training Results (2026-05-14)
+
+Training: SGJM-25M on Python stdlib (4.7 MiB), 5000 steps, MLX, Apple Silicon.  
+Duration: **32.5 minutes**.
+
+| Step | Eval Token NLL | Eval Accept Acc | Finding |
+|------|---------------:|----------------:|---------|
+| 500 | 1.809 | 90.5% | All four losses active from step 1 |
+| 1 000 | 1.232 | 95.2% | NLL dropping fast |
+| **2 000** | **1.126** | **96.8%** | **← best checkpoint (eval loss minimum)** |
+| 2 500 | 1.209 | 97.1% | Eval loss rises — overfitting on 4.7 MiB corpus |
+| 4 000 | 1.450 | 96.4% | Training NLL continues to fall; eval diverges |
+| 4 999 | (train) 0.102 | 98.7% | Heavily overfit to training set |
+
+**Early stopping at step 2000** is automatic ��� `best.safetensors` is saved at the eval loss minimum. This is the key constraint at small corpus sizes: 4.7 MiB of Python stdlib is enough to learn syntax and common idioms, but the model memorizes it before 5000 steps. For production autocomplete, train on 100+ MiB of domain code.
+
+### Eval gate results (same-budget comparison, TinyShakespeare run):
+
+*(Gate comparison was run on TinyShakespeare. Python corpus gate run pending larger dataset.)*
+
+| Condition | SGJM | Same-budget baseline | Advantage |
+|-----------|-----:|--------------------:|-----------|
+| Token NLL | 0.025 | 0.024 | ≈ parity |
+| Branch acceptance | 100% | — | SGJM only |
+| JEPA top-1 acc | 99.6% | — | SGJM only |
+| Merge precision | 10 607× | — | SGJM only |
+| Compute per token | 1× | **13.92×** | SGJM wins |
+
+### Actual demo completions (Python stdlib checkpoint, step 2000, greedy):
+
+**Prompt 1**: `def fibonacci(n):` (128 tokens, temperature=0)
+
+```
+def fibonacci(n):
+                raise ValueError("ISO week directive '%V' is incompatible with "
+```
+
+The model has never seen a fibonacci implementation — Python stdlib doesn't contain one. It correctly identifies that `def …(n):` is followed by a function body, and pattern-matches to the nearest stdlib pattern it knows: `strptime` format directives. **This is the correct behavior for a domain-specific model**: it generates valid Python syntax drawn from its training distribution. For a real autocomplete system, train on the target codebase.
+
+**Prompt 2**: `import json\n\ndef load_config(path):\n    """Load configuration from a JSON file."""`
+
+```
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    # The Parse all are nounded build all archs to allowed be used ben absen are
+```
+
+The model knows that docstrings are followed by comment lines — it produces syntactically correct Python. The comment content is garbled because the vocabulary of "configuration file" comments doesn't appear verbatim in stdlib. With a larger training corpus containing actual config-loading patterns, this would be coherent.
+
+**Prompt 3**: `class Transformer(nn.Module):\n    def __init__(self, d_model=512):`
+
+```
+class Transformer(nn.Module):
+    def __init__(self, d_model=512):
+        return self.__name__ == name * (days2, 256))
+```
+
+Again, syntactically plausible (a method body with a return statement) but semantically wrong because `nn.Module` subclasses don't appear in Python stdlib. A model trained on PyTorch source would complete this correctly.
+
+### Throughput (Python checkpoint, step 2000, Apple Silicon MX):
+
+| Method | Tokens | Time (s) | Tok/s | Speedup |
+|--------|-------:|---------:|------:|---------|
+| Autoregressive | 128 | 0.54 | **236.1** | baseline |
+| Speculative (fibonacci) | 128 | 1.02 | 125.1 | 0.53× |
+| Speculative (load_config) | 128 | 0.92 | 139.5 | 0.69× |
+| Speculative (transformer) | 128 | 0.91 | 140.1 | 0.67× |
+
+The speculative path is slower here because the Python harness overhead dominates for 128-token generation on a fast NPU. As noted in the benchmark (§Generation Benchmark), the 13.92× FLOPs advantage materializes with KV-cache and native kernel implementation, not in a Python harness. Branch acceptance is 94–100% in all three cases — the speculative mechanism is working correctly.
+
+### 250M vs 25M Python Corpus Comparison (2026-05-14)
+
+The 250M model was trained on `python_extended` (stdlib + site-packages, 32 MiB) for 10 000 steps on the same Apple Silicon host. Direct comparison at equivalent task:
+
+| | 25M (stdlib, 4.7 MiB) | 250M (extended, 32 MiB) |
+|--|----------------------|------------------------|
+| Params | ~25M | ~251M |
+| Corpus | 4.7 MiB Python stdlib | 32 MiB stdlib + site-packages |
+| Training time | 32.5 min | 365.8 min |
+| Best eval token NLL | 1.126 (step 2000) | 0.887 (step 7500) |
+| Best eval total loss | — | 1.823 (step 6500) |
+| Best eval accept acc | 96.8% | 99.1% |
+| Overfit? | Yes — eval rises after step 2000 | No — plateau, not overfit |
+
+**21% lower NLL** (1.126 → 0.887) from 10× more parameters on 7× more corpus data. The 250M model saturates at step 6500 without overfitting — the 32 MiB corpus provides enough diversity to prevent memorization.
+
+**Demo completions (250M, step 6500, temperature=0)**:
+
+*Fibonacci:*
+```
+def fibonacci(n):
+    [160 spaces — model generates whitespace continuation at greedy temperature]
+```
+Greedy temperature=0 with a plateau-converged model produces degenerate output for ambiguous one-line prompts. Use temperature > 0 for open-ended generation.
+
+*load_config:*
+```python
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    if path is None:
+        return path
+    if path is None:
+        return path
+    ...
+```
+Syntactically valid Python; repetition pattern typical of a model that has learned `if path is None:` from many stdlib guard clauses but lacks a stopping signal. Correct idiom, stuck in a loop.
+
+*DataLoader:*
+```python
+class DataLoader:
+    def __init__(self, dataset, batch_size=32):
+        self.dataset = dataset
+        self.dataset = dataset
+        ...
+```
+The model correctly writes `self.dataset = dataset` — this exact pattern appears in site-packages (PyTorch-style DataLoaders). Again repetition, same root cause.
+
+**Throughput (250M, Apple Silicon)**:
+
+| Prompt | AR tok/s | Spec tok/s | Speedup | Accept |
+|--------|----------|------------|---------|--------|
+| fibonacci | 31.9 | 40.9 | **1.28×** | 100% |
+| load_config | 23.3 | 7.2 | 0.31× | 100% |
+| DataLoader | 23.4 | 25.2 | **1.07×** | 100% |
+
+The 250M AR throughput (23–32 tok/s) is lower than the 25M's 236 tok/s due to 10× larger weight matrices. The 1.28× speculative speedup on fibonacci confirms the drafter mechanism scales to 250M — the draft-and-verify cycle remains beneficial on short-context prompts.
+
+### Key finding: domain specificity is the primary lever
+
+The most important takeaway from this demo is **not the speedup** (which requires kernel-level implementation) but **the domain-specificity effect**:
+
+- GPT-2 small (117M, WebText): produces English text for Python prompts — wrong domain entirely
+- SGJM-25M (Python stdlib): produces syntactically valid Python — right structure, constrained vocabulary
+- SGJM-25M (target codebase): would produce semantically correct completions — train on what you want to autocomplete
+
+At 25M parameters, you cannot have a general-purpose model. You can have an *excellent* domain-specific model that runs at 200+ tokens/second on any hardware with an MLX or CUDA backend.
+
+---
+
+## 5. Running the Demo
+
+Train on Python source code:
+
+```bash
+# Train SGJM on Python stdlib (~27 min, Apple Silicon)
+python -m sgjm.training --size 25m --backend mlx \
+  --data-source python --steps 5000 \
+  --checkpoint-dir runs/sgjm-python-25m
+
+# Side-by-side demo: speculative vs autoregressive
+python -m sgjm.demo \
+  --checkpoint runs/sgjm-python-25m/best.safetensors \
+  --prompt "def fibonacci(n):\n    " \
+  --n-tokens 128
+
+# Try other prompts
+python -m sgjm.demo \
+  --checkpoint runs/sgjm-python-25m/best.safetensors \
+  --prompt "import json\n\ndef load_config(path):\n    " \
+  --n-tokens 200
+
+python -m sgjm.demo \
+  --checkpoint runs/sgjm-python-25m/best.safetensors \
+  --prompt "class Transformer(nn.Module):\n    def __init__(self" \
+  --n-tokens 256
+```
+
+---
+
+## 6. Key Takeaways
+
+1. **25M is viable for domain-specific, latency-sensitive, or on-device tasks** — not as a GPT-4 replacement but as a specialized inference component.
+
+2. **The speculative mechanism is a serving strategy, not a quality improvement**. SGJM's NLL matches the baseline. Its value is FLOPs per accepted token — meaningful at edge scale, transformative with proper kernel implementation.
+
+3. **Byte-level models have a practical advantage for code and structured text**: no tokenizer vocabulary, handles all Unicode naturally, trains on any text file without preprocessing.
+
+4. **Domain training on 4.7 MiB of Python stdlib produces a functional code completion model in 27 minutes** on a MacBook. The same approach applies to any domain with a few megabytes of representative text.
+
+5. **The 25M tier is underexplored**. Most research focuses on 1B+ models. The ablation results here — particularly the JEPA load-bearing result — are likely generalizable upward and are easier to study at small scale.

data/sgjm_manifest.jsonl

@@ -0,0 +1,107 @@
+{"id": "ae8b71658778e1e3", "path": "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/adapters.py", "lane": "code_long_context", "branch_conflict_score": 0.1, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 3268}
+{"id": "05972991034b0675", "path": "/Users/adam/Development/SGJM/results/demo-python/demo_fibonacci.txt", "lane": "reasoning_trajectories", "branch_conflict_score": 0.025, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1366}
+{"id": "01e75719c8e3df11", "path": "/Users/adam/Development/SGJM/tests/test_research_mlx.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.25, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1279}
+{"id": "eef761372877a8bf", "path": "/Users/adam/Development/SGJM/tests/test_python_corpus.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.475, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 3168}
+{"id": "c4d641b8a4b3a748", "path": "/Users/adam/Development/SGJM/USE_CASES.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.925, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "low", "contradiction_tag": "local", "n_chars": 18365}
+{"id": "30f5dbe1bb1e576f", "path": "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 363}
+{"id": "449b3bbb943c5155", "path": "/Users/adam/Development/SGJM/scripts/build_sgjm_manifest.py", "lane": "code_long_context", "branch_conflict_score": 0.925, "latent_horizon": 4, "verifier_hardness": "hard", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 3566}
+{"id": "a6cd1bed43cb88bb", "path": "/Users/adam/Development/SGJM/src/sgjm/eval/mlx_metrics.py", "lane": "code_long_context", "branch_conflict_score": 0.425, "latent_horizon": 8, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 8465}
+{"id": "61ebd60550e1ebfd", "path": "/Users/adam/Development/SGJM/src/sgjm/research/sweep.py", "lane": "code_long_context", "branch_conflict_score": 0.225, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 4227}
+{"id": "b4630a7103b7557d", "path": "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/losses.py", "lane": "code_long_context", "branch_conflict_score": 0.3, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2734}
+{"id": "1fb7f7eb4ca601d7", "path": "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/trainer.py", "lane": "code_long_context", "branch_conflict_score": 0.7, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 7712}
+{"id": "175d02a98283128f", "path": "/Users/adam/Development/SGJM/src/sgjm/harness/runner.py", "lane": "code_long_context", "branch_conflict_score": 0.375, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 3100}
+{"id": "b9fde385313b43c4", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/merge_radius/merge_r2.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1824}
+{"id": "3792550769bdd1ca", "path": "/Users/adam/Development/SGJM/results/demo-python/demo_load_config.txt", "lane": "reasoning_trajectories", "branch_conflict_score": 0.0, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1629}
+{"id": "a806e7e37f79684a", "path": "/Users/adam/Development/SGJM/scripts/smoke.py", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1018}
+{"id": "03fc67cd623c2ff0", "path": "/Users/adam/Development/SGJM/results/phase5-ablation-25m-mlx/sgjm_token_only.json", "lane": "code_long_context", "branch_conflict_score": 0.05, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1991}
+{"id": "7fbdb375038794d6", "path": "/Users/adam/Development/SGJM/src/sgjm/research/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 456}
+{"id": "4d7dda8fbebee1e8", "path": "/Users/adam/Development/SGJM/results/phase5-ablation-25m-mlx/sgjm_full.json", "lane": "code_long_context", "branch_conflict_score": 0.075, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1829}
+{"id": "92a4b782c1f9722b", "path": "/Users/adam/Development/SGJM/src/sgjm/modules/judge.py", "lane": "code_long_context", "branch_conflict_score": 0.05, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 755}
+{"id": "790c4477885fdef9", "path": "/Users/adam/Development/SGJM/src/sgjm/training/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 399}
+{"id": "e374917155980992", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/merge_radius/merge_r12.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1882}
+{"id": "3ce1d105fa4d58cd", "path": "/Users/adam/Development/SGJM/results/phase5-bench/benchmark_report.txt", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 979}
+{"id": "843f1ed7b85857c6", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/block_size/block_4.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1874}
+{"id": "d70d908899cade76", "path": "/Users/adam/Development/SGJM/src/sgjm/demo/__main__.py", "lane": "code_long_context", "branch_conflict_score": 0.175, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 3559}
+{"id": "a7327a0588066182", "path": "/Users/adam/Development/SGJM/results/phase5-ablation-25m-mlx/summary.json", "lane": "code_long_context", "branch_conflict_score": 0.3, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 10862}
+{"id": "45737e34cf94c12b", "path": "/Users/adam/Development/SGJM/src/sgjm/branch/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.2, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 354}
+{"id": "55a8c135f702477f", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/loss_weight/summary.json", "lane": "code_long_context", "branch_conflict_score": 0.125, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 10743}
+{"id": "e54a32298fbe33ad", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/loss_weight/jepa_w_0.05.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1870}
+{"id": "c61081a340a46043", "path": "/Users/adam/Development/SGJM/tests/test_harness.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.125, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 801}
+{"id": "59edac7317548412", "path": "/Users/adam/Development/SGJM/results/autoresearch/sgjm_autoresearch_20260515_051230.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.275, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 3944}
+{"id": "9b7d6c44ad27c4e6", "path": "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/baseline.py", "lane": "code_long_context", "branch_conflict_score": 0.15, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1732}
+{"id": "f4dd7a333c70c907", "path": "/Users/adam/Development/SGJM/results/autoresearch/latest_manifest.json", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 862}
+{"id": "5a4222a51c1c3539", "path": "/Users/adam/Development/SGJM/src/sgjm/training/config.py", "lane": "code_long_context", "branch_conflict_score": 0.3, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 8662}
+{"id": "defee79b94691602", "path": "/Users/adam/Development/SGJM/tests/test_eval_cli_mlx.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.275, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 2542}
+{"id": "a131f05128bdb3bf", "path": "/Users/adam/Development/SGJM/results/phase5-ablation-25m-mlx/sgjm_no_jepa.json", "lane": "code_long_context", "branch_conflict_score": 0.05, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2015}
+{"id": "09d350d0839387d3", "path": "/Users/adam/Development/SGJM/results/demo-python/demo_transformer.txt", "lane": "reasoning_trajectories", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1577}
+{"id": "e7c27ea9ca4eefc6", "path": "/Users/adam/Development/SGJM/src/sgjm/eval/checkpoint.py", "lane": "code_long_context", "branch_conflict_score": 0.275, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2201}
+{"id": "226daa28a1948eb7", "path": "/Users/adam/Development/SGJM/results/demo-250m/demo_dataloader.txt", "lane": "reasoning_trajectories", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1632}
+{"id": "3a2806ee93eafa2c", "path": "/Users/adam/Development/SGJM/tests/test_250m_config.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.475, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 2965}
+{"id": "aa71a9acbc238c37", "path": "/Users/adam/Development/SGJM/src/sgjm/demo/generate.py", "lane": "code_long_context", "branch_conflict_score": 0.2, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 3021}
+{"id": "526a522369f1c940", "path": "/Users/adam/Development/SGJM/results/demo-250m/demo_load_config.txt", "lane": "reasoning_trajectories", "branch_conflict_score": 0.125, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1686}
+{"id": "4015a45e53c1a2f2", "path": "/Users/adam/Development/SGJM/src/sgjm/training/__main__.py", "lane": "code_long_context", "branch_conflict_score": 0.9, "latent_horizon": 4, "verifier_hardness": "hard", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 4101}
+{"id": "58fbb7b75baa5354", "path": "/Users/adam/Development/SGJM/src/sgjm/modules/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 342}
+{"id": "6b6839cd2742bb73", "path": "/Users/adam/Development/SGJM/results/sgjm-250m-mlx-run1/config.json", "lane": "code_long_context", "branch_conflict_score": 0.05, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1013}
+{"id": "296d99a26431a126", "path": "/Users/adam/Development/SGJM/scripts/autoresearch_sgjm.py", "lane": "code_long_context", "branch_conflict_score": 0.625, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "low", "contradiction_tag": "local", "n_chars": 7665}
+{"id": "6aad5a65d407121d", "path": "/Users/adam/Development/SGJM/src/sgjm/harness/metrics.py", "lane": "code_long_context", "branch_conflict_score": 0.175, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1469}
+{"id": "4b7d996013f20573", "path": "/Users/adam/Development/SGJM/src/sgjm/research/runner.py", "lane": "code_long_context", "branch_conflict_score": 0.8, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 10029}
+{"id": "ffd40696769f730d", "path": "/Users/adam/Development/SGJM/src/sgjm/research/cards.py", "lane": "code_long_context", "branch_conflict_score": 0.125, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1388}
+{"id": "d5494492a1a1a8f9", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/merge_radius/summary.json", "lane": "code_long_context", "branch_conflict_score": 0.125, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 10555}
+{"id": "014adb1d3ef93459", "path": "/Users/adam/Development/SGJM/results/autoresearch/papers_20260515_051230.json", "lane": "code_long_context", "branch_conflict_score": 0.95, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "low", "contradiction_tag": "local", "n_chars": 20453}
+{"id": "61c35742ad2a50bf", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/merge_radius/merge_r4.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1825}
+{"id": "fff0efc211c2866d", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/merge_radius/merge_r8.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1881}
+{"id": "d2cd00251ccc5d85", "path": "/Users/adam/Development/SGJM/tests/test_branch.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.625, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 2065}
+{"id": "bda9269faa79726c", "path": "/Users/adam/Development/SGJM/src/sgjm/bench/mlx_bench.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.15, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 8058}
+{"id": "4b15183b8396c15a", "path": "/Users/adam/Development/SGJM/tests/test_address.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.2, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1137}
+{"id": "606ad997d5014157", "path": "/Users/adam/Development/SGJM/results/autoresearch/SGJM_DATASET_AND_TRAINING_OPTIMIZATION_2026-05-15.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.525, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "low", "contradiction_tag": "local", "n_chars": 5430}
+{"id": "2f04538bb1811a24", "path": "/Users/adam/Development/SGJM/tests/test_training_torch.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.325, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2834}
+{"id": "3441414ff5081a7c", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/loss_weight/jepa_w_0.25.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1864}
+{"id": "5659bd501a5f97ff", "path": "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/baseline.py", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 968}
+{"id": "63d1814c949356f5", "path": "/Users/adam/Development/SGJM/results/sgjm-25m-mlx-run1/README.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.1, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 2422}
+{"id": "36001a2f08634ca5", "path": "/Users/adam/Development/SGJM/BLOG.md", "lane": "reasoning_trajectories", "branch_conflict_score": 1.0, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "low", "contradiction_tag": "local", "n_chars": 14433}
+{"id": "d7657d1c38b9c929", "path": "/Users/adam/Development/SGJM/results/phase5-ablation-25m-mlx/sgjm_no_verifier.json", "lane": "code_long_context", "branch_conflict_score": 0.1, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1910}
+{"id": "048e1a491011f286", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/loss_weight/jepa_w_1.0.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1807}
+{"id": "4611da505acd1964", "path": "/Users/adam/Development/SGJM/results/phase5-eval-gate/gate_report.json", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 759}
+{"id": "42bf4e130f670398", "path": "/Users/adam/Development/SGJM/results/sgjm-25m-mlx-run1/config.json", "lane": "code_long_context", "branch_conflict_score": 0.05, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 994}
+{"id": "3cb417058a9459fc", "path": "/Users/adam/Development/SGJM/tests/test_eval_mlx.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.2, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2842}
+{"id": "38aa6640d4696cf7", "path": "/Users/adam/Development/SGJM/results/phase5-ablation-25m-mlx/sgjm_no_drafter.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1840}
+{"id": "2c422d91742851ee", "path": "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/model.py", "lane": "code_long_context", "branch_conflict_score": 0.225, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 5291}
+{"id": "4bf5fdce10328fdc", "path": "/Users/adam/Development/SGJM/src/sgjm/harness/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 212}
+{"id": "f6a77ca93c95fdd9", "path": "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 208}
+{"id": "9fa1d0ac3f3c1a8a", "path": "/Users/adam/Development/SGJM/tests/test_mlx_checkpoint.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.25, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2314}
+{"id": "cdac8244abe032ab", "path": "/Users/adam/Development/SGJM/src/sgjm/branch/verifier.py", "lane": "code_long_context", "branch_conflict_score": 0.225, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 848}
+{"id": "119c91b922256f81", "path": "/Users/adam/Development/SGJM/results/sgjm-250m-mlx-run1/README.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.275, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 6757}
+{"id": "09484aec42828143", "path": "/Users/adam/Development/SGJM/src/sgjm/graph/address.py", "lane": "code_long_context", "branch_conflict_score": 0.275, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2683}
+{"id": "31c8c5a4a6c58828", "path": "/Users/adam/Development/SGJM/tests/test_eval.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.55, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 4811}
+{"id": "4e10115335981f05", "path": "/Users/adam/Development/SGJM/pyproject.toml", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1088}
+{"id": "3ecf6e90d5378775", "path": "/Users/adam/Development/SGJM/src/sgjm/eval/__main__.py", "lane": "code_long_context", "branch_conflict_score": 0.325, "latent_horizon": 8, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 6600}
+{"id": "3c992a03fd6daf6d", "path": "/Users/adam/Development/SGJM/src/sgjm/graph/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 268}
+{"id": "472a9c370f01006d", "path": "/Users/adam/Development/SGJM/results/demo-250m/demo_fibonacci.txt", "lane": "reasoning_trajectories", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1423}
+{"id": "8dc923e58c424a26", "path": "/Users/adam/Development/SGJM/src/sgjm/eval/__init__.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 379}
+{"id": "57bee687ec1ad2bb", "path": "/Users/adam/Development/SGJM/results/sgjm-100m-mlx-run1/config.json", "lane": "code_long_context", "branch_conflict_score": 0.05, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 998}
+{"id": "f482b14a3c11abb5", "path": "/Users/adam/Development/SGJM/src/sgjm/graph/manager.py", "lane": "code_long_context", "branch_conflict_score": 0.425, "latent_horizon": 8, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 5811}
+{"id": "193b8be1dc3a4995", "path": "/Users/adam/Development/SGJM/src/sgjm/training/data.py", "lane": "code_long_context", "branch_conflict_score": 1.0, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 6643}
+{"id": "691d38a42749369d", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/block_size/summary.json", "lane": "code_long_context", "branch_conflict_score": 0.075, "latent_horizon": 8, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 6272}
+{"id": "117191d69c8b5e4e", "path": "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/losses.py", "lane": "code_long_context", "branch_conflict_score": 0.275, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2865}
+{"id": "76b50e21a41e6d18", "path": "/Users/adam/Development/SGJM/src/sgjm/branch/lifecycle.py", "lane": "code_long_context", "branch_conflict_score": 0.35, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2490}
+{"id": "f62011d481c0bcec", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/loss_weight/jepa_w_4.0.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1911}
+{"id": "3d28e3f865de9054", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/block_size/block_8.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1817}
+{"id": "2294a641d7ad15a0", "path": "/Users/adam/Development/SGJM/src/sgjm/graph/node.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 2, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 833}
+{"id": "799b32ef6e888869", "path": "/Users/adam/Development/SGJM/src/sgjm/modules/drafter.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1376}
+{"id": "77ad3725846be4dc", "path": "/Users/adam/Development/SGJM/tests/test_training_config.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.65, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 2137}
+{"id": "78f61174fb50b096", "path": "/Users/adam/Development/SGJM/results/sgjm-100m-mlx-run1/README.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.05, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "medium", "contradiction_tag": "none", "n_chars": 2627}
+{"id": "de1674324db75dab", "path": "/Users/adam/Development/SGJM/src/sgjm/branch/policy.py", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1189}
+{"id": "e9e9178708851b9f", "path": "/Users/adam/Development/SGJM/tests/test_research.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.625, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 2875}
+{"id": "932258b326dff6c7", "path": "/Users/adam/Development/SGJM/src/sgjm/research/__main__.py", "lane": "code_long_context", "branch_conflict_score": 0.475, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 3417}
+{"id": "fb18eaacab40bdc3", "path": "/Users/adam/Development/SGJM/src/sgjm/eval/metrics.py", "lane": "code_long_context", "branch_conflict_score": 0.525, "latent_horizon": 8, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 10527}
+{"id": "1bf5759214923025", "path": "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/model.py", "lane": "code_long_context", "branch_conflict_score": 0.45, "latent_horizon": 8, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 6899}
+{"id": "378e3f482f53a57a", "path": "/Users/adam/Development/SGJM/tests/test_mlx_baseline.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.2, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1996}
+{"id": "51708d1e522d8498", "path": "/Users/adam/Development/SGJM/src/sgjm/modules/backbone.py", "lane": "code_long_context", "branch_conflict_score": 0.0, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1293}
+{"id": "7e895099f18a7cc8", "path": "/Users/adam/Development/SGJM/tests/test_graph.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.25, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1354}
+{"id": "3fe51af7e25dbdee", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/block_size/block_2.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1818}
+{"id": "4e425fcc73e06753", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/loss_weight/jepa_w_0.0.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 2019}
+{"id": "e54596077c23e52f", "path": "/Users/adam/Development/SGJM/results/phase5-sweeps/merge_radius/merge_r6.json", "lane": "code_long_context", "branch_conflict_score": 0.025, "latent_horizon": 4, "verifier_hardness": "easy", "mergeability_bucket": "high", "contradiction_tag": "none", "n_chars": 1886}
+{"id": "4f01202273488761", "path": "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/trainer.py", "lane": "code_long_context", "branch_conflict_score": 0.45, "latent_horizon": 8, "verifier_hardness": "medium", "mergeability_bucket": "medium", "contradiction_tag": "local", "n_chars": 7760}
+{"id": "1d63370f86c9fa68", "path": "/Users/adam/Development/SGJM/README.md", "lane": "reasoning_trajectories", "branch_conflict_score": 0.675, "latent_horizon": 8, "verifier_hardness": "hard", "mergeability_bucket": "low", "contradiction_tag": "local", "n_chars": 18247}
+{"id": "41df96efad6fd33b", "path": "/Users/adam/Development/SGJM/tests/test_bench_mlx.py", "lane": "adversarial_branch_conflict", "branch_conflict_score": 0.45, "latent_horizon": 4, "verifier_hardness": "medium", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 2594}
+{"id": "a67322702ab60fac", "path": "/Users/adam/Development/SGJM/src/sgjm/training/backends.py", "lane": "code_long_context", "branch_conflict_score": 0.45, "latent_horizon": 4, "verifier_hardness": "hard", "mergeability_bucket": "high", "contradiction_tag": "local", "n_chars": 1832}

pyproject.toml

@@ -0,0 +1,49 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "sgjm"
+version = "2026.6.5"
+description = "Speculative Graph JEPA Model — hybrid Mamba-2/attention speculative decoder, MLX + PyTorch (CUDA/ROCm) training"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "Apache-2.0"
+license-files = ["LICENSE", "NOTICE"]
+authors = [{ name = "Adam Pippert", email = "adam.pippert@gmail.com" }]
+keywords = ["speculative-decoding", "mamba", "jepa", "language-model", "mlx", "pytorch"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = []
+
+[project.urls]
+Homepage = "https://github.com/AdamPippert/SGJM"
+Repository = "https://github.com/AdamPippert/SGJM"
+"Model Card" = "https://huggingface.co/adampippert/SGJM"
+
+[project.optional-dependencies]
+# Pick exactly one backend extra plus dev:
+#   pip install -e '.[cpu,dev]'                         # any platform, slow
+#   pip install -e '.[cuda,dev]'                        # NVIDIA + CUDA wheels (default PyPI)
+#   pip install -e '.[mlx,dev]'                         # macOS Apple Silicon (M1/M2/M3/M4)
+# ROCm/Strix Halo: do NOT use the [rocm] extra alone — torch ROCm wheels live
+# on a separate index. Install with:
+#   pip install --index-url https://download.pytorch.org/whl/rocm6.2 torch
+#   pip install -e '.[rocm,dev]'
+cpu = ["torch>=2.4", "numpy>=1.26"]
+cuda = ["torch>=2.4", "numpy>=1.26"]
+rocm = ["numpy>=1.26"]
+mlx = ["mlx>=0.18", "numpy>=1.26"]
+dev = ["pytest>=8.0"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"

results/autoresearch/SGJM_DATASET_AND_TRAINING_OPTIMIZATION_2026-05-15.md

@@ -0,0 +1,137 @@
+# SGJM Dataset + Training Optimization Plan (25M, 250M, 1B)
+
+## Scope
+This plan satisfies two optimization tracks:
+1) dataset optimization tailored to SGJM capabilities
+2) training-method optimization for retraining 25M, 250M, and upcoming 1B
+
+Grounding inputs:
+- results/phase5-eval-gate/gate_report.json
+- results/phase5-ablation-25m-mlx/summary.json
+- results/phase5-sweeps/{loss_weight,block_size,merge_radius}/summary.json
+- results/autoresearch/sgjm_autoresearch_20260515_051230.md
+
+## First-principles diagnosis
+- SGJM wins on compute-per-accepted-token and verifier/merge behavior.
+- Current datasets are too small and too easy for measuring SGJM-specific benefits at scale.
+- 25M run uses 1 MiB corpus; 250M run plateaus on 32 MiB corpus.
+- JEPA + verifier are load-bearing; optimization must directly stress branch disagreement and latent predictability.
+
+---
+
+## Optimization Track A: Optimal SGJM dataset design
+
+### A1. Dataset objective
+Maximize signal for SGJM’s unique modules:
+- drafter quality under branch diversity
+- JEPA latent forecast quality under long horizons
+- verifier discrimination under near-miss branches
+- merge precision under semantically equivalent but lexically different candidates
+
+### A2. Mixture blueprint (by token share)
+- 40% Code long-context corpus
+  - language-balanced: Python, TS/JS, Go, Rust
+  - include multi-file projects, tests, docs, and refactor commits
+- 35% Reasoning trajectories
+  - chain-like derivations, theorem/program proof sketches, structured planning traces
+  - include positive and corrected-negative trajectories
+- 25% Adversarial branch-conflict corpus
+  - pairs/sets of continuations where top-1 token probability is misleading
+  - near-duplicate semantics with lexical variance for merge pressure
+
+### A3. SGJM-specific annotations per sample
+- branch_conflict_score (0-1)
+- latent_horizon (how many steps till disambiguation)
+- verifier_hardness bucket (easy/medium/hard)
+- mergeability signature (high/medium/low)
+- contradiction tag (none/local/global)
+
+### A4. Token budgets
+- 25M retrain: 8B–12B tokens
+- 250M retrain: 35B–60B tokens
+- 1B pretrain/retrain: 120B–220B tokens
+
+### A5. Curriculum schedule
+- Phase D1 (stability): block_size=2-heavy samples, low conflict
+- Phase D2 (diversity): mix in block_size=4, medium conflict
+- Phase D3 (stress): 10-15% block_size=8 hard cases, high conflict, contradiction-heavy
+
+### A6. Data quality gates (reject if failed)
+- duplicate n-gram rate < 2%
+- contamination checks against held-out eval suites
+- per-source perplexity sanity bounds
+- adversarial split actually increases branch_conflict_score distribution mean by >= 30%
+
+---
+
+## Optimization Track B: Training methodology
+
+### B1. Global method (all model sizes)
+- Stage 1 (stabilization): token + JEPA only
+  - loss weights: token=1.0, drafter=0.3, jepa ramp 0.05 -> 0.2, verifier=0.0
+- Stage 2 (full SGJM): enable verifier + full drafter
+  - loss weights: token=1.0, drafter=0.5 (decay to 0.35 after accept>=0.7), jepa=0.25, verifier ramp 0.02 -> 0.1
+- Stage 3 (efficiency tuning): acceptance/merge co-optimization
+  - enforce block-size curriculum and merge-radius tuning per checkpoint
+
+Adaptive controls:
+- If accept_rate < threshold for 3 evals: LR *= 0.8 and increase hard-negative mining
+- If merge_advantage stagnates for 5 evals: increase adversarial batch share by +10%
+- If token_nll regresses while accept rises: lower verifier weight by 20% and increase token CE share
+
+### B2. 25M retrain profile
+Targets:
+- token_nll <= baseline + 0.03
+- accept_rate >= 0.70
+- merge_advantage >= 2.0
+- compute_advantage >= 4.0
+
+Config direction:
+- block_size default 2 (from sweep signal)
+- seq_len 512 -> 1024 progressive
+- warmup 2-3% of total steps
+- aggressive regularization: dropout 0.05-0.1, weight decay 0.1-0.15
+
+### B3. 250M retrain profile
+Targets:
+- token_nll <= baseline + 0.025
+- accept_rate >= 0.75
+- merge_advantage >= 2.5
+- compute_advantage >= 6.0
+
+Config direction:
+- data scale 35B+ tokens mandatory (current 32 MiB is non-starter)
+- long-context emphasis (2k context by mid-run)
+- block_size 2 for early/mid, partial 4 in late curriculum
+- checkpoint triage every 1k steps using SGJM gate metrics, not just NLL
+
+### B4. 1B training profile
+Targets:
+- token_nll <= baseline + 0.02
+- accept_rate >= 0.80
+- merge_advantage >= 3.0
+- compute_advantage >= 8.0
+
+Config direction:
+- start from new preset: size=1b
+- distributed: FSDP or ZeRO-3, bf16, grad checkpointing, flash attention
+- architecture: GQA/MQA optional, RoPE scaling, context extension schedule 2k -> 4k -> 8k
+- training plan: 3 phases with increasing conflict/hard-negative exposure
+
+---
+
+## Immediate execution checklist
+1) Build dataset manifest + sampler that emits SGJM annotations.
+2) Add curriculum-aware dataloader knobs (block curriculum + hard-negative ratio).
+3) Add 1B config preset and CLI support (completed in this work).
+4) Run pilot retrains:
+   - 25M: 3 short runs to calibrate acceptance dynamics
+   - 250M: 2 medium runs on expanded corpus
+   - 1B: smoke + infrastructure validation, then full schedule
+5) Compare all runs using existing gate metrics + new conflict-stratified slices.
+
+## Exit criteria for “tasks satisfied today”
+- AutoResearch run completed against SGJM variants (done)
+- Dataset optimization spec completed (done)
+- Training methodology for 25M/250M/1B completed (done)
+- Repo artifacts produced and test for 1B config added (done)

results/autoresearch/latest_manifest.json

@@ -0,0 +1,25 @@
+{
+  "generated_at": "2026-05-15T12:12:30.107603+00:00",
+  "report": "/Users/adam/Development/SGJM/results/autoresearch/sgjm_autoresearch_20260515_051230.md",
+  "papers": "/Users/adam/Development/SGJM/results/autoresearch/papers_20260515_051230.json",
+  "variants": [
+    {
+      "name": "sgjm-25m",
+      "params_m": 25.0,
+      "data_source": "auto",
+      "corpus_bytes": 1048576,
+      "best_token_nll": 0.024779903003945947,
+      "best_accept_rate": 0.9999302455357143,
+      "notes": "Gate-pass config; tiny corpus, likely saturated; near-perfect acceptance."
+    },
+    {
+      "name": "sgjm-250m",
+      "params_m": 251.0,
+      "data_source": "python_extended",
+      "corpus_bytes": 33554432,
+      "best_token_nll": 0.889,
+      "best_accept_rate": 0.991,
+      "notes": "Converged then plateaued at 32 MiB python corpus capacity ceiling."
+    }
+  ]
+}

results/autoresearch/papers_20260515_051230.json

@@ -0,0 +1,242 @@
+[
+  {
+    "category": "cs.AI",
+    "title": "Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents",
+    "link": "https://arxiv.org/abs/2605.12620",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12620v1 Announce Type: new Abstract: Building generalist embodied agents capable of solving complex real-world tasks remains a fundamental challenge in AI. Multimodal Large Language Models (MLLMs) have significantly advanced the reasoning capabilities of such agents through strong vision-language knowledge and chain-of-thought (CoT) reasoning, yet remain brittle when faced with challenging out-of-distribut",
+    "score": 4.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agents",
+    "link": "https://arxiv.org/abs/2605.13941",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13941v1 Announce Type: new Abstract: Long-term memory is essential for LLM agents that operate across multiple sessions, yet existing memory systems treat retrieval infrastructure as fixed: stored content evolves while scoring functions, fusion strategies, and answer-generation policies remain frozen at deployment. We argue that truly adaptive memory requires co-evolution at two levels: the stored knowledg",
+    "score": 3.2018
+  },
+  {
+    "category": "cs.AI",
+    "title": "Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack",
+    "link": "https://arxiv.org/abs/2605.12673",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12673v1 Announce Type: new Abstract: Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges spontaneously in frontier models without overfitting. We argue that benchmarks must be secure by design. From past incidents of reward hacks,",
+    "score": 3.1018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction",
+    "link": "https://arxiv.org/abs/2605.13950",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13950v1 Announce Type: new Abstract: Autonomous language-model agents are increasingly evaluated on long-horizon tool-use tasks, but existing benchmarks rarely capture the complexity and nuance of real scientific work. To address this gap, we introduce Collider-Bench, a benchmark for evaluating whether LLM agents can reproduce experimental analyses from the Large Hadron Collider (LHC) using only public pap",
+    "score": 3.1018
+  },
+  {
+    "category": "cs.AI",
+    "title": "Macro-Action Based Multi-Agent Instruction Following through Value Cancellation",
+    "link": "https://arxiv.org/abs/2605.12655",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12655v1 Announce Type: new Abstract: Multi-agent reinforcement learning (MARL) in real-world use cases may need to adapt to external natural language instructions that interrupt ongoing behavior and conflict with long-horizon objectives. However, conditioning rewards on instructions introduces a fundamental failure mode as Bellman updates couple value estimates across instruction contexts, leading to incon",
+    "score": 2.0018
+  },
+  {
+    "category": "cs.AI",
+    "title": "CHAL: Council of Hierarchical Agentic Language",
+    "link": "https://arxiv.org/abs/2605.12718",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12718v1 Announce Type: new Abstract: Multi-agent debate has emerged as a promising approach for improving LLM reasoning on ground-truth tasks, yet current methodologies face certain structural limitations: debate tends to induce a martingale over belief trajectories, majority voting accounts for most observed gains, and LLMs exhibit confidence escalation rather than calibration across rounds. We argue that",
+    "score": 2.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Dual Hierarchical Dialogue Policy Learning for Legal Inquisitive Conversational Agents",
+    "link": "https://arxiv.org/abs/2605.14057",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14057v1 Announce Type: new Abstract: Most existing dialogue systems are user-driven, primarily designed to fulfill user requests. However, in many critical real-world scenarios, a conversational agent must proactively extract information to achieve its own objectives rather than merely respond. To address this gap, we introduce \\emph{Inquisitive Conversational Agents (ICAs)} and develop an ICA specifically",
+    "score": 2.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning",
+    "link": "https://arxiv.org/abs/2605.14040",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14040v1 Announce Type: new Abstract: We audit the multimodal-physics evaluation pipeline end-to-end and document three undetected construction practices that distort how the field measures vision-language reasoning: train-eval contamination, translation drift, and MCQ saturation. (1) Public training pools (UGPhysics-Train, SciInstruct, MMK12) pass single-stage 5-gram-Jaccard audits with zero hits across al",
+    "score": 1.5018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding",
+    "link": "https://arxiv.org/abs/2605.14005",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14005v1 Announce Type: new Abstract: Speculative decoding has become a widely adopted technique for accelerating large language model (LLM) inference by drafting multiple candidate tokens and verifying them with a target model in parallel. Its efficiency, however, critically depends on the average accepted length $\\tau$, i.e., how many draft tokens survive each verification step. In this work, we identify ",
+    "score": 1.3018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation",
+    "link": "https://arxiv.org/abs/2605.14053",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14053v1 Announce Type: new Abstract: The application of Large Language Models to Question Answering has shown great promise, but important challenges such as hallucinations and erroneous reasoning arise when using these models, particularly in knowledge-intensive, domain-specific tasks. To address these issues, we introduce Derivation Prompting, a novel prompting technique for the generation step of the Re",
+    "score": 1.3018
+  },
+  {
+    "category": "cs.CL",
+    "title": "When Evidence Conflicts: Uncertainty and Order Effects in Retrieval-Augmented Biomedical Question Answering",
+    "link": "https://arxiv.org/abs/2605.14115",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14115v1 Announce Type: new Abstract: Biomedical retrieval-augmented large language models (LLMs) often face evidence that is incomplete, misleading, or internally contradictory, yet evaluation usually emphasizes answer accuracy under helpful context rather than reliability under conflict. Using HealthContradict, we evaluate six open-weight LLMs under five controlled evidence conditions: no retrieved contex",
+    "score": 1.3018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning",
+    "link": "https://arxiv.org/abs/2605.13936",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13936v1 Announce Type: new Abstract: The recent success of large language models (LLMs) has been largely driven by vast public datasets. However, the next frontier for LLM development lies beyond public data. Much of the world's most valuable information is private, especially in highly regulated sectors such as healthcare and finance, where data include patient histories or customer communications. Unlock",
+    "score": 1.1018
+  },
+  {
+    "category": "cs.AI",
+    "title": "Revealing Interpretable Failure Modes of VLMs",
+    "link": "https://arxiv.org/abs/2605.12674",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12674v1 Announce Type: new Abstract: Vision-Language Models (VLMs) are increasingly used in safety-critical applications because of their broad reasoning capabilities and ability to generalize with minimal task-specific engineering. Despite these advantages, they can exhibit catastrophic failures in specific real-world situations, constituting failure modes. We introduce REVELIO, a framework for systematic",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.AI",
+    "title": "Learning Transferable Latent User Preferences for Human-Aligned Decision Making",
+    "link": "https://arxiv.org/abs/2605.12682",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12682v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as reasoning modules in many applications. While they are efficient in certain tasks, LLMs often struggle to produce human-aligned solutions. Human-aligned decision making requires accounting for both explicitly stated goals and latent user preferences that shape how ambiguous situations should be resolved. Existing app",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.AI",
+    "title": "On the Size Complexity and Decidability of First-Order Progression",
+    "link": "https://arxiv.org/abs/2605.12691",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12691v1 Announce Type: new Abstract: Progression, the task of updating a knowledge base to reflect action effects, generally requires second-order logic. Identifying first-order special cases, by restricting either the knowledge base or action effects, has long been a central topic in reasoning about actions. It is known that local-effect, normal, and acyclic actions, three increasingly expressive classes,",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.AI",
+    "title": "DisaBench: A Participatory Evaluation Framework for Disability Harms in Language Models",
+    "link": "https://arxiv.org/abs/2605.12702",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12702v1 Announce Type: new Abstract: General-purpose safety benchmarks for large language models do not adequately evaluate disability-related harms. We introduce DisaBench: a taxonomy of twelve disability harm categories co-created with people with disabilities and red teaming experts, a taxonomy-driven evaluation methodology that pairs benign and adversarial prompts across seven life domains, and a datas",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.AI",
+    "title": "BEHAVE: A Hybrid AI Framework for Real-Time Modeling of Collective Human Dynamics",
+    "link": "https://arxiv.org/abs/2605.12730",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12730v1 Announce Type: new Abstract: Existing AI systems for modeling human behavior operate at the level of individuals or detect events after they occur. As a result, they systematically fail to capture the collective dynamics that determine whether a group remains stable or transitions into escalation or breakdown. We propose a different foundation: a group of interacting humans constitutes a complex dy",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.AI",
+    "title": "State-Centric Decision Process",
+    "link": "https://arxiv.org/abs/2605.12755",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.12755v1 Announce Type: new Abstract: Language environments such as web browsers, code terminals, and interactive simulations emit raw text rather than states, and provide none of the runtime structure that MDP analysis requires. No explicit state space, no observation-to-state mapping, no certified transitions, and no termination criterion. We introduce the State-Centric Decision Process (SDP), a runtime f",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey",
+    "link": "https://arxiv.org/abs/2605.13919",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13919v1 Announce Type: new Abstract: Multilingual knowledge editing (MKE) remains challenging because language-specific edits interfere with one another, even when locate-then-edit methods work well in monolingual settings. This paper focuses on three issues: the effectiveness of vector merging methods for MKE, the extent to which Task Singular Vectors for Merging (TSVM) can reduce multilingual interferenc",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model with Curriculum Learning and Native Tool Use",
+    "link": "https://arxiv.org/abs/2605.13989",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13989v1 Announce Type: new Abstract: We present VectraYX-Nano, a 41.95M-parameter decoder-only language model trained from scratch in Spanish for cybersecurity, with a Latin-American focus and native tool invocation via the Model Context Protocol (MCP). Four contributions: (i) Corpus: VectraYX-Sec-ES, a 170M-token Spanish corpus from an eight-VM pipeline (~$25 USD) partitioned into conversational (42M toke",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts",
+    "link": "https://arxiv.org/abs/2605.14055",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14055v1 Announce Type: new Abstract: Parameter-Efficient Fine-Tuning (PEFT) is widely used for adapting Large Language Models (LLMs) for various tasks. Recently, there has been an increasing demand for fine-tuning a single LLM for multiple tasks because it requires overall less data for fine-tuning thanks to the common features shared among tasks. More importantly, LLMs are resource demanding and deploying",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Distribution Corrected Offline Data Distillation for Large Language Models",
+    "link": "https://arxiv.org/abs/2605.14071",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14071v1 Announce Type: new Abstract: Distilling reasoning traces from strong large language models into smaller ones is a promising route to improve intelligence in resource-constrained settings. Existing approaches face a fundamental trade-off: offline distillation from teacher-generated traces provides high-quality, sample-efficient supervision but suffers from distributional drift: during training, the ",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.CL",
+    "title": "Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study",
+    "link": "https://arxiv.org/abs/2605.14087",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14087v1 Announce Type: new Abstract: Large Language Models (LLMs), when trained on web-scale corpora, inherently absorb toxic patterns from their training data. This leads to ``toxic degeneration'' where even innocuous prompts can trigger harmful outputs. This phenomenon poses significant risks for real-world deployments. Thus, necessitating effective mitigation strategies that should maintain model utilit",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Vision-Based Runtime Monitoring under Varying Specifications using Semantic Latent Representations",
+    "link": "https://arxiv.org/abs/2605.13923",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13923v1 Announce Type: new Abstract: We study certified runtime monitoring of past-time signal temporal logic (ptSTL) from visual observations under partial observability. The monitor must infer safety-relevant quantities from images and provide finite-sample guarantees, while being \\emph{reusable}: once trained and calibrated, it should certify any formula in a target fragment without per-formula retraini",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders",
+    "link": "https://arxiv.org/abs/2605.13930",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13930v1 Announce Type: new Abstract: EEG foundation models achieve state-of-the-art clinical performance, yet the internal computations driving their predictions remain opaque: a barrier to clinical trust. We apply TopK Sparse Autoencoders (SAEs) across three architecturally distinct EEG transformers: SleepFM, REVE, and LaBraM to extract sparse feature dictionaries from their embeddings. By grounding these",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Rethinking Molecular OOD Generalization via Target-Aware Source Selection",
+    "link": "https://arxiv.org/abs/2605.13932",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13932v1 Announce Type: new Abstract: Robust prediction of molecular properties under extreme out-of-distribution (OOD) scenarios is a pivotal bottleneck in AI-driven drug discovery. Current scaffold-splitting protocols fail to obstruct microscopic semantic overlap, predisposing models to shortcut learning and overestimating their true extrapolation capability; meanwhile, conventional domain adaptation para",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models",
+    "link": "https://arxiv.org/abs/2605.13935",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13935v1 Announce Type: new Abstract: Diffusion language models are a promising alternative to autoregressive models, yet post-training methods for them largely adapt reward-maximizing objectives. We identify a central failure mode in this setting we call trajectory locking: sampled reward-driven updates over-concentrate probability mass onto a narrow set of denoising paths, reducing coverage of alternative",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "TabPFN-3: Technical Report",
+    "link": "https://arxiv.org/abs/2605.13986",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13986v1 Announce Type: new Abstract: Tabular data underpins most high-value prediction problems in science and industry, and TabPFN has driven the foundation model revolution for this modality. Designed with feedback from our users, TabPFN-3 builds on this foundation to scale state-of-the-art performance to datasets with 1M training rows and substantially reduce training and inference time. Pretrained excl",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Neural Fields for NV-Center Inverse Sensing",
+    "link": "https://arxiv.org/abs/2605.13988",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.13988v1 Announce Type: new Abstract: Inverse problems in scientific sensing are often solved with either hand-designed regularizers or supervised networks trained on simulated labels, yet both can fail when the forward model is nonlinear, spectrally coupled, and physically delicate. We study this issue for noise sensing based on nitrogen-vacancy (NV) centers in diamond, where a quantum sensor measures magn",
+    "score": 0.0018
+  },
+  {
+    "category": "cs.LG",
+    "title": "Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility",
+    "link": "https://arxiv.org/abs/2605.14037",
+    "published": "2026-05-15T00:00:00-04:00",
+    "summary": "arXiv:2605.14037v1 Announce Type: new Abstract: Under modern test-time compute and agentic paradigms, language models process ever-longer sequences. Efficient text generation with transformer architectures is increasingly constrained by the Key-Value cache memory footprint and bandwidth. To address this limitation, we introduce Self-Pruned Key-Value Attention (SP-KV), a mechanism designed to predict future KV utility",
+    "score": 0.0018
+  }
+]

results/autoresearch/sgjm_autoresearch_20260515_051230.md

@@ -0,0 +1,49 @@
+# SGJM AutoResearch Report
+
+Generated: 2026-05-15T12:12:30.107334+00:00
+
+## Variant Snapshot
+
+- sgjm-25m: params=25.0M, data=auto, corpus=1.0 MiB, best_token_nll=0.0248, accept=1.000. Gate-pass config; tiny corpus, likely saturated; near-perfect acceptance.
+- sgjm-250m: params=251.0M, data=python_extended, corpus=32.0 MiB, best_token_nll=0.8890, accept=0.991. Converged then plateaued at 32 MiB python corpus capacity ceiling.
+
+## Diagnosis (first principles)
+
+1) 25M is architecture-validated but data-understressed (tiny corpus, near-perfect acceptance).
+2) 250M is data-bottlenecked (32 MiB corpus ceiling), not capacity-limited.
+3) Merge precision and verifier utility are the real SGJM differentiators; dataset must stress branch disagreement, not just next-token CE.
+
+## Latest arXiv Signals (keyword-filtered)
+
+- [Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents](https://arxiv.org/abs/2605.12620) | cs.AI | score=4.0018
+- [EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agents](https://arxiv.org/abs/2605.13941) | cs.LG | score=3.2018
+- [Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack](https://arxiv.org/abs/2605.12673) | cs.AI | score=3.1018
+- [Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction](https://arxiv.org/abs/2605.13950) | cs.LG | score=3.1018
+- [Macro-Action Based Multi-Agent Instruction Following through Value Cancellation](https://arxiv.org/abs/2605.12655) | cs.AI | score=2.0018
+- [CHAL: Council of Hierarchical Agentic Language](https://arxiv.org/abs/2605.12718) | cs.AI | score=2.0018
+- [Dual Hierarchical Dialogue Policy Learning for Legal Inquisitive Conversational Agents](https://arxiv.org/abs/2605.14057) | cs.CL | score=2.0018
+- [Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning](https://arxiv.org/abs/2605.14040) | cs.CL | score=1.5018
+- [Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding](https://arxiv.org/abs/2605.14005) | cs.CL | score=1.3018
+- [Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation](https://arxiv.org/abs/2605.14053) | cs.CL | score=1.3018
+- [When Evidence Conflicts: Uncertainty and Order Effects in Retrieval-Augmented Biomedical Question Answering](https://arxiv.org/abs/2605.14115) | cs.CL | score=1.3018
+- [Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning](https://arxiv.org/abs/2605.13936) | cs.LG | score=1.1018
+
+## Optimization A: Dataset Design (capability-targeted)
+
+- Build a three-lane mixture with fixed weights: 40% code long-context, 35% reasoning trajectories, 25% adversarial branch-conflict samples.
+- Add SGJM-specific labels per sample: branch conflict score, latent transition smoothness, verifier hardness, mergeability bucket.
+- Curriculum by block length: start with block=2 tasks, then 30% block=4, finally 10% block=8 hard cases.
+- Data scale targets: 25M=8-12B tokens, 250M=35-60B, 1B=120-220B. Current corpora are orders of magnitude too small.
+
+## Optimization B: Training Method (25M/250M/1B)
+
+- Two-stage schedule: Stage-1 token+JEPA warm start, Stage-2 full SGJM with verifier anneal.
+- Dynamic loss weighting: jepa 0.05->0.25 ramp, verifier 0.0->0.1 ramp; keep drafter 0.5 until accept>0.6 then decay to 0.35.
+- Acceptance-controlled LR: if accept<0.45 for 3 evals, reduce LR 20% and increase verifier margin mining.
+- Model-size specifics: 25M prioritize robustness/regularization; 250M prioritize throughput+long context; 1B use FSDP/ZeRO + GQA + RoPE scaling and staged context extension.
+
+## Concrete Retrain Targets
+
+- 25M: token_nll <= baseline+0.03, accept>=0.70, merge_adv>=2.0, compute_adv>=4.0
+- 250M: token_nll <= baseline+0.025, accept>=0.75, merge_adv>=2.5, compute_adv>=6.0
+- 1B: token_nll <= baseline+0.02, accept>=0.80, merge_adv>=3.0, compute_adv>=8.0

results/demo-250m/demo_dataloader.txt

@@ -0,0 +1,38 @@
+[demo] loading checkpoint: runs/sgjm-250m/best.safetensors
+[demo] model: step=6500  d_model=1024  n_layers=14  block_size=4
+[demo] prompt: 'class DataLoader:\n    def __init__(self, dataset, batch_size=32):\n        '
+[demo] generating 160 tokens...
+
+────────────────────────────────────────────────────────────
+  Prompt
+────────────────────────────────────────────────────────────
+class DataLoader:
+    def __init__(self, dataset, batch_size=32):
+        
+────────────────────────────────────────────────────────────
+  Autoregressive (23.4 tok/s, 6.82s)
+────────────────────────────────────────────────────────────
+class DataLoader:
+    def __init__(self, dataset, batch_size=32):
+        self.dataset = dataset
+        self.dataset = dataset
+        self.dataset = dataset
+        self.dataset = dataset
+        self.dataset = dataset
+        self.
+
+────────────────────────────────────────────────────────────
+  Speculative (block=4, accept=100%, 25.2 tok/s, 6.36s)
+────────────────────────────────────────────────────────────
+class DataLoader:
+    def __init__(self, dataset, batch_size=32):
+        setup(= daaftee, badcass=se * 10))  #                                                                                                                           
+
+────────────────────────────────────────────────────────────
+  Comparison
+────────────────────────────────────────────────────────────
+Method                   Tokens  Time(s)    Tok/s
+────────────────────── ──────── ──────── ────────
+Autoregressive              160     6.82     23.4
+Speculative                 160     6.36     25.2  (1.07× speedup, 100% accept)
+

results/demo-250m/demo_fibonacci.txt

@@ -0,0 +1,30 @@
+[demo] loading checkpoint: runs/sgjm-250m/best.safetensors
+[demo] model: step=6500  d_model=1024  n_layers=14  block_size=4
+[demo] prompt: 'def fibonacci(n):\n    '
+[demo] generating 160 tokens...
+
+────────────────────────────────────────────────────────────
+  Prompt
+────────────────────────────────────────────────────────────
+def fibonacci(n):
+    
+────────────────────────────────────────────────────────────
+  Autoregressive (31.9 tok/s, 5.01s)
+────────────────────────────────────────────────────────────
+def fibonacci(n):
+                                                                                                                                                                    
+
+────────────────────────────────────────────────────────────
+  Speculative (block=4, accept=100%, 40.9 tok/s, 3.92s)
+────────────────────────────────────────────────────────────
+def fibonacci(n):
+                                                                                                                                                                    
+
+────────────────────────────────────────────────────────────
+  Comparison
+────────────────────────────────────────────────────────────
+Method                   Tokens  Time(s)    Tok/s
+────────────────────── ──────── ──────── ────────
+Autoregressive              160     5.01     31.9
+Speculative                 160     3.92     40.9  (1.28× speedup, 100% accept)
+

results/demo-250m/demo_load_config.txt

@@ -0,0 +1,49 @@
+[demo] loading checkpoint: runs/sgjm-250m/best.safetensors
+[demo] model: step=6500  d_model=1024  n_layers=14  block_size=4
+[demo] prompt: 'import json\n\ndef load_config(path):\n    """Load configuration from a JSON file."""\n    '
+[demo] generating 160 tokens...
+
+────────────────────────────────────────────────────────────
+  Prompt
+────────────────────────────────────────────────────────────
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    
+────────────────────────────────────────────────────────────
+  Autoregressive (23.3 tok/s, 6.88s)
+────────────────────────────────────────────────────────────
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    if path is None:
+        return path
+    if path is None:
+        return path
+    if path is None:
+        return path
+    if path is None:
+        return path
+
+
+────────────────────────────────────────────────────────────
+  Speculative (block=4, accept=100%, 7.2 tok/s, 22.16s)
+────────────────────────────────────────────────────────────
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    if upia is None:
+        return pane((
+            "Cone oo con fo  poe io  po  po  io  po  po  po  po  p   po  "   "   "   "   "   "   "   "   "   "   "   "   
+
+────────────────────────────────────────────────────────────
+  Comparison
+────────────────────────────────────────────────────────────
+Method                   Tokens  Time(s)    Tok/s
+────────────────────── ──────── ──────── ────────
+Autoregressive              160     6.88     23.3
+Speculative                 160    22.16      7.2  (0.31× speedup, 100% accept)
+

results/demo-python/demo_fibonacci.txt

@@ -0,0 +1,31 @@
+[demo] loading checkpoint: runs/sgjm-python-25m/best.safetensors
+[demo] model: step=2000  d_model=384  n_layers=10  block_size=4
+[demo] prompt: 'def fibonacci(n):\n    '
+[demo] generating 128 tokens...
+
+────────────────────────────────────────────────────────────
+  Prompt
+────────────────────────────────────────────────────────────
+def fibonacci(n):
+    
+────────────────────────────────────────────────────────────
+  Autoregressive (236.1 tok/s, 0.54s)
+────────────────────────────────────────────────────────────
+def fibonacci(n):
+                raise ValueError("ISO week directive '%V' is incompatible with "
+                                                   
+
+────────────────────────────────────────────────────────────
+  Speculative (block=4, accept=100%, 125.1 tok/s, 1.02s)
+────────────────────────────────────────────────────────────
+def fibonacci(n):
+                r   i   i                                                                                                           
+
+────────────────────────────────────────────────────────────
+  Comparison
+────────────────────────────────────────────────────────────
+Method                   Tokens  Time(s)    Tok/s
+────────────────────── ──────── ──────── ────────
+Autoregressive              128     0.54    236.1
+Speculative                 128     1.02    125.1  (0.53× speedup, 100% accept)
+

results/demo-python/demo_load_config.txt

@@ -0,0 +1,40 @@
+[demo] loading checkpoint: runs/sgjm-python-25m/best.safetensors
+[demo] model: step=2000  d_model=384  n_layers=10  block_size=4
+[demo] prompt: 'import json\n\ndef load_config(path):\n    """Load configuration from a JSON file."""\n    '
+[demo] generating 128 tokens...
+
+────────────────────────────────────────────────────────────
+  Prompt
+────────────────────────────────────────────────────────────
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    
+────────────────────────────────────────────────────────────
+  Autoregressive (202.7 tok/s, 0.63s)
+────────────────────────────────────────────────────────────
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    # The Parse all are nounded build all archs to allowed be used ben absen are
+    # an expressed because the aren the aren not pr
+
+────────────────────────────────────────────────────────────
+  Speculative (block=4, accept=100%, 139.5 tok/s, 0.92s)
+────────────────────────────────────────────────────────────
+import json
+
+def load_config(path):
+    """Load configuration from a JSON file."""
+    #                                                                                                                               
+
+────────────────────────────────────────────────────────────
+  Comparison
+────────────────────────────────────────────────────────────
+Method                   Tokens  Time(s)    Tok/s
+────────────────────── ──────── ──────── ────────
+Autoregressive              128     0.63    202.7
+Speculative                 128     0.92    139.5  (0.69× speedup, 100% accept)
+

results/demo-python/demo_transformer.txt

@@ -0,0 +1,36 @@
+[demo] loading checkpoint: runs/sgjm-python-25m/best.safetensors
+[demo] model: step=2000  d_model=384  n_layers=10  block_size=4
+[demo] prompt: 'class Transformer(nn.Module):\n    def __init__(self, d_model=512):\n        '
+[demo] generating 128 tokens...
+
+────────────────────────────────────────────────────────────
+  Prompt
+────────────────────────────────────────────────────────────
+class Transformer(nn.Module):
+    def __init__(self, d_model=512):
+        
+────────────────────────────────────────────────────────────
+  Autoregressive (209.2 tok/s, 0.61s)
+────────────────────────────────────────────────────────────
+class Transformer(nn.Module):
+    def __init__(self, d_model=512):
+        return self.__name__ == name * (days2, 256))
+
+    def __name__(self, days=0, days=0, hour, minutes=0, seconds=0, microseconds=0,
+
+────────────────────────────────────────────────────────────
+  Speculative (block=4, accept=94%, 140.1 tok/s, 0.91s)
+────────────────────────────────────────────────────────────
+class Transformer(nn.Module):
+    def __init__(self, d_model=512):
+        return self.__iexp__((
+line = -1)                                                                                               
+
+────────────────────────────────────────────────────────────
+  Comparison
+────────────────────────────────────────────────────────────
+Method                   Tokens  Time(s)    Tok/s
+────────────────────── ──────── ──────── ────────
+Autoregressive              128     0.61    209.2
+Speculative                 128     0.91    140.1  (0.67× speedup, 94% accept)
+

results/execution-logs/sgjm1b_smoke.log

@@ -0,0 +1,4 @@
+[sgjm] resolved backend=mlx size=25m
+[sgjm] backend=mlx params=1400.67M
+[sgjm] step=     0 lr=2.00e-08 total=8.9748 tok=5.8542 draft=5.5188 jepa=1.1670 ver=0.6944 acc=0.500
+[sgjm] step=     1 lr=4.00e-08 total=9.5179 tok=6.2640 draft=5.7794 jepa=1.1775 ver=0.6976 acc=0.500

results/execution-logs/sgjm250_calib_a.log

@@ -0,0 +1,112 @@
+[sgjm] resolved backend=mlx size=25m
+[sgjm] backend=mlx params=252.35M
+[sgjm] step=     0 lr=1.00e-07 total=8.8236 tok=5.9432 draft=5.4094 jepa=1.1857 ver=0.7146 acc=0.454
+[sgjm] step=    25 lr=2.60e-06 total=6.5174 tok=4.0505 draft=4.5899 jepa=1.1656 ver=0.6933 acc=0.410
+[sgjm] step=    50 lr=5.10e-06 total=5.5059 tok=3.3622 draft=3.9571 jepa=1.0966 ver=0.6934 acc=0.502
+[sgjm] step=    75 lr=7.60e-06 total=5.3952 tok=3.3809 draft=3.7246 jepa=0.9652 ver=0.6928 acc=0.407
+[sgjm] step=   100 lr=1.01e-05 total=5.6028 tok=3.5472 draft=3.8437 jepa=0.7830 ver=0.6929 acc=0.501
+[sgjm] step=   125 lr=1.26e-05 total=4.8161 tok=3.0773 draft=3.2475 jepa=0.5963 ver=0.6927 acc=0.522
+[sgjm] step=   150 lr=1.51e-05 total=4.9062 tok=3.1264 draft=3.3633 jepa=0.4275 ver=0.6920 acc=0.555
+[sgjm] step=   175 lr=1.76e-05 total=4.7839 tok=3.0792 draft=3.2309 jepa=0.3374 ver=0.6933 acc=0.500
+[sgjm] step=   200 lr=2.01e-05 total=5.3684 tok=3.4536 draft=3.6608 jepa=0.3339 ver=0.6375 acc=0.709
+[sgjm] step=   225 lr=2.26e-05 total=4.5727 tok=2.9250 draft=3.1493 jepa=0.1784 ver=0.6898 acc=0.408
+[sgjm] step=   250 lr=2.51e-05 total=3.8022 tok=2.4097 draft=2.6410 jepa=0.1716 ver=0.6859 acc=0.601
+[sgjm] eval@250: {'total': 4.913861950238545, 'token': 3.1354472239812217, 'drafter': 3.4083130757013955, 'jepa': 0.20380433648824692, 'verifier': 0.67347119251887, 'accept_acc': 0.6151961286862692}
+[sgjm] step=   275 lr=2.76e-05 total=3.4268 tok=2.1571 draft=2.4032 jepa=0.1508 ver=0.6624 acc=0.558
+[sgjm] step=   300 lr=3.01e-05 total=4.6768 tok=3.0209 draft=3.1944 jepa=0.2622 ver=0.4056 acc=0.811
+[sgjm] step=   325 lr=3.26e-05 total=4.4669 tok=2.8719 draft=3.0588 jepa=0.1259 ver=0.6630 acc=0.645
+[sgjm] step=   350 lr=3.51e-05 total=4.5198 tok=2.8959 draft=3.1159 jepa=0.1194 ver=0.6749 acc=0.535
+[sgjm] step=   375 lr=3.76e-05 total=4.2136 tok=2.6868 draft=2.9249 jepa=0.1167 ver=0.6581 acc=0.574
+[sgjm] step=   400 lr=4.01e-05 total=4.5889 tok=2.9535 draft=3.1392 jepa=0.1190 ver=0.6737 acc=0.562
+[sgjm] step=   425 lr=4.26e-05 total=3.7954 tok=2.3599 draft=2.7629 jepa=0.1167 ver=0.5294 acc=0.811
+[sgjm] step=   450 lr=4.51e-05 total=3.9148 tok=2.4771 draft=2.7785 jepa=0.1677 ver=0.3959 acc=0.952
+[sgjm] step=   475 lr=4.76e-05 total=4.1574 tok=2.6304 draft=2.9253 jepa=0.1141 ver=0.6612 acc=0.607
+[sgjm] step=   500 lr=5.01e-05 total=3.9123 tok=2.4770 draft=2.7533 jepa=0.0932 ver=0.6176 acc=0.672
+[sgjm] eval@500: {'total': 4.121642430623372, 'token': 2.627545158068339, 'drafter': 2.8734511534372964, 'jepa': 0.10830457632740338, 'verifier': 0.5817671120166779, 'accept_acc': 0.6979575355847677}
+[sgjm] step=   525 lr=5.26e-05 total=3.9883 tok=2.5342 draft=2.7887 jepa=0.1088 ver=0.6113 acc=0.671
+[sgjm] step=   550 lr=5.51e-05 total=4.1433 tok=2.6689 draft=2.8954 jepa=0.1147 ver=0.1901 acc=0.950
+[sgjm] step=   575 lr=5.76e-05 total=4.2547 tok=2.7212 draft=2.9432 jepa=0.1192 ver=0.6241 acc=0.687
+[sgjm] step=   600 lr=6.01e-05 total=4.1410 tok=2.6166 draft=2.9269 jepa=0.1031 ver=0.6329 acc=0.617
+[sgjm] step=   625 lr=6.26e-05 total=4.1282 tok=2.6550 draft=2.8765 jepa=0.1049 ver=0.3049 acc=0.901
+[sgjm] step=   650 lr=6.51e-05 total=4.1074 tok=2.6179 draft=2.8711 jepa=0.0990 ver=0.5514 acc=0.728
+[sgjm] step=   675 lr=6.76e-05 total=4.7129 tok=3.0019 draft=3.3101 jepa=0.0988 ver=0.5766 acc=0.701
+[sgjm] step=   700 lr=7.01e-05 total=4.0597 tok=2.5890 draft=2.8942 jepa=0.1342 ver=0.1270 acc=0.953
+[sgjm] step=   725 lr=7.26e-05 total=3.5759 tok=2.2593 draft=2.5145 jepa=0.0878 ver=0.6322 acc=0.633
+[sgjm] step=   750 lr=7.51e-05 total=4.1969 tok=2.6764 draft=2.9186 jepa=0.1102 ver=0.6268 acc=0.663
+[sgjm] eval@750: {'total': 4.3698010842005415, 'token': 2.7960795958836875, 'drafter': 3.054114023844401, 'jepa': 0.11374000211556752, 'verifier': 0.4411301960547765, 'accept_acc': 0.8014706472555796}
+[sgjm] step=   775 lr=7.76e-05 total=4.2756 tok=2.7226 draft=2.9929 jepa=0.1011 ver=0.5804 acc=0.683
+[sgjm] step=   800 lr=8.01e-05 total=4.4240 tok=2.7976 draft=3.1384 jepa=0.1035 ver=0.5860 acc=0.683
+[sgjm] step=   825 lr=8.26e-05 total=3.4709 tok=2.1840 draft=2.4590 jepa=0.0980 ver=0.5947 acc=0.684
+[sgjm] step=   850 lr=8.51e-05 total=4.7916 tok=3.0861 draft=3.2952 jepa=0.0875 ver=0.6143 acc=0.651
+[sgjm] step=   875 lr=8.76e-05 total=4.0297 tok=2.5642 draft=2.8146 jepa=0.0997 ver=0.6029 acc=0.680
+[sgjm] step=   900 lr=9.01e-05 total=3.7768 tok=2.3918 draft=2.6602 jepa=0.0788 ver=0.5886 acc=0.689
+[sgjm] step=   925 lr=9.26e-05 total=3.7167 tok=2.3556 draft=2.6380 jepa=0.0890 ver=0.4151 acc=0.829
+[sgjm] step=   950 lr=9.51e-05 total=3.7216 tok=2.3193 draft=2.7487 jepa=0.0763 ver=0.2547 acc=0.908
+[sgjm] step=   975 lr=9.76e-05 total=4.3725 tok=2.7832 draft=3.0752 jepa=0.1058 ver=0.5138 acc=0.737
+[sgjm] step=  1000 lr=1.00e-04 total=3.6290 tok=2.2855 draft=2.5788 jepa=0.0954 ver=0.5571 acc=0.739
+[sgjm] eval@1000: {'total': 4.217257936795552, 'token': 2.6767027775446572, 'drafter': 2.9822624127070108, 'jepa': 0.09203251947959264, 'verifier': 0.5027579168478647, 'accept_acc': 0.7491830488046011}
+[sgjm] step=  1025 lr=9.99e-05 total=3.8445 tok=2.4461 draft=2.7027 jepa=0.0864 ver=0.4800 acc=0.765
+[sgjm] step=  1050 lr=9.97e-05 total=5.5899 tok=3.6703 draft=3.7675 jepa=0.0884 ver=0.3381 acc=0.907
+[sgjm] step=  1075 lr=9.94e-05 total=4.6550 tok=3.0043 draft=3.2345 jepa=0.0995 ver=0.2944 acc=0.880
+[sgjm] step=  1100 lr=9.89e-05 total=3.8009 tok=2.4394 draft=2.6180 jepa=0.0914 ver=0.5425 acc=0.745
+[sgjm] step=  1125 lr=9.83e-05 total=3.8900 tok=2.4641 draft=2.7717 jepa=0.0724 ver=0.4111 acc=0.840
+[sgjm] step=  1150 lr=9.76e-05 total=3.8432 tok=2.4461 draft=2.6860 jepa=0.0910 ver=0.5631 acc=0.704
+[sgjm] step=  1175 lr=9.67e-05 total=3.3120 tok=2.1228 draft=2.2639 jepa=0.0788 ver=0.6176 acc=0.692
+[sgjm] step=  1200 lr=9.57e-05 total=4.1836 tok=2.6754 draft=2.9529 jepa=0.1169 ver=0.2498 acc=0.900
+[sgjm] step=  1225 lr=9.46e-05 total=4.0956 tok=2.6293 draft=2.8734 jepa=0.0920 ver=0.2553 acc=0.892
+[sgjm] step=  1250 lr=9.33e-05 total=4.0518 tok=2.6034 draft=2.7941 jepa=0.0913 ver=0.5281 acc=0.757
+[sgjm] eval@1250: {'total': 3.8821394443511963, 'token': 2.4629796942075095, 'drafter': 2.7439624071121216, 'jepa': 0.0900350275139014, 'verifier': 0.47718606144189835, 'accept_acc': 0.8058823943138123}
+[sgjm] step=  1275 lr=9.19e-05 total=2.2343 tok=1.4030 draft=1.5883 jepa=0.0619 ver=0.3871 acc=0.887
+[sgjm] step=  1300 lr=9.05e-05 total=4.2141 tok=2.7169 draft=2.8822 jepa=0.1022 ver=0.5732 acc=0.692
+[sgjm] step=  1325 lr=8.89e-05 total=3.5649 tok=2.2859 draft=2.5175 jepa=0.0986 ver=0.1303 acc=0.975
+[sgjm] step=  1350 lr=8.72e-05 total=3.8055 tok=2.4287 draft=2.6747 jepa=0.1125 ver=0.3534 acc=0.846
+[sgjm] step=  1375 lr=8.54e-05 total=4.1362 tok=2.6566 draft=2.8934 jepa=0.0978 ver=0.2894 acc=0.890
+[sgjm] step=  1400 lr=8.35e-05 total=3.7015 tok=2.3823 draft=2.5933 jepa=0.0892 ver=0.1701 acc=0.951
+[sgjm] step=  1425 lr=8.15e-05 total=3.9932 tok=2.5713 draft=2.7918 jepa=0.1022 ver=0.1962 acc=0.942
+[sgjm] step=  1450 lr=7.94e-05 total=3.5451 tok=2.2645 draft=2.5338 jepa=0.1121 ver=0.0313 acc=0.996
+[sgjm] step=  1475 lr=7.72e-05 total=3.7876 tok=2.4199 draft=2.6347 jepa=0.0987 ver=0.5072 acc=0.746
+[sgjm] step=  1500 lr=7.50e-05 total=3.8094 tok=2.4369 draft=2.6626 jepa=0.0992 ver=0.3901 acc=0.824
+[sgjm] eval@1500: {'total': 3.912659525871277, 'token': 2.4763396581014, 'drafter': 2.772951086362203, 'jepa': 0.09495894486705463, 'verifier': 0.5043564929316441, 'accept_acc': 0.7949346800645193}
+[sgjm] step=  1525 lr=7.27e-05 total=3.8630 tok=2.4955 draft=2.6875 jepa=0.1108 ver=0.1581 acc=0.974
+[sgjm] step=  1550 lr=7.03e-05 total=3.9559 tok=2.5556 draft=2.7521 jepa=0.0999 ver=0.1786 acc=0.953
+[sgjm] step=  1575 lr=6.79e-05 total=4.3678 tok=2.8185 draft=3.0652 jepa=0.1199 ver=0.0582 acc=0.983
+[sgjm] step=  1600 lr=6.55e-05 total=3.6813 tok=2.3936 draft=2.5188 jepa=0.1069 ver=0.2212 acc=0.926
+[sgjm] step=  1625 lr=6.29e-05 total=3.5627 tok=2.2858 draft=2.5225 jepa=0.1154 ver=0.0515 acc=0.990
+[sgjm] step=  1650 lr=6.04e-05 total=3.3831 tok=2.1661 draft=2.3889 jepa=0.0905 ver=0.1689 acc=0.960
+[sgjm] step=  1675 lr=5.78e-05 total=3.8032 tok=2.4591 draft=2.6632 jepa=0.0961 ver=0.0370 acc=0.997
+[sgjm] step=  1700 lr=5.52e-05 total=4.3951 tok=2.8203 draft=3.1007 jepa=0.1281 ver=0.1456 acc=0.968
+[sgjm] step=  1725 lr=5.26e-05 total=2.8833 tok=1.8251 draft=2.0415 jepa=0.0865 ver=0.3601 acc=0.851
+[sgjm] step=  1750 lr=5.00e-05 total=3.5389 tok=2.2485 draft=2.4894 jepa=0.1103 ver=0.4340 acc=0.792
+[sgjm] eval@1750: {'total': 3.7574750979741416, 'token': 2.3934351603190103, 'drafter': 2.65031627813975, 'jepa': 0.11621163909633954, 'verifier': 0.3407587632536888, 'accept_acc': 0.8523693283398946}
+[sgjm] step=  1775 lr=4.74e-05 total=4.2801 tok=2.7675 draft=2.9872 jepa=0.1524 ver=0.0470 acc=0.994
+[sgjm] step=  1800 lr=4.48e-05 total=3.3294 tok=2.1236 draft=2.3774 jepa=0.1233 ver=0.0592 acc=0.993
+[sgjm] step=  1825 lr=4.22e-05 total=4.1103 tok=2.6403 draft=2.8832 jepa=0.1305 ver=0.1914 acc=0.936
+[sgjm] step=  1850 lr=3.96e-05 total=3.5846 tok=2.2635 draft=2.6101 jepa=0.1168 ver=0.0554 acc=0.990
+[sgjm] step=  1875 lr=3.71e-05 total=3.6502 tok=2.3255 draft=2.5971 jepa=0.1398 ver=0.1512 acc=0.936
+[sgjm] step=  1900 lr=3.45e-05 total=3.6868 tok=2.3492 draft=2.5834 jepa=0.1200 ver=0.4235 acc=0.806
+[sgjm] step=  1925 lr=3.21e-05 total=3.6307 tok=2.3346 draft=2.5390 jepa=0.1135 ver=0.1901 acc=0.915
+[sgjm] step=  1950 lr=2.97e-05 total=3.5599 tok=2.2722 draft=2.5435 jepa=0.1312 ver=0.0356 acc=0.988
+[sgjm] step=  1975 lr=2.73e-05 total=2.9200 tok=1.8440 draft=2.0987 jepa=0.1124 ver=0.1917 acc=0.913
+[sgjm] step=  2000 lr=2.50e-05 total=4.0002 tok=2.5717 draft=2.7959 jepa=0.1077 ver=0.2473 acc=0.915
+[sgjm] eval@2000: {'total': 3.4374317725499473, 'token': 2.1826334595680237, 'drafter': 2.440246125062307, 'jepa': 0.12176858882109325, 'verifier': 0.281230635009706, 'accept_acc': 0.8906046251455942}
+[sgjm] step=  2025 lr=2.28e-05 total=4.4268 tok=2.7678 draft=3.2776 jepa=0.1601 ver=0.0519 acc=0.992
+[sgjm] step=  2050 lr=2.06e-05 total=3.2019 tok=2.0520 draft=2.2077 jepa=0.1119 ver=0.4348 acc=0.789
+[sgjm] step=  2075 lr=1.85e-05 total=4.0595 tok=2.6178 draft=2.8211 jepa=0.1295 ver=0.2289 acc=0.908
+[sgjm] step=  2100 lr=1.65e-05 total=3.7572 tok=2.3926 draft=2.6652 jepa=0.1372 ver=0.2290 acc=0.903
+[sgjm] step=  2125 lr=1.46e-05 total=2.8929 tok=1.8335 draft=2.0543 jepa=0.1162 ver=0.2573 acc=0.890
+[sgjm] step=  2150 lr=1.28e-05 total=2.6656 tok=1.6598 draft=1.9708 jepa=0.1281 ver=0.0952 acc=0.972
+[sgjm] step=  2175 lr=1.11e-05 total=3.6159 tok=2.3273 draft=2.5175 jepa=0.1248 ver=0.2178 acc=0.910
+[sgjm] step=  2200 lr=9.55e-06 total=3.1119 tok=1.9811 draft=2.2319 jepa=0.1177 ver=0.0388 acc=0.995
+[sgjm] step=  2225 lr=8.07e-06 total=3.3400 tok=2.1468 draft=2.3192 jepa=0.1273 ver=0.2618 acc=0.879
+[sgjm] step=  2250 lr=6.70e-06 total=3.8396 tok=2.4628 draft=2.7030 jepa=0.1305 ver=0.1531 acc=0.950
+[sgjm] eval@2250: {'total': 3.8283124367396035, 'token': 2.4631226658821106, 'drafter': 2.684314767519633, 'jepa': 0.13592140128215155, 'verifier': 0.11800381975869338, 'accept_acc': 0.9588235914707184}
+[sgjm] step=  2275 lr=5.45e-06 total=3.4722 tok=2.2177 draft=2.4585 jepa=0.1355 ver=0.1469 acc=0.954
+[sgjm] step=  2300 lr=4.32e-06 total=3.0799 tok=1.9385 draft=2.2438 jepa=0.1468 ver=0.0601 acc=0.993
+[sgjm] step=  2325 lr=3.32e-06 total=3.4060 tok=2.1376 draft=2.4422 jepa=0.1308 ver=0.4275 acc=0.809
+[sgjm] step=  2350 lr=2.45e-06 total=2.6122 tok=1.6391 draft=1.8862 jepa=0.1145 ver=0.2312 acc=0.892
+[sgjm] step=  2375 lr=1.70e-06 total=3.3817 tok=2.1656 draft=2.3862 jepa=0.1304 ver=0.1249 acc=0.968
+[sgjm] step=  2400 lr=1.09e-06 total=3.4588 tok=2.2081 draft=2.4194 jepa=0.1349 ver=0.3441 acc=0.832
+[sgjm] step=  2425 lr=6.16e-07 total=3.3214 tok=2.1237 draft=2.3386 jepa=0.1254 ver=0.1977 acc=0.923
+[sgjm] step=  2450 lr=2.74e-07 total=3.4760 tok=2.2279 draft=2.4391 jepa=0.1335 ver=0.1899 acc=0.922
+[sgjm] step=  2475 lr=6.85e-08 total=3.4741 tok=2.2082 draft=2.4660 jepa=0.1404 ver=0.2345 acc=0.903
+[sgjm] step=  2499 lr=1.10e-10 total=3.2509 tok=2.0600 draft=2.3102 jepa=0.1416 ver=0.2712 acc=0.888

results/execution-logs/sgjm250_calib_b.log

@@ -0,0 +1,112 @@
+[sgjm] resolved backend=mlx size=25m
+[sgjm] backend=mlx params=252.35M
+[sgjm] step=     0 lr=1.00e-07 total=9.0174 tok=5.9432 draft=5.4127 jepa=1.1856 ver=0.7146 acc=0.454
+[sgjm] step=    25 lr=2.60e-06 total=6.7079 tok=4.0498 draft=4.5964 jepa=1.1623 ver=0.6933 acc=0.401
+[sgjm] step=    50 lr=5.10e-06 total=5.6840 tok=3.3625 draft=3.9630 jepa=1.0827 ver=0.6933 acc=0.500
+[sgjm] step=    75 lr=7.60e-06 total=5.5518 tok=3.3810 draft=3.7368 jepa=0.9327 ver=0.6928 acc=0.415
+[sgjm] step=   100 lr=1.01e-05 total=5.7424 tok=3.5469 draft=3.8833 jepa=0.7379 ver=0.6929 acc=0.500
+[sgjm] step=   125 lr=1.26e-05 total=4.9435 tok=3.0906 draft=3.2972 jepa=0.5399 ver=0.6927 acc=0.517
+[sgjm] step=   150 lr=1.51e-05 total=4.9876 tok=3.1263 draft=3.3972 jepa=0.3738 ver=0.6921 acc=0.554
+[sgjm] step=   175 lr=1.76e-05 total=4.8616 tok=3.0835 draft=3.2709 jepa=0.2933 ver=0.6936 acc=0.500
+[sgjm] step=   200 lr=2.01e-05 total=5.2800 tok=3.2993 draft=3.6929 jepa=0.2783 ver=0.6463 acc=0.666
+[sgjm] step=   225 lr=2.26e-05 total=4.6429 tok=2.9321 draft=3.2107 jepa=0.1448 ver=0.6932 acc=0.505
+[sgjm] step=   250 lr=2.51e-05 total=3.9134 tok=2.4312 draft=2.7534 jepa=0.1453 ver=0.6920 acc=0.517
+[sgjm] eval@250: {'total': 5.024669488271077, 'token': 3.1559399366378784, 'drafter': 3.5143254597981772, 'jepa': 0.1746974935134252, 'verifier': 0.6789228916168213, 'accept_acc': 0.6201607684294382}
+[sgjm] step=   275 lr=2.76e-05 total=3.5353 tok=2.1657 draft=2.5396 jepa=0.1298 ver=0.6732 acc=0.570
+[sgjm] step=   300 lr=3.01e-05 total=4.7757 tok=3.0242 draft=3.2996 jepa=0.2226 ver=0.4610 acc=0.904
+[sgjm] step=   325 lr=3.26e-05 total=4.5860 tok=2.8871 draft=3.2043 jepa=0.1103 ver=0.6918 acc=0.510
+[sgjm] step=   350 lr=3.51e-05 total=4.6059 tok=2.9019 draft=3.2185 jepa=0.1037 ver=0.6877 acc=0.518
+[sgjm] step=   375 lr=3.76e-05 total=4.2885 tok=2.6761 draft=3.0296 jepa=0.1177 ver=0.6813 acc=0.537
+[sgjm] step=   400 lr=4.01e-05 total=4.6142 tok=2.9191 draft=3.1975 jepa=0.1046 ver=0.7021 acc=0.510
+[sgjm] step=   425 lr=4.26e-05 total=3.9398 tok=2.3725 draft=2.9597 jepa=0.1300 ver=0.5495 acc=0.850
+[sgjm] step=   450 lr=4.51e-05 total=4.0454 tok=2.5202 draft=2.8934 jepa=0.1424 ver=0.4287 acc=0.942
+[sgjm] step=   475 lr=4.76e-05 total=4.2709 tok=2.6530 draft=3.0408 jepa=0.1108 ver=0.6976 acc=0.539
+[sgjm] step=   500 lr=5.01e-05 total=3.9967 tok=2.4748 draft=2.8600 jepa=0.0958 ver=0.6791 acc=0.576
+[sgjm] eval@500: {'total': 4.189104636510213, 'token': 2.590902845064799, 'drafter': 3.006748676300049, 'jepa': 0.12216155976057053, 'verifier': 0.6428715089956919, 'accept_acc': 0.6096620659033457}
+[sgjm] step=   525 lr=5.26e-05 total=4.0888 tok=2.5498 draft=2.8881 jepa=0.1142 ver=0.6635 acc=0.620
+[sgjm] step=   550 lr=5.51e-05 total=4.3261 tok=2.7421 draft=3.0586 jepa=0.1431 ver=0.1892 acc=0.926
+[sgjm] step=   575 lr=5.76e-05 total=4.3195 tok=2.6915 draft=3.0627 jepa=0.1204 ver=0.6663 acc=0.593
+[sgjm] step=   600 lr=6.01e-05 total=4.2492 tok=2.6336 draft=3.0490 jepa=0.0931 ver=0.6785 acc=0.568
+[sgjm] step=   625 lr=6.26e-05 total=4.1832 tok=2.6414 draft=2.9813 jepa=0.1023 ver=0.2562 acc=0.914
+[sgjm] step=   650 lr=6.51e-05 total=4.2283 tok=2.6316 draft=3.0137 jepa=0.1014 ver=0.6449 acc=0.639
+[sgjm] step=   675 lr=6.76e-05 total=4.8010 tok=2.9844 draft=3.4601 jepa=0.1015 ver=0.6115 acc=0.684
+[sgjm] step=   700 lr=7.01e-05 total=4.1883 tok=2.6230 draft=3.0307 jepa=0.1353 ver=0.1613 acc=0.925
+[sgjm] step=   725 lr=7.26e-05 total=3.6801 tok=2.2757 draft=2.6274 jepa=0.0858 ver=0.6932 acc=0.544
+[sgjm] step=   750 lr=7.51e-05 total=4.2232 tok=2.6313 draft=2.9980 jepa=0.1008 ver=0.6773 acc=0.589
+[sgjm] eval@750: {'total': 4.476851622263591, 'token': 2.798321564992269, 'drafter': 3.200577139854431, 'jepa': 0.12066792572538058, 'verifier': 0.48074520876010257, 'accept_acc': 0.707595149676005}
+[sgjm] step=   775 lr=7.76e-05 total=4.3611 tok=2.7037 draft=3.1436 jepa=0.1065 ver=0.5899 acc=0.696
+[sgjm] step=   800 lr=8.01e-05 total=4.5312 tok=2.7965 draft=3.2920 jepa=0.0967 ver=0.6455 acc=0.611
+[sgjm] step=   825 lr=8.26e-05 total=3.5899 tok=2.1964 draft=2.5965 jepa=0.1062 ver=0.6871 acc=0.607
+[sgjm] step=   850 lr=8.51e-05 total=4.8282 tok=3.0591 draft=3.3576 jepa=0.0839 ver=0.6931 acc=0.546
+[sgjm] step=   875 lr=8.76e-05 total=4.1435 tok=2.5634 draft=2.9770 jepa=0.1044 ver=0.6545 acc=0.597
+[sgjm] step=   900 lr=9.01e-05 total=3.8770 tok=2.3817 draft=2.8231 jepa=0.0780 ver=0.6430 acc=0.637
+[sgjm] step=   925 lr=9.26e-05 total=3.7923 tok=2.3478 draft=2.7555 jepa=0.0997 ver=0.4181 acc=0.890
+[sgjm] step=   950 lr=9.51e-05 total=3.7970 tok=2.2889 draft=2.9354 jepa=0.1000 ver=0.1543 acc=0.970
+[sgjm] step=   975 lr=9.76e-05 total=4.4681 tok=2.7918 draft=3.2154 jepa=0.1132 ver=0.4035 acc=0.868
+[sgjm] step=  1000 lr=1.00e-04 total=3.7791 tok=2.2950 draft=2.8090 jepa=0.1081 ver=0.5261 acc=0.772
+[sgjm] eval@1000: {'total': 4.3636040687561035, 'token': 2.691996137301127, 'drafter': 3.1683589220046997, 'jepa': 0.10623813420534134, 'verifier': 0.6086898346741995, 'accept_acc': 0.7019356985886892}
+[sgjm] step=  1025 lr=9.99e-05 total=3.9701 tok=2.4472 draft=2.9370 jepa=0.1036 ver=0.2855 acc=0.945
+[sgjm] step=  1050 lr=9.97e-05 total=5.7076 tok=3.6328 draft=4.0309 jepa=0.0991 ver=0.3460 acc=0.744
+[sgjm] step=  1075 lr=9.94e-05 total=4.7798 tok=3.0037 draft=3.4471 jepa=0.1088 ver=0.2544 acc=0.900
+[sgjm] step=  1100 lr=9.89e-05 total=3.9324 tok=2.4192 draft=2.8704 jepa=0.0960 ver=0.5397 acc=0.741
+[sgjm] step=  1125 lr=9.83e-05 total=4.0420 tok=2.4930 draft=3.0124 jepa=0.0776 ver=0.2339 acc=0.960
+[sgjm] step=  1150 lr=9.76e-05 total=4.0275 tok=2.4617 draft=2.9405 jepa=0.1067 ver=0.6885 acc=0.605
+[sgjm] step=  1175 lr=9.67e-05 total=3.4012 tok=2.1011 draft=2.4292 jepa=0.0815 ver=0.6514 acc=0.617
+[sgjm] step=  1200 lr=9.57e-05 total=4.3720 tok=2.6773 draft=3.2914 jepa=0.1368 ver=0.1477 acc=0.978
+[sgjm] step=  1225 lr=9.46e-05 total=4.2920 tok=2.6576 draft=3.1762 jepa=0.1060 ver=0.1982 acc=0.940
+[sgjm] step=  1250 lr=9.33e-05 total=4.2201 tok=2.6219 draft=3.0310 jepa=0.0979 ver=0.5823 acc=0.676
+[sgjm] eval@1250: {'total': 4.065106709798177, 'token': 2.4834326903025308, 'drafter': 3.009075403213501, 'jepa': 0.10450337454676628, 'verifier': 0.5101048946380615, 'accept_acc': 0.7981463372707367}
+[sgjm] step=  1275 lr=9.19e-05 total=2.3648 tok=1.4317 draft=1.7800 jepa=0.0725 ver=0.2502 acc=0.913
+[sgjm] step=  1300 lr=9.05e-05 total=4.4132 tok=2.7168 draft=3.2253 jepa=0.1179 ver=0.5427 acc=0.731
+[sgjm] step=  1325 lr=8.89e-05 total=3.8260 tok=2.3976 draft=2.7903 jepa=0.1013 ver=0.0798 acc=0.997
+[sgjm] step=  1350 lr=8.72e-05 total=3.9348 tok=2.4004 draft=2.9235 jepa=0.1229 ver=0.4198 acc=0.799
+[sgjm] step=  1375 lr=8.54e-05 total=4.3665 tok=2.6977 draft=3.2157 jepa=0.1079 ver=0.3394 acc=0.914
+[sgjm] step=  1400 lr=8.35e-05 total=3.8756 tok=2.4129 draft=2.8362 jepa=0.0990 ver=0.1979 acc=0.931
+[sgjm] step=  1425 lr=8.15e-05 total=4.1635 tok=2.6054 draft=3.0288 jepa=0.1085 ver=0.1656 acc=0.956
+[sgjm] step=  1450 lr=7.94e-05 total=3.7825 tok=2.3375 draft=2.8183 jepa=0.1310 ver=0.0310 acc=0.996
+[sgjm] step=  1475 lr=7.72e-05 total=3.8017 tok=2.3149 draft=2.8279 jepa=0.1057 ver=0.4639 acc=0.796
+[sgjm] step=  1500 lr=7.50e-05 total=3.9649 tok=2.4479 draft=2.9400 jepa=0.1012 ver=0.2166 acc=0.945
+[sgjm] eval@1500: {'total': 4.164750854174296, 'token': 2.553877592086792, 'drafter': 3.0715770721435547, 'jepa': 0.1082302841047446, 'verifier': 0.4802721468731761, 'accept_acc': 0.7829724450906118}
+[sgjm] step=  1525 lr=7.27e-05 total=4.0372 tok=2.5121 draft=2.9536 jepa=0.1121 ver=0.2022 acc=0.976
+[sgjm] step=  1550 lr=7.03e-05 total=4.1415 tok=2.5841 draft=3.0182 jepa=0.1115 ver=0.2034 acc=0.949
+[sgjm] step=  1575 lr=6.79e-05 total=4.5270 tok=2.8353 draft=3.3088 jepa=0.1204 ver=0.0719 acc=0.983
+[sgjm] step=  1600 lr=6.55e-05 total=3.8422 tok=2.4219 draft=2.7267 jepa=0.1175 ver=0.2761 acc=0.885
+[sgjm] step=  1625 lr=6.29e-05 total=3.7480 tok=2.3033 draft=2.8095 jepa=0.1350 ver=0.0617 acc=0.994
+[sgjm] step=  1650 lr=6.04e-05 total=3.5013 tok=2.1526 draft=2.6207 jepa=0.1018 ver=0.1294 acc=0.968
+[sgjm] step=  1675 lr=5.78e-05 total=3.9008 tok=2.4346 draft=2.8645 jepa=0.1077 ver=0.0704 acc=0.993
+[sgjm] step=  1700 lr=5.52e-05 total=4.5299 tok=2.8212 draft=3.3001 jepa=0.1415 ver=0.2329 acc=0.932
+[sgjm] step=  1725 lr=5.26e-05 total=3.0399 tok=1.8396 draft=2.3222 jepa=0.0998 ver=0.1423 acc=0.985
+[sgjm] step=  1750 lr=5.00e-05 total=3.7020 tok=2.2528 draft=2.7435 jepa=0.1240 ver=0.4644 acc=0.769
+[sgjm] eval@1750: {'total': 3.953872799873352, 'token': 2.4093175729115806, 'drafter': 2.9392066399256387, 'jepa': 0.13360331083337465, 'verifier': 0.41551091397802037, 'accept_acc': 0.8187336027622223}
+[sgjm] step=  1775 lr=4.74e-05 total=4.4432 tok=2.7409 draft=3.3141 jepa=0.1579 ver=0.0584 acc=0.996
+[sgjm] step=  1800 lr=4.48e-05 total=3.5034 tok=2.1481 draft=2.6310 jepa=0.1339 ver=0.0629 acc=0.993
+[sgjm] step=  1825 lr=4.22e-05 total=4.2735 tok=2.6516 draft=3.1277 jepa=0.1442 ver=0.2202 acc=0.933
+[sgjm] step=  1850 lr=3.96e-05 total=3.7620 tok=2.2637 draft=2.9168 jepa=0.1384 ver=0.0532 acc=0.993
+[sgjm] step=  1875 lr=3.71e-05 total=3.8182 tok=2.3117 draft=2.8822 jepa=0.1626 ver=0.2479 acc=0.909
+[sgjm] step=  1900 lr=3.45e-05 total=3.8712 tok=2.3611 draft=2.8585 jepa=0.1358 ver=0.4696 acc=0.759
+[sgjm] step=  1925 lr=3.21e-05 total=3.7614 tok=2.3377 draft=2.7413 jepa=0.1235 ver=0.2216 acc=0.916
+[sgjm] step=  1950 lr=2.97e-05 total=3.7104 tok=2.2655 draft=2.8026 jepa=0.1500 ver=0.0610 acc=0.984
+[sgjm] step=  1975 lr=2.73e-05 total=3.0795 tok=1.8308 draft=2.3459 jepa=0.1387 ver=0.4099 acc=0.812
+[sgjm] step=  2000 lr=2.50e-05 total=4.1215 tok=2.5677 draft=3.0223 jepa=0.1196 ver=0.1283 acc=0.968
+[sgjm] eval@2000: {'total': 3.6005011796951294, 'token': 2.1786193450291953, 'drafter': 2.6891546646753945, 'jepa': 0.1390422210097313, 'verifier': 0.425440164282918, 'accept_acc': 0.8356299300988516}
+[sgjm] step=  2025 lr=2.28e-05 total=5.0133 tok=3.0321 draft=3.8792 jepa=0.1392 ver=0.0675 acc=0.989
+[sgjm] step=  2050 lr=2.06e-05 total=3.3514 tok=2.0649 draft=2.3947 jepa=0.1286 ver=0.5697 acc=0.693
+[sgjm] step=  2075 lr=1.85e-05 total=4.2220 tok=2.6249 draft=3.0476 jepa=0.1381 ver=0.3878 acc=0.821
+[sgjm] step=  2100 lr=1.65e-05 total=3.9465 tok=2.4134 draft=2.9593 jepa=0.1578 ver=0.1402 acc=0.968
+[sgjm] step=  2125 lr=1.46e-05 total=3.0670 tok=1.8517 draft=2.3251 jepa=0.1331 ver=0.1945 acc=0.935
+[sgjm] step=  2150 lr=1.28e-05 total=2.8690 tok=1.6660 draft=2.2894 jepa=0.1557 ver=0.1943 acc=0.915
+[sgjm] step=  2175 lr=1.11e-05 total=3.7526 tok=2.3253 draft=2.7255 jepa=0.1396 ver=0.2971 acc=0.865
+[sgjm] step=  2200 lr=9.55e-06 total=3.2801 tok=1.9984 draft=2.4807 jepa=0.1394 ver=0.0653 acc=0.995
+[sgjm] step=  2225 lr=8.07e-06 total=3.4667 tok=2.1552 draft=2.5033 jepa=0.1443 ver=0.2378 acc=0.911
+[sgjm] step=  2250 lr=6.70e-06 total=3.9847 tok=2.4700 draft=2.9302 jepa=0.1465 ver=0.1294 acc=0.972
+[sgjm] eval@2250: {'total': 4.0585306485493975, 'token': 2.503700057665507, 'drafter': 2.9931986729303994, 'jepa': 0.15632390603423119, 'verifier': 0.1915043480694294, 'accept_acc': 0.9324146807193756}
+[sgjm] step=  2275 lr=5.45e-06 total=3.6753 tok=2.2296 draft=2.7504 jepa=0.1588 ver=0.3078 acc=0.878
+[sgjm] step=  2300 lr=4.32e-06 total=3.3043 tok=1.9672 draft=2.5788 jepa=0.1593 ver=0.0785 acc=0.990
+[sgjm] step=  2325 lr=3.32e-06 total=3.5825 tok=2.1471 draft=2.6798 jepa=0.1512 ver=0.5772 acc=0.722
+[sgjm] step=  2350 lr=2.45e-06 total=2.7341 tok=1.6297 draft=2.1101 jepa=0.1382 ver=0.1475 acc=0.954
+[sgjm] step=  2375 lr=1.70e-06 total=3.5147 tok=2.1602 draft=2.6172 jepa=0.1453 ver=0.0955 acc=0.988
+[sgjm] step=  2400 lr=1.09e-06 total=3.6006 tok=2.2090 draft=2.6443 jepa=0.1457 ver=0.3307 acc=0.854
+[sgjm] step=  2425 lr=6.16e-07 total=3.4826 tok=2.1343 draft=2.5696 jepa=0.1411 ver=0.2819 acc=0.889
+[sgjm] step=  2450 lr=2.74e-07 total=3.6254 tok=2.2369 draft=2.6867 jepa=0.1437 ver=0.0919 acc=0.987
+[sgjm] step=  2475 lr=6.85e-08 total=3.6611 tok=2.2220 draft=2.7337 jepa=0.1637 ver=0.3129 acc=0.856
+[sgjm] step=  2499 lr=1.10e-10 total=3.4203 tok=2.0628 draft=2.5654 jepa=0.1620 ver=0.3438 acc=0.859

results/execution-logs/sgjm25_calib_a.log

@@ -0,0 +1,67 @@
+=== START sgjm25_calib_a 2026-05-17T16:34:53Z ===
+[sgjm] resolved backend=mlx size=25m
+Traceback (most recent call last):
+  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
+    return _run_code(code, main_globals, None,
+  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
+    exec(code, run_globals)
+  File "/Users/adam/Development/SGJM/src/sgjm/training/__main__.py", line 102, in <module>
+    sys.exit(main())
+  File "/Users/adam/Development/SGJM/src/sgjm/training/__main__.py", line 94, in main
+    from sgjm.training.mlx_backend.trainer import train as mlx_train
+  File "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/__init__.py", line 1, in <module>
+    from sgjm.training.mlx_backend.model import SGJM
+  File "/Users/adam/Development/SGJM/src/sgjm/training/mlx_backend/model.py", line 5, in <module>
+    import mlx.core as mx
+ModuleNotFoundError: No module named 'mlx'
+=== START sgjm25_calib_a 2026-05-17T16:35:56Z ===
+[sgjm] resolved backend=cpu size=25m
+
+A module that was compiled using NumPy 1.x cannot be run in
+NumPy 2.0.2 as it may crash. To support both 1.x and 2.x
+versions of NumPy, modules must be compiled with NumPy 2.0.
+Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
+
+If you are a user of the module, the easiest solution will be to
+downgrade to 'numpy<2' or try to upgrade the affected module.
+We expect that some modules will need time to support NumPy 2.
+
+Traceback (most recent call last):  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
+    return _run_code(code, main_globals, None,
+  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
+    exec(code, run_globals)
+  File "/Users/adam/Development/SGJM/src/sgjm/training/__main__.py", line 102, in <module>
+    sys.exit(main())
+  File "/Users/adam/Development/SGJM/src/sgjm/training/__main__.py", line 91, in main
+    from sgjm.training.torch_backend.trainer import train as torch_train
+  File "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/__init__.py", line 1, in <module>
+    from sgjm.training.torch_backend.baseline import BaselineLM, compute_baseline_losses
+  File "/Users/adam/Development/SGJM/src/sgjm/training/torch_backend/baseline.py", line 5, in <module>
+    import torch
+  File "/Users/adam/Library/Python/3.9/lib/python/site-packages/torch/__init__.py", line 1477, in <module>
+    from .functional import *  # noqa: F403
+  File "/Users/adam/Library/Python/3.9/lib/python/site-packages/torch/functional.py", line 9, in <module>
+    import torch.nn.functional as F
+  File "/Users/adam/Library/Python/3.9/lib/python/site-packages/torch/nn/__init__.py", line 1, in <module>
+    from .modules import *  # noqa: F403
+  File "/Users/adam/Library/Python/3.9/lib/python/site-packages/torch/nn/modules/__init__.py", line 35, in <module>
+    from .transformer import TransformerEncoder, TransformerDecoder, \
+  File "/Users/adam/Library/Python/3.9/lib/python/site-packages/torch/nn/modules/transformer.py", line 20, in <module>
+    device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
+/Users/adam/Library/Python/3.9/lib/python/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)
+  device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
+[sgjm:sgjm] backend=cpu device=cpu params=25.86M backbone=23.90M drafter=1.38M judge=0.39M verifier=0.20M
+[sgjm:sgjm] step=     0 lr=1.50e-06 total=8.5883 token=5.6671 drafter=5.6688 jepa=1.0435 verifier=0.6941 accept_acc=0.4943
+[sgjm:sgjm] step=    25 lr=3.90e-05 total=6.4446 token=3.9944 drafter=4.7345 jepa=0.9640 verifier=0.6937 accept_acc=0.5000
+[sgjm:sgjm] step=    50 lr=7.65e-05 total=5.6545 token=3.4744 drafter=4.2214 jepa=0.6978 verifier=0.6907 accept_acc=0.5002
+[sgjm:sgjm] step=    75 lr=1.14e-04 total=4.9613 token=3.0331 drafter=3.7581 jepa=0.3196 verifier=0.6644 accept_acc=0.6395
+[sgjm:sgjm] step=   100 lr=1.51e-04 total=4.3109 token=2.6401 drafter=3.2454 jepa=0.3358 verifier=0.6256 accept_acc=0.6646
+[sgjm:sgjm] step=   125 lr=1.89e-04 total=4.2401 token=2.6145 drafter=3.1469 jepa=0.4249 verifier=0.6192 accept_acc=0.6580
+[sgjm:sgjm] step=   150 lr=2.26e-04 total=4.2967 token=2.6452 drafter=3.1923 jepa=0.5055 verifier=0.6008 accept_acc=0.6777
+[sgjm:sgjm] step=   175 lr=2.64e-04 total=4.0413 token=2.5051 drafter=2.9664 jepa=0.4706 verifier=0.5908 accept_acc=0.6882
+[sgjm:sgjm] step=   200 lr=3.00e-04 total=3.8437 token=2.3547 drafter=2.8630 jepa=0.5804 verifier=0.5692 accept_acc=0.7121
+[sgjm:sgjm] step=   225 lr=3.00e-04 total=3.5720 token=2.1792 drafter=2.6785 jepa=0.5428 verifier=0.5277 accept_acc=0.7441
+[sgjm:sgjm] step=   250 lr=2.99e-04 total=4.1226 token=2.5578 drafter=3.0115 jepa=0.6062 verifier=0.5745 accept_acc=0.7057
+[sgjm:sgjm] eval@250: {'total': 3.9560472667217255, 'token': 2.463844805955887, 'drafter': 2.8697565495967865, 'jepa': 0.5883685424923897, 'verifier': 0.5581154897809029, 'accept_acc': 0.7123523578047752}
+[sgjm:sgjm] step=   275 lr=2.99e-04 total=3.9171 token=2.4268 drafter=2.8652 jepa=0.6252 verifier=0.5289 accept_acc=0.7323
+[sgjm:sgjm] step=   300 lr=2.98e-04 total=3.5431 token=2.1649 drafter=2.6432 jepa=0.6124 verifier=0.5198 accept_acc=0.7480

results/execution-logs/sgjm25_calib_b.log

@@ -0,0 +1,90 @@
+[sgjm] resolved backend=mlx size=25m
+[sgjm] backend=mlx params=25.96M
+[sgjm] step=     0 lr=1.50e-06 total=8.9681 tok=5.8719 draft=5.7279 jepa=1.1789 ver=0.6921 acc=0.503
+[sgjm] step=    25 lr=3.90e-05 total=5.8780 tok=3.4957 draft=4.3273 jepa=1.0880 ver=0.6930 acc=0.478
+[sgjm] step=    50 lr=7.65e-05 total=5.6224 tok=3.4591 draft=3.9883 jepa=0.7589 ver=0.6925 acc=0.510
+[sgjm] step=    75 lr=1.14e-04 total=4.8359 tok=2.9910 draft=3.4666 jepa=0.3781 ver=0.6859 acc=0.596
+[sgjm] step=   100 lr=1.51e-04 total=4.8132 tok=3.0249 draft=3.3813 jepa=0.3030 ver=0.6535 acc=0.687
+[sgjm] step=   125 lr=1.89e-04 total=4.0727 tok=2.5373 draft=2.8978 jepa=0.2706 ver=0.5745 acc=0.703
+[sgjm] step=   150 lr=2.26e-04 total=4.3377 tok=2.7039 draft=3.1061 jepa=0.2726 ver=0.4983 acc=0.781
+[sgjm] step=   175 lr=2.64e-04 total=4.2000 tok=2.6408 draft=2.9498 jepa=0.2785 ver=0.5322 acc=0.738
+[sgjm] step=   200 lr=3.00e-04 total=4.0446 tok=2.5297 draft=2.8908 jepa=0.2917 ver=0.3220 acc=0.880
+[sgjm] step=   225 lr=3.00e-04 total=4.3032 tok=2.7157 draft=3.0392 jepa=0.2601 ver=0.3607 acc=0.847
+[sgjm] step=   250 lr=2.99e-04 total=4.2812 tok=2.6948 draft=3.0207 jepa=0.2282 ver=0.5228 acc=0.741
+[sgjm] eval@250: {'total': 4.2140171229839325, 'token': 2.67084276676178, 'drafter': 2.9318204522132874, 'jepa': 0.2598923146724701, 'verifier': 0.4785040020942688, 'accept_acc': 0.7704847455024719}
+[sgjm] step=   275 lr=2.99e-04 total=4.5727 tok=2.8978 draft=3.2223 jepa=0.2459 ver=0.3362 acc=0.872
+[sgjm] step=   300 lr=2.98e-04 total=3.8076 tok=2.4029 draft=2.6730 jepa=0.2448 ver=0.3930 acc=0.826
+[sgjm] step=   325 lr=2.96e-04 total=3.4442 tok=2.1297 draft=2.5136 jepa=0.2648 ver=0.2247 acc=0.905
+[sgjm] step=   350 lr=2.95e-04 total=3.6592 tok=2.3124 draft=2.5792 jepa=0.2251 ver=0.2932 acc=0.876
+[sgjm] step=   375 lr=2.93e-04 total=3.7855 tok=2.3686 draft=2.7086 jepa=0.2467 ver=0.3200 acc=0.874
+[sgjm] step=   400 lr=2.91e-04 total=3.8657 tok=2.4298 draft=2.7259 jepa=0.2906 ver=0.3670 acc=0.842
+[sgjm] step=   425 lr=2.89e-04 total=3.6571 tok=2.2827 draft=2.6225 jepa=0.2692 ver=0.2836 acc=0.879
+[sgjm] step=   450 lr=2.86e-04 total=3.3291 tok=2.0797 draft=2.3983 jepa=0.2690 ver=0.1235 acc=0.974
+[sgjm] step=   475 lr=2.83e-04 total=3.7375 tok=2.3596 draft=2.6077 jepa=0.2774 ver=0.4061 acc=0.817
+[sgjm] step=   500 lr=2.80e-04 total=3.7936 tok=2.3941 draft=2.6674 jepa=0.2819 ver=0.2927 acc=0.894
+[sgjm] eval@500: {'total': 3.6762998700141907, 'token': 2.3035406172275543, 'drafter': 2.6179537773132324, 'jepa': 0.2750014141201973, 'verifier': 0.28165259398519993, 'accept_acc': 0.8951464146375656}
+[sgjm] step=   525 lr=2.77e-04 total=3.4939 tok=2.2091 draft=2.4548 jepa=0.2651 ver=0.2201 acc=0.918
+[sgjm] step=   550 lr=2.73e-04 total=3.4111 tok=2.1372 draft=2.4442 jepa=0.2546 ver=0.1708 acc=0.950
+[sgjm] step=   575 lr=2.69e-04 total=3.5373 tok=2.2355 draft=2.4921 jepa=0.2616 ver=0.2073 acc=0.918
+[sgjm] step=   600 lr=2.65e-04 total=3.5009 tok=2.2010 draft=2.4843 jepa=0.2915 ver=0.1748 acc=0.953
+[sgjm] step=   625 lr=2.61e-04 total=3.4779 tok=2.1974 draft=2.4508 jepa=0.2968 ver=0.1315 acc=0.968
+[sgjm] step=   650 lr=2.56e-04 total=3.3773 tok=2.1217 draft=2.3983 jepa=0.2950 ver=0.1518 acc=0.939
+[sgjm] step=   675 lr=2.51e-04 total=3.6910 tok=2.3161 draft=2.6073 jepa=0.3168 ver=0.2968 acc=0.912
+[sgjm] step=   700 lr=2.46e-04 total=3.0683 tok=1.9167 draft=2.2040 jepa=0.2940 ver=0.0694 acc=0.982
+[sgjm] step=   725 lr=2.41e-04 total=3.3245 tok=2.0667 draft=2.3741 jepa=0.3419 ver=0.2441 acc=0.890
+[sgjm] step=   750 lr=2.36e-04 total=3.2739 tok=2.0549 draft=2.3279 jepa=0.3171 ver=0.0944 acc=0.968
+[sgjm] eval@750: {'total': 3.165252208709717, 'token': 1.9577558934688568, 'drafter': 2.2846018075942993, 'jepa': 0.32470009103417397, 'verifier': 0.2061301078647375, 'accept_acc': 0.9176919311285019}
+[sgjm] step=   775 lr=2.31e-04 total=3.1770 tok=1.9645 draft=2.2897 jepa=0.3553 ver=0.1800 acc=0.926
+[sgjm] step=   800 lr=2.25e-04 total=3.3302 tok=2.0650 draft=2.4001 jepa=0.3628 ver=0.1345 acc=0.947
+[sgjm] step=   825 lr=2.19e-04 total=3.0450 tok=1.8968 draft=2.1859 jepa=0.3312 ver=0.0698 acc=0.981
+[sgjm] step=   850 lr=2.13e-04 total=2.9914 tok=1.8384 draft=2.1921 jepa=0.3326 ver=0.0886 acc=0.974
+[sgjm] step=   875 lr=2.07e-04 total=3.0903 tok=1.9052 draft=2.2574 jepa=0.3463 ver=0.0561 acc=0.987
+[sgjm] step=   900 lr=2.01e-04 total=3.1312 tok=1.9198 draft=2.3068 jepa=0.3428 ver=0.0825 acc=0.973
+[sgjm] step=   925 lr=1.95e-04 total=3.2010 tok=1.9880 draft=2.2982 jepa=0.3626 ver=0.1196 acc=0.955
+[sgjm] step=   950 lr=1.89e-04 total=2.7004 tok=1.6504 draft=1.9724 jepa=0.3849 ver=0.0766 acc=0.972
+[sgjm] step=   975 lr=1.82e-04 total=3.2505 tok=2.0069 draft=2.3581 jepa=0.4059 ver=0.0459 acc=0.989
+[sgjm] step=  1000 lr=1.76e-04 total=3.2998 tok=2.0369 draft=2.3949 jepa=0.3955 ver=0.0763 acc=0.976
+[sgjm] eval@1000: {'total': 2.942210406064987, 'token': 1.7914329767227173, 'drafter': 2.159805417060852, 'jepa': 0.3770422637462616, 'verifier': 0.17897918075323105, 'accept_acc': 0.9327017664909363}
+[sgjm] step=  1025 lr=1.70e-04 total=2.9799 tok=1.8398 draft=2.1415 jepa=0.4013 ver=0.1141 acc=0.958
+[sgjm] step=  1050 lr=1.63e-04 total=2.7650 tok=1.6659 draft=2.0684 jepa=0.3865 ver=0.0865 acc=0.969
+[sgjm] step=  1075 lr=1.57e-04 total=2.7876 tok=1.6952 draft=2.0542 jepa=0.4007 ver=0.0653 acc=0.982
+[sgjm] step=  1100 lr=1.50e-04 total=2.9380 tok=1.7803 draft=2.1624 jepa=0.4249 ver=0.1597 acc=0.954
+[sgjm] step=  1125 lr=1.43e-04 total=2.9036 tok=1.7668 draft=2.1245 jepa=0.4183 ver=0.1476 acc=0.939
+[sgjm] step=  1150 lr=1.37e-04 total=2.8550 tok=1.7558 draft=2.0656 jepa=0.4090 ver=0.0631 acc=0.980
+[sgjm] step=  1175 lr=1.30e-04 total=2.5999 tok=1.5560 draft=1.9420 jepa=0.4284 ver=0.1076 acc=0.957
+[sgjm] step=  1200 lr=1.24e-04 total=2.7386 tok=1.6596 draft=2.0250 jepa=0.4171 ver=0.0492 acc=0.986
+[sgjm] step=  1225 lr=1.18e-04 total=2.6577 tok=1.6138 draft=1.9607 jepa=0.3953 ver=0.0522 acc=0.986
+[sgjm] step=  1250 lr=1.11e-04 total=2.4757 tok=1.4649 draft=1.8909 jepa=0.4021 ver=0.0624 acc=0.980
+[sgjm] eval@1250: {'total': 2.639838546514511, 'token': 1.5879154354333878, 'drafter': 1.9545578956604004, 'jepa': 0.42004794254899025, 'verifier': 0.1454625865444541, 'accept_acc': 0.945189468562603}
+[sgjm] step=  1275 lr=1.05e-04 total=2.5446 tok=1.5354 draft=1.8925 jepa=0.3939 ver=0.0479 acc=0.985
+[sgjm] step=  1300 lr=9.87e-05 total=2.3781 tok=1.4099 draft=1.7961 jepa=0.4252 ver=0.0801 acc=0.974
+[sgjm] step=  1325 lr=9.26e-05 total=2.4123 tok=1.4342 draft=1.8243 jepa=0.3976 ver=0.0794 acc=0.971
+[sgjm] step=  1350 lr=8.66e-05 total=2.5577 tok=1.5342 draft=1.9032 jepa=0.4384 ver=0.0780 acc=0.976
+[sgjm] step=  1375 lr=8.07e-05 total=2.7081 tok=1.6436 draft=1.9887 jepa=0.4192 ver=0.0910 acc=0.968
+[sgjm] step=  1400 lr=7.50e-05 total=2.6023 tok=1.5699 draft=1.9320 jepa=0.4201 ver=0.0426 acc=0.989
+[sgjm] step=  1425 lr=6.94e-05 total=2.2385 tok=1.3220 draft=1.6887 jepa=0.4406 ver=0.0761 acc=0.977
+[sgjm] step=  1450 lr=6.40e-05 total=2.7020 tok=1.6405 draft=1.9908 jepa=0.4136 ver=0.0506 acc=0.983
+[sgjm] step=  1475 lr=5.87e-05 total=2.4785 tok=1.4481 draft=1.9023 jepa=0.4807 ver=0.0897 acc=0.972
+[sgjm] step=  1500 lr=5.36e-05 total=2.6429 tok=1.5991 draft=1.9540 jepa=0.4287 ver=0.0303 acc=0.996
+[sgjm] eval@1500: {'total': 2.619326949119568, 'token': 1.5689085274934769, 'drafter': 1.950766235589981, 'jepa': 0.43744663521647453, 'verifier': 0.11772921495139599, 'accept_acc': 0.9573388248682022}
+[sgjm] step=  1525 lr=4.87e-05 total=2.4467 tok=1.4670 draft=1.8297 jepa=0.4048 ver=0.0510 acc=0.986
+[sgjm] step=  1550 lr=4.39e-05 total=2.4680 tok=1.4741 draft=1.8512 jepa=0.4242 ver=0.0590 acc=0.983
+[sgjm] step=  1575 lr=3.94e-05 total=2.3104 tok=1.3634 draft=1.7558 jepa=0.4263 ver=0.0639 acc=0.981
+[sgjm] step=  1600 lr=3.51e-05 total=2.5945 tok=1.5680 draft=1.9090 jepa=0.4460 ver=0.0640 acc=0.982
+[sgjm] step=  1625 lr=3.10e-05 total=2.4399 tok=1.4480 draft=1.8343 jepa=0.4597 ver=0.0724 acc=0.976
+[sgjm] step=  1650 lr=2.71e-05 total=2.1810 tok=1.2825 draft=1.6493 jepa=0.4583 ver=0.0642 acc=0.980
+[sgjm] step=  1675 lr=2.35e-05 total=2.4963 tok=1.4931 draft=1.8774 jepa=0.4118 ver=0.0341 acc=0.991
+[sgjm] step=  1700 lr=2.01e-05 total=2.3169 tok=1.3602 draft=1.7635 jepa=0.4535 ver=0.0872 acc=0.972
+[sgjm] step=  1725 lr=1.69e-05 total=2.3693 tok=1.4003 draft=1.7951 jepa=0.4463 ver=0.0576 acc=0.982
+[sgjm] step=  1750 lr=1.41e-05 total=2.4909 tok=1.4765 draft=1.8734 jepa=0.4657 ver=0.0987 acc=0.961
+[sgjm] eval@1750: {'total': 2.391613095998764, 'token': 1.4267115890979767, 'drafter': 1.7819698601961136, 'jepa': 0.43197276815772057, 'verifier': 0.11400794330984354, 'accept_acc': 0.9587536826729774}
+[sgjm] step=  1775 lr=1.14e-05 total=2.4752 tok=1.4796 draft=1.8503 jepa=0.4481 ver=0.0400 acc=0.990
+[sgjm] step=  1800 lr=9.05e-06 total=2.4219 tok=1.4483 draft=1.8075 jepa=0.4403 ver=0.0477 acc=0.988
+[sgjm] step=  1825 lr=6.94e-06 total=2.2416 tok=1.3194 draft=1.7082 jepa=0.4239 ver=0.0557 acc=0.983
+[sgjm] step=  1850 lr=5.11e-06 total=2.3249 tok=1.3684 draft=1.7641 jepa=0.4463 ver=0.0939 acc=0.966
+[sgjm] step=  1875 lr=3.56e-06 total=2.4714 tok=1.4867 draft=1.8358 jepa=0.4248 ver=0.0382 acc=0.990
+[sgjm] step=  1900 lr=2.28e-06 total=2.2274 tok=1.3192 draft=1.6735 jepa=0.4310 ver=0.0848 acc=0.973
+[sgjm] step=  1925 lr=1.28e-06 total=2.1661 tok=1.3025 draft=1.5898 jepa=0.3925 ver=0.1228 acc=0.962
+[sgjm] step=  1950 lr=5.71e-07 total=2.2515 tok=1.3547 draft=1.6728 jepa=0.3818 ver=0.0391 acc=0.987
+[sgjm] step=  1975 lr=1.43e-07 total=2.0858 tok=1.2255 draft=1.5827 jepa=0.4286 ver=0.0579 acc=0.982
+[sgjm] step=  1999 lr=2.28e-10 total=2.1263 tok=1.2519 draft=1.6051 jepa=0.4374 ver=0.0773 acc=0.972

results/execution-logs/sgjm25_calib_c.log

@@ -0,0 +1,90 @@
+[sgjm] resolved backend=mlx size=25m
+[sgjm] backend=mlx params=25.96M
+[sgjm] step=     0 lr=1.50e-06 total=9.1030 tok=5.8719 draft=5.7335 jepa=1.1807 ver=0.6921 acc=0.500
+[sgjm] step=    25 lr=3.90e-05 total=6.0035 tok=3.4964 draft=4.3395 jepa=1.0720 ver=0.6930 acc=0.480
+[sgjm] step=    50 lr=7.65e-05 total=5.7358 tok=3.4637 draft=4.0466 jepa=0.7184 ver=0.6927 acc=0.511
+[sgjm] step=    75 lr=1.14e-04 total=4.8374 tok=2.9207 draft=3.5196 jepa=0.3521 ver=0.6888 acc=0.561
+[sgjm] step=   100 lr=1.51e-04 total=4.9076 tok=3.0437 draft=3.4525 jepa=0.2854 ver=0.6631 acc=0.652
+[sgjm] step=   125 lr=1.89e-04 total=4.1893 tok=2.5471 draft=3.0314 jepa=0.2688 ver=0.5927 acc=0.751
+[sgjm] step=   150 lr=2.26e-04 total=4.4476 tok=2.6996 draft=3.2591 jepa=0.2710 ver=0.5064 acc=0.781
+[sgjm] step=   175 lr=2.64e-04 total=4.3099 tok=2.6372 draft=3.0871 jepa=0.2872 ver=0.5728 acc=0.685
+[sgjm] step=   200 lr=3.00e-04 total=4.1883 tok=2.5393 draft=3.0813 jepa=0.2972 ver=0.3404 acc=0.868
+[sgjm] step=   225 lr=3.00e-04 total=4.3804 tok=2.6826 draft=3.1813 jepa=0.2725 ver=0.3907 acc=0.813
+[sgjm] step=   250 lr=2.99e-04 total=4.3973 tok=2.6960 draft=3.1686 jepa=0.2562 ver=0.5294 acc=0.750
+[sgjm] eval@250: {'total': 4.36805123090744, 'token': 2.6730625331401825, 'drafter': 3.1302269101142883, 'jepa': 0.2995202988386154, 'verifier': 0.5499511919915676, 'accept_acc': 0.7242373898625374}
+[sgjm] step=   275 lr=2.99e-04 total=4.6571 tok=2.8866 draft=3.3527 jepa=0.2452 ver=0.3290 acc=0.883
+[sgjm] step=   300 lr=2.98e-04 total=3.9533 tok=2.4090 draft=2.8588 jepa=0.2819 ver=0.4443 acc=0.799
+[sgjm] step=   325 lr=2.96e-04 total=3.5360 tok=2.0384 draft=2.7990 jepa=0.3172 ver=0.1887 acc=0.944
+[sgjm] step=   350 lr=2.95e-04 total=3.8128 tok=2.3212 draft=2.7870 jepa=0.2527 ver=0.3491 acc=0.837
+[sgjm] step=   375 lr=2.93e-04 total=3.9692 tok=2.3793 draft=2.9603 jepa=0.3043 ver=0.3372 acc=0.844
+[sgjm] step=   400 lr=2.91e-04 total=4.0421 tok=2.4379 draft=2.9648 jepa=0.3269 ver=0.4006 acc=0.822
+[sgjm] step=   425 lr=2.89e-04 total=4.0119 tok=2.4341 draft=2.9533 jepa=0.3070 ver=0.2443 acc=0.909
+[sgjm] step=   450 lr=2.86e-04 total=3.5288 tok=2.0791 draft=2.7092 jepa=0.3320 ver=0.1210 acc=0.977
+[sgjm] step=   475 lr=2.83e-04 total=3.9189 tok=2.3604 draft=2.8812 jepa=0.2999 ver=0.4294 acc=0.810
+[sgjm] step=   500 lr=2.80e-04 total=3.9485 tok=2.3761 draft=2.9223 jepa=0.3251 ver=0.2998 acc=0.883
+[sgjm] eval@500: {'total': 3.8309080004692078, 'token': 2.284691721200943, 'drafter': 2.863490045070648, 'jepa': 0.33332348987460136, 'verifier': 0.3114043343812227, 'accept_acc': 0.8825955465435982}
+[sgjm] step=   525 lr=2.77e-04 total=3.6337 tok=2.1953 draft=2.6810 jepa=0.2972 ver=0.2359 acc=0.902
+[sgjm] step=   550 lr=2.73e-04 total=3.5641 tok=2.1171 draft=2.7078 jepa=0.2976 ver=0.1861 acc=0.943
+[sgjm] step=   575 lr=2.69e-04 total=3.7377 tok=2.2603 draft=2.7493 jepa=0.3234 ver=0.2188 acc=0.912
+[sgjm] step=   600 lr=2.65e-04 total=3.6706 tok=2.1951 draft=2.7395 jepa=0.3470 ver=0.1904 acc=0.949
+[sgjm] step=   625 lr=2.61e-04 total=3.6691 tok=2.2140 draft=2.7178 jepa=0.3368 ver=0.1200 acc=0.970
+[sgjm] step=   650 lr=2.56e-04 total=3.5676 tok=2.1208 draft=2.6805 jepa=0.3577 ver=0.1707 acc=0.938
+[sgjm] step=   675 lr=2.51e-04 total=3.8774 tok=2.3129 draft=2.8739 jepa=0.3708 ver=0.3482 acc=0.891
+[sgjm] step=   700 lr=2.46e-04 total=3.2095 tok=1.8906 draft=2.4419 jepa=0.3652 ver=0.0665 acc=0.985
+[sgjm] step=   725 lr=2.41e-04 total=3.5108 tok=2.0638 draft=2.6439 jepa=0.3875 ver=0.2818 acc=0.878
+[sgjm] step=   750 lr=2.36e-04 total=3.4878 tok=2.0745 draft=2.6106 jepa=0.3858 ver=0.1156 acc=0.956
+[sgjm] eval@750: {'total': 3.4201184809207916, 'token': 1.9776086062192917, 'drafter': 2.615806519985199, 'jepa': 0.41085900366306305, 'verifier': 0.31891899555921555, 'accept_acc': 0.8705047592520714}
+[sgjm] step=   775 lr=2.31e-04 total=3.3919 tok=1.9651 draft=2.6104 jepa=0.4114 ver=0.1880 acc=0.936
+[sgjm] step=   800 lr=2.25e-04 total=3.5459 tok=2.0644 draft=2.7171 jepa=0.4241 ver=0.1694 acc=0.935
+[sgjm] step=   825 lr=2.19e-04 total=3.2660 tok=1.9074 draft=2.4912 jepa=0.4063 ver=0.1144 acc=0.966
+[sgjm] step=   850 lr=2.13e-04 total=3.2082 tok=1.8502 draft=2.4859 jepa=0.4084 ver=0.1293 acc=0.956
+[sgjm] step=   875 lr=2.07e-04 total=3.2792 tok=1.8805 draft=2.5650 jepa=0.4290 ver=0.0897 acc=0.980
+[sgjm] step=   900 lr=2.01e-04 total=3.3264 tok=1.8938 draft=2.6292 jepa=0.4207 ver=0.1280 acc=0.958
+[sgjm] step=   925 lr=1.95e-04 total=3.4554 tok=2.0154 draft=2.6299 jepa=0.4393 ver=0.1517 acc=0.949
+[sgjm] step=   950 lr=1.89e-04 total=2.9586 tok=1.6634 draft=2.3376 jepa=0.4622 ver=0.1088 acc=0.964
+[sgjm] step=   975 lr=1.82e-04 total=3.5335 tok=2.0326 draft=2.7437 jepa=0.4891 ver=0.0671 acc=0.984
+[sgjm] step=  1000 lr=1.76e-04 total=3.5530 tok=2.0592 draft=2.7375 jepa=0.4616 ver=0.0969 acc=0.965
+[sgjm] eval@1000: {'total': 3.2041093707084656, 'token': 1.8007187247276306, 'drafter': 2.5217553973197937, 'jepa': 0.4605707973241806, 'verifier': 0.27370264381170273, 'accept_acc': 0.8912141025066376}
+[sgjm] step=  1025 lr=1.70e-04 total=3.2283 tok=1.8362 draft=2.5094 jepa=0.4839 ver=0.1647 acc=0.938
+[sgjm] step=  1050 lr=1.63e-04 total=3.0194 tok=1.6660 draft=2.4489 jepa=0.4792 ver=0.0916 acc=0.970
+[sgjm] step=  1075 lr=1.57e-04 total=3.0527 tok=1.7071 draft=2.4221 jepa=0.4870 ver=0.1281 acc=0.964
+[sgjm] step=  1100 lr=1.50e-04 total=3.2116 tok=1.7930 draft=2.5432 jepa=0.5144 ver=0.1842 acc=0.941
+[sgjm] step=  1125 lr=1.43e-04 total=3.1978 tok=1.7874 draft=2.5255 jepa=0.5042 ver=0.2154 acc=0.906
+[sgjm] step=  1150 lr=1.37e-04 total=3.0144 tok=1.7171 draft=2.3314 jepa=0.4967 ver=0.0739 acc=0.976
+[sgjm] step=  1175 lr=1.30e-04 total=2.9215 tok=1.5933 draft=2.3737 jepa=0.5093 ver=0.1396 acc=0.945
+[sgjm] step=  1200 lr=1.24e-04 total=2.9979 tok=1.6604 draft=2.4043 jepa=0.5144 ver=0.0676 acc=0.980
+[sgjm] step=  1225 lr=1.18e-04 total=2.9546 tok=1.6425 draft=2.3659 jepa=0.4930 ver=0.0589 acc=0.988
+[sgjm] step=  1250 lr=1.11e-04 total=2.7873 tok=1.4900 draft=2.3165 jepa=0.5111 ver=0.1120 acc=0.965
+[sgjm] eval@1250: {'total': 2.949668914079666, 'token': 1.6112764477729797, 'drafter': 2.3732622265815735, 'jepa': 0.5289503484964371, 'verifier': 0.19523879513144493, 'accept_acc': 0.9210999980568886}
+[sgjm] step=  1275 lr=1.05e-04 total=2.8018 tok=1.5439 draft=2.2538 jepa=0.4902 ver=0.0852 acc=0.973
+[sgjm] step=  1300 lr=9.87e-05 total=2.6682 tok=1.4059 draft=2.2312 jepa=0.5386 ver=0.1208 acc=0.961
+[sgjm] step=  1325 lr=9.26e-05 total=2.7059 tok=1.4493 draft=2.2361 jepa=0.5048 ver=0.1239 acc=0.959
+[sgjm] step=  1350 lr=8.66e-05 total=2.8565 tok=1.5539 draft=2.3096 jepa=0.5530 ver=0.0960 acc=0.969
+[sgjm] step=  1375 lr=8.07e-05 total=2.9814 tok=1.6656 draft=2.3564 jepa=0.5105 ver=0.1006 acc=0.967
+[sgjm] step=  1400 lr=7.50e-05 total=2.8974 tok=1.5966 draft=2.3256 jepa=0.5172 ver=0.0872 acc=0.969
+[sgjm] step=  1425 lr=6.94e-05 total=2.5394 tok=1.3260 draft=2.1342 jepa=0.5483 ver=0.0918 acc=0.975
+[sgjm] step=  1450 lr=6.40e-05 total=3.0136 tok=1.6740 draft=2.4035 jepa=0.5171 ver=0.0868 acc=0.970
+[sgjm] step=  1475 lr=5.87e-05 total=2.8338 tok=1.4632 draft=2.4302 jepa=0.5805 ver=0.1037 acc=0.974
+[sgjm] step=  1500 lr=5.36e-05 total=2.9251 tok=1.6209 draft=2.3323 jepa=0.5352 ver=0.0421 acc=0.992
+[sgjm] eval@1500: {'total': 2.937375783920288, 'token': 1.587001010775566, 'drafter': 2.3920841217041016, 'jepa': 0.5458228513598442, 'verifier': 0.17876905528828502, 'accept_acc': 0.9316096603870392}
+[sgjm] step=  1525 lr=4.87e-05 total=2.7073 tok=1.4720 draft=2.2092 jepa=0.4991 ver=0.0586 acc=0.987
+[sgjm] step=  1550 lr=4.39e-05 total=2.7641 tok=1.4882 draft=2.2748 jepa=0.5224 ver=0.0791 acc=0.977
+[sgjm] step=  1575 lr=3.94e-05 total=2.6117 tok=1.3839 draft=2.1633 jepa=0.5320 ver=0.1311 acc=0.952
+[sgjm] step=  1600 lr=3.51e-05 total=2.8741 tok=1.5775 draft=2.2971 jepa=0.5539 ver=0.0965 acc=0.968
+[sgjm] step=  1625 lr=3.10e-05 total=2.7845 tok=1.4739 draft=2.3071 jepa=0.5808 ver=0.1183 acc=0.959
+[sgjm] step=  1650 lr=2.71e-05 total=2.5335 tok=1.3111 draft=2.1356 jepa=0.5759 ver=0.1059 acc=0.965
+[sgjm] step=  1675 lr=2.35e-05 total=2.7698 tok=1.5070 draft=2.2580 jepa=0.5173 ver=0.0446 acc=0.992
+[sgjm] step=  1700 lr=2.01e-05 total=2.6665 tok=1.3886 draft=2.2448 jepa=0.5687 ver=0.1332 acc=0.951
+[sgjm] step=  1725 lr=1.69e-05 total=2.7182 tok=1.4400 draft=2.2627 jepa=0.5510 ver=0.0910 acc=0.967
+[sgjm] step=  1750 lr=1.41e-05 total=2.8234 tok=1.5018 draft=2.3270 jepa=0.5765 ver=0.1399 acc=0.944
+[sgjm] eval@1750: {'total': 2.7203762233257294, 'token': 1.4492271095514297, 'drafter': 2.2312930822372437, 'jepa': 0.5472100973129272, 'verifier': 0.18700036499649286, 'accept_acc': 0.9266803711652756}
+[sgjm] step=  1775 lr=1.14e-05 total=2.7856 tok=1.4894 draft=2.3021 jepa=0.5583 ver=0.0562 acc=0.991
+[sgjm] step=  1800 lr=9.05e-06 total=2.7224 tok=1.4569 draft=2.2385 jepa=0.5534 ver=0.0794 acc=0.976
+[sgjm] step=  1825 lr=6.94e-06 total=2.5533 tok=1.3331 draft=2.1598 jepa=0.5254 ver=0.0894 acc=0.971
+[sgjm] step=  1850 lr=5.11e-06 total=2.6588 tok=1.3909 draft=2.2249 jepa=0.5639 ver=0.1446 acc=0.941
+[sgjm] step=  1875 lr=3.56e-06 total=2.7611 tok=1.4923 draft=2.2583 jepa=0.5342 ver=0.0602 acc=0.983
+[sgjm] step=  1900 lr=2.28e-06 total=2.5448 tok=1.3436 draft=2.1108 jepa=0.5335 ver=0.1243 acc=0.958
+[sgjm] step=  1925 lr=1.28e-06 total=2.4495 tok=1.3363 draft=1.9427 jepa=0.5062 ver=0.1525 acc=0.950
+[sgjm] step=  1950 lr=5.71e-07 total=2.5200 tok=1.3679 draft=2.0473 jepa=0.4969 ver=0.0424 acc=0.991
+[sgjm] step=  1975 lr=1.43e-07 total=2.4132 tok=1.2540 draft=2.0291 jepa=0.5379 ver=0.1015 acc=0.968
+[sgjm] step=  1999 lr=2.28e-10 total=2.4536 tok=1.2587 draft=2.0797 jepa=0.5661 ver=0.1346 acc=0.950

results/hyde-rocm/250m.json

@@ -0,0 +1,17 @@
+{
+  "run": "sgjm-250m-rocm",
+  "machine": "hyde",
+  "backend": "rocm",
+  "gpu": "AMD Radeon 8060S (Strix Halo)",
+  "hip_version": "7.2.53211",
+  "pytorch_version": "2.11.0",
+  "date": "2026-05-18",
+  "steps": 10000,
+  "elapsed_s": 4128.5,
+  "elapsed_min": 68.8,
+  "steps_per_sec": 2.42,
+  "final_total_loss": 1.3466,
+  "final_token_loss": 0.5843,
+  "final_accept_acc": 0.9956,
+  "initial_total_loss": 8.8398
+}

results/hyde-rocm/25m.json

@@ -0,0 +1,17 @@
+{
+  "run": "sgjm-25m-rocm",
+  "machine": "hyde",
+  "backend": "rocm",
+  "gpu": "AMD Radeon 8060S (Strix Halo)",
+  "hip_version": "7.2.53211",
+  "pytorch_version": "2.11.0",
+  "date": "2026-05-18",
+  "steps": 5000,
+  "elapsed_s": 1252.1,
+  "elapsed_min": 20.9,
+  "steps_per_sec": 3.99,
+  "final_total_loss": 0.6943,
+  "final_token_loss": 0.1523,
+  "final_accept_acc": 0.9799,
+  "initial_total_loss": 8.7234
+}

results/phase5-ablation-25m-mlx/sgjm_full.json

@@ -0,0 +1,58 @@
+{
+  "card": {
+    "name": "sgjm_full",
+    "hypothesis": "All four losses (token + drafter + jepa + verifier) contribute.",
+    "overrides": {},
+    "expected_signal": "Best on combined score; sets the ceiling.",
+    "arch": "sgjm",
+    "pair_with_baseline": true
+  },
+  "elapsed_sec": 436.6732909679413,
+  "sgjm_metrics": {
+    "n_tokens": 65536,
+    "n_positions": 64512,
+    "token_nll": 0.1010679779574275,
+    "token_ppl": 1.1063518467459919,
+    "branch_acceptance_rate": 0.6330605158730159,
+    "jepa_top1_acc": 0.9747798859126984,
+    "jepa_chance_top1": 0.1111111111111111,
+    "merge_precision_js": 0.5779976844787598,
+    "random_pair_js": 0.6891302749183073,
+    "merge_precision_advantage": 1.1922716879735724,
+    "compute_per_accepted_token": 2990892.9755142014
+  },
+  "baseline_metrics": {
+    "n_tokens": 65536,
+    "token_nll": 0.08837755443528295,
+    "token_ppl": 1.0924004848267488,
+    "compute_per_token": 26354304.0
+  },
+  "comparison": {
+    "sgjm": {
+      "n_tokens": 65536,
+      "n_positions": 64512,
+      "token_nll": 0.1010679779574275,
+      "token_ppl": 1.1063518467459919,
+      "branch_acceptance_rate": 0.6330605158730159,
+      "jepa_top1_acc": 0.9747798859126984,
+      "jepa_chance_top1": 0.1111111111111111,
+      "merge_precision_js": 0.5779976844787598,
+      "random_pair_js": 0.6891302749183073,
+      "merge_precision_advantage": 1.1922716879735724,
+      "compute_per_accepted_token": 2990892.9755142014
+    },
+    "baseline": {
+      "n_tokens": 65536,
+      "token_nll": 0.08837755443528295,
+      "token_ppl": 1.0924004848267488,
+      "compute_per_token": 26354304.0
+    },
+    "nll_delta": 0.012690423522144556,
+    "compute_advantage": 8.811516900055278,
+    "gate_passed": false,
+    "gate_reasons": [
+      "merge_precision_advantage=1.19 < 1.5"
+    ]
+  },
+  "error": null
+}

results/phase5-ablation-25m-mlx/sgjm_no_drafter.json

@@ -0,0 +1,60 @@
+{
+  "card": {
+    "name": "sgjm_no_drafter",
+    "hypothesis": "Drafter loss isn't needed; backbone hidden states are enough.",
+    "overrides": {
+      "loss.drafter": 0.0
+    },
+    "expected_signal": "Drafter outputs become incoherent; branch_acceptance drops.",
+    "arch": "sgjm",
+    "pair_with_baseline": true
+  },
+  "elapsed_sec": 432.4698359966278,
+  "sgjm_metrics": {
+    "n_tokens": 65536,
+    "n_positions": 64512,
+    "token_nll": 0.09155610762536526,
+    "token_ppl": 1.0958782620946006,
+    "branch_acceptance_rate": 1.0,
+    "jepa_top1_acc": 0.9548766121031746,
+    "jepa_chance_top1": 0.1111111111111111,
+    "merge_precision_js": 0.6918166677157084,
+    "random_pair_js": 0.6893483032931689,
+    "merge_precision_advantage": 0.9964320541297599,
+    "compute_per_accepted_token": 1893416.25
+  },
+  "baseline_metrics": {
+    "n_tokens": 65536,
+    "token_nll": 0.08837755443528295,
+    "token_ppl": 1.0924004848267488,
+    "compute_per_token": 26354304.0
+  },
+  "comparison": {
+    "sgjm": {
+      "n_tokens": 65536,
+      "n_positions": 64512,
+      "token_nll": 0.09155610762536526,
+      "token_ppl": 1.0958782620946006,
+      "branch_acceptance_rate": 1.0,
+      "jepa_top1_acc": 0.9548766121031746,
+      "jepa_chance_top1": 0.1111111111111111,
+      "merge_precision_js": 0.6918166677157084,
+      "random_pair_js": 0.6893483032931689,
+      "merge_precision_advantage": 0.9964320541297599,
+      "compute_per_accepted_token": 1893416.25
+    },
+    "baseline": {
+      "n_tokens": 65536,
+      "token_nll": 0.08837755443528295,
+      "token_ppl": 1.0924004848267488,
+      "compute_per_token": 26354304.0
+    },
+    "nll_delta": 0.0031785531900823116,
+    "compute_advantage": 13.918917195307689,
+    "gate_passed": false,
+    "gate_reasons": [
+      "merge_precision_advantage=1.00 < 1.5"
+    ]
+  },
+  "error": null
+}

results/phase5-ablation-25m-mlx/sgjm_no_jepa.json

@@ -0,0 +1,63 @@
+{
+  "card": {
+    "name": "sgjm_no_jepa",
+    "hypothesis": "JEPA pruning is unnecessary; verifier alone is enough.",
+    "overrides": {
+      "loss.jepa": 0.0
+    },
+    "expected_signal": "jepa_top1_acc drops to chance; branch_acceptance flat.",
+    "arch": "sgjm",
+    "pair_with_baseline": true
+  },
+  "elapsed_sec": 433.9900426864624,
+  "sgjm_metrics": {
+    "n_tokens": 65536,
+    "n_positions": 64512,
+    "token_nll": 0.09915689704939723,
+    "token_ppl": 1.104239537893558,
+    "branch_acceptance_rate": 0.02683221726190476,
+    "jepa_top1_acc": 0.11546688988095238,
+    "jepa_chance_top1": 0.1111111111111111,
+    "merge_precision_js": 0.6183551251888275,
+    "random_pair_js": 0.6893096806704325,
+    "merge_precision_advantage": 1.1147472586403122,
+    "compute_per_accepted_token": 70565031.26516464
+  },
+  "baseline_metrics": {
+    "n_tokens": 65536,
+    "token_nll": 0.08837755443528295,
+    "token_ppl": 1.0924004848267488,
+    "compute_per_token": 26354304.0
+  },
+  "comparison": {
+    "sgjm": {
+      "n_tokens": 65536,
+      "n_positions": 64512,
+      "token_nll": 0.09915689704939723,
+      "token_ppl": 1.104239537893558,
+      "branch_acceptance_rate": 0.02683221726190476,
+      "jepa_top1_acc": 0.11546688988095238,
+      "jepa_chance_top1": 0.1111111111111111,
+      "merge_precision_js": 0.6183551251888275,
+      "random_pair_js": 0.6893096806704325,
+      "merge_precision_advantage": 1.1147472586403122,
+      "compute_per_accepted_token": 70565031.26516464
+    },
+    "baseline": {
+      "n_tokens": 65536,
+      "token_nll": 0.08837755443528295,
+      "token_ppl": 1.0924004848267488,
+      "compute_per_token": 26354304.0
+    },
+    "nll_delta": 0.010779342614114285,
+    "compute_advantage": 0.37347541023495795,
+    "gate_passed": false,
+    "gate_reasons": [
+      "branch_acceptance_rate=0.027 < 0.5",
+      "jepa_top1_acc=0.115 not meaningfully above chance=0.111",
+      "merge_precision_advantage=1.11 < 1.5",
+      "compute_advantage=0.37 < 1.0"
+    ]
+  },
+  "error": null
+}

results/phase5-ablation-25m-mlx/sgjm_no_verifier.json

@@ -0,0 +1,61 @@
+{
+  "card": {
+    "name": "sgjm_no_verifier",
+    "hypothesis": "Verifier loss isn't needed; rely on judge for acceptance.",
+    "overrides": {
+      "loss.verifier": 0.0
+    },
+    "expected_signal": "branch_acceptance_rate uninformative (~0.5).",
+    "arch": "sgjm",
+    "pair_with_baseline": true
+  },
+  "elapsed_sec": 433.9157350063324,
+  "sgjm_metrics": {
+    "n_tokens": 65536,
+    "n_positions": 64512,
+    "token_nll": 0.10087120719254017,
+    "token_ppl": 1.1061341704637397,
+    "branch_acceptance_rate": 0.21265811011904762,
+    "jepa_top1_acc": 0.9665333581349206,
+    "jepa_chance_top1": 0.1111111111111111,
+    "merge_precision_js": 0.581750750541687,
+    "random_pair_js": 0.6891995248234426,
+    "merge_precision_advantage": 1.1846989869488034,
+    "compute_per_accepted_token": 8903569.438005686
+  },
+  "baseline_metrics": {
+    "n_tokens": 65536,
+    "token_nll": 0.08837755443528295,
+    "token_ppl": 1.0924004848267488,
+    "compute_per_token": 26354304.0
+  },
+  "comparison": {
+    "sgjm": {
+      "n_tokens": 65536,
+      "n_positions": 64512,
+      "token_nll": 0.10087120719254017,
+      "token_ppl": 1.1061341704637397,
+      "branch_acceptance_rate": 0.21265811011904762,
+      "jepa_top1_acc": 0.9665333581349206,
+      "jepa_chance_top1": 0.1111111111111111,
+      "merge_precision_js": 0.581750750541687,
+      "random_pair_js": 0.6891995248234426,
+      "merge_precision_advantage": 1.1846989869488034,
+      "compute_per_accepted_token": 8903569.438005686
+    },
+    "baseline": {
+      "n_tokens": 65536,
+      "token_nll": 0.08837755443528295,
+      "token_ppl": 1.0924004848267488,
+      "compute_per_token": 26354304.0
+    },
+    "nll_delta": 0.012493652757257223,
+    "compute_advantage": 2.9599706256576477,
+    "gate_passed": false,
+    "gate_reasons": [
+      "branch_acceptance_rate=0.213 < 0.5",
+      "merge_precision_advantage=1.18 < 1.5"
+    ]
+  },
+  "error": null
+}

results/phase5-ablation-25m-mlx/sgjm_token_only.json

@@ -0,0 +1,64 @@
+{
+  "card": {
+    "name": "sgjm_token_only",
+    "hypothesis": "Aux losses don't help; equivalent to baseline plus dead weight.",
+    "overrides": {
+      "loss.drafter": 0.0,
+      "loss.jepa": 0.0,
+      "loss.verifier": 0.0
+    },
+    "expected_signal": "Should approximate baseline NLL with all aux metrics dead.",
+    "arch": "sgjm",
+    "pair_with_baseline": true
+  },
+  "elapsed_sec": 436.67354011535645,
+  "sgjm_metrics": {
+    "n_tokens": 65536,
+    "n_positions": 64512,
+    "token_nll": 0.08899632468819618,
+    "token_ppl": 1.093076638921474,
+    "branch_acceptance_rate": 0.18621341765873015,
+    "jepa_top1_acc": 0.11362227182539683,
+    "jepa_chance_top1": 0.1111111111111111,
+    "merge_precision_js": NaN,
+    "random_pair_js": 0.6896529843490999,
+    "merge_precision_advantage": 1.0,
+    "compute_per_accepted_token": 10167990.437026555
+  },
+  "baseline_metrics": {
+    "n_tokens": 65536,
+    "token_nll": 0.08837755443528295,
+    "token_ppl": 1.0924004848267488,
+    "compute_per_token": 26354304.0
+  },
+  "comparison": {
+    "sgjm": {
+      "n_tokens": 65536,
+      "n_positions": 64512,
+      "token_nll": 0.08899632468819618,
+      "token_ppl": 1.093076638921474,
+      "branch_acceptance_rate": 0.18621341765873015,
+      "jepa_top1_acc": 0.11362227182539683,
+      "jepa_chance_top1": 0.1111111111111111,
+      "merge_precision_js": NaN,
+      "random_pair_js": 0.6896529843490999,
+      "merge_precision_advantage": 1.0,
+      "compute_per_accepted_token": 10167990.437026555
+    },
+    "baseline": {
+      "n_tokens": 65536,
+      "token_nll": 0.08837755443528295,
+      "token_ppl": 1.0924004848267488,
+      "compute_per_token": 26354304.0
+    },
+    "nll_delta": 0.0006187702529132366,
+    "compute_advantage": 2.5918891410471114,
+    "gate_passed": false,
+    "gate_reasons": [
+      "branch_acceptance_rate=0.186 < 0.5",
+      "jepa_top1_acc=0.114 not meaningfully above chance=0.111",
+      "merge_precision_advantage=1.00 < 1.5"
+    ]
+  },
+  "error": null
+}

results/phase5-ablation-25m-mlx/summary.json

@@ -0,0 +1,311 @@
+{
+  "sweep": "ablation",
+  "ranked": [
+    {
+      "card": {
+        "name": "sgjm_no_drafter",
+        "hypothesis": "Drafter loss isn't needed; backbone hidden states are enough.",
+        "overrides": {
+          "loss.drafter": 0.0
+        },
+        "expected_signal": "Drafter outputs become incoherent; branch_acceptance drops.",
+        "arch": "sgjm",
+        "pair_with_baseline": true
+      },
+      "elapsed_sec": 432.4698359966278,
+      "sgjm_metrics": {
+        "n_tokens": 65536,
+        "n_positions": 64512,
+        "token_nll": 0.09155610762536526,
+        "token_ppl": 1.0958782620946006,
+        "branch_acceptance_rate": 1.0,
+        "jepa_top1_acc": 0.9548766121031746,
+        "jepa_chance_top1": 0.1111111111111111,
+        "merge_precision_js": 0.6918166677157084,
+        "random_pair_js": 0.6893483032931689,
+        "merge_precision_advantage": 0.9964320541297599,
+        "compute_per_accepted_token": 1893416.25
+      },
+      "baseline_metrics": {
+        "n_tokens": 65536,
+        "token_nll": 0.08837755443528295,
+        "token_ppl": 1.0924004848267488,
+        "compute_per_token": 26354304.0
+      },
+      "comparison": {
+        "sgjm": {
+          "n_tokens": 65536,
+          "n_positions": 64512,
+          "token_nll": 0.09155610762536526,
+          "token_ppl": 1.0958782620946006,
+          "branch_acceptance_rate": 1.0,
+          "jepa_top1_acc": 0.9548766121031746,
+          "jepa_chance_top1": 0.1111111111111111,
+          "merge_precision_js": 0.6918166677157084,
+          "random_pair_js": 0.6893483032931689,
+          "merge_precision_advantage": 0.9964320541297599,
+          "compute_per_accepted_token": 1893416.25
+        },
+        "baseline": {
+          "n_tokens": 65536,
+          "token_nll": 0.08837755443528295,
+          "token_ppl": 1.0924004848267488,
+          "compute_per_token": 26354304.0
+        },
+        "nll_delta": 0.0031785531900823116,
+        "compute_advantage": 13.918917195307689,
+        "gate_passed": false,
+        "gate_reasons": [
+          "merge_precision_advantage=1.00 < 1.5"
+        ]
+      },
+      "error": null
+    },
+    {
+      "card": {
+        "name": "sgjm_full",
+        "hypothesis": "All four losses (token + drafter + jepa + verifier) contribute.",
+        "overrides": {},
+        "expected_signal": "Best on combined score; sets the ceiling.",
+        "arch": "sgjm",
+        "pair_with_baseline": true
+      },
+      "elapsed_sec": 436.6732909679413,
+      "sgjm_metrics": {
+        "n_tokens": 65536,
+        "n_positions": 64512,
+        "token_nll": 0.1010679779574275,
+        "token_ppl": 1.1063518467459919,
+        "branch_acceptance_rate": 0.6330605158730159,
+        "jepa_top1_acc": 0.9747798859126984,
+        "jepa_chance_top1": 0.1111111111111111,
+        "merge_precision_js": 0.5779976844787598,
+        "random_pair_js": 0.6891302749183073,
+        "merge_precision_advantage": 1.1922716879735724,
+        "compute_per_accepted_token": 2990892.9755142014
+      },
+      "baseline_metrics": {
+        "n_tokens": 65536,
+        "token_nll": 0.08837755443528295,
+        "token_ppl": 1.0924004848267488,
+        "compute_per_token": 26354304.0
+      },
+      "comparison": {
+        "sgjm": {
+          "n_tokens": 65536,
+          "n_positions": 64512,
+          "token_nll": 0.1010679779574275,
+          "token_ppl": 1.1063518467459919,
+          "branch_acceptance_rate": 0.6330605158730159,
+          "jepa_top1_acc": 0.9747798859126984,
+          "jepa_chance_top1": 0.1111111111111111,
+          "merge_precision_js": 0.5779976844787598,
+          "random_pair_js": 0.6891302749183073,
+          "merge_precision_advantage": 1.1922716879735724,
+          "compute_per_accepted_token": 2990892.9755142014
+        },
+        "baseline": {
+          "n_tokens": 65536,
+          "token_nll": 0.08837755443528295,
+          "token_ppl": 1.0924004848267488,
+          "compute_per_token": 26354304.0
+        },
+        "nll_delta": 0.012690423522144556,
+        "compute_advantage": 8.811516900055278,
+        "gate_passed": false,
+        "gate_reasons": [
+          "merge_precision_advantage=1.19 < 1.5"
+        ]
+      },
+      "error": null
+    },
+    {
+      "card": {
+        "name": "sgjm_no_verifier",
+        "hypothesis": "Verifier loss isn't needed; rely on judge for acceptance.",
+        "overrides": {
+          "loss.verifier": 0.0
+        },
+        "expected_signal": "branch_acceptance_rate uninformative (~0.5).",
+        "arch": "sgjm",
+        "pair_with_baseline": true
+      },
+      "elapsed_sec": 433.9157350063324,
+      "sgjm_metrics": {
+        "n_tokens": 65536,
+        "n_positions": 64512,
+        "token_nll": 0.10087120719254017,
+        "token_ppl": 1.1061341704637397,
+        "branch_acceptance_rate": 0.21265811011904762,
+        "jepa_top1_acc": 0.9665333581349206,
+        "jepa_chance_top1": 0.1111111111111111,
+        "merge_precision_js": 0.581750750541687,
+        "random_pair_js": 0.6891995248234426,
+        "merge_precision_advantage": 1.1846989869488034,
+        "compute_per_accepted_token": 8903569.438005686
+      },
+      "baseline_metrics": {
+        "n_tokens": 65536,
+        "token_nll": 0.08837755443528295,
+        "token_ppl": 1.0924004848267488,
+        "compute_per_token": 26354304.0
+      },
+      "comparison": {
+        "sgjm": {
+          "n_tokens": 65536,
+          "n_positions": 64512,
+          "token_nll": 0.10087120719254017,
+          "token_ppl": 1.1061341704637397,
+          "branch_acceptance_rate": 0.21265811011904762,
+          "jepa_top1_acc": 0.9665333581349206,
+          "jepa_chance_top1": 0.1111111111111111,
+          "merge_precision_js": 0.581750750541687,
+          "random_pair_js": 0.6891995248234426,
+          "merge_precision_advantage": 1.1846989869488034,
+          "compute_per_accepted_token": 8903569.438005686
+        },
+        "baseline": {
+          "n_tokens": 65536,
+          "token_nll": 0.08837755443528295,
+          "token_ppl": 1.0924004848267488,
+          "compute_per_token": 26354304.0
+        },
+        "nll_delta": 0.012493652757257223,
+        "compute_advantage": 2.9599706256576477,
+        "gate_passed": false,
+        "gate_reasons": [
+          "branch_acceptance_rate=0.213 < 0.5",
+          "merge_precision_advantage=1.18 < 1.5"
+        ]
+      },
+      "error": null
+    },
+    {
+      "card": {
+        "name": "sgjm_token_only",
+        "hypothesis": "Aux losses don't help; equivalent to baseline plus dead weight.",
+        "overrides": {
+          "loss.drafter": 0.0,
+          "loss.jepa": 0.0,
+          "loss.verifier": 0.0
+        },
+        "expected_signal": "Should approximate baseline NLL with all aux metrics dead.",
+        "arch": "sgjm",
+        "pair_with_baseline": true
+      },
+      "elapsed_sec": 436.67354011535645,
+      "sgjm_metrics": {
+        "n_tokens": 65536,
+        "n_positions": 64512,
+        "token_nll": 0.08899632468819618,
+        "token_ppl": 1.093076638921474,
+        "branch_acceptance_rate": 0.18621341765873015,
+        "jepa_top1_acc": 0.11362227182539683,
+        "jepa_chance_top1": 0.1111111111111111,
+        "merge_precision_js": NaN,
+        "random_pair_js": 0.6896529843490999,
+        "merge_precision_advantage": 1.0,
+        "compute_per_accepted_token": 10167990.437026555
+      },
+      "baseline_metrics": {
+        "n_tokens": 65536,
+        "token_nll": 0.08837755443528295,
+        "token_ppl": 1.0924004848267488,
+        "compute_per_token": 26354304.0
+      },
+      "comparison": {
+        "sgjm": {
+          "n_tokens": 65536,
+          "n_positions": 64512,
+          "token_nll": 0.08899632468819618,
+          "token_ppl": 1.093076638921474,
+          "branch_acceptance_rate": 0.18621341765873015,
+          "jepa_top1_acc": 0.11362227182539683,
+          "jepa_chance_top1": 0.1111111111111111,
+          "merge_precision_js": NaN,
+          "random_pair_js": 0.6896529843490999,
+          "merge_precision_advantage": 1.0,
+          "compute_per_accepted_token": 10167990.437026555
+        },
+        "baseline": {
+          "n_tokens": 65536,
+          "token_nll": 0.08837755443528295,
+          "token_ppl": 1.0924004848267488,
+          "compute_per_token": 26354304.0
+        },
+        "nll_delta": 0.0006187702529132366,
+        "compute_advantage": 2.5918891410471114,
+        "gate_passed": false,
+        "gate_reasons": [
+          "branch_acceptance_rate=0.186 < 0.5",
+          "jepa_top1_acc=0.114 not meaningfully above chance=0.111",
+          "merge_precision_advantage=1.00 < 1.5"
+        ]
+      },
+      "error": null
+    },
+    {
+      "card": {
+        "name": "sgjm_no_jepa",
+        "hypothesis": "JEPA pruning is unnecessary; verifier alone is enough.",
+        "overrides": {
+          "loss.jepa": 0.0
+        },
+        "expected_signal": "jepa_top1_acc drops to chance; branch_acceptance flat.",
+        "arch": "sgjm",
+        "pair_with_baseline": true
+      },
+      "elapsed_sec": 433.9900426864624,
+      "sgjm_metrics": {
+        "n_tokens": 65536,
+        "n_positions": 64512,
+        "token_nll": 0.09915689704939723,
+        "token_ppl": 1.104239537893558,
+        "branch_acceptance_rate": 0.02683221726190476,
+        "jepa_top1_acc": 0.11546688988095238,
+        "jepa_chance_top1": 0.1111111111111111,
+        "merge_precision_js": 0.6183551251888275,
+        "random_pair_js": 0.6893096806704325,
+        "merge_precision_advantage": 1.1147472586403122,
+        "compute_per_accepted_token": 70565031.26516464
+      },
+      "baseline_metrics": {
+        "n_tokens": 65536,
+        "token_nll": 0.08837755443528295,
+        "token_ppl": 1.0924004848267488,
+        "compute_per_token": 26354304.0
+      },
+      "comparison": {
+        "sgjm": {
+          "n_tokens": 65536,
+          "n_positions": 64512,
+          "token_nll": 0.09915689704939723,
+          "token_ppl": 1.104239537893558,
+          "branch_acceptance_rate": 0.02683221726190476,
+          "jepa_top1_acc": 0.11546688988095238,
+          "jepa_chance_top1": 0.1111111111111111,
+          "merge_precision_js": 0.6183551251888275,
+          "random_pair_js": 0.6893096806704325,
+          "merge_precision_advantage": 1.1147472586403122,
+          "compute_per_accepted_token": 70565031.26516464
+        },
+        "baseline": {
+          "n_tokens": 65536,
+          "token_nll": 0.08837755443528295,
+          "token_ppl": 1.0924004848267488,
+          "compute_per_token": 26354304.0
+        },
+        "nll_delta": 0.010779342614114285,
+        "compute_advantage": 0.37347541023495795,
+        "gate_passed": false,
+        "gate_reasons": [
+          "branch_acceptance_rate=0.027 < 0.5",
+          "jepa_top1_acc=0.115 not meaningfully above chance=0.111",
+          "merge_precision_advantage=1.11 < 1.5",
+          "compute_advantage=0.37 < 1.0"
+        ]
+      },
+      "error": null
+    }
+  ]
+}

results/phase5-bench/benchmark_report.txt

@@ -0,0 +1,19 @@
+============================================================
+Generation Benchmark — SGJM-25M vs Autoregressive
+============================================================
+Checkpoint      : runs/sgjm-25m/best.safetensors
+Model step      : 4500
+Prompt length   : 64 tokens
+Tokens generated: 200 (SGJM: 50 steps×4; AR: 200 steps×1)
+Note: SGJM encodes 4-token node contexts; AR encodes growing context.
+      Theoretical compute advantage (full-context eval): 13.92× (gate run).
+
+Metric                                 SGJM  AR Baseline
+--------------------------------------------------------
+Tokens generated                        200          200
+Steps (model fwd passes)                100          200
+Acceptance rate (harness)            25.0%      100.0%
+Elapsed (s)                            1.32         1.31
+Tokens / sec                          151.7        153.0
+Speedup (SGJM/AR)                     0.99×
+============================================================