Add RLM long-context note

Public note for the local RLM long-context result.

POST-rlm-local-longcontext.md

@@ -0,0 +1,64 @@
+# Recursive Language Models on a local 35B: a long-context test on dual RTX PRO 4000 (Blackwell)
+
+Short version: on a pair of RTX PRO 4000 Blackwell desktop GPUs, wrapping a local quantised 35B in
+the Recursive Language Models (RLM) inference scaffold made it much more reliable at answering
+questions from long documents. The gain comes from letting the model search its context with code,
+not from deep recursion, which fired but did not help on my tests. All local, no frontier API.
+
+## Setup
+
+- GPUs: 2x RTX PRO 4000 Blackwell (desktop variant, 24 GB GDDR7 each, 48 GB total, no NVLink).
+- Model: Qwen3.6-35B-A3B (Q4_K_M GGUF), served by llama.cpp's OpenAI-compatible server, 131,072-token
+  context, layer-split across both cards.
+- Technique: RLM (Zhang, Kraska, Khattab; arXiv 2512.24601; github.com/alexzhang13/rlm), using the
+  library's OpenAI backend pointed at my local server, with the code REPL running in a Docker sandbox.
+- The paper's experiments used frontier API models (GPT-5 / GPT-5-mini) as the RLM root. I wanted to
+  know how RLM behaves when the root is a local quantised 35B on 24 GB workstation cards. I could not
+  find existing numbers for that, so here are mine.
+
+## What I tested
+
+A plain baseline (the long document placed directly in the prompt, one call) versus the RLM-wrapped
+model, on the same questions:
+
+- 100 long-context QA items from OOLONG-synth (two disjoint slices of 50), with contexts up to about
+  34,000 whitespace-delimited words, comfortably inside the model's context window.
+- One synthetic item of about 166,000 words, well beyond the 131k-token window, with the answer placed
+  past the truncation point.
+
+Scoring was deterministic matching against the answer key, hand-checked on the baseline's misses.
+
+## Results
+
+- In-window: the plain baseline got 37 of 100 right (37%). The RLM-wrapped model got 95 of 100 (95%).
+  Same model, same questions; the only difference is that RLM let it search and read the context with
+  code instead of swallowing the whole thing at once. The gap held across two disjoint 50-item slices
+  (baseline 38% then 36%, RLM 94% then 96%), so it is not a block-specific fluke.
+- Beyond-window: RLM answered the ~166k-word item by working through it in pieces. The plain baseline
+  could not, because the document does not fit the window.
+- Recursion: the deeper mode, where the model spawns child calls to break a task down, did fire when I
+  built items to force it (15 sub-calls at depth 2). But on that small, hard probe it scored worse than
+  the plain approach (1 of 5 versus 3 of 5). So the uplift here is from the REPL-over-context
+  mechanism, not from nested recursion. With a local Q4 35B, recursion switched on but did not earn its
+  keep on these tasks.
+- Speed: RLM is somewhat slower per question because it makes several passes over the context, but it
+  stayed in a practical range for offline document work.
+
+## What this is and is not
+
+- One model on one hardware setup, on a public benchmark slice plus one synthetic stress item.
+- OOLONG-synth is public, so some contamination is possible, but it would lift both the baseline and
+  RLM equally, so it does not explain the gap between them.
+- The recursion result rests on only 5 hard items, so read it as directional, not settled.
+- Whether recursion helps with a stronger root model is outside the scope of this note; these numbers
+  are specifically for a local quantised 35B.
+
+## Why it might matter
+
+If you run local and care about long-context reliability, this is a cheap lever: an open inference
+scaffold turned a 37% baseline into 95% on long-document QA, on 24 GB workstation cards, with no
+frontier API and no extra VRAM. The model's own long window already exists; RLM made it more reliable
+inside that window and let it reach past it.
+
+Credit to the RLM authors for the technique and the library. The setup and method above are enough to
+reproduce.