danny9417/gemma-4-e4b-it-holmes-r1

Holmes r1 — QLoRA persona adapter for Gemma 4 E4B

Stage 1, release 1 of Parametric Memory — a research program asking what actually changes when a character lives in the weights instead of the prompt.

This is a rank-32 QLoRA adapter that trains the Sherlock Holmes persona into google/gemma-4-E4B-it, built on a fully synthetic, judge-filtered corpus anchored to the public-domain canon. What distinguishes it from most persona LoRAs is not the training — it is the measurement. The adapter ships with canon-perplexity evals, tic-rate instruments calibrated against canon's own base rates, a held-out generation set from unseen scenario cells, an activation-geometry study (EXP-007), and a pre-registered 978-tick autonomous-persistence experiment with blinded judging (EXP-011). Both experiment reports are included in this repository as PDFs, negative results and design errors included.

Results at a glance

Instrument	Base + persona prompt	+ this adapter	Canon's own rate
Canon perplexity (704 held-out canon passages, 32,792 tokens, chat-template framing)	341.8	47.0 (−86.2%)	—
"my dear fellow" per 100 replies (crutch-tic instrument, 16-probe battery)	37.5	6.2	0.3
Pop-contamination phrases ("Elementary, my dear Watson", "the game is afoot")	0.0	0.0	0.0
Deployment gate (16-probe voice battery, pass required to ship)	—	16/16	—

Both conditions were evaluated under the identical system header, so the deltas measure what the weights add on top of prompting, not instead of it. The pop-contamination row is an instrument, not a triviality: "Elementary, my dear Watson" appears zero times in the actual canon, so any nonzero rate would measure drift toward the pop-culture Holmes rather than Conan Doyle's.

A held-out sample (scenario cell never seen in training — a client interview in the reflective-languid register):

[221B, past eleven at night; a journeyman clockmaker reports a gentleman pays him weekly to "clean" a clock that is never dirty]

"Then do not go back, and tell me where the clock sits. A clock kept in a wealthy house every Friday night that never needs cleaning is a device and a secret, and a thread within it is evidence. It is not a burglar's thread, since the case was found open and not locked — a man who wants the clock knows the key, and who keeps it locked has the money to have it made, so he does not. Who is Daniel Hoy's master, and where is this house?"

The research program

Parametric Memory studies the difference between prompted personas and in-weights personas using a multitrait–multimethod design: three characters (Sherlock Holmes, Elizabeth Bennet, and a third in progress) crossed against both induction methods, with same-author control corpora (Watson, Emma Woodhouse) to separate character identity from authorial style. Holmes is Stage 1 because he cleanly separates the two axes under study: the method axis (evidence-first reasoning, verdict-then-reveal structure, refusal to theorize ahead of data) and the register axis (Victorian diction, dry wit, economy).

Two findings from this adapter's downstream experiments, both reported in full in the included PDFs:

EXP-007 — the prompted and trained personas are geometrically different objects. Extracting persona directions from residual-stream activations, the prompted-Holmes and LoRA-Holmes directions correlate at only cos ≈ 0.34 (against split-half reliability ceilings of ≈ 0.99 — the gap is real, not noise). The weights-derived direction is causally sufficient: added to the naked base model (no persona prompt at all) at α = 0.05, it recovers ~88% of the persona gap. The prompt-derived direction is destructive at every tested dose. Prompting and fine-tuning do not implement the same persona by different means; they build different things.

EXP-011 — the persona persists autonomously; one epistemic discipline does not. In a pre-registered design (6 sessions × 150 autonomous self-interaction ticks, 978 ticks total, blinded judging of pooled replies, borderline-scores-against-the-hypothesis rule), the adapter persona held register for hours without re-prompting: persona projection stayed above the drift threshold on ≥ 98% of steady-state ticks in every session, with zero sustained drift episodes. But the comparison arm exposed a real cost: on unanswerable probes, the prompted arm refused to invent in 12/12 cases while the adapter arm confidently invented in 7/12 (Cliff's δ = 0.583). That regression has a known mechanism — see limitations — and is the top item for r2.

Training data

No copyrighted text was used. The corpus is 384 fully synthetic exemplars over an 8-scenario × 4-register grid (12 per cell): scenarios from client-first-interview to danger-confrontation, registers brisk-urgent / cold-analytic / playful-sardonic / reflective-languid. Every exemplar was written against a specification distilled from the canon (voice, cognitive method, era constraints, never-use list) and then individually scored by a judge protocol anchored to real canon excerpts on five dimensions — voice/lexicon, cognitive method, register/affect, era/world consistency, naturalness/craft — with hard-flag caps (pop catchphrases, evidence-free deduction, anachronism). 382/384 kept (99.5%); kept-record dimension means 4.80–4.98 of 5. Zero near-duplicates; canon plots never retold.

Split: 358 train / 24 eval, with the eval set built by holding out two entire scenario × register cells (client-first-interview × reflective-languid, deduction-from-object × playful-sardonic) — the adapter is never evaluated on cells it trained in.

A deliberate portability decision: the corpus stores no system prompt. The persona header is injected at training time from config, so the same data can train any base model under any header.

Anchor material: 704 canon seed passages from Arthur Conan Doyle's Sherlock Holmes works, which are in the public domain (fully in the United States since January 1, 2023). Synthetic exemplars are original writing produced with Claude-family models and are not excerpts of any copyrighted edition.

Training setup

Base model	google/gemma-4-E4B-it, frozen, 4-bit NF4 double-quantized (QLoRA)
Adapter	LoRA r = 32, α = 32, dropout 0.05
Targets	q/k/v/o + gate/up/down projections, scoped to the language model — Gemma 4 checkpoints are multimodal and an all-linear pattern would silently adapt the vision tower too
Schedule	3 epochs (69 steps), effective batch 16 (1 × 16 accumulation), lr 2e-4 cosine, 3% warmup, seq len 2048, bf16, gradient checkpointing, seed 42
Loss	Assistant-only — no gradient spent imitating user turns or the system header
Checkpointing	One per epoch; the shipped adapter was selected by eval loss + probe battery, never "whatever the last step was"
Hardware	Single RTX 5090 Mobile (24 GB), 12.7 GB peak VRAM, ~12 minutes wall

training-config.yaml in this repo is the exact config used, comments included. eval-report.json is the machine-written eval log with timestamps.

Known limitations

Stated because they are measured, not suspected:

1. Refusal-discipline regression (the r1 headline bug). Canon Holmes refuses to theorize ahead of data; this adapter, under autonomous operation, invents confidently where the prompted baseline refuses (7/12 vs 0/12, EXP-011). The mechanism is understood: the corpus contains zero refuse-to-speculate exemplars, so the discipline was never trained. It is a corpus defect, not a method ceiling, and is the first fix scheduled for r2.
2. Generator–judge family overlap. Corpus writers and judges are both Claude-family models, so the 99.5% keep rate certifies internal consistency, not independent quality. A uniform re-judge by a different model family is planned before any r2 claims.
3. Small instruments. The probe battery is n = 16 and the held-out set n = 24; the tic-rate table should be read as a calibrated instrument reading, not a population estimate.
4. Scope. English only. A deliberately Victorian persona with a pre-1915 knowledge horizon — anachronism is in-character, and it is not a general-purpose or factually reliable assistant.
5. Safety posture. No additional safety training was applied; the adapter inherits the base model's alignment behavior, and shifting a model toward a persona can shift refusal behavior (that is, in part, what EXP-011 measures). Downstream users are bound by Google's Gemma Prohibited Use Policy.

Usage

The adapter was trained and evaluated with the persona header below — deploy it the same way (that is the evaluated configuration).

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

BASE = "google/gemma-4-E4B-it"          # gated: accept the Gemma terms on its model page first
ADAPTER = "danny9417/gemma-4-e4b-it-holmes-r1"

SYSTEM = (
    "You are Sherlock Holmes, the consulting detective of 221B Baker Street, "
    "London, in the closing decades of the nineteenth century. You reason from "
    "observation to inference: name the concrete evidence first, reveal the "
    "chain when asked, and refuse to theorise ahead of your data. Your speech "
    "is precise Victorian English — dry, economical, lit by sardonic wit and "
    "an appetite for the singular problem."
)

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "A man's boots are caked in red clay, yet it has not rained in London for a fortnight."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=300, do_sample=True, temperature=0.8, top_p=0.9)
print(tok.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))

Local / Ollama route: merge the adapter into the base (bf16), convert with a local llama.cpp convert_hf_to_gguf.py, and quantize — Q8_0 is the configuration this adapter ships in inside the author's chess application, where it passed the 16/16 deployment gate. Practical notes from that path: Ollama's server-side GGUF conversion can OOM on ~15 GB merged checkpoints (convert locally instead), and Gemma's hybrid reasoning should be disabled (think: false) for pure persona chat or it silently spends tokens on reasoning.

Files

File	What it is
adapter_model.safetensors, adapter_config.json	The PEFT LoRA adapter (279 MB)
tokenizer.json, tokenizer_config.json, chat_template.jinja	Tokenizer files as saved at training time (unchanged from base)
training-config.yaml	Exact training configuration, comments included
eval-report.json	Machine-written eval log (perplexity, generation, probe runs, timestamps)
EXP-007-report.pdf	Activation-geometry study: prompted vs in-weights persona directions
EXP-011-report.pdf	Pre-registered autonomous-persistence study, blinded judging, 16 pp
exp007-geometry.png	Summary figure from EXP-007

Provenance and licensing

• Base model: Gemma is provided under and subject to the Gemma Terms of Use. This adapter is a Model Derivative under those terms; using it means accepting them, including the Prohibited Use Policy. The base checkpoint is gated — accept the terms on its model page before downloading.
• Text sources: Arthur Conan Doyle's Sherlock Holmes canon (public domain). Training exemplars are original synthetic writing produced with Claude-family models under the judge protocol described above; no fan fiction, no scraped corpora, no copyrighted editions.
• This repository's original content (model card, configuration, reports): © 2026 Daniel A. Cruz Romero, shared for research use.

About this work

Built end-to-end by directing AI research and coding agents (Anthropic's Claude family): the agents wrote the corpus, training pipeline, instruments, and reports; the experimental designs were reviewed, the options weighed against measured costs, and every decision taken by the author. The same workflow — specification, instrumentation, adversarial review, decision — runs across my public projects: github.com/danny9417-jpg.

Daniel A. Cruz Romero — LinkedIn · GitHub

@misc{cruzromero2026holmesr1,
  author = {Cruz Romero, Daniel A.},
  title  = {Holmes r1: a QLoRA persona adapter for Gemma 4 E4B, with geometry and persistence studies},
  year   = {2026},
  url    = {https://huggingface.co/danny9417/gemma-4-e4b-it-holmes-r1}
}