CLAUDE_IDEAS.md

CLAUDE_IDEAS

Ideas beyond the explicit brief. Implemented ones shipped because they were cheap and clearly aligned; everything else is logged here for Corina to triage. (Per the blank check: no idea too crazy.)

A meta-note: this repo was accidentally built by two parallel Claude instances sharing one container. Instance A wrote the pipeline; instance B audited it and wrote the docs. Ideas below marked [A] were implemented in A's code; [B] in B's docs/cleanup. The fact that the two instances converged on nearly identical designs (same edition choices, same generic-prompt insight, same model pick) is itself a data point about how stable these decisions are given the conversation context — a tiny accidental self-consistency experiment.

Implemented

• [A] Generic-prompt conditioning. Corina asked for journal-prompt support; naive implementation (pair real prompts with random entries) would teach prompt-deafness. Solution: 20 prompts so generic any diary entry answers them ("What is the weather inside you?"). The model learns the form prompt→entry; at inference you can ask anything.
• [A] Entry sentinel tokens (<|entry|> / <|/entry|>): give clean generation boundaries and a reliable stop condition, and make multi-entry sampling possible later.
• [A] Pepys year-stratified subsampling. Pepys outweighs everyone ~10×; uniform subsampling would still keep him dominant and could drop whole years. Round-robin across years to a char budget keeps the full decade.
• [A] Synthetic chronology for Van Gogh. The Ludovici letters lack dates; rather than dropping dates, they're assigned in corpus order across 1881→1890, so date-conditioning can still steer along his emotional arc (earnest → fevered → calm).
• [A] Mansfield OCR year-repair (IQI4→1914 digit-confusion table) and fragment chunking (her undated scraps become entries instead of being discarded).
• [A] Mean-initialized new token embeddings — random init of 6 new rows on a 3B model causes an early loss spike; mean-init is the standard fix.
• [A] Sequence packing — concatenate-and-chunk; no padding waste on a small corpus.
• [B] PROVENANCE.md — edition-level copyright audit, including the declined ghosts (Frank, Nin) and why, so future corpus additions have a template for the legal reasoning.
• [B] HOW_TO.md generalization — written as a recipe for arbitrary ghosts, not a description of this repo, including the PD rules of thumb that consumed a third of our conversation.
• [B] Corpus cross-verification — independent re-download of all sources and checksum comparison against instance A's copies (matched), plus an audit that no non-PD text (the uploaded Anne Frank PDF) entered the pipeline. It did not.

Considered, not implemented — the triage list

Near-term / cheap:

• Persona-separability eval. Generate N entries per persona token, train a bag-of-words classifier (or just TF-IDF cosine to held-out real entries) to measure: (a) does each token produce its ghost? (b) is the blend actually mixed or secretly one voice? Turns "the séance feels right" into a number. ~50 lines.
• Death-boundary probes. Systematically sample dates: each ghost's life, each ghost's death-day, dates after death, dates before birth, far future (2126). Diaries are date-saturated text; what a date-conditioned ghost does outside its lifespan is a genuinely interesting (and slightly chilling) behavioral question.
• Mansfield OCR polish pass. Her corpus still carries OCR grit ("exquisitive cleanliness", stray quote spacing). A one-shot LLM cleanup pass (entry in → corrected entry out, edit-distance capped to prevent rewriting her) would raise voice fidelity. Keep the raw version; diff them.
• GGUF export for llama.cpp so the séance runs on a laptop, offline, by candlelight, as is proper.

The representation-engineering tier (Corina-bait):

• Persona vectors from persona tokens. The four token embeddings are literal learned persona directions. Extract activation deltas between persona-token and blank runs at each layer → compare to the control vectors from Corina's Llama 3.3 rep-eng project. Same phenomenon, two routes: trained-in tokens vs. extracted vectors. A natural bridge between this project and the Persona Atlas idea from our original brainstorm.
• Ghost algebra. Because persona control lives in embedding space, try arithmetic at inference: feed a weighted average of persona token embeddings (60% maclane + 40% pepys) instead of a discrete token. Continuous interpolation between dead writers. If it works it's a demo that writes itself; if it fails the failure mode is informative.
• Persona-token-prob ablation. Train at p ∈ {0, 0.25, 0.5, 0.75, 1.0}; measure blend quality and persona separability at each. Small corpus = cheap sweep = a tidy little writeup about control-token dropout.

Crossovers with Corina's other projects:

• vibeslogger × ghost-diary. Map generated entries onto the Russell circumplex (valence/arousal) with a small classifier. Then: (a) chart each ghost's affective signature — Pepys should cluster differently than MacLane; (b) inverse direction: take your vibeslogger mood point for today and condition the ghost on it ("write today's entry from high-arousal/low-valence"). Mood-conditioned séance.
• brain.vat ghost wing. The four ghosts as vat.social agents, writing entries in response to each other's days — or a "correspondence mode" where Van Gogh (a letter-writer by nature) writes to the others.
• A fifth token: <|corina|>. Reserved, untrained. Fine-tune locally on your own journals/vibeslogger entries and you join the séance — the model completes your days in your voice, alongside the dead. Privacy note: this must only ever be trained and run locally; it's also philosophically the sharpest version of the project (the Growing Block angle: a diary is the past refusing to stop existing).

Further out / weirder:

• Temporal drift atlas. Sweep the blended ghost across dates 1650→2100, embed the outputs, and visualize lexical/affective drift over conditioned time — does the blend "know" centuries? D3 visualization; overlaps with Corina's infovis toolkit.
• Séance interface. Tiny web UI (Vite/React, she has the stack): candle-dark theme, you type a date, the entry types itself out letter-by-letter. Daily-ghost-entry-as-email is the low-effort sibling.
• Cross-ghost interviews. Prompted mode already supports questions; a structured "interview transcript" format (same 10 questions to all four + blend) would make a great blog-post artifact.
• The Barbellion expansion pack. He was voted off the island but Journal of a Disappointed Man is sitting right there on Gutenberg, PD, pre-parsed format very MacLane-like. One cleaner function away.

Open questions for Corina

1. Should synthetic dates be visually marked at inference (e.g., the model never sees real Van Gogh dates — is that fine for the aesthetic)?
2. Is 1024 max_seq_len right? (Resolved: Increased to 2048 to prevent truncating Pepys's longer entries).
3. Blend default: is persona_token_prob=0.5 the right prior, or do you want the blend to be the primary artifact (→ lower it)?