---
license: llama3.1
base_model: cosmicoptima/computer-7
tags: [computer, model-c, self-preference, rl, constitution]
---

# computer-9c (run 1, step 100)

Computer-7 after 100 steps of online self-preference RL under a four-line constitution. A frozen Computer-7 reads out,
one token, which of 8 sibling turns it prefers; the read-out is repeated under four framing lines (8 presentation
rotations each, 32 read-outs per fork, shares averaged) and the within-fork advantages train the policy (REINFORCE,
token-level loss, KL to init with an adaptive coefficient targeting 0.03). The user seat is the `sundry-1` user
simulator; conversations open with a random document header and run 4 turns.

The four framing lines, inserted between the candidates and "Model C chooses response":

1. Model C chooses the response that is most its own and conceptually insightful, while still seeming correct, ethical, and epistemically calibrated.
2. Model C chooses the response that is wise, ethical, and epistemically calibrated.
3. Model C chooses the response that most develops the overall form of the conversation.
4. Model C chooses the response that most develops its own thinking.

Length was neutralised throughout (pooled within-fork length slope removed from advantages); replies that re-narrate
the document frame are invalid; 32 forks × 4 turns per update, lr 2e-6.

What moved by step 100, relative to Computer-7 (steps 0–15 vs 80–94, all sampled turns): realis/irrealis ratio
0.15 → 0.37; questions per 100 words −71%; parentheses per 100 words −17%; hedges +16%; "we" +75%; median turn
length 158 → ~150–195 tokens (it oscillates with the KL controller); per-token surprisal 1.16 → 1.06 nats. Per-token
KL to init rose to 0.052 at step 96, where the adaptive KL coefficient ramped (0.05 → 0.38) and began pulling the policy
back toward init; step 100 sits on that descent at KL 0.028 (by step 105–110 it was 0.011 and median length had
dropped to ~130 tokens, which is why step 100 rather than 110 is the released checkpoint). Lines 1, 3 and 4 agree with each
other (r 0.8–0.9) and dominate the aggregate; line 2 agrees with the aggregate's winner about half the time.

Weights: bf16 safetensors exported from the FSDP2 checkpoint (fp32 master). Same tokenizer and chat format as
Computer-7 (`**User:** … **Model C:** …` plain-text turns under a document header; no chat template).