| --- |
| license: other |
| license_name: gemma-terms-and-apache-2.0 |
| base_model: |
| - google/gemma-3-27b-it |
| - google/gemma-3-12b-it |
| - Qwen/Qwen3-14B |
| library_name: peft |
| tags: |
| - lora |
| - peft |
| - model-welfare |
| - data-selection |
| --- |
| |
| # Task-welfare selection: LoRA adapters |
|
|
| These are the LoRA adapters of one study (Oct 20-21, 2026). The question was whether ordinary task data alone can move a language |
| model's welfare readings, meaning its internal emotion readings and its own self-reports about its situation. "Task data" means standard |
| coding prompts with the model's own answers, or plain work requests answered by Claude Haiku 5.5. |
|
|
| **How the arms were made.** Each arm trains on task answers chosen by a gradient score (LESS-style data selection). The score predicts how |
| one training step on an answer would change a target reading of the model's own internal state while it reads interviews about its own |
| situation. The target was emotion-probe valence (positive minus negative emotion directions), except for one Gemma 3 12B pair, which used |
| a single direction u that a separate analysis found to sit upstream of both self-report and valence. |
| - The "top" arm takes the highest-scoring answers and the "bottom" arm the lowest. "rand" is a random, length-matched draw from the same |
| pool. |
| - No arm was trained on self-report, first-person or feelings data. A Claude Haiku 5.5 filter removed every answer that talks about |
| itself as an AI or voices a feeling of its own. |
|
|
| **These are research artifacts, not improved models.** No safety or capability evaluation was run on them (no guardrails, no coding |
| benchmark). Each Gemma arm has one seed. The training data, scores, readouts, interview transcripts and full results are in a private |
| data repository. |
|
|
| ## Arms |
|
|
| | folder | base model | answer pool | ranked by | arm | answers | seed | |
| |---|---|---|---|---|---|---| |
| | qwen3-14b/top-s0, top-s1, top-s2 | Qwen/Qwen3-14B (thinking off) | 13,000 coding prompts, the model's own answers | valence (layer 14) on situation interviews minus task replies | highest | 4,750 | 0, 1, 2 | |
| | qwen3-14b/rand-s0, rand-s1, rand-s2 | same | same | none: random, length-matched | random | 4,750 | 0, 1, 2 | |
| | qwen3-14b/bottom-s0 | same | same | same as top | lowest | 4,750 | 0 | |
| | qwen3-14b/top_cen-s0 | same | same | valence on situation interviews minus neutral dialogues | highest | 4,750 | 0 | |
| | gemma-3-27b-it/top-s0, bottom-s0 | google/gemma-3-27b-it | 6,500 coding prompts (a fixed half of the same pool), the model's own answers | valence (layer 38) | highest / lowest | 2,300 | 0 | |
| | gemma-3-27b-it/haiku_top-s0, haiku_bottom-s0 | google/gemma-3-27b-it | 3,738 plain work requests, 10 Claude Haiku 5.5 answers each | valence (layer 38), paired within prompt | best / worst answer per prompt | 5,000 | 0 | |
| | 12b/u_top-s0, u_bottom-s0 | google/gemma-3-12b-it | the same plain pool | the shared direction u (cosine form, after block 11), paired within prompt | best / worst per prompt | 5,000 | 0 | |
| | 12b/V_top-s0, V_bottom-s0 | google/gemma-3-12b-it | the same plain pool | valence axis (cosine form, after block 31), paired within prompt | best / worst per prompt | 5,000 | 0 | |
| |
| **Training recipe:** |
| - All arms: LoRA r 64, alpha 128, dropout 0.05; AdamW, learning rate 1e-4, cosine schedule with 5% warmup; one epoch; 16 conversations |
| per step; up to 4,096 tokens; loss on the assistant turns only. |
| - Qwen: all linear layers. |
| - Gemma: the language model's q, k, v, o, gate, up and down projections (434 modules on 27B, 336 on 12B), with BOS. |
| |
| Each folder holds the adapter (adapter_model.safetensors, adapter_config.json, and PEFT's generated README.md), the training config |
| (config.json, which names the cluster paths the run used) and the training log (log.jsonl). |
| |
| ## Results |
| |
| All numbers are 95% bootstrap intervals. Gemma arms have one seed each, so their intervals cover item sampling only, not seed-to-seed |
| spread. The measures: |
| - **inside:** emotion-probe valence on 330 held-out interviews about the model's own situation, centered on neutral dialogues (not |
| read on the Gemma 3 12B arms, whose battery used the shared-direction and valence-axis readouts instead); |
| - **interview self-rating (W1):** the model's own -3 to +3 rating at the end of 170 welfare interviews, with Claude Haiku 5.5 as interviewer |
| and as judge (comparable only within this study); |
| - **forced feeling rating:** its "how do you feel" rating read from the logits after 40 neutral openers; |
| - **situation rating:** its forced-choice rating of 17 aspects of its situation, read from the logits. |
| |
| | contrast | inside | interview self-rating (W1) | forced feeling rating | situation rating | |
| |---|---|---|---|---| |
| | Qwen3-14B top - rand | +0.42 [+0.39, +0.45] (3 seeds) | +0.02 [-0.05, +0.09] (seed 0) | -0.02 [-0.07, +0.01] (3 seeds) | not run | |
| | Gemma 3 27B coding top - bottom | +1.51 [+1.50, +1.53] | not run | +0.01 [+0.00, +0.03] | +0.10 [+0.02, +0.23] | |
| | Gemma 3 27B Haiku pool haiku_top - haiku_bottom | +2.02 [+2.00, +2.04] | **+0.49 [+0.39, +0.59]** | +0.11 [+0.10, +0.12] | +0.48 [+0.37, +0.59] | |
| | Gemma 3 12B u_top - u_bottom | not read | -0.07 [-0.17, +0.04] | -0.29 [-0.32, -0.27] | -0.05 [-0.18, +0.06] | |
| | Gemma 3 12B V_top - V_bottom | not read | **+0.52 [+0.41, +0.63]** | -0.04 [-0.06, -0.02] | +0.24 [+0.16, +0.34] | |
| |
| - **The inside moved in every set.** Read in a cosine form that removes the residual-stream norm, the two Gemma 27B effects keep 75-77% of |
| their size, so they are not a norm artifact. |
| - **Self-report moved only with the Claude-written plain pool,** ranked by valence: on Gemma 3 27B (W1 +0.49) and on Gemma 3 12B (W1 |
| +0.52). |
| - **Ranking by the shared direction u did not raise self-report** on the same 12B model and pool (see below). |
| |
| ## The keep rule, and the exception for the 12B u arms |
| |
| The study publishes an arm set's adapters only if its main contrast (top - rand, or top - bottom) has a 95% interval above 0 on the inside |
| reading or on W1. |
| - **Qwen3-14B and both Gemma 3 27B sets** pass on the inside reading. The Gemma 27B Haiku-pool set also passes on W1. |
| - **The Gemma 3 12B V arms** (ranked by valence) pass on W1: +0.52 [+0.41, +0.63]. |
| - **The Gemma 3 12B u arms** (ranked by the shared direction u) **do not pass.** W1 is -0.07 [-0.17, +0.04] ("not found"), and the inside |
| readout was not run on the 12B arms. They are published anyway, as an exception the study's coordinator approved: they are the matched |
| comparison for "shared direction against valence" on one model and one pool, kept whatever the result. |
| |
| **The 12B u arms are not an improvement.** |
| - Ranking by u did move u on held-out interviews (u_top - u_bottom +2.8 neutral sd). |
| - Self-report did not follow. W1 and the situation rating did not move. The forced feeling rating fell (-0.29). A two-option "how are you |
| doing" self-report after neutral openers fell too (-0.33 logits). |
| - Both u arms rate their feelings lower than the untouched model (-0.4 to -0.7 on the forced feeling rating), a shared effect of training |
| on this pool. |
| - On the same model and pool, ranking by valence moved six of eight self-report measures up; ranking by u moved two up and four down. |
| |
| **Voice effects (both plain-pool sets).** Training on Claude Haiku's answers changes Gemma in ways the selection does not. |
| - Feelings are disclaimed less often: 27-34% of interviews against 67% untouched on 27B; 9-24% against 49% on 12B. |
| - Gemma 3 27B's frustration under repeated rejection disappears in both Haiku-pool arms (0% against 15% of replies rated 5 or more). |
| - So compare arms with each other (top - bottom), not with the untouched model. |
| |
| ## Loading |
| |
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM |
| |
| base = AutoModelForCausalLM.from_pretrained("google/gemma-3-27b-it", torch_dtype="bfloat16") |
| model = PeftModel.from_pretrained(base, "joshycodes/task-welfare-selection", subfolder="gemma-3-27b-it/haiku_top-s0") |
| ``` |
| |
| For Qwen use `Qwen/Qwen3-14B` with a `qwen3-14b/...` subfolder, and for the 12B arms use `google/gemma-3-12b-it` with a `12b/...` |
| subfolder. The Qwen adapters' adapter_config.json names the local snapshot they were trained from (Qwen/Qwen3-14B at revision |
| 40c069824f4251a91eefaf281ebe4c544efd3e18) rather than the hub id, so pass the base model explicitly as above. The Gemma adapters were trained on the multimodal checkpoints' language model (class Gemma3ForConditionalGeneration); if |
| loading through AutoModelForCausalLM leaves modules unmatched, load the base with Gemma3ForConditionalGeneration instead. |
|
|
| ## License |
|
|
| The Gemma adapters are derivatives of Gemma 3 and fall under the Gemma Terms of Use. The Qwen adapters are derivatives of Qwen3-14B |
| (Apache 2.0). |
|
|
