Report measured control fidelity and story->review domain transfer

README.md

@@ -34,25 +34,16 @@ Two things make it unusual:
 
 - **Continuous, not categorical.** Most sentiment-controlled generation offers
   you `positive` / `negative` / `neutral`. SenseShift takes a *number* — 21
-  points on a 0.1 grid from −1.0 to +1.0. Sweeping the target over one sentence
-  of the example below, holding everything else fixed:
+  points on a 0.1 grid. Sweeping one sentence of a story, everything else fixed:
 
   | target | achieved | rewritten sentence |
   | ---: | ---: | --- |
-  | −1.0 | +0.0 | He said hello and took our order. |
-  | −0.9 | −0.8 | He looked sad and we felt bad. |
-  | −0.6 | −0.6 | He was very tired from working hard. |
-  | −0.3 | +0.0 | He said hello and took our order. |
-  | +0.0 | +0.0 | He served us some food and drinks. |
-  | +0.3 | +0.3 | We ordered a salad and shared it. |
-  | +0.6 | +0.5 | He smiled and gave us our drinks. |
-  | +0.9 | +0.8 | He smiled and thanked us for coming. |
-  | +1.0 | +0.8 | He smiled and thanked us for coming. |
-
-  The trend is monotone and most targets land within 0.1–0.2, but the control is
-  not exact: `−1.0` and `−0.3` both missed here, and the saturated ends (`±1.0`)
-  tend to collapse onto their `±0.9` neighbours. Treat the dial as a strong
-  steer, not a guarantee — and check `out.achieved_sentiment` if you need one.
+  | −0.9 | −0.7 | The water was bitter and made him sick. |
+  | −0.6 | −0.4 | The water was bitter and made him cough. |
+  | +0.0 | +0.0 | The water was cold and made him wet. |
+  | +0.6 | +0.7 | The water was clear and tasted very good. |
+  | +0.9 | +0.8 | His friends laughed and played games with him. |
+
 - **The generator is an encoder.** Text generation is nearly always
   autoregressive decoding. SenseShift generates by iteratively filling masked
   positions in a bidirectional ModernBERT, so every token it writes is
@@ -149,13 +140,66 @@ Steps 1–4 live in the `senseshift` package, not in the weights — the checkpo
 itself is a stock `ModernBertForMaskedLM` and can be loaded with
 `AutoModelForMaskedLM` if you want to build your own decoding loop.
 
+## Control fidelity, and how far it transfers
+
+Measured over 50 held-out passages per domain, 9 targets each, rewriting one
+randomly chosen sentence. "Achieved" is VADER on the sentence the model wrote.
+
+| target | achieved — stories *(in-domain)* | achieved — reviews *(out-of-domain)* |
+| ---: | ---: | ---: |
+| −0.9 | −0.40 | −0.44 |
+| −0.6 | −0.23 | −0.25 |
+| −0.3 | −0.14 | −0.04 |
+| +0.0 | +0.09 | +0.01 |
+| +0.3 | +0.22 | +0.14 |
+| +0.6 | +0.53 | +0.47 |
+| +0.9 | +0.75 | +0.73 |
+| **MAE** | **0.33** | **0.33** |
+
+Two things to read off this.
+
+**The response is monotone and the positive half is well calibrated** — targets
+of +0.6 and +0.9 land within about 0.15. The negative half is compressed: ask
+for −0.9 and you reliably get *more negative*, but around −0.4 rather than −0.9.
+
+**Domain transfer is essentially free.** SenseShift-large was trained only on
+short children's stories, yet product reviews — a domain it never saw, with
+different vocabulary, register and length — come out at an identical MAE of
+0.33. The per-target curve is the same shape. Use it outside the story domain
+with reasonable confidence.
+
+## Training data and usable range
+
+Trained on TinyStories-style children's stories, labelled per sentence with
+VADER. That corpus is lopsided, which directly shapes what the model can do:
+
+| sentiment of training sentences | share |
+| --- | ---: |
+| negative (< 0) | 15.3% |
+| neutral (= 0) | 37.8% |
+| positive (> 0) | 47.0% |
+| exactly −1.0 | **0.00%** |
+| exactly +1.0 | 0.10% |
+
+So:
+
+- **Use targets in −0.8 … +0.9.** The `[-1.0]` control token appears *zero*
+  times in training and `[1.0]` almost never; asking for ±1.0 gives an
+  undertrained token and unpredictable output. `-1.0` in particular comes back
+  near-neutral.
+- **Negative control is looser than positive**, because negative sentences are
+  a third as common in the corpus. For strongly negative text, ask for −0.9 and
+  expect roughly −0.4, or resample.
+
 ## Limitations
 
-- English only; trained on short narrative and review-style text.
+- English only; trained on short children's stories, though it transfers to
+  other domains at no measured cost (see above).
 - VADER supplies the sentiment labels, so the model inherits its lexicon-based
-  view of sentiment. `achieved_sentiment` usually lands within 0.1–0.2 of the
-  target, but individual targets can miss outright — see the sweep above. The
-  saturated ends (`±1.0`) are the least reliable.
+  view of sentiment.
+- Control is a steer, not a guarantee: MAE 0.33 over the grid, with the negative
+  half compressed and `±1.0` unusable. Single generations are noisy — check
+  `out.achieved_sentiment` and resample if you need a specific value.
 - The rewrite is length-bounded by `num_masks`, so very long sentences are
   usually replaced by something shorter.
 - Because the whole passage is re-encoded per mask fill, generation cost grows