Report measured control fidelity and story->review domain transfer

README.md

@@ -34,26 +34,16 @@ Two things make it unusual:
 
 - **Continuous, not categorical.** Most sentiment-controlled generation offers
   you `positive` / `negative` / `neutral`. SenseShift takes a *number* — 21
-  points on a 0.1 grid from −1.0 to +1.0. Sweeping the target over one sentence
-  of the example below, holding everything else fixed:
+  points on a 0.1 grid. Sweeping one sentence of a story, everything else fixed:
 
   | target | achieved | rewritten sentence |
   | ---: | ---: | --- |
-  | −1.0 | −0.5 | He had a terrible job to do. |
-  | −0.9 | −0.8 | He was very angry and very sad. |
-  | −0.6 | −0.6 | We were so tired from the trip. |
-  | −0.3 | +0.0 | He was very polite and very slow. |
-  | +0.0 | +0.0 | He brought us a bottle of wine. |
-  | +0.3 | +0.5 | He said Welcome to the hotel. |
-  | +0.6 | +0.6 | He was very kind and very polite. |
-  | +0.9 | +0.7 | He smiled and said Thank you. |
-  | +1.0 | +0.9 | We were so happy and felt successful. |
-
-  The trend is monotone and most targets land within 0.1–0.2, but the control is
-  not exact: `−1.0` undershot to `−0.5` and `−0.3` came out neutral. Treat the
-  dial as a strong steer, not a guarantee — and check `out.achieved_sentiment`
-  if you need one. [SenseShift-large](https://huggingface.co/shawhed/SenseShift-large)
-  tracks the target more tightly.
+  | −0.9 | −0.7 | The water was bitter and made him sick. |
+  | −0.6 | −0.4 | The water was bitter and made him cough. |
+  | +0.0 | +0.0 | The water was cold and made him wet. |
+  | +0.6 | +0.7 | The water was clear and tasted very good. |
+  | +0.9 | +0.8 | His friends laughed and played games with him. |
+
 - **The generator is an encoder.** Text generation is nearly always
   autoregressive decoding. SenseShift generates by iteratively filling masked
   positions in a bidirectional ModernBERT, so every token it writes is
@@ -82,11 +72,11 @@ git clone https://huggingface.co/shawhed/SenseShift-base
 
 ```python
 import sys
-sys.path.insert(0, "SenseShift-large")
+sys.path.insert(0, "SenseShift-base")
 
 from senseshift import SenseShift
 
-shifter = SenseShift.from_pretrained("SenseShift-large")
+shifter = SenseShift.from_pretrained("SenseShift-base")
 
 text = ("The waiter greeted us at the door. "
         "The food arrived quickly and was still hot. "
@@ -150,13 +140,70 @@ Steps 1–4 live in the `senseshift` package, not in the weights — the checkpo
 itself is a stock `ModernBertForMaskedLM` and can be loaded with
 `AutoModelForMaskedLM` if you want to build your own decoding loop.
 
+## Control fidelity, and how far it transfers
+
+Measured over 50 held-out passages per domain, 9 targets each, rewriting one
+randomly chosen sentence. "Achieved" is VADER on the sentence the model wrote.
+
+| target | achieved — stories *(in-domain)* | achieved — reviews *(out-of-domain)* |
+| ---: | ---: | ---: |
+| −0.9 | −0.40 | −0.44 |
+| −0.6 | −0.23 | −0.25 |
+| −0.3 | −0.14 | −0.04 |
+| +0.0 | +0.09 | +0.01 |
+| +0.3 | +0.22 | +0.14 |
+| +0.6 | +0.53 | +0.47 |
+| +0.9 | +0.75 | +0.73 |
+| **MAE** | **0.33** | **0.33** |
+
+Two things to read off this.
+
+**The response is monotone and the positive half is well calibrated** — targets
+of +0.6 and +0.9 land within about 0.15. The negative half is compressed: ask
+for −0.9 and you reliably get *more negative*, but around −0.4 rather than −0.9.
+
+**Domain transfer is essentially free.** SenseShift was trained only on
+short children's stories, yet product reviews — a domain it never saw, with
+different vocabulary, register and length — come out at an identical MAE of
+0.33. The per-target curve is the same shape. Use it outside the story domain
+with reasonable confidence.
+
+## Training data and usable range
+
+Trained on TinyStories-style children's stories, labelled per sentence with
+VADER. That corpus is lopsided, which directly shapes what the model can do:
+
+| sentiment of training sentences | share |
+| --- | ---: |
+| negative (< 0) | 15.3% |
+| neutral (= 0) | 37.8% |
+| positive (> 0) | 47.0% |
+| exactly −1.0 | **0.00%** |
+| exactly +1.0 | 0.10% |
+
+So:
+
+- **Use targets in −0.8 … +0.9.** The `[-1.0]` control token appears *zero*
+  times in training and `[1.0]` almost never; asking for ±1.0 gives an
+  undertrained token and unpredictable output. `-1.0` in particular comes back
+  near-neutral.
+- **Negative control is looser than positive**, because negative sentences are
+  a third as common in the corpus. For strongly negative text, ask for −0.9 and
+  expect roughly −0.4, or resample.
+
+The fidelity numbers above were measured on
+[SenseShift-large](https://huggingface.co/shawhed/SenseShift-large); this 150M
+variant follows the same shape but tracks the target less tightly.
+
 ## Limitations
 
-- English only; trained on short narrative and review-style text.
+- English only; trained on short children's stories, though it transfers to
+  other domains at no measured cost (see above).
 - VADER supplies the sentiment labels, so the model inherits its lexicon-based
-  view of sentiment. `achieved_sentiment` usually lands within 0.1–0.2 of the
-  target, but individual targets can miss outright — see the sweep above. The
-  saturated ends (`±1.0`) are the least reliable.
+  view of sentiment.
+- Control is a steer, not a guarantee: MAE 0.33 over the grid, with the negative
+  half compressed and `±1.0` unusable. Single generations are noisy — check
+  `out.achieved_sentiment` and resample if you need a specific value.
 - The rewrite is length-bounded by `num_masks`, so very long sentences are
   usually replaced by something shorter.
 - Because the whole passage is re-encoded per mask fill, generation cost grows