v154: FOUR of six predicates are vacuous (150/150) - my prediction was wrong on pathprefix, whose prefix belongs to the SCAFFOLD not the content. Content gating is available almost exclusively where it does not matter: the vacuous gain transforms hold 96.58% of stack value. Caught two instrument defects - a default-OFF flag my probe never set, and a space-only predicate blind to 59 tabs. 1223 tests, 26/26 mutations fire (part 3)

test_compressors.py

@@ -19246,5 +19246,247 @@ class TestPerRepairPredicatesAndMarkerPricing(unittest.TestCase):
                 tok, src,
                 tok + " reached the shipped compressor while unpromoted")
 
+
+class TestPredicateSelectivityAcrossTheStack(unittest.TestCase):
+    """Last turn the truncation marker's predicate turned out vacuous (at-cap bodies in 150/150 challenges). The
+    open question was whether any OTHER transform I hold has a genuinely selective predicate. I measured all six.
+
+    MY PREDICTION WAS WRONG ON PATHPREFIX. I expected it to be selective - "not every challenge mentions the
+    workspace path" - and it fires on 150/150. In hindsight the reason is structural: the harness constructs every
+    challenge inside that workspace, so the prefix is a property of the SCAFFOLD, not of task content. I was
+    reasoning about content when the predicate was about the container.
+
+    FOUR OF SIX PREDICATES ARE VACUOUS (marker, pathprefix, relabel, rolemark at 100% of challenges each); only
+    tilde (31.3%) and wscollapse (81.3%) are selective.
+
+    TRAP 1 IS THE FINDING, and it is not a happy one. Pairing each predicate with its MEASURED gain: the three
+    vacuous gain transforms carry +0.0243866763 = 96.58% of the stack's value, while the one selective gain
+    transform carries +0.0008634527 = 3.42%. CONTENT GATING IS AVAILABLE ALMOST EXCLUSIVELY WHERE IT DOES NOT
+    MATTER. The transforms worth money fire everywhere; the transform with a real predicate is the weakest member.
+
+    TRAP 3 gives a third kind of answer, neither "gate it" nor "the transform is bad": a real gate exists for
+    wscollapse, it is free, and it protects the smallest gain in the stack. AVAILABLE AND NEARLY POINTLESS IS NOT
+    THE SAME AS UNAVAILABLE.
+
+    TWO INSTRUMENT DEFECTS, BOTH CAUGHT BY VALIDATING AGAINST THE LIVE VARIANT. First, wscollapse produced output
+    byte-identical to shipped on all 150 challenges while I have a gain booked for it - because WS_COLLAPSE is
+    default OFF and my probe never set SOMA_WS_COLLAPSE. The zero was my harness, not the transform. Second, with
+    the flag on, the variant changed output on 2 challenges my predicate called empty, so my "forfeits exactly 0
+    occurrences" claim was WRONG: I searched only for interior SPACE runs, and the variant also collapses TAB runs.
+    Challenges 1649 and 1661 hold 59 tabs with no multi-space run. The corrected predicate is EXACT on all 150 -
+    0 too strict, 0 too loose - and moves occurrences 589 -> 623, skipped challenges 30 -> 28, surface reduction
+    10.0% -> 9.53%.
+    """
+
+    RP = Path("/var/lib/octave/sn114/repo/sel_report.json")
+    PR = Path("/var/lib/octave/sn114/repo/sel_predicates.json")
+    CO = Path("/var/lib/octave/sn114/repo/sel_concentration.json")
+    FX = Path("/var/lib/octave/sn114/repo/sel_fixed.json")
+    MS = Path("/var/lib/octave/sn114/repo/sel_missed.json")
+
+    @classmethod
+    def setUpClass(cls):
+        for name, path in (("rp", cls.RP), ("pr", cls.PR), ("co", cls.CO),
+                           ("fx", cls.FX), ("ms", cls.MS)):
+            if not path.exists():
+                raise unittest.SkipTest("selectivity result missing: " + path.name)
+            setattr(cls, name, json.loads(path.read_text()))
+
+    def test_population_is_non_vacuous(self):
+        self.assertEqual(self.pr["n"], 150)
+        self.assertEqual(
+            len(self.pr["predicates"]), 6,
+            msg="the predicate census no longer covers all six transforms")
+
+    def test_four_predicates_are_vacuous(self):
+        v = set(self.pr["vacuous"])
+        self.assertEqual(
+            v, {"marker", "pathprefix", "relabel", "rolemark"},
+            msg="the set of vacuous predicates changed, so the central census result "
+                "moved")
+        for k in v:
+            self.assertGreaterEqual(
+                self.pr["predicates"][k]["pct"], 95.0,
+                msg=k + " is recorded as vacuous but no longer fires on >=95% of "
+                        "challenges")
+
+    def test_two_predicates_are_selective(self):
+        s = set(self.pr["selective"])
+        self.assertEqual(
+            s, {"tilde", "wscollapse"},
+            msg="the set of selective predicates changed")
+        for k in s:
+            self.assertLess(
+                self.pr["predicates"][k]["pct"], 95.0,
+                msg=k + " is recorded as selective but now fires nearly everywhere")
+            self.assertGreater(
+                self.pr["predicates"][k]["occurrences"], 0,
+                msg=k + " has no occurrences at all, so the predicate is empty rather "
+                        "than selective")
+
+    def test_pathprefix_prediction_failure_recorded(self):
+        self.assertIn(
+            "WRONG ON PATHPREFIX", self.rp["prediction_outcome"],
+            msg="the record that my selectivity prediction failed was removed")
+        self.assertIn(
+            "SCAFFOLD", self.rp["prediction_outcome"],
+            msg="the explanation - the path belongs to the container, not the content - "
+                "was removed")
+        self.assertEqual(
+            self.pr["predicates"]["pathprefix"]["challenges"], self.pr["n"],
+            msg="pathprefix no longer fires on every challenge, which was the surprise")
+
+    def test_gating_is_available_where_it_does_not_matter(self):
+        """TRAP 1: selectivity is not value."""
+        t = self.rp["trap1_selectivity_is_not_value"]
+        g = t["measured_gains"]
+        vac = g["pathprefix"] + g["relabel"] + g["rolemark"]
+        sel = g["wscollapse"]
+        self.assertGreater(
+            vac, 0.0, "the vacuous gain transforms no longer carry positive value")
+        self.assertGreater(
+            sel, 0.0, "the selective gain transform no longer carries positive value")
+        self.assertGreater(
+            vac, sel * 10,
+            msg="the vacuous transforms no longer dominate the selective one by 10x, so "
+                "the 'gating where it does not matter' conclusion is gone")
+        self.assertGreater(
+            t["vacuous_share_pct"], 90.0,
+            msg="the vacuous share of stack value fell below 90%")
+        # anchor the derived share to the raw gains
+        self.assertAlmostEqual(
+            100.0 * vac / (vac + sel), t["vacuous_share_pct"], places=1,
+            msg="the recorded vacuous share no longer matches the measured gains")
+
+    def test_wscollapse_is_the_weakest_member(self):
+        g = self.rp["trap1_selectivity_is_not_value"]["measured_gains"]
+        for k in ("pathprefix", "relabel", "rolemark"):
+            self.assertGreater(
+                g[k], g["wscollapse"],
+                msg=k + " no longer exceeds wscollapse, so the selective transform is "
+                        "not the weakest member")
+
+    def test_default_off_harness_defect_recorded(self):
+        d = self.rp["harness_defect_caught_by_validation"]
+        self.assertIn(
+            "MY HARNESS, NOT THE TRANSFORM", d["diagnosis"],
+            msg="the record that the zero-change result was a harness defect was removed")
+        self.assertIn("SOMA_WS_COLLAPSE", d["diagnosis"])
+
+    def test_predicate_defect_recorded_and_fixed(self):
+        d = self.rp["predicate_defect_caught_and_fixed"]
+        self.assertIn(
+            "TAB runs", d["cause"],
+            msg="the cause of the incomplete predicate - unhandled tab runs - was "
+                "removed")
+        self.assertGreater(
+            len(d["missed"]), 0,
+            msg="the record of the challenges my predicate missed is empty, so this "
+                "test would pass without evidence")
+        self.assertEqual(
+            len(d["missed"]), len(self.ms),
+            msg="the recorded miss list no longer matches the measurement file")
+        for m in self.ms:
+            self.assertGreater(
+                m["tabs_in_output"], 0,
+                msg="a missed challenge has no tabs, contradicting the diagnosis")
+
+    def test_corrected_predicate_is_exact(self):
+        f = self.fx
+        self.assertEqual(
+            f["too_strict"], 0,
+            msg="the corrected predicate is too strict again, so the gate would forfeit "
+                "real occurrences")
+        self.assertEqual(
+            f["too_loose"], 0,
+            msg="the corrected predicate became loose, overstating the gate surface")
+        self.assertTrue(f["exact"])
+        self.assertGreater(
+            f["agree"], 100,
+            msg="the corrected predicate now agrees with the variant on fewer than 100 "
+                "challenges")
+        self.assertGreater(
+            f["occurrences"], self.co["occurrences"],
+            msg="the corrected predicate no longer finds MORE occurrences than the "
+                "space-only probe, which was the whole correction")
+        self.assertLess(
+            f["skipped_challenges"], self.co["zero_challenges"],
+            msg="the corrected predicate no longer skips fewer challenges")
+
+    def test_wscollapse_default_off_LIVE(self):
+        """Drive the live variant: it must be inert without the env flag and active with
+        it. This is the defect that fooled my own probe."""
+        import importlib.util as _iu
+        import os as _os
+        pl = "/var/lib/octave/sn114/external/SOMA-plugin/"
+
+        def _load(nm, fn):
+            spec = _iu.spec_from_file_location(nm, pl + fn)
+            m = _iu.module_from_spec(spec)
+            spec.loader.exec_module(m)
+            return m
+
+        nl = chr(10)
+        tab = chr(9)
+        probe = nl.join(['<message role="user">', "<text>",
+                         "def f():", tab + tab + "return 1",
+                         "a" + (" " * 6) + "b",
+                         "</text>", "</message>"])
+        prev = _os.environ.get("SOMA_WS_COLLAPSE")
+        try:
+            _os.environ.pop("SOMA_WS_COLLAPSE", None)
+            sh = _load("sel_sh_live", "structural_cot_compressor.py")
+            off = _load("sel_ws_off", "structural_wscollapse_compressor.py")
+            self.assertFalse(
+                off.WS_COLLAPSE,
+                msg="the wscollapse variant is no longer default OFF, which would mean "
+                    "an unpromoted transform is live")
+            base = sh.compress_content(probe)
+            self.assertEqual(
+                off.compress_content(probe), base,
+                msg="the disabled variant no longer matches shipped byte for byte")
+            _os.environ["SOMA_WS_COLLAPSE"] = "1"
+            on = _load("sel_ws_on", "structural_wscollapse_compressor.py")
+            self.assertTrue(
+                on.WS_COLLAPSE,
+                msg="setting SOMA_WS_COLLAPSE=1 no longer enables the variant")
+            got = on.compress_content(probe)
+            self.assertNotEqual(
+                got, base,
+                msg="the enabled variant no longer changes output on a probe holding "
+                    "both a tab run and a 6-space run")
+            # assert on values COPIED from the printed output: the 6-space INTERIOR run
+            # collapses, while the leading tab indentation is preserved by design (v140)
+            self.assertIn(
+                "a b", got,
+                msg="the enabled variant no longer collapses an interior 6-space run")
+            self.assertNotIn(
+                "a" + (" " * 6) + "b", got,
+                msg="the interior space run survived, so the variant did nothing")
+            self.assertIn(
+                tab + tab + "return 1", got,
+                msg="leading tab indentation was destroyed, which is the v140 defect")
+            # and a tab in INTERIOR position must collapse
+            probe2 = nl.join(['<message role="user">', "<text>",
+                              "x = 1" + tab + tab + "# note", "</text>", "</message>"])
+            got2 = on.compress_content(probe2)
+            self.assertNotIn(
+                tab + tab, got2,
+                msg="an interior tab run survived, which was the behaviour my predicate "
+                    "had missed on challenges 1649 and 1661")
+        finally:
+            if prev is None:
+                _os.environ.pop("SOMA_WS_COLLAPSE", None)
+            else:
+                _os.environ["SOMA_WS_COLLAPSE"] = prev
+
+    def test_shipped_has_no_ws_collapse(self):
+        src = Path("/var/lib/octave/sn114/external/SOMA-plugin/"
+                   "structural_cot_compressor.py").read_text()
+        for tok in ("WS_COLLAPSE", "WS_TAB_RUN", "collapse_whitespace"):
+            self.assertNotIn(
+                tok, src,
+                tok + " reached the shipped compressor while unpromoted")
+
 if __name__ == "__main__":
     unittest.main(verbosity=2)

variance_analysis.json

@@ -153,6 +153,14 @@
     ]
   },
   "findings": [
+    "FOUR OF SIX PREDICATES ARE VACUOUS: marker, pathprefix, relabel and rolemark each fire on 150/150 challenges; only tilde (31.33%) and wscollapse (80.0%) are selective",
+    "MY PREDICTION WAS WRONG ON PATHPREFIX. I expected it selective - 'not every challenge mentions the workspace path' - and it fires everywhere. The reason is structural: the harness builds every challenge inside that workspace, so the prefix is a property of the SCAFFOLD, not of task content. I was reasoning about content when the predicate was about the container",
+    "TRAP 1 IS THE FINDING: the three vacuous GAIN transforms carry +0.0243866763 = 96.58% of the stack's value while the one selective gain transform carries +0.0008634527 = 3.42%. CONTENT GATING IS AVAILABLE ALMOST EXCLUSIVELY WHERE IT DOES NOT MATTER - the transforms worth money fire everywhere, and the transform with a real predicate is the weakest stack member",
+    "TRAP 3 gives a third kind of answer, neither 'gate it' nor 'the transform is bad': a real free gate exists for wscollapse and it protects the smallest gain in the stack. AVAILABLE AND NEARLY POINTLESS IS NOT THE SAME AS UNAVAILABLE",
+    "A HARNESS DEFECT CAUGHT BY VALIDATION: wscollapse produced output byte-identical to shipped on all 150 challenges while I have a gain booked for it - because WS_COLLAPSE is default OFF and my probe never set SOMA_WS_COLLAPSE. The zero was my harness, not the transform",
+    "A PREDICATE DEFECT CAUGHT THE SAME WAY: with the flag on, the variant changed output on 2 challenges my predicate called empty, so my 'forfeits exactly 0 occurrences' claim was WRONG. I searched only for interior SPACE runs; the variant also collapses TAB runs, and challenges 1649/1661 hold 59 tabs with no multi-space run. The corrected predicate is EXACT on all 150 (0 too strict, 0 too loose), moving occurrences 589->623, skipped challenges 30->28, surface reduction 10.0%->9.53%. A PREDICATE MUST BE VALIDATED AGAINST THE TRANSFORM IT GATES, BY DRIVING IT",
+    "AND MY FIRST TEST ASSERTION WAS RECALLED RATHER THAN COPIED FROM THE VALUE, again: I asserted the variant collapses a tab run, but my probe put the tabs in LEADING INDENTATION, which wscollapse preserves by design (v140). Fixed by asserting on the printed interior collapse and adding an interior-tab probe",
+    "1223 tests pass; 26 mutations attempted and all 26 fire (22 stored-measurement, 4 live source including two behavioural regex neuterings). All five sources byte-identical to pristine",
     "THE MARKER PREDICATE IS VACUOUS: an at-cap span body is present in 150 of 150 challenges (100.0%) and in all 76 unclamped ones. There is NO content gate for the truncation marker - truncated bodies are what the compressor produces. Trap 2 caught this before I could claim a gate helped",
     "SO LAST TURN'S FRAMING WAS WRONG. I called needfix 'content-triggered repair'; it was the marker RIDING on the tilde's unrelated predicate - 'markers wherever tildes happen to be'. A GATE THAT SAVES MONEY IS NOT THEREBY A PRINCIPLED GATE: check that the predicate belongs to the thing it gates",
     "AND THE COINCIDENTAL GATE WAS ADVERSE, NOT MERELY ARBITRARY: the 47 tilde-bearing challenges average 23.596 at-cap bodies against 11.65 for the other 103 (2.025x) and hold 48.03% of all 2309 at-cap bodies while being 31.3% of challenges. Marking them cost 22.9 tokens/challenge against 15.1 on average - the gate selected exactly the MOST expensive challenges to mark",
@@ -25909,5 +25917,160 @@
     "policy_now_supported": "tilde repair on the 47 challenges that contain a tilde, compression where unclamped, and NO truncation marker - because the marker cannot be gated and its benefit cannot be priced with the instruments available",
     "still_not_promoted": "shipped remains byte-identical to pristine; the flip-probability asymmetry is still unmeasured",
     "mutation_note": "a third live mutation (renaming a _truncate helper) was DISCARDED because its anchor count was 0 - the symbol does not exist in shipped. v151's lesson: a mutation that changes nothing produces a meaningless result either way"
+  },
+  "predicate_selectivity_across_the_stack": {
+    "question": "last turn's marker predicate was vacuous. Does any OTHER transform I hold have a genuinely selective predicate?",
+    "prediction": "pathprefix selective; relabel and rolemark vacuous like the marker",
+    "prediction_outcome": "WRONG ON PATHPREFIX - it fires on 150/150. I was reasoning about task content when the workspace path is a property of the SCAFFOLD: the harness builds every challenge inside it",
+    "selectivity": {
+      "n": 150,
+      "predicates": {
+        "pathprefix": {
+          "challenges": 150,
+          "pct": 100.0,
+          "occurrences": 1991,
+          "mean_per_challenge": 13.273,
+          "max_per_challenge": 71,
+          "vacuous_at_challenge_level": true
+        },
+        "relabel": {
+          "challenges": 150,
+          "pct": 100.0,
+          "occurrences": 3381,
+          "mean_per_challenge": 22.54,
+          "max_per_challenge": 166,
+          "vacuous_at_challenge_level": true
+        },
+        "rolemark": {
+          "challenges": 150,
+          "pct": 100.0,
+          "occurrences": 3639,
+          "mean_per_challenge": 24.26,
+          "max_per_challenge": 168,
+          "vacuous_at_challenge_level": true
+        },
+        "wscollapse": {
+          "challenges": 120,
+          "pct": 80.0,
+          "occurrences": 589,
+          "mean_per_challenge": 3.927,
+          "max_per_challenge": 75,
+          "vacuous_at_challenge_level": false
+        },
+        "marker": {
+          "challenges": 150,
+          "pct": 100.0,
+          "occurrences": 2309,
+          "mean_per_challenge": 15.393,
+          "max_per_challenge": 104,
+          "vacuous_at_challenge_level": true
+        },
+        "tilde": {
+          "challenges": 47,
+          "pct": 31.33,
+          "occurrences": 269,
+          "mean_per_challenge": 1.793,
+          "max_per_challenge": 38,
+          "vacuous_at_challenge_level": false
+        }
+      },
+      "vacuous": [
+        "marker",
+        "pathprefix",
+        "relabel",
+        "rolemark"
+      ],
+      "selective": [
+        "tilde",
+        "wscollapse"
+      ]
+    },
+    "four_of_six_vacuous": {
+      "vacuous": [
+        "marker",
+        "pathprefix",
+        "relabel",
+        "rolemark"
+      ],
+      "selective": [
+        "tilde",
+        "wscollapse"
+      ],
+      "note": "threshold 95% of challenges"
+    },
+    "trap1_selectivity_is_not_value": {
+      "measured_gains": {
+        "pathprefix": 0.0137724979,
+        "relabel": 0.0066846686,
+        "rolemark": 0.0039295098,
+        "wscollapse": 0.0008634527,
+        "marker": -0.0028215624,
+        "tilde": -2.10764e-05
+      },
+      "vacuous_gain_transforms_component": 0.0243866763,
+      "selective_gain_transform_component": 0.0008634527,
+      "vacuous_share_pct": 96.58,
+      "finding": "the three vacuous GAIN transforms carry +0.0243866763 = 96.6% of the stack value; the one selective gain transform carries +0.0008634527 = 3.4%. CONTENT GATING IS AVAILABLE ALMOST EXCLUSIVELY WHERE IT DOES NOT MATTER - the transforms worth money fire everywhere, and the transform with a real predicate is the weakest stack member"
+    },
+    "trap2_both_granularities": {
+      "concentration": {
+        "occurrences": 589,
+        "zero_challenges": 30,
+        "median": 2.0,
+        "top1_pct": 12.73,
+        "top5_pct": 31.75,
+        "top10_pct": 43.8,
+        "top30_pct": 69.44,
+        "chars_skipped": 98780,
+        "chars_total": 987484,
+        "surface_reduction_pct": 10.0
+      },
+      "finding": "wscollapse occurrences are concentrated - top 10 challenges hold 43.8% of them, one alone 12.73% - but the gate is at challenge level and skips only zero-occurrence challenges, so concentration does not threaten it"
+    },
+    "trap3_third_kind_of_answer": "neither \"gate it\" nor \"the transform is bad\": a real gate exists, it is free, and it protects the smallest gain in the stack. AVAILABLE AND NEARLY POINTLESS IS NOT THE SAME AS UNAVAILABLE",
+    "harness_defect_caught_by_validation": {
+      "symptom": "the wscollapse variant produced output IDENTICAL to shipped on all 150 challenges while I have +0.0008634527 booked for it",
+      "diagnosis": "WS_COLLAPSE = os.environ.get(\"SOMA_WS_COLLAPSE\",\"0\")==\"1\" is default OFF by design, and my validation probe never set the env var. THE 0-CHANGES RESULT WAS MY HARNESS, NOT THE TRANSFORM",
+      "lesson": "a default-OFF variant is inert unless the probe enables it - diagnose a zero result before interpreting it (v149)"
+    },
+    "predicate_defect_caught_and_fixed": {
+      "symptom": "with the flag on, the variant changed output on 2 challenges my predicate called empty, so the 0-forfeit claim was WRONG",
+      "cause": "my probe only looked for interior SPACE runs; the variant also collapses TAB runs via WS_TAB_RUN. Challenges 1649 and 1661 hold 59 tabs with no multi-space run",
+      "missed": [
+        {
+          "cid": "1649",
+          "n_diff": 14,
+          "tabs_in_output": 56,
+          "sample": [
+            "'-\\t\\t\\tret = fcntl.flock(_fd(f), flags)'",
+            "'-\\t\\t\\treturn ret == 0'",
+            "'+\\t\\t\\ttry:'"
+          ]
+        },
+        {
+          "cid": "1661",
+          "n_diff": 1,
+          "tabs_in_output": 3,
+          "sample": [
+            "\">>> o.c8 = F('name')\\t# model has char fields 'c8' and 'name'\""
+          ]
+        }
+      ],
+      "corrected": {
+        "agree": 122,
+        "too_loose": 0,
+        "too_strict": 0,
+        "neither": 28,
+        "occurrences": 623,
+        "skipped_challenges": 28,
+        "chars_skipped": 94137,
+        "chars_total": 987484,
+        "surface_reduction_pct": 9.53,
+        "exact": true
+      },
+      "result": "the corrected predicate (space runs OR tabs) is EXACT on all 150: 0 too strict, 0 too loose. Occurrences 589 -> 623, skipped challenges 30 -> 28, surface reduction 10.0% -> 9.53%",
+      "lesson": "A PREDICATE MUST BE VALIDATED AGAINST THE TRANSFORM IT GATES, by driving it - my probe was the incomplete artefact, as it has been ~66 times now"
+    },
+    "still_not_promoted": "shipped remains byte-identical to pristine; the flip-probability asymmetry is still unmeasured"
   }
 }