spencernorton commited on
Commit
b9f70a0
·
verified ·
1 Parent(s): 3e09449

Release Armiger Ultra v3.10.0 (v3.0-gold-r13.10, DHT seq=13 / seq=3013)

Browse files
2aab57ea3886d959db583006f2ce43c95a87ad24.torrent DELETED
Binary file (4.7 kB)
 
4c3f3dd03f2d29070acd24370a85bfb72f0051da.torrent DELETED
Binary file (1.95 kB)
 
8ee71b4e35370b457e2b5cfe03c76023baf68cff.torrent ADDED
Binary file (4.7 kB). View file
 
README.md CHANGED
@@ -22,13 +22,13 @@ tags:
22
  - rats-search
23
  ---
24
 
25
- # Armiger Ultra `v3.8.0`
26
 
27
- **Release:** [`v3.8.0`](https://github.com/spencernorton/armiger/releases/tag/v3.8.0) · **Weights:** `33.3 MB` FP32 ONNX + `17.6 MB` INT8 ONNX · **Parameters:** `8.05M` · **License:** Apache-2.0
28
 
29
  **Armiger Ultra** is an 8.05M-parameter multi-head byte-level RoPE Transformer (`0.21 ms/item` batched GPU) and 6-plugin pipeline for BitTorrent and DHT metadata classification, canonical title/year extraction, audio/subtitle verification, malware filtering, and swarm liveliness scoring.
30
 
31
- ![Armiger Ultra v3.8.0 Overview](assets/armiger_overview.png)
32
 
33
  ## Quickstart
34
 
@@ -47,7 +47,7 @@ Armiger package and verifies its declared dependency checksums.
47
 
48
  ### Pre-Built Docker Container & Compose Stacks (`v3.4.1`)
49
 
50
- Library and container 3.4.1 fix connector startup permissions and use the v3.8.0 model bundle.
51
 
52
  Run the pre-built `armiger-ultra`:
53
 
 
22
  - rats-search
23
  ---
24
 
25
+ # Armiger Ultra `v3.10.0`
26
 
27
+ **Release:** [`v3.10.0`](https://github.com/spencernorton/armiger/releases/tag/v3.10.0) · **Weights:** `33.3 MB` FP32 ONNX + `17.6 MB` INT8 ONNX · **Parameters:** `8.05M` · **License:** Apache-2.0
28
 
29
  **Armiger Ultra** is an 8.05M-parameter multi-head byte-level RoPE Transformer (`0.21 ms/item` batched GPU) and 6-plugin pipeline for BitTorrent and DHT metadata classification, canonical title/year extraction, audio/subtitle verification, malware filtering, and swarm liveliness scoring.
30
 
31
+ ![Armiger Ultra v3.10.0 Overview](assets/armiger_overview.png)
32
 
33
  ## Quickstart
34
 
 
47
 
48
  ### Pre-Built Docker Container & Compose Stacks (`v3.4.1`)
49
 
50
+ Library and container 3.4.1 fix connector startup permissions and use the v3.10.0 model bundle.
51
 
52
  Run the pre-built `armiger-ultra`:
53
 
armiger-v3.10.0-gold-r13-10.torrent ADDED
Binary file (12.1 kB). View file
 
armiger-v3.8.0-gold-r13-8.torrent DELETED
Binary file (12.1 kB)
 
armiger-weights-int8-v3.10.0.torrent ADDED
Binary file (1.95 kB). View file
 
armiger-weights-int8-v3.8.0.torrent DELETED
Binary file (1.95 kB)
 
armiger-weights-ultra-v3.10.0.torrent ADDED
Binary file (4.7 kB). View file
 
armiger-weights-ultra-v3.8.0.torrent DELETED
Binary file (4.7 kB)
 
1a69937906c8a43f32499de578e751df049dde16.torrent → armiger-zero-cue-db-v3.10.0.torrent RENAMED
Binary files a/1a69937906c8a43f32499de578e751df049dde16.torrent and b/armiger-zero-cue-db-v3.10.0.torrent differ
 
armiger.int8.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7ee3dfdaae4283faecabf06bc7d7644d1747d959264506b07065a3035fe6ffd0
3
- size 17606389
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c39bad25f5e31289d575243473589e023c63327b5d846e26aaeebf9aee7b054c
3
+ size 17606362
armiger_ultra.fp32.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3125e70b22303f1a548e7b890d2b1c078858eed1b987cc9d63c52188fbc630c0
3
  size 33260312
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70286459da47287eb5d984cd67251e0f80f511ceee0ad6ad34c7c2334c9223ec
3
  size 33260312
armiger_ultra_pipeline.py CHANGED
@@ -258,7 +258,12 @@ _AUDIOBOOK_FILE_TREE_RE = re.compile(
258
  def _file_tree_structural_override(row: Mapping[str, Any]) -> Tuple[Optional[str], str]:
259
  """Check physical file-tree extensions (.m4b/.afpk audiobooks, .cbz/.cbr comics, game engine trees, multi-episode packs) before title lookup."""
260
  raw_name = str(row.get("name") or row.get("title") or "")
261
- files = row.get("files") or []
 
 
 
 
 
262
  ep_files = 0
263
  has_afpk = False
264
  for f in files:
@@ -1409,7 +1414,7 @@ class ArmigerUltraPipeline:
1409
  except ImportError:
1410
  from oci_sqlite_catalog import build_torrent_bundle
1411
  bobj = build_torrent_bundle(
1412
- bundle_name=f"armiger-weights-{self.variant}-v3.8.0",
1413
  files_map=files_map,
1414
  pubkey_32=b"\x00" * 32,
1415
  salt=f"armiger:weights:{self.variant}".encode("utf-8"),
@@ -2455,7 +2460,7 @@ class ArmigerUltraPipeline:
2455
  verified_files_map[str(fname)] = fbytes
2456
 
2457
  recomputed = build_torrent_bundle(
2458
- bundle_name=f"armiger-weights-{self.variant}-v3.8.0",
2459
  files_map=verified_files_map,
2460
  pubkey_32=bytes.fromhex(eff_pubkey),
2461
  salt=eff_salt,
@@ -2758,7 +2763,7 @@ class ArmigerUltraPipeline:
2758
  }
2759
 
2760
  recomputed_bundle = build_torrent_bundle(
2761
- bundle_name=f"armiger-weights-{self.variant}-v3.8.0",
2762
  files_map=files_map,
2763
  pubkey_32=bytes.fromhex(eff_pubkey),
2764
  salt=eff_salt,
 
258
  def _file_tree_structural_override(row: Mapping[str, Any]) -> Tuple[Optional[str], str]:
259
  """Check physical file-tree extensions (.m4b/.afpk audiobooks, .cbz/.cbr comics, game engine trees, multi-episode packs) before title lookup."""
260
  raw_name = str(row.get("name") or row.get("title") or "")
261
+ files = list(row.get("files") or [])
262
+ if not files:
263
+ ext_hint = str(row.get("extension") or row.get("ext") or "").strip().lstrip(".").lower()
264
+ synth = f"{raw_name}.{ext_hint}" if ext_hint and not raw_name.lower().endswith(f".{ext_hint}") else raw_name
265
+ if synth:
266
+ files = [synth]
267
  ep_files = 0
268
  has_afpk = False
269
  for f in files:
 
1414
  except ImportError:
1415
  from oci_sqlite_catalog import build_torrent_bundle
1416
  bobj = build_torrent_bundle(
1417
+ bundle_name=f"armiger-weights-{self.variant}-v3.10.0",
1418
  files_map=files_map,
1419
  pubkey_32=b"\x00" * 32,
1420
  salt=f"armiger:weights:{self.variant}".encode("utf-8"),
 
2460
  verified_files_map[str(fname)] = fbytes
2461
 
2462
  recomputed = build_torrent_bundle(
2463
+ bundle_name=f"armiger-weights-{self.variant}-v3.10.0",
2464
  files_map=verified_files_map,
2465
  pubkey_32=bytes.fromhex(eff_pubkey),
2466
  salt=eff_salt,
 
2763
  }
2764
 
2765
  recomputed_bundle = build_torrent_bundle(
2766
+ bundle_name=f"armiger-weights-{self.variant}-v3.10.0",
2767
  files_map=files_map,
2768
  pubkey_32=bytes.fromhex(eff_pubkey),
2769
  salt=eff_salt,
assets/armiger_categorical_breakdown.png CHANGED

Git LFS Details

  • SHA256: ca3a401933a90b94746c2ff6b8b0f789238171ef8056c59447814f1cd9562202
  • Pointer size: 131 Bytes
  • Size of remote file: 383 kB

Git LFS Details

  • SHA256: 40199e761e72a8ac1efc9dec1a3868763322cf59505110dd8f7674474c8a0709
  • Pointer size: 131 Bytes
  • Size of remote file: 382 kB
assets/armiger_overview.png CHANGED

Git LFS Details

  • SHA256: 570d00e50ac5765eca6a85c9d81fe7388b62d65780f2ef6fb1d7b39f19b09cdc
  • Pointer size: 131 Bytes
  • Size of remote file: 304 kB

Git LFS Details

  • SHA256: e2369cd1f3eb3f7faf8accd9e789b0718cbb2626a2d58a219bcb1ab400f28235
  • Pointer size: 131 Bytes
  • Size of remote file: 301 kB
assets/armiger_vs_llms_cost_comparison.png CHANGED

Git LFS Details

  • SHA256: 52cfab0941633bb672f1a4dfe5424bd569a5748f054eee76348ed47f98ed95bf
  • Pointer size: 131 Bytes
  • Size of remote file: 320 kB

Git LFS Details

  • SHA256: e0ceec9427b8c45b99d6dba731be94442147404162ef42a1866e0bc50f92e0c7
  • Pointer size: 131 Bytes
  • Size of remote file: 320 kB
assets/armiger_vs_llms_overall_and_6dims.png CHANGED

Git LFS Details

  • SHA256: 4573d788bb0041faae1e56adcb6a0130ce0195c50926c7fba5db418047ba2fb6
  • Pointer size: 131 Bytes
  • Size of remote file: 438 kB

Git LFS Details

  • SHA256: 87927d984f072396ea7891ad39f8084c697fcc513462d0040afb885e972fa500
  • Pointer size: 131 Bytes
  • Size of remote file: 436 kB
assets/armiger_vs_torrent_classifiers_6dims.png CHANGED

Git LFS Details

  • SHA256: 09f25889a1b7ed369509e06464a6022ea1986b0e1cc28ca866d2e79bbf75598f
  • Pointer size: 131 Bytes
  • Size of remote file: 656 kB

Git LFS Details

  • SHA256: 4e4325804114267658676b5586dc57e7580aec6a20d55aab8c77a8d98fc47585
  • Pointer size: 131 Bytes
  • Size of remote file: 653 kB
bundle.torrent CHANGED
Binary files a/bundle.torrent and b/bundle.torrent differ
 
c2417a2db5d38f3a8158cacd2c99aa085721a459.torrent ADDED
Binary file (1.95 kB). View file
 
calibration.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "calibrated_onnx_fp32_sha256": "3125e70b22303f1a548e7b890d2b1c078858eed1b987cc9d63c52188fbc630c0",
3
  "calibration_version": "v3.0-gold-r13_6-val-only",
4
  "feature_stats_dim": 41,
5
  "fitted_snapshot_id": "snap_r13_20261024T140340Z_064c8e6ccb1f",
 
1
  {
2
+ "calibrated_onnx_fp32_sha256": "70286459da47287eb5d984cd67251e0f80f511ceee0ad6ad34c7c2334c9223ec",
3
  "calibration_version": "v3.0-gold-r13_6-val-only",
4
  "feature_stats_dim": 41,
5
  "fitted_snapshot_id": "snap_r13_20261024T140340Z_064c8e6ccb1f",
calibration_and_dht.py CHANGED
@@ -27,7 +27,10 @@ except ImportError: # pragma: no cover
27
  import sre_parse as _sre_parse # type: ignore[no-redef]
28
  import sre_constants as _sre_constants # type: ignore[no-redef]
29
 
30
- from pq_xmss_merkle import QuorumSignatureCache, XMSSSignature, XMSSStatefulVerifier
 
 
 
31
 
32
 
33
  MAX_CUSTOM_REGEX_PATTERN_LEN = 512
@@ -642,6 +645,157 @@ def validate_plugin_overrides_schema(raw_overrides: Any) -> dict[str, dict[str,
642
  return validated
643
 
644
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
645
  class ArmigerCalibrator:
646
  """Applies `calibration.json` temperature scaling, Menon prior adjustments, Mondrian gates,
647
  and schema-validated data-driven plugin overrides (`Gap S3-01`).
@@ -710,7 +864,9 @@ class ArmigerCalibrator:
710
  for k_m, gk in (("adult", "adult"), ("junk", "junk"), ("owner_scope", "owner_scope")):
711
  v_m = m_Legacy.get(k_m)
712
  if isinstance(v_m, dict):
713
- t_v = v_m.get("threshold", v_m.get("threshold_G250", v_m.get("threshold_H350", self.DEFAULT_MONDRIAN[gk])))
 
 
714
  raw_m[gk] = float(t_v)
715
  elif isinstance(v_m, (int, float)):
716
  raw_m[gk] = float(v_m)
@@ -730,6 +886,29 @@ class ArmigerCalibrator:
730
  "owner_scope": scope_t,
731
  }
732
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
733
  raw_overrides = (
734
  active_block.get("plugin_overrides")
735
  if "plugin_overrides" in active_block
@@ -776,11 +955,24 @@ class ArmigerCalibrator:
776
  out[head_name] = rounded
777
  return out
778
 
 
 
 
 
 
 
 
 
 
 
 
779
  def evaluate_safety_gates(
780
  self,
781
  p_adult: float,
782
  p_junk: float,
783
  p_excluded_scope: float,
 
 
784
  ) -> dict[str, bool]:
785
  def _flag(val: float, thresh: float) -> bool:
786
  if isinstance(val, bool) or not isinstance(val, (int, float)) or not math.isfinite(val) or val < 0.0 or val > 1.0:
@@ -788,9 +980,9 @@ class ArmigerCalibrator:
788
  return bool(val >= thresh)
789
 
790
  return {
791
- "adult_flagged": _flag(p_adult, float(self.mondrian["adult"])),
792
- "junk_flagged": _flag(p_junk, float(self.mondrian["junk"])),
793
- "scope_excluded": _flag(p_excluded_scope, float(self.mondrian["owner_scope"])),
794
  }
795
 
796
 
@@ -844,6 +1036,8 @@ class ArmigerDHTUpdater:
844
  raise ValueError(f"Unsupported DHT update mode: {mode}")
845
  if not isinstance(require_pq, bool):
846
  raise TypeError(f"require_pq must be bool, got {type(require_pq).__name__}")
 
 
847
  self.require_pq = require_pq
848
  self._min_quorum_is_bool = isinstance(min_quorum, bool)
849
  self._seq_is_bool = isinstance(current_seq, bool)
@@ -878,7 +1072,10 @@ class ArmigerDHTUpdater:
878
  @staticmethod
879
  @functools.lru_cache(maxsize=512)
880
  def _verify_wots_cached(pq_pub_bytes: bytes, payload: bytes, pq_sig_bytes: bytes) -> bool:
881
- from csam_guard import WOTSPlusSHAKE256
 
 
 
882
  return bool(WOTSPlusSHAKE256.verify(pq_pub_bytes, payload, pq_sig_bytes))
883
 
884
  def verify_bundle(
@@ -906,10 +1103,9 @@ class ArmigerDHTUpdater:
906
  return False, f"stale_or_replayed_seq:{manifest.seq}<={self.current_seq}"
907
  if manifest.seq > (2**63 - 1) or manifest.seq > self.current_seq + 1024:
908
  return False, f"seq_jump_exceeds_max_window:{manifest.seq}"
909
- if self._xmss_verifier is None:
910
- now_ts = int(time.time())
911
- if (manifest.timestamp - now_ts) > 300 or (now_ts - manifest.timestamp) > 14 * 86400:
912
- return False, f"manifest_timestamp_out_of_bounds:{manifest.timestamp}"
913
  if self.min_quorum <= 0 or manifest.quorum_attestations < self.min_quorum:
914
  return False, f"insufficient_quorum:{manifest.quorum_attestations}<{self.min_quorum}"
915
  if (
@@ -956,6 +1152,7 @@ class ArmigerDHTUpdater:
956
  xmss_ok, xmss_reason = self._xmss_verifier.verify_and_advance(
957
  message=payload,
958
  xmss_sig=xmss_sig_obj,
 
959
  )
960
  except (ValueError, TypeError, KeyError) as exc:
961
  return False, f"xmss_signature_parse_error:{exc}"
@@ -985,14 +1182,25 @@ class ArmigerDHTUpdater:
985
  if not pq_ok:
986
  return False, "pq_wots_signature_invalid"
987
 
988
- if bool(commit_seq):
989
  self.commit_bundle(manifest)
990
  return True, "verified"
991
 
992
  def commit_bundle(self, manifest: SignedDHTManifest) -> int:
993
- """Advance `self.current_seq` after a verified bundle has been staged and accepted."""
994
  if isinstance(manifest.seq, bool) or not isinstance(manifest.seq, int) or manifest.seq <= self.current_seq:
995
  raise ValueError(f"invalid_or_non_monotonic_commit_seq:{manifest.seq}<={self.current_seq}")
 
 
 
 
 
 
 
 
 
 
 
996
  self.current_seq = int(manifest.seq)
997
  return self.current_seq
998
 
@@ -1004,25 +1212,26 @@ class ArmigerDHTUpdater:
1004
  canary_fn: Any | None = None,
1005
  rollback_fn: Any | None = None,
1006
  ) -> tuple[bool, str]:
1007
- """Verify bundle, run `apply_fn` and optional `canary_fn`, and commit `current_seq` only on success."""
 
 
 
 
1008
  ok, reason = self.verify_bundle(manifest, bundle_bytes, commit_seq=False)
1009
  if not ok:
1010
  return False, reason
1011
  try:
1012
  apply_res = apply_fn(bundle_bytes)
1013
  if apply_res is False:
1014
- if rollback_fn is not None:
1015
- rollback_fn()
1016
  return False, "bundle_application_rejected"
1017
  if canary_fn is not None:
1018
  canary_ok = canary_fn()
1019
  if not canary_ok:
1020
- if rollback_fn is not None:
1021
- rollback_fn()
1022
  return False, "canary_regression_rejected"
1023
  except Exception as exc:
1024
- if rollback_fn is not None:
1025
- rollback_fn()
1026
  return False, f"bundle_application_failed:{exc}"
1027
  self.commit_bundle(manifest)
1028
  return True, "verified_applied_and_committed"
 
27
  import sre_parse as _sre_parse # type: ignore[no-redef]
28
  import sre_constants as _sre_constants # type: ignore[no-redef]
29
 
30
+ try:
31
+ from armiger.pq_xmss_merkle import QuorumSignatureCache, XMSSSignature, XMSSStatefulVerifier
32
+ except ImportError:
33
+ from pq_xmss_merkle import QuorumSignatureCache, XMSSSignature, XMSSStatefulVerifier # type: ignore[no-redef]
34
 
35
 
36
  MAX_CUSTOM_REGEX_PATTERN_LEN = 512
 
645
  return validated
646
 
647
 
648
+ # --- Horvitz-Thompson inverse-stratum-weighted temperature scaling & split-conformal Mondrian quantiles ---
649
+
650
+ def horvitz_thompson_stratum_weights(classes: list[Any], strata: list[Any]) -> list[float]:
651
+ """Return design weights `w_i = 1 / n_{c,k}` for row `i` in class `c` and stratum `k`.
652
+
653
+ Every `(class, stratum)` cell then carries total mass 1, so a cell of 332 identical
654
+ 100%-confident LibriVox clones counts exactly as much as a cell of 38 course rows.
655
+ """
656
+ if len(classes) != len(strata):
657
+ raise ValueError("classes and strata must have the same length")
658
+ counts: dict[tuple[Any, Any], int] = {}
659
+ for c, k in zip(classes, strata):
660
+ counts[(c, k)] = counts.get((c, k), 0) + 1
661
+ return [1.0 / counts[(c, k)] for c, k in zip(classes, strata)]
662
+
663
+
664
+ def weighted_nll(probs: list[list[float]], y: list[int], weights: list[float]) -> float:
665
+ """Weighted mean negative log-likelihood of the true class."""
666
+ wsum = sum(weights)
667
+ if wsum <= 0.0:
668
+ raise ValueError("weights must have positive total mass")
669
+ return sum(-w * math.log(max(p[t], 1e-12)) for p, t, w in zip(probs, y, weights)) / wsum
670
+
671
+
672
+ def fit_weighted_temperature(
673
+ logits: list[list[float]],
674
+ y: list[int],
675
+ weights: list[float] | None = None,
676
+ *,
677
+ t_min: float = 0.5,
678
+ t_max: float = 2.5,
679
+ grid_steps: int = 81,
680
+ guard: bool = True,
681
+ ) -> dict[str, Any]:
682
+ """Grid-search the temperature minimising (weighted) NLL; with `guard=True` ship `T=1.0`
683
+ unless the best candidate beats it (same guard as the R5clean/R6 trainers).
684
+ """
685
+ n = len(logits)
686
+ if n == 0 or n != len(y):
687
+ raise ValueError("logits and y must be non-empty and aligned")
688
+ w = list(weights) if weights is not None else [1.0] * n
689
+ nll_at_1 = weighted_nll([_softmax(z) for z in logits], y, w)
690
+ best_t, best_nll = 1.0, nll_at_1
691
+ for i in range(grid_steps):
692
+ t_cand = round(t_min + (t_max - t_min) * i / max(1, grid_steps - 1), 4)
693
+ nll_c = weighted_nll([_softmax(z, temperature=t_cand) for z in logits], y, w)
694
+ if nll_c < best_nll - 1e-6:
695
+ best_nll, best_t = nll_c, t_cand
696
+ shipped = best_t if (not guard or best_nll < nll_at_1 - 1e-6) else 1.0
697
+ return {
698
+ "temperature": shipped,
699
+ "best_T_unclamped": best_t,
700
+ "weighted_nll_at_T1": round(nll_at_1, 6),
701
+ "weighted_nll_at_best_T": round(best_nll, 6),
702
+ "n": n,
703
+ "weighting": "uniform" if weights is None else "horvitz_thompson_inverse_stratum",
704
+ }
705
+
706
+
707
+ def weighted_conformal_quantile(scores: list[float], weights: list[float] | None, alpha: float) -> float:
708
+ """Weighted split-conformal `(1 - alpha)` quantile (Tibshirani et al. 2019).
709
+
710
+ The calibration scores get normalised weights `w_i / (sum_j w_j + w_test)` plus a point mass
711
+ at `+inf` for the test row (`w_test = max(w)`, conservative). With uniform weights this is
712
+ exactly `s_(ceil((n + 1)(1 - alpha)))`, the unweighted rule the trainers already use.
713
+ Returns `1.0` (never fires) if the `+inf` mass is needed to reach `1 - alpha`.
714
+ """
715
+ n = len(scores)
716
+ if n == 0:
717
+ raise ValueError("scores cannot be empty")
718
+ if not (0.0 < alpha < 1.0):
719
+ raise ValueError("alpha must be in (0, 1)")
720
+ w = list(weights) if weights is not None else [1.0] * n
721
+ if len(w) != n or any(x <= 0.0 for x in w):
722
+ raise ValueError("weights must be positive and aligned with scores")
723
+ w_test = max(w)
724
+ total = sum(w) + w_test
725
+ order = sorted(range(n), key=lambda i: scores[i])
726
+ cum = 0.0
727
+ for i in order:
728
+ cum += w[i] / total
729
+ if cum >= (1.0 - alpha) - 1e-12:
730
+ return float(scores[i])
731
+ return 1.0
732
+
733
+
734
+ def fit_stratum_weighted_mondrian_gate(
735
+ p_positive: list[float],
736
+ y: list[int],
737
+ strata: list[Any],
738
+ alpha: float,
739
+ *,
740
+ shrinkage_m: float = 0.0,
741
+ ) -> dict[str, Any]:
742
+ """Fit the `scope_excluded`-style one-sided gate `p_positive >= tau` with FPR control on negatives.
743
+
744
+ `tau_global` is the Horvitz-Thompson-weighted conformal quantile over all negatives (`y == 0`);
745
+ `per_stratum[k]` is the Mondrian (stratum-conditional) quantile over the negatives of stratum `k`.
746
+ Strata with no negatives (every row positive) inherit `tau_global` and are listed in
747
+ `strata_without_negatives` — no FPR constraint exists there, so nothing lower can be certified.
748
+ """
749
+ if not (len(p_positive) == len(y) == len(strata)) or not p_positive:
750
+ raise ValueError("p_positive, y and strata must be non-empty and aligned")
751
+ w = horvitz_thompson_stratum_weights(y, strata)
752
+ neg_idx = [i for i, t in enumerate(y) if t == 0]
753
+ if not neg_idx:
754
+ raise ValueError("no negatives (y == 0) to fit an FPR-controlled gate on")
755
+ tau_global = weighted_conformal_quantile([p_positive[i] for i in neg_idx], [w[i] for i in neg_idx], alpha)
756
+ per_stratum: dict[str, float] = {}
757
+ audit: dict[str, dict[str, Any]] = {}
758
+ no_neg: list[str] = []
759
+ for k in sorted({str(s) for s in strata}):
760
+ idx = [i for i, s in enumerate(strata) if str(s) == k]
761
+ k_neg = [i for i in idx if y[i] == 0]
762
+ k_pos = [i for i in idx if y[i] == 1]
763
+ if k_neg:
764
+ raw_tau_k = weighted_conformal_quantile([p_positive[i] for i in k_neg], None, alpha)
765
+ if shrinkage_m > 0.0:
766
+ lam = len(k_neg) / (len(k_neg) + float(shrinkage_m))
767
+ tau_k = lam * raw_tau_k + (1.0 - lam) * tau_global
768
+ else:
769
+ tau_k = raw_tau_k
770
+ else:
771
+ tau_k = tau_global
772
+ no_neg.append(k)
773
+ per_stratum[k] = round(tau_k, 6)
774
+ audit[k] = {
775
+ "n_neg": len(k_neg),
776
+ "n_pos": len(k_pos),
777
+ "tau": round(tau_k, 6),
778
+ "val_fpr_at_tau": round(sum(1 for i in k_neg if p_positive[i] >= tau_k) / len(k_neg), 6) if k_neg else None,
779
+ "val_recall_at_tau": round(sum(1 for i in k_pos if p_positive[i] >= tau_k) / len(k_pos), 6) if k_pos else None,
780
+ }
781
+ n_neg = len(neg_idx)
782
+ return {
783
+ "alpha": alpha,
784
+ "tau_global": round(tau_global, 6),
785
+ "per_stratum": per_stratum,
786
+ "strata_without_negatives": no_neg,
787
+ "n_neg": n_neg,
788
+ "n_pos": len(y) - n_neg,
789
+ "val_fpr_at_tau_global": round(sum(1 for i in neg_idx if p_positive[i] >= tau_global) / n_neg, 6),
790
+ "val_recall_at_tau_global": round(
791
+ sum(1 for i, t in enumerate(y) if t == 1 and p_positive[i] >= tau_global) / max(1, len(y) - n_neg), 6
792
+ ),
793
+ "stratum_audit": audit,
794
+ "method": "Horvitz-Thompson inverse-stratum-weighted (w_i = 1/n_{c,k}) split-conformal quantile on val negatives; "
795
+ "Mondrian per-stratum quantiles; +inf test-point mass (Tibshirani et al. 2019)",
796
+ }
797
+
798
+
799
  class ArmigerCalibrator:
800
  """Applies `calibration.json` temperature scaling, Menon prior adjustments, Mondrian gates,
801
  and schema-validated data-driven plugin overrides (`Gap S3-01`).
 
864
  for k_m, gk in (("adult", "adult"), ("junk", "junk"), ("owner_scope", "owner_scope")):
865
  v_m = m_Legacy.get(k_m)
866
  if isinstance(v_m, dict):
867
+ # accept `threshold`, `threshold_G250`, `threshold_r5clean`, ... (trainers suffix the key per run)
868
+ t_keys = ["threshold"] + sorted(k for k in v_m if str(k).startswith("threshold_"))
869
+ t_v = next((v_m[k] for k in t_keys if k in v_m and isinstance(v_m[k], (int, float))), self.DEFAULT_MONDRIAN[gk])
870
  raw_m[gk] = float(t_v)
871
  elif isinstance(v_m, (int, float)):
872
  raw_m[gk] = float(v_m)
 
886
  "owner_scope": scope_t,
887
  }
888
 
889
+ # Mondrian per-stratum gate thresholds (written by recalibrate_mondrian_stratum_weighted.py):
890
+ # {"owner_scope": {"stratum_key": "category", "strata": {"movie": 0.21, ...}}}
891
+ self.mondrian_strata: dict[str, dict[str, float]] = {}
892
+ self.mondrian_stratum_key: dict[str, str] = {}
893
+ raw_strat = active_block.get("mondrian_stratified_thresholds")
894
+ if raw_strat is None:
895
+ raw_strat = raw_cfg.get("mondrian_stratified_thresholds")
896
+ if isinstance(raw_strat, dict):
897
+ for gate, block in raw_strat.items():
898
+ if gate not in ("adult", "junk", "owner_scope", "scope") or not isinstance(block, dict):
899
+ raise ValueError(f"invalid mondrian_stratified_thresholds block for {gate!r}")
900
+ strata = block.get("strata")
901
+ if not isinstance(strata, dict):
902
+ raise ValueError(f"mondrian_stratified_thresholds[{gate!r}].strata must be an object")
903
+ clean: dict[str, float] = {}
904
+ for k, v in strata.items():
905
+ if isinstance(v, bool) or not isinstance(v, (int, float)) or not math.isfinite(float(v)) or not (0.0 < float(v) < 1.0):
906
+ raise ValueError(f"invalid stratified mondrian threshold {gate}.{k}={v!r}")
907
+ clean[str(k)] = float(v)
908
+ g_key = "owner_scope" if gate == "scope" else gate
909
+ self.mondrian_strata[g_key] = clean
910
+ self.mondrian_stratum_key[g_key] = str(block.get("stratum_key", "category"))
911
+
912
  raw_overrides = (
913
  active_block.get("plugin_overrides")
914
  if "plugin_overrides" in active_block
 
955
  out[head_name] = rounded
956
  return out
957
 
958
+ def gate_threshold(self, gate: str, stratum: str | None = None) -> float:
959
+ """Threshold for `gate` (`adult` / `junk` / `owner_scope`): the Mondrian per-stratum value when
960
+ `stratum` (e.g. the predicted category) is known and fitted, else the global value.
961
+ """
962
+ g_key = "owner_scope" if gate == "scope" else gate
963
+ if stratum is not None:
964
+ per = self.mondrian_strata.get(g_key, {})
965
+ if str(stratum) in per:
966
+ return float(per[str(stratum)])
967
+ return float(self.mondrian[g_key])
968
+
969
  def evaluate_safety_gates(
970
  self,
971
  p_adult: float,
972
  p_junk: float,
973
  p_excluded_scope: float,
974
+ *,
975
+ stratum: str | None = None,
976
  ) -> dict[str, bool]:
977
  def _flag(val: float, thresh: float) -> bool:
978
  if isinstance(val, bool) or not isinstance(val, (int, float)) or not math.isfinite(val) or val < 0.0 or val > 1.0:
 
980
  return bool(val >= thresh)
981
 
982
  return {
983
+ "adult_flagged": _flag(p_adult, self.gate_threshold("adult", stratum)),
984
+ "junk_flagged": _flag(p_junk, self.gate_threshold("junk", stratum)),
985
+ "scope_excluded": _flag(p_excluded_scope, self.gate_threshold("owner_scope", stratum)),
986
  }
987
 
988
 
 
1036
  raise ValueError(f"Unsupported DHT update mode: {mode}")
1037
  if not isinstance(require_pq, bool):
1038
  raise TypeError(f"require_pq must be bool, got {type(require_pq).__name__}")
1039
+ if not require_pq:
1040
+ raise ValueError("require_pq=False is forbidden; post-quantum verification is mandatory")
1041
  self.require_pq = require_pq
1042
  self._min_quorum_is_bool = isinstance(min_quorum, bool)
1043
  self._seq_is_bool = isinstance(current_seq, bool)
 
1072
  @staticmethod
1073
  @functools.lru_cache(maxsize=512)
1074
  def _verify_wots_cached(pq_pub_bytes: bytes, payload: bytes, pq_sig_bytes: bytes) -> bool:
1075
+ try:
1076
+ from armiger.csam_guard import WOTSPlusSHAKE256
1077
+ except ImportError:
1078
+ from csam_guard import WOTSPlusSHAKE256
1079
  return bool(WOTSPlusSHAKE256.verify(pq_pub_bytes, payload, pq_sig_bytes))
1080
 
1081
  def verify_bundle(
 
1103
  return False, f"stale_or_replayed_seq:{manifest.seq}<={self.current_seq}"
1104
  if manifest.seq > (2**63 - 1) or manifest.seq > self.current_seq + 1024:
1105
  return False, f"seq_jump_exceeds_max_window:{manifest.seq}"
1106
+ now_ts = int(time.time())
1107
+ if (manifest.timestamp - now_ts) > 300 or (now_ts - manifest.timestamp) > 14 * 86400:
1108
+ return False, f"manifest_timestamp_out_of_bounds:{manifest.timestamp}"
 
1109
  if self.min_quorum <= 0 or manifest.quorum_attestations < self.min_quorum:
1110
  return False, f"insufficient_quorum:{manifest.quorum_attestations}<{self.min_quorum}"
1111
  if (
 
1152
  xmss_ok, xmss_reason = self._xmss_verifier.verify_and_advance(
1153
  message=payload,
1154
  xmss_sig=xmss_sig_obj,
1155
+ advance=(commit_seq is None or bool(commit_seq)),
1156
  )
1157
  except (ValueError, TypeError, KeyError) as exc:
1158
  return False, f"xmss_signature_parse_error:{exc}"
 
1182
  if not pq_ok:
1183
  return False, "pq_wots_signature_invalid"
1184
 
1185
+ if commit_seq is None or bool(commit_seq):
1186
  self.commit_bundle(manifest)
1187
  return True, "verified"
1188
 
1189
  def commit_bundle(self, manifest: SignedDHTManifest) -> int:
1190
+ """Advance both XMSS verifier state and `self.current_seq` after a verified bundle has been staged and accepted."""
1191
  if isinstance(manifest.seq, bool) or not isinstance(manifest.seq, int) or manifest.seq <= self.current_seq:
1192
  raise ValueError(f"invalid_or_non_monotonic_commit_seq:{manifest.seq}<={self.current_seq}")
1193
+ if self._xmss_verifier is not None and manifest.xmss_signature_hex:
1194
+ xmss_sig_bytes = bytes.fromhex(manifest.xmss_signature_hex)
1195
+ xmss_sig_obj = XMSSSignature.from_bytes(xmss_sig_bytes)
1196
+ if self._xmss_verifier.current_seq < int(manifest.seq):
1197
+ xmss_ok, xmss_reason = self._xmss_verifier.verify_and_advance(
1198
+ message=manifest.canonical_payload(),
1199
+ xmss_sig=xmss_sig_obj,
1200
+ advance=True,
1201
+ )
1202
+ if not xmss_ok:
1203
+ raise ValueError(f"xmss_commit_verification_failed:{xmss_reason}")
1204
  self.current_seq = int(manifest.seq)
1205
  return self.current_seq
1206
 
 
1212
  canary_fn: Any | None = None,
1213
  rollback_fn: Any | None = None,
1214
  ) -> tuple[bool, str]:
1215
+ """Verify bundle without advancing state, run `apply_fn` and optional `canary_fn`, and commit only on success."""
1216
+ if not callable(apply_fn):
1217
+ return False, "apply_fn_required"
1218
+ if not callable(rollback_fn):
1219
+ return False, "rollback_fn_required_for_safe_apply"
1220
  ok, reason = self.verify_bundle(manifest, bundle_bytes, commit_seq=False)
1221
  if not ok:
1222
  return False, reason
1223
  try:
1224
  apply_res = apply_fn(bundle_bytes)
1225
  if apply_res is False:
1226
+ rollback_fn()
 
1227
  return False, "bundle_application_rejected"
1228
  if canary_fn is not None:
1229
  canary_ok = canary_fn()
1230
  if not canary_ok:
1231
+ rollback_fn()
 
1232
  return False, "canary_regression_rejected"
1233
  except Exception as exc:
1234
+ rollback_fn()
 
1235
  return False, f"bundle_application_failed:{exc}"
1236
  self.commit_bundle(manifest)
1237
  return True, "verified_applied_and_committed"
config.json CHANGED
@@ -1,18 +1,18 @@
1
  {
2
  "model_name": "armiger-ultra",
3
- "version": "3.8.0",
4
  "bundle_type": "unified_armiger_plus_ultra_plus_swarm",
5
  "architecture": "armiger-v2-byte-rope-transformer-8head + residual-swarm-health-adapter",
6
  "artifacts": {
7
  "armiger_cpu_int8_onnx": {
8
  "file": "armiger.int8.onnx",
9
- "size_bytes": 17606389,
10
- "sha256": "7ee3dfdaae4283faecabf06bc7d7644d1747d959264506b07065a3035fe6ffd0"
11
  },
12
  "armiger_ultra_fp32_onnx": {
13
  "file": "armiger_ultra.fp32.onnx",
14
  "size_bytes": 33260312,
15
- "sha256": "3125e70b22303f1a548e7b890d2b1c078858eed1b987cc9d63c52188fbc630c0"
16
  },
17
  "armiger_ultra_pytorch": {
18
  "file": "armiger_ultra.pt",
@@ -139,7 +139,7 @@
139
  "test_armiger_dht_rsi.py"
140
  ],
141
  "sha256": {
142
- "calibration.json": "ac2fe3b3de8cda9d3e7bca3b334157ac0d90e052e2ff03c3787900b991b13b34",
143
  "calibration_and_dht.py": "11be587b60202185abbcf507b935464fed5be5d7c0b28adc38fe5213ca088db5",
144
  "armiger_dht_rsi.py": "3b709c506e0826fded14d3180516982f91b003f4ad3381a9b13e3909900e11e4",
145
  "test_armiger_dht_rsi.py": "f284d290cc2a847f152e51ff40112556caf6c256ee2629aa9f31afbefa73d2df"
@@ -176,7 +176,7 @@
176
  "csam_safety"
177
  ],
178
  "sha256": {
179
- "armiger_ultra_pipeline.py": "7d6c9588a0ac047b68bd7a296160b77f37f17ffd34049c25ab7d8e75529399bc",
180
  "test_armiger_ultra_pipeline.py": "b309d0edf87a189f41e95c18511fd26b0d748f981d93f1a0782cd8163015e18d"
181
  }
182
  },
 
1
  {
2
  "model_name": "armiger-ultra",
3
+ "version": "3.10.0",
4
  "bundle_type": "unified_armiger_plus_ultra_plus_swarm",
5
  "architecture": "armiger-v2-byte-rope-transformer-8head + residual-swarm-health-adapter",
6
  "artifacts": {
7
  "armiger_cpu_int8_onnx": {
8
  "file": "armiger.int8.onnx",
9
+ "size_bytes": 17606362,
10
+ "sha256": "c39bad25f5e31289d575243473589e023c63327b5d846e26aaeebf9aee7b054c"
11
  },
12
  "armiger_ultra_fp32_onnx": {
13
  "file": "armiger_ultra.fp32.onnx",
14
  "size_bytes": 33260312,
15
+ "sha256": "70286459da47287eb5d984cd67251e0f80f511ceee0ad6ad34c7c2334c9223ec"
16
  },
17
  "armiger_ultra_pytorch": {
18
  "file": "armiger_ultra.pt",
 
139
  "test_armiger_dht_rsi.py"
140
  ],
141
  "sha256": {
142
+ "calibration.json": "92aa6aac07a9868f7a54fc0089eec6b06aa86c759723dd0731e0afd13e617cdd",
143
  "calibration_and_dht.py": "11be587b60202185abbcf507b935464fed5be5d7c0b28adc38fe5213ca088db5",
144
  "armiger_dht_rsi.py": "3b709c506e0826fded14d3180516982f91b003f4ad3381a9b13e3909900e11e4",
145
  "test_armiger_dht_rsi.py": "f284d290cc2a847f152e51ff40112556caf6c256ee2629aa9f31afbefa73d2df"
 
176
  "csam_safety"
177
  ],
178
  "sha256": {
179
+ "armiger_ultra_pipeline.py": "018de352f0984ad2a7eeacbdd31a8188e11bd2d3c349ccc1740cfd1a92cb3916",
180
  "test_armiger_ultra_pipeline.py": "b309d0edf87a189f41e95c18511fd26b0d748f981d93f1a0782cd8163015e18d"
181
  }
182
  },
e7067600b7423a9f112bd3d05d99f54890278ff8.torrent ADDED
Binary file (12.1 kB). View file
 
armiger-zero-cue-db-v3.8.0.torrent → ef2d34cd25969f5fa4f79e48c1ea26ff67b5543e.torrent RENAMED
Binary files a/armiger-zero-cue-db-v3.8.0.torrent and b/ef2d34cd25969f5fa4f79e48c1ea26ff67b5543e.torrent differ
 
fd18286b9b164f055878c3e63c6c442182745ff2.torrent DELETED
Binary file (12.1 kB)
 
zero_cue_plugin.py CHANGED
@@ -78,6 +78,8 @@ _CONSOLE_TITLE_ID = (
78
  )
79
  CONSOLE_TITLE_ID_RE = re.compile("(?i)" + _CONSOLE_TITLE_ID)
80
 
 
 
81
  GAME_ARCHIVE_RE = re.compile(
82
  r"(?i)(?:\bNTSC-U\b|\bNTSC-J\b|\[linux-x86\]|\[linux-x64\]|\[win64\]|"
83
  r"\bInteractive\s+Fiction\b|\bGOG\s+Edition\b|\bLinux\s+Native\b|\((?:Europe|USA|Japan|World)\)\.(?:zip|7z|rar|iso|chd)\b|"
 
78
  )
79
  CONSOLE_TITLE_ID_RE = re.compile("(?i)" + _CONSOLE_TITLE_ID)
80
 
81
+ # Gazetteer entries that matched snapshot_r10 val/test names with no train support were removed in R12
82
+ # (tests/test_r12_plugin_hygiene_and_invariance.py lists them).
83
  GAME_ARCHIVE_RE = re.compile(
84
  r"(?i)(?:\bNTSC-U\b|\bNTSC-J\b|\[linux-x86\]|\[linux-x64\]|\[win64\]|"
85
  r"\bInteractive\s+Fiction\b|\bGOG\s+Edition\b|\bLinux\s+Native\b|\((?:Europe|USA|Japan|World)\)\.(?:zip|7z|rar|iso|chd)\b|"