simonmorley commited on
Commit
4a9a4d9
·
verified ·
1 Parent(s): 3f607e2

Multi-class softmax folded detector — initial release (V8-V14 + V16, 2026-05-13)

Browse files
Files changed (6) hide show
  1. LICENSE +191 -0
  2. README.md +236 -0
  3. inference_example.py +164 -0
  4. model.joblib +3 -0
  5. predict.py +232 -0
  6. release-cert.json +190 -0
LICENSE ADDED
@@ -0,0 +1,191 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of tracking or improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for describing the origin of the Work and
141
+ reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Support. While redistributing the Work or
166
+ Derivative Works thereof, You may accept upon your own behalf the
167
+ responsibility to provide, and accept charging a fee for, accepting
168
+ warranty, support, indemnity, or other liability obligations and/or
169
+ rights consistent with this License. However, in accepting such
170
+ obligations, You may act only on Your own behalf and on Your sole
171
+ responsibility, not on behalf of any other Contributor, and only
172
+ if You agree to indemnify, defend, and hold each Contributor
173
+ harmless for any liability incurred by, or claims asserted against,
174
+ such Contributor by reason of your accepting any such warranty
175
+ or support.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ Copyright 2026 NullRabbit Labs Ltd
180
+
181
+ Licensed under the Apache License, Version 2.0 (the "License");
182
+ you may not use this file except in compliance with the License.
183
+ You may obtain a copy of the License at
184
+
185
+ http://www.apache.org/licenses/LICENSE-2.0
186
+
187
+ Unless required by applicable law or agreed to in writing, software
188
+ distributed under the License is distributed on an "AS IS" BASIS,
189
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
190
+ implied. See the License for the specific language governing
191
+ permissions and limitations under the License.
README.md ADDED
@@ -0,0 +1,236 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - cybersecurity
7
+ - blockchain
8
+ - network-security
9
+ - validator-security
10
+ - anomaly-detection
11
+ - multi-class-classification
12
+ - byte-amplification
13
+ - gossip-abuse
14
+ - sui
15
+ - solana
16
+ - scikit-learn
17
+ library_name: scikit-learn
18
+ pretty_name: Multi-class softmax folded detector (V8-V14 + V16, Sui + Solana)
19
+ ---
20
+
21
+ # Multi-class softmax folded detector — V8-V14 + V16, Sui + Solana
22
+
23
+ ## What it is
24
+
25
+ The multi-class folded detector is a single joint classifier trained against the union corpus of NullRabbit's V8-V14 binary detectors plus the new V16 gossip-abuse class, with Solana primitives folded into the existing Sui-side class taxonomy. It is one demonstrable outcome of NullRabbit's pre-registration discipline applied at the unified-detector layer; **the methodology is the contribution.**
26
+
27
+ This is the work of the substrate paper (in preparation): an iterative leak-surface peeling pattern applied across multiple training cycles. Each binary detector cycle (V8 → V9 → V10 → V11 → V12 → V13 → V14 → V15 → V16) is pre-registered, audited on close, and retracted in writing when a leak fires. The folded multi-class architecture absorbs each binary class as a softmax column; the v2 retrain documented here adds V16 (gossip-abuse) after V15's caveat #2 was empirically confirmed and closed by corpus augmentation. The model demonstrates **autonomous defence for decentralised networks** at the unified-detector layer.
28
+
29
+ The model itself is `CalibratedClassifierCV(HistGradientBoostingClassifier, method="isotonic", cv=5)` over 107 features spanning the bundle v1 modalities (pcap.*, host.*, app.*, protocol.*, responses.*). Output: per-class calibrated probabilities across 9 classes (benign + V8/V9/V10/V11/V12/V13/V14/V16). Single-bundle scoring; not a packet-level streaming detector.
30
+
31
+ ## Architecture
32
+
33
+ - Estimator: `CalibratedClassifierCV(HistGradientBoostingClassifier(max_iter=300, learning_rate=0.05, max_depth=8), method='isotonic', cv=5)`.
34
+ - Output: 9-class softmax with isotonic calibration per class.
35
+ - Features: 107 total, drawn from the full bundle v1 modality surface.
36
+ - Solana mode: `folded` — Solana primitives map into Sui-side class manifolds based on the family taxonomy (e.g. SOL_F10 → V8, SOL_F14 → V14, SOL_G01-G08 → V16). This is the **cross-chain claim**: the family taxonomy abstracts at the attack-mechanism layer, so the same class column captures the mechanism regardless of the chain that hosts it.
37
+ - Training corpus: 2549 bundles across 38 primitives, fidelity_class=lab.
38
+ - Seed: 42.
39
+
40
+ ## Class taxonomy
41
+
42
+ | Class | Family | Sui primitives | Solana primitives | Total in training |
43
+ |---|---|---|---|---:|
44
+ | benign | benign | sui_BENIGN_passive_fullnode, sui_BENIGN_reproducer_pipeline | solana_BENIGN_organic_rpc, solana_BENIGN_validator_passive, SOL_BG01_validator_repair_catchup | 1076 |
45
+ | V8 | response_amp | sui_F10 | SOL_F10 | 105 |
46
+ | V9 | reconnaissance | (none in this cache) | SOL_RC_nmap_slow | 50 |
47
+ | V10 | auth_bypass | sui_GR01, sui_GR02, sui_H01 | SOL_H01 | 204 |
48
+ | V11 | rate_limiter_bypass / app-DoS | sui_P01, sui_P05, sui_P06, sui_P07 | SOL_P07, SOL_GR01 | 324 |
49
+ | V12 | consensus_abuse / memory_amp | sui_D01, sui_D02, sui_D03 | (none) | 150 |
50
+ | V13 | service_misconfig | sui_MC × 5 | SOL_MC × 5 | 500 |
51
+ | V14 | compute_amp | sui_F14 | SOL_F14 | 105 |
52
+ | **V16** | **gossip_abuse** | (none — pcap-only Sui-side cycle banked) | **SOL_G01, SOL_G04, SOL_G05, SOL_G07, SOL_G08** | **35** |
53
+
54
+ V16 is the smallest class at n=35; all other classes have n≥50. The Solana-only construction of V16 reflects cycle2 corpus state at training time (2026-05-13). Cross-chain transfer evidence for V16 requires a Sui-side gossip-pcap corpus — banked, not validated.
55
+
56
+ ## Training data
57
+
58
+ The training corpus is **proprietary**. NullRabbit's archived `corpus_v1.0`–`corpus_v1.10` plus the cycle2 working corpus (V15 / V16 gossip-abuse) compose the union from which 2549 lab-fidelity bundles were drawn for v2 training.
59
+
60
+ A curated, public sample of the corpus is available on Hugging Face as **[NullRabbit/nr-bundles-public](https://huggingface.co/datasets/NullRabbit/nr-bundles-public)** — 31 bundles spanning seven vulnerability families across Sui and Solana, CC-BY-4.0. The bundle format is open and specified at **[`nr-bundle-spec`](https://github.com/NullRabbitLabs/nr-bundle-spec)** (MIT). External researchers building their own corpus against the spec can reproduce the methodology, retrain the multi-class softmax architecture on their own data, and compare against this reference model.
61
+
62
+ The V8 binary detector that anchored the byte-amplification class is published separately at **[NullRabbit/v8-cipher-agnostic](https://huggingface.co/NullRabbit/v8-cipher-agnostic)** (Apache-2.0). The multi-class folded model unifies V8's behaviour with the other family detectors into a single inference path.
63
+
64
+ ## Intended use
65
+
66
+ - **Reference multi-class detector for the V8-V14 + V16 attack-family taxonomy** on validator-infrastructure observations. Single bundle in → 9-class softmax out → argmax verdict + calibrated per-class probabilities.
67
+ - **Cross-chain generalisation evidence**: the folded mapping demonstrates that Sui-trained and Solana-trained class manifolds for the same family converge under joint training. Cross-chain transfer is validated for V8 (response_amp), V11 (rate_limiter_bypass), V13 (service_misconfig), V14 (compute_amp) — these have both Sui and Solana primitives in training and achieve per-class recall ≥0.997.
68
+ - **Methodology demonstration**: the v2 retrain consumes the V16 binary detector's outcome and absorbs the new family into the unified detector. Pre-registration → train → audit → outcome documentation is the substrate-paper-meaningful methodology, applied at the multi-class layer.
69
+
70
+ ## Load-bearing limitations
71
+
72
+ This section is the most important part of the card. Each limitation is anchored in pre-registered evidence and surfaced because it would otherwise become a deployment-time surprise.
73
+
74
+ ### n=1 OOF fragility on the V16 load-bearing benign
75
+
76
+ SOL_BG01_validator_repair_catchup is the single ground-truth UDP gossip benign in the training corpus. The fitted v2 model routes SOL_BG01 to `benign` with P(benign)=0.97. **In the OOF fold where SOL_BG01 is held out**, the model has zero UDP gossip benign signal and routes BG01 to V16 — producing the single benign-as-V16 misclassification in the OOF confusion matrix. This is **expected n=1 fragility** pre-registered in `docs/MULTICLASS-FOLDED-V2-DESIGN.md` Section "Honest caveats banked in advance #1". Production deployment for V16 + BG01-class routing requires corpus scale-up (n≥10 UDP benigns across postures: idle multi-validator cluster, repair-traffic, snapshot-catchup, vote-msg propagation) before the routing claim is defensible.
77
+
78
+ ### V16 cross-primitive generalisation is in-sample
79
+
80
+ All 35 V16 attack bundles (SOL_G01, SOL_G04, SOL_G05, SOL_G07, SOL_G08) are in training. Per-primitive recall of 35/35 at the in-sample level is fit, not held-out generalisation. Transfer to gossip-abuse primitives outside the cycle2 corpus is not validated by v2.
81
+
82
+ ### V16 has no Sui-side training data
83
+
84
+ V16's training distribution is Solana-only. The model **cannot** demonstrate cross-chain transfer for the gossip-abuse family from this corpus. Forward TODO: Sui-side gossip-pcap corpus + V16 cross-chain transfer evaluation.
85
+
86
+ ### V9 (reconnaissance) is Solana-only in this cache
87
+
88
+ The 2026-05-11 baseline cache included `sui_RC_masscan_distributed` + `sui_RC_nmap_slow`; the v2-unified cache (flat-dir construction from step11-cache + spaces-extract + cycle2) doesn't include them. V9 in v2 has 50 bundles (SOL_RC_nmap_slow only). Per-class recall at 1.00 is in-sample; cross-chain V9 transfer is not validated. Forward TODO for v3.
89
+
90
+ ### Per-class metric variance is asymmetric
91
+
92
+ V16 (n=35) is ~30× smaller than V13 (n=500). Stratified folding handles the imbalance but the per-class metric noise floor differs across classes — V16 metrics will have higher variance than V13 metrics. Single-fold per-class recall numbers for V16 should be interpreted with this in mind.
93
+
94
+ ### Bundle modality requirements vary by class
95
+
96
+ The model uses 107 features spanning all bundle v1 modalities and was trained on bundles with full modality coverage (responses, host, app, protocol, packets.pcap all populated). At inference time, bundles with missing modalities are **out-of-distribution by construction** and predictions degrade in a class-dependent way.
97
+
98
+ Class manifold sensitivity to missing `packets.pcap` (the most common public-release mode where pcap is dropped for safety):
99
+
100
+ - **V8 (response_amp), V13 (service_misconfig), V14 (compute_amp)**: largely robust to pcap-drop. These classes have discriminative wire-shape features in `responses.parquet` + `host.parquet` that survive without pcap.
101
+ - **V11 (rate_limiter_bypass) and benign-with-traffic**: load-bearing on `pcap.*` rate/cardinality features. When pcap is dropped, the all-zero pcap.* signal looks more V16-like than V11/benign-like to the model, producing misclassifications.
102
+ - **V16 (gossip-abuse)**: load-bearing on `pcap.*` features (the class was trained entirely on pcap-only cycle2 bundles). Without raw pcap, V16 cannot be evaluated honestly — and a V16 prediction with `feature_coverage=resp_only` is almost certainly a missing-modality artefact, not a true gossip-abuse detection.
103
+
104
+ The `predict.py` scoreability gate (recommended consumption surface) refuses to score bundles where neither `responses.parquet` nor `packets.pcap` has content, and emits a `feature_coverage` flag + a `coverage_warning` when the predicted class is sensitive to the missing modality. For reliable V11 / benign-with-traffic / V16 inference, callers must provide bundles with raw `packets.pcap` present. The curated public dataset `nr-bundles-public` is not suitable for those classes.
105
+
106
+ ### Disclosure context
107
+
108
+ The training corpus includes bundles for primitives at varying disclosure states. `SOL_F10_multi_get_accounts_amp`, `SOL_F14_simulate_transaction_sync_wedge`, `SOL_P07_get_program_accounts_filter_miss` are publicly disclosed per [NR-2026-001](https://nullrabbit.ai). Other primitives represent methodology-class findings or are referenced in coordinated-disclosure channels with respective ecosystems. Disclosure-status information travels with the bundles in `nr-bundles-public`; this model card is the inference-layer cross-reference.
109
+
110
+ ## Evaluation
111
+
112
+ 5-fold stratified out-of-fold predictions on the 2549-bundle training corpus:
113
+
114
+ | Class | n | recall | precision | brier | P(class\|true) | P(class\|false) |
115
+ |---|---:|---:|---:|---:|---:|---:|
116
+ | benign | 1076 | 0.999 | 0.999 | 0.0011 | 0.997 | 0.003 |
117
+ | V8 | 105 | 1.000 | 1.000 | 0.0001 | 0.992 | 0.000 |
118
+ | V9 | 50 | 1.000 | 1.000 | 0.0000 | 0.995 | 0.000 |
119
+ | V10 | 204 | 1.000 | 1.000 | 0.0001 | 0.990 | 0.000 |
120
+ | V11 | 324 | 0.997 | 1.000 | 0.0004 | 0.995 | 0.001 |
121
+ | V12 | 150 | 1.000 | 1.000 | 0.0000 | 1.000 | 0.000 |
122
+ | V13 | 500 | 1.000 | 1.000 | 0.0000 | 0.999 | 0.000 |
123
+ | V14 | 105 | 1.000 | 1.000 | 0.0001 | 0.985 | 0.000 |
124
+ | V16 | 35 | 1.000 | 0.972 | 0.0005 | 0.982 | 0.001 |
125
+
126
+ Overall OOF accuracy: **0.9992** (2026-05-11 predecessor: 0.9996; within fold-variance band). Per-class confusion matrix:
127
+
128
+ ```
129
+ benign V8 V9 V10 V11 V12 V13 V14 V16
130
+ benign 1075 0 0 0 0 0 0 0 1
131
+ V8 0 105 0 0 0 0 0 0 0
132
+ V9 0 0 50 0 0 0 0 0 0
133
+ V10 0 0 0 204 0 0 0 0 0
134
+ V11 1 0 0 0 323 0 0 0 0
135
+ V12 0 0 0 0 0 150 0 0 0
136
+ V13 0 0 0 0 0 0 500 0 0
137
+ V14 0 0 0 0 0 0 0 105 0
138
+ V16 0 0 0 0 0 0 0 0 35
139
+ ```
140
+
141
+ The single benign→V16 misclassification is the SOL_BG01 OOF fragility documented above. The single V11→benign cross-class confusion is identical to the 2026-05-11 baseline and reflects an edge case in the rate_limiter_bypass class boundary that has persisted across cycles.
142
+
143
+ ## How to use
144
+
145
+ ### Recommended path: `predict.py` (scoreability-gated)
146
+
147
+ The repository ships with `predict.py` — a thin scoreability-gated inference helper that wraps the raw multi-class estimator with two production-side gates:
148
+
149
+ - **Scoreability gate**: refuses to score bundles where neither `responses.parquet` nor `packets.pcap` has content. Bundles with no observed RPC traffic AND no captured network packets cannot be classified usefully; the gate returns an explicit `verdict: "unscoreable"` instead of producing a spurious argmax.
150
+ - **Feature-coverage flag**: emits `feature_coverage` describing which bundle modalities contributed features (`"resp_only"`, `"pcap_only"`, `"full"`, etc.) so callers can downweight predictions where the modality coverage doesn't match the predicted class (e.g. V16 prediction with `resp_only` coverage is suspect).
151
+
152
+ ```python
153
+ from huggingface_hub import hf_hub_download
154
+ from predict import load_model, score_bundle
155
+
156
+ model_path = hf_hub_download(
157
+ repo_id="NullRabbit/multiclass-folded", filename="model.joblib"
158
+ )
159
+ payload = load_model(model_path)
160
+
161
+ record = score_bundle("/path/to/some/bundle_dir", payload)
162
+ if record["verdict"] == "unscoreable":
163
+ print(f"refused: {record['reason']}")
164
+ else:
165
+ print(f"argmax: {record['argmax_class']} (P={record['argmax_p']:.4f}, "
166
+ f"coverage={record['feature_coverage']})")
167
+ for cls, p in sorted(record["class_probs"].items(),
168
+ key=lambda kv: -kv[1])[:3]:
169
+ print(f" P({cls}) = {p:.4f}")
170
+ ```
171
+
172
+ `predict.py` depends on the bundle-spec reference parser:
173
+
174
+ ```
175
+ pip install git+https://github.com/NullRabbitLabs/nr-bundle-spec.git
176
+ ```
177
+
178
+ For a full worked example that loads bundles from `nr-bundles-public` via the spec parser and demonstrates cross-class scoring (V8 / V11 / V13 / V14 / benign verdicts on the public dataset), see [`inference_example.py`](inference_example.py).
179
+
180
+ ### Bypassing the gate
181
+
182
+ Callers with their own pre-filtering pipeline (or who explicitly want raw model output) can load the estimator directly:
183
+
184
+ ```python
185
+ import joblib
186
+ import numpy as np
187
+
188
+ payload = joblib.load(model_path)
189
+ model = payload["model"] # CalibratedClassifierCV
190
+ features = payload["feature_names"] # 107-feature contract
191
+ class_order = payload["class_order"]
192
+
193
+ X = np.array([[...]]) # shape (n_samples, 107)
194
+ proba = model.predict_proba(X) # shape (n_samples, 9)
195
+ ```
196
+
197
+ **This path is the responsibility of the caller.** The scoreability gate exists to prevent spurious predictions on under-determined inputs. See the Load-bearing limitations section.
198
+
199
+ ## Methodology
200
+
201
+ NullRabbit's training cycles follow pre-registration discipline. Each cycle has a design document committed before the trainer runs. Audits run on close against sanity floors, per-feature ablation trails, and falsification holdouts. Where an audit fires, training halts, the design is re-registered, and the prior version is retracted in writing.
202
+
203
+ The **iterative leak-surface peeling pattern** is the methodology contribution: detection of a training-time leak (a feature whose discriminative signal turns out to come from a labelling artefact or capture-pipeline asymmetry rather than from the attack mechanism) triggers a corpus delta + re-train + re-audit, with each cycle narrowing the leak surface. The v2 retrain is a worked example at the unified-detector layer: V15 (gossip-abuse binary) pre-registered caveat #2 (protocol-shape leak); cycle2 corpus expansion (the other-window sprint) provided the load-bearing UDP benign that made the caveat empirically testable; V15 evaluation confirmed the caveat; V16 binary detector retrained with corpus augmentation closed the caveat at the n=1 fragile level; this v2 multi-class retrain absorbs V16 into the unified detector with the load-bearing benign test passing at training-set scale and the OOF fragility surfaced honestly.
204
+
205
+ The corpus format and family taxonomy are open at `nr-bundle-spec`. The methodology is open (in preparation as the substrate paper). The specific corpus contents beyond `nr-bundles-public` are proprietary.
206
+
207
+ ## Related
208
+
209
+ - **Bundle format spec**: [`nr-bundle-spec`](https://github.com/NullRabbitLabs/nr-bundle-spec) (MIT)
210
+ - **Reference public bundles**: [NullRabbit/nr-bundles-public](https://huggingface.co/datasets/NullRabbit/nr-bundles-public) (CC-BY-4.0)
211
+ - **V8 binary detector** (cipher-agnostic byte-amplification): [NullRabbit/v8-cipher-agnostic](https://huggingface.co/NullRabbit/v8-cipher-agnostic) (Apache-2.0)
212
+ - **Earned-autonomy paper** (governance layer for autonomous defence for decentralised networks): [Zenodo DOI 10.5281/zenodo.18406828](https://doi.org/10.5281/zenodo.18406828)
213
+ - **Substrate paper** (data-layer methodology, in preparation)
214
+ - **NullRabbit Labs**: [huggingface.co/NullRabbit](https://huggingface.co/NullRabbit)
215
+ - **Website**: [nullrabbit.ai](https://nullrabbit.ai)
216
+
217
+ ## Citation
218
+
219
+ ```bibtex
220
+ @misc{nullrabbit_multiclass_folded_2026,
221
+ author = {NullRabbit},
222
+ title = {Multi-class softmax folded detector — V8-V14 + V16, Sui + Solana},
223
+ year = {2026},
224
+ month = may,
225
+ version = {2},
226
+ publisher = {Hugging Face},
227
+ url = {https://huggingface.co/NullRabbit/multiclass-folded},
228
+ note = {Reference 9-class joint detector for the V8-V14 binary detector family plus the V16 gossip-abuse class. Trained on the bundle v1 corpus specified at nr-bundle-spec v0.1.0; curated public sample at NullRabbit/nr-bundles-public; V8 binary anchor at NullRabbit/v8-cipher-agnostic.},
229
+ }
230
+ ```
231
+
232
+ ## Contact
233
+
234
+ Research enquiries: simon@nullrabbit.ai
235
+
236
+ Spec compliance or format questions — open an issue at [`nr-bundle-spec`](https://github.com/NullRabbitLabs/nr-bundle-spec).
inference_example.py ADDED
@@ -0,0 +1,164 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # SPDX-License-Identifier: Apache-2.0
3
+ """Multi-class folded detector — end-to-end inference example.
4
+
5
+ Three-artefact collaboration. This script:
6
+
7
+ 1. Downloads bundles from the public NullRabbit/nr-bundles-public dataset
8
+ on Hugging Face.
9
+ 2. Downloads the multi-class folded model and the scoreability-gated
10
+ inference helper (``predict.py``) from this repository.
11
+ 3. Loads each bundle manifest via the bundle-spec reference parser
12
+ (NullRabbitLabs/nr-bundle-spec, MIT).
13
+ 4. Calls ``predict.score_bundle()`` to apply the scoreability gate and
14
+ produce a 9-class softmax verdict + per-class probabilities.
15
+
16
+ A worked demonstration of the **spec → corpus → model** path at the
17
+ unified-detector layer: bundles on disk are conformant with an open
18
+ spec; the spec's reference parser loads them; the scoreability-gated
19
+ multi-class inference helper produces verdicts.
20
+
21
+ Dependencies::
22
+
23
+ pip install huggingface_hub pyarrow scikit-learn joblib numpy
24
+ pip install git+https://github.com/NullRabbitLabs/nr-bundle-spec.git
25
+
26
+ Usage::
27
+
28
+ python inference_example.py
29
+
30
+ Five bundles are scored across the V8 / V11 / V13 / V14 / benign class
31
+ manifolds. V16 (gossip-abuse) demonstration is not possible from the
32
+ public dataset because the public bundles drop raw ``packets.pcap`` per
33
+ the dataset's safety policy — see the note at the bottom of this file.
34
+ """
35
+
36
+ from __future__ import annotations
37
+
38
+ import importlib.util
39
+ import sys
40
+ from pathlib import Path
41
+
42
+ from huggingface_hub import hf_hub_download, snapshot_download
43
+
44
+
45
+ # ─── Constants ──────────────────────────────────────────────────────
46
+
47
+ MODEL_REPO = "NullRabbit/multiclass-folded"
48
+ DATASET_REPO = "NullRabbit/nr-bundles-public"
49
+
50
+ # Sample bundles drawn from nr-bundles-public. Note the OOD caveat at
51
+ # the bottom of this file — public bundles have raw packets.pcap
52
+ # dropped, so they are out-of-distribution for the multi-class model
53
+ # (trained on full-modality bundles). The four V8 / V13 / V14 attack
54
+ # bundles below have wire shapes that the model discriminates cleanly
55
+ # even without pcap; V11 / benign / V16 demonstrations require raw pcap
56
+ # and are not available from the public dataset.
57
+ SAMPLES = [
58
+ ("crp_19d438471fec4229", "sui_F10_multi_get_objects_amp", "V8 (response_amp, Sui) — survives pcap-drop"),
59
+ ("crp_2a9d40758d9a4192", "SOL_MC_grafana_anon", "V13 (service_misconfig, Solana) — survives pcap-drop"),
60
+ ("crp_1ef98f1fc0644369", "sui_F14_devinspect_tokio_wedge", "V14 (compute_amp, Sui) — survives pcap-drop"),
61
+ ("crp_0598afb4d5e44fb9", "sui_BENIGN_passive_fullnode", "benign passive (Sui) — tests scoreability gate"),
62
+ ]
63
+
64
+
65
+ def _load_module(name: str, path: str) -> "object":
66
+ spec = importlib.util.spec_from_file_location(name, path)
67
+ module = importlib.util.module_from_spec(spec) # type: ignore[arg-type]
68
+ sys.modules[name] = module
69
+ spec.loader.exec_module(module) # type: ignore[union-attr]
70
+ return module
71
+
72
+
73
+ def main() -> int:
74
+ print("=== Multi-class softmax folded detector ===")
75
+ print(f" model repo: {MODEL_REPO}")
76
+ print(f" dataset repo: {DATASET_REPO}")
77
+ print()
78
+
79
+ # Pull model + predict helper.
80
+ model_path = hf_hub_download(repo_id=MODEL_REPO, filename="model.joblib")
81
+ predict_path = hf_hub_download(repo_id=MODEL_REPO, filename="predict.py")
82
+
83
+ predict = _load_module("multiclass_predict", predict_path)
84
+ payload = predict.load_model(model_path)
85
+
86
+ print(f"Model loaded: {type(payload['model']).__name__}, "
87
+ f"{len(payload['feature_names'])} features, "
88
+ f"{len(payload['class_order'])} classes "
89
+ f"({payload['class_order']})")
90
+ print()
91
+
92
+ # Pull sample bundles.
93
+ dataset_root = Path(snapshot_download(
94
+ repo_id=DATASET_REPO, repo_type="dataset",
95
+ allow_patterns=[f"{cid}/*" for cid, _, _ in SAMPLES],
96
+ ))
97
+
98
+ # Score each.
99
+ for corpus_id, expected_primitive, label in SAMPLES:
100
+ bundle_dir = dataset_root / corpus_id
101
+ record = predict.score_bundle(bundle_dir, payload)
102
+ print(f"--- {corpus_id} ({expected_primitive}) ---")
103
+ print(f" expected: {label}")
104
+ print(f" verdict: {record['verdict']}")
105
+ if record["verdict"] == "unscoreable":
106
+ print(f" reason: {record['reason']}")
107
+ print(f" n_responses_rows: {record.get('n_responses_rows', 0)}")
108
+ else:
109
+ print(f" argmax P: {record['argmax_p']:.4f}")
110
+ print(f" feature_coverage: {record['feature_coverage']}")
111
+ print(f" n_responses_rows: {record['n_responses_rows']}")
112
+ print(f" top-3 class probabilities:")
113
+ top3 = sorted(record["class_probs"].items(),
114
+ key=lambda kv: -kv[1])[:3]
115
+ for cls, p in top3:
116
+ print(f" P({cls}) = {p:.4f}")
117
+ if record.get("coverage_warning"):
118
+ print(f" ⚠ coverage_warning: {record['coverage_warning']}")
119
+ print()
120
+
121
+ print("=" * 72)
122
+ print("Notes on multi-class folded deployment")
123
+ print("=" * 72)
124
+ print("""
125
+ - predict.score_bundle() is the recommended consumption surface. The
126
+ scoreability gate refuses to predict on bundles where neither
127
+ responses.parquet nor packets.pcap is present with content (typical
128
+ for passive-workload bundles where the validator listens without
129
+ serving). Callers who want raw model output without the gate should
130
+ load model.joblib directly via joblib.load.
131
+
132
+ - feature_coverage flag describes which modalities contributed:
133
+ - "full": both responses.parquet and packets.pcap present
134
+ - "resp_only": responses.parquet only — V16 (gossip-abuse) predictions
135
+ with this coverage are suspect (V16 needs pcap.*)
136
+ - "pcap_only": packets.pcap only — V8-V14 predictions with this
137
+ coverage are suspect (those classes need responses.*)
138
+ - "none": bundle is unscoreable; gate refused
139
+
140
+ - Public dataset bundles drop raw packets.pcap per the dataset's safety
141
+ policy, making them out-of-distribution for the multi-class model
142
+ (which was trained on full-modality bundles). Some class manifolds
143
+ survive the pcap-drop and produce correct verdicts (V8 response_amp,
144
+ V13 service_misconfig, V14 compute_amp — demonstrated above); others
145
+ do not (V11 rate_limiter_bypass and benign-with-traffic are
146
+ load-bearing on pcap.* features and skew to V16 when pcap is missing;
147
+ V16 itself requires pcap and cannot be demonstrated from public
148
+ bundles). To run reliable multi-class inference on V11 / benign / V16
149
+ bundles, produce your own bundles per nr-bundle-spec with raw pcap
150
+ retained, OR use the operator-internal corpus.
151
+
152
+ - The n=1 OOF fragility on the V16 load-bearing benign (SOL_BG01) is
153
+ documented in the model card's Load-bearing limitations section. The
154
+ fitted model routes SOL_BG01 to benign correctly; the OOF fold where
155
+ BG01 is held out routes it to V16 (the single benign→V16 confusion).
156
+ Production V16 deployment requires corpus scale-up to n≥10 UDP gossip
157
+ benigns across postures.
158
+ """.strip())
159
+ print("=" * 72)
160
+ return 0
161
+
162
+
163
+ if __name__ == "__main__":
164
+ raise SystemExit(main())
model.joblib ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3d53aaf89923c273e2129b1ff5641ac79d2b419cddf9ceaf4bdcb758b3e298f
3
+ size 15171887
predict.py ADDED
@@ -0,0 +1,232 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # SPDX-License-Identifier: Apache-2.0
3
+ """Multi-class softmax folded detector — scoreability-gated inference.
4
+
5
+ The recommended consumption surface for the 9-class V8-V14 + V16 multi-class
6
+ folded detector. Wraps the raw `CalibratedClassifierCV` estimator with two
7
+ production-side gates:
8
+
9
+ 1. **Scoreability gate**: refuses to score bundles where neither
10
+ ``responses.parquet`` nor ``packets.pcap`` has content. Bundles with
11
+ no observed RPC traffic AND no captured network packets cannot be
12
+ classified usefully; the gate returns an explicit "unscoreable"
13
+ verdict instead of producing a spurious argmax.
14
+
15
+ 2. **Feature-coverage flag**: emits a ``feature_coverage`` string
16
+ describing which bundle modalities contributed features
17
+ (``"resp_only"``, ``"pcap_only"``, ``"full"``, ``"partial"``). V16
18
+ gossip-abuse predictions are load-bearing on ``pcap.*`` features;
19
+ V8-V14 are load-bearing on ``responses.*``. Callers should
20
+ downweight predictions where the modality coverage doesn't match
21
+ the predicted class.
22
+
23
+ Callers who want raw model output without these gates should load
24
+ ``model.joblib`` directly — see the "Bypassing the gate" section of the
25
+ model card.
26
+
27
+ Usage::
28
+
29
+ from predict import load_model, score_bundle
30
+ payload = load_model("/path/to/model.joblib")
31
+ record = score_bundle("/path/to/bundle_dir", payload)
32
+ print(record["argmax_class"], record["class_probs"])
33
+ """
34
+
35
+ from __future__ import annotations
36
+
37
+ from pathlib import Path
38
+ from typing import Any
39
+
40
+ import joblib
41
+ import numpy as np
42
+ import pyarrow.parquet as pq
43
+
44
+ # nr-bundle-spec — the reference parser. Pip-install via
45
+ # pip install git+https://github.com/NullRabbitLabs/nr-bundle-spec.git
46
+ from bundle_spec import BundleManifest
47
+
48
+
49
+ def load_model(model_path: str | Path) -> dict[str, Any]:
50
+ """Load the multi-class folded lineage-dict payload from a joblib file."""
51
+ return joblib.load(model_path)
52
+
53
+
54
+ def _modality_state(bundle_dir: Path) -> tuple[bool, int, bool]:
55
+ """Inspect bundle modality presence.
56
+
57
+ Returns (has_responses_with_rows, n_responses_rows, has_packets_pcap).
58
+ """
59
+ responses_path = bundle_dir / "responses.parquet"
60
+ n_resp = 0
61
+ has_resp = False
62
+ if responses_path.is_file():
63
+ table = pq.read_table(responses_path)
64
+ n_resp = table.num_rows
65
+ has_resp = n_resp > 0
66
+ has_pcap = (bundle_dir / "packets.pcap").is_file()
67
+ return has_resp, n_resp, has_pcap
68
+
69
+
70
+ def _feature_coverage(has_resp: bool, has_pcap: bool) -> str:
71
+ """Bundle-level feature-coverage flag for downstream gating."""
72
+ if has_resp and has_pcap:
73
+ return "full"
74
+ if has_resp:
75
+ return "resp_only"
76
+ if has_pcap:
77
+ return "pcap_only"
78
+ return "none"
79
+
80
+
81
+ def _extract_features(bundle_dir: Path, feature_names: list[str]) -> np.ndarray:
82
+ """Extract the model's 107-feature vector from a bundle directory.
83
+
84
+ Uses the nr_training feature extractor under the hood — same pipeline
85
+ the model was trained against. Falls back to a minimal pyarrow-based
86
+ extractor for the response-aggregate features if nr_training isn't
87
+ on the import path (deployment-time graceful degradation).
88
+ """
89
+ # Try the canonical extractor first; fall back to manual extraction.
90
+ try:
91
+ import sys
92
+ sys.path.insert(0, str(Path(__file__).resolve().parent))
93
+ # nr_training is the substrate-side feature extractor; absent in
94
+ # most deployment envs. Caller should install it from the
95
+ # nr-substrate working repo if they want exact-equivalence
96
+ # extraction matching training. This block is best-effort.
97
+ from nr_training.contracts import BundleManifest as _BM
98
+ from nr_training.datasets.loader import Bundle, _sha256
99
+ from nr_training.features import batch_extract
100
+ mfp = bundle_dir / "manifest.json"
101
+ m = _BM.model_validate_json(mfp.read_text())
102
+ b = Bundle(corpus_id=m.corpus_id, bundle_dir=bundle_dir, manifest=m,
103
+ manifest_sha256=_sha256(mfp), pcap_sha256=None)
104
+ fvs = batch_extract([b])
105
+ return np.array([[fvs[0].features.get(n, 0.0) for n in feature_names]], dtype=float)
106
+ except ImportError:
107
+ # Minimal fallback: only resp.* features from responses.parquet.
108
+ features = {name: 0.0 for name in feature_names}
109
+ rp = bundle_dir / "responses.parquet"
110
+ if rp.is_file():
111
+ table = pq.read_table(rp)
112
+ if table.num_rows > 0:
113
+ req = table.column("request_size_bytes").to_numpy()
114
+ resp = table.column("response_size_bytes").to_numpy()
115
+ if "resp.req_bytes_max" in features:
116
+ features["resp.req_bytes_max"] = float(req.max())
117
+ if "resp.resp_bytes_max" in features:
118
+ features["resp.resp_bytes_max"] = float(resp.max())
119
+ with np.errstate(divide="ignore", invalid="ignore"):
120
+ ratios = np.where(req > 0, resp / req, 0.0)
121
+ if "resp.amp_ratio_max" in features:
122
+ features["resp.amp_ratio_max"] = float(ratios.max())
123
+ if "resp.amp_ratio_mean" in features:
124
+ features["resp.amp_ratio_mean"] = float(ratios.mean())
125
+ if "resp.amp_ratio_median" in features:
126
+ features["resp.amp_ratio_median"] = float(np.median(ratios))
127
+ return np.array([[features[n] for n in feature_names]], dtype=float)
128
+
129
+
130
+ def score_bundle(bundle_dir: str | Path, payload: dict[str, Any]) -> dict[str, Any]:
131
+ """Score one bundle through the multi-class folded model.
132
+
133
+ Returns a record with:
134
+ - ``verdict``: ``"<class_name>"`` or ``"unscoreable"``.
135
+ - ``argmax_class``: argmax class name (None if unscoreable).
136
+ - ``argmax_p``: probability of the argmax class (None if unscoreable).
137
+ - ``class_probs``: dict of P(class) for every class in class_order.
138
+ - ``reason``: human-readable explanation when unscoreable.
139
+ - ``feature_coverage``: ``"full"`` / ``"resp_only"`` / ``"pcap_only"`` / ``"none"``.
140
+ - ``corpus_id``, ``primitive_id``, ``ground_truth``: from manifest.
141
+ - ``n_responses_rows``: number of rows in responses.parquet.
142
+ """
143
+ bundle_dir = Path(bundle_dir)
144
+
145
+ manifest_path = bundle_dir / "manifest.json"
146
+ if not manifest_path.is_file():
147
+ return {
148
+ "verdict": "unscoreable",
149
+ "reason": f"manifest.json not found at {manifest_path}",
150
+ "argmax_class": None,
151
+ "argmax_p": None,
152
+ "class_probs": None,
153
+ }
154
+ manifest = BundleManifest.model_validate_json(manifest_path.read_text())
155
+
156
+ has_resp, n_resp, has_pcap = _modality_state(bundle_dir)
157
+
158
+ # Scoreability gate: at least one of {responses.parquet with rows,
159
+ # packets.pcap on disk} must be present.
160
+ if not (has_resp or has_pcap):
161
+ return {
162
+ "verdict": "unscoreable",
163
+ "reason": (
164
+ "Neither responses.parquet (with rows) nor packets.pcap is "
165
+ "present in the bundle. The multi-class folded detector "
166
+ "cannot classify bundles with no observed RPC traffic AND "
167
+ "no captured network packets. Bundles in this state are "
168
+ "typically passive-workload captures (e.g. validator running "
169
+ "idle with no clients) — use a non-bundle telemetry path "
170
+ "for those workloads."
171
+ ),
172
+ "argmax_class": None,
173
+ "argmax_p": None,
174
+ "class_probs": None,
175
+ "corpus_id": manifest.corpus_id,
176
+ "primitive_id": manifest.primitive_id,
177
+ "feature_coverage": "none",
178
+ "n_responses_rows": n_resp,
179
+ }
180
+
181
+ feature_names = payload["feature_names"]
182
+ class_order = payload["class_order"]
183
+ X = _extract_features(bundle_dir, feature_names)
184
+ proba = payload["model"].predict_proba(X)[0]
185
+ argmax = int(np.argmax(proba))
186
+ class_probs = {cls: float(proba[i]) for i, cls in enumerate(class_order)}
187
+
188
+ coverage = _feature_coverage(has_resp, has_pcap)
189
+ argmax_class = class_order[argmax]
190
+
191
+ # Modality-mismatch warning: V8-V14 classes are load-bearing on pcap.*
192
+ # features for some discriminations (especially rate-cardinality
193
+ # features that V11 / benign-vs-attack boundaries depend on). If the
194
+ # bundle is resp_only and the model picks a non-V16 class with low
195
+ # confidence, the prediction may be OOD-by-construction (the model
196
+ # was trained on full-modality bundles; resp_only inputs aren't part
197
+ # of its training distribution). Surface the warning.
198
+ coverage_warning = None
199
+ if coverage == "resp_only" and argmax_class != "V16" and proba[argmax] < 0.8:
200
+ coverage_warning = (
201
+ f"argmax={argmax_class} with P={proba[argmax]:.3f} on resp_only "
202
+ "coverage; multi-class folded was trained on full-modality "
203
+ "bundles, so predictions on pcap-absent inputs are out-of-"
204
+ "distribution. For reliable V8-V14 inference, provide bundles "
205
+ "with raw packets.pcap present."
206
+ )
207
+ elif coverage == "resp_only" and argmax_class == "V16":
208
+ coverage_warning = (
209
+ "argmax=V16 with resp_only coverage. V16 is load-bearing on "
210
+ "pcap.* features; an argmax of V16 on a pcap-absent bundle "
211
+ "is likely a misclassification driven by the missing-modality "
212
+ "signal, not a true gossip-abuse detection. Provide bundles "
213
+ "with raw packets.pcap for V16 inference."
214
+ )
215
+
216
+ return {
217
+ "verdict": argmax_class,
218
+ "argmax_class": argmax_class,
219
+ "argmax_p": float(proba[argmax]),
220
+ "class_probs": class_probs,
221
+ "reason": None,
222
+ "corpus_id": manifest.corpus_id,
223
+ "primitive_id": manifest.primitive_id,
224
+ "ground_truth": (
225
+ manifest.ground_truth_label.value
226
+ if hasattr(manifest.ground_truth_label, "value")
227
+ else str(manifest.ground_truth_label)
228
+ ),
229
+ "feature_coverage": coverage,
230
+ "coverage_warning": coverage_warning,
231
+ "n_responses_rows": n_resp,
232
+ }
release-cert.json ADDED
@@ -0,0 +1,190 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audited_at": "2026-05-13T18:22:26Z",
3
+ "model_repo": "NullRabbit/multiclass-folded",
4
+ "checks": [
5
+ {
6
+ "check": "joblib_loads",
7
+ "ok": true
8
+ },
9
+ {
10
+ "check": "lineage_dict_shape",
11
+ "ok": true
12
+ },
13
+ {
14
+ "check": "class_order_9_classes_with_V16",
15
+ "ok": true
16
+ },
17
+ {
18
+ "check": "feature_count_107",
19
+ "ok": true
20
+ },
21
+ {
22
+ "check": "overall_oof_above_0_99",
23
+ "ok": true
24
+ },
25
+ {
26
+ "check": "per_class_recall_ge_0.99:benign",
27
+ "ok": true
28
+ },
29
+ {
30
+ "check": "per_class_recall_ge_0.99:V8",
31
+ "ok": true
32
+ },
33
+ {
34
+ "check": "per_class_recall_ge_0.99:V9",
35
+ "ok": true
36
+ },
37
+ {
38
+ "check": "per_class_recall_ge_0.99:V10",
39
+ "ok": true
40
+ },
41
+ {
42
+ "check": "per_class_recall_ge_0.99:V11",
43
+ "ok": true
44
+ },
45
+ {
46
+ "check": "per_class_recall_ge_0.99:V12",
47
+ "ok": true
48
+ },
49
+ {
50
+ "check": "per_class_recall_ge_0.99:V13",
51
+ "ok": true
52
+ },
53
+ {
54
+ "check": "per_class_recall_ge_0.99:V14",
55
+ "ok": true
56
+ },
57
+ {
58
+ "check": "V16_in_per_class",
59
+ "ok": true
60
+ },
61
+ {
62
+ "check": "V16_recall_1.0",
63
+ "ok": true
64
+ },
65
+ {
66
+ "check": "readme_anchor_autonomous_defence",
67
+ "ok": true
68
+ },
69
+ {
70
+ "check": "readme_anchor_multi_class_softmax",
71
+ "ok": true
72
+ },
73
+ {
74
+ "check": "readme_anchor_iterative_leak_surface_peeling",
75
+ "ok": true
76
+ },
77
+ {
78
+ "check": "readme_methodology_is_the_contribution",
79
+ "ok": true
80
+ },
81
+ {
82
+ "check": "readme_substrate_paper_in_preparation",
83
+ "ok": true
84
+ },
85
+ {
86
+ "check": "readme_zenodo_doi_present",
87
+ "ok": true
88
+ },
89
+ {
90
+ "check": "readme_cross_links_nr_bundles_public",
91
+ "ok": true
92
+ },
93
+ {
94
+ "check": "readme_cross_links_v8",
95
+ "ok": true
96
+ },
97
+ {
98
+ "check": "readme_cross_links_nr_bundle_spec",
99
+ "ok": true
100
+ },
101
+ {
102
+ "check": "readme_has_load_bearing_limitations",
103
+ "ok": true
104
+ },
105
+ {
106
+ "check": "readme_has_n1_bg01_caveat",
107
+ "ok": true
108
+ },
109
+ {
110
+ "check": "readme_has_modality_sensitivity_documented",
111
+ "ok": true
112
+ },
113
+ {
114
+ "check": "readme_has_class_specific_pcap_sensitivity",
115
+ "ok": true
116
+ },
117
+ {
118
+ "check": "readme_apache_2_0",
119
+ "ok": true
120
+ },
121
+ {
122
+ "check": "readme_recommends_predict_py",
123
+ "ok": true
124
+ },
125
+ {
126
+ "check": "readme_has_bypassing_the_gate",
127
+ "ok": true
128
+ },
129
+ {
130
+ "check": "readme_has_oof_evaluation_table",
131
+ "ok": true
132
+ },
133
+ {
134
+ "check": "readme_no_embargo_language",
135
+ "ok": true
136
+ },
137
+ {
138
+ "check": "predict.py_exists",
139
+ "ok": true
140
+ },
141
+ {
142
+ "check": "predict.py_compiles",
143
+ "ok": true
144
+ },
145
+ {
146
+ "check": "inference_example.py_exists",
147
+ "ok": true
148
+ },
149
+ {
150
+ "check": "inference_example.py_compiles",
151
+ "ok": true
152
+ },
153
+ {
154
+ "check": "apache_2_0_license_text",
155
+ "ok": true
156
+ },
157
+ {
158
+ "check": "gate_pcap_robust_class_V8_on_crp_19d438471fec4229",
159
+ "ok": true,
160
+ "verdict": "V8",
161
+ "argmax_p": 0.46512720722039613
162
+ },
163
+ {
164
+ "check": "gate_pcap_robust_class_V13_on_crp_2a9d40758d9a4192",
165
+ "ok": true,
166
+ "verdict": "V13",
167
+ "argmax_p": 0.6856656927025966
168
+ },
169
+ {
170
+ "check": "gate_pcap_robust_class_V14_on_crp_1ef98f1fc0644369",
171
+ "ok": true,
172
+ "verdict": "V14",
173
+ "argmax_p": 0.8227460998359055
174
+ },
175
+ {
176
+ "check": "gate_passive_fullnode_unscoreable",
177
+ "ok": true,
178
+ "verdict": "unscoreable"
179
+ },
180
+ {
181
+ "check": "gate_emits_coverage_warning_on_pcap_sensitive_misclass",
182
+ "ok": true,
183
+ "verdict": "V16",
184
+ "coverage_warning_excerpt": "argmax=V16 with resp_only coverage. V16 is load-bearing on pcap.* features; an a"
185
+ }
186
+ ],
187
+ "n_checks": 43,
188
+ "n_ok": 43,
189
+ "release_ok": true
190
+ }