| # Psy Evaluation Protocol |
|
|
| ## Rules |
|
|
| 1. Evaluate one artifact family at a time. |
| 2. Report majority-class baseline for every probe. |
| 3. Report train, validation, and test row counts. |
| 4. Report class distribution. |
| 5. Mark probes with severe class imbalance as marginal or unusable. |
| 6. For CVE probes, remove direct CVSS/severity fields before claiming structural signal. |
| 7. Do not use mixed-family accuracy as proof of cyber judgment. |
| 8. Do not ship raw corpus, training shards, full logs, or private source paths in result artifacts. |
|
|
| ## Current Bundle Results |
|
|
| Included result JSONs are sanitized summaries, not full logs: |
|
|
| - `results/cve_sanitized_result.json` |
| - `results/rule_ast_result.json` |
| - `results/network_flow_result.json` |
|
|
| ## Validity Checklist |
|
|
| - Single artifact family only. |
| - Baseline reported. |
| - Class distribution reported. |
| - Label source disclosed. |
| - Leakage caveat included where relevant. |
| - Result JSON includes source note and status. |
| - Probe-head weights included only if saved and verified. |
|
|
|
|
