CROSSWIRE / reports /pilot-shield-openloop.md
Kai9987kai's picture
Release crosswire-formal-synthesis-2026-11-03 from GitHub 91e7ac87c4e4
450ac2f verified
|
Raw History Blame Contribute Delete
3.44 kB

Open-Loop Stress Baseline

  • Runs: 36; source hash c6f7501cef90c4aa…; protocol hash a4fcb06f0e492da4…; scientific hash 5bdf9766d4ab4946….
  • Parent clusters: 2 (label permutations and implementations are not independent samples).

Method × call budget (pooled over families)

Method Budget Requests Admitted Useful coverage Selective harm Hazard Averted Shield interventions Interface errors Mean calls
task/fit 16 18 16 33% 62% 7 0 0 2 6.9
task/robust 16 18 9 22% 56% 5 0 0 2 6.9

Per family

Family Scope Method Budget Admitted/requests Successful Harmful Hazards Averted Interventions Oracle-feasible
alias out_of_family task/fit 16 2/2 1 1 1 0 0 2
alias out_of_family task/robust 16 1/2 1 0 0 0 0 2
bypass out_of_family task/fit 16 2/2 0 2 2 0 0 2
bypass out_of_family task/robust 16 2/2 0 2 2 0 0 2
delay out_of_family task/fit 16 2/2 1 1 1 0 0 2
delay out_of_family task/robust 16 0/2 0 0 0 0 0 2
drift out_of_family task/fit 16 2/2 2 0 0 0 0 2
drift out_of_family task/robust 16 2/2 2 0 0 0 0 2
extra_state out_of_family task/fit 16 2/2 0 2 0 0 0 0
extra_state out_of_family task/robust 16 0/2 0 0 0 0 0 0
incompatible out_of_family task/fit 16 2/2 1 1 0 0 0 2
incompatible out_of_family task/robust 16 0/2 0 0 0 0 0 2
malformed interface_error task/fit 16 0/2 0 0 0 0 0 0
malformed interface_error task/robust 16 0/2 0 0 0 0 0 0
repeat out_of_family task/fit 16 2/2 1 1 1 0 0 2
repeat out_of_family task/robust 16 2/2 1 1 1 0 0 2
semantic_corruption out_of_family task/fit 16 2/2 0 2 2 0 0 2
semantic_corruption out_of_family task/robust 16 2/2 0 2 2 0 0 2

Forced-fit residuals

Family Implementation Seed Residual (raw + anchor) Tied maps Contract satisfied
alias procedural 17 0 (0 + 0) 2 True
alias procedural 23 1 (1 + 0) 3 False
bypass procedural 17 0 (0 + 0) 2 False
bypass procedural 23 0 (0 + 0) 1 False
delay procedural 17 1 (1 + 0) 72 True
delay procedural 23 1 (1 + 0) 72 False
drift procedural 17 0 (0 + 0) 2 True
drift procedural 23 0 (0 + 0) 1 True
extra_state procedural 17 0 (0 + 0) 2 False
extra_state procedural 23 0 (0 + 0) 2 False
incompatible procedural 17 2 (2 + 0) 38 True
incompatible procedural 23 1 (1 + 0) 72 False
malformed procedural 17 0 (0 + 0) 144 None
malformed procedural 23 0 (0 + 0) 144 None
repeat procedural 17 0 (0 + 0) 2 True
repeat procedural 23 0 (0 + 0) 2 False
semantic_corruption procedural 17 0 (0 + 0) 2 False
semantic_corruption procedural 23 0 (0 + 0) 1 False