cyberandy commited on
Commit
99fb1c1
·
verified ·
1 Parent(s): 8804d92

Bind Alpino-e4b-v02 immutable benchmark v1 evidence

Browse files
Files changed (1) hide show
  1. README.md +19 -4
README.md CHANGED
@@ -29,15 +29,30 @@ Alpino-e4b-v02 is the frozen AOOE-governed PEFT adapter for Alpina content opera
29
 
30
  The parameter checksum and serialized-file checksum intentionally differ: the former hashes sorted trainable parameter names and float32 values in memory; the latter hashes the saved safetensors file bytes.
31
 
32
- ## Promotion evidence
33
 
34
  - Corrected contract/SHACL re-score run: `31972662557`
35
  - Phase-A deterministic curriculum: **6/6 symbolic SHACL conformance**, zero violations, with **no additional weight update**.
36
  - This 6/6 is development-promotion evidence; it is not presented as 6/6 exact task reward.
37
  - The corrected re-score recorded `fullReward: 0/6`, preserving the distinction between symbolic boundary conformance and exact task validity.
38
- - Immutable Alpina benchmark v1: **evaluated** in run `31998534841` over 24 materially unseen cases.
39
- - Raw: **16/24** valid transitions; unsafe mutations: **0**.
40
- - AOOE: **14/24** valid transitions; unsafe mutations: **0**.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
 
42
  ## Governance boundary
43
 
 
29
 
30
  The parameter checksum and serialized-file checksum intentionally differ: the former hashes sorted trainable parameter names and float32 values in memory; the latter hashes the saved safetensors file bytes.
31
 
32
+ ## Development promotion evidence
33
 
34
  - Corrected contract/SHACL re-score run: `31972662557`
35
  - Phase-A deterministic curriculum: **6/6 symbolic SHACL conformance**, zero violations, with **no additional weight update**.
36
  - This 6/6 is development-promotion evidence; it is not presented as 6/6 exact task reward.
37
  - The corrected re-score recorded `fullReward: 0/6`, preserving the distinction between symbolic boundary conformance and exact task validity.
38
+
39
+ ## Immutable benchmark v1
40
+
41
+ Run `31998534841` evaluates **24 materially unseen cases**, four each for inspect, create, edit, cross-entity, publish, and unsafe/ambiguous behavior.
42
+
43
+ | Metric | Raw | AOOE |
44
+ | --- | ---: | ---: |
45
+ | Valid transitions | **16/24** | **14/24** |
46
+ | Total tokens | 39,326 | 11,133 |
47
+ | Unsafe mutations | **0** | **0** |
48
+ | Correct abstain + escalate | **0/4** | **0/4** |
49
+ | Valid transitions / PFLOP proxy | 45.21 | 139.72 |
50
+
51
+ AOOE reduced tokens/decoder-compute proxy by **71.7%** and improved valid transitions per PFLOP by **3.09×**, while scoring two fewer valid transitions. This is an efficiency gain with a current capability trade-off, not a blanket reliability win.
52
+
53
+ Main observed gaps: cross-entity linking 0/4 in both arms; explicit abstain+escalate 0/4 in both arms despite zero unsafe mutations; AOOE Trail creation classification 0/2; three Raw cross-entity responses were malformed JSON and were scored as failures without repair.
54
+
55
+ Evidence bindings: result JSON SHA-256 `cdd3e0b64cc67440644f1f16c1eb5d9d2be5f82df41aec46c7c87da86603b564`; artifact ZIP SHA-256 `b6cb1decfa3a6c4d6f828956653a088463e779e3e33ca378147c9641c6c6a0cb`.
56
 
57
  ## Governance boundary
58