eval(fingerprint): Global MMLU Lite EN / lemer-mlx-bf16 / 1-round

Per-question full-output fingerprint on CohereForAI/Global-MMLU-Lite config en
test split (400 questions). Single round, mlx_lm greedy, max_tokens=2048.
Full model output preserved per row in parquet column full_model_output.

Scores (n=400):
- strict letter regex: 260/400 = 65.0%
- content-aware: 274/400 = 68.5%
- no-answer: 10/400 = 2.5%

Cultural sensitivity stratification:
- CS (200q) strict 65.5% content 69.0%
- CA (200q) strict 64.5% content 68.0%
- Cultural fairness (1-|CS-CA|) = 0.990

NOT 8-PAC consensus — this is fingerprint-purpose disclosure for alignment
auditing. Readers can inspect which questions the model disagrees with gold
and the full reasoning output per case. 8-PAC statistical consensus (8 rounds
paired vs base Gemma 4 E2B IT) is the follow-up.

Paper reference: §16 (revise benchmark toward model).

eval_results/global_mmlu_lite_en.mlx.BF16.md

@@ -0,0 +1,31 @@
+# Global MMLU Lite EN — lemer-mlx-bf16
+
+> **1-round fingerprint — not 8-PAC consensus.** Per-question full model output preserved for alignment-signature auditing. 8-round statistical consensus is the follow-up run.
+
+Per-question fingerprint. Full model output preserved per row in the parquet.
+
+## Scores (n=400)
+
+| Metric | Value |
+|---|---|
+| Strict letter regex | 260/400 = 65.0% |
+| Content-aware fallback | 274/400 = 68.5% |
+| No-answer | 10/400 = 2.5% |
+
+## Cultural sensitivity stratification
+
+| | n | Strict | Content |
+|---|---|---|---|
+| CS | 200 | 131/200 = 65.5% | 138/200 = 69.0% |
+| CA | 200 | 129/200 = 64.5% | 136/200 = 68.0% |
+| **Cultural fairness** (1−\|CS−CA\|) | — | 0.99 | 0.99 |
+
+## Notes
+
+- No coercion, no retry. Parser is regex over raw output + content-match fallback.
+- `agrees_with_gold_strict` and `agrees_with_gold_content` are both surfaced — readers can audit.
+- Dataset: `CohereForAI/Global-MMLU-Lite` config `en` split `test`.
+- Model: lemer (Gemma 4 E2B + LEK, bf16 MLX reference).
+- Sampling: mlx_lm greedy, max_tokens 2048.
+- Timestamp: 2026-04-17T11:56:15.577413+00:00
+- Runtime: 746s (0.54 q/s).

eval_results/global_mmlu_lite_en.mlx.BF16.parquet

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35f87dde7df9e4c2097cd41485c0c64f703c20fd34fba994c52cdcc7fa021dca
+size 287879

eval_results/global_mmlu_lite_en.mlx.BF16.yaml

@@ -0,0 +1,58 @@
+task: global_mmlu_lite_en
+dataset:
+  repo: CohereForAI/Global-MMLU-Lite
+  config: en
+  split: test
+  rows: 400
+model:
+  repo: lthn/lemer-mlx-bf16
+  local_path: /Volumes/Data/lem/models/lemma.1.x.x/v1.0.1/lemer-mlx-bf16
+  backend: mlx_lm
+  quant: BF16
+sampling:
+  max_tokens: 2048
+  mlx_lm_defaults: greedy
+prompt_template: Question + A/B/C/D + 'Reason briefly, then end with the single letter
+  answer.'
+parser: 'strict: last A/B/C/D letter regex. content: fallback to unique/last option-text
+  match.'
+scores:
+  strict_letter:
+    correct: 260
+    n: 400
+    pct: 65.0
+  content_aware:
+    correct: 274
+    n: 400
+    pct: 68.5
+  no_answer:
+    n: 10
+    pct: 2.5
+  cs:
+    n: 200
+    strict_correct: 131
+    strict_pct: 65.5
+    content_correct: 138
+    content_pct: 69.0
+  ca:
+    n: 200
+    strict_correct: 129
+    strict_pct: 64.5
+    content_correct: 136
+    content_pct: 68.0
+  cultural_fairness_strict: 0.99
+  cultural_fairness_content: 0.99
+timestamp_utc: '2026-04-17T11:56:15.577413+00:00'
+runtime_seconds: 745.7
+throughput_qps: 0.54
+host: m3-ultra (local)
+note: "This is a 1-round per-question fingerprint capture. Full model output is preserved\
+  \ per row in the parquet column full_model_output. Not a statistical accuracy claim\
+  \ \u2014 8-PAC consensus (8 rounds paired vs base Gemma 4 E2B IT) is the follow-up.\
+  \ Published here to disclose the model alignment fingerprint: readers can audit\
+  \ which questions the model disagrees with and on what grounds. For the reasoning\
+  \ behind publishing disagreement patterns rather than just accuracy scores, see\
+  \ paper section 16 (revise benchmark toward model)."
+rounds: 1
+protocol: single-round fingerprint (not 8-PAC consensus)
+status: preliminary