Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,149 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-llmfao/general_knowledge_model`](https://huggingface.co/cs-552-2026-llmfao/general_knowledge_model)
+- **Owner(s):** group **llmfao**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`a2d1d7f`](https://huggingface.co/cs-552-2026-llmfao/general_knowledge_model/commit/a2d1d7fe9856a5ed5ba10b6fb98d305d77ca0456)
+- **Message:** Upload ./general_knowledge/model/qwen-GN-model-08_06_20h_15/merged
+- **Committed:** 2026-06-09T20:48:33+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.2300 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `B`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `B`
+- **completion**:
+
+    ```text
+    <think>
+    ['The Hamiltonian operator is given by $H = \\varepsilon \\vec{\\sigma} \\cdot \\vec{n}$.', 'The Pauli matrices have eigenvalues 1 and -1.', 'So, the Hamiltonian has eigenvalues $\\varepsilon (1 \\cdot \\vec{n}) = \\varepsilon$ and $\\varepsilon (-1 \\cdot \\vec{n}) = -\\varepsilon$.', 'The answer is (B).']
+    </think>
+    
+    \boxed{B}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `<no answer>`
+- **completion**:
+
+    ```text
+    <think>
+    ['The speed of light in a medium is given by $v_{glass} = \n\\frac{c}{n}$.', 'The velocity of the glass in the laboratory frame is $v$.', 'Since the velocity of the glass is $v$, the velocity of the light in the glass is $v_{glass} = \n\\frac{c}{n} = \n\\frac{v}{1+v/v_glass}$.', 'However, since the glass is moving with velocity $v$ in the laboratory frame, the velocity of the light in the
+    
+    ... [truncated 812 chars] ...
+    
+    ab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{v}{1+v/n}
+    $$
+    
+    $$
+    v_{lab} = \n\\frac{c}{n} + \n\\frac{vn}{v+n}
+    $$
+    ```