Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,168 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-RatGPT/math_model`](https://huggingface.co/cs-552-2026-RatGPT/math_model)
+- **Owner(s):** group **RatGPT**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`7c489c6`](https://huggingface.co/cs-552-2026-RatGPT/math_model/commit/7c489c6863dd1f8c063b6bd51711422c3e431a0f)
+- **Message:** Upload group-model RFT v2 math checkpoint
+- **Committed:** 2026-06-09T09:59:39+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | 0.3600 | ok |
+| Knowledge | — | not run |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Math
+
+**Correct** (1 shown)
+
+- **reference**: `$\frac{a}{2}$`
+- **overall** (1/8 completions correct)
+- **extracted** (✓): `\frac{a}{2}`
+- **completion**:
+
+    ```text
+    <think>
+    
+    </think>
+    
+    1. **Identify Key Points and Properties**:
+       - \(P\) is the circumcenter of \(\triangle XYZ\).
+       - \(Q\) is the midpoint of \(YZ\).
+       - \(R\) is the reflection of \(Q\) over \(XZ\).
+       - \(S\) is chosen such that \(XQYS\) is a rectangle.
+       - Given \(\angle YXZ = a\).
+    
+    2. **Reflection and Midpoint**:
+       - Since \(Q\) is the midpoint of \(YZ\), \(XQ\) is the median.
+       - Refle
+    
+    ... [truncated 512 chars] ...
+    
+    \) over \(XZ\), is at \(\left(\frac{c + d}{2}, -\frac{e}{2}\right)\).
+    
+    5. **Equations and Symmetry**:
+       - The rectangle \(XQYS\) implies \(YS\) is horizontal.
+       - \(S\) is at \(\left(\frac{c + d}{2}, 0\right)\).
+    
+    6. **Circumcenter \(P\)**:
+       - \(P\) is the circumcenter of \(\triangle XYZ\).
+       - The circumcenter \(P\) lies on the perpendicular bisector of \(XZ\), which is the y-axis.
+       - Therefore, \(P\) is at \((0, k)\).
+    
+    7. **Angle Calculation**:
+       - The angle \(\angle PRS\) is calculated using vector analysis and properties of the circumcenter.
+       - Vectors \(PR\) and \(PS\) are calculated as:
+         \[
+         PR = \left( \frac{c + d}{2}, -\frac{e}{2} - k \right)
+         \]
+         \[
+         PS = \left( \frac{c + d}{2}, -k \right)
+         \]
+       - The angle \(\angle PRS\) is found using the dot product:
+         \[
+         \cos \theta = \frac{\vec{PR} \cdot \vec{PS}}{|\vec{PR}| |\vec{PS}|}
+         \]
+       - Simplifying the dot product and magnitudes, we find:
+         \[
+         \cos \theta = \frac{e^2}{2e^2 + 4k^2}
+         \]
+       - The angle \(\theta\) is found to be \(a/2\).
+    
+    ### Conclusion
+    The angle \(\angle PRS\) in terms of \(a\) is \(a/2\).
+    
+    \[
+    \angle PRS = \frac{a}{2}
+    \]
+    
+    Final answer:
+    \boxed{\frac{a}{2}}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `756`
+- **overall** (0/8 completions correct)
+- **extracted** (✗): `111`
+- **completion**:
+
+    ```text
+    <think>
+    
+    </think>
+    
+    1. **Identify the centers and radii of the balloons:**
+       - The radii of the balloons are 11, 13, and 19.
+       - The centers of the balloons are denoted by \( X \), \( Y \), and \( Z \).
+    
+    2. **Use the distance formula between the centers:**
+       - Given \( XY^2 = 560 \), we can express the distance between centers \( X \) and \( Y \) as \( \sqrt{560} \).
+    
+    3. **Use the Pythagorean th
+    
+    ... [truncated 770 chars] ...
+    
+    5)^2
+         \]
+         Solving for \( h_3 \):
+         \[
+         h_3^2 = 225 - 361 = -136 \implies h_3 = \sqrt{136}
+         \]
+    
+    5. **Use the distance formula for the centers \( X \), \( Y \), and \( Z \):**
+       - The distance between the centers \( X \) and \( Y \) is given by:
+         \[
+         XY^2 = (h_1 - h_2)^2 + (13 - 11)^2 = 560
+         \]
+         Simplifying:
+         \[
+         (h_1 - h_2)^2 + 4 = 560 \implies (h_1 - h_2)^2 = 556
+         \]
+    
+    6. **Use the Pythagorean theorem in the triangle formed by \( X \), \( Y \), and \( Z \):**
+       - The distance \( XZ^2 \) is given by:
+         \[
+         XZ^2 = (h_1 - h_3)^2 + (19 - 11)^2
+         \]
+         Substituting \( h_1 = \sqrt{23} \), \( h_3 = \sqrt{136} \):
+         \[
+         XZ^2 = (\sqrt{23} - \sqrt{136})^2 + (8)^2
+         \]
+         Simplify \( (\sqrt{23} - \sqrt{136})^2 \):
+         \[
+         (\sqrt{23} - \sqrt{136})^2 = 23 + 136 - 2\sqrt{23 \cdot 136} = 159 - 2\sqrt{3128}
+         \]
+         Since \( \sqrt{3128} = \sqrt{16 \cdot 196} = 4 \cdot 14 = 56 \):
+         \[
+         (\sqrt{23} - \sqrt{136})^2 = 159 - 2 \cdot 56 = 159 - 112 = 47
+         \]
+         Therefore:
+         \[
+         XZ^2 = 47 + 64 = 111
+         \]
+    
+    7. **Conclusion:**
+       - The value of \( XZ^2 \) is \( 111 \).
+    
+    \[
+    111
+    \]
+    
+    Final answer:
+    \boxed{111}
+    ```