Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,91 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-MandMP/general_knowledge_model`](https://huggingface.co/cs-552-2026-MandMP/general_knowledge_model)
+- **Owner(s):** group **MandMP**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`7f910fa`](https://huggingface.co/cs-552-2026-MandMP/general_knowledge_model/commit/7f910fabdfb320e0b8d69069d534ee60b5ac2e3b)
+- **Message:** gk_sft_mixed_v1 - mixed 4+10+20 options DeepSeek CoT
+- **Committed:** 2026-06-10T10:03:48+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.3500 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `F`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `F`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's tackle this legal question. So, the scenario is about a baseball player trying to get a contract. His friend agreed to drive him to the tryout and pay for his housing and meals, in exchange for 10% of his first-year earnings. Then the team offered him a contract, but he said he'd take $1k instead of $10k. The friend wants to sue to enforce the assignment.
+    
+    Hmm. What's the key h
+    
+    ... [truncated 823 chars] ...
+    
+    he mistake is in thinking that the assignment is unenforceable. But given the options, the correct answer is F.
+    </think>
+    
+    The answer is (F).
+    
+    **Explanation:**  
+    The assignment in question involves a right to earnings from a future event (signing a contract with the professional team). Under contract law, such assignments are generally enforceable once the event (future right) arises. The athlete signed the contract with the team, fulfilling the condition of the assignment. While the agreement was verbal, the court typically recognizes oral assignments for rights tied to future events. The friend’s reliance on the assignment (e.g., expecting financial gain) is irrelevant to the enforceability of the agreement, as the right to earnings already exists. The court would likely find the assignment enforceable at the time it was made, unless the right is contingent on further events not yet occurred. However, the question specifies the assignment is unenforceable, making (F) the most plausible explanation. Other options, such as reliance or lack of consideration, do not align with the established principles governing assignments of rights tied to future events. 
+    
+    **The answer is \boxed{F}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `C`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `<no answer>`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, so I need to find the ratio of the equilibrium temperatures of Planet 4 to Planet 2 using the TTV (Transit Timing Variations) method. Hmm, let me think about how this works.
+    
+    First, I remember that the equilibrium temperature of an exoplanet depends on several factors like its distance from its star, the star's luminosity, and other things related to the planet's properties. The form
+    
+    ... [truncated 823 chars] ...
+    
+    ^(1/4 ) = 1.75 → R_planet4 = (1.75^4 ) * r_planet4².
+    
+    Similarly, for Planet_2: R_planet2 = (1.75^4 ) * r_planet2².
+    
+    But we don't know the actual values of r_planet4 and r_planet2. However, maybe we can find the ratio of their temperatures, which is already given as 1.75. Therefore, the answer is just 1.75, which would be option B (0.69?) Wait, no. Wait 1.75 is the ratio. Looking at the choices, none of them is 1.75. The options are A to D. Let me check the options again.
+    
+    Choices:
+    A. ~0.75
+    B. ~0.69
+    C. ~0.83
+    D. ~0.57
+    
+    Wait, none of these match 1.75. So clearly, my approach is wrong. What went wrong here?
+    
+    Alternative approach: Perhaps I misapplied the formula. Maybe the formula is T_planet^4 = L_star / (4σ(r_planet + r_stellar))². Or is there another factor I'm missing?
+    
+    Wait, I might have confused the formula. Another possibility is that the formula for equilibrium temperature includes the distance from the star (d) and the planet's radius (R_planet), so T^4 = (L_star * R_planet ) / (4σ d^2 ). In that case, the ratio T4/T2 = (R4/R2) * (d2/d4 ). So the ratio of temperatures is product of the ratio of their radii and the ratio of their distances.
+    
+    If all five planets have the same al
+    ```