Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,85 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-ma-que/general_knowledge_model`](https://huggingface.co/cs-552-2026-ma-que/general_knowledge_model)
+- **Owner(s):** group **ma-que**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`ea02e87`](https://huggingface.co/cs-552-2026-ma-que/general_knowledge_model/commit/ea02e87444e4cbb2130a13dcff064cab2582098f)
+- **Message:** Remove model card for repo consistency
+- **Committed:** 2026-06-10T19:04:19+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.3900 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `G`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `G`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's try to figure out this question. So, the scenario is a 3-year-old girl in preschool who, on her first day, is holding her mom's hand and watching other kids play. Then, after some encouragement, she starts helping another child with a puzzle. The question is asking which temperament style in the Thomas and Chess model this child has.
+    
+    First, I need to recall what the Thomas and
+    
+    ... [truncated 822 chars] ...
+    
+    interacting much. Then, after encouragement, she helps. So she's a 3-year-old who is a slow to warm up (G), which is a type of difficult temperament. So the answer is G. But the options are labeled as A-J, so the answer is G. Alternatively, maybe the answer is I (Difficult) because she's a difficult child. But the slow to warm up is a subpoint. So the answer is G. But I'm not entirely sure. Alternatively, maybe the answer is H (Passive), but that's more withdrawal. The scenario says she starts helping after encouragement, so maybe she's adapting. So the answer is G (Slow to warm up) as the main style. I think that's the most likely answer.
+    </think>
+    
+    <answer>The 3-year-old girl in the scenario is described as holding her mother's hand and observing others before starting to help a child after encouragement. This suggests she is initially hesitant to engage with the environment, indicating a slow-to-warm-up temperament. In the Thomas and Chess model, the slow-to-warm-up subtype is a type of difficult temperament. While passive (H) is another difficult subpoint, the scenario highlights her eventual adaptation, aligning more closely with the slow-to-warm-up style. 
+    
+    \boxed{G}</answer>
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `I`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `B`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's tackle this question step by step. The question is about executory contracts and which statement is the least correct. The options are A to J, and I need to figure out which one is incorrect.
+    
+    First, I need to recall what an executory contract is. From what I remember, an executory contract is one where one or both parties still have obligations to perform. So, there's no compl
+    
+    ... [truncated 823 chars] ...
+    
+    ully executed contract exists when neither party has remaining obligations.  
+    - **B.** Incorrect. If one party has completed performance and the other has not begun, the contract is **completed**, not executory.  
+    - **C.** Correct. An executory contract is when one party has performed and the other has not yet begun.  
+    - **D.** Correct. A wholly executory contract involves obligations exchanged without performance.  
+    - **E.** Correct. Partially executed means at least one party has begun performance.  
+    - **F.** Correct. Executory contract when both parties have yet to begin.  
+    - **G.** Correct. Fully executed when both parties have completed performance.  
+    - **H.** Correct. Partially executed when both have begun but not completed.  
+    - **I.** Incorrect. If one party has completed performance, the contract is done, not executory.  
+    - **J.** Correct. Executed when both have fully performed.  
+    
+    **Key Analysis**:  
+    - **B** incorrectly describes a completed contract, not an executory one.  
+    - **I** also incorrectly describes a completed contract. However, **B** is the least correct because it directly contradicts the definition of an executory contract.  
+    
+    **Answer**: \boxed{B}</answer>
+    ```