Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,81 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-centralesupechec/general_knowledge_model`](https://huggingface.co/cs-552-2026-centralesupechec/general_knowledge_model)
+- **Owner(s):** group **centralesupechec**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`ec123fa`](https://huggingface.co/cs-552-2026-centralesupechec/general_knowledge_model/commit/ec123fa81a27f340718fba34f371ceb990ac19b0)
+- **Message:** sft_v3 ours_v2 max_new_tokens=16384
+- **Committed:** 2026-06-09T16:28:58+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.4200 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `I`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `I`
+- **completion**:
+
+    ```text
+    Okay, let's try to figure out this question. The question is about Archaic Greek statues that are rigid, facing forward with clenched fists, similar to ancient Egyptian statuary. The options are various names like Atlantes, Gorgons, Minotaurs, etc.
+    
+    First, I need to recall what I know about Archaic Greek art. The Archaic period was around 700-480 BCE. They had a lot of statues, especially of male
+    
+    ... [truncated 823 chars] ...
+    
+    ntions clenched fists. Hmm. Maybe the answer is Kouroi, even if the description is slightly different. Or maybe there's another term.
+    
+    Wait, another possibility: the Discoboloi are sometimes called "Discoboloi" because they are in the act of discus throwing. But their pose is not clenched fists. The Kouroi are the ones with arms crossed. Maybe the answer is Kouroi. Let me check the options again. The choices include I. Kouroi. So maybe the answer is I. Kouroi. But the question says "clenched fists." Wait, maybe the Kouroi are depicted with arms crossed, which is similar to clenched fists? Or maybe the answer is different.
+    
+    Alternatively, maybe the answer is the Atlantes, but I don't think they have that pose. Or maybe the answer is the Gorgons, but they are female and not in a rigid pose. 
+    
+    Wait, another thought: in the Archaic period, there were statues of male figures, like the Kouroi, which are standing with arms crossed, and they are often considered to be in a rigid, formal pose. The question mentions "clenched fists," which might be a specific detail. But maybe the answer is Kouroi. Because that's the most likely option. So I think the answer is I. Kouroi.
+    </think>
+    
+    \boxed{I}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `B`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `D`
+- **completion**:
+
+    ```text
+    Okay, let's try to figure out this problem. So, the question is about the interaction of a high-energy gamma ray with a CMB photon, leading to the creation of an electron-positron pair. We need to find the minimum gamma-ray energy required so that this process limits the gamma-ray lifetime in the universe. The average photon energy in the CMB is given as 10^-3 eV.
+    
+    First, I remember that when a ph
+    
+    ... [truncated 822 chars] ...
+    
+    is 1.022e-3 GeV. But since none of the options are that, I must have made a mistake.
+    
+    Alternatively, maybe the question is about the energy required for the process to occur, but the answer is 1.022 MeV, which is 1.022e-3 GeV, and the options are in GeV, so the answer is 1.022e-3 GeV, which is 1.022e-3 GeV. But since none of the options are that, I must conclude that there's a mistake in the problem.
+    
+    But since I have to choose from the options, and the closest one is D. 1.8*1e5 GeV, which is 180,000 GeV, which is much higher than 1.022 MeV, I think there must be a mistake in the problem. However, given that the options are provided, and the correct answer is likely D, but I'm not sure. Alternatively, maybe the question is about the energy required for the process to occur, and the answer is 1.022 MeV, which is 1.022e-3 GeV, and the options are in GeV, so the answer is 1.022e-3 GeV, which is 1.022e-3 GeV. But since none of the options are that, I must choose the closest one, which is D. 1.8*1e5 GeV.
+    
+    But this is not correct. I think there's a mistake in the problem. However, given that the options are provided, and the correct answer is likely D, I'll choose D.
+    </think>
+    
+    \boxed{D}
+    ```