Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,84 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-momy/multilingual_model`](https://huggingface.co/cs-552-2026-momy/multilingual_model)
+- **Owner(s):** group **momy**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`bd84ee8`](https://huggingface.co/cs-552-2026-momy/multilingual_model/commit/bd84ee812a82bea10ae4286aa413c3c505f26e75)
+- **Message:** Upload folder using huggingface_hub
+- **Committed:** 2026-06-09T10:00:35+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | — | not run |
+| Multilingual | 0.5700 | ok |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Multilingual
+
+**Correct** (1 shown)
+
+- **reference**: `B`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `B`
+- **completion**:
+
+    ```text
+    Okay, let's see. The question is asking which of the given options is not a lesson (урок) of control of knowledge, skills, and abilities. The choices are A. лабораторная работа (laboratory work), B. суггестивный (suggestive), C. сочинение (composition), D. компьютерный (computer).
+    
+    First, I need to recall what each of these terms means in the context of education. 
+    
+    Laboratory work (A) is typicall
+    
+    ... [truncated 708 chars] ...
+    
+    рный" refers to a computer-based lesson or activity. These can be assessments, like using software to demonstrate skills, so that's a control lesson.
+    
+    So the question is which one is not a control lesson. If B is a teaching method rather than a control lesson, then B would be the answer. But I need to confirm. 
+    
+    In Russian education, "suggestive" (суггестивный) might be a term used for a specific teaching approach, but not necessarily a lesson that controls knowledge. Alternatively, maybe it's a type of assessment. But I'm not sure. Alternatively, maybe "suggestive" is a method, not a lesson type. 
+    
+    Alternatively, maybe "computer" (D) is not a lesson type but a method. But the options are all lesson types. Wait, the question says "урок" which is a lesson. So the options are types of lessons. 
+    
+    So, if "suggestive" is a method, not a lesson type, then B would be the answer. But I need to check. 
+    
+    Alternatively, maybe "suggestive" is a type of lesson. But I'm not sure. Maybe in Russian education, "suggestive" is a method, not a lesson. Therefore, the answer is B. 
+    
+    Alternatively, maybe "computer" (D) is a lesson type. But I think the answer is B. So the answer is B.
+    </think>
+    \boxed{B}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `C`
+- **completion**:
+
+    ```text
+    Okay, let's try to figure out this question. So, the user is asking which Supreme Court decision contradicted William Lloyd Garrison's argument from his 1831 speech. The passage mentions that Garrison believes people of color, who are free and naturalized in the US, should have the same rights as citizens, and that the Constitution might be unfair in distinguishing based on color. He argues that t
+    
+    ... [truncated 822 chars] ...
+    
+    I think the correct answer is C. Because the question is about the argument that the Constitution is unfair in distinguishing based on color, and the case of Plessy (1896) was a decision that upheld segregation, which was a contradiction. So the answer is C.
+    </think>
+    La respuesta correcta es **C. Caso Plessy contra Ferguson (1896)**. 
+    
+    **Explicación:**  
+    William Lloyd Garrison argumentaba que la Constitución de los Estados Unidos no reconocía los derechos iguales a las personas de color, y que el trato injusto hacia ellos era contrario a la libertad. El **Caso Plessy contra Ferguson (1896)**, aunque inicialmente permitió que los blancos y negros se separaran en lugar de ser iguales, fue una decisión que **afirmó la separación pero no la igualdad**, lo que contradecía el argumento de Garrison de que los derechos de las personas de color deberían ser reconocidos y garantizados. Aunque el caso de **Dred Scott (1857)** también fue un punto de conflicto, el **Caso Plessy** es más directamente contrario a la idea de igualdad que el pasaje de Garrison, ya que su decisión fue un paso hacia la segregación, lo que contradecía su visión de justicia y equidad. 
+    
+    **Respuesta final:** \boxed{C}
+    ```