Automated MNLP evaluation report (2026-06-04)

EVAL_REPORT.md

@@ -0,0 +1,87 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-barn/multilingual_model`](https://huggingface.co/cs-552-2026-barn/multilingual_model)
+- **Owner(s):** group **barn**
+- **Generated at:** 2026-06-04T19:45:28+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | — | not run |
+| Multilingual | 0.2300 | ok |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Multilingual
+
+**Correct** (1 shown)
+
+- **reference**: `A`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `A`
+- **completion**:
+
+    ```text
+    <think>
+    
+    </think>
+    
+    A. Квалиметрической шкалой
+    
+    **Обоснование:**  
+    Тройка формальных объектов — это концепция, которая используется в квалиметрии (науке о количественных характеристиках). Она описывает три основные ступени или уровни, на которых можно оценивать объекты, обычно в рамках шкалирования. Эти ступени обычно включают:  
+    1. **Нулевая** (или **нулевая**),  
+    2. **Положительная**,  
+    3. **Негативная**.  
+    
+    Эта тройка формальных объектов называется **квалиметрической шкалой**.  
+    
+    Таким образом, правильный ответ:  
+    $\boxed{A}$
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `C`
+- **completion**:
+
+    ```text
+    <think>
+    
+    </think>
+    
+    The most likely cause of the patient's condition is **C. जन्मजात महाधमनी स्टेनोसिस (Congenital Patent Ductus Arteriosus, PDA)**.
+    
+    ### Explanation:
+    
+    - The patient is **32 weeks pregnant** and has **preeclampsia** (high blood pressure and proteinuria), which is a **high-risk pregnancy**.
+    - She has **tachycardia (132 bpm)**, **tachypnea (32 bpm)**, and **tachypnea (32 bpm)**, which
+    
+    ... [truncated 453 chars] ...
+    
+    vated**, and **tachycardia** is a common finding in **preeclampsia** and **cardiomegaly**.
+    - The **pulsation** and **oxygen saturation** are **normal**, but the **tachycardia and tachypnea** are consistent with **cardiomegaly** and **preeclampsia**.
+    - The **heart sounds** and **heart rate** suggest **increased cardiac workload**, which is commonly seen in **preeclampsia** and **congenital heart disease**.
+    
+    ### Key clues:
+    - The **tachycardia** and **tachypnea** are **not** due to **respiratory distress** (e.g., asthma, COPD), but rather due to **cardiac disease** and **preeclampsia**.
+    - The **tachycardia** is **not** due to **anemia** or **hypothyroidism** (the patient is **not** hypothyroid, and **thyroxine** is being used).
+    - The **heart rate** is **not** due to **tachypnea** or **respiratory distress**, but due to **cardiac workload** and **preeclampsia**.
+    
+    ### Conclusion:
+    The most likely **cause of the patient's condition** is **C. जन्मजात महाधमनी स्टेनोसिस (Congenital Patent Ductus Arteriosus)**, which is a **congenital heart defect** that can lead to **increased cardiac workload**, **tachycardia**, and **tachypnea** in **pregnant women** with **preeclampsia**.
+    
+    $$
+    \boxed{C}
+    $$
+    ```