Automated MNLP evaluation report (2026-06-04)

EVAL_REPORT.md

@@ -0,0 +1,75 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-ChatMODS/multilingual_model`](https://huggingface.co/cs-552-2026-ChatMODS/multilingual_model)
+- **Owner(s):** group **ChatMODS**
+- **Generated at:** 2026-06-04T19:45:28+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | — | not run |
+| Multilingual | 0.4900 | ok |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Multilingual
+
+**Correct** (1 shown)
+
+- **reference**: `C`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `C`
+- **completion**:
+
+    ```text
+    <think>
+    \boxed{C}
+    </think>
+    
+    \boxed{C}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `B`
+- **completion**:
+
+    ```text
+    <think>
+    Okay, let's tackle this question about the taxation of investment instruments in managed funds in Italy. The question is asking about the tax treatment for a resident individual, not in business, when investing in a fund of funds (fondi comuni di investimento mobiliari) in Italy.
+    
+    First, I need to recall the Italian tax laws related to investment funds. In Italy, there are different types
+    
+    ... [truncated 823 chars] ...
+    
+    to separate taxation with a 15% rate (which is not standard, as the standard rate is 24% for most income)
+    
+    So, the correct answer would be B, as the proventi from the investment in a fund are taxable in the income tax declaration, and the tax rate is 24%. However, I need to check if there are any specific rules for the funds in question.
+    
+    Alternatively, if the fund is a "fondo comune di investimento" and the investor is a resident, the proventi might be subject to the 24% tax rate. Therefore, the correct answer would be B, as the proventi are imposable in the income tax declaration, and the tax rate is 24%. However, the options given are different, so I need to check again.
+    
+    Wait, the options are:
+    
+    A. Subject to 26% under the cash basis principle (which is not correct, as the standard rate is 24%)
+    
+    B. Imposable in the income tax declaration (which is correct, as the proventi are taxable)
+    
+    C. Subject to 12.5% withholding tax (which might be the case for certain types of funds, but not sure)
+    
+    D. Subject to 15% separate taxation (which is not standard)
+    
+    Therefore, the correct answer is B, as the proventi are imposable in the income tax declaration at a rate of 24%.
+    </think>
+    
+    \boxed{B}
+    ```