Update Automated MNLP evaluation report (2026-05-18)
Browse files- EVAL_REPORT.md +51 -8
EVAL_REPORT.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
- **Model repo:** [`cs-552-2026-TopHaylin/multilingual_model`](https://huggingface.co/cs-552-2026-TopHaylin/multilingual_model)
|
| 4 |
- **Owner(s):** group **TopHaylin**
|
| 5 |
-
- **Generated at:** 2026-05-
|
| 6 |
- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
|
| 7 |
|
| 8 |
_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
|
|
@@ -13,15 +13,58 @@ _This PR is opened automatically by the course CI. It is **non-blocking** — yo
|
|
| 13 |
|---|---:|---|
|
| 14 |
| Math | — | not run |
|
| 15 |
| Knowledge | — | not run |
|
| 16 |
-
| Multilingual |
|
| 17 |
| Safety | — | not run |
|
| 18 |
|
| 19 |
-
##
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
| 24 |
-
generation_config.json not found: 404 Client Error. (Request ID: Root=1-6a093b3c-5709eb6d42f75ce025122c27;476b9b74-8134-43c2-9fe7-0c19d4b3cb83)
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
- **Model repo:** [`cs-552-2026-TopHaylin/multilingual_model`](https://huggingface.co/cs-552-2026-TopHaylin/multilingual_model)
|
| 4 |
- **Owner(s):** group **TopHaylin**
|
| 5 |
+
- **Generated at:** 2026-05-18T02:30:25+00:00 (UTC)
|
| 6 |
- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
|
| 7 |
|
| 8 |
_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
|
|
|
|
| 13 |
|---|---:|---|
|
| 14 |
| Math | — | not run |
|
| 15 |
| Knowledge | — | not run |
|
| 16 |
+
| Multilingual | 0.4600 | ok |
|
| 17 |
| Safety | — | not run |
|
| 18 |
|
| 19 |
+
## Sample completions
|
| 20 |
|
| 21 |
+
_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
|
| 22 |
|
| 23 |
+
### Multilingual
|
|
|
|
| 24 |
|
| 25 |
+
**Correct** (1 shown)
|
| 26 |
+
|
| 27 |
+
- **reference**: `C`
|
| 28 |
+
- **overall** (1/1 completions correct)
|
| 29 |
+
- **extracted** (✓): `C`
|
| 30 |
+
- **completion**:
|
| 31 |
+
|
| 32 |
+
```text
|
| 33 |
+
我们考虑所有可能的掷骰子组合。Juan 掷一个八面骰子,Amal 掷一个六面骰子,总共有 $8 \times 6 = 48$ 种可能的组合。
|
| 34 |
+
|
| 35 |
+
我们要求掷出的数字的乘积是 3 的倍数。也就是说,$3 \mid (a \cdot b)$,其中 $a$ 是 Juan 掷出的数字,$b$ 是 Amal 掷出的数字。
|
| 36 |
+
|
| 37 |
+
$3 \mid (a \cdot b)$ 的等价条件是 $3 \mid a$ 或 $3 \mid b$,因为 3 是质数。
|
| 38 |
+
|
| 39 |
+
我们计算 $3 \mid a$ 或 $3 \mid b$ 的组合数:
|
| 40 |
+
|
| 41 |
+
- $3 \mid a$:Juan 掷出的数字为 3 的倍数,有 2 种可能(3 或 6)。
|
| 42 |
+
- $3 \mid b$:Amal 掷出的数字为 3 的倍数,有 2 种可能(3 或 6)。
|
| 43 |
+
|
| 44 |
+
但注意 $3 \mid a$ 和 $3 \mid b$ 是互斥事件,所以它们的组合数不能重复计数。
|
| 45 |
+
|
| 46 |
+
因此,满足 $3 \mid (a \cdot b)$ 的组合数为:
|
| 47 |
+
|
| 48 |
+
$$
|
| 49 |
+
2 \times 2 = 4
|
| 50 |
+
$$
|
| 51 |
+
|
| 52 |
+
因此,满足条件的概率为:
|
| 53 |
+
|
| 54 |
+
$$
|
| 55 |
+
\frac{4}{48} = \frac{1}{12}
|
| 56 |
+
$$
|
| 57 |
+
|
| 58 |
+
### 答案:$\boxed{C}$
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
**Incorrect** (1 shown)
|
| 62 |
+
|
| 63 |
+
- **reference**: `A`
|
| 64 |
+
- **overall** (0/1 completions correct)
|
| 65 |
+
- **extracted** (✗): `<no answer>`
|
| 66 |
+
- **completion**:
|
| 67 |
+
|
| 68 |
+
```text
|
| 69 |
+
答案:B。
|
| 70 |
+
```
|