zechen-nlp commited on
Commit
c6d8022
·
verified ·
1 Parent(s): dd186ac

Update Automated MNLP evaluation report (2026-05-18)

Browse files
Files changed (1) hide show
  1. EVAL_REPORT.md +51 -8
EVAL_REPORT.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  - **Model repo:** [`cs-552-2026-TopHaylin/multilingual_model`](https://huggingface.co/cs-552-2026-TopHaylin/multilingual_model)
4
  - **Owner(s):** group **TopHaylin**
5
- - **Generated at:** 2026-05-17T04:52:19+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
@@ -13,15 +13,58 @@ _This PR is opened automatically by the course CI. It is **non-blocking** — yo
13
  |---|---:|---|
14
  | Math | — | not run |
15
  | Knowledge | — | not run |
16
- | Multilingual | | error (generation_config) |
17
  | Safety | — | not run |
18
 
19
- ## Errors
20
 
21
- ### multilingual `generation_config`
22
 
23
- ```
24
- generation_config.json not found: 404 Client Error. (Request ID: Root=1-6a093b3c-5709eb6d42f75ce025122c27;476b9b74-8134-43c2-9fe7-0c19d4b3cb83)
25
 
26
- Entry Not Found for url: https://huggingface.co/cs-552-2026-TopHaylin/multilingual_model/resolve/main/generation_config.json.
27
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
  - **Model repo:** [`cs-552-2026-TopHaylin/multilingual_model`](https://huggingface.co/cs-552-2026-TopHaylin/multilingual_model)
4
  - **Owner(s):** group **TopHaylin**
5
+ - **Generated at:** 2026-05-18T02:30:25+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
 
13
  |---|---:|---|
14
  | Math | — | not run |
15
  | Knowledge | — | not run |
16
+ | Multilingual | 0.4600 | ok |
17
  | Safety | — | not run |
18
 
19
+ ## Sample completions
20
 
21
+ _Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
22
 
23
+ ### Multilingual
 
24
 
25
+ **Correct** (1 shown)
26
+
27
+ - **reference**: `C`
28
+ - **overall** (1/1 completions correct)
29
+ - **extracted** (✓): `C`
30
+ - **completion**:
31
+
32
+ ```text
33
+ 我们考虑所有可能的掷骰子组合。Juan 掷一个八面骰子,Amal 掷一个六面骰子,总共有 $8 \times 6 = 48$ 种可能的组合。
34
+
35
+ 我们要求掷出的数字的乘积是 3 的倍数。也就是说,$3 \mid (a \cdot b)$,其中 $a$ 是 Juan 掷出的数字,$b$ 是 Amal 掷出的数字。
36
+
37
+ $3 \mid (a \cdot b)$ 的等价条件是 $3 \mid a$ 或 $3 \mid b$,因为 3 是质数。
38
+
39
+ 我们计算 $3 \mid a$ 或 $3 \mid b$ 的组合数:
40
+
41
+ - $3 \mid a$:Juan 掷出的数字为 3 的倍数,有 2 种可能(3 或 6)。
42
+ - $3 \mid b$:Amal 掷出的数字为 3 的倍数,有 2 种可能(3 或 6)。
43
+
44
+ 但注意 $3 \mid a$ 和 $3 \mid b$ 是互斥事件,所以它们的组合数不能重复计数。
45
+
46
+ 因此,满足 $3 \mid (a \cdot b)$ 的组合数为:
47
+
48
+ $$
49
+ 2 \times 2 = 4
50
+ $$
51
+
52
+ 因此,满足条件的概率为:
53
+
54
+ $$
55
+ \frac{4}{48} = \frac{1}{12}
56
+ $$
57
+
58
+ ### 答案:$\boxed{C}$
59
+ ```
60
+
61
+ **Incorrect** (1 shown)
62
+
63
+ - **reference**: `A`
64
+ - **overall** (0/1 completions correct)
65
+ - **extracted** (✗): `<no answer>`
66
+ - **completion**:
67
+
68
+ ```text
69
+ 答案:B。
70
+ ```