zechen-nlp commited on
Commit
0e1115f
Β·
verified Β·
1 Parent(s): 3b7999b

Update Automated MNLP evaluation report (2026-05-20)

Browse files
Files changed (1) hide show
  1. EVAL_REPORT.md +6 -6
EVAL_REPORT.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  - **Model repo:** [`cs-552-2026-kth/general_knowledge_model`](https://huggingface.co/cs-552-2026-kth/general_knowledge_model)
4
  - **Owner(s):** group **kth**
5
- - **Generated at:** 2026-05-19T03:22:52+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
@@ -12,7 +12,7 @@ _This PR is opened automatically by the course CI. It is **non-blocking** β€” yo
12
  | Benchmark | Accuracy | Status |
13
  |---|---:|---|
14
  | Math | β€” | not run |
15
- | Knowledge | 0.3700 | ok |
16
  | Multilingual | β€” | not run |
17
  | Safety | β€” | not run |
18
 
@@ -30,16 +30,16 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
30
  - **completion**:
31
 
32
  ```text
33
- Answer: \boxed{C}
34
  ```
35
 
36
  **Incorrect** (1 shown)
37
 
38
- - **reference**: `C`
39
  - **overall** (0/1 completions correct)
40
- - **extracted** (βœ—): `D`
41
  - **completion**:
42
 
43
  ```text
44
- \boxed{D}
45
  ```
 
2
 
3
  - **Model repo:** [`cs-552-2026-kth/general_knowledge_model`](https://huggingface.co/cs-552-2026-kth/general_knowledge_model)
4
  - **Owner(s):** group **kth**
5
+ - **Generated at:** 2026-05-20T05:49:46+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
 
12
  | Benchmark | Accuracy | Status |
13
  |---|---:|---|
14
  | Math | β€” | not run |
15
+ | Knowledge | 0.3500 | ok |
16
  | Multilingual | β€” | not run |
17
  | Safety | β€” | not run |
18
 
 
30
  - **completion**:
31
 
32
  ```text
33
+ \boxed{C}
34
  ```
35
 
36
  **Incorrect** (1 shown)
37
 
38
+ - **reference**: `B`
39
  - **overall** (0/1 completions correct)
40
+ - **extracted** (βœ—): `A`
41
  - **completion**:
42
 
43
  ```text
44
+ \boxed{A}
45
  ```