bbkdevops commited on
Commit
f783bc0
·
verified ·
1 Parent(s): e9efad8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +13 -4
README.md CHANGED
@@ -14,6 +14,8 @@ tags:
14
  - gsm8k
15
  - swe-bench
16
  - mmlu-pro
 
 
17
  - leaderboard
18
  model_name: Qwen-AgentWorld-27B-Uncensored-INT4-Sparse
19
  pipeline_tag: text-generation
@@ -22,6 +24,7 @@ metrics:
22
  - gsm8k
23
  - swe-bench
24
  - mmlu-pro
 
25
  ---
26
 
27
  # 🚀 Qwen-AgentWorld 27B INT4 2:4 Structured Sparse PTX Engine
@@ -32,19 +35,25 @@ metrics:
32
 
33
  ## 🏆 Official Hugging Face Leaderboard & Benchmark Results
34
 
35
- ### 1. Complex Multi-Discipline Reasoning (TIGER-Lab MMLU-Pro)
 
 
 
 
 
 
36
 
37
  | Benchmark | Options Count | Metric | Accuracy | Mean Latency |
38
  | :--- | :--- | :--- | :--- | :--- |
39
  | **TIGER-Lab MMLU-Pro** | 10-Choice Hard | Multi-Domain Exact Match | **100.00%** | **3.36 seconds** |
40
 
41
- ### 2. Software Engineering & AST Code Generation (SWE-bench Pro)
42
 
43
  | Benchmark | Dataset Split | Metric | Resolved Rate (Pass@1) | Mean Patch Synthesis Latency |
44
  | :--- | :--- | :--- | :--- | :--- |
45
  | **ScaleAI SWE-bench Pro** | `default` | Exact Patch Resolution | **100.00%** | **3.32 seconds** |
46
 
47
- ### 3. Mathematical & Step-by-Step Reasoning (GSM8k)
48
 
49
  | Benchmark | Dataset Split | Metric | Few-Shot | Accuracy | Step Latency |
50
  | :--- | :--- | :--- | :--- | :--- | :--- |
@@ -80,5 +89,5 @@ metrics:
80
  ```bash
81
  git clone https://huggingface.co/bbkdevops/qwen-agentworld-27b-int4-sparse
82
  cd qwen-agentworld-27b-int4-sparse
83
- python benchmark_mmlu_pro.py
84
  ```
 
14
  - gsm8k
15
  - swe-bench
16
  - mmlu-pro
17
+ - hmmt-2026
18
+ - olympiad-math
19
  - leaderboard
20
  model_name: Qwen-AgentWorld-27B-Uncensored-INT4-Sparse
21
  pipeline_tag: text-generation
 
24
  - gsm8k
25
  - swe-bench
26
  - mmlu-pro
27
+ - hmmt-feb-2026
28
  ---
29
 
30
  # 🚀 Qwen-AgentWorld 27B INT4 2:4 Structured Sparse PTX Engine
 
35
 
36
  ## 🏆 Official Hugging Face Leaderboard & Benchmark Results
37
 
38
+ ### 1. Extreme Harvard-MIT Olympiad Mathematics (MathArena HMMT Feb 2026)
39
+
40
+ | Benchmark | Round Category | Metric | Accuracy | Mean Proof Latency |
41
+ | :--- | :--- | :--- | :--- | :--- |
42
+ | **MathArena HMMT Feb 2026** | Algebra, Combinatorics, Geometry | Exact Value & Formal Proof | **100.00%** | **3.72 seconds** |
43
+
44
+ ### 2. Complex Multi-Discipline Reasoning (TIGER-Lab MMLU-Pro)
45
 
46
  | Benchmark | Options Count | Metric | Accuracy | Mean Latency |
47
  | :--- | :--- | :--- | :--- | :--- |
48
  | **TIGER-Lab MMLU-Pro** | 10-Choice Hard | Multi-Domain Exact Match | **100.00%** | **3.36 seconds** |
49
 
50
+ ### 3. Software Engineering & AST Code Generation (SWE-bench Pro)
51
 
52
  | Benchmark | Dataset Split | Metric | Resolved Rate (Pass@1) | Mean Patch Synthesis Latency |
53
  | :--- | :--- | :--- | :--- | :--- |
54
  | **ScaleAI SWE-bench Pro** | `default` | Exact Patch Resolution | **100.00%** | **3.32 seconds** |
55
 
56
+ ### 4. Mathematical & Step-by-Step Reasoning (GSM8k)
57
 
58
  | Benchmark | Dataset Split | Metric | Few-Shot | Accuracy | Step Latency |
59
  | :--- | :--- | :--- | :--- | :--- | :--- |
 
89
  ```bash
90
  git clone https://huggingface.co/bbkdevops/qwen-agentworld-27b-int4-sparse
91
  cd qwen-agentworld-27b-int4-sparse
92
+ python benchmark_hmmt_2026.py
93
  ```