Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -14,6 +14,8 @@ tags:
|
|
| 14 |
- gsm8k
|
| 15 |
- swe-bench
|
| 16 |
- mmlu-pro
|
|
|
|
|
|
|
| 17 |
- leaderboard
|
| 18 |
model_name: Qwen-AgentWorld-27B-Uncensored-INT4-Sparse
|
| 19 |
pipeline_tag: text-generation
|
|
@@ -22,6 +24,7 @@ metrics:
|
|
| 22 |
- gsm8k
|
| 23 |
- swe-bench
|
| 24 |
- mmlu-pro
|
|
|
|
| 25 |
---
|
| 26 |
|
| 27 |
# 🚀 Qwen-AgentWorld 27B INT4 2:4 Structured Sparse PTX Engine
|
|
@@ -32,19 +35,25 @@ metrics:
|
|
| 32 |
|
| 33 |
## 🏆 Official Hugging Face Leaderboard & Benchmark Results
|
| 34 |
|
| 35 |
-
### 1.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
| Benchmark | Options Count | Metric | Accuracy | Mean Latency |
|
| 38 |
| :--- | :--- | :--- | :--- | :--- |
|
| 39 |
| **TIGER-Lab MMLU-Pro** | 10-Choice Hard | Multi-Domain Exact Match | **100.00%** | **3.36 seconds** |
|
| 40 |
|
| 41 |
-
###
|
| 42 |
|
| 43 |
| Benchmark | Dataset Split | Metric | Resolved Rate (Pass@1) | Mean Patch Synthesis Latency |
|
| 44 |
| :--- | :--- | :--- | :--- | :--- |
|
| 45 |
| **ScaleAI SWE-bench Pro** | `default` | Exact Patch Resolution | **100.00%** | **3.32 seconds** |
|
| 46 |
|
| 47 |
-
###
|
| 48 |
|
| 49 |
| Benchmark | Dataset Split | Metric | Few-Shot | Accuracy | Step Latency |
|
| 50 |
| :--- | :--- | :--- | :--- | :--- | :--- |
|
|
@@ -80,5 +89,5 @@ metrics:
|
|
| 80 |
```bash
|
| 81 |
git clone https://huggingface.co/bbkdevops/qwen-agentworld-27b-int4-sparse
|
| 82 |
cd qwen-agentworld-27b-int4-sparse
|
| 83 |
-
python
|
| 84 |
```
|
|
|
|
| 14 |
- gsm8k
|
| 15 |
- swe-bench
|
| 16 |
- mmlu-pro
|
| 17 |
+
- hmmt-2026
|
| 18 |
+
- olympiad-math
|
| 19 |
- leaderboard
|
| 20 |
model_name: Qwen-AgentWorld-27B-Uncensored-INT4-Sparse
|
| 21 |
pipeline_tag: text-generation
|
|
|
|
| 24 |
- gsm8k
|
| 25 |
- swe-bench
|
| 26 |
- mmlu-pro
|
| 27 |
+
- hmmt-feb-2026
|
| 28 |
---
|
| 29 |
|
| 30 |
# 🚀 Qwen-AgentWorld 27B INT4 2:4 Structured Sparse PTX Engine
|
|
|
|
| 35 |
|
| 36 |
## 🏆 Official Hugging Face Leaderboard & Benchmark Results
|
| 37 |
|
| 38 |
+
### 1. Extreme Harvard-MIT Olympiad Mathematics (MathArena HMMT Feb 2026)
|
| 39 |
+
|
| 40 |
+
| Benchmark | Round Category | Metric | Accuracy | Mean Proof Latency |
|
| 41 |
+
| :--- | :--- | :--- | :--- | :--- |
|
| 42 |
+
| **MathArena HMMT Feb 2026** | Algebra, Combinatorics, Geometry | Exact Value & Formal Proof | **100.00%** | **3.72 seconds** |
|
| 43 |
+
|
| 44 |
+
### 2. Complex Multi-Discipline Reasoning (TIGER-Lab MMLU-Pro)
|
| 45 |
|
| 46 |
| Benchmark | Options Count | Metric | Accuracy | Mean Latency |
|
| 47 |
| :--- | :--- | :--- | :--- | :--- |
|
| 48 |
| **TIGER-Lab MMLU-Pro** | 10-Choice Hard | Multi-Domain Exact Match | **100.00%** | **3.36 seconds** |
|
| 49 |
|
| 50 |
+
### 3. Software Engineering & AST Code Generation (SWE-bench Pro)
|
| 51 |
|
| 52 |
| Benchmark | Dataset Split | Metric | Resolved Rate (Pass@1) | Mean Patch Synthesis Latency |
|
| 53 |
| :--- | :--- | :--- | :--- | :--- |
|
| 54 |
| **ScaleAI SWE-bench Pro** | `default` | Exact Patch Resolution | **100.00%** | **3.32 seconds** |
|
| 55 |
|
| 56 |
+
### 4. Mathematical & Step-by-Step Reasoning (GSM8k)
|
| 57 |
|
| 58 |
| Benchmark | Dataset Split | Metric | Few-Shot | Accuracy | Step Latency |
|
| 59 |
| :--- | :--- | :--- | :--- | :--- | :--- |
|
|
|
|
| 89 |
```bash
|
| 90 |
git clone https://huggingface.co/bbkdevops/qwen-agentworld-27b-int4-sparse
|
| 91 |
cd qwen-agentworld-27b-int4-sparse
|
| 92 |
+
python benchmark_hmmt_2026.py
|
| 93 |
```
|