File size: 1,251 Bytes
cc8e625 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | ---
title: SOV33 Benchmark Runner
emoji: 🧪
colorFrom: indigo
colorTo: purple
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: SOV33 general + agentic benchmark on HuggingFace
---
# SOV33 Benchmark Runner — HuggingFace Space
Hosts the SOV33 benchmark harness as a Gradio app. Runs general capability
(MMLU-Pro, GSM8K, AIME, HellaSwag, ARC-C, HumanEval, TruthfulQA) and agentic
(GAIA-lite, tau-bench-retail, ALFWorld-text, HotpotQA, SWE-bench-lite) suites
against any uploaded model.
## Use
1. Push the model you want to evaluate to the Hub (or use one of the suggested)
2. Open the Space → paste model id → pick suite → Run
3. Results download as signed JSON; sigil chain stored in `/data/sigil_chain.jsonl`
## Files in this Space
- `sov33_benchmark_general.py` — general capability suite
- `sov33_benchmark_agentic.py` — agentic capability suite
- `harness_loader.py` — HF transformers wrapper
- `app.py` — Gradio UI
## Suggested models
- `Qwen/Qwen2.5-3B-Instruct` (3B, fits T4 small)
- `Qwen/Qwen3-30B-A3B` (MoE 30B/A3B, fits A100 80GB)
- `meta-llama/Meta-Llama-3-8B-Instruct`
- `mistralai/Mistral-7B-Instruct-v0.3`
## Credentials
Set `HF_TOKEN` as a Space secret to enable private models. |