README.md

metadata

title: SLM Arena
emoji: 🏟️
colorFrom: yellow
colorTo: blue
sdk: gradio
sdk_version: 6.19.0
app_file: app.py
pinned: false
hf_oauth: true
hf_oauth_scopes:
  - inference-api
license: apache-2.0
short_description: Compact model arena with GLM and GPT OSS commentary
models:
  - HuggingFaceTB/SmolLM2-135M
  - MaliosDark/Isabel-50M
  - AxiomicLabs/GPT-X2-125M
  - joelhenwang/OdinNext-138M-Base
  - UniversalComputingResearch/Atom2.7m
  - SupraLabs/Supra-1.5-50M-base-exp
  - fromziro/Er-Tiny-1.3M
  - fromziro/Er-Medium-12.5M
  - veyra-ai/Veyra2-Apricot-50M-Base
  - veyra-ai/Veyra2-30M-Base
  - veyra-ai/Veyra2-15M-Base
  - Harley-ml/Dillionv2-1.3M
  - User01110/tinyLM-8M-exp-256
  - Glint-Research/Glint-1.3
  - AtomixLabs/AtomixS2-5M-v1.0
  - BananaMind/MiniBananaMind-v4-9M
  - MihaiPopa-1/CinnabarLM-1.4M-Base
  - finnianx/Gros-Michel-90m-Base
  - LH-Tech-AI/Spark-5M-Base-v4
  - Eclipse-Senpai/KeyLM-75M
  - GODELEV/Archaea-74M-V1.1
  - Quazim0t0/Escarda-86M-Base
  - StentorLabs/Stentor3-20M
  - StentorLabs/Stentor3-50M
  - jhu-clsp/ettin-decoder-17m
  - jhu-clsp/ettin-decoder-32m
  - jhu-clsp/ettin-decoder-68m
  - jhu-clsp/ettin-decoder-150m
  - 56m/Dumb-1.2-RC1
  - Quazim0t0/Escarda-86M-Identity
  - MultivexAI/Supra-1.6-50M-Instruct-Ultra-exp
  - HuggingFaceTB/SmolLM2-135M-Instruct
  - ThingAI/Quark-135m
  - ThingAI/Quark-72M
  - ThingAI/Quark-50m
  - joelhenwang/OdinNext-138M-Instruct
  - veyra-ai/Veyra2-30M-Instruct-Early
  - MinimaLabs/KeyLM-75M-Instruct
tags:
  - text-generation
  - small-language-model
  - model-arena
  - blind-evaluation
  - huggingface

SLM Arena

Standalone Hugging Face Space for comparing 2 to 5 compact language models at once.

What it does

• Pick and See lets you choose the models and keeps their identities visible while they generate.
• Pick Blind lets you choose the models manually, but the responses stay anonymous until Reveal.
• Random Blind samples hidden models from the full catalog, with options for:
  • allowing or blocking same-organization matches
  • enforcing a strict <2x size spread across the random group
• Model Catalog switches between Base Models and Instruct Models and keeps the dropdowns aligned with the active family.

Commentary

After a run finishes, the Space can ask a provider-backed commentary model for neutral commentary:

• zai-glm-4.7 on Cerebras, using CEREBRAS_TOKEN and recommended by default
• openai/gpt-oss-120b on Groq, using GROQ_TOKEN

The commentary prompt still uses these text-task sampling settings:

• temperature=0.2
• top_p=1.0

In Pick and See, commentary appears after the generations finish and the commentator call completes.

In Pick Blind and Random Blind, commentary stays locked until Reveal exposes the identities behind each response. After Reveal, it appears in the same panel once the commentator call completes. The commentary panel also has an on/off toggle and a provider selector for Cerebras or Groq. If GLM 4.7 hits a rate limit, Cerebras currently lists it at 5 RPM, so switch to Groq's GPT OSS 120B. Groq currently lists GPT OSS 120B at 30 RPM, or about 1 request every 2 seconds, which is usually enough for arena runs.

Notes

• The catalog is preloaded at startup, but arena contestant weights still load lazily so the Space can keep the full roster without pinning every model at once.
• The catalog view is sorted by organization first and then by parameter count, and the dropdowns follow the active family.
• Commentary requests are sent through the provider OpenAI-compatible chat APIs using the selected repo secret.
• Some upstream repos are gated or use custom research code. Those models stay in the arena catalog, but generation still depends on the upstream repo exposing a working transformers-compatible loading path.