Meditor-2-3.4B-Universal โ€” Standard Qwen2Moe & GGUF Edition

Meditor-2-3.4B-Universal flattens the hierarchical routing graph of Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE into standard Hugging Face Qwen2MoeForCausalLM (qwen2_moe) and ready-to-run GGUF quantizations (BF16 through Q2_K / IQ4_XS).

Meditor Model Series

Generation & Variant Hugging Face Repo Base Backbone Params (Total / Active) UniBench-100 Mean (/10) Pass Rate VRAM Footprint
Meditor 1 (Native MoE) Yo-Purple/Meditor-1-2.2B-A500M-MoE Qwen2.5-0.5B 2.22B / 0.57B 2.09 (37/100) ยท 2.42 (Best M1) 1.0% (Best M1) 4.16 GB (BF16)
Meditor 1 (Universal) Yo-Purple/Meditor-1-500M-Universal Qwen2.5-0.5B (qwen2) 0.49B / 0.49B 2.42 (100/100 BF16) 1.0% 0.93 GB / GGUF
Meditor 2 (Native HMoE) Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE SmolLM2-1.7B 3.41B / 1.83B 3.28 (37/100) ยท 4.12 (Best M2) 22.0% (Best M2) 6.42 GB (BF16)
Meditor 2 (Universal) (This Model) Yo-Purple/Meditor-2-3.4B-Universal SmolLM2-1.7B (qwen2_moe) 3.41B / 1.83B 4.12 (100/100 BF16) ยท 4.00 (Q4_K_M) 22.0% / 18.0% 6.38 GB / GGUF
Meditor 3 (Native MoE) Yo-Purple/Meditor-3-4B-MoE Qwen3-4B (meditor3) 4.16B / 4.04Bโ€“4.07B 7.31 (100/100, Rank #1/36) 69.0% (7.0% Uniq) 5.43 GiB peak (bf16_low_vram)
Meditor 3 (Universal GGUF) Yo-Purple/Meditor-3-4B-Universal Qwen3-4B (k2-horizon) 4.16B (BF16) ยท Q8_0 ยท Q4_K_M Compiled GGUF from M3 (7.31 native) LM Studio / llama.cpp 4.37โ€“5.95 GiB (quantized)

Universal LLM Benchmark (UniBench-100) Cross-Generation Comparison

All benchmark comparisons below come directly from the unified UniBench-100 evaluation suite (100 open-ended synthesis questions across 10 cognitive domains, graded out of 10.0 with pass threshold >= 6.0 by gemini-3.8-flash).

1. Cross-Generation Progression (Full 100/100 Runs)

Rank (of 36 Runs) Model Run Series Generation Evaluated Mean Score (/10) Pass Rate (>=6.0) Unique Solve Rate Eval Speed
#1 Meditor-3-4B-MoE (Released) Meditor 3 (Native) 100 / 100 7.31 69.0% 7.0% 35.7 tok/s
#2 Meditor-3 Prior Uncontested (BF16) Meditor 3 (Prior R&D) 100 / 100 4.62 30.0% 1.0% 35.7 tok/s
#5 Meditor 3 Beta (BF16) Meditor 3 (Prior R&D) 100 / 100 4.31 22.0% 1.0% 19.0 tok/s
#11 Meditor 2 Universal (BF16) Meditor 2 Best 100 / 100 4.12 22.0% 0.0% 27.5 tok/s
#18 Meditor 2 Universal (Q4_K_M) Meditor 2 Quantized 100 / 100 4.00 18.0% 0.0% 141.2 tok/s
#29 Meditor 2 (Native HMoE) Meditor 2 Native 37 / 100 3.28 8.1% 0.0% 48.7 tok/s
#30 Meditor 1 Universal (BF16) Meditor 1 Best 100 / 100 2.42 1.0% 0.0% 163.4 tok/s
#31 Meditor 1 Universal (Quant) Meditor 1 Quantized 100 / 100 2.35 0.0% 0.0% 171.3 tok/s
#33 Meditor 1 (Native MoE) Meditor 1 Native 37 / 100 2.09 2.7% 0.0% 141.9 tok/s

Key Takeaway: Meditor-3-4B-MoE delivers +3.19 points (+77.4% relative score jump) over the best Meditor 2 run (4.12), +4.89 points (3.02x score) over the best Meditor 1 run (2.42), and +2.69 points over the strongest earlier Meditor 3 research run (4.62).

UniBench-100 Progression Across Meditor Generations

2. Per-Domain Score Breakdown (Best Runs of Meditor 1, Meditor 2, and Meditor 3)

UniBench-100 Domain Meditor 1 Best (2.42) Meditor 2 Best (4.12) Meditor 3 Released (7.31) Gain vs M2 M3 Pass Rate
Critical Thinking & Argument Evaluation 3.51 5.98 9.09 +3.11 100.0%
Creative & Stylistic Expression 2.91 4.95 8.38 +3.43 90.0%
Humanities, Philosophy & Ethics 2.30 4.91 8.35 +3.44 80.0%
Physical & Life Sciences 1.68 3.38 7.41 +4.03 80.0%
Mathematics & Quantitative Estimation 2.36 1.99 7.27 +5.28 60.0%
Factuality & Hallucination Resistance 1.32 5.56 7.09 +1.53 60.0%
Real-World Tactical Strategy & Decision-Making 2.10 3.72 6.98 +3.26 60.0%
Instruction Following & Strict Constraints 3.65 4.62 6.82 +2.20 50.0%
Coding & Software Engineering 2.15 3.26 6.46 +3.20 70.0%
Logic & Abstract Reasoning 2.19 2.82 5.27 +2.45 40.0%

UniBench-100 Per-Domain Comparison

3. Full Leaderboard Landscape

UniBench-100 Leaderboard Rankings

Architecture Specifications (Meditor-2-3.4B-Universal)

Architectural Parameter Value
Model Class Qwen2MoeForCausalLM (qwen2_moe)
Backbone Specs 24 layers, d_model=2048, 32Q / 32KV heads
FFN / MoE Layout 1 Shared Expert (8192) + 15 Routed Experts (768, top-1 per token)
Parameters 3,411,003,392 Total ยท 1,825,507,328 Active (53.52%)
Context Window 524,288 tokens (rope_theta=130,000.0)

Local Hardware & Probe Evaluations

Evaluation Benchmark Observed Result
UniBench-100 (BF16) 4.12 / 10 (22.0% pass, 100/100 evaluated @ 27.5 tok/s โ€” Rank #11 / 36)
UniBench-100 (Q4_K_M) 4.00 / 10 (18.0% pass, 100/100 evaluated @ 141.2 tok/s โ€” Rank #18 / 36)
WikiText-2 Perplexity 2.48 (loss 0.9063)
ARC Subset (12 Questions) 100.0% (12 / 12)
Arithmetic (10 Questions) 60.0% raw ยท 100.0% with tool assist
PyTorch Speed & VRAM (seq_256) 55.51 tok/s ยท 6.38 GB BF16 VRAM

Available Files & Quantizations

  • Standard HF Safetensors (qwen2_moe): model.safetensors, config.json, tokenizer files
  • Official GGUF Quants: meditor-2-3.4b-universal-bf16.gguf, meditor-2-3.4b-universal-q8_0.gguf, meditor-2-3.4b-universal-q6_k.gguf, meditor-2-3.4b-universal-q5_k_m.gguf, meditor-2-3.4b-universal-q5_k_s.gguf, meditor-2-3.4b-universal-q4_k_m.gguf, meditor-2-3.4b-universal-q4_k_s.gguf, meditor-2-3.4b-universal-q3_k_m.gguf
  • Community GGUF Quants (credit: mradermacher): Meditor-2-3.4B-Universal.IQ4_XS.gguf, Meditor-2-3.4B-Universal.Q3_K_L.gguf, Meditor-2-3.4B-Universal.Q3_K_S.gguf, Meditor-2-3.4B-Universal.Q2_K.gguf

Quickstart

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Yo-Purple/Meditor-2-3.4B-Universal"
tok = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")
inputs = tok("Summarize Newton's second law:", return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**inputs, max_new_tokens=96)[0], skip_special_tokens=True))

Or run via llama.cpp:

llama-server -m meditor-2-3.4b-universal-q4_k_m.gguf -ngl 99 -c 8192

License

Released under Apache 2.0. Built on HuggingFaceTB/SmolLM2-1.7B-Instruct.

Downloads last month
1,395
Safetensors
Model size
4B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Yo-Purple/Meditor-2-3.4B-Universal

Quantized
(133)
this model
Quantizations
2 models

Space using Yo-Purple/Meditor-2-3.4B-Universal 1

Evaluation results

  • BF16 Mean Score (/10) on UniBench-100 (10 Domains, 100 Questions)
    self-reported
    4.120
  • BF16 Pass Rate (%) on UniBench-100 (10 Domains, 100 Questions)
    self-reported
    22.000
  • Q4_K_M Mean Score (/10) on UniBench-100 (10 Domains, 100 Questions)
    self-reported
    4.000
  • ARC Accuracy (%) on ARC Challenge Subset
    self-reported
    100.000
  • Perplexity on WikiText-2
    self-reported
    2.480