Instructions to use Yo-Purple/Meditor-2-3.4B-Universal with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Yo-Purple/Meditor-2-3.4B-Universal with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Yo-Purple/Meditor-2-3.4B-Universal") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Yo-Purple/Meditor-2-3.4B-Universal") model = AutoModelForCausalLM.from_pretrained("Yo-Purple/Meditor-2-3.4B-Universal", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Yo-Purple/Meditor-2-3.4B-Universal with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M # Run inference directly in the terminal: llama cli -hf Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M # Run inference directly in the terminal: llama cli -hf Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
Use Docker
docker model run hf.co/Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Yo-Purple/Meditor-2-3.4B-Universal with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Yo-Purple/Meditor-2-3.4B-Universal" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Yo-Purple/Meditor-2-3.4B-Universal", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
- SGLang
How to use Yo-Purple/Meditor-2-3.4B-Universal with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Yo-Purple/Meditor-2-3.4B-Universal" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Yo-Purple/Meditor-2-3.4B-Universal", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Yo-Purple/Meditor-2-3.4B-Universal" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Yo-Purple/Meditor-2-3.4B-Universal", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use Yo-Purple/Meditor-2-3.4B-Universal with Ollama:
ollama run hf.co/Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use Yo-Purple/Meditor-2-3.4B-Universal with Docker Model Runner:
docker model run hf.co/Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
- Lemonade
How to use Yo-Purple/Meditor-2-3.4B-Universal with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Yo-Purple/Meditor-2-3.4B-Universal:Q4_K_M
Run and chat with the model
lemonade run user.Meditor-2-3.4B-Universal-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Meditor-2-3.4B-Universal โ Standard Qwen2Moe & GGUF Edition
Meditor-2-3.4B-Universal flattens the hierarchical routing graph of Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE into standard Hugging Face Qwen2MoeForCausalLM (qwen2_moe) and ready-to-run GGUF quantizations (BF16 through Q2_K / IQ4_XS).
Meditor Model Series
| Generation & Variant | Hugging Face Repo | Base Backbone | Params (Total / Active) | UniBench-100 Mean (/10) | Pass Rate | VRAM Footprint |
|---|---|---|---|---|---|---|
| Meditor 1 (Native MoE) | Yo-Purple/Meditor-1-2.2B-A500M-MoE |
Qwen2.5-0.5B | 2.22B / 0.57B | 2.09 (37/100) ยท 2.42 (Best M1) | 1.0% (Best M1) | 4.16 GB (BF16) |
| Meditor 1 (Universal) | Yo-Purple/Meditor-1-500M-Universal |
Qwen2.5-0.5B (qwen2) |
0.49B / 0.49B | 2.42 (100/100 BF16) | 1.0% | 0.93 GB / GGUF |
| Meditor 2 (Native HMoE) | Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE |
SmolLM2-1.7B | 3.41B / 1.83B | 3.28 (37/100) ยท 4.12 (Best M2) | 22.0% (Best M2) | 6.42 GB (BF16) |
| Meditor 2 (Universal) (This Model) | Yo-Purple/Meditor-2-3.4B-Universal |
SmolLM2-1.7B (qwen2_moe) |
3.41B / 1.83B | 4.12 (100/100 BF16) ยท 4.00 (Q4_K_M) | 22.0% / 18.0% | 6.38 GB / GGUF |
| Meditor 3 (Native MoE) | Yo-Purple/Meditor-3-4B-MoE |
Qwen3-4B (meditor3) |
4.16B / 4.04Bโ4.07B | 7.31 (100/100, Rank #1/36) | 69.0% (7.0% Uniq) | 5.43 GiB peak (bf16_low_vram) |
| Meditor 3 (Universal GGUF) | Yo-Purple/Meditor-3-4B-Universal |
Qwen3-4B (k2-horizon) |
4.16B (BF16) ยท Q8_0 ยท Q4_K_M |
Compiled GGUF from M3 (7.31 native) |
LM Studio / llama.cpp | 4.37โ5.95 GiB (quantized) |
Universal LLM Benchmark (UniBench-100) Cross-Generation Comparison
All benchmark comparisons below come directly from the unified UniBench-100 evaluation suite (100 open-ended synthesis questions across 10 cognitive domains, graded out of 10.0 with pass threshold >= 6.0 by gemini-3.8-flash).
1. Cross-Generation Progression (Full 100/100 Runs)
| Rank (of 36 Runs) | Model Run | Series Generation | Evaluated | Mean Score (/10) | Pass Rate (>=6.0) | Unique Solve Rate | Eval Speed |
|---|---|---|---|---|---|---|---|
| #1 | Meditor-3-4B-MoE (Released) | Meditor 3 (Native) | 100 / 100 | 7.31 | 69.0% | 7.0% | 35.7 tok/s |
| #2 | Meditor-3 Prior Uncontested (BF16) | Meditor 3 (Prior R&D) | 100 / 100 | 4.62 | 30.0% | 1.0% | 35.7 tok/s |
| #5 | Meditor 3 Beta (BF16) | Meditor 3 (Prior R&D) | 100 / 100 | 4.31 | 22.0% | 1.0% | 19.0 tok/s |
| #11 | Meditor 2 Universal (BF16) | Meditor 2 Best | 100 / 100 | 4.12 | 22.0% | 0.0% | 27.5 tok/s |
| #18 | Meditor 2 Universal (Q4_K_M) | Meditor 2 Quantized | 100 / 100 | 4.00 | 18.0% | 0.0% | 141.2 tok/s |
| #29 | Meditor 2 (Native HMoE) | Meditor 2 Native | 37 / 100 | 3.28 | 8.1% | 0.0% | 48.7 tok/s |
| #30 | Meditor 1 Universal (BF16) | Meditor 1 Best | 100 / 100 | 2.42 | 1.0% | 0.0% | 163.4 tok/s |
| #31 | Meditor 1 Universal (Quant) | Meditor 1 Quantized | 100 / 100 | 2.35 | 0.0% | 0.0% | 171.3 tok/s |
| #33 | Meditor 1 (Native MoE) | Meditor 1 Native | 37 / 100 | 2.09 | 2.7% | 0.0% | 141.9 tok/s |
Key Takeaway: Meditor-3-4B-MoE delivers +3.19 points (
+77.4%relative score jump) over the best Meditor 2 run (4.12), +4.89 points (3.02xscore) over the best Meditor 1 run (2.42), and +2.69 points over the strongest earlier Meditor 3 research run (4.62).
2. Per-Domain Score Breakdown (Best Runs of Meditor 1, Meditor 2, and Meditor 3)
| UniBench-100 Domain | Meditor 1 Best (2.42) |
Meditor 2 Best (4.12) |
Meditor 3 Released (7.31) |
Gain vs M2 | M3 Pass Rate |
|---|---|---|---|---|---|
| Critical Thinking & Argument Evaluation | 3.51 | 5.98 | 9.09 | +3.11 |
100.0% |
| Creative & Stylistic Expression | 2.91 | 4.95 | 8.38 | +3.43 |
90.0% |
| Humanities, Philosophy & Ethics | 2.30 | 4.91 | 8.35 | +3.44 |
80.0% |
| Physical & Life Sciences | 1.68 | 3.38 | 7.41 | +4.03 |
80.0% |
| Mathematics & Quantitative Estimation | 2.36 | 1.99 | 7.27 | +5.28 |
60.0% |
| Factuality & Hallucination Resistance | 1.32 | 5.56 | 7.09 | +1.53 |
60.0% |
| Real-World Tactical Strategy & Decision-Making | 2.10 | 3.72 | 6.98 | +3.26 |
60.0% |
| Instruction Following & Strict Constraints | 3.65 | 4.62 | 6.82 | +2.20 |
50.0% |
| Coding & Software Engineering | 2.15 | 3.26 | 6.46 | +3.20 |
70.0% |
| Logic & Abstract Reasoning | 2.19 | 2.82 | 5.27 | +2.45 |
40.0% |
3. Full Leaderboard Landscape
Architecture Specifications (Meditor-2-3.4B-Universal)
| Architectural Parameter | Value |
|---|---|
| Model Class | Qwen2MoeForCausalLM (qwen2_moe) |
| Backbone Specs | 24 layers, d_model=2048, 32Q / 32KV heads |
| FFN / MoE Layout | 1 Shared Expert (8192) + 15 Routed Experts (768, top-1 per token) |
| Parameters | 3,411,003,392 Total ยท 1,825,507,328 Active (53.52%) |
| Context Window | 524,288 tokens (rope_theta=130,000.0) |
Local Hardware & Probe Evaluations
| Evaluation Benchmark | Observed Result |
|---|---|
UniBench-100 (BF16) |
4.12 / 10 (22.0% pass, 100/100 evaluated @ 27.5 tok/s โ Rank #11 / 36) |
UniBench-100 (Q4_K_M) |
4.00 / 10 (18.0% pass, 100/100 evaluated @ 141.2 tok/s โ Rank #18 / 36) |
| WikiText-2 Perplexity | 2.48 (loss 0.9063) |
ARC Subset (12 Questions) |
100.0% (12 / 12) |
Arithmetic (10 Questions) |
60.0% raw ยท 100.0% with tool assist |
PyTorch Speed & VRAM (seq_256) |
55.51 tok/s ยท 6.38 GB BF16 VRAM |
Available Files & Quantizations
- Standard HF Safetensors (
qwen2_moe):model.safetensors,config.json, tokenizer files - Official GGUF Quants:
meditor-2-3.4b-universal-bf16.gguf,meditor-2-3.4b-universal-q8_0.gguf,meditor-2-3.4b-universal-q6_k.gguf,meditor-2-3.4b-universal-q5_k_m.gguf,meditor-2-3.4b-universal-q5_k_s.gguf,meditor-2-3.4b-universal-q4_k_m.gguf,meditor-2-3.4b-universal-q4_k_s.gguf,meditor-2-3.4b-universal-q3_k_m.gguf - Community GGUF Quants (credit:
mradermacher):Meditor-2-3.4B-Universal.IQ4_XS.gguf,Meditor-2-3.4B-Universal.Q3_K_L.gguf,Meditor-2-3.4B-Universal.Q3_K_S.gguf,Meditor-2-3.4B-Universal.Q2_K.gguf
Quickstart
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "Yo-Purple/Meditor-2-3.4B-Universal"
tok = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")
inputs = tok("Summarize Newton's second law:", return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**inputs, max_new_tokens=96)[0], skip_special_tokens=True))
Or run via llama.cpp:
llama-server -m meditor-2-3.4b-universal-q4_k_m.gguf -ngl 99 -c 8192
License
Released under Apache 2.0. Built on HuggingFaceTB/SmolLM2-1.7B-Instruct.
- Downloads last month
- 1,395
Model tree for Yo-Purple/Meditor-2-3.4B-Universal
Space using Yo-Purple/Meditor-2-3.4B-Universal 1
Evaluation results
- BF16 Mean Score (/10) on UniBench-100 (10 Domains, 100 Questions)self-reported4.120
- BF16 Pass Rate (%) on UniBench-100 (10 Domains, 100 Questions)self-reported22.000
- Q4_K_M Mean Score (/10) on UniBench-100 (10 Domains, 100 Questions)self-reported4.000
- ARC Accuracy (%) on ARC Challenge Subsetself-reported100.000
- Perplexity on WikiText-2self-reported2.480


