Instructions to use okolxc/openmythos-2b-edge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use okolxc/openmythos-2b-edge with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="okolxc/openmythos-2b-edge")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("okolxc/openmythos-2b-edge", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use okolxc/openmythos-2b-edge with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "okolxc/openmythos-2b-edge" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "okolxc/openmythos-2b-edge", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/okolxc/openmythos-2b-edge
- SGLang
How to use okolxc/openmythos-2b-edge with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "okolxc/openmythos-2b-edge" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "okolxc/openmythos-2b-edge", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "okolxc/openmythos-2b-edge" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "okolxc/openmythos-2b-edge", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use okolxc/openmythos-2b-edge with Docker Model Runner:
docker model run hf.co/okolxc/openmythos-2b-edge
OpenMythos-2B β Edge (SFT + GRPO)
Qwen3.5-2B weights running inside the OpenMythos architecture
(hybrid linear attention + GQA + MoE-ready blocks), converted 1:1, then
instruction-tuned on UltraChat-200k and reinforced with GRPO using a
LiquidAI/d1-3B decision-model verifier.
| Base model | Qwen/Qwen3.5-2B |
| Params | 2.393 B (bf16) |
| Layers | 24 (18 linear-attention + 6 full-attention), 11 + 1 recurrent + 12 |
| Attention | hybrid: Gated DeltaNet (linear) + GQA full-attention |
| Context | 248 320 vocab, RoPE ΞΈ=1e7, 256k native positions |
| Weights | 5.80 GB, 4 safetensors shards |
π Verified results (Tesla T4, bf16)
Chat perplexity β identical held-out set: 69 UltraChat-200k test samples + 5 general-text samples = 12 313 tokens, same tokenizer and code path:
| model | held-out ppl | vs Qwen |
|---|---|---|
Qwen/Qwen3.5-2B (original) |
7.9467 | 1.00Γ |
| OpenMythos-2B (converted, pre-SFT) | 11.8564 | 1.49Γ |
| OpenMythos-2B (SFT β this repo) | 6.2327 | 0.78Γ |
Math reasoning (GSM8K test, greedy generation):
| model | accuracy |
|---|---|
| OpenMythos-2B (converted, pre-SFT) | 4 % (1/24) |
| OpenMythos-2B (after distillation) | 29.2 % (7/24) |
The conversion alone cost 1.49Γ perplexity; 150 steps of distillation from a Qwen3.5-2B teacher closed the gap and overshot.
π¦ Files
| file | what |
|---|---|
model-00001..00004-of-00004.safetensors |
the model (5.80 GB, bf16, 335 tensors) |
model.safetensors.index.json |
shard index |
config.json |
architecture metadata + training hyperparameters |
sft2_distill_step150.safetensors |
the distillation checkpoint (29.2 % GSM8K) |
grpo_step12.safetensors |
after GRPO with the d1-3B verifier |
scratch_2b/ |
earlier from-scratch checkpoints (different, smaller architecture) β see its README |
Load with the OpenMythos architecture (see config.json for the layer map);
the weights are a 1:1 copy of the converted Qwen3.5-2B tensors.
ποΈ Architecture
Embed β [11 Γ TransformerBlock] β [RecurrentBlock] β [12 Γ TransformerBlock] β RMSNorm β tied Head
β β
β ββ ACT / LoRA recurrent reasoning block
ββ hybrid attention: Gated DeltaNet + GQA full-attention
Each TransformerBlock = pre-norm attention + SwiGLU feed-forward, with optional Mixture-of-Experts, Mixture-of-Depths routing, Multi-Token-Prediction head, sparse-attention (DSA) and long-term KV memory available as fine-tuning targets.
β Conversion fidelity
| Component | Transfer |
|---|---|
| Gated DeltaNet attention (18 layers) | direct copy β module parity 0.43 % |
| GQA full attention (6 layers) | direct copy β layer parity cos 0.996 |
| SwiGLU feed-forward (6144) | direct copy |
| Norms (zero-centred RMSNorm) | direct copy |
| Embedding / output head | direct copy (tied) |
load_state_dict on the promoted checkpoint reports
0 missing / 0 unexpected tensors.
Post-hoc compression of K and V into a shared MLA latent was measured too (SVD energy retention 86β98 %) and found lossy, so the converted checkpoint keeps per-head K/V; the compressed-latent attention path is kept in the project for training from scratch.
π§ͺ Weight-Space Programming (gradient-free editing)
Weights can be edited directly, without gradients, as in Exploring
Weight-Space Programming (arXiv:2611.09382). The admissible edit space is
transformer-block parameters only (ffn.down.weight, attn.o_proj.weight).
The paper's headline edit was replicated on this model: contrasting the
"reconsideration" discourse (Wait, However, But, Maybe, Perhaps) against the
"forward-progress" discourse (Therefore, Thus, So, Now, The, First, Next, Let) in token-embedding space, then subtracting that direction from a single
MLP column:
u = N( mean_A E - mean_B E ) * (1 + gamma) gamma = final RMSNorm weight
W[:, j] -= u only one column changes
| weights changed | 2 046 bf16 (0.0163 %) β the paper changed 1 023 (0.016 %) |
| target | coda.11.ffn.down.weight column 869 |
| dev NLL | 3.0670 β 3.0564 |
| held-out NLL | 3.0452 β 3.0350 |
A control edit with the opposite sign regressed both dev and held-out, confirming the direction is meaningful rather than noise.
Edits are validated on a development split and only then measured on a held-out split β the same discipline the paper found necessary, where 6 of 16 trajectories had to be reverted because gains did not transfer.
π Attribution
Weights are derived from Qwen3.5-2B (Apache-2.0). Please respect the upstream license and keep the attribution when redistributing.
- Downloads last month
- 1,356