Add moe-l2 model card

README.md

@@ -1,3 +1,84 @@
 ---
+language:
+- en
+- zh
 license: apache-2.0
+tags:
+- moe
+- llm
+- llama.cpp
+- expert-offload
+- low-vram
+- deepseek
+- qwen
+- mixtral
+library_name: moe-l2
+pipeline_tag: text-generation
 ---
+
+# moe-l2 — Run 100B+ MoE models on low-VRAM NVIDIA GPUs
+
+**moe-l2 is an expert-offload scheduler for llama.cpp that lets you run large MoE (Mixture-of-Experts) models — DeepSeek, Qwen, Mixtral, 100B+ params — on consumer NVIDIA GPUs with far less VRAM than the model would normally need.**
+
+## Why
+
+MoE models are like a hospital: every token only activates a few experts (the "doctors on duty"), but all experts must stay "on shift" (loaded into VRAM). A 16B MoE model with a 6 GB file can require **23.3 GB of VRAM** when fully loaded — an 8 GB card can't even start it.
+
+moe-l2 keeps the experts in host RAM (zero VRAM) and only moves the **activated** experts to the GPU per step, with an LRU hot-expert cache. The GPU reads pinned expert weights directly via PCIe DMA.
+
+## Measured results (RTX 4090, full chain `moe-l2 start --gpu`)
+
+| Model | Full-load VRAM | With moe-l2 | Speed |
+|---|---:|---:|---:|
+| DeepSeek-V2-Lite (16B MoE) | 23.3 GB | **1.6-2.0 GB** | **139-154 t/s** |
+| Qwen3.6-A3B (32B MoE) | 7.6 GB+ | **2.9 GB** | 50.2 t/s (single) |
+| DeepSeek-V4-Flash (157B MoE, 85 GB file) | OOM | **8.3-9.1 GB VRAM** | N/A (upstream bug) |
+
+Three-card full-chain measurements (2026-08-19, bins-v0.6.0):
+
+| GPU | DeepSeek-V2-Lite | Qwen3.6-A3B |
+|---|---:|---:|
+| RTX 4090 | 139-154 t/s | 25.5-44.2 t/s |
+| RTX 2080 Ti (11 GB) | 86-94 t/s | 16.6-28.6 t/s |
+| RTX 5090 | 141-151 t/s | 28-52.5 t/s |
+
+## How it works (3 layers)
+
+1. **Domain predictor** — classifies your prompt's domain (8 domains) and preloads the most likely experts.
+2. **L2 cache** — mmap shared-memory LRU cache, async preload, zero-copy GGUF reading.
+3. **A3 GPU expert cache** — per-slot locked LRU in VRAM; hot experts stay resident, cold experts are evicted. Router-map driven **selective pin** keeps only top-K experts per layer locked, so an 85 GB model runs at ~10.4 GB RSS with zero speed regression.
+
+Transparent OpenAI-compatible proxy on localhost:11435 (SSE streaming), so it works with Open WebUI, LangChain, etc.
+
+## Quick start
+
+```bash
+# Linux x86_64 + NVIDIA GPU + CUDA driver
+pip install moe-l2
+moe-l2 download-bins
+moe-l2 start --model /path/to/model.gguf --gpu
+```
+
+`moe-l2 doctor` checks your environment; `moe-l2 model download` fetches models with resumable downloads.
+
+## Requirements
+
+- Linux x86_64
+- NVIDIA GPU (GTX 1080 → RTX 50, one prebuilt binary for all, sm_61-sm_120a)
+- Python 3.9+
+- Native Windows / macOS / ARM Linux not supported (WSL2 unverified)
+
+## Version history
+
+13 versions in one month: 23.3 GB → 1.2 GB VRAM, from "can't run" to 140+ t/s. Full changelog on [GitHub](https://github.com/yalun753/moe-l2/blob/main/CHANGELOG.md).
+
+## Links
+
+- **Source code**: [github.com/yalun753/moe-l2](https://github.com/yalun753/moe-l2)
+- **PyPI**: [pypi.org/project/moe-l2](https://pypi.org/project/moe-l2)
+- **Benchmarks & verification reports**: in the GitHub `references/` directory (EN/ZH)
+- **White paper**: [white-paper](https://github.com/yalun753/moe-l2/blob/main/references/zh/white-paper-zh.md)
+
+## License
+
+Apache-2.0