🤗 Hugging Face | 🤖 ModelScope | 🐙 OpenRouter
# Introduction We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.3B** activated parameters per token. It is designed to deliver strong reasoning and agentic capabilities under a small inference compute footprint, making advanced model capabilities more accessible for local and resource-constrained deployment. BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings. Key highlights of the model are summarized below: + **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost. + **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following. + **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length. # Model Overview Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.3B parameters activated per token. The architecture of Ling-3.0-tiny is not designed to optimize a single technical metric; instead, it aims to translate efficiency into real-world task performance. + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer in every 4-layer block) enables more efficient long-context processing; + A sparse MoE with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.3B activated parameters per token. + Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model. Together, these designs target three goals: + Reduce the computational resources required for inference; + Lower the barriers to local deployment and downstream development; + Enable lightweight models to participate in real-world agent workflows.  # Evaluation We evaluated Ling-3.0-tiny across agentic tasks, coding, long-context understanding, knowledge reliability, mathematical and scientific reasoning, and instruction following. Ling-3.0-tiny achieves a score of **25** on the Artificial Analysis Intelligence Index v4.1.1 and **16** on the Artificial Analysis Agentic Index. In Artificial Analysis testing, Ling-3.0-tiny reaches an output speed of over **160 tokens/s**, with approximately **18 seconds** of end-to-end latency for a 500-token response, including reasoning time. These results highlight the model's efficiency relative to its 1.3B activated parameter footprint. The following table presents representative benchmarks for Ling-3.0-tiny:  > + All Ling-3.0-tiny evaluation results use Thinking mode; the recommended sampling parameters are `temperature=1.0`, `top_p=0.95`, and `top_k=20`. > + Terminal-Bench 2.1: Evaluated under the Artificial Analysis (AA) protocol using the default Terminus 2 harness, a unified 2-hour timeout, the provided JSON parser in preserve-thinking mode, and 3 runs per task (mean). Decoding uses temperature=1.0, max_new_tokens=32K, with a 256K context window. # Quickstart ## SGLang The hardware- and recipe-specific launch matrix (BF16/FP8 × Low-Latency / High-Throughput / HiCache + Mooncake), with a live command generator and verified configurations, lives in the SGLang cookbook: **Cookbook:** https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ling-3.0-tiny ### Install SGLang Use the pre-built image that tracks the Ling-3.0 runtime: ```bash docker pull lmsysorg/sglang:dev-Ling-3.0-tiny ``` ### Run Inference Recommended low-latency recipe (built-in MTP / NEXTN, 256K YaRN context) on 1× 141GB-class GPU (H20-3e) or a 1-GPU Blackwell node: **Server** ```bash docker run --rm --gpus all --ipc=host --shm-size 32g \ -p 30000:30000 \ -e HF_TOKEN=