CatherineZY commited on
Commit
a2ee06c
·
verified ·
1 Parent(s): 38c6b3b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -12
README.md CHANGED
@@ -8,29 +8,25 @@ license: mit
8
 
9
  # Introduction
10
  We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.3B** activated parameters per token.
11
- It is designed to deliver strong reasoning and agentic capabilities under a small inference compute footprint, making advanced model capabilities more accessible for local and resource-constrained deployment.
12
  BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings.
13
 
14
  Key highlights of the model are summarized below:
15
 
16
- + **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
17
- + **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following.
18
  + **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
19
 
20
  # Model Overview
21
  Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
22
 
23
- The architecture of Ling-3.0-tiny is not designed to optimize a single technical metric; instead, it aims to translate efficiency into real-world task performance.
24
 
25
- + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer in every 4-layer block) enables more efficient long-context processing;
26
- + A sparse MoE with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.3B activated parameters per token.
27
  + Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model.
28
 
29
- Together, these designs target three goals:
30
-
31
- + Reduce the computational resources required for inference;
32
- + Lower the barriers to local deployment and downstream development;
33
- + Enable lightweight models to participate in real-world agent workflows.
34
 
35
  <!-- Ling-3.0-tiny architecture diagram -->
36
  ![](https://intranetproxy.alipay.com/skylark/lark/0/2026/png/202156655/1786334509742-e888d58f-c9af-44d0-a64a-a9b6ba0cb652.png)
@@ -45,7 +41,7 @@ The following table presents representative benchmarks for Ling-3.0-tiny:
45
 
46
  ![image](https://cdn-uploads.huggingface.co/production/uploads/6502cf8fbdaeae26417cd3c9/g9Thw4ohjkDGYw0Cq7mKi.png)
47
 
48
- > + All Ling-3.0-tiny evaluation results use Thinking mode; the recommended sampling parameters are `temperature=1.0`, `top_p=0.95`, and `top_k=20`.
49
  > + Terminal-Bench 2.1: Evaluated under the Artificial Analysis (AA) protocol using the default Terminus 2 harness, a unified 2-hour timeout, the provided JSON parser in preserve-thinking mode, and 3 runs per task (mean). Decoding uses temperature=1.0, max_new_tokens=32K, with a 256K context window.
50
 
51
  # Quickstart
 
8
 
9
  # Introduction
10
  We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.3B** activated parameters per token.
11
+ It is designed to deliver strong reasoning and agentic capabilities at low inference cost, making advanced model capabilities more accessible for local and resource-constrained deployment.
12
  BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings.
13
 
14
  Key highlights of the model are summarized below:
15
 
16
+ + **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny integrates a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer per 4-layer block) with a sparse MoE FFN comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
17
+ + **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, coding, mathematical and scientific reasoning, and instruction following.
18
  + **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
19
 
20
  # Model Overview
21
  Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
22
 
23
+ The architecture of Ling-3.0-tiny is designed to make computational efficiency serve real-world agentic performance.
24
 
25
+ + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer per 4-layer block) provides more efficient long-context processing;
26
+ + A sparse MoE FFN with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.3B activated parameters per token.
27
  + Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model.
28
 
29
+ Overall, these designs deliver the inference efficiency needed to deploy lightweight models in real-world agentic workflows.
 
 
 
 
30
 
31
  <!-- Ling-3.0-tiny architecture diagram -->
32
  ![](https://intranetproxy.alipay.com/skylark/lark/0/2026/png/202156655/1786334509742-e888d58f-c9af-44d0-a64a-a9b6ba0cb652.png)
 
41
 
42
  ![image](https://cdn-uploads.huggingface.co/production/uploads/6502cf8fbdaeae26417cd3c9/g9Thw4ohjkDGYw0Cq7mKi.png)
43
 
44
+ > + Thinking mode is enabled by default. The recommended sampling parameters for Ling-3.0-tiny are `temperature=1.0`, `top_p=0.95`, and `top_k=20`.
45
  > + Terminal-Bench 2.1: Evaluated under the Artificial Analysis (AA) protocol using the default Terminus 2 harness, a unified 2-hour timeout, the provided JSON parser in preserve-thinking mode, and 3 runs per task (mean). Decoding uses temperature=1.0, max_new_tokens=32K, with a 256K context window.
46
 
47
  # Quickstart