Update README.md
Browse files
README.md
CHANGED
|
@@ -8,29 +8,25 @@ license: mit
|
|
| 8 |
|
| 9 |
# Introduction
|
| 10 |
We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.3B** activated parameters per token.
|
| 11 |
-
It is designed to deliver strong reasoning and agentic capabilities
|
| 12 |
BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings.
|
| 13 |
|
| 14 |
Key highlights of the model are summarized below:
|
| 15 |
|
| 16 |
-
+ **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny
|
| 17 |
-
+ **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks,
|
| 18 |
+ **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
|
| 19 |
|
| 20 |
# Model Overview
|
| 21 |
Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
|
| 22 |
|
| 23 |
-
The architecture of Ling-3.0-tiny is
|
| 24 |
|
| 25 |
-
+ A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer
|
| 26 |
-
+ A sparse MoE with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.3B activated parameters per token.
|
| 27 |
+ Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model.
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
+ Reduce the computational resources required for inference;
|
| 32 |
-
+ Lower the barriers to local deployment and downstream development;
|
| 33 |
-
+ Enable lightweight models to participate in real-world agent workflows.
|
| 34 |
|
| 35 |
<!-- Ling-3.0-tiny architecture diagram -->
|
| 36 |

|
|
@@ -45,7 +41,7 @@ The following table presents representative benchmarks for Ling-3.0-tiny:
|
|
| 45 |
|
| 46 |

|
| 47 |
|
| 48 |
-
> +
|
| 49 |
> + Terminal-Bench 2.1: Evaluated under the Artificial Analysis (AA) protocol using the default Terminus 2 harness, a unified 2-hour timeout, the provided JSON parser in preserve-thinking mode, and 3 runs per task (mean). Decoding uses temperature=1.0, max_new_tokens=32K, with a 256K context window.
|
| 50 |
|
| 51 |
# Quickstart
|
|
|
|
| 8 |
|
| 9 |
# Introduction
|
| 10 |
We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.3B** activated parameters per token.
|
| 11 |
+
It is designed to deliver strong reasoning and agentic capabilities at low inference cost, making advanced model capabilities more accessible for local and resource-constrained deployment.
|
| 12 |
BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings.
|
| 13 |
|
| 14 |
Key highlights of the model are summarized below:
|
| 15 |
|
| 16 |
+
+ **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny integrates a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer per 4-layer block) with a sparse MoE FFN comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
|
| 17 |
+
+ **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, coding, mathematical and scientific reasoning, and instruction following.
|
| 18 |
+ **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
|
| 19 |
|
| 20 |
# Model Overview
|
| 21 |
Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
|
| 22 |
|
| 23 |
+
The architecture of Ling-3.0-tiny is designed to make computational efficiency serve real-world agentic performance.
|
| 24 |
|
| 25 |
+
+ A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer per 4-layer block) provides more efficient long-context processing;
|
| 26 |
+
+ A sparse MoE FFN with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.3B activated parameters per token.
|
| 27 |
+ Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model.
|
| 28 |
|
| 29 |
+
Overall, these designs deliver the inference efficiency needed to deploy lightweight models in real-world agentic workflows.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
<!-- Ling-3.0-tiny architecture diagram -->
|
| 32 |

|
|
|
|
| 41 |
|
| 42 |

|
| 43 |
|
| 44 |
+
> + Thinking mode is enabled by default. The recommended sampling parameters for Ling-3.0-tiny are `temperature=1.0`, `top_p=0.95`, and `top_k=20`.
|
| 45 |
> + Terminal-Bench 2.1: Evaluated under the Artificial Analysis (AA) protocol using the default Terminus 2 harness, a unified 2-hour timeout, the provided JSON parser in preserve-thinking mode, and 3 runs per task (mean). Decoding uses temperature=1.0, max_new_tokens=32K, with a 256K context window.
|
| 46 |
|
| 47 |
# Quickstart
|