Update README.md
Browse files
README.md
CHANGED
|
@@ -7,13 +7,13 @@ license: mit
|
|
| 7 |
<p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a> | 🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a> | 🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a> </p>
|
| 8 |
|
| 9 |
# Introduction
|
| 10 |
-
We are introducing Ling-3.0-tiny, a native hybrid reasoning model with 7.9B total parameters and only 1.3B activated parameters during inference. This release provides BF16, FP8, and INT4 variants
|
| 11 |
|
| 12 |
Key highlights of the model are summarized below:
|
| 13 |
|
| 14 |
+ **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
|
| 15 |
+ **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following.
|
| 16 |
-
+ **Local and Edge Deployment:**
|
| 17 |
|
| 18 |
# Model Overview
|
| 19 |
Ling-3.0-tiny adopts the native hybrid linear attention architecture of Ling-3.0 series and is further optimized for lightweight, accessible deployment scenarios. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
|
|
|
|
| 7 |
<p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a> | 🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a> | 🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a> </p>
|
| 8 |
|
| 9 |
# Introduction
|
| 10 |
+
We are introducing Ling-3.0-tiny, a native hybrid reasoning model with 7.9B total parameters and only 1.3B activated parameters during inference. This release provides BF16, FP8, and INT4 variants for different hardware, performance, and deployment requirements.
|
| 11 |
|
| 12 |
Key highlights of the model are summarized below:
|
| 13 |
|
| 14 |
+ **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
|
| 15 |
+ **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following.
|
| 16 |
+
+ **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
|
| 17 |
|
| 18 |
# Model Overview
|
| 19 |
Ling-3.0-tiny adopts the native hybrid linear attention architecture of Ling-3.0 series and is further optimized for lightweight, accessible deployment scenarios. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
|