CatherineZY commited on
Commit
7a1c740
·
verified ·
1 Parent(s): 99d8ac9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -7,13 +7,13 @@ license: mit
7
  <p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a>&nbsp;&nbsp; | &nbsp;&nbsp;🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a>&nbsp;&nbsp; | &nbsp;&nbsp;🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a>&nbsp;&nbsp;</p>
8
 
9
  # Introduction
10
- We are introducing Ling-3.0-tiny, a native hybrid reasoning model with 7.9B total parameters and only 1.3B activated parameters during inference. This release provides BF16, FP8, and INT4 variants, allowing developers to choose flexibly based on their hardware environment, capability requirements, and deployment costs.
11
 
12
  Key highlights of the model are summarized below:
13
 
14
  + **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
15
  + **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following.
16
- + **Local and Edge Deployment:** BF16, FP8, and INT4 weights are provided to support different requirements, including high-precision evaluation, balancing throughput and resource usage, and deployment in resource-constrained environments.
17
 
18
  # Model Overview
19
  Ling-3.0-tiny adopts the native hybrid linear attention architecture of Ling-3.0 series and is further optimized for lightweight, accessible deployment scenarios. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
 
7
  <p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a>&nbsp;&nbsp; | &nbsp;&nbsp;🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a>&nbsp;&nbsp; | &nbsp;&nbsp;🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a>&nbsp;&nbsp;</p>
8
 
9
  # Introduction
10
+ We are introducing Ling-3.0-tiny, a native hybrid reasoning model with 7.9B total parameters and only 1.3B activated parameters during inference. This release provides BF16, FP8, and INT4 variants for different hardware, performance, and deployment requirements.
11
 
12
  Key highlights of the model are summarized below:
13
 
14
  + **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model to balance long-context modeling capability, parameter efficiency, and computational cost.
15
  + **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following.
16
+ + **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
17
 
18
  # Model Overview
19
  Ling-3.0-tiny adopts the native hybrid linear attention architecture of Ling-3.0 series and is further optimized for lightweight, accessible deployment scenarios. The model has 7.9B total parameters, with only 1.3B parameters activated per token.