inclusionAI
/

Ling-3.0-tiny

Text Generation

Model card Files Files and versions

CatherineZY commited on 17 days ago

Commit

2d2391e

·

verified ·

1 Parent(s): 4243383

Update README.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -22,8 +22,7 @@ The architecture of Ling-3.0-tiny is not designed to optimize a single technical
 + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer in every 4-layer block) enables more efficient long-context processing;
 + A sparse MoE with 128 experts activates 8 routed experts and 1 shared expert per token, enabling the model to deliver more comprehensive capabilities with only 1.3B activated parameters;
-+ Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model;
-+ The Multi-Token Prediction training objective provides a foundation for more efficient token prediction and further inference acceleration.
 Together, these designs target three goals:

 + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer in every 4-layer block) enables more efficient long-context processing;
 + A sparse MoE with 128 experts activates 8 routed experts and 1 shared expert per token, enabling the model to deliver more comprehensive capabilities with only 1.3B activated parameters;
++ Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within one single model.
 Together, these designs target three goals: