Update README.md

README.md

@@ -1,3 +1,200 @@
 ---
 license: mit
 ---
+<p align="center">
+    <img src="https://mdn.alipayobjects.com/huamei_qa8qxu/afts/img/A*4QxcQrBlTiAAAAAAQXAAAAgAemJ7AQ/original" width="100"/>
+</p>
+<p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a>&nbsp;&nbsp; | &nbsp;&nbsp;🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a>&nbsp;&nbsp; | &nbsp;&nbsp;🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a>&nbsp;&nbsp;</p>
+
+# Introduction
+We are introducing Ling-3.0-tiny, a native hybrid reasoning model with 7.9B total parameters and only 1.3B activated parameters during inference. This release provides BF16, FP8, and INT4 variants, allowing developers to choose flexibly based on their hardware environment, capability requirements, and deployment costs.
+
+Key highlights of the model are summarized below:
+
++ **Efficient Hybrid-Linear Architecture:** Ling-3.0-tiny uses a 3:1 alternating stacking of KDA and MLA (3 Kimi Delta Attention layers followed by 1 Multi-Head Latent Attention layer in every 4-layer block), combined with a sparse MoE comprising 128 routed experts. Each token activates only 8 routed experts and 1 shared expert, allowing the model—with 7.9B total parameters and 1.3B activated parameters—to balance long-context modeling capability, parameter efficiency, and computational cost.
++ **Native Hybrid Reasoning and Agentic Capabilities:** Ling-3.0-tiny supports both fast responses and multi-step reasoning, with thinking mode configurable per request through `enable_thinking`. It delivers balanced performance across general agent tasks, tool use, mathematical and scientific reasoning, and instruction following.
++ **Local and Edge Deployment:** BF16, FP8, and INT4 weights are provided to support different requirements, including high-precision evaluation, balancing throughput and resource usage, and deployment in resource-constrained environments.
+
+# Model Overview
+Ling-3.0-tiny adopts the native hybrid linear attention architecture of Ling-3.0 series and is further optimized for lightweight, accessible deployment scenarios. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
+
+The architecture of Ling-3.0-tiny is not designed to optimize a single technical metric; instead, it aims to translate efficiency into real-world task performance.
+
++ A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer in every 4-layer block) enables more efficient long-context processing;
++ A sparse MoE with 128 experts activates 8 routed experts and 1 shared expert per token, enabling the model to deliver more comprehensive capabilities with only 1.3B activated parameters;
++ Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model;
++ The Multi-Token Prediction training objective provides a foundation for more efficient token prediction and further inference acceleration.
+
+Together, these designs target three goals:
+
++ Reduce the computational resources required for inference;
++ Lower the barriers to local deployment and downstream development;
++ Enable lightweight models to participate in real-world agent workflows.
+
+<!-- Ling-3.0-tiny architecture diagram -->
+![](https://intranetproxy.alipay.com/skylark/lark/0/2026/png/202156655/1786334509742-e888d58f-c9af-44d0-a64a-a9b6ba0cb652.png)
+
+# Evaluation
+We evaluated Ling-3.0-tiny across general agent tasks, coding agents, code generation, long-context understanding, knowledge reliability, mathematical and scientific reasoning, and instruction following.
+
+Ling-3.0-tiny scores **25** on the Artificial Analysis Intelligence Index v4.1.1 and **16** on the Artificial Analysis Agentic Index. In Artificial Analysis testing, the model achieves an output speed of more than **160 tokens/s** and an end-to-end latency of approximately **18 seconds** for generating 500 tokens (including the reasoning model's thinking time), reflecting a balance among overall capability, activated parameter scale, and response efficiency.
+
+The following table presents representative benchmarks for Ling-3.0-tiny:
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/6502cf8fbdaeae26417cd3c9/g9Thw4ohjkDGYw0Cq7mKi.png)
+
+> All Ling-3.0-tiny evaluation results use Thinking mode; the recommended sampling parameters are `temperature=1.0`, `top_p=0.95`, and `top_k=20`.
+
+# Quickstart
+## SGLang
+The hardware- and recipe-specific launch matrix (BF16/FP8 × Low-Latency / High-Throughput / HiCache + Mooncake) includes a live command generator and verified configurations:
+
+### Install SGLang
+Use the pre-built image that tracks the Ling-3.0 runtime:
+
+```bash
+docker pull lmsysorg/sglang:dev-Ling-3.0-tiny
+```
+
+### Run Inference
+We recommend the low-latency recipe (built-in MTP / NEXTN, 256K YaRN context) on 1× 141GB-class GPU (H20-3e) or a 1-GPU Blackwell node:
+
+**Server**
+
+```bash
+docker run --rm --gpus all --ipc=host --shm-size 32g \
+  -p 30000:30000 \
+  -e HF_TOKEN=<your-hf-token> \
+  lmsysorg/sglang:dev-Ling-3.0-tiny \
+  env SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 \
+  python3 -m sglang.launch_server \
+    --model-path inclusionAI/Ling-3.0-tiny \
+    --tp 1 \
+    --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":2.0,"rope_theta":6000000,"partial_rotary_factor":0.5,"original_max_position_embeddings":131072}}' \
+    --context-length 262144 \
+    --speculative-algorithm NEXTN \
+    --mem-fraction-static 0.8 \
+    --host 0.0.0.0 \
+    --port 30000
+```
+
+**Client**
+
+The chat template and the `ling3` reasoning parser enable thinking mode by default. To disable it for an individual request, set `"chat_template_kwargs": {"enable_thinking": false}`. We recommend the sampling parameters `temperature=1.0`, `top_p=0.95`, and `top_k=20`.
+
+```bash
+curl -s http://localhost:30000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model": "auto",
+       "messages": [{"role": "user", "content": "What is the capital of France?"}],
+       "stream": true,
+       "temperature": 1.0,
+       "top_k": 20,
+       "top_p": 0.95
+     }'
+```
+
+## vLLM
+### Install our vLLM
+```bash
+pip install uv
+
+uv venv ~/my_ling_env
+
+source ~/my_ling_env/bin/activate
+
+git clone -b ling_3_0 https://github.com/inclusionAI/vllm-ling-v3.git
+
+cd vllm-ling-v3
+
+VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto
+```
+
+### Run Inference
+The following example runs Ling-3.0-tiny on 1 GPU, with the server listening on port `${PORT}`:
+
+**Server**
+
+```bash
+vllm serve "$MODEL_PATH" \
+    --port "$PORT" \
+    --trust-remote-code \
+    --served-model-name auto \
+    --tensor-parallel-size 1 \
+    --gpu-memory-utilization 0.85 \
+    --enable-prefix-caching \
+    --mamba-cache-mode align \
+    --enable-auto-tool-choice \
+    --tool-call-parser ling3 \
+    --reasoning-parser ling3
+```
+
+**Client**
+
+We recommend using the sampling parameters `temperature=1.0`, `top_p=0.95`, and `top_k=20`, and enabling `enable_thinking` for better performance.
+
+```bash
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model": "auto",
+       "messages": [{"role": "user", "content": "What is the capital of France?"}],
+       "chat_template_kwargs": {"enable_thinking": true},
+       "stream": true,
+       "temperature": 1.0, 
+       "top_k": 20,
+       "top_p": 0.95
+     }'
+```
+
+## Ollama
+> This configuration has been verified on an M4 Pro Mac with 48 GB of unified memory.
+>
+
+### Preparation and Build
+```bash
+git clone https://github.com/ollama/ollama.git
+cd ollama
+git fetch origin refs/pull/17643/head:bailing-moe-v3
+git switch bailing-moe-v3
+
+cmake -B build .
+cmake --build build --parallel 8
+```
+
+> Support is currently provided by [ollama/ollama#17643](https://github.com/ollama/ollama/pull/17643) and is limited to running via MLX on Apple Silicon.
+>
+> Use the local `./ollama` executable built from source in this section. This functionality is not yet included in the official Ollama release.
+>
+
+### Import Model
+Replace `/absolute/path/to/fp8_weights` with the absolute path to the FP8 model weights directory. The imported model will be named `ling-tiny-fp8`:
+
+```bash
+printf 'FROM /absolute/path/to/fp8_weights\n' > /tmp/Modelfile.ling
+./ollama create ling-tiny-fp8 --experimental -f /tmp/Modelfile.ling
+```
+
+### Start Service
+Set the default context length to 8192, and then start the Ollama service:
+
+```bash
+# The service listens on http://127.0.0.1:11434 by default
+OLLAMA_CONTEXT_LENGTH=8192 ./ollama serve
+```
+
+### Call API
+```bash
+curl -sS http://127.0.0.1:11434/api/generate -d '{
+  "model": "ling-tiny-fp8",
+  "prompt": "<role>SYSTEM</role>detailed thinking on<|role_end|><role>HUMAN</role>Calculate 17 × 23 and output only the number.<|role_end|><role>ASSISTANT</role>\n<think>",
+  "raw": true,
+  "think": true,
+  "stream": false,
+  "options": {
+    "temperature": 1.0,
+    "top_p": 0.95,
+    "top_k": 20,
+    "num_predict": 2048
+  }
+}' | jq -r .response
+```