inclusionAI

/

Ling-3.0-flash-fp8

like 21

Follow

inclusionAI 2.39k

Summary

Replaces the Quickstart → SGLang section with the recipe from the official SGLang cookbook, now that first-class Ling-3.0-flash support has landed in upstream SGLang:

• Cookbook page: https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ling-3.0-flash
• Upstream cookbook PR: https://github.com/sgl-project/sglang/pull/33556 (merged)
• Runtime support in upstream SGLang: https://github.com/sgl-project/sglang/pull/33561

Changes:

• Install instructions point at the pre-built lmsysorg/sglang:dev-Ling-3.0-flash image instead of building from the inclusionAI/sglang_ling_v3 fork.
• Launch command follows the cookbook Low-Latency cell (NEXTN + 256K YaRN + --mem-fraction-static 0.8), with hardware-specific TP/EP variants documented in the cookbook.
• FP8 card: uses the TP4+EP4 recipe, replacing the earlier TP2 workaround (--disable-shared-experts-fusion) that predated the expert-parallel fix.
• Notes that thinking is enabled by default and can be disabled per request via chat_template_kwargs.

The vLLM section and the rest of the model card are unchanged.