Sage-A1.5-Concise FP8

FP8 quantization of TheFusionCube/Sage-A1.5-Concise. That model is Sage-A1.5 after reinforcement learning to shorten its chain of thought, with 11 to 26% fewer tokens on five benchmarks and no significant loss on any of them. See its card for results and limitations.

The quantization follows InternScience/Sage-A1.5-FP8 exactly. It uses the same compressed-tensors config, with FP8 E4M3 weights, per-channel BF16 scales and dynamic per-token FP8 activations. It quantizes the same layers: routed experts, the shared expert and full-attention projections. Linear attention, routers, embeddings, the LM head and the vision encoder stay in BF16. Applied to the base model, this recipe reproduces the official FP8 weights bit for bit, so tensors the training did not change are identical to the official FP8 release. The file layout matches it too, with the same tensor names, dtypes, shapes and offsets. The MTP head was not trained, so model_mtp.safetensors is byte-identical to the official one. The tokenizer and config files are copied from the official FP8 release.

Files

File Contents Size
model-00001-of-00002.safetensors vision encoder, embeddings, language layers 0 to 20 20.0 GB
model-00002-of-00002.safetensors language layers 20 to 39, final norm, LM head 17.7 GB
model_mtp.safetensors MTP head for speculative decoding 1.7 GB

Status

The benchmark numbers on the main card were measured on the BF16 model. This FP8 version has not been benchmarked, and it has not yet been loaded in vLLM or SGLang. Because its config and layout match the official FP8 release tensor for tensor, it should load wherever that release does. A 4-bit GGUF build of the same model was tested against the base model and is described on the GGUF card.

Usage

hf download TheFusionCube/Sage-A1.5-Concise-FP8 --local-dir Sage-A1.5-Concise-FP8

Or serve it straight from the Hub:

vllm serve TheFusionCube/Sage-A1.5-Concise-FP8 \
  --port 8000 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Use the base model's recommended sampling: temperature 1.0, top_p 0.95, top_k 20, presence_penalty 1.5.

See the base model card for SGLang, MTP, long-context and agentic settings.

Downloads last month
34
Safetensors
Model size
36B params
Tensor type
BF16
ยท
F8_E4M3
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for TheFusionCube/Sage-A1.5-Concise-FP8

Quantized
(4)
this model