Instructions to use TheFusionCube/Sage-A1.5-Concise-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TheFusionCube/Sage-A1.5-Concise-FP8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="TheFusionCube/Sage-A1.5-Concise-FP8") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("TheFusionCube/Sage-A1.5-Concise-FP8") model = AutoModelForMultimodalLM.from_pretrained("TheFusionCube/Sage-A1.5-Concise-FP8", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use TheFusionCube/Sage-A1.5-Concise-FP8 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "TheFusionCube/Sage-A1.5-Concise-FP8" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheFusionCube/Sage-A1.5-Concise-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/TheFusionCube/Sage-A1.5-Concise-FP8
- SGLang
How to use TheFusionCube/Sage-A1.5-Concise-FP8 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "TheFusionCube/Sage-A1.5-Concise-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheFusionCube/Sage-A1.5-Concise-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "TheFusionCube/Sage-A1.5-Concise-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheFusionCube/Sage-A1.5-Concise-FP8", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use TheFusionCube/Sage-A1.5-Concise-FP8 with Docker Model Runner:
docker model run hf.co/TheFusionCube/Sage-A1.5-Concise-FP8
Sage-A1.5-Concise FP8
FP8 quantization of TheFusionCube/Sage-A1.5-Concise. That model is Sage-A1.5 after reinforcement learning to shorten its chain of thought, with 11 to 26% fewer tokens on five benchmarks and no significant loss on any of them. See its card for results and limitations.
The quantization follows InternScience/Sage-A1.5-FP8 exactly. It uses the same compressed-tensors config, with FP8 E4M3 weights, per-channel BF16 scales and dynamic per-token FP8 activations. It quantizes the same layers: routed experts, the shared expert and full-attention projections. Linear attention, routers, embeddings, the LM head and the vision encoder stay in BF16. Applied to the base model, this recipe reproduces the official FP8 weights bit for bit, so tensors the training did not change are identical to the official FP8 release. The file layout matches it too, with the same tensor names, dtypes, shapes and offsets. The MTP head was not trained, so model_mtp.safetensors is byte-identical to the official one. The tokenizer and config files are copied from the official FP8 release.
Files
| File | Contents | Size |
|---|---|---|
model-00001-of-00002.safetensors |
vision encoder, embeddings, language layers 0 to 20 | 20.0 GB |
model-00002-of-00002.safetensors |
language layers 20 to 39, final norm, LM head | 17.7 GB |
model_mtp.safetensors |
MTP head for speculative decoding | 1.7 GB |
Status
The benchmark numbers on the main card were measured on the BF16 model. This FP8 version has not been benchmarked, and it has not yet been loaded in vLLM or SGLang. Because its config and layout match the official FP8 release tensor for tensor, it should load wherever that release does. A 4-bit GGUF build of the same model was tested against the base model and is described on the GGUF card.
Usage
hf download TheFusionCube/Sage-A1.5-Concise-FP8 --local-dir Sage-A1.5-Concise-FP8
Or serve it straight from the Hub:
vllm serve TheFusionCube/Sage-A1.5-Concise-FP8 \
--port 8000 \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder
Use the base model's recommended sampling: temperature 1.0, top_p 0.95, top_k 20, presence_penalty 1.5.
See the base model card for SGLang, MTP, long-context and agentic settings.
- Downloads last month
- 34
Model tree for TheFusionCube/Sage-A1.5-Concise-FP8
Base model
InternScience/Sage-A1.5