Text Generation
Transformers
Safetensors
qwen3_5_moe
image-text-to-text
Mixture of Experts
reasoning
efficient-reasoning
reinforcement-learning
conversational
Instructions to use TheFusionCube/Sage-A1.5-Concise with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TheFusionCube/Sage-A1.5-Concise with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="TheFusionCube/Sage-A1.5-Concise") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("TheFusionCube/Sage-A1.5-Concise") model = AutoModelForMultimodalLM.from_pretrained("TheFusionCube/Sage-A1.5-Concise", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use TheFusionCube/Sage-A1.5-Concise with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "TheFusionCube/Sage-A1.5-Concise" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheFusionCube/Sage-A1.5-Concise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/TheFusionCube/Sage-A1.5-Concise
- SGLang
How to use TheFusionCube/Sage-A1.5-Concise with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "TheFusionCube/Sage-A1.5-Concise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheFusionCube/Sage-A1.5-Concise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "TheFusionCube/Sage-A1.5-Concise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheFusionCube/Sage-A1.5-Concise", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use TheFusionCube/Sage-A1.5-Concise with Docker Model Runner:
docker model run hf.co/TheFusionCube/Sage-A1.5-Concise
Add quantized versions and a limitation from the 4-bit test
Browse files
README.md
CHANGED
|
@@ -17,6 +17,15 @@ Sage-A1.5-Concise is [InternScience/Sage-A1.5](https://huggingface.co/InternScie
|
|
| 17 |
|
| 18 |
It is a drop-in replacement for the base model. The architecture, tokenizer, chat template, vision encoder and MTP head are unchanged. Only `model-00012-of-00014.safetensors` differs from the base repository at revision `0e3205cf`.
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
## Results
|
| 21 |
|
| 22 |
Each benchmark compares the model with base Sage-A1.5 on the same tasks. The difference is in score points, with a paired bootstrap 95% interval over tasks. The base reference pools two or three independent base runs. The numbers come from fresh samples that were drawn after the checkpoint had been chosen. The draws used to choose it are not included.
|
|
@@ -35,6 +44,7 @@ Sampling followed the base model's recommendation: temperature 1.0, top_p 0.95,
|
|
| 35 |
|
| 36 |
- ChemCoTBench passes by a narrow margin. The upper bound of its interval is +0.02, and the point estimate is about 1 point below base. A loss of up to about 2 points is not ruled out on ChemCoTBench or CMT.
|
| 37 |
- Many training variants were scored on the same test sets during development, and this checkpoint was the second candidate to go through a confirmation round. The more often a test set is consulted, the more likely a model that is truly worse than base is to pass by chance.
|
|
|
|
| 38 |
- On IFBench about 7 in 600 responses repeat themselves verbatim until the token limit.
|
| 39 |
- Only five benchmarks were checked. Agentic, tool-use and long-context abilities of the base model were not evaluated.
|
| 40 |
- The MTP head was not trained. Speculative decoding still works, but its acceptance rate may be lower than with the base model. This was not measured.
|
|
|
|
| 17 |
|
| 18 |
It is a drop-in replacement for the base model. The architecture, tokenizer, chat template, vision encoder and MTP head are unchanged. Only `model-00012-of-00014.safetensors` differs from the base repository at revision `0e3205cf`.
|
| 19 |
|
| 20 |
+
## Quantized versions
|
| 21 |
+
|
| 22 |
+
| Repository | Format | Size | For |
|
| 23 |
+
|---|---|---|---|
|
| 24 |
+
| [Sage-A1.5-Concise-FP8](https://huggingface.co/TheFusionCube/Sage-A1.5-Concise-FP8) | compressed-tensors FP8, same recipe as the official [Sage-A1.5-FP8](https://huggingface.co/InternScience/Sage-A1.5-FP8) | 39 GB | vLLM, SGLang |
|
| 25 |
+
| [Sage-A1.5-Concise-GGUF](https://huggingface.co/TheFusionCube/Sage-A1.5-Concise-GGUF) | Q4_K_M, Q5_K_M, Q6_K, Q8_0, plus the vision projector | 22 to 38 GB | llama.cpp |
|
| 26 |
+
|
| 27 |
+
The benchmark results below were measured on this BF16 model. The quantized versions have not been benchmarked.
|
| 28 |
+
|
| 29 |
## Results
|
| 30 |
|
| 31 |
Each benchmark compares the model with base Sage-A1.5 on the same tasks. The difference is in score points, with a paired bootstrap 95% interval over tasks. The base reference pools two or three independent base runs. The numbers come from fresh samples that were drawn after the checkpoint had been chosen. The draws used to choose it are not included.
|
|
|
|
| 44 |
|
| 45 |
- ChemCoTBench passes by a narrow margin. The upper bound of its interval is +0.02, and the point estimate is about 1 point below base. A loss of up to about 2 points is not ruled out on ChemCoTBench or CMT.
|
| 46 |
- Many training variants were scored on the same test sets during development, and this checkpoint was the second candidate to go through a confirmation round. The more often a test set is consulted, the more likely a model that is truly worse than base is to pass by chance.
|
| 47 |
+
- The chains of thought are shorter, but not short. On hard problems a response can still run past 20,000 tokens, and much of that is checking an answer the model already found. In one 4-bit test it reached the right answer to a physics question 14% of the way through its reasoning and spent the rest verifying it.
|
| 48 |
- On IFBench about 7 in 600 responses repeat themselves verbatim until the token limit.
|
| 49 |
- Only five benchmarks were checked. Agentic, tool-use and long-context abilities of the base model were not evaluated.
|
| 50 |
- The MTP head was not trained. Speculative decoding still works, but its acceptance rate may be lower than with the base model. This was not measured.
|