Instructions to use rdbell/Sage-S2.1-Flash-oQ4e-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use rdbell/Sage-S2.1-Flash-oQ4e-mtp with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("rdbell/Sage-S2.1-Flash-oQ4e-mtp") config = load_config("rdbell/Sage-S2.1-Flash-oQ4e-mtp") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use rdbell/Sage-S2.1-Flash-oQ4e-mtp with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "rdbell/Sage-S2.1-Flash-oQ4e-mtp"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "rdbell/Sage-S2.1-Flash-oQ4e-mtp" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use rdbell/Sage-S2.1-Flash-oQ4e-mtp with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "rdbell/Sage-S2.1-Flash-oQ4e-mtp"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default rdbell/Sage-S2.1-Flash-oQ4e-mtp
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use rdbell/Sage-S2.1-Flash-oQ4e-mtp with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "rdbell/Sage-S2.1-Flash-oQ4e-mtp"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "rdbell/Sage-S2.1-Flash-oQ4e-mtp" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Sage-S2.1-Flash-oQ4e-mtp
A mixed-precision MLX checkpoint of internlm/Sage-S2.1-Flash (Shanghai AI Laboratory / InternLM), a fine-tune of GLM-5.3-Flash with the same architecture: a natively multimodal mixture-of-experts model with 320B total and 18B active parameters.
It was quantized with oQ (oMLX v0.7.4.dev1) from the official FP8 release, internlm/Sage-S2.1-Flash-FP8 at revision 716e6436905650b7be93e1b7dedfb0bb95afd3a3, using the same recipe as Jundot/GLM-5.3-Flash-oQ4e-mtp: oQ4e, with the MTP head preserved.
The effective average is 4.641 bits/weight for the complete checkpoint, the same layout as that GLM pack.
Quantization and Bit Distribution
oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.
One difference from the GLM pack's build: oMLX 0.7.4.dev1 streams this architecture's calibration one decoder layer at a time from the FP8 source, where the GLM pack (oMLX v0.7.3.dev1) calibrated on a temporary 4-bit proxy.
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| Affine 4-bit, group 64 | 311.654B | 96.991% | 163.266 GiB | 4.50 |
| Affine 8-bit, group 64 | 8.976B | 2.793% | 8.882 GiB | 8.50 |
| BF16 | 0.607B | 0.189% | 1.131 GiB | 16.00 |
| FP32 | 0.086B | 0.027% | 0.322 GiB | 32.00 |
| Total | 321.323B | 100% | 173.600 GiB | 4.641 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language backbone | 313.327B | 180.966 GB / 168.538 GiB |
| Vision encoder | 0.564B | 1.127 GB / 1.050 GiB |
| MTP head | 7.433B | 4.309 GB / 4.013 GiB |
| Total | 321.323B | 186.402 GB / 173.600 GiB |
Notes
- Serving: built with oMLX 0.7.4.dev1 and served with it on a 256 GB M3 Ultra (MTP on, chunked prefill).
- Quality: we have not measured this pack against the original FP8 or BF16 weights.
- Chat template: Sage's own. Unlike GLM-5.3-Flash's, it keeps earlier turns' reasoning in the conversation history, so multi-turn agent prompts are longer.
- Not included: Sage's scientific-input projectors (molecules, crystals, DNA/RNA, spectra, EEG). They run outside the language model; see the source repository.
License
MIT, following the source model. Sage-S2.1-Flash's code and weights are released under the MIT License by Shanghai AI Laboratory / InternLM. Sage-S2.1-Flash is built on GLM-5.3-Flash, whose MIT license notice (Z.AI) is included as LICENSE.
- Downloads last month
- 71
4-bit
Model tree for rdbell/Sage-S2.1-Flash-oQ4e-mtp
Base model
internlm/Sage-S2.1-Flash