AndyCarrot/MagenticBrain-PrismaAURA-5bit

MagenticBrain PrismaAURA 5-bit

Mixed-precision PrismaAURA export of microsoft/MagenticBrain for NVIDIA Blackwell and stock vLLM compressed-tensors serving.

MagenticBrain is a 14B orchestration model from Microsoft Research AI Frontiers. It plans multi-step tasks, selects declared tools, and coordinates sub-agents inside a host-controlled execution harness. This repository changes the weight representation only; it does not add or retrain capabilities.

Official MagenticLite model pairing

Microsoft Magentic-UI is the officially recommended host application for this two-model stack. Microsoft describes MagenticLite as pairing an on-device-friendly orchestrator model, MagenticBrain, with a specialized browser-use model, Fara.

Use these two PrismaAURA repositories together for that pairing:

• Orchestrator model: MagenticBrain PrismaAURA 5-bit (this repository)
• Browser-use model: Fara1.5-9B PrismaAURA 5-bit
• Official host application: microsoft/magentic-ui

These are independently produced quantized derivatives of the Microsoft models; Magentic-UI, MagenticBrain, and Fara are the official Microsoft components and model roles.

Quantization

• Method: PrismaAURA mixed-precision allocation
• Container: vLLM compressed-tensors
• Format menu: NVFP4 + FP8 E4M3
• Selected point: 5.000625 bits per quantizable parameter
• Assignment: 212 NVFP4 and 68 FP8 E4M3 Linears
• Held-out real KL vs source model: 0.031031487
• Source: 14,768,307,200 parameters, F32 checkpoint
• Export: approximately 13.49 GiB across four safetensors shards

Actual bpp	Held-out KL vs source	NVFP4	FP8 E4M3
4.750357	0.036007880	248	32
5.000625	0.031031487	212	68
5.249561	0.027971407	181	99

PrismaAURA bpp covers allocator-eligible parameters. Embeddings, the LM head, norms, and other fixed/pass-through tensors are excluded, so the physical whole-checkpoint average is higher than 5 bpp.

Text calibration used 32 samples of 1,024 tokens. Frontier validation used a held-out split of 8 samples at 512 tokens. The source checkpoint has no MTP weights, so this artifact does not provide MTP speculative decoding.

See QUANTIZATION.md for the detailed production record.

Recommended deployment

Microsoft recommends using MagenticBrain with MagenticLite or another host-controlled tool-execution harness. If integrating it directly, the host must declare the tool schema, parse tool calls, execute actions under scoped permissions, feed tool results back, and stop on the submit terminator.

vLLM

vllm serve AndyCarrot/MagenticBrain-PrismaAURA-5bit \
  --quantization compressed-tensors \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --default-chat-template-kwargs '{"enable_thinking":false}' \
  --max-model-len 32768

The included launch.sh manages a local vLLM server in tmux:

./launch.sh start
./launch.sh status
./launch.sh stop
./launch.sh restart

The artifact was load-tested with vLLM 0.24.0 on an NVIDIA GB10. vLLM selected FlashInferCutlassNvFp4LinearKernel for NVFP4 and CutlassFP8ScaledMMLinearKernel for FP8, then completed a generation smoke test.

Recommended request settings

{
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "min_p": 0,
  "presence_penalty": 1,
  "chat_template_kwargs": {"enable_thinking": false}
}

Do not use greedy decoding; the source model card warns that it can cause repetition on this model family.

Intended use and safety

This model has no execution authority by itself. All actions must pass through a host-owned tool layer. Use least-privilege tool schemas, sandbox execution, surface plans and tool calls, validate tool outputs, and require human confirmation for sensitive or irreversible operations.

The intended uses, limitations, responsible-AI considerations, and execution boundary of the original model continue to apply. Consult the official model card.

Attribution and license

The original MagenticBrain model was developed by Microsoft Research AI Frontiers and released under the MIT License. This repository is an independently produced quantized derivative and is not an official Microsoft release. The original MIT license and model terms continue to apply; no additional restrictions are introduced here.

• Original model: https://huggingface.co/microsoft/MagenticBrain
• Base model: https://huggingface.co/Qwen/Qwen3-14B
• Quantization project: https://github.com/RobTand/prismaquant