midium-ai/elephant-1-8bit

elephant-1 — MLX 8-bit

An 8-bit MLX quantization of baselabs/elephant-1 (revision b3f8a93462fb1051a9de031e94a7ff380e254072): baselabs' 9B typed decision model (a full fine-tune of Qwen3.5-9B). Give it a state and typed questions (choice, yes/no, score) and it returns a probability for every option without generating text. Built and served by Midium's System One runtime (POST /v1/systemone).

What was quantized

• Language model: affine 8-bit, group size 64, round-to-nearest (no calibration data).
• Kept as in the source: nothing else: the vision tower is not part of this text-only build.
• Tokenizer, chat template and the decision configuration are unchanged.

Weights: 9.5 GB (bf16 source: 18.8 GB).

Evaluation against the bf16 model

Same runtime, same requests, question by question, on a seeded sample of up to 150 questions per set.

Suite	Questions	Top answer agrees with bf16	Accuracy change per set	Answers lost / gained
14 decision sets (routing, reply checks, tool preselection, model routing)	2,058	99.2–100%	-0.7 to +0.9 pts	1 / 1
Public sets, 150-way intent choice, 0–9 scores, option-shuffle and typo robustness	1,720	99.2–100%	-0.8 to +0.7 pts	1 / 1

Mean total-variation distance between the 8-bit and bf16 answer distributions: at most 0.008 per set.

Usage

This checkpoint keeps the source's decision protocol — OneJev's qev protocol: the answer read from the LM-head logits of the slot labels (A.., 256 slots), with block-bidirectional full attention (modeling_overrides in config.json, readout.json). It needs a runtime that implements that protocol: Midium's System One runtime loads it directly; generic MLX text-generation loaders do not.

License

MIT, as the source model.