README.md

---
library_name: mlx
license: mit
license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model: deepreinforce-ai/Ornith-1.0-9B
base_model_relation: quantized
tags:
- mlx
- quantized
- mixed-precision
- 6bit
- 8bit
- optiq
- apple-silicon
- image-text-to-text
- vision-language
- qwen3.5
---

# codelion/Ornith-1.0-9B-OptiQ-6bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Try the Lab](https://mlx-optiq.com/docs/lab/) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

A 6-bit mixed-precision MLX quant of [deepreinforce-ai/Ornith-1.0-9B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B), built on the Qwen3.5-9B architecture. Sensitive layers are kept at 8-bit and robust ones at 4-bit.

17.6 GB of bf16 weights become **8.0 GB**, which fits a 16 GB Mac.

**Image input works.** The vision tower is kept at bf16 in a sidecar, so this quant takes images as well as text.

## Quantization details

| Property | Value |
|---|---|
| Predominant precision | 6-bit |
| Layers at 8-bit (sensitive) | 145 |
| Layers at 4-bit (robust) | 104 |
| Total quantized layers | 249 |
| Group size | 64 |
| Vision tower | bf16, 333 tensors, in `optiq/optiq_vision.safetensors` |
| Size on disk | 8.0 GB, from a 17.6 GB bf16 base |

We follow the same naming convention `llama.cpp` uses for Q6_K and similar mixed-precision quants: the "6-bit" label is the predominant precision, not the weighted average.

The base model ships no MTP head, so this quant has no speculative-decoding sidecar.

### How the bit-widths were chosen

Every layer was measured directly on this model. Each (layer, bit-width) pair was scored by KL divergence against the bf16 reference on a [six-domain calibration mix](https://mlx-optiq.com/blog/calibration-mix), and a knapsack solver spent the bit budget where the measured error was largest.

The full sweep ships with the model as `optiq/sensitivity.json`: 249 layers scored at both candidate widths. That is the measurement, not just the outcome, so this architecture can be re-quantized at another target without repeating it.

Only the language tower is quantized. The vision tower stays at bf16, which is how every OptiQ VLM ships.

## Usage

### Text

Everything OptiQ-specific lives in an `optiq/` subfolder, so a stock `*.safetensors` glob ignores it and `mlx-lm` sees a clean language model.

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("codelion/Ornith-1.0-9B-OptiQ-6bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain the difference between TCP and UDP."}],
    add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```

This is a reasoning model: it thinks inside `<think>...</think>` before answering, so give it enough `max_tokens` to finish.

### Images

Image input needs [`mlx-optiq`](https://mlx-optiq.com/), which loads the bf16 vision sidecar and feeds the merged embeddings to the quantized language tower:

```bash
pip install mlx-optiq
```

```python
from PIL import Image
from optiq.runtime.engine import OptiqEngine

engine = OptiqEngine("codelion/Ornith-1.0-9B-OptiQ-6bit")
answer = engine.generate("What is in this image?",
                         images=[Image.open("photo.jpg")], max_tokens=512)
print(answer.text)
```

Or serve it over an OpenAI-compatible endpoint that accepts image content parts:

```bash
optiq serve --model codelion/Ornith-1.0-9B-OptiQ-6bit
```

## Verification

Text, arithmetic reasoning, and image understanding were all exercised on the finished artifact before release.

No task benchmarks were run on this quant; for measured quality numbers on the base architecture, see the [Qwen3.5-9B OptiQ card](https://huggingface.co/mlx-community/Qwen3.5-9B-OptiQ-4bit).

Quantization does not change the behaviour or alignment of the base model. Use it under the same terms as [the original](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B).