README.md

---
license: mit
library_name: mlx
pipeline_tag: text-generation
language:
- en
base_model: deepreinforce-ai/Ornith-1.0-397B
base_model_relation: quantized
tags:
- mlx
- qwen3_5_moe
- moe
---

# Ornith-1.0-397B-mlx-4bit

This is an **MLX** conversion of [deepreinforce-ai/Ornith-1.0-397B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-397B), quantized to **4-bit** for use on Apple Silicon with [mlx-lm](https://github.com/ml-explore/mlx-lm).

- **Base model:** [deepreinforce-ai/Ornith-1.0-397B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-397B) (Qwen3.5-MoE, `Qwen3_5MoeForConditionalGeneration`, 397B total / MoE)
- **Format:** MLX, 4-bit (affine)
- **Approx. size on disk:** ~223 GB
- **Converted with:** mlx-lm 0.31.2

> **Note — text-only.** The original Ornith-1.0-397B is multimodal (vision encoder + language model). mlx-lm converts the **language model only**; the vision tower is not included. This build is for **text generation**. The tokenizer, chat template, and `generation_config` are included.

## Requirements

This is a large MoE model. You need an Apple Silicon Mac with enough unified memory to hold the weights (roughly **~223 GB** plus runtime overhead/KV cache). A 512 GB M3 Ultra runs all of these comfortably.

## Usage

```bash
pip install -U mlx-lm
```

```bash
mlx_lm.generate --model pipenetwork/Ornith-1.0-397B-mlx-4bit \
  --prompt "Write a haiku about Apple Silicon." --max-tokens 256
```

```python
from mlx_lm import load, generate

model, tokenizer = load("pipenetwork/Ornith-1.0-397B-mlx-4bit")
messages = [{"role": "user", "content": "Explain mixture-of-experts in one paragraph."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True))
```

## License

MIT, inherited from the base model.