---
license: apache-2.0
tags:
- vision
- moondream
- mlx
- int8
- quantized
base_model: moondream/moondream3-preview
---

# MD3P-Int8 - INT8 Quantized Moondream3 for MLX

An INT8 quantized version of Moondream3, offering a balance between model quality and size for MLX deployment.

## Model Details

| Component | Original (BF16) | This Model |
|-----------|-----------------|------------|
| MoE Experts (layers 4-23) | BF16 | **int8** |
| Vision Encoder | BF16 | BF16 (preserved) |
| Text Attention | BF16 | **int8** |
| Text MLP (layers 0-3) | BF16 | **int8** |
| Embeddings | BF16 | BF16 (preserved) |
| **Total Size** | ~12 GB | **~10 GB** |

## Quantization Details

- **Method**: Affine quantization (bits=8, group_size=64)
- **Target**: Text model layers (attention, MLP, MoE experts)
- **Preserved**: Vision encoder and embeddings at BF16 for quality

## Comparison with INT4 Variants

| Model | Size | Quality | Use Case |
|-------|------|---------|----------|
| md3p-int8 (this) | 10 GB | Higher | Desktop/Server MLX |
| md3p-int4 | 6.48 GB | Medium | Memory-constrained |
| md3p-int4-smol | 5.43 GB | Lower | iOS (~6GB limit) |

## Usage

This model is designed for use with MLX-based Moondream implementations.

```python
# Example with mlx-lm or similar
from mlx_lm import load, generate

model, tokenizer = load("lewi/md3p-int8")
```

## Source & License

- **Original Model**: [moondream/moondream3-preview](https://huggingface.co/moondream/moondream3-preview)
- **License**: Apache 2.0 (same as original)

## Acknowledgments

Thanks to the [Moondream](https://moondream.ai/) team for the original model and Apache 2.0 license.