---
license: other
license_name: openmdw1.1-license
license_link: https://openmdw.ai/license/1-1/
base_model: nvidia/Cosmos3-Edge
pipeline_tag: image-text-to-text
library_name: mlx
tags:
  - mlx
  - cosmos
  - vision-language
  - video
  - apple-silicon
---

# Cosmos3-Edge reasoner for Apple Silicon (MLX, 8-bit)

An MLX port of the reasoner in [nvidia/Cosmos3-Edge](https://huggingface.co/nvidia/Cosmos3-Edge) (revision `344d602`). It takes
text, images and video and answers in text, with optional thinking. This build quantizes the language model to 8-bit
(affine, group 64) and keeps the SigLIP2 vision tower and projector in bf16. Size: 3.1 GB.

Published by [NetProjects](https://netprojects.com) (Internet Projects Ltd). Not affiliated with NVIDIA.

| Build | Size | Same next token as NVIDIA's reference | RealWorldQA (765) | CV-Bench (25% sample) |
|---|---|---|---|---|
| [netprojects/Cosmos3-Edge-mlx-8bit](https://huggingface.co/netprojects/Cosmos3-Edge-mlx-8bit) (recommended) | 3.1 GB | 98.4% | 69.2 | 87.0 |
| [netprojects/Cosmos3-Edge-mlx-6bit](https://huggingface.co/netprojects/Cosmos3-Edge-mlx-6bit) | 2.6 GB | 97.7% | 68.6 | 87.3 |
| bf16 (original weights, same code) | 4.9 GB | 98.7% | 68.9 | 87.2 |
| NVIDIA's published scores | — | — | 70.5 | 85.3 (full set) |

Measured on a MacBook Pro M5 Max (128 GB), 7 November 2026. "Same next token" compares each build with NVIDIA's model as
implemented in transformers 5.22.2 (bf16, CPU) over 19 prompts (text, single and multiple images, video), 48 teacher-forced
tokens each; mean KL over the top 64 tokens is 0.002 (8-bit) and 0.007 (6-bit). Benchmarks run with thinking off, greedy
decoding and exact-answer scoring (letters, words and numbers); CV-Bench uses every fourth item of each source, scored the official way. NVIDIA's own
evaluation settings are not published, so the scores show the port behaves like the model rather than reproduce NVIDIA's setup.

## Use

```bash
git clone https://github.com/internet-projects/cosmos3-edge-mlx && cd cosmos3-edge-mlx
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -r requirements.txt
hf download netprojects/Cosmos3-Edge-mlx-8bit --local-dir cosmos3-edge-mlx-8bit

# one prompt
.venv/bin/python -m cosmos3_edge_mlx.generate --model cosmos3-edge-mlx-8bit --image photo.jpg --prompt "Describe the scene."
.venv/bin/python -m cosmos3_edge_mlx.generate --model cosmos3-edge-mlx-8bit --video clip.mp4 --fps 4 --prompt "What happens?"

# OpenAI-compatible server: /v1/chat/completions with text, image_url and video_url parts, streaming
.venv/bin/python -m cosmos3_edge_mlx.server --model cosmos3-edge-mlx-8bit --port 8796
```

Thinking is on by default, as in NVIDIA's chat template; turn it off with `--no-think` or
`"chat_template_kwargs": {"enable_thinking": false}`. The server returns the thinking text as `reasoning_content`.
Input preprocessing uses NVIDIA's own processor from transformers (5.22.2 or later), so image resizing, frame sampling and
timestamps match the original. Videos are decoded with PyAV.
Memory: the MLX buffer cache is capped (`COSMOS_CACHE_LIMIT_GB`, default 2) and cleared after each request, and
`COSMOS_MEMORY_LIMIT_GB` (default 24) sets a memory limit. With the 8-bit build, a 16K-token prompt peaked at 5.5 GB and a 32K-token prompt at 7.3 GB. Long-context answers
match NVIDIA's code: in a needle test up to 8K tokens both give the same answer on every prompt, and at 16K the port
finds the needle on the same prompts as NVIDIA's code or more.

## How it works

- **Language model:** 28-layer decoder (hidden 2048, 16 query / 8 key-value heads, ReLU² MLP, vocabulary 131,072) with
  interleaved 3D M-RoPE (sections 24/20/20, theta 1e8), as in NVIDIA's code.
- **Vision:** SigLIP2-style tower (27 layers, patch 16) at native resolution. The learned 16×16 position grid is resized per
  image or frame with the same antialiased bilinear filter as the original, and attention stays inside each image or frame.
  A 2×2 patch merger projects to the language model.
- **Video:** frames are encoded one by one with NVIDIA's timestamps between them, and positions follow NVIDIA's
  `get_rope_index`.
- **Reasoner only:** the checkpoint also holds Cosmos3-Edge's video generator. Those tensors are not part of this build.

## Build and check it yourself

```bash
hf download nvidia/Cosmos3-Edge --revision 344d602b128d1bbdacb43b08d0a3626f46343e29 --local-dir cosmos3-edge-hf \
  --include "*.json" --include "*.jinja" --include "*.md" --include "LICENSE*" --include "NOTICE*" \
  --include "transformer/*" --include "vision_encoder/*"
.venv/bin/python convert.py --src cosmos3-edge-hf --dst cosmos3-edge-mlx-8bit --bits 8      # or --bits 6
.venv/bin/python tests/test_tiny.py          # tiny random model vs transformers' Cosmos3-Edge, no real weights
.venv/bin/python parity.py ref --model cosmos3-edge-hf --out ref.npz     # NVIDIA's model in transformers, CPU
.venv/bin/python parity.py mlx --model cosmos3-edge-mlx-8bit --ref ref.npz --out q8.json && .venv/bin/python parity.py report q8.json
.venv/bin/python bench.py --model cosmos3-edge-mlx-8bit --data evaldata --out bench-q8.json
```

## License

Cosmos3-Edge is released by NVIDIA under the [OpenMDW-1.1](https://openmdw.ai/license/1-1/) license, and these conversions
keep it: `LICENSE` is the OpenMDW-1.1 text and `NOTICE` describes the conversion.
NVIDIA's model documents (`BIAS.md`, `EXPLAINABILITY.md`, `PRIVACY.md`, `SAFETY.md`) are included unchanged.

Credits: NVIDIA (Cosmos3-Edge), Hugging Face transformers (reference implementation and processor), Apple MLX.
Port by Paris Michalas, Partner at [NetProjects](https://netprojects.com). Not affiliated with NVIDIA.