netprojects/Cosmos3-Edge-mlx-8bit

Cosmos3-Edge reasoner for Apple Silicon (MLX, 8-bit)

An MLX port of the reasoner in nvidia/Cosmos3-Edge (revision 344d602). It takes text, images and video and answers in text, with optional thinking. This build quantizes the language model to 8-bit (affine, group 64) and keeps the SigLIP2 vision tower and projector in bf16. Size: 3.1 GB.

Published by NetProjects (Internet Projects Ltd). Not affiliated with NVIDIA.

Build	Size	Same next token as NVIDIA's reference	RealWorldQA (765)	CV-Bench (25% sample)
netprojects/Cosmos3-Edge-mlx-8bit (recommended)	3.1 GB	98.4%	69.2	87.0
netprojects/Cosmos3-Edge-mlx-6bit	2.6 GB	97.7%	68.6	87.3
bf16 (original weights, same code)	4.9 GB	98.7%	68.9	87.2
NVIDIA's published scores	—	—	70.5	85.3 (full set)

Measured on a MacBook Pro M5 Max (128 GB), 7 November 2026. "Same next token" compares each build with NVIDIA's model as implemented in transformers 5.22.2 (bf16, CPU) over 19 prompts (text, single and multiple images, video), 48 teacher-forced tokens each; mean KL over the top 64 tokens is 0.002 (8-bit) and 0.007 (6-bit). Benchmarks run with thinking off, greedy decoding and exact-answer scoring (letters, words and numbers); CV-Bench uses every fourth item of each source, scored the official way. NVIDIA's own evaluation settings are not published, so the scores show the port behaves like the model rather than reproduce NVIDIA's setup.

Use

git clone https://github.com/internet-projects/cosmos3-edge-mlx && cd cosmos3-edge-mlx
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -r requirements.txt
hf download netprojects/Cosmos3-Edge-mlx-8bit --local-dir cosmos3-edge-mlx-8bit

# one prompt
.venv/bin/python -m cosmos3_edge_mlx.generate --model cosmos3-edge-mlx-8bit --image photo.jpg --prompt "Describe the scene."
.venv/bin/python -m cosmos3_edge_mlx.generate --model cosmos3-edge-mlx-8bit --video clip.mp4 --fps 4 --prompt "What happens?"

# OpenAI-compatible server: /v1/chat/completions with text, image_url and video_url parts, streaming
.venv/bin/python -m cosmos3_edge_mlx.server --model cosmos3-edge-mlx-8bit --port 8796

Thinking is on by default, as in NVIDIA's chat template; turn it off with --no-think or "chat_template_kwargs": {"enable_thinking": false}. The server returns the thinking text as reasoning_content. Input preprocessing uses NVIDIA's own processor from transformers (5.22.2 or later), so image resizing, frame sampling and timestamps match the original. Videos are decoded with PyAV. Memory: the MLX buffer cache is capped (COSMOS_CACHE_LIMIT_GB, default 2) and cleared after each request, and COSMOS_MEMORY_LIMIT_GB (default 24) sets a memory limit. With the 8-bit build, a 16K-token prompt peaked at 5.5 GB and a 32K-token prompt at 7.3 GB. Long-context answers match NVIDIA's code: in a needle test up to 8K tokens both give the same answer on every prompt, and at 16K the port finds the needle on the same prompts as NVIDIA's code or more.

How it works

• Language model: 28-layer decoder (hidden 2048, 16 query / 8 key-value heads, ReLU² MLP, vocabulary 131,072) with interleaved 3D M-RoPE (sections 24/20/20, theta 1e8), as in NVIDIA's code.
• Vision: SigLIP2-style tower (27 layers, patch 16) at native resolution. The learned 16×16 position grid is resized per image or frame with the same antialiased bilinear filter as the original, and attention stays inside each image or frame. A 2×2 patch merger projects to the language model.
• Video: frames are encoded one by one with NVIDIA's timestamps between them, and positions follow NVIDIA's get_rope_index.
• Reasoner only: the checkpoint also holds Cosmos3-Edge's video generator. Those tensors are not part of this build.

Build and check it yourself

hf download nvidia/Cosmos3-Edge --revision 344d602b128d1bbdacb43b08d0a3626f46343e29 --local-dir cosmos3-edge-hf \
  --include "*.json" --include "*.jinja" --include "*.md" --include "LICENSE*" --include "NOTICE*" \
  --include "transformer/*" --include "vision_encoder/*"
.venv/bin/python convert.py --src cosmos3-edge-hf --dst cosmos3-edge-mlx-8bit --bits 8      # or --bits 6
.venv/bin/python tests/test_tiny.py          # tiny random model vs transformers' Cosmos3-Edge, no real weights
.venv/bin/python parity.py ref --model cosmos3-edge-hf --out ref.npz     # NVIDIA's model in transformers, CPU
.venv/bin/python parity.py mlx --model cosmos3-edge-mlx-8bit --ref ref.npz --out q8.json && .venv/bin/python parity.py report q8.json
.venv/bin/python bench.py --model cosmos3-edge-mlx-8bit --data evaldata --out bench-q8.json

License

Cosmos3-Edge is released by NVIDIA under the OpenMDW-1.1 license, and these conversions keep it: LICENSE is the OpenMDW-1.1 text and NOTICE describes the conversion. NVIDIA's model documents (BIAS.md, EXPLAINABILITY.md, PRIVACY.md, SAFETY.md) are included unchanged.

Credits: NVIDIA (Cosmos3-Edge), Hugging Face transformers (reference implementation and processor), Apple MLX. Port by Paris Michalas, Partner at NetProjects. Not affiliated with NVIDIA.