| --- |
| license: other |
| license_name: nvidia-open-model-license |
| library_name: mlx |
| pipeline_tag: image-feature-extraction |
| base_model: |
| - nvidia/C-RADIOv4-SO400M |
| - nvidia/C-RADIOv4-H |
| tags: |
| - mlx |
| - c-radio |
| - vision |
| - embeddings |
| - quantized |
| - apple-silicon |
| --- |
| |
| # C-RADIOv4 Quantized MLX Bundles |
|
|
| This repository contains self-contained MLX bundles for NVIDIA C-RADIOv4 image embedding |
| models on Apple Silicon. |
|
|
| Implementation repository: |
|
|
| https://github.com/stephansturges/c-radio_v4_MLX |
|
|
| ## Repository Structure |
|
|
| | Path | Model | Format | Status | |
| | --- | --- | --- | --- | |
| | `so400m/8bit-affine` | `nvidia/C-RADIOv4-SO400M` | 8-bit affine, group size 64 | Compact/high-precision, not a throughput tier | |
| | `h/8bit-affine` | `nvidia/C-RADIOv4-H` | 8-bit affine, group size 64 | Compact/high-precision, not a throughput tier | |
| | `so400m/cider-w8a8` | `nvidia/C-RADIOv4-SO400M` | Cider W8A8, per-channel | M5+ compact/runtime low-bit | |
| | `h/cider-w8a8` | `nvidia/C-RADIOv4-H` | Cider W8A8, per-channel | M5+ compact/runtime low-bit | |
| | `so400m/cider-w8a8-g128` | `nvidia/C-RADIOv4-SO400M` | Cider W8A8, group size 128 | M5+ balanced precision/speed | |
| | `h/cider-w8a8-g128` | `nvidia/C-RADIOv4-H` | Cider W8A8, group size 128 | M5+ balanced precision/speed | |
| | `so400m/cider-w8a8-p9999` | `nvidia/C-RADIOv4-SO400M` | Cider W8A8, 99.99 percentile clip | M5+ fastest experimental | |
| | `h/cider-w8a8-p9999` | `nvidia/C-RADIOv4-H` | Cider W8A8, 99.99 percentile clip | M5+ fastest experimental | |
| | `so400m/mxfp8` | `nvidia/C-RADIOv4-SO400M` | `mxfp8`, group size 32 | Experimental/lower precision, not recommended | |
| | `h/mxfp8` | `nvidia/C-RADIOv4-H` | `mxfp8`, group size 32 | Experimental/lower precision, not recommended | |
|
|
| Each subdirectory contains: |
|
|
| - `model.safetensors` |
| - `manifest.json` |
| - upstream config and preprocessor metadata |
| - a subdirectory `README.md` with model-specific provenance, measurements, and usage |
|
|
| ## Source Models |
|
|
| - `nvidia/C-RADIOv4-SO400M`: https://huggingface.co/nvidia/C-RADIOv4-SO400M |
| - Revision: `c0457f5dc26ca145f954cd4fc5bb6114e5705ad8` |
| - `nvidia/C-RADIOv4-H`: https://huggingface.co/nvidia/C-RADIOv4-H |
| - Revision: `0057b339059c0b9e1b4ba996f975410ebbfdfcc8` |
|
|
| ## Accuracy Summary |
|
|
| Measured against local bf16 MLX bundles at `512x512` on 12 WALDO crop images. |
|
|
| | Bundle | Summary cosine mean/min | Spatial cosine mean/min | |
| | --- | ---: | ---: | |
| | `so400m/8bit-affine` | 0.999907 / 0.999868 | 0.999930 / 0.999876 | |
| | `h/8bit-affine` | 0.999899 / 0.999878 | 0.999830 / 0.999764 | |
| | `so400m/mxfp8` | 0.989820 / 0.950717 | 0.993502 / 0.977879 | |
| | `h/mxfp8` | 0.990217 / 0.974710 | 0.988696 / 0.976071 | |
|
|
| The 8-bit affine bundles are the recommended compact/high-precision artifacts, not the |
| throughput tier. They are packed weight-only MLX artifacts and are not dequantized back to |
| dense bf16 at load time. Cider W8A8 is the real weight/activation low-bit runtime path for |
| Apple M5+ machines and trades a little more embedding drift for lower memory and modest |
| speedups in some cells. The `mxfp8` bundles are included for experimentation and are lower |
| precision in these checks. |
|
|
| Smoke-image Cider W8A8 precision versus local bf16 MLX at `512x512`: |
|
|
| | Bundle | Summary cosine | Spatial cosine | |
| | --- | ---: | ---: | |
| | `so400m/cider-w8a8` | 0.998164 | 0.998889 | |
| | `h/cider-w8a8` | 0.997202 | 0.996210 | |
|
|
| WALDO 12-image Cider W8A8 precision versus local bf16 MLX at `512x512`: |
|
|
| | Bundle | Summary cosine mean/min | Spatial cosine mean/min | |
| | --- | ---: | ---: | |
| | `so400m/cider-w8a8-g128` | 0.998808 / 0.998460 | 0.999269 / 0.998657 | |
| | `h/cider-w8a8-g128` | 0.997935 / 0.997436 | 0.997821 / 0.996704 | |
| | `so400m/cider-w8a8-p9999` | 0.998638 / 0.998112 | 0.999240 / 0.998586 | |
| | `h/cider-w8a8-p9999` | 0.997642 / 0.996978 | 0.997628 / 0.996634 | |
|
|
| ## Speed Summary |
|
|
| MLX measurements on Apple M5 Max at `512x512`, batch 1: |
|
|
| | Bundle | p50 latency | Throughput | |
| | --- | ---: | ---: | |
| | `so400m/8bit-affine` | 49.6 ms | 20.2 images/s | |
| | `h/8bit-affine` | 74.2 ms | 13.5 images/s | |
| | `so400m/cider-w8a8` | 32.5 ms | 30.8 images/s | |
| | `h/cider-w8a8` | 47.1 ms | 21.2 images/s | |
| | `so400m/cider-w8a8-g128` | 31.3 ms | 32.0 images/s | |
| | `h/cider-w8a8-g128` | 48.3 ms | 20.7 images/s | |
| | `so400m/cider-w8a8-p9999` | 29.8 ms | 33.5 images/s | |
| | `h/cider-w8a8-p9999` | 43.7 ms | 22.9 images/s | |
| | `so400m/mxfp8` | 49.8 ms | 20.1 images/s | |
| | `h/mxfp8` | 52.6 ms | 19.0 images/s | |
|
|
| There are no supported dequantize-at-load artifacts in this repository. The MLX affine and |
| `mxfp8` bundles keep weights packed but are weight-only, so they prioritize compact storage |
| and lower runtime weight memory over throughput. Cider W8A8 is the faster |
| weight/activation low-bit runtime path found so far, but it requires Apple M5+ hardware |
| and the optional Cider package. |
|
|
| ## Usage |
|
|
| Install the implementation package from the GitHub repository, then point `--checkpoint` at |
| one of the downloaded subdirectories: |
|
|
| ```sh |
| cradio-mlx embed \ |
| --backend mlx-so400m \ |
| --checkpoint so400m/8bit-affine \ |
| --image image.jpg \ |
| --image-size 512 \ |
| --dtype bfloat16 \ |
| --save-npz embedding.npz |
| ``` |
|
|
| The g128 variants are the balanced Cider choice. The p99.99 variants are faster and |
| slightly lower precision; validate them against downstream task metrics before replacing |
| bf16 or g128. |
|
|
| Use `--backend mlx-h` for the H model: |
|
|
| ```sh |
| cradio-mlx embed \ |
| --backend mlx-h \ |
| --checkpoint h/8bit-affine \ |
| --image image.jpg \ |
| --image-size 512 \ |
| --dtype bfloat16 \ |
| --save-npz embedding.npz |
| ``` |
|
|
| Cider W8A8 bundles require Python `>=3.12`, Apple M5+ hardware, and Cider: |
|
|
| ```sh |
| python -m pip install "cider @ git+https://github.com/Mininglamp-AI/cider.git" |
| cradio-mlx embed \ |
| --backend mlx-h \ |
| --checkpoint h/cider-w8a8 \ |
| --image image.jpg \ |
| --image-size 512 \ |
| --dtype bfloat16 \ |
| --save-npz embedding.npz |
| ``` |
|
|
| ## License |
|
|
| The implementation code in `c-radio_v4_MLX` is MIT licensed. The model weights and these |
| converted bundles are governed by NVIDIA's Open Model License Agreement, not by the MIT |
| license. Preserve NVIDIA provenance and license terms when redistributing these bundles. |
|
|
| NVIDIA Open Model License Agreement: |
|
|
| https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf |
|
|
