README.md

---
license: cc-by-nc-4.0
pipeline_tag: image-to-3d
tags:
  - image-to-3d
  - gaussian-splatting
  - mlx
  - apple-silicon
base_model: veichta/zipsplat
library_name: mlx
---

# ZipSplat — MLX weights (fp16 safetensors)

Format conversion of the [ZipSplat](https://github.com/cvg/ZipSplat) `zipsplat-da3g-252p`
checkpoint for [mlx-swift](https://github.com/ml-explore/mlx-swift), used by
[mlx-swift-ZipSplat](https://github.com/mnmly/mlx-swift-ZipSplat).

**This is not a new model.** It is the original checkpoint re-serialised so it can be loaded
on Apple Silicon without PyTorch. All credit for the model belongs to the original authors.

## Original work

ZipSplat: Fewer Gaussians, Better Splats — Alexander Veicht, Sunghwan Hong, Dániel Baráth,
Marc Pollefeys (ETH Zürich / Microsoft).

- Paper: https://arxiv.org/abs/2606.05102
- Code: https://github.com/cvg/ZipSplat
- Original weights: https://huggingface.co/veichta/zipsplat

## Licence

**CC BY-NC 4.0 — non-commercial use only.**
https://creativecommons.org/licenses/by-nc/4.0/

Inherited from the original weights, which carry it because the checkpoint is initialised
from [DA3-Giant](https://huggingface.co/depth-anything/DA3-GIANT) (CC BY-NC 4.0) and trained
on [DL3DV-10K](https://github.com/DL3DV-10K/Dataset) (CC BY-NC 4.0). The ZipSplat *code* is
Apache-2.0; the weights are not. This conversion is a derivative and carries the same terms.

## Changes from the original

`zipsplat-da3g-252p.tar` (5.79 GB, fp32 PyTorch) → `zipsplat-da3g-252p-f16.safetensors`
(2.90 GB, fp16). 907 tensors, 1.4477 B parameters, verified against the reference model
structure with 0 missing and 0 unexpected keys. Three mechanical changes, no retraining and
no architectural modification:

1. **fp16 cast.** Storage only; the port loads at whatever dtype the caller asks for.
2. **Two structural key remaps.** The ViT's `patch_embed.*`, `cls_token` and `pos_embed` are
   nested under an `embeddings.` prefix, matching the module tree in
   [mlx-swift-da3](https://github.com/mnmly/mlx-swift-da3).
3. **Two Conv2d transposes.** Both patch-embed weights go NCHW → NHWC (`0,2,3,1`), as MLX
   convolutions are channels-last.

Reproduce with
[`Scripts/convert_weights.py`](https://github.com/mnmly/mlx-swift-ZipSplat/blob/main/Scripts/convert_weights.py).

## Fidelity

The port was checked against the PyTorch reference at three levels:

| check | result |
|---|---|
| per-stage activations (patch embed → backbone → fuse → head) | within fp16 tolerance |
| end-to-end `.ply`, every Gaussian parameter | worst field mean-rel 0.043, all corr ≥ 0.9996 |
| novel views rendered through gsplat's CUDA rasteriser | mean PSNR 46.10 dB, worst 38.23 dB |

For scale, the model's own eval PSNR against ground truth is 21.77 dB, so the conversion's
deviation sits about 24 dB below the model's own error.

## Usage

```swift
import MLXZipSplat

let session = try ZipSplatSession(weights: weightsURL)
session.loadViews(images)
let gaussians = session.gaussians(compression: 1.0)[0]
try gaussians.writePLY(to: outputURL)
```

See [mlx-swift-ZipSplat](https://github.com/mnmly/mlx-swift-ZipSplat) for the CLI and the
SwiftUI viewer.