File size: 3,188 Bytes
b4b8695 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
license: cc-by-nc-4.0
pipeline_tag: image-to-3d
tags:
- image-to-3d
- gaussian-splatting
- mlx
- apple-silicon
base_model: veichta/zipsplat
library_name: mlx
---
# ZipSplat — MLX weights (fp16 safetensors)
Format conversion of the [ZipSplat](https://github.com/cvg/ZipSplat) `zipsplat-da3g-252p`
checkpoint for [mlx-swift](https://github.com/ml-explore/mlx-swift), used by
[mlx-swift-ZipSplat](https://github.com/mnmly/mlx-swift-ZipSplat).
**This is not a new model.** It is the original checkpoint re-serialised so it can be loaded
on Apple Silicon without PyTorch. All credit for the model belongs to the original authors.
## Original work
ZipSplat: Fewer Gaussians, Better Splats — Alexander Veicht, Sunghwan Hong, Dániel Baráth,
Marc Pollefeys (ETH Zürich / Microsoft).
- Paper: https://arxiv.org/abs/2606.05102
- Code: https://github.com/cvg/ZipSplat
- Original weights: https://huggingface.co/veichta/zipsplat
## Licence
**CC BY-NC 4.0 — non-commercial use only.**
https://creativecommons.org/licenses/by-nc/4.0/
Inherited from the original weights, which carry it because the checkpoint is initialised
from [DA3-Giant](https://huggingface.co/depth-anything/DA3-GIANT) (CC BY-NC 4.0) and trained
on [DL3DV-10K](https://github.com/DL3DV-10K/Dataset) (CC BY-NC 4.0). The ZipSplat *code* is
Apache-2.0; the weights are not. This conversion is a derivative and carries the same terms.
## Changes from the original
`zipsplat-da3g-252p.tar` (5.79 GB, fp32 PyTorch) → `zipsplat-da3g-252p-f16.safetensors`
(2.90 GB, fp16). 907 tensors, 1.4477 B parameters, verified against the reference model
structure with 0 missing and 0 unexpected keys. Three mechanical changes, no retraining and
no architectural modification:
1. **fp16 cast.** Storage only; the port loads at whatever dtype the caller asks for.
2. **Two structural key remaps.** The ViT's `patch_embed.*`, `cls_token` and `pos_embed` are
nested under an `embeddings.` prefix, matching the module tree in
[mlx-swift-da3](https://github.com/mnmly/mlx-swift-da3).
3. **Two Conv2d transposes.** Both patch-embed weights go NCHW → NHWC (`0,2,3,1`), as MLX
convolutions are channels-last.
Reproduce with
[`Scripts/convert_weights.py`](https://github.com/mnmly/mlx-swift-ZipSplat/blob/main/Scripts/convert_weights.py).
## Fidelity
The port was checked against the PyTorch reference at three levels:
| check | result |
|---|---|
| per-stage activations (patch embed → backbone → fuse → head) | within fp16 tolerance |
| end-to-end `.ply`, every Gaussian parameter | worst field mean-rel 0.043, all corr ≥ 0.9996 |
| novel views rendered through gsplat's CUDA rasteriser | mean PSNR 46.10 dB, worst 38.23 dB |
For scale, the model's own eval PSNR against ground truth is 21.77 dB, so the conversion's
deviation sits about 24 dB below the model's own error.
## Usage
```swift
import MLXZipSplat
let session = try ZipSplatSession(weights: weightsURL)
session.loadViews(images)
let gaussians = session.gaussians(compression: 1.0)[0]
try gaussians.writePLY(to: outputURL)
```
See [mlx-swift-ZipSplat](https://github.com/mnmly/mlx-swift-ZipSplat) for the CLI and the
SwiftUI viewer.
|