mnmly commited on
Commit
baa2471
·
verified ·
1 Parent(s): fa6d9c4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +82 -0
README.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ library_name: mlx
4
+ pipeline_tag: graph-ml
5
+ tags:
6
+ - mlx
7
+ - mlx-swift
8
+ - 3d
9
+ - point-cloud
10
+ - self-supervised-learning
11
+ - point-transformer-v3
12
+ base_model: Pointcept/Utonia
13
+ ---
14
+
15
+ # Utonia — MLX (safetensors) weights for Apple Silicon
16
+
17
+ MLX-converted weights for **[Utonia](https://huggingface.co/Pointcept/Utonia)** (*Utonia: Toward One Encoder for All Point Clouds*, [arXiv:2603.03283](https://arxiv.org/abs/2603.03283)) — a Point Transformer V3 (mode-3) encoder pretrained across indoor RGB-D, outdoor LiDAR, remote sensing, object CAD, and video-lifted point clouds.
18
+
19
+ These weights power **[mlx-swift-utonia](https://github.com/mnmly/mlx-swift-utonia)**, a numerically verified Swift/[mlx-swift](https://github.com/ml-explore/mlx-swift) port that runs Utonia natively on Apple Silicon (macOS): full-resolution multi-million-point clouds, ScanNet-20 semantic segmentation, and PCA feature visualization.
20
+
21
+ ## Files
22
+
23
+ | File | Contents |
24
+ | --- | --- |
25
+ | `utonia.safetensors` | Encoder weights (137,253,816 params, float32) — converted from `utonia.pth` |
26
+ | `utonia_config.json` | Model config extracted from the checkpoint (`enc_channels=(54,108,216,432,576)`, `enc_depths=(3,3,3,12,3)`, `enc_num_head=(3,6,12,24,32)`, 3D RoPE, 4 serialization curves) |
27
+ | `utonia_seg_head_sc.safetensors` | ScanNet-20 linear-probe segmentation head (`Linear(1386→20)`) — converted from `utonia_linear_prob_head_sc.pth` |
28
+ | `utonia_seg_head_sc_config.json` | Seg-head config |
29
+ | `*_manifest.txt` | Tensor name/shape manifests for both checkpoints |
30
+
31
+ ## Changes from the original
32
+
33
+ Converted from the PyTorch checkpoints in [Pointcept/Utonia](https://huggingface.co/Pointcept/Utonia) (no retraining, no fine-tuning — the numbers are byte-identical modulo the format changes below):
34
+
35
+ - `.pth` (pickled state dict) → `.safetensors`, float32.
36
+ - Config dict extracted from the checkpoint into a standalone JSON.
37
+ - Tensor **values are unchanged**; the Swift loader transposes the spconv `SubMConv3d` kernels from `(C_out, k, k, k, C_in)` to `(k, k, k, C_in, C_out)` at load time.
38
+
39
+ Numerical parity of the Swift port against the reference PyTorch implementation: bit-exact serialization, encoder relative error 0.27 % (fp32 GPU drift), 99.87 % semantic-segmentation argmax agreement.
40
+
41
+ ## Usage (mlx-swift)
42
+
43
+ ```swift
44
+ import Utonia
45
+
46
+ let session = try UtoniaSession.load(SessionConfig(weightsDir: weightsDirURL))
47
+ let result = session.run(RawCloud(coord: coords, color: colors, normal: normals))
48
+ ```
49
+
50
+ Or via the CLI from [mlx-swift-utonia](https://github.com/mnmly/mlx-swift-utonia):
51
+
52
+ ```sh
53
+ utonia-cli semseg --weights-dir weights --input scene/ --output segmented.ply
54
+ utonia-cli pca --weights-dir weights --input scene/ --output features.ply
55
+ ```
56
+
57
+ ## License
58
+
59
+ **CC-BY-NC-4.0**, inherited from the original [Pointcept/Utonia](https://huggingface.co/Pointcept/Utonia) weights — **non-commercial use only**. The mlx-swift-utonia *code* is licensed separately (see its repository); this restriction applies to the weights.
60
+
61
+ All credit for the model belongs to the Utonia authors (Pointcept / The University of Hong Kong and collaborators).
62
+
63
+ ## Citation
64
+
65
+ ```bibtex
66
+ @misc{zhang2026utoniaencoderpointclouds,
67
+ title={Utonia: Toward One Encoder for All Point Clouds},
68
+ author={Yujia Zhang and Xiaoyang Wu and Yunhan Yang and Xianzhe Fan and Han Li and Yuechen Zhang and Zehao Huang and Naiyan Wang and Hengshuang Zhao},
69
+ year={2026},
70
+ eprint={2603.03283},
71
+ archivePrefix={arXiv},
72
+ primaryClass={cs.CV},
73
+ url={https://arxiv.org/abs/2603.03283},
74
+ }
75
+
76
+ @misc{pointcept2023,
77
+ title={Pointcept: A Codebase for Point Cloud Perception Research},
78
+ author={Pointcept Contributors},
79
+ howpublished = {\url{https://github.com/Pointcept/Pointcept}},
80
+ year={2023}
81
+ }
82
+ ```