docs and settings

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+test.ply filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,7 @@
+.DS_Store
+__pycache__/
+onnx__*
+monodepth_*
+feature_model*
+_Constant_*
+_init_model_*

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "ml-sharp"]
+	path = ml-sharp
+	url = https://github.com/apple/ml-sharp

README.md

@@ -1,3 +1,152 @@
 ---
 license: apple-amlr
+library_name: ml-sharp
+pipeline_tag: image-to-3d
+base_model: apple/Sharp
 ---
+
+
+# Sharp Monocular View Synthesis in Less Than a Second (Core ML Edition)
+
+[![Project Page](https://img.shields.io/badge/Project-Page-green)](https://apple.github.io/ml-sharp/)
+[![arXiv](https://img.shields.io/badge/arXiv-2512.10685-b31b1b.svg)](https://arxiv.org/abs/2512.10685)
+
+> This software project accompanies the research paper: _Sharp Monocular View Synthesis in Less Than a Second_
+by _Lars Mescheder, Wei Dong, Shiwei Li, Xuyang Bai, Marcel Santos, Peiyun Hu, Bruno Lecouat, Mingmin Zhen, Amaël Delaunoy, Tian Fang, Yanghai Tsin, Stephan Richter and Vladlen Koltun_.
+
+We present SHARP, an approach to photorealistic view synthesis from a single image. Given a single photograph, SHARP regresses the parameters of a 3D Gaussian representation of the depicted scene. This is done in less than a second on a standard GPU via a single feedforward pass through a neural network. The 3D Gaussian representation produced by SHARP can then be rendered in real time, yielding high-resolution photorealistic images for nearby views. The representation is metric, with absolute scale, supporting metric camera movements.
+
+#### ✅ **Now available natively on Apple devices** — This release includes a fully validated **Core ML (.mlpackage)** version of SHARP, optimized for CPU, GPU, and Neural Engine inference on macOS and iOS.
+
+![](viewer.gif)
+
+Rendered using [Splat Viewer](https://huggingface.co/spaces/pearsonkyle/Gaussian-Splat-Viewer)
+
+## Getting started
+
+### 📦 Download the Core ML Model
+
+Make sure `git-xet` is installed (https://hf.co/docs/hub/git-xet):
+
+```bash
+brew install git-xet
+git xet install
+```
+
+Clone the model repository:
+
+```bash
+git clone --recursive https://huggingface.co/pearsonkyle/Sharp-mlmodel
+```
+
+### 📱 Run Inference on Apple Devices
+
+Use the provided [sharp.swift](sharp.swift) inference script to load the model and generate 3D Gaussian splats (PLY) from any image:
+
+```bash
+# Compile the Swift runner (requires Xcode command-line tools)
+swiftc -O -o run_sharp sharp.swift -framework CoreML -framework CoreImage -framework AppKit
+
+# Run inference on an image and decimate the output by 50%
+./run_sharp sharp.mlpackage test.png test.ply -d 0.5
+```
+
+> Inference on an Apple M4 Max takes ~1.9 seconds.
+
+```bash
+Usage: \(execName) [OPTIONS] <model> <input_image> <output.ply>
+
+SHARP Model Inference - Generate 3D Gaussian Splats from a single image
+
+Arguments:
+    model              Path to the SHARP Core ML model (.mlpackage, .mlmodel, or .mlmodelc)
+    input_image        Path to input image (PNG, JPEG, etc.)
+    output.ply         Path for output PLY file
+
+Options: 
+    -m, --model PATH           Path to Core ML model
+    -i, --input PATH           Path to input image
+    -o, --output PATH          Path for output PLY file
+    -f, --focal-length FLOAT   Focal length in pixels (default: 1536)
+    -d, --decimation FLOAT     Decimation ratio 0.0-1.0 or percentage 1-100 (default:  1.0 = keep all)
+                                Example: 0.5 or 50 keeps 50% of Gaussians
+    -h, --help                 Show this help message
+```
+
+**CLI Features:**
+- Automatic model compilation and caching  
+- Decimation to reduce point cloud size while preserving visual fidelity
+- Input is expected as a standard RGB image; conversion to [0,1] and CHW format happens inside the model
+- PLY output compatible with [Splat Viewer](https://huggingface.co/spaces/pearsonkyle/Gaussian-Splat-Viewer), [MetalSplatter](https://github.com/scier/MetalSplatter), and [Three.js](https://threejs.org)
+
+## Model Input and Output
+
+### 📥 Input
+The Core ML model accepts two inputs:
+
+- **`image`**: A 3-channel RGB image in `uint8` format with shape `(1, 3, H, W)`.  
+  - Values are expected in range `[0, 255]` (no manual normalization required).  
+  - Recommended resolution: `1536×1536` (matches training size).  
+  - Aspect ratio is preserved; input will be resized internally if needed.
+
+- **`disparity_factor`**: A scalar tensor of shape `(1,)` representing the ratio `focal_length / image_width`.  
+  - Use `1.0` for standard cameras (e.g., typical smartphone or DSLR).  
+  - Adjust slightly to control depth scale: higher values = closer objects, lower values = farther scenes.
+  - If using the `sharp.swift` runner, this input is automatically computed from your image dimensions.
+
+### 📤 Output
+The model outputs five tensors representing a 3D Gaussian splat representation:
+
+| Output | Shape | Description |
+|--------|-------|-------------|
+| `mean_vectors_3d_positions` | `(1, N, 3)` | 3D positions in Normalized Device Coordinates (NDC) — x, y, z. |
+| `singular_values_scales` | `(1, N, 3)` | Scale parameters along each principal axis (width, height, depth). |
+| `quaternions_rotations` | `(1, N, 4)` | Unit quaternions `[w, x, y, z]` encoding orientation of each Gaussian. |
+| `colors_rgb_linear` | `(1, N, 3)` | Linear RGB color values in range `[0, 1]` (no gamma correction). |
+| `opacities_alpha_channel` | `(1, N)` | Opacity (alpha) values per Gaussian, in range `[0, 1]`. |
+
+> ✅ These outputs are fully compatible with [Splat Viewer](https://huggingface.co/spaces/pearsonkyle/Gaussian-Splat-Viewer) and [MetalSplatter](https://github.com/scier/MetalSplatter).
+
+The total number of Gaussians `N` is approximately 1,179,648 for the default model.
+
+### 🔍 Model Validation Results
+
+The Core ML model has been rigorously validated against the original PyTorch implementation. Below are the numerical accuracy metrics across all 5 output tensors:
+
+| Output | Max Diff | Mean Diff | P99 Diff | Angular Diff (°) | Status |
+|--------|----------|-----------|----------|------------------|--------|
+| Mean Vectors (3D Positions) | 0.000794 | 0.000049 | 0.000094 | - | ✅ PASS |
+| Singular Values (Scales) | 0.000035 | 0.000000 | 0.000002 | - | ✅ PASS |
+| Quaternions (Rotations) | 1.425558 | 0.000024 | 0.000067 | 9.2519 / 0.0019 / 0.0396 | ✅ PASS |
+| Colors (RGB Linear) | 0.001440 | 0.000005 | 0.000055 | - | ✅ PASS |
+| Opacities (Alpha) | 0.004183 | 0.000005 | 0.000114 | - | ✅ PASS |
+
+> **Validation Notes:**
+> - All outputs match PyTorch within 0.01% mean error.
+> - Quaternion angular errors are below 1° for 99% of Gaussians.
+
+## Reproducing the Conversion
+
+To reproduce the conversion from PyTorch to Core ML, follow these steps:
+```
+conda create -n sharp python=3.13
+conda activate sharp
+pip install -r ml-sharp/requirements.txt
+pip install coremltools
+python convert.py
+```
+
+## Citation
+
+If you find this work useful, please cite the original paper:
+
+```bibtex
+@inproceedings{Sharp2025:arxiv,
+  title      = {Sharp Monocular View Synthesis in Less Than a Second},
+  author     = {Lars Mescheder and Wei Dong and Shiwei Li and Xuyang Bai and Marcel Santos and Peiyun Hu and Bruno Lecouat and Mingmin Zhen and Ama\"{e}l Delaunoy and Tian Fang and Yanghai Tsin and Stephan R. Richter and Vladlen Koltun},
+  journal    = {arXiv preprint arXiv:2512.10685},
+  year       = {2025},
+  url        = {https://arxiv.org/abs/2512.10685},
+}
+```
+