| --- |
| license: mit |
| base_model: |
| - VAST-AI/TripoSplat |
| pipeline_tag: image-to-3d |
| tags: |
| - image-to-3d |
| - image2mesh |
| - 3d |
| - gaussian-splatting |
| - 3d-gaussian-splatting w |
| - ebgpu |
| - onnx |
| - browser |
| - local-inference |
| - triposplat |
| - computer-vision |
| --- |
| |
|
|
| ⸻ |
|
|
| TripoSplat WebGPU |
|
|
| https://triposplat-webgpu.vercel.app/ |
|
|
| Browser-local WebGPU conversion of TripoSplat for single-image generation of 3D Gaussian splats. |
|
|
| This repository contains ONNX model artifacts prepared for the experimental TripoSplat WebGPU runtime. |
|
|
| The goal is to run the TripoSplat image-to-3D pipeline locally in a WebGPU-capable browser using ONNX Runtime WebGPU, without uploading the source image to a remote inference service. |
|
|
| Alpha engineering release |
|
|
| This is not yet a production-ready browser model. The complete five-stage pipeline executes and exports structurally valid Gaussian scenes on the recorded test system, but official end-to-end numerical and rendered-image parity has not yet been established. See Validation status and Limitations. |
|
|
| Model description |
|
|
| TripoSplat converts one 2D image into a variable-resolution 3D Gaussian representation. |
|
|
| The original model was developed by TripoAI and released by VAST-AI Research. This repository does not train a new model or alter the intended learned task. It converts and packages the released TripoSplat pipeline for execution through ONNX Runtime WebGPU. |
|
|
| Inputs |
|
|
| A single RGB image containing an object or foreground subject. |
|
|
| For the current WebGPU runtime: |
|
|
| * Prepared images with an isolated foreground can be processed directly. |
| * Arbitrary photographs require a caller-provided background-removal stage. |
| * Browser-local BiRefNet background removal is planned but is not yet bundled and qualified. |
| * Results are generally more predictable when the subject is clearly visible, centered, and not heavily occluded. |
|
|
| Outputs |
|
|
| A 3D Gaussian scene containing up to 262,144 Gaussians, exportable as: |
|
|
| * .ply |
| * .splat |
|
|
| The scene contains Gaussian positions, appearance, opacity, orientation, and scale data suitable for compatible Gaussian-splat viewers and downstream tools. |
|
|
| Intended uses |
|
|
| This model repository is intended for: |
|
|
| * Browser-local image-to-3D research |
| * WebGPU portability experiments |
| * ONNX Runtime WebGPU validation |
| * Gaussian-splat generation prototypes |
| * Interactive 3D creation tools |
| * AR, VR, spatial-computing, game, and simulation experiments |
| * Comparing browser inference against the official PyTorch implementation |
| * Research into local and privacy-preserving generative inference |
|
|
| It is currently best treated as an engineering and interoperability release, not a production inference endpoint. |
|
|
| Out-of-scope uses |
|
|
| This release should not currently be relied upon for: |
|
|
| * Guaranteed numerical equivalence with official TripoSplat |
| * Guaranteed visual equivalence with official TripoSplat |
| * Production workloads requiring stable latency or memory bounds |
| * Automated processing on unqualified low-memory devices |
| * Safety-critical, medical, legal, forensic, or measurement applications |
| * Accurate physical reconstruction from one photograph |
| * Recovery of unseen geometry as factual ground truth |
| * Deployment without reviewing browser, CDN, analytics, and application privacy behavior |
|
|
| Generated geometry and appearance are model predictions. They should not be interpreted as a complete or physically verified reconstruction of the depicted object. |
|
|
| Pipeline architecture |
|
|
| The browser pipeline consists of five principal stages: |
|
|
| 1. DINOv3 image encoder |
| * Encodes the prepared source image into visual features. |
| 2. Flux VAE encoder |
| * Produces latent image conditioning. |
| 3. Diffusion Transformer |
| * Performs classifier-free-guided flow sampling. |
| * The browser runtime supports the official fast 4-step and quality 20-step schedules. |
| 4. Dynamic octree |
| * Predicts and refines the spatial distribution of Gaussian centers over eight octree levels. |
| 5. Gaussian feature decoder |
| * Converts the final learned features into Gaussian scene attributes. |
|
|
| Sampling, octree traversal, scene construction, and export coordination are implemented in TypeScript around ONNX Runtime WebGPU sessions. |
|
|
| Repository contents |
|
|
| The model bundle uses an immutable manifest describing every required ONNX graph and external-data object. |
|
|
| Current bundle characteristics: |
|
|
| Property Value |
| Numerical format FP32 |
| Total declared model size 6,465,182,402 bytes |
| Number of manifest objects 10 |
| Maximum generated Gaussian count 262,144 |
| Browser execution backend ONNX Runtime WebGPU |
| WASM fallback used for reported tests No |
|
|
| Some ONNX graphs use external-data sidecars. The graph and sidecar files must retain the exact paths encoded in each ONNX graph. |
|
|
| Do not rename or flatten external-data files without rewriting the corresponding ONNX graph. |
|
|
| Using the model |
|
|
| The model artifacts are designed for the companion runtime: |
|
|
| https://github.com/yosun/TripoSplatWebGPU |
|
|
| Clone and start the development application: |
|
|
| git clone https://github.com/yosun/TripoSplatWebGPU.git |
| cd TripoSplatWebGPU |
| corepack enable |
| pnpm install --frozen-lockfile |
| pnpm dev |
|
|
| Open: |
|
|
| http://localhost:<port>/e2e-lab.html |
|
|
| The end-to-end lab can: |
|
|
| 1. Load a prepared source image |
| 2. Execute the five-stage WebGPU pipeline |
| 3. Construct the Gaussian scene |
| 4. Export .ply and .splat |
| 5. Load the exported result into the browser viewer |
| 6. Produce a structured qualification report |
|
|
| The laboratory pages are validation surfaces, not polished end-user interfaces. |
|
|
| Hosting the model files |
|
|
| Because the current model bundle is approximately 6.47 GB, production deployments should serve it from static object storage or a model CDN rather than proxying it through an application server. |
|
|
| The host should support: |
|
|
| * HTTPS |
| * Cross-origin resource sharing |
| * Byte-range requests |
| * Long-lived caching |
| * Immutable versioned object paths |
| * Correct MIME types |
| * Exact ONNX external-data locations |
| * Sufficient file-size limits |
|
|
| Appropriate deployment patterns include object storage combined with a CDN. |
|
|
| Application serverless functions should not relay the model objects unless the platform is explicitly designed for multi-gigabyte static model delivery. |
|
|
| Validation status |
|
|
| Validation was performed against deterministic fixtures generated from the untouched official PyTorch implementation. |
|
|
| Component-level validation |
|
|
| Component Current result |
| TripoSplat preprocessing and DINOv3 encoder Strict pass on one recorded fixture |
| Flux VAE encoder Pass on one recorded fixture |
| One FP32 DiT invocation Strict pass |
| Four-step CFG/Euler loop Qualification pass, strict diagnostic fail |
| Twenty-step guided loop Qualification fail and strict diagnostic fail |
| Full eight-level octree trajectory Pass |
| Raw Gaussian feature decoder Pass |
| Packaged five-stage browser execution Structural and viewer pass |
| Official whole-scene parity Not established |
| Official render parity Not established |
| Repeated-generation memory qualification Not established |
| 16 GB device qualification Not established |
| Microsoft Edge qualification Not established |
|
|
| A component pass means that the recorded fixture met the declared numerical thresholds for that specific test. It does not establish correctness for all images or devices. |
|
|
| Recorded test environment |
|
|
| The published browser measurements were collected on: |
|
|
| Property Test environment |
| Device Apple M3 Max Mac |
| Unified memory 128 GB |
| Operating system macOS 26.3 |
| Browser Chrome 150 |
| Runtime ONNX Runtime WebGPU 1.27.0 |
| WASM fallback Disabled |
|
|
| These results must not be extrapolated to lower-memory systems. |
|
|
| Recorded component measurements |
|
|
| Stage Recorded result |
| DINOv3 8,702.1 ms cold model/session load; 4,262.4 ms inference |
| Flux VAE 1,304.1 ms median inference |
| One FP32 DiT invocation 12,584.6 ms cold model/session load; 11,844.6 ms inference |
| Four-step sampling loop 102,268.8 ms sampling wall time across eight DiT calls |
| Twenty-step sampling loop 676,669.1 ms sampling wall time across forty DiT calls |
| Eight-level octree 1,327.4 ms model load; 5,713.7 ms primary trajectory inference |
| Gaussian decoder 7,929.6 ms model load; 5,370.6 ms inference |
| Packaged four-step generation 247,901.5 ms through generate() in the recorded qualification run |
| Final packaged consumer run 210,158.7 ms through PLY and .splat export |
|
|
| These measurements include different combinations of loading, temporary artifact preparation, pipeline execution, export, and viewer qualification. They are engineering measurements from one machine, not standardized product latency claims. |
|
|
| Known limitations |
|
|
| End-to-end parity remains open |
|
|
| Individual stages can closely match their PyTorch fixtures while small discrepancies accumulate across iterative sampling and scene decoding. |
|
|
| The current release does not claim: |
|
|
| * Whole-scene numerical parity |
| * Rendered-image parity |
| * Equivalent image quality to the official implementation |
| * Equivalent behavior across arbitrary inputs |
|
|
| Twenty-step sampling is not qualified |
|
|
| The official 20-step quality schedule completes all expected browser model calls, but its final latent state currently fails the declared qualification thresholds. |
|
|
| Use the 20-step path for investigation, not as a verified parity path. |
|
|
| Four-step sampling has a diagnostic discrepancy |
|
|
| The 4-step schedule passes the current qualification envelope but fails a stricter diagnostic comparison. It is sufficient for structural end-to-end testing, not for claiming exact official parity. |
|
|
| High memory requirements |
|
|
| The current model bundle is FP32 and approximately 6.47 GB before accounting for: |
|
|
| * WebGPU buffers |
| * Worker allocations |
| * Decoded model data |
| * Intermediate tensors |
| * Browser overhead |
| * Export buffers |
| * Viewer resources |
| * Unified-memory duplication or driver behavior |
|
|
| Browser heap counters do not expose all worker, GPU, driver, or unified-memory allocations. A reliable peak-memory figure is not yet available. |
|
|
| Limited hardware qualification |
|
|
| The current measured environment has 128 GB of unified memory. |
|
|
| No claim is made yet for: |
|
|
| * 16 GB Macs |
| * Integrated GPUs with limited shared memory |
| * Mobile browsers |
| * Mobile GPUs |
| * Intel Macs |
| * Discrete Windows GPUs |
| * Linux browser configurations |
| * Safari WebGPU |
| * Firefox WebGPU |
|
|
| Background removal is not bundled |
|
|
| The TripoSplat WebGPU package currently expects either: |
|
|
| * A prepared foreground input, or |
| * A caller-provided local removeBackground implementation |
|
|
| Background-removal behavior can materially affect the generated scene. |
|
|
| Single-view ambiguity |
|
|
| A single image does not fully specify hidden surfaces, depth, scale, material, or topology. The model may invent or simplify unseen geometry. |
|
|
| Input sensitivity |
|
|
| Results may degrade for: |
|
|
| * Multiple overlapping objects |
| * Heavy occlusion |
| * Transparent or reflective objects |
| * Very thin structures |
| * Motion blur |
| * Extreme crops |
| * Small subjects |
| * Highly cluttered backgrounds |
| * Ambiguous silhouettes |
| * Unusual camera projections |
| * Text-heavy or diagrammatic inputs |
|
|
| Privacy |
|
|
| The companion runtime is designed for browser-local inference. |
|
|
| Under the intended architecture: |
|
|
| * Source images are decoded in the browser. |
| * Source images are not sent to a model inference server. |
| * Model computation runs through local WebGPU execution. |
| * Generated Gaussian data can remain local unless the surrounding application uploads it. |
|
|
| Browser-local inference does not make every deployment private by default. Deployers must separately review: |
|
|
| * Analytics |
| * Error reporting |
| * Authentication |
| * CDN logs |
| * Signed URLs |
| * Browser extensions |
| * Application uploads |
| * Published result pages |
| * Content Security Policy |
| * Retention and deletion behavior |
|
|
| Do not log source images or signed model URLs unintentionally. |
|
|
| Security considerations |
|
|
| Model files should be treated as immutable executable inputs to the ONNX runtime. |
|
|
| Deployers should: |
|
|
| * Verify artifact hashes against the manifest |
| * Serve versioned immutable files |
| * Restrict allowed model origins |
| * Configure a suitable Content Security Policy |
| * Avoid loading arbitrary user-supplied ONNX graphs |
| * Validate exported scene sizes before allocating viewer resources |
| * Keep ONNX Runtime and browser versions current |
| * Avoid exposing private CDN credentials to browser code |
| * Rate-limit any optional publishing or storage service |
|
|
| Bias and representational limitations |
|
|
| This conversion inherits the behavior and limitations of the original TripoSplat model and its training data. |
|
|
| The WebGPU project did not train the underlying model and does not currently have sufficient information to provide a complete audit of: |
|
|
| * Training-data composition |
| * Geographic representation |
| * Cultural representation |
| * Object-category balance |
| * Copyright status of every training example |
| * Performance disparities across image domains |
|
|
| Users should evaluate the model on their own intended data and document domain-specific failure patterns before deployment. |
|
|
| Environmental considerations |
|
|
| Browser-local execution transfers computation from centralized inference infrastructure to the user’s device. |
|
|
| Potential advantages include: |
|
|
| * No dedicated inference server for each generation |
| * Local reuse of cached model artifacts |
| * Reduced image-upload requirements |
|
|
| Potential costs include: |
|
|
| * A large initial model download |
| * Significant local GPU and memory use |
| * Long FP32 execution times |
| * Repeated energy use on client hardware |
|
|
| Quantized models, reduced precision, graph fusion, model partitioning, and artifact deduplication have not yet been fully qualified. |
|
|
| Relationship to the original model |
|
|
| This repository is an unofficial WebGPU conversion and engineering project. |
|
|
| It is not the official TripoSplat repository and is not presented as an official TripoAI browser release. |
|
|
| The official sources remain the reference for: |
|
|
| * Model architecture |
| * Learned weights |
| * PyTorch numerical behavior |
| * Intended generation schedules |
| * Research claims |
| * Original licensing |
| * Scientific citation |
|
|
| Official model: |
|
|
| VAST-AI/TripoSplat |
|
|
| Official implementation: |
|
|
| https://github.com/VAST-AI-Research/TripoSplat |
|
|
| WebGPU implementation: |
|
|
| https://github.com/yosun/TripoSplatWebGPU |
|
|
| Licensing |
|
|
| The original TripoSplat code and released model weights are provided under the MIT License by their authors. |
|
|
| The ONNX artifacts in this repository are conversions of those released weights and remain subject to the applicable upstream license and notices. |
|
|
| The companion WebGPU source repository contains additional runtime and inherited software components. Review its THIRD_PARTY_NOTICES.md before redistributing the complete application or publishing a derived software package. |
|
|
| This repository does not grant rights over user-provided input images or generated content that are not otherwise held by the user. |
|
|
| Attribution |
|
|
| Original model |
|
|
| TripoSplat was developed by TripoAI and released through VAST-AI Research. |
|
|
| WebGPU conversion and runtime |
|
|
| The ONNX/WebGPU conversion, browser runtime, validation harnesses, packaging, and Gaussian export integration are maintained by Yosun Chang / AI3D. |
|
|
| This work builds from the ml-sharp-web browser application chassis while keeping SHARP-specific inference and TripoSplat inference logically separate. |
|
|
| Citation |
|
|
| Please cite the original TripoSplat paper when using the model: |
|
|
| @misc{yan2026generative3dgaussianslearned, |
| title = {Generative 3D Gaussians with Learned Density Control}, |
| author = {Runjie Yan and Yan-Pei Cao and Peng Wang and Ding Liang and Yuan-Chen Guo}, |
| year = {2026}, |
| eprint = {2605.16355}, |
| archivePrefix = {arXiv}, |
| primaryClass = {cs.GR}, |
| url = {https://arxiv.org/abs/2605.16355} |
| } |
|
|
| When discussing this specific browser conversion, also reference: |
|
|
| Yosun Chang. TripoSplat WebGPU: an experimental browser-local |
| ONNX Runtime WebGPU conversion of TripoSplat. 2026. |
| https://github.com/yosun/TripoSplatWebGPU |
|
|
| Development status |
|
|
| This project is under active development. |
|
|
| The principal remaining release gates are: |
|
|
| * Resolve iterative sampling drift |
| * Establish official whole-scene numerical parity |
| * Establish rendered-image parity |
| * Qualify repeated generations |
| * Measure peak total memory |
| * Test 16 GB hardware |
| * Test additional browsers and GPU families |
| * Bundle and validate local background removal |
| * Qualify reduced-precision or compressed artifacts |
| * Publish a stable package and immutable model revision |
|
|
| For current evidence, reports, fixtures, and contribution instructions, see the companion source repository. |
