docs/compatibility-and-benchmarks.md

Compatibility and measured benchmarks

Compatibility policy

The target is desktop Chrome and Edge on Apple Silicon Macs with at least 16 GB unified memory. That is a target, not a current support claim.

The only recorded browser environment is Chrome 150 on an Apple M3 Max Mac with 128 GB unified memory. No Microsoft Edge configuration, 16 GB Apple Silicon, Safari, Firefox, integrated Intel GPU, or discrete Windows GPU qualification has been completed.

Until the complete pipeline and package are available, compatibility should be described as experimental even on the measured machine.

What a compatibility check can know

A browser can report whether navigator.gpu exists, whether an adapter/device can be requested, and selected device limits and features. It cannot reliably report total GPU memory, free unified memory, or the ONNX Runtime/driver peak allocation that a future generation will require.

A structured compatibility report should therefore include:

• browser and platform strings;
• WebGPU availability;
• adapter metadata when the browser exposes it;
• relevant adapter/device limits;
• estimated static model transfer bytes from the manifest;
• only measured or carefully derived peak-byte data, clearly labeled;
• warnings for unqualified browsers and fallback paths;
• blockers such as a missing required WebGPU feature or graph larger than a supported buffer limit.

It must not convert JavaScript heap size or maxBufferSize into a claim about available GPU memory.

Measured results

Environment shared by all browser measurements:

Field	Value
Hardware	Apple M3 Max Mac, 128 GB unified memory
Operating system	macOS 26.3 (25D125)
Browser	Chrome 150.0.0.0
Execution provider	ONNX Runtime WebGPU
ONNX Runtime Web	1.27.0
Cross-origin isolated	No
WASM fallback	Disabled

Flux VAE encoder

Metric	Value
Model transfer	137,800,544 bytes
Model load	817 ms
Median inference, three runs	1,304.1 ms
Median source preprocessing + encode	1,601.7 ms
Maximum absolute tensor error	0.0001683235
Cosine similarity	0.9999999997

One fp32 DiT invocation

Metric	Value
Model transfer	1,643,895,982 bytes
Cold model/session load	12,584.6 ms
Inference	11,844.6 ms
Readback	0.9 ms
Latent maximum absolute error	0.0000057220
Latent cosine similarity	0.9999999999996
Camera maximum absolute error	0.0000017881
Camera cosine similarity	0.9999999999997
Declared strict gate	Passed

Four-step fp32 CFG/Euler sampling

Metric	Value
WebGPU DiT invocations	8
Model load	11,411.3 ms
Sampling wall	102,268.8 ms
Summed inference	102,236.0 ms
Latent maximum absolute error	0.0044521093
Latent cosine similarity	0.9999999885
Camera cosine similarity	0.9999999986
Qualification envelope	Passed
Separate strict diagnostic	Failed

Four-step teacher-forced DiT trajectory

The browser graph was also run eight times with the exact official PyTorch sample and timestep injected at every conditional and unconditional invocation. This removes autoregressive CFG/Euler state drift from each comparison.

Invocation class	Strict result
Conditional, steps 1–4	Passed
Unconditional, step 1	Passed
Unconditional, steps 2–4	Failed for latent; camera passed
All camera outputs	Passed

The failing unconditional latent maximum errors were 0.000330925, 0.000472546, and 0.000844243 at steps 2–4. Their fractions within combined tolerance were 0.9994812, 0.9990692, and 0.9958420. The strict gate required every value within atol=0.0001 plus rtol=0.001 * abs(reference) and cosine similarity of at least 0.99999999.

Timing/size	Value
Model transfer	1,643,895,982 bytes
Model load	12,231.8 ms
WebGPU invocations	8
Summed inference	107,931.5 ms
Sampling wall	108,012.6 ms

This measured result localizes the strict defect to invocation-dependent behavior, especially the all-zero conditioning branch; it is not explained only by autoregressive Euler accumulation. Evidence: teacher-forced browser report.

An explicit stable-RMS graph rewrite was measured as a diagnostic experiment. It failed the same step-2 through step-4 unconditional latent gates, with maximum errors of 0.000346661, 0.000476122, and 0.000827583. This is not a material improvement over the canonical graph, so the manifest remains unchanged. Its 95,085.9 ms wall time is an engineering diagnostic, not a promoted performance result. Evidence: stable-RMS teacher-forced report.

Fresh CPU ONNX Runtime reproduces most of the step-4 asymmetry with no browser or reused session: invocation 7 conditional passes at 0.000043750 maximum latent error, while invocation 8—same sample/timestep, zero context—fails at 0.000764251. Adapted PyTorch remains much closer to untouched official PyTorch on the unconditional input (0.000071019 maximum), and an earlier ordinary output-projection ONNX graph fails almost identically at 0.000745058. Static Sobol and the real-RoPE primitive are already gated. The evidence therefore points to the long repeated-token ONNX attention reductions, not stable RMS, RoPE, the output-projection workaround, WebGPU, or session reuse. This is a numerical diagnosis, not a browser performance result. Evidence: conditional CPU report, unconditional CPU report, and projection A/B report.

The follow-up 39-boundary probe localizes the first material divergence to the attention residual of context_refiner.0, before its MLP and before any joint block. CPU ORT versus official PyTorch measures 3.0994e-6 conditional maximum error and 1.1063e-4 unconditional maximum error at this boundary (35.69×). Adapted PyTorch remains within tolerance on the same unconditional boundary (6.6757e-6), while ORT versus adapted PyTorch fails (1.1730e-4). The next diagnostic must split this one attention operation into Q/K normalization, logits, softmax, value accumulation, and projection. Evidence: block-boundary report.

Twenty-step fp32 CFG/Euler sampling

Metric	Value
WebGPU DiT invocations	40
Model load	12,510.0 ms
Sampling wall	676,669.1 ms
Summed inference	676,525.5 ms
Official PyTorch/MPS sampling	169,155.4 ms
Latent maximum absolute error	0.0487092733
Latent mean absolute error	0.0003517170
Latent cosine similarity	0.9999996593
Latent fraction within qualification tolerance	0.9983292
Camera cosine similarity	0.9999999807
Qualification envelope	Failed
Strict diagnostic	Failed

The control loop completed all 40 WebGPU invocations without fallback. This is a measured failure from accumulated final-state drift, not an extrapolation or an incomplete run.

Eight-level fp32 octree trajectory

Metric	Value
Model transfer	221,419,396 bytes
Model/session load	1,327.4 ms
Primary official-frontier calls	8
Varied-padding probes	8
Primary summed inference	5,713.700000166893 ms
Padding-probe summed inference	5,673.1 ms
All-call trajectory wall	11,415.3 ms
Active-logit levels passing	8/8
Active padding-probe maximum error	0
Sampled child-count mismatches	0
Recorded random values consumed	41,972/41,972
Final point-coordinate maximum error	0
Final log-probability maximum error	0.0000028610
Full trajectory gate	Passed

Every active WebGPU occupancy logit passes the declared atol=0.005, rtol=0.01, minimum-cosine 0.99999 gate. The second call at each level replaces the inactive zero-padded tail with varied normalized coordinates; active outputs remain bit-identical. TypeScript systematic resampling reproduces every official child count, consumes every captured random value exactly once, and produces bit-identical final points after compaction, expansion, and jitter replay.

The 5,713.700000166893 ms primary sum covers the eight official neural calls. The 11,415.3 ms wall time deliberately includes eight extra padding probes and host comparison work, so neither is an end-to-end generation benchmark. Evidence: full octree trajectory report.

fp32 Gaussian raw feature decoder

Metric	Value
Model transfer	1,094,219,284 bytes
Model/session load	7,929.6 ms
Inference	5,370.6 ms
Readback	1.3 ms
Values compared	3,932,160
Maximum absolute error	0.0032768250
Mean absolute error	0.0000083869
RMSE	0.0000279985
Cosine similarity	0.99999999997
Fraction within tolerance	1.0
Declared browser gate	Passed

This validates raw [1,8192,480] graph features. A separate small official _build_gaussians fixture validates the host activation equations and export tests validate the official .splat layout. The upstream octree points are now qualified, but a live Gaussian decode from those points through activated arrays and a rendered final scene remains open.

These numbers are single-machine engineering records. They do not constitute a distribution or service-level objective; the 20-step entry is an actual completed run rather than a forecast.

Packaged four-step end-to-end structural/viewer run

Metric	Value
Declared verified artifact set	6,465,182,402 bytes
generate() including non-persistent staging	247,901.5 ms
DINO inference	4,339.6 ms
Flux VAE inference	1,745.3 ms
DiT inference, eight calls	120,168.9 ms
Octree inference, eight calls	7,405.3 ms
Gaussian decoder inference	7,138.4 ms
PLY viewer load	569.2 ms
Total through both export hashes and viewer load	248,951.4 ms
Gaussian count	262,144
PLY bytes	17,826,208
.splat bytes	8,388,608
Viewer drawing buffer	1916×954
Structural/export/viewer gate	Passed
Whole-scene numerical/render parity	Not claimed

The run used cache: 'none': each graph was streamed into temporary OPFS, verified, loaded, and released before the next model stage. All scene arrays were present, correctly sized, and finite; both exports matched their structural byte contracts; the PLY reached viewer-ready state with non-zero drawing/display dimensions; the scene and model were disposed. The reported post-disposal origin usage of 216 bytes is not a peak-memory or peak-storage measurement. Evidence: end-to-end viewer report.

The same run exercised retained worker-side DiT conditioning. Deterministic host-to-worker payload accounting falls from 184,774,656 to 46,193,664 bytes for four steps and from 923,873,280 to 46,193,664 bytes for twenty steps. The measured run is not an A/B benchmark, and the optimization does not prove GPU-resident input reuse inside ONNX Runtime.

Exact packed Vite consumer — full five-stage run

The final @ai3d/triposplat-webgpu@0.1.0-alpha.0 tarball, SHA-256 a2ba5ce623c7b6a9d86225742789ef2e7f93e9b8aeec95bc44bcb0725cb21859, was installed with npm into a fresh temporary Vite project. No workspace package link or custom workspace worker was used. Vite emitted the package's module worker and ONNX Runtime assets; the installed bundle first passed a WebGPU identity graph and then completed the full prepared-image TripoSplat path.

Metric	Value
Tarball bytes	6,128,163
Declared model artifacts	6,465,182,402 bytes
generate() including non-persistent verified staging	209,712.2 ms
DINO inference	4,285.2 ms
Flux VAE inference	1,756.5 ms
DiT inference, eight calls	111,538.3 ms
Octree inference, eight calls	6,455.0 ms
Gaussian decoder inference	6,111.2 ms
Total through PLY and .splat export	210,158.7 ms
Gaussian count	262,144
All required arrays finite	Yes
PLY bytes	17,826,208
.splat bytes	8,388,608
Clean-install structural/export gate	Passed

The model origin was a loopback cross-origin static server with CORS, so this is not a public-CDN download benchmark and may benefit from operating-system file caching. It used cache: 'none', did not upload the image, and does not claim whole-scene numerical/render parity or peak memory. Evidence: packed Vite full-run report.

Memory reporting

The VAE record includes performance.memory values for the main JavaScript realm. After three runs, its recorded used heap was 59,219,068 bytes. This excludes the ONNX worker, WebGPU buffers, the Metal driver, and other unified-memory residency. It must not be reported as model or peak memory.

The full attention design avoids materializing the largest naïve score matrices by using query chunking and export adaptations. That addresses a known graph-shape hazard but is not proof that the complete pipeline fits a 16 GB device.

Benchmark protocol for future claims

Before publishing a performance claim:

1. identify exact hardware, memory capacity, OS, browser, ONNX Runtime version, provider, model revision, graph hashes, settings, and whether fallback occurred;
2. separate download, cache hit, session construction, preprocessing, each graph, readback, packing, and total wall time;
3. include warm-up policy and all raw runs, not only the fastest result;
4. record parity or output-quality gate beside timing;
5. report only memory counters whose coverage is understood;
6. test cancellation and a second generation after disposal;
7. store the machine-readable result under docs/benchmarks/.

Do not publish a “20-step estimate” obtained by multiplying a one-step timing. Session staging, cache state, thermal behavior, readbacks, and decoding make that extrapolation unreliable.