moge-2-p150 on Tenstorrent p150
Derived from Ruicheng/moge-2-vitl-normal, the ViT-L checkpoint with the normal head of MoGe-2 by Microsoft Research (microsoft/MoGe), with unmodified weights. Paper: MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details. See the original model card and repository for training details.
MoGe-2 ViT-L with the normal head, 331M parameters: a DINOv2 ViT-L/14 encoder (309M parameters), a convolutional neck and four heads (point map, surface normals, validity mask, metric scale). One RGB image goes in and is placed on a fixed 1920x1080 canvas (1800 ViT tokens), letterboxed by default. Out come, at the input resolution, a metric point map in camera space, metric depth, unit surface normals, a validity mask and the camera intrinsics (focal length and field of view) estimated from the image. The network runs on the chip; image decoding, the canvas fit, the recovery of focal length and depth shift, the resize back to the input size and the output encoding run on the host CPU.
Runs on Tenstorrent p150 (mesh P150).
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
At a glance
| Hardware | p150 |
| License | mit |
Intended use
Monocular geometry from single RGB images: depth, 3D point maps, surface normals and the camera's field of view, for research, prototyping and offline analysis such as 3D scene previews and dataset annotation.
Prerequisites
- The Tenstorrent CLI,
tt:uv tool install tenstorrent(or use tt-model alone, see the Quickstart) - Docker
- Tenstorrent hardware: p150
- An x86-64 host CPU with AVX2 and FMA (x86-64-v3) for the host-side speeds under Expected performance; on other CPUs the server uses a slower host path
Quickstart
tt serve changh95/moge-2-p150
tt serve downloads the Docker image and the Ruicheng/moge-2-vitl-normal at cb0e8bbd6b1e243589717c78e750b1ba4c093acf weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.
Without tt-cli: tt-model serve changh95/moge-2-p150.
Send the sample image of this repo to /predict, on the port tt serve reported. The response is JSON with the
dense outputs as a base64 .npz (about 39 MB for this 1920x1080 image):
curl -sLO https://huggingface.co/changh95/moge-2-p150/resolve/main/media/source.png
printf '{"image":"%s"}' "$(base64 -w0 source.png)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' --data-binary @req.json -o resp.json
python3 -c "import base64, io, json, numpy as np; r = json.load(open('resp.json')); \
o = np.load(io.BytesIO(base64.b64decode(r['outputs']['npz']))); print(o['depth'].shape, r['fov_x_deg'])"
Capabilities
This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API β its request and response shapes are the model's own. See the author's notes for the payload it expects.
POST /predict takes one base64 image and returns its geometry. GET /health and GET /info report the server
and device state; GET /v1/models is a stub, not a chat API.
| request field | meaning |
|---|---|
image |
base64 PNG or JPEG of one image, at least 8x8 pixels and at most 32 MB; a data:image/...;base64, prefix is accepted |
output_format (optional) |
npz (default): the arrays as a base64 .npz in the JSON; npz_binary: the same .npz as the response body (application/x-npz), the other response fields as JSON in the X-MoGe-Meta header; png: depth as a 16-bit PNG, normals and mask as 8-bit PNGs; json: nested lists, only for images up to 512x512 pixels |
fit (optional) |
pad (default): letterbox onto the 1920x1080 canvas, keeping the aspect ratio; stretch: resize to the canvas |
fov_x (optional) |
the image's horizontal field of view in degrees, if known (between 1 and 179); then only the depth shift is estimated |
apply_mask (optional) |
default true: invalid pixels get inf depth and points and a zero normal |
force_projection (optional) |
default true: the points are re-projected from the depth and the intrinsics |
include_depth_png (optional) |
default false: with npz or json, also return the 16-bit depth PNG |
Response of the quickstart request above (media/source.png, 1920x1080), npz data and the format notes in
encoding elided:
{"model": "MoGe-2 ViT-L (normal)", "weights": {"repo": "Ruicheng/moge-2-vitl-normal", "revision": "cb0e8bbd6b1e243589717c78e750b1ba4c093acf"},
"height": 1080, "width": 1920,
"canonical": {"width": 1920, "height": 1080, "fit": "pad", "region": [0, 0, 1920, 1080], "num_tokens": 1800, "token_grid": [32, 57]},
"metric_scale": 10.332945823669434, "focal_rel_half_diag": 0.9335933327674866, "shift": -0.10347557067871094,
"intrinsics": [[0.5355780124664307, 0.0, 0.5], [0.0, 0.9521387219429016, 0.5], [0.0, 0.0, 1.0]],
"intrinsics_pixels": [[1028.3097839355469, 0.0, 960.0], [0.0, 1028.3098196983337, 540.0], [0.0, 0.0, 1.0]],
"fov_x_deg": 86.06467404887222, "fov_y_deg": 55.410943192460806, "mask_coverage": 0.9054615162037037,
"depth_m": {"min": 2.506209135055542, "median": 13.321823120117188, "max": 255.86788940429688},
"apply_mask": true, "force_projection": true,
"outputs": {"npz": "..."}, "encoding": {"output_format": "npz", "npz": "..."},
"timing_ms": {"decode": 14.8, "preprocess": 0.1, "device": 32.2, "postprocess": 34.5, "encode": 34.0, "total": 115.6}}
outputs.npzdecodes withnp.loadto arrays at the input resolution:pointsfloat32 [H, W, 3] in metres, camera space with OpenCV axes (x right, y down, z forward);depthfloat32 [H, W] in metres;normalfloat16 [H, W, 3], unit normals in camera space;maskuint8 [H, W], 1 = valid;intrinsicsfloat32 3x3;metric_scalefloat32.intrinsicsis normalized (fx,cxin units of the image width,fy,cyin units of the height);intrinsics_pixelsis the same matrix in pixels.metric_scaleis already applied topointsanddepth.mask_coverageis the share of valid pixels anddepth_mthe minimum, median and maximum valid depth.canonical.regionis where the image sits on the 1920x1080 canvas (x0, y0, x1, y1).- Pixels without valid geometry, such as the sky, have
mask0 and, withapply_mask: true,infdepth and points and a zero normal. pngreturnsoutputs.depth_png16(metres = value xencoding.depth_png_scale, 0 = invalid),outputs.normal_png(n = value / 255 x 2 - 1) andoutputs.mask_png(255 = valid); points are not encoded, unproject the depth with the intrinsics. For a 1920x1080 image thenpzJSON response is about 39 MB and thenpz_binarybody about 29 MB.timing_msgives the server-side stages:decode(base64 and image),preprocess(canvas fit),device(the model call: upload, chip, readback),postprocess(focal and shift recovery, resize to the input size),encodeandtotal.- Errors: 400 for an image that cannot be decoded, is smaller than 8x8 or larger than 32 MB,
jsonabove 512x512 pixels,include_depth_pngwithnpz_binary, or invalid request fields; 503 while the server starts. - A Python API with the same outputs (
tt_moge.MoGeModel; its arrays equal/predict'snpzoutput bit for bit) is described incode/PYTHON.md, with an example incode/examples/quickstart.py. It is installed in this package's image; elsewhere it needs a host ttnn build of the tt-metal commit in Provenance withpatches/tt-metal-8b98410.local.patchapplied.
Example output
The depth and normals of media/source.png as code/examples/quickstart.py writes them with this package's
image: depth drawn as inverse depth with the Spectral colormap (near is red, far is blue), normals as RGB
(x to red, up to green, towards the camera to blue); invalid pixels (the sky) are black in the depth and gray in
the normals.
Expected performance
Geometry accuracy against the ground truth of three public benchmarks, beside the figure the MoGe-2 paper publishes and the same model run in fp32 on the CPU with the same inputs (Rel % / Ξ΄1 %; lower Rel and higher Ξ΄1 are better):
| eval | this package | fp32 CPU reference | MoGe-2 paper | images |
|---|---|---|---|---|
| NYUv2: metric point map | 4.62 / 98.1 | 4.54 / 98.1 | 4.44 / 98.3 | 218 of 654 |
| NYUv2: metric depth | 7.79 / 96.4 | 7.76 / 96.4 | 7.33 / 96.1 | 218 of 654 |
| NYUv2: metric depth, true FoV given | 6.76 / 97.2 | 6.59 / 97.3 | 6.46 / 96.9 | 218 of 654 |
| NYUv2: affine-invariant depth | 2.98 / 98.5 | 2.98 / 98.5 | 2.89 / 98.6 | 218 of 654 |
| KITTI: metric point map | 6.93 / 94.9 | 7.00 / 94.7 | 7.44 / 94.4 | 163 of 652 |
| KITTI: metric depth | 15.0 / 76.4 | 15.2 / 76.0 | 18.1 / 62.9 | 163 of 652 |
| KITTI: metric depth, true FoV given | 8.88 / 93.3 | 8.61 / 93.4 | 8.64 / 93.7 | 163 of 652 |
| KITTI: affine-invariant depth | 3.84 / 98.0 | 3.83 / 98.0 | 3.75 / 98.1 | 163 of 652 |
| iBims-1: metric point map | 6.06 / 97.4 | β | 5.63 / 97.4 | 100 of 100 |
| iBims-1: metric depth | 13.3 / 85.0 | β | 13.6 / 83.0 | 100 of 100 |
| iBims-1: metric depth, true FoV given | 10.9 / 87.4 | β | 9.92 / 92.4 | 100 of 100 |
| iBims-1: affine-invariant depth | 2.66 / 98.6 | β | 2.36 / 98.8 | 100 of 100 |
| Depth against the fp32 reference: AbsRel mean (max) / PCC mean (min) | 0.43 % (1.58 %) / 0.99996 (0.99970) | β | β | 381 |
| Normals against the fp32 reference: mean angle (max of the per-image means) | 0.37Β° (0.83Β°) | β | β | 381 |
| Metric scale against the fp32 reference, mean (range) | +0.65 % (β0.56 to +3.1 %) | β | β | 381 |
model.forward outputs against the fp32 forward on media/source.png: points / depth / normal / mask PCC |
0.999953 / 0.999919 / 0.999958 / 0.999996 | β | β | 1 |
Sets: MoGe's processed evaluation data (the Hugging Face dataset
Ruicheng/monocular-geometry-evaluation):
NYUv2 official test split (654 images, every 3rd used), KITTI Eigen test split (652, every 4th) and iBims-1
(100, all). Metrics: those of MoGe's own evaluation code. Rel is the mean relative error and Ξ΄1 the share of
pixels within a factor 1.25 of the ground truth (for point maps: with a point error below 25 % of the point's
distance), both per-image means; the metric point map is compared after the optimal
translation, metric depth without any alignment, affine-invariant depth after a scale and shift fit. The field
of view is estimated from the image except in the "true FoV given" rows, which pass the ground-truth horizontal
FoV as fov_x. This package and the fp32 reference run the served pipeline (the image on the 1920x1080 canvas,
1800 tokens) through the Python API; the fp32 reference is the upstream MoGe-2 PyTorch model (microsoft/MoGe)
with the same weights, run on NYUv2 and KITTI only. MoGe-2 paper: Table B.4, row "Ours", which is the ViT-L
model without the normal head (Ruicheng/moge-2-vitl, 326M parameters) evaluated on the full sets at 3600
tokens on each image's own aspect ratio.
| speed (batch 1, warm) | input | median | p99 | throughput |
|---|---|---|---|---|
| Device time per image, back-to-back trace replays | 1920x1080 canvas, 1800 tokens | 17.86 ms | 17.96 ms | 56.0 images/s |
Python model.forward(x) (upload, device run, readback, host output pass) |
media/source.png as float32 |
45.6 ms | 107.0 ms | 21.9 images/s |
Python model(array) call, returning a MoGeOutput |
media/source.png, decoded uint8 1920x1080 |
76.2 ms | 184.9 ms | 13.1 images/s |
Python model(path) call, also decoding the PNG |
media/source.png |
93.8 ms | 240.5 ms | 10.7 images/s |
Served POST /predict (npz), server timing_ms.total |
media/source.png |
164.7 ms | 439.5 ms | β |
Served POST /predict (png), server timing_ms.total |
800x600 resize of media/source.png |
100.3 ms | 252.6 ms | β |
Served POST /predict (npz), client round trip on loopback |
media/source.png |
409.7 ms | 445.9 ms | 2.4 requests/s |
Served POST /predict (npz_binary), client round trip on loopback |
media/source.png |
229.5 ms | 327.4 ms | 4.4 requests/s |
Served POST /predict (png), client round trip on loopback |
800x600 resize of media/source.png |
112.6 ms | 130.6 ms | 8.9 requests/s |
Served POST /predict (npz), 2 concurrent clients |
media/source.png |
458.0 ms | 533.0 ms | 2.45 to 3.40 requests/s |
Measured with this package's image on one p150 in an 8-vCPU AMD EPYC-Rome host shared with other jobs (1-minute
load average 11 to 31 during the runs), with code/scripts/bench_inproc.py (5 timings of 50 back-to-back
replays for the device row, 200 calls after 20 warm-up calls for the others, 100 for model(path)) and
code/scripts/bench_http.py (100 requests per case, one at a time on one keep-alive
connection after 10 warm-up requests, then 2 clients with 100 requests each), in 3 runs (median of the runs'
medians, highest p99; throughput 1000 / median); the client round-trip rows are one further run of the same
requests, timed from sending the request to the parsed JSON response (the client_ms that bench_http.py
reports also includes a hash of the outputs), and the 2-client throughput spans 4 runs. The host-side rows
rise with the host's load (model(array) medians of 52 to 134 ms across the runs); the device row does not.
Limitations
- Not for driving, robot navigation, obstacle avoidance or any other safety function, and not for measurements that people or machines rely on: with the field of view estimated from the image, the metric depth error (Rel) was 7.8 % on NYUv2, 15.0 % on KITTI and 13.3 % on iBims-1 above. Validate the metric scale on your own scenes before you use distances.
- Not a MoGe-2 accuracy benchmark: this package serves the normal-head checkpoint on a fixed 1920x1080 canvas with 1800 tokens, while the paper's figures are for the checkpoint without the normal head at 3600 tokens on each image's own aspect ratio. The fixed canvas costs accuracy on some data: on 73 of the NYUv2 images above, the fp32 model's metric point map Rel is 4.68 % with this package's settings and 4.12 % with MoGe's own.
- Not for training or fine-tuning: inference only, with the
moge-2-vitl-normalweights. The other MoGe and MoGe-2 checkpoints are not packaged. - One image per request, batch 1, requests served one at a time. Every image is fitted onto the 1920x1080 canvas
(letterboxed with
fit: pad, distorted withfit: stretch), so very wide or tall images use fewer of the 1800 tokens; other canvases and token budgets were not tested. The EXIF orientation of JPEG files is not applied. - Accuracy was measured on NYUv2, KITTI and iBims-1 only; the other evaluation sets of the paper (ETH3D, GSO, Sintel, DDAD, DIODE, Spring, HAMMER) were not evaluated, and the surface normals were compared with the fp32 reference only, not with ground truth.
- The estimated field of view can be far off: on the KITTI images above it was off by 10.0Β° on average (9.6Β° in
fp32 on the CPU), and the metric depth error fell from 15.0 % to 8.88 % when the true FoV was given. Pass
fov_xwhen you know it. - The chip computes in bfloat16, with 8-bit block floating point weights in parts of the network. Against the fp32 CPU reference, the depth differs by 0.43 % on average (1.58 % at most per image), the normals by 0.37Β° and the metric scale by +0.65 % on average (β0.56 to +3.1 %); the estimated horizontal FoV is 0.44Β° wider on average.
- One p150 only (one chip): multi-chip meshes are not supported, and other boards were not tested. The Python API was tested inside this package's image only.
- The latency of
model()and of the served requests depends on the host CPU and rises with its load; the device time does not. On a CPU without AVX2 and FMA, the host part falls back to a slower torch path: forced onto that path here,model(array)took 642 ms instead of 52 to 134 ms.
Licensing
- Weights: Ruicheng/moge-2-vitl-normal at commit
cb0e8bbd6b1e243589717c78e750b1ba4c093acf, MIT (licence), downloaded bytt serveat that commit, not redistributed here. - Port and serving code (
code/): Apache-2.0.code/tt_moge/reference/carries upstream code under its own licences: microsoft/MoGe (MIT, Β© Microsoft Corporation) with its DINOv2 modules (Apache-2.0, Β© Meta Platforms, Inc. and affiliates) and utils3d (MIT); the licence texts and the file list are inlicenses/.code/models/common/lightweightmodule.pyis from tt-metal (Apache-2.0).code/tt_moge/host_ext/prebuilt/is a compiled build ofcode/tt_moge/host_ext/moge_host.cpp, made bycode/scripts/build_host_ext.sh. patches/tt-metal-8b98410.local.patch: all the changes the image's tt-metal carries on top of the public commit in Provenance;patches/tt-metal-eth-dispatch.patchis a subset of it. tt-metal is Apache-2.0.media/depth.pngandmedia/normal.pngare this package's outputs formedia/source.png.
Feedback
Questions or problems with this package: open a discussion at https://huggingface.co/changh95/moge-2-p150/discussions β that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.
Provenance
The exact sources the image was built from β code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | 8b98410e730bb504fea43a88609756e34821d91d (dirty tree β the image includes uncommitted changes) |
code/ digest |
ebc4f823a4c89591 (sha256, first 16 hex digits) |
| built | 2026-11-12T14:54:16+00:00 by tt-model 0.1.0 |


