moge-2-p150

MoGe-2 (Microsoft's monocular metric geometry model, ViT-L/normal checkpoint) running on one Tenstorrent Blackhole p150a via tt-nn: one RGB image in, metric depth, camera-space point map, surface normals, validity mask and intrinsics out. Weights: Ruicheng/moge-2-vitl-normal · Paper: arXiv:2507.02546 · Upstream code: microsoft/MoGe · Port: changh95/tt-MoGe

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  changh95/moge-2-p150 --with-weights
tt-model serve changh95/moge-2-p150

• Weights Ruicheng/moge-2-vitl-normal at cb0e8bbd6b1e go to your HF cache; the image does not contain them.
• Serves on port 20000 (or the next free port); ready when the log says Application startup complete.

Run with tt-cli

tt serve changh95/moge-2-p150
printf '{"image":"%s"}' "$(base64 -w0 media/source.png)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/moge-2-p150

• POST /predict: image (base64 PNG/JPEG); optional output_format (npz default | png | json, json only up to 512×512), fit (pad default | stretch), fov_x (known horizontal FoV in degrees), apply_mask (true), force_projection (true), include_depth_png (false).
• GET /health, GET /info.

Response

{"model": "MoGe-2 ViT-L (normal)", "height": 1080, "width": 1920,
 "canonical": {"width": 1920, "height": 1080, "fit": "pad", "num_tokens": 1800, "token_grid": [32, 57]},
 "metric_scale": 10.34, "fov_x_deg": 86.40, "mask_coverage": 0.9055,
 "intrinsics": [[0.5325, 0.0, 0.5], [0.0, 0.9466, 0.5], [0.0, 0.0, 1.0]],
 "depth_m": {"min": 2.490, "median": 13.28, "max": 258.4},
 "outputs": {"npz": "..."}, "encoding": {"output_format": "npz"},
 "timing_ms": {"device": 115.3, "total": 1342.1}}

• outputs.npz is base64 of np.savez_compressed at the original resolution: points f32 [H,W,3] metres (camera space, OpenCV axes), depth f32 [H,W] metres, normal f16 [H,W,3], mask u8 (1 = valid), intrinsics f32 3×3 normalized (multiply row 0 by W and row 1 by H; also returned as intrinsics_pixels), metric_scale. Invalid pixels are inf depth/points and zero normal.
• output_format: png returns depth_png16 (16-bit PNG, metres = value × encoding.depth_png_scale, 0 = invalid), normal_png (8-bit RGB, n = v/255·2−1) and mask_png (255 = valid) instead.

Demo

Input (media/source.png)	Depth on p150a (media/depth.png)	Normals on p150a (media/normal.png)

Accuracy and speed

Metric	Value
Point map / depth / normal / mask PCC vs torch reference (real image, 1920×1080)	0.9999 / 0.9998 / 0.9999 / 1.0000
End-to-end PCC vs torch on a random image (points / depth / normal / mask)	0.9999 / 0.9997 / 0.9998 / 0.9997
Metric depth vs torch reference after focal/shift recovery (real image): AbsRel / δ1 / mask IoU	0.0050 / 1.000 / 0.9999
Inference, served over HTTP (warm, batch 1, 1920×1080, npz; median of 30 requests)	115 ms device (min 112 / max 125) · 1.34 s end-to-end (~1.2 s is host post-processing + npz compression)
Inference, served over HTTP (warm, batch 1, 800×600, png; median of 30 requests)	116 ms device · 0.34 s end-to-end
Same, previous path (TT_FUSED=0: eager encoder + decoder-only trace)	215 ms device · 1.45 s (npz) / 213 ms · 0.43 s (png)
Same forward on an RTX 5090 (same host, port's torch reference, eager PyTorch, batch 1, incl. H2D/D2H; bf16 / fp16 autocast)	30.9 / 29.5 ms → GPU 3.7× / 3.8× faster than the p150a's 112.8 ms device; fp32-strict 65.5 ms (1.7×); best torch.compile 20.0 ms (5.6×). End-to-end npz is host-bound on both sides (~1.2 s compression): GPU 1.18 s vs p150a 1.36 s

Caveats

• Every image is placed on a fixed 1920×1080 canvas (1800 ViT tokens, 32×57 grid): fit: pad letterboxes and crops the border back out, stretch squashes; one image per request, batch 1, requests are serialised on the chip.
• bf16 encoder and bfp8 conv weights on device: outputs differ slightly from the fp32 reference (see the PCC figures above; the bf16 metric scale sits ~0.5 % from the fp32 reference on the demo image); depth/points stay float32 because the exp remap reaches ~1e11 in invalid regions.
• The whole device graph (ViT-L encoder, projection fold, conv decoder, merged output) is ONE metal trace captured at boot; TT_FUSED=0 in the environment restores the previous eager-encoder path (~215 ms device).
• Dense outputs are base64 npz (default) or 16-bit/8-bit PNGs inside the JSON envelope; json nested lists are refused above 512×512.
• Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model ready card does not 404.
• Validated on tt-metal v0.78.0-dev20260820 (main 8b98410e730), single p150a only.
• GPU comparison: GPU bf16/fp16 3.7–3.8× faster on the device forward; the served npz request is dominated by host post-processing/compression on both sides (GPU 1.18 s vs 1.36 s). RTX 5090 rows (2026-09-14): same host, the port's own torch reference (same weights) run eagerly in PyTorch 2.11 cu128 with fp32 weights + autocast unless stated, no TensorRT; medians of 50 iterations after warm-up, H2D/D2H included; the p150a rows are the served bf16 fused path incl. upload/readback. p150a power was not measured, so no efficiency comparison is made. Full table: GPU_COMPARISON.md.

Licensing

• Weights: Ruicheng/moge-2-vitl-normal, MIT (licence).
• Port and serving code (code/): Apache-2.0, from changh95/tt-MoGe; the vendored microsoft/MoGe reference and utils3d under code/tt_moge/reference/ keep their MIT licence.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	8b98410e730bb504fea43a88609756e34821d91d
code/ digest	36f1eb12a72865ea (sha256, first 16 hex digits)
built	2026-09-13T15:29:57+00:00 by tt-model 0.1.0