Marigold V2

Revisiting Diffusion Transformers for Monocular Depth Estimation

ACM Transactions on Graphics (SIGGRAPH Asia 2026)

Igor Pavlovic^1,2,*,†, Thiemo Wandel^2,*, Anton Obukhov^2,§
Luca Bartolomei³, Andrey Davydov², Fabio Tosi³, Matteo Poggi³, Sabine Süsstrunk¹, Dengxin Dai²

¹EPFL · ²HUAWEI Bayer Lab · ³University of Bologna
^*Equal contribution · ^†Internship · ^§Project lead

Marigold V2 is a family of models and a cost-effective fine-tuning protocol that repurposes a pretrained diffusion transformer into single-step dense predictors: depth, see-through depth, surface normals, albedo, and other dense modalities. Fine-tuning takes less than a week on a single consumer GPU, within reach of individual practitioners and small labs, and the results are state of the art, faithfully reproducing sharp edges, fur, and hair-thin details. The same models also unlock applications such as metric depth completion.

Each checkpoint here is a set of rank-128 LoRA adapters for the 4-bit quantized DiT plus, where trained, the fine-tuned VAE decoder. The frozen base model is downloaded separately.

Marigold V2 depth predictions compared to prior work

Usage

Inference, evaluation, and training code lives in the GitHub repository, which downloads this repository and the base model into its assets/ folder:

git clone https://github.com/huawei-bayerlab/marigold-v2.git && cd marigold-v2
bash setup/setup_env.sh && conda activate marigold-v2
python scripts/download_assets.py --skip-datasets
python scripts/infer.py --modality depth --image_dir /path/to/images --output_dir output/depth
python scripts/infer.py --checkpoint assets/checkpoints/Marigold-V2/depth/Log-layered --image_dir ...   # any checkpoint below

Inference needs about 17 GB of GPU memory at 1024² and 29 GB at 2048², so a 24 GB card covers 1024² and 2048² needs 32 GB. Outputs are affine-invariant, i.e. depth up to an unknown scale and shift per image.

Checkpoints

Path	Output	Training
`depth/Log-stage2`	affine-invariant log depth	Stage 1 → Stage 2 (SinkLoss, VAE decoder fine-tuned). The paper model; default in the code.
`depth/Log-stage1`	affine-invariant log depth	Stage 1 only: latent MSE + L1 + gradient + iREPA. Initialization for Stage 2 and the layered variant.
`depth/Log-layered`	see-through log depth	`Log-stage1` fine-tuned with SinkLoss on layer 8 of LayeredDepth-Syn; predicts geometry behind glass.
`depth/Uniform-base`	affine-invariant linear depth (Marigold V1 style)	Stage 1 recipe, 30k steps. Parameterization ablation.
`depth/Disparity-base`	affine-invariant inverse depth	Stage 1 recipe with VAE decoder fine-tuning, 30k steps. Parameterization ablation.
`depth/Disparity-layered`	see-through inverse depth	Stage 1 recipe on layer 8 of LayeredDepth-Syn.
`depth/Uniform-layered`	see-through linear depth	LayeredDepth-Syn variant of `Uniform-base`.
`normals`	camera-space unit normals	angular loss + iREPA + SinkLoss, VAE decoder fine-tuned.
`albedo`	linear RGB albedo in [0, 1]	L1 + iREPA, VAE decoder fine-tuned.

Log and linear depth increase with distance, disparity decreases. Each folder holds one trainables.safetensors with keys <component>.<parameter> (Diffuser.* LoRA weights, VAE.* decoder weights when present). qwen_text_embeddings/ contains the precomputed prompt embeddings and masks per modality (<prefix>_prompt_embeds.pt, <prefix>_prompt_mask.pt), so the text encoder of the base model is never needed. manifest.json lists sizes and SHA-256 checksums of all files.

Results

Zero-shot depth with the Pixel-Perfect Depth protocol, depth/Log-stage2, AbsRel ↓ / δ1 ↑ in percent:

NYUv2	KITTI	ETH3D	ScanNet	DIODE
3.6 / 98.0	5.4 / 97.4	2.8 / 99.2	3.7 / 97.9	5.2 / 97.1

Surface normals, normals, mean angular error ↓ / % within 11.25° ↑:

NYUv2	ScanNet	iBims-1	Sintel
16.6 / 61.2	14.1 / 67.4	15.9 / 70.9	28.7 / 27.6

Albedo on the Hypersim test split, albedo: PSNR 20.78, SSIM 0.811, LPIPS 0.195.

Training data

Depth: Hypersim and Virtual KITTI 2 (about 74k images, as repackaged for Marigold V1). See-through depth: LayeredDepth-Syn. Normals and albedo: Hypersim. All released models were trained on a single 32 GB GPU.

Citation

@article{pavlovic2026marigoldv2,
    author = {Pavlovic, Igor and Wandel, Thiemo and Obukhov, Anton and Bartolomei, Luca and Davydov, Andrey and Tosi, Fabio and Poggi, Matteo and S{\"u}sstrunk, Sabine and Dai, Dengxin},
    title = {Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation},
    year = {2026},
    issue_date = {December 2026},
    publisher = {Association for Computing Machinery},
    volume = {45},
    number = {6},
    url = {https://doi.org/10.1145/3842528},
    doi = {10.1145/3842528},
    journal = {ACM Trans. Graph.},
    month = dec,
    articleno = {204},
    numpages = {14}
}

License

The weights in this repository are released under the Apache License, Version 2.0 (see LICENSE). The base model Qwen-Image-Edit-2509 keeps its own license.