gen2seg: Generative Models Enable Generalizable Instance Segmentation
Paper • 2505.15263 • Published • 3
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("reachomk/gen2seg-sd", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]
This is the official model release for the Stable Diffusion 2 (SD) variant of our gen2seg generative instance segmenter. It is the same checkpoint we used to generate figures in the paper.
Paper: https://arxiv.org/abs/2505.15263
Please see our website https://reachomk.github.io/gen2seg for demos and additional qualitative samples.
If you are looking for our MAE-H variant, you can find that at https://huggingface.co/reachomk/gen2seg-mae-h
You can run this model at our GitHub: https://github.com/UCDVision/gen2seg or our Huggingface Space: https://huggingface.co/spaces/reachomk/gen2seg
Base model
stabilityai/stable-diffusion-2