arxiv:2608.29253

QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation

Published on Aug 29

· Submitted by

Yaroslav Prytula on Sep 4

Upvote

Authors:

Yaroslav Prytula ,

Abstract

QCell is a query-based model that improves instance segmentation of overlapping microscopy cells through latent-space recombination and contrastive query alignment.

Generated by thinkingmachines/Inkling-Small

Instance segmentation of overlapping cells in microscopy remains challenging due to semi-transparent structures that produce weak boundaries and mixed visual evidence in overlap regions. Existing methods address this through local regions of interest or shape priors but lack global reasoning across overlapping objects. We present QCell, a novel query-based model that de-overlaps cell instances in microscopy scenes. Our approach combines (i) an instance recombination module that decomposes and recombines query representations in latent space, enabling the model to reason about complete object structure under overlap, and (ii) a contrastive query alignment objective that combines distinctive instance feature learning and separation of overlapping cell queries. We additionally introduce a new Organoid dataset benchmark for overlapping cell segmentation. We show that QCell outperforms state-of-the-art methods across multiple benchmarks, achieving +2.2 AP and +2.7 AJI on ISBI2014. Code is available at https://github.com/SlavkoPrytula/QCell

View arXiv page View PDF Project page GitHub 9 Add to collection

Community

YaroslavPrytula

Paper author Paper submitter 1 day ago

•

edited 1 day ago

QCell: Query-Based Cell Instance Segmentation (BMVC 2026)

In this work, we present:

QCell, a novel query-based model for overlapping cell instance segmentation ⭐️
A new overlapping object segmentation dataset: Organoids 🔥

🔗 GitHub: https://github.com/SlavkoPrytula/QCell
🌐 Project page: https://slavkoprytula.github.io/QCell/

If you find this work useful, consider giving it a ⭐️ on GitHub to support further open-source research!

YaroslavPrytula

Paper author Paper submitter 1 day ago

•

edited 1 day ago

Overview of QCell. QCell builds on a MaskDINO-style query-based segmentation pipeline, where multi-scale image features and positional embeddings are processed by the encoder and refined by transformer decoder layers with content and DN queries. The proposed modules are shown above: (a) instance recombination decomposes each query into amodal, visible, and occluded sub-representations and recombines them into a refined full-instance query; (b) contrastive query learning uses matched instance queries q̂_i by Hungarian matching as anchors, corresponding DN queries d̂_i^+ across all groups as positives, and other DN queries as negatives to align queries of the same cell and separate queries of different cells in latent space.

YaroslavPrytula

Paper author Paper submitter 1 day ago

Organoids. One of our key contributions is a novel Organoids dataset for overlapping object segmentation in brightfield microscopy. The dataset contains 1,186 training images, 1,199 validation images, and 201 test images at a resolution of 540 × 540. The dataset presents dense and highly overlapping scenes, with up to 105 instances per training image and an average of 96 instances per test image, reaching a maximum of 223. This makes Organoids a challenging real-world benchmark for evaluating instance separation and de-overlapping in microscopy. Additional dataset details are provided in the supplementary material.

YaroslavPrytula

Paper author Paper submitter about 20 hours ago

QCell segments overlapping cells using an instance recombination module and contrastive query alignment, achieving state-of-the-art results against prior amodal segmentation methods. We additionally introduce a new Organoid dataset benchmark for overlapping cell segmentation. Accepted at BMVC 2026.