RGBD-GSD-Net — RGB-D Glass Surface Detection Network

Pre-trained weights for the model introduced in:

Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection
Jiaying Lin*, Yuen-Hei Yeung*, Shuquan Ye, Rynson W. H. Lau
AAAI 2025
arXiv · Project Page · Dataset (RGBD-GSD)

Model Summary

RGBD-GSD-Net detects glass surfaces by jointly processing RGB images and depth maps. It introduces two novel modules:

• Cross-Modal Context Mining (CCM): adaptively learns individual and mutual context features from RGB and depth information.
• Depth-Missing Aware Attention (DAA): explicitly exploits spatial locations where depth is missing (a strong indicator of glass surfaces) to guide detection.

The backbone is a ResNeXt encoder shared across both modalities.

File	Description
best.pth	Best checkpoint (204 MB), saved as {'model': state_dict, ...}
results/our_best_results.zip	Model predictions on the RGBD-GSD test set

Loading the Weights

import torch
from networks.your_network import RGBDGlassNet   # from the code release

model = RGBDGlassNet()
checkpoint = torch.load("best.pth", map_location="cpu")
model.load_state_dict(checkpoint["model"])
model.eval()

Download the checkpoint:

huggingface-cli download garrying/RGBD-GSD-Net best.pth --local-dir ./weights

Training Dataset

This model was trained and evaluated on RGBD-GSD, the first large-scale RGB-D glass surface detection dataset:

• 3,009 RGB-D images with binary glass surface masks and depth maps
• Available at garrying/RGBD-GSD

Citation

@article{aaai2025_rgbdglass,
  author    = {Lin, Jiaying and Yeung, Yuen-Hei and Ye, Shuquan and Lau, Rynson W.H.},
  title     = {Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection},
  journal   = {AAAI},
  year      = {2025},
}

License

Non-commercial use only — CC BY-NC 4.0.