RGBT — thermal human perception (sample)

Sample release for the paper Towards Privacy-Preserving Thermal Human Perception from Dataset to Deployment.

This repository contains one thermal-infrared image and its annotation per room for the 20 rooms of the RGBT dataset. The full dataset (paired RGB + thermal, ~96k frames) will be released separately.

It also hosts one released model per task (see below), together with minimal inference code in the GitHub repository.

Contents

images/roomNN.png      # thermal-infrared pseudo-color frame, 80x62
thermal/roomNN.bin     # matching raw thermal frame (uint16, 4-byte header)
annotations.json       # per-image annotation
preview.png            # grid of all 20 samples with boxes drawn

Released models

Task	Method	Path in this repo	Headline metric
Infrared person counting	ResNet18	resnet18/weights/best.pt	accuracy 0.8165
RGB to thermal field	U-Net	unet_rgb2t/weights/checkpoint.pt	MAE 0.654 C, R2 0.923
Infrared to RGB	BBDM	bbdm_ir2rgb/weights/last_model.pth	PSNR 19.47, SSIM 0.797

The infrared person-state detection model (4 states: lie / sit / other / off_bed) is coming soon.

Download everything at once:

hf download skl24/RGBT --local-dir checkpoints

Inference code (one folder per method) lives in kailaisun/RGBT.

Annotation format

annotations.json is keyed by image filename. Each entry:

{
  "room_id": "room01",
  "session_id": "room01_cam2",
  "frame_id": "room01_cam2#1757055337161",
  "ts_ms": 1757055337161,
  "condition": null,
  "role": "cam2",
  "canvas": [80, 62],
  "image": "images/room01.png",
  "objects": [
    {
      "bbox": [15.802, 6.161, 61.996, 32.742],
      "label": "1_1人|1_人在床上|1_坐",
      "n_person": "1_1人",
      "place": "1_人在床上",
      "body_part": null,
      "posture": "1_坐",
      "polygon": [...]
    }
  ]
}

bbox is [x0, y0, x1, y1] in the canvas coordinate system (80x62, origin at the top-left). label is the compound label n_person | place | body_part | posture; the individual fields are also given separately.

Field vocabulary (Chinese, as annotated):

• n_person: 1_1人, 2_2人, 3_3人及以上, 0_无人
• place: 0_人在床下, 1_人在床上
• body_part: 0_人体完整, 1_人体上半身, 2_人体躯干
• posture: 0_躺, 1_坐, 2_其他行为

Raw thermal format

thermal/roomNN.bin is uint16, little-endian, with a 4-byte header followed by 62 * 80 values in row-major order; divide by 10 for degrees Celsius.

Links

• Code / paper: https://github.com/kailaisun/RGBT
• Model checkpoints: https://huggingface.co/skl24/RGBT

License

MIT.