File size: 6,593 Bytes

---
license: mit
tags:
- low-light
- instance-segmentation
- raw
- mmdetection
---

# LIS-models: Verified Checkpoints for Instance Segmentation in the Dark (IJCV 2023)

Official checkpoints for the IJCV 2023 paper *"Instance Segmentation in the Dark"* (Chen, Fu, Wei, Zheng, Heide). Every checkpoint below was **re-evaluated on a single NVIDIA RTX 3090** with the released code (mmdetection 2.15.1 fork + mmcv-full 1.4.8 + PyTorch 1.11.0+cu113) before publishing; the numbers match the paper within 0.1 AP.

论文《Instance Segmentation in the Dark》(IJCV 2023) 官方权重。发布前每个权重均用开源代码（mmdetection 2.15.1 fork + mmcv-full 1.4.8 + PyTorch 1.11.0+cu113）在单卡 RTX 3090 上复评，数字与论文一致（最大偏差 0.1 AP）。

Code: [github.com/Linwei-Chen/LIS](https://github.com/Linwei-Chen/LIS) · Dataset: [Linwei-Chen/LIS](https://huggingface.co/datasets/Linwei-Chen/LIS)

## Checkpoints / 权重清单

Notation: `segm / bbox` = AP^mask / AP^bbox on the LIS test protocol (dark side).

### Our method (AWD + SCB + DSL) / 论文方法

| File | Size | Paper | Re-eval | Config (in repo) |
|---|---|---|---|---|
| `mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS=31.8.pth` | 369M | 31.8 / 37.6 | **31.8 / 37.6** ✅ | `Configs/mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS.py` |
| `mask_rcnn_convnext-t_p4_w7_fpn_AWD_SCB_DSL_SynCOCO2LIS=36.8.pth` | 614M | 36.8 / 42.7 | **36.8 / 42.7** ✅ | `Configs/mask_rcnn_convnext-t_p4_w7_fpn_AWD_SCB_DSL_SynCOCO2LIS.py` |
| `pointrend_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS=32.8.pth` | 466M | 32.8 / 37.1 | **32.8 / 37.1** ✅ | `Configs/pointrend_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS.py` |
| `mask2former_r50_AWD_SCB_DSL_SynCOCO2LIS=37.8.pth` | 554M | 35.6 / 37.8 | 35.5 / 37.8 ✅ | `Configs/mask2former_r50_caffe_AWD_SCB_DSL_SynCOCO2LIS.py` |
| `mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_COCO80_1x_COCO=36.1.pth` | 372M | 36.1 / 39.5 (COCO val2017, normal light) | 36.1 / 39.5 ✅ | `Configs/mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_COCO80.py` |

### Baselines / 基线（same detector, without our components / 同检测器去掉论文组件）

| File | Size | Paper | Re-eval | Config (in repo) |
|---|---|---|---|---|
| `mask_rcnn_r50_fpn_COCO8=19.8.pth` | 352M | 19.8 / 22.8 | **19.8 / 22.8** ✅ | `Configs/mask_rcnn_r50_fpn_caffe_COCO2LIS.py` |
| `mask_rcnn_convnext-t_p4_w7_fpn_COCO2LIS=23.7.pth` | 573M | 23.7 / 27.9 | **23.7 / 27.9** ✅ | `Configs/mask_rcnn_convnext-t_p4_w7_fpn_COCO2LIS.py` |
| `pointrend_r50_fpn_caffe_COCO2LIS=20.6.pth` | 449M | 20.6 / 23.5 | **20.6 / 23.5** ✅ | `Configs/pointrend_r50_fpn_caffe_COCO2LIS.py` |
| `mask2former_r50_SynCOCO2LIS=21.4.pth` | 528M | 21.4 / 22.9 | 21.4 / 22.9 ✅ | `Configs/mask2former_r50_COCO2LIS.py` |

MD5 checksums are provided in `md5.txt`. File sizes are decimal MB (e.g. `369M` = 369 MB ≈ 352 MiB).

`Configs/` 指向代码仓库中的 config 文件。`mask2former` 系的 config 文件名以实际仓库为准（`mask2former_r50_caffe_*` / `mask2former_r50_*`）。

## Evaluation Protocols / 评测协议

Two test protocols are used by the two config families, exactly as in the paper:

- **Ours 系列（png 协议）**: `lis_coco_png_test+1.json` on `RAW_Dark/` png images. The paper/re-eval numbers use the full 669 images / 2,934 instances; the released (privacy-filtered) json ships 604 images / 2,680 instances. Repo config paths expect `data/LIS/` with the layout described in the code README.
- **Baseline 系列（JPG 相机原图协议）**: `lis_coco_JPG_test+1.json` on camera original `.JPG` files (`RGB_Dark/` in the HF dataset).

论文与复评均在完整测试集（669 图）上进行；HF 数据集公开版过滤了 179 对隐私敏感图像，用公开版测试集复现会有细微数字差异（详见数据卡）。实测：R50 Ours 在公开版测试集（604 图）上 segm 32.0 / bbox 37.5，与论文数字基本一致。

⚠️ **Class-name note / 类别名说明**: the configs pair label ids with class names via the tuple `(bicycle, car, motorbike, bus, bottle, chair, diningtable, tvmonitor)`, which matches the **true id semantics** of the dataset's 0-based category ids. The `categories/name` block inside the released json files is historically misaligned with these true semantics (verified instance-by-instance) — do not rename it, and see the dataset card's *Category mapping* section before any class-dependent post-processing. / config 通过元组 `(bicycle, car, motorbike, bus, bottle, chair, diningtable, tvmonitor)` 绑定标签与类名，该元组与数据集 0 起始类别 id 的**真实语义**一致；发布 json 内的 categories 名字块与真实语义存在历史错位（已逐类验证），请勿改名，做任何依赖类名的后处理前先阅读数据卡的 *Category mapping* 一节。

## Usage / 用法

```bash
git clone https://github.com/Linwei-Chen/LIS && cd LIS
huggingface-cli download Linwei-Chen/LIS --repo-type dataset --local-dir data/LIS   # eval set
# or download only what evaluation needs (~1.6G instead of the full dataset):
# huggingface-cli download Linwei-Chen/LIS --repo-type dataset --local-dir data/LIS --include "RAW_Dark/*" "annotations/*"
huggingface-cli download Linwei-Chen/LIS-models mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS=31.8.pth --local-dir checkpoints
ln -sfn ../data mmdetection/data && cd mmdetection
python tools/test.py ../Configs/mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS.py \
    ../checkpoints/mask_rcnn_r50_fpn_caffe_AWD_SCB_DSL_SynCOCO2LIS=31.8.pth --eval bbox segm
# → segm mAP 32.0 / bbox mAP 37.5 on the privacy-filtered release test set (measured)
#   (paper: 31.8 / 37.6 on the full 669-image test set)
```

For inference-time speedup, `tools/reparameter.py` folds the adaptive weighted downsampling into a single convolution (see repo README). / `tools/reparameter.py` 可将自适应加权下采样折叠为单卷积以加速推理（见仓库 README）。

Environment: PyTorch 1.11.0 + cu113, mmcv-full 1.4.8, mmdet 2.15.1 (the repo's bundled fork).

## License / 许可

Checkpoints are released under the [MIT license](https://opensource.org/licenses/MIT), same as the code.

权重与代码同样以 [MIT 许可](https://opensource.org/licenses/MIT) 发布。

## Citation / 引用

```bibtex
@article{2023lis,
  title={Instance Segmentation in the Dark},
  author={Chen, Linwei and Fu, Ying and Wei, Kaixuan and Zheng, Dezhi and Heide, Felix},
  journal={International Journal of Computer Vision},
  volume={131},
  number={8},
  pages={2198--2218},
  year={2023},
  publisher={Springer}
}
```