| --- |
| license: apache-2.0 |
| --- |
| |
| # SFM: Spatial Frequency Modulation for Semantic Segmentation (TPAMI 2025) |
|
|
| Official checkpoints for the paper **Spatial Frequency Modulation for Semantic Segmentation** |
| (TPAMI 2025, [arXiv:2507.11893](https://arxiv.org/abs/2507.11893), |
| DOI [10.1109/TPAMI.2025.3592621](https://doi.org/10.1109/TPAMI.2025.3592621)). |
|
|
| Code: **https://github.com/Linwei-Chen/SFM** |
|
|
| Two released technical lines: |
|
|
| - **Line 1 — FCN-style models**: ARS + qhull non-uniform upsampling (MSAU, paper Eq. 17/18) |
| + cascaded LPRM, built on mmsegmentation 0.25.0. |
| Eval code and the 18 configs: [`SFM_mmseg0x/`](https://github.com/Linwei-Chen/SFM/tree/main/SFM_mmseg0x). |
| - **Line 2 — FPN**: ARS + CARAFE-style feature reassembly with 3-point dynamic-sampling |
| weighted alignment, on Mask2Former (mmsegmentation 1.x). |
| Code and config: [`SFM_mask2former/`](https://github.com/Linwei-Chen/SFM/tree/main/SFM_mask2former). |
|
|
| Every checkpoint below is **verified from training/eval logs**: each measured mIoU either |
| exceeds the paper-reported number or matches it after one-decimal rounding |
| (e.g. PSPNet-R50 D16 measures 78.05, rounding to the paper's 78.1). Each model folder |
| contains `config.py` (the exact config paired with that checkpoint) and the weight file. |
|
|
| ## Line 1 — Cityscapes (val mIoU, ResNet-50) |
|
|
| | folder | paper | measured | |
| | --- | --- | --- | |
| | `line1/cityscapes/ccnet_r50SS3PR-d32_cityscapes` | 76.3 | **76.34** | |
| | `line1/cityscapes/ccnet_r50SS3PR-d16_cityscapes` | 78.5 | 78.45 | |
| | `line1/cityscapes/ccnet_r50SS3PR-d8_cityscapes` | 79.3 | **79.39** | |
| | `line1/cityscapes/isanet_r50SS3PR-d32_cityscapes` (OCNet) | 75.8 | **75.82** | |
| | `line1/cityscapes/isanet_r50SS3PR-d16_cityscapes` (OCNet) | 78.0 | **78.00** | |
| | `line1/cityscapes/isanet_r50SS3PR-d8_cityscapes` (OCNet) | 79.7 | **79.74** | |
| | `line1/cityscapes/FCN_r50SS3PR-d32_cityscapes` | 76.2 | **76.20** | |
| | `line1/cityscapes/FCN_r50SS3PR-d16_cityscapes` | 77.8 | 77.78 | |
| | `line1/cityscapes/FCN_r50SS3PR-d8_cityscapes` | 78.7 | 78.65 | |
| | `line1/cityscapes/pspnet_r50SS3PR-d32_cityscapes` | 76.0 | **76.18** | |
| | `line1/cityscapes/pspnet_r50SS3PR-d16_cityscapes` | 78.1 | 78.05 | |
| | `line1/cityscapes/pspnet_r50SS3PR-d8_cityscapes` | 79.5 | **79.52** | |
| | `line1/cityscapes/pcaa_r50SSPR-d32_cityscapes` (PCAA, col. 1) | 76.8 | **76.83** | |
| | `line1/cityscapes/pcaa_r50SSPR-d16_cityscapes` (PCAA, col. 2) | 77.9 | 77.86 | |
|
|
| Weight file in each folder: `iter_40000.pth`. |
|
|
| ## Line 1 — PascalContext-59 (val mIoU, ResNet-50) |
|
|
| | folder | paper | measured | |
| | --- | --- | --- | |
| | `line1/pascalcontext/FCN_r50SS3PR-d32_pascalcontext` | 46.0 | **46.35** | |
| | `line1/pascalcontext/pspnet_r50SS3PR-d32_pascalcontext` | 49.7 | 49.68 | |
| | `line1/pascalcontext/ccnet_r50SS3PR-d32_pascalcontext` | 49.1 | 49.07 | |
| | `line1/pascalcontext/isanet_r50SS3PR-d32_pascalcontext` (OCNet) | 49.9 | 49.85 | |
|
|
| ## Line 2 — ADE20K (val mIoU, Mask2Former Swin-T) |
|
|
| | folder | paper | measured | |
| | --- | --- | --- | |
| | `line2/mask2former_swin-t_8xb2-160k_ade20k-512x512_SFMv2` | 49.1 | **49.34** (re-evaluated 2025-10) | |
|
|
| Weight file: `best_mIoU_iter_150000.pth`. |
|
|
| ## Evaluation |
|
|
| **Line 1** (see [`SFM_mmseg0x/README.md`](https://github.com/Linwei-Chen/SFM/blob/main/SFM_mmseg0x/README.md) |
| for environment setup and qhull compilation): |
|
|
| ```bash |
| cd SFM_mmseg0x # from the GitHub repo checkout |
| PYTHONPATH=<SFM_mmseg0x> python tools/test.py \ |
| <model_folder>/config.py <model_folder>/iter_40000.pth --eval mIoU |
| ``` |
|
|
| **Line 2** (code: [`SFM_mask2former/`](https://github.com/Linwei-Chen/SFM/tree/main/SFM_mask2former), |
| mmsegmentation 1.x): |
|
|
| ```bash |
| cd SFM_mask2former |
| python tools/test.py \ |
| <model_folder>/config.py <model_folder>/best_mIoU_iter_150000.pth |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{chen2023spatialfrequency, |
| title={Spatial Frequency Modulation for Semantic Segmentation}, |
| author={Chen, Linwei and Fu, Ying and Gu, Lin and Zheng, Dezhi and Dai, Jifeng}, |
| journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, |
| year={2025}, |
| volume={47}, |
| number={11}, |
| pages={9767-9784}, |
| doi={10.1109/TPAMI.2025.3592621} |
| } |
| ``` |
|
|
| Code is released under the Apache-2.0 license. |
|
|
