| --- |
| language: |
| - en |
| license: gpl-3.0 |
| pipeline_tag: image-segmentation |
| library_name: reinpp |
| tags: |
| - semantic-segmentation |
| - domain-generalization |
| - domain-adaptation |
| - rein |
| - arxiv:2508.01667 |
| --- |
| |
| # Rein++ |
|
|
| Checkpoints for **Rein++: Efficient Generalization and Adaptation for Semantic Segmentation With Vision Foundation Models** (IEEE TPAMI 2026). |
|
|
| [Paper](https://doi.org/10.1109/TPAMI.2026.3681631) · [Inference and evaluation code](https://github.com/w1oves/Rein/tree/train/projects/rein%2B%2B) · [Validation report](VALIDATION.md) |
|
|
| ## First checkpoint release |
|
|
| We release the first batch of Rein++ checkpoints and inference and evaluation code for reproducing our paper results. This release includes five Rein-G models based on DINOv3-L and DINOv3-7B, and one Rein-A model, covering experiments in Tables II and VII. All six models have been evaluated on their corresponding validation sets. The remaining checkpoints and training code will be released progressively. |
|
|
| ## Models |
|
|
| Scores are mIoU (%). Paper results and results measured with the release are listed separately. |
|
|
| | Model ID | Method / backbone | Paper table | Paper | Re-evaluated | Download | |
| |---|---|---|---|---|---| |
| | `rdg_usg2cbm_dinov3_rein` | Rein-G / DINOv3-L | II | 78.9 / 63.4 / 74.8 | 78.91 / 63.44 / 74.78 | [config](rdg_usg2cbm_dinov3_rein/config.py) · [weights](rdg_usg2cbm_dinov3_rein/model.pth) | |
| | `rdg_usg2cbm_dinov3-7b_rein` | Rein-G / DINOv3-7B | II | 80.0 / 66.3 / 75.9 | 80.02 / 66.32 / 75.91 | [config](rdg_usg2cbm_dinov3-7b_rein/config.py) · [weights](rdg_usg2cbm_dinov3-7b_rein/model.pth) | |
| | `rdg_c2abm_dinov3_rein` | Rein-G / DINOv3-L | II | 72.5 / 65.0 / 77.4 | 72.52 / 65.04 / 77.41 | [config](rdg_c2abm_dinov3_rein/config.py) · [weights](rdg_c2abm_dinov3_rein/model.pth) | |
| | `rdg_c2abm_dinov3-7b_rein` | Rein-G / DINOv3-7B | II | 75.9 / 68.0 / 77.5 | 75.92 / 67.97 / 77.54 | [config](rdg_c2abm_dinov3-7b_rein/config.py) · [weights](rdg_c2abm_dinov3-7b_rein/model.pth) | |
| | `rdg_g2cbm_dinov3-7b_rein` | Rein-G / DINOv3-7B | II | 73.9 / 65.5 / 72.6 | 73.89 / 65.49 / 72.58 | [config](rdg_g2cbm_dinov3-7b_rein/config.py) · [weights](rdg_g2cbm_dinov3-7b_rein/model.pth) | |
| | `rda_g2c_dinov3_rein_ta` | Rein-A / DINOv3-L | VII | 79.0 | 79.03 | [config](rda_g2c_dinov3_rein_ta/config.py) · [weights](rda_g2c_dinov3_rein_ta/model.pth) | |
|
|
| Metric order and settings: |
|
|
| - `usg2cbm`: UrbanSyn + SYNTHIA + GTAV → Cityscapes / BDD100K / Mapillary. |
| - `c2abm`: Cityscapes → ACDC / BDD100K / Mapillary. |
| - `g2cbm`: GTAV → Cityscapes / BDD100K / Mapillary. |
| - `g2c`: GTAV → Cityscapes (domain adaptation). |
|
|
| All scores use validation sets. ACDC is the arithmetic mean of its night, snow, fog, and rain mIoU values. Table II's DINOv3-L GTAV-only model is planned for a later release. |
|
|
| Each model folder contains `model.pth`, `config.py`, and `manifest.json`. The checkpoint contains the Rein adapter parameters and segmentation head from the student branch used for evaluation. Load the matching DINOv3 backbone first, then these parameters. The manifests record complete SHA-256 checksums and the required backbone version; [model-index.json](model-index.json) collects the model metadata and results. |
|
|
| ## Download |
|
|
| ```bash |
| pip install -U huggingface_hub |
| hf download zhixiangwei/ReinPlusPlus --local-dir /absolute/path/ReinPlusPlus |
| ``` |
|
|
| To download one model: |
|
|
| ```bash |
| hf download zhixiangwei/ReinPlusPlus --include "rdg_usg2cbm_dinov3_rein/*" \ |
| --local-dir /absolute/path/ReinPlusPlus |
| ``` |
|
|
| For a full repository download, run `sha256sum -c SHA256SUMS` inside the downloaded directory to check the release files. |
|
|
| ## Base models |
|
|
| Obtain the original PyTorch LVD-1689M pretrained checkpoints from [DINOv3's official repository](https://github.com/facebookresearch/dinov3#pretrained-models): |
|
|
| | Backbone | File | SHA-256 | |
| |---|---|---| |
| | DINOv3-L | `dinov3_vitl16_pretrain_lvd1689m-8aa4cbdd.pth` | `8aa4cbddda325040fc78db2c272754af6ebe8ff2c55f6ec4f1964d8890f66035` | |
| | DINOv3-7B | `dinov3_vit7b16_pretrain_lvd1689m-a955f4ea.pth` | `a955f4ea3bec4fcd666bf363630da4386383069b482c8a927e17a3e1154965b7` | |
|
|
| Use the original PyTorch files with these configurations. Base checkpoints are downloaded separately under Meta's access and license terms. |
|
|
| ## Inference and evaluation |
|
|
| Inference and evaluation code is available in [projects/rein++](https://github.com/w1oves/Rein/tree/train/projects/rein%2B%2B). Follow the subproject README for installation and data preparation. |
|
|
| The validation environment uses Linux, Python 3.10, PyTorch 2.1.0 with CUDA 11.8, MMCV 2.1.0, MMEngine 0.10.7, MMSegmentation 1.2.2, and MMDetection 3.3.0. |
|
|
| From the Rein++ subproject directory, replace `MODEL_ID` with an entry in the table: |
|
|
| ```bash |
| python tools/infer.py /absolute/path/ReinPlusPlus/MODEL_ID/config.py \ |
| /absolute/path/ReinPlusPlus/MODEL_ID/model.pth /absolute/path/image.png \ |
| --backbone /absolute/path/dinov3_pretrain.pth \ |
| --output-dir /absolute/path/predictions |
| |
| python tools/test.py /absolute/path/ReinPlusPlus/MODEL_ID/config.py \ |
| /absolute/path/ReinPlusPlus/MODEL_ID/model.pth \ |
| --backbone /absolute/path/dinov3_pretrain.pth \ |
| --data-root /absolute/path/data --work-dir /absolute/path/evaluation |
| ``` |
|
|
| Evaluation also accepts `--cfg-options key=value`. |
|
|
| ## Labels and datasets |
|
|
| Predictions use Cityscapes train IDs 0–18, in this order: road, sidewalk, building, wall, fence, pole, traffic light, traffic sign, vegetation, terrain, sky, person, rider, car, truck, bus, train, motorcycle, bicycle. Ground-truth ID 255 is ignored. |
|
|
| The evaluations use Cityscapes validation (500 images), BDD100K validation (1,000), Mapillary Vistas v1.2 validation (2,000), and ACDC validation (406 across four conditions), as selected by each configuration. Mapillary labels use the experiment's mapping to the 19 Cityscapes classes. The code release provides data preparation instructions and label conversion tools. ACDC results here refer to validation, not the official test server. |
|
|
| ## Training |
|
|
| Training code will be released progressively. |
|
|
| ## License |
|
|
| The Rein adapter and segmentation-head weights are released under [GPL-3.0](LICENSE.txt). DINOv3 base models retain their original license. DINOv3-derived code and OpenMMLab components retain their respective licenses, documented in the code repository. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{Wei2026ReinPlusPlus, |
| author = {Wei, Zhixiang and Ma, Xiaoxiao and Yan, Ruishen and Tu, Tao and Chen, Huaian and Zheng, Jinjin and Jin, Yi and Chen, Enhong}, |
| title = {Rein++: Efficient Generalization and Adaptation for Semantic Segmentation With Vision Foundation Models}, |
| journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, |
| year = {2026}, |
| volume = {48}, |
| number = {8}, |
| doi = {10.1109/TPAMI.2026.3681631} |
| } |
| ``` |
|
|
