xumingzhu989 commited on
Commit
f6d7c55
Β·
verified Β·
1 Parent(s): 10c1ff8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +162 -3
README.md CHANGED
@@ -1,3 +1,162 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - semantic-segmentation
5
+ - knowledge-distillation
6
+ - multimodal
7
+ - model-compression
8
+ - pytorch
9
+ ---
10
+
11
+ <a id="top"></a>
12
+ <div align="center">
13
+ <h1>πŸš€ HMKD-ICMR: Heterogeneous Model Knowledge Distillation via Dual Alignment for Semantic Segmentation</h1>
14
+
15
+ <p>
16
+ <b>Mingzhu Xu</b><sup>1</sup>&nbsp;
17
+ <b>Jing Wang</b><sup>1</sup>&nbsp;
18
+ <b>Mingcai Wang</b><sup>1</sup>&nbsp;
19
+ <b>Yiping Li</b><sup>1</sup>&nbsp;
20
+ <b>Yupeng Hu</b><sup>1βœ‰</sup>&nbsp;
21
+ <b>Xuemeng Song</b><sup>1</sup>&nbsp;
22
+ <b>Weili Guan</b><sup>1</sup>
23
+ </p>
24
+
25
+ <p>
26
+ <sup>1</sup>Affiliation (Please update if needed)
27
+ </p>
28
+ </div>
29
+
30
+ Official implementation of **HMKD**, a Heterogeneous Model Knowledge Distillation framework with Dual Alignment for Semantic Segmentation.
31
+
32
+ πŸ”— **Conference:** ICMR 2025
33
+ πŸ”— **Task:** Semantic Segmentation
34
+ πŸ”— **Framework:** PyTorch
35
+
36
+ ---
37
+
38
+ ## πŸ“Œ Model Information
39
+
40
+ ### 1. Model Name
41
+ **HMKD** (Heterogeneous Model Knowledge Distillation)
42
+
43
+ ---
44
+
45
+ ### 2. Task Type & Applicable Tasks
46
+ - **Task Type:** Semantic Segmentation / Model Compression
47
+ - **Core Task:** Knowledge Distillation for segmentation
48
+ - **Applicable Scenarios:**
49
+ - Lightweight model deployment
50
+ - Cross-architecture distillation
51
+ - Efficient semantic understanding
52
+
53
+ ---
54
+
55
+ ### 3. Project Introduction
56
+
57
+ Semantic segmentation models often rely on heavy architectures, limiting their deployment in resource-constrained environments. Knowledge distillation (KD) provides a promising solution by transferring knowledge from a large teacher model to a compact student model.
58
+
59
+ **HMKD** introduces a **Dual Alignment Distillation Framework**, which:
60
+
61
+ - Aligns heterogeneous architectures between teacher and student models
62
+ - Performs **feature-level and prediction-level alignment**
63
+ - Bridges the representation gap across different model families
64
+ - Improves segmentation accuracy while maintaining efficiency
65
+
66
+ ---
67
+
68
+ ### 4. Training Data Source
69
+
70
+ Supported datasets:
71
+
72
+ - **Cityscapes**
73
+ - **CamVid**
74
+
75
+ | Dataset | Train | Val | Test | Classes |
76
+ |--------|------|-----|------|--------|
77
+ | Cityscapes | 2975 | 500 | 1525 | 19 |
78
+ | CamVid | 367 | 101 | 233 | 11 |
79
+
80
+ ---
81
+
82
+ ## πŸš€ Environment Setup
83
+
84
+ - Ubuntu 20.04.4 LTS
85
+ - Python 3.8.10 (Anaconda recommended)
86
+ - CUDA 11.3
87
+ - PyTorch 1.11.0
88
+ - NCCL 2.10.3
89
+
90
+ ### Install dependencies:
91
+
92
+ ```bash
93
+ pip install timm==0.3.2
94
+ pip install mmcv-full==1.2.7
95
+ pip install opencv-python==4.5.1.48
96
+ ```
97
+
98
+ ---
99
+
100
+ ## βš™οΈ Pre-trained Weights
101
+
102
+ ### Initialization Weights
103
+
104
+ - ResNet-18
105
+ - ResNet-101
106
+ - SegFormer-B0
107
+ - SegFormer-B4
108
+
109
+ (Download from official PyTorch and Google Drive links)
110
+
111
+ ---
112
+
113
+ ### Trained Weights
114
+
115
+ Download trained HMKD models:
116
+
117
+ - Baidu Cloud: https://pan.baidu.com/s/1xw_6ts5VNV73vXeOLAokwQ?pwd=jvx8
118
+
119
+ ---
120
+
121
+ ## πŸš€ Training
122
+
123
+ 1. Download datasets and pre-trained weights
124
+ 2. Generate dataset path lists (.txt files)
125
+ 3. Update dataset paths in the code
126
+
127
+ ### Run training:
128
+
129
+ ```bash
130
+ CUDA_VISIBLE_DEVICES=0,1 nohup python -m torch.distributed.launch --nproc_per_node=2 train_NEW_AEU_kd.py > train.log 2>&1 &
131
+
132
+ CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 train_NEW_AEU_kd.py
133
+ ```
134
+
135
+ ---
136
+
137
+ ## ⚠️ Notes
138
+
139
+ - Designed for research purposes
140
+ - Performance depends on teacher-student architecture pairing
141
+ - Multi-GPU training is recommended
142
+
143
+ ---
144
+
145
+ ## πŸ“ Citation
146
+
147
+ ```bibtex
148
+ @ARTICLE{HMKD,
149
+ author={Xu, Mingzhu and Wang, Jing and Wang, Mingcai and Li, Yiping and Hu, Yupeng and Song, Xuemeng and Guan, Weili},
150
+ journal={ICMR},
151
+ title={Heterogeneous Model Knowledge Distillation via Dual Alignment for Semantic Segmentation},
152
+ year={2025}
153
+ }
154
+ ```
155
+
156
+ ---
157
+
158
+ ## πŸ“¬ Contact
159
+
160
+ For questions or collaboration, please contact the corresponding author.
161
+
162
+ ---