Mingze commited on
Commit
6373a17
·
1 Parent(s): 199fdb3

Document model attribution and UBC results

Browse files
Files changed (2) hide show
  1. README.md +66 -13
  2. app.py +3 -12
README.md CHANGED
@@ -37,19 +37,42 @@ short_description: Street image segmentation with SegFormer.
37
  ## Live demo
38
 
39
  The public [Hugging Face Space](https://huggingface.co/spaces/Mingze/StreetSceneSegmentation)
40
- runs entirely in the browser interface. Students can upload an image, choose a
41
- built-in example, inspect the results, and download the outputs without installing
42
  Python or providing an API key.
43
 
44
  ### Featured classroom example: UBC campus street
45
 
46
- ![UBC campus street with trees, pedestrians, and a cyclist](https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-main-mall.jpeg)
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  This scene provides a useful teaching example because it combines vegetation,
49
  sky, terrain, paving, buildings, poles, pedestrians, and a cyclist in one image.
50
  Students can compare the predicted area shares and inspect challenging boundaries
51
  between the tree canopy and sky, or between the paved route and grass.
52
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  ## What the project does
54
 
55
  The application uses
@@ -67,6 +90,24 @@ For each image, the app produces:
67
  | Raw class-ID PNG | Preserves the numeric prediction for later analysis |
68
  | CSV summary | Supports statistics, plotting, and classroom exercises |
69
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
70
  ## The 19 Cityscapes classes
71
 
72
  | Group | Classes | Example questions |
@@ -107,11 +148,10 @@ to limit memory use while preserving aspect ratio.
107
  - **Model literacy:** inspect boundary errors and discuss why predictions differ from human interpretation.
108
  - **Change over time:** process repeat photographs of a location and compare their CSV summaries.
109
 
110
- Five sample street scenes, led by the UBC campus example above, are loaded from the
 
111
  [HNIVision example collection](https://github.com/LabMingzeChen/HNIVision/tree/main/space/examples)
112
- and appear directly beneath the upload panel in the live app. Hosting samples
113
- through GitHub Raw follows the same lightweight pattern as HNI-VLM and avoids
114
- duplicating binary assets in the Space repository.
115
 
116
  ## Run locally
117
 
@@ -185,15 +225,28 @@ StreetSceneSegmentation/
185
  - Model-license and dataset terms remain governed by the linked model card and
186
  [Cityscapes](https://www.cityscapes-dataset.com/) terms.
187
 
188
- ## References
 
 
 
189
 
190
- - Enze Xie et al., [“SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers”](https://arxiv.org/abs/2105.15203), NeurIPS 2021.
191
- - Marius Cordts et al., [“The Cityscapes Dataset for Semantic Urban Scene Understanding”](https://arxiv.org/abs/1604.01685), CVPR 2016.
192
- - NVIDIA, [SegFormer-B0 Cityscapes model card](https://huggingface.co/nvidia/segformer-b0-finetuned-cityscapes-1024-1024).
 
 
 
 
193
 
194
- ## Citation
 
 
 
 
 
 
195
 
196
- If this software supports your teaching or research, cite the repository:
197
 
198
  ```bibtex
199
  @software{chen2026streetscenesegmentation,
 
37
  ## Live demo
38
 
39
  The public [Hugging Face Space](https://huggingface.co/spaces/Mingze/StreetSceneSegmentation)
40
+ runs entirely in the browser interface. Students can upload an image, choose the
41
+ UBC example, inspect the results, and download the outputs without installing
42
  Python or providing an API key.
43
 
44
  ### Featured classroom example: UBC campus street
45
 
46
+ <table>
47
+ <tr>
48
+ <th>Original image</th>
49
+ <th>Segmentation overlay</th>
50
+ <th>Cityscapes color mask</th>
51
+ </tr>
52
+ <tr>
53
+ <td><img src="https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-main-mall.jpeg" alt="UBC campus street input"></td>
54
+ <td><img src="https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-segmentation-overlay.webp" alt="UBC campus segmentation overlay"></td>
55
+ <td><img src="https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-segmentation-mask.webp" alt="UBC campus Cityscapes color mask"></td>
56
+ </tr>
57
+ </table>
58
 
59
  This scene provides a useful teaching example because it combines vegetation,
60
  sky, terrain, paving, buildings, poles, pedestrians, and a cyclist in one image.
61
  Students can compare the predicted area shares and inspect challenging boundaries
62
  between the tree canopy and sky, or between the paved route and grass.
63
 
64
+ The result above was generated by the live Space at 2048 × 1536 px with an
65
+ overlay opacity of 0.55. The largest predicted classes were:
66
+
67
+ | Predicted class | Image share |
68
+ |---|---:|
69
+ | Vegetation | 42.87% |
70
+ | Sky | 20.86% |
71
+ | Road | 20.73% |
72
+ | Terrain | 11.50% |
73
+ | Building | 1.76% |
74
+ | Sidewalk | 1.07% |
75
+
76
  ## What the project does
77
 
78
  The application uses
 
90
  | Raw class-ID PNG | Preserves the numeric prediction for later analysis |
91
  | CSV summary | Supports statistics, plotting, and classroom exercises |
92
 
93
+ ## Model and dataset attribution
94
+
95
+ This repository provides the Gradio application and output utilities; it did not
96
+ train the segmentation model or create its training dataset. The external
97
+ resources used by the app are credited below.
98
+
99
+ | Borrowed resource | Creator and citation | How it is used |
100
+ |---|---|---|
101
+ | [SegFormer architecture](https://arxiv.org/abs/2105.15203) | Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo (2021) | Transformer encoder and lightweight MLP decoder architecture |
102
+ | [`nvidia/segformer-b0-finetuned-cityscapes-1024-1024`](https://huggingface.co/nvidia/segformer-b0-finetuned-cityscapes-1024-1024) | NVIDIA checkpoint; model card maintained on Hugging Face | Public pretrained weights used for inference |
103
+ | [Cityscapes dataset](https://www.cityscapes-dataset.com/) | Marius Cordts et al. (CVPR 2016) | Fine-tuning dataset, 19 evaluation classes, and standard class palette |
104
+
105
+ The MIT license in this repository covers **only this project's original code**.
106
+ It does not relicense the model weights or Cityscapes. The Hugging Face model
107
+ card lists the checkpoint license as `other`, and Cityscapes is subject to its
108
+ [official terms](https://www.cityscapes-dataset.com/license/), including
109
+ non-commercial-use and attribution conditions. Review those terms before reuse.
110
+
111
  ## The 19 Cityscapes classes
112
 
113
  | Group | Classes | Example questions |
 
148
  - **Model literacy:** inspect boundary errors and discuss why predictions differ from human interpretation.
149
  - **Change over time:** process repeat photographs of a location and compare their CSV summaries.
150
 
151
+ The UBC campus image above is the only built-in example and appears directly
152
+ beneath the upload panel in the live app. It is hosted in the
153
  [HNIVision example collection](https://github.com/LabMingzeChen/HNIVision/tree/main/space/examples)
154
+ and loaded through GitHub Raw, following the same lightweight pattern as HNI-VLM.
 
 
155
 
156
  ## Run locally
157
 
 
225
  - Model-license and dataset terms remain governed by the linked model card and
226
  [Cityscapes](https://www.cityscapes-dataset.com/) terms.
227
 
228
+ ## Cite the model and dataset
229
+
230
+ If you use this application in research, cite both the SegFormer architecture
231
+ and the Cityscapes dataset:
232
 
233
+ ```bibtex
234
+ @article{xie2021segformer,
235
+ title = {SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers},
236
+ author = {Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anandkumar, Anima and Alvarez, Jose M. and Luo, Ping},
237
+ journal = {Advances in Neural Information Processing Systems},
238
+ year = {2021}
239
+ }
240
 
241
+ @inproceedings{Cordts2016Cityscapes,
242
+ title = {The Cityscapes Dataset for Semantic Urban Scene Understanding},
243
+ author = {Cordts, Marius and Omran, Mohamed and Ramos, Sebastian and Rehfeld, Timo and Enzweiler, Markus and Benenson, Rodrigo and Franke, Uwe and Roth, Stefan and Schiele, Bernt},
244
+ booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
245
+ year = {2016}
246
+ }
247
+ ```
248
 
249
+ You may additionally cite this software:
250
 
251
  ```bibtex
252
  @software{chen2026streetscenesegmentation,
app.py CHANGED
@@ -41,16 +41,7 @@ SAMPLE_ROOT = (
41
  "https://raw.githubusercontent.com/"
42
  "LabMingzeChen/HNIVision/main/space/examples"
43
  )
44
- SAMPLE_IMAGES = [
45
- f"{SAMPLE_ROOT}/{filename}"
46
- for filename in (
47
- "ubc-campus-main-mall.jpeg",
48
- "140546.jpg",
49
- "145541.jpg",
50
- "148831.jpg",
51
- "160710.jpg",
52
- )
53
- ]
54
 
55
 
56
  @lru_cache(maxsize=1)
@@ -185,8 +176,8 @@ with gr.Blocks(title="Street Scene Segmentation", theme=gr.themes.Soft(), css=CS
185
  gr.Examples(
186
  examples=SAMPLE_IMAGES,
187
  inputs=image_input,
188
- label="Or try a sample street scene",
189
- examples_per_page=4,
190
  )
191
  with gr.Accordion("Display settings", open=False):
192
  opacity_input = gr.Slider(
 
41
  "https://raw.githubusercontent.com/"
42
  "LabMingzeChen/HNIVision/main/space/examples"
43
  )
44
+ SAMPLE_IMAGES = [f"{SAMPLE_ROOT}/ubc-campus-main-mall.jpeg"]
 
 
 
 
 
 
 
 
 
45
 
46
 
47
  @lru_cache(maxsize=1)
 
176
  gr.Examples(
177
  examples=SAMPLE_IMAGES,
178
  inputs=image_input,
179
+ label="Try the UBC campus street example",
180
+ examples_per_page=1,
181
  )
182
  with gr.Accordion("Display settings", open=False):
183
  opacity_input = gr.Slider(