Spaces:
Running on Zero
Running on Zero
Document model attribution and UBC results
Browse files
README.md
CHANGED
|
@@ -37,19 +37,42 @@ short_description: Street image segmentation with SegFormer.
|
|
| 37 |
## Live demo
|
| 38 |
|
| 39 |
The public [Hugging Face Space](https://huggingface.co/spaces/Mingze/StreetSceneSegmentation)
|
| 40 |
-
runs entirely in the browser interface. Students can upload an image, choose
|
| 41 |
-
|
| 42 |
Python or providing an API key.
|
| 43 |
|
| 44 |
### Featured classroom example: UBC campus street
|
| 45 |
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
This scene provides a useful teaching example because it combines vegetation,
|
| 49 |
sky, terrain, paving, buildings, poles, pedestrians, and a cyclist in one image.
|
| 50 |
Students can compare the predicted area shares and inspect challenging boundaries
|
| 51 |
between the tree canopy and sky, or between the paved route and grass.
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
## What the project does
|
| 54 |
|
| 55 |
The application uses
|
|
@@ -67,6 +90,24 @@ For each image, the app produces:
|
|
| 67 |
| Raw class-ID PNG | Preserves the numeric prediction for later analysis |
|
| 68 |
| CSV summary | Supports statistics, plotting, and classroom exercises |
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
## The 19 Cityscapes classes
|
| 71 |
|
| 72 |
| Group | Classes | Example questions |
|
|
@@ -107,11 +148,10 @@ to limit memory use while preserving aspect ratio.
|
|
| 107 |
- **Model literacy:** inspect boundary errors and discuss why predictions differ from human interpretation.
|
| 108 |
- **Change over time:** process repeat photographs of a location and compare their CSV summaries.
|
| 109 |
|
| 110 |
-
|
|
|
|
| 111 |
[HNIVision example collection](https://github.com/LabMingzeChen/HNIVision/tree/main/space/examples)
|
| 112 |
-
and
|
| 113 |
-
through GitHub Raw follows the same lightweight pattern as HNI-VLM and avoids
|
| 114 |
-
duplicating binary assets in the Space repository.
|
| 115 |
|
| 116 |
## Run locally
|
| 117 |
|
|
@@ -185,15 +225,28 @@ StreetSceneSegmentation/
|
|
| 185 |
- Model-license and dataset terms remain governed by the linked model card and
|
| 186 |
[Cityscapes](https://www.cityscapes-dataset.com/) terms.
|
| 187 |
|
| 188 |
-
##
|
|
|
|
|
|
|
|
|
|
| 189 |
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 193 |
|
| 194 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 195 |
|
| 196 |
-
|
| 197 |
|
| 198 |
```bibtex
|
| 199 |
@software{chen2026streetscenesegmentation,
|
|
|
|
| 37 |
## Live demo
|
| 38 |
|
| 39 |
The public [Hugging Face Space](https://huggingface.co/spaces/Mingze/StreetSceneSegmentation)
|
| 40 |
+
runs entirely in the browser interface. Students can upload an image, choose the
|
| 41 |
+
UBC example, inspect the results, and download the outputs without installing
|
| 42 |
Python or providing an API key.
|
| 43 |
|
| 44 |
### Featured classroom example: UBC campus street
|
| 45 |
|
| 46 |
+
<table>
|
| 47 |
+
<tr>
|
| 48 |
+
<th>Original image</th>
|
| 49 |
+
<th>Segmentation overlay</th>
|
| 50 |
+
<th>Cityscapes color mask</th>
|
| 51 |
+
</tr>
|
| 52 |
+
<tr>
|
| 53 |
+
<td><img src="https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-main-mall.jpeg" alt="UBC campus street input"></td>
|
| 54 |
+
<td><img src="https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-segmentation-overlay.webp" alt="UBC campus segmentation overlay"></td>
|
| 55 |
+
<td><img src="https://raw.githubusercontent.com/LabMingzeChen/HNIVision/main/space/examples/ubc-campus-segmentation-mask.webp" alt="UBC campus Cityscapes color mask"></td>
|
| 56 |
+
</tr>
|
| 57 |
+
</table>
|
| 58 |
|
| 59 |
This scene provides a useful teaching example because it combines vegetation,
|
| 60 |
sky, terrain, paving, buildings, poles, pedestrians, and a cyclist in one image.
|
| 61 |
Students can compare the predicted area shares and inspect challenging boundaries
|
| 62 |
between the tree canopy and sky, or between the paved route and grass.
|
| 63 |
|
| 64 |
+
The result above was generated by the live Space at 2048 × 1536 px with an
|
| 65 |
+
overlay opacity of 0.55. The largest predicted classes were:
|
| 66 |
+
|
| 67 |
+
| Predicted class | Image share |
|
| 68 |
+
|---|---:|
|
| 69 |
+
| Vegetation | 42.87% |
|
| 70 |
+
| Sky | 20.86% |
|
| 71 |
+
| Road | 20.73% |
|
| 72 |
+
| Terrain | 11.50% |
|
| 73 |
+
| Building | 1.76% |
|
| 74 |
+
| Sidewalk | 1.07% |
|
| 75 |
+
|
| 76 |
## What the project does
|
| 77 |
|
| 78 |
The application uses
|
|
|
|
| 90 |
| Raw class-ID PNG | Preserves the numeric prediction for later analysis |
|
| 91 |
| CSV summary | Supports statistics, plotting, and classroom exercises |
|
| 92 |
|
| 93 |
+
## Model and dataset attribution
|
| 94 |
+
|
| 95 |
+
This repository provides the Gradio application and output utilities; it did not
|
| 96 |
+
train the segmentation model or create its training dataset. The external
|
| 97 |
+
resources used by the app are credited below.
|
| 98 |
+
|
| 99 |
+
| Borrowed resource | Creator and citation | How it is used |
|
| 100 |
+
|---|---|---|
|
| 101 |
+
| [SegFormer architecture](https://arxiv.org/abs/2105.15203) | Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo (2021) | Transformer encoder and lightweight MLP decoder architecture |
|
| 102 |
+
| [`nvidia/segformer-b0-finetuned-cityscapes-1024-1024`](https://huggingface.co/nvidia/segformer-b0-finetuned-cityscapes-1024-1024) | NVIDIA checkpoint; model card maintained on Hugging Face | Public pretrained weights used for inference |
|
| 103 |
+
| [Cityscapes dataset](https://www.cityscapes-dataset.com/) | Marius Cordts et al. (CVPR 2016) | Fine-tuning dataset, 19 evaluation classes, and standard class palette |
|
| 104 |
+
|
| 105 |
+
The MIT license in this repository covers **only this project's original code**.
|
| 106 |
+
It does not relicense the model weights or Cityscapes. The Hugging Face model
|
| 107 |
+
card lists the checkpoint license as `other`, and Cityscapes is subject to its
|
| 108 |
+
[official terms](https://www.cityscapes-dataset.com/license/), including
|
| 109 |
+
non-commercial-use and attribution conditions. Review those terms before reuse.
|
| 110 |
+
|
| 111 |
## The 19 Cityscapes classes
|
| 112 |
|
| 113 |
| Group | Classes | Example questions |
|
|
|
|
| 148 |
- **Model literacy:** inspect boundary errors and discuss why predictions differ from human interpretation.
|
| 149 |
- **Change over time:** process repeat photographs of a location and compare their CSV summaries.
|
| 150 |
|
| 151 |
+
The UBC campus image above is the only built-in example and appears directly
|
| 152 |
+
beneath the upload panel in the live app. It is hosted in the
|
| 153 |
[HNIVision example collection](https://github.com/LabMingzeChen/HNIVision/tree/main/space/examples)
|
| 154 |
+
and loaded through GitHub Raw, following the same lightweight pattern as HNI-VLM.
|
|
|
|
|
|
|
| 155 |
|
| 156 |
## Run locally
|
| 157 |
|
|
|
|
| 225 |
- Model-license and dataset terms remain governed by the linked model card and
|
| 226 |
[Cityscapes](https://www.cityscapes-dataset.com/) terms.
|
| 227 |
|
| 228 |
+
## Cite the model and dataset
|
| 229 |
+
|
| 230 |
+
If you use this application in research, cite both the SegFormer architecture
|
| 231 |
+
and the Cityscapes dataset:
|
| 232 |
|
| 233 |
+
```bibtex
|
| 234 |
+
@article{xie2021segformer,
|
| 235 |
+
title = {SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers},
|
| 236 |
+
author = {Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anandkumar, Anima and Alvarez, Jose M. and Luo, Ping},
|
| 237 |
+
journal = {Advances in Neural Information Processing Systems},
|
| 238 |
+
year = {2021}
|
| 239 |
+
}
|
| 240 |
|
| 241 |
+
@inproceedings{Cordts2016Cityscapes,
|
| 242 |
+
title = {The Cityscapes Dataset for Semantic Urban Scene Understanding},
|
| 243 |
+
author = {Cordts, Marius and Omran, Mohamed and Ramos, Sebastian and Rehfeld, Timo and Enzweiler, Markus and Benenson, Rodrigo and Franke, Uwe and Roth, Stefan and Schiele, Bernt},
|
| 244 |
+
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
|
| 245 |
+
year = {2016}
|
| 246 |
+
}
|
| 247 |
+
```
|
| 248 |
|
| 249 |
+
You may additionally cite this software:
|
| 250 |
|
| 251 |
```bibtex
|
| 252 |
@software{chen2026streetscenesegmentation,
|
app.py
CHANGED
|
@@ -41,16 +41,7 @@ SAMPLE_ROOT = (
|
|
| 41 |
"https://raw.githubusercontent.com/"
|
| 42 |
"LabMingzeChen/HNIVision/main/space/examples"
|
| 43 |
)
|
| 44 |
-
SAMPLE_IMAGES = [
|
| 45 |
-
f"{SAMPLE_ROOT}/{filename}"
|
| 46 |
-
for filename in (
|
| 47 |
-
"ubc-campus-main-mall.jpeg",
|
| 48 |
-
"140546.jpg",
|
| 49 |
-
"145541.jpg",
|
| 50 |
-
"148831.jpg",
|
| 51 |
-
"160710.jpg",
|
| 52 |
-
)
|
| 53 |
-
]
|
| 54 |
|
| 55 |
|
| 56 |
@lru_cache(maxsize=1)
|
|
@@ -185,8 +176,8 @@ with gr.Blocks(title="Street Scene Segmentation", theme=gr.themes.Soft(), css=CS
|
|
| 185 |
gr.Examples(
|
| 186 |
examples=SAMPLE_IMAGES,
|
| 187 |
inputs=image_input,
|
| 188 |
-
label="
|
| 189 |
-
examples_per_page=
|
| 190 |
)
|
| 191 |
with gr.Accordion("Display settings", open=False):
|
| 192 |
opacity_input = gr.Slider(
|
|
|
|
| 41 |
"https://raw.githubusercontent.com/"
|
| 42 |
"LabMingzeChen/HNIVision/main/space/examples"
|
| 43 |
)
|
| 44 |
+
SAMPLE_IMAGES = [f"{SAMPLE_ROOT}/ubc-campus-main-mall.jpeg"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
|
| 47 |
@lru_cache(maxsize=1)
|
|
|
|
| 176 |
gr.Examples(
|
| 177 |
examples=SAMPLE_IMAGES,
|
| 178 |
inputs=image_input,
|
| 179 |
+
label="Try the UBC campus street example",
|
| 180 |
+
examples_per_page=1,
|
| 181 |
)
|
| 182 |
with gr.Accordion("Display settings", open=False):
|
| 183 |
opacity_input = gr.Slider(
|