Update README.md
Browse files
README.md
CHANGED
|
@@ -1,7 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# Anime Line Art Extraction Segmentation Model
|
| 2 |
<div style="display:flex;">
|
| 3 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/6kCBB668giXjJoCLXAzfy.png" width="50%">
|
| 4 |
-
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/Q5pwvCCAfhl5ctqgsVEPa.png" width="50%">
|
|
|
|
| 5 |
|
| 6 |
## Model Description
|
| 7 |
|
|
@@ -35,7 +53,7 @@ Frameworks used:
|
|
| 35 |
|
| 36 |
Since no pretrained model exists specifically for anime line extraction, the model was trained using a custom dataset and automatically generated edge masks.
|
| 37 |
|
| 38 |
-
|
| 39 |
|
| 40 |
### Intended Use Cases
|
| 41 |
Potential applications include:
|
|
@@ -198,7 +216,7 @@ IoU: ~0.21
|
|
| 198 |
|
| 199 |
These metrics indicate that the model is able to detect meaningful edge structures but struggles with extremely thin line details.
|
| 200 |
|
| 201 |
-
|
| 202 |
|
| 203 |
## Key Observations
|
| 204 |
|
|
@@ -208,7 +226,7 @@ What worked well:
|
|
| 208 |
- Captured hair boundaries
|
| 209 |
- Detected facial structures
|
| 210 |
|
| 211 |
-
|
| 212 |
|
| 213 |
Failure cases:
|
| 214 |
|
|
@@ -216,10 +234,10 @@ Failure cases:
|
|
| 216 |
- Dark scenes
|
| 217 |
- Shading lines interpreted as edges
|
| 218 |
- Excessive background detail
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
These results show that the model learned meaningful edge structures despite the noisy annotations generated from Canny edge detection.
|
| 224 |
|
| 225 |
## Visual Examples
|
|
@@ -243,11 +261,11 @@ However, the task presents several challenges:
|
|
| 243 |
3. Low contrast scenes reduce edge detectability
|
| 244 |
|
| 245 |
Because the model was only trained for 30 epochs, additional training may improve performance. However, improving annotation quality or training at higher resolution would likely have a larger impact.
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
|
| 249 |
-
|
| 250 |
-
|
| 251 |
--------------------------------------------------
|
| 252 |
|
| 253 |
# Limitations and Biases
|
|
@@ -319,12 +337,12 @@ Additional research directions include:
|
|
| 319 |
|
| 320 |
- Object detection models for automatic removal of occlusions
|
| 321 |
|
| 322 |
-
|
| 323 |
|
| 324 |
- Line art upscaling techniques
|
| 325 |
|
| 326 |
-
|
| 327 |
|
| 328 |
- Using detected edges for stitching animation panning shots
|
| 329 |
|
| 330 |
-
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
tags:
|
| 5 |
+
- computer-vision
|
| 6 |
+
- image-segmentation
|
| 7 |
+
- edge-detection
|
| 8 |
+
- line-art
|
| 9 |
+
- anime
|
| 10 |
+
datasets:
|
| 11 |
+
- custom
|
| 12 |
+
metrics:
|
| 13 |
+
- dice
|
| 14 |
+
- iou
|
| 15 |
+
pipeline_tag: image-segmentation
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
# Anime Line Art Extraction Segmentation Model
|
| 19 |
<div style="display:flex;">
|
| 20 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/6kCBB668giXjJoCLXAzfy.png" width="50%">
|
| 21 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/Q5pwvCCAfhl5ctqgsVEPa.png" width="50%">
|
| 22 |
+
</div>
|
| 23 |
|
| 24 |
## Model Description
|
| 25 |
|
|
|
|
| 53 |
|
| 54 |
Since no pretrained model exists specifically for anime line extraction, the model was trained using a custom dataset and automatically generated edge masks.
|
| 55 |
|
| 56 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/h06ej-ODkw5tDAx3X6KfL.png" width="80%">
|
| 57 |
|
| 58 |
### Intended Use Cases
|
| 59 |
Potential applications include:
|
|
|
|
| 216 |
|
| 217 |
These metrics indicate that the model is able to detect meaningful edge structures but struggles with extremely thin line details.
|
| 218 |
|
| 219 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/zvCczs-TB241YW4FuVIOF.png" width="35%">
|
| 220 |
|
| 221 |
## Key Observations
|
| 222 |
|
|
|
|
| 226 |
- Captured hair boundaries
|
| 227 |
- Detected facial structures
|
| 228 |
|
| 229 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/dlkCHCrPtBJPvy7sGSc8j.png" width="75%">
|
| 230 |
|
| 231 |
Failure cases:
|
| 232 |
|
|
|
|
| 234 |
- Dark scenes
|
| 235 |
- Shading lines interpreted as edges
|
| 236 |
- Excessive background detail
|
| 237 |
+
<div>
|
| 238 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/Hi0LQhIQZvWlAd_44o88H.png" width="75%">
|
| 239 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/KnIMiKkePB9aDNausGNDp.png" width="75%">
|
| 240 |
+
</div>
|
| 241 |
These results show that the model learned meaningful edge structures despite the noisy annotations generated from Canny edge detection.
|
| 242 |
|
| 243 |
## Visual Examples
|
|
|
|
| 261 |
3. Low contrast scenes reduce edge detectability
|
| 262 |
|
| 263 |
Because the model was only trained for 30 epochs, additional training may improve performance. However, improving annotation quality or training at higher resolution would likely have a larger impact.
|
| 264 |
+
<div style="display:flex;">
|
| 265 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/PL9-L1MHMEhqmNxY4WQkm.png" width="50%">
|
| 266 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/ru-gguNfSDzxbCXT6kmeS.png" width="50%">
|
| 267 |
+
</div>
|
| 268 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/wT3J4LSINPNHVVjLUaqcR.png" width="50%">
|
| 269 |
--------------------------------------------------
|
| 270 |
|
| 271 |
# Limitations and Biases
|
|
|
|
| 337 |
|
| 338 |
- Object detection models for automatic removal of occlusions
|
| 339 |
|
| 340 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/NKCNnMBSAzzhAPjaZiX9y.png" width="50%">
|
| 341 |
|
| 342 |
- Line art upscaling techniques
|
| 343 |
|
| 344 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/9cisCYIkU_y45UJtJRNcE.png" width="50%">
|
| 345 |
|
| 346 |
- Using detected edges for stitching animation panning shots
|
| 347 |
|
| 348 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/6972a2622ef5ed3b50628995/ZDIrGENzx4oy-Vj_jyQMa.gif" width="50%">
|