anonimkaq4/manga-page-element-segmentation

Manga Page Element Segmentation

YOLO11 instance-segmentation checkpoints for three core elements of a manga page: panel frames, people, and speech bubbles.

What the model recognizes

The model predicts an instance mask, bounding box, class label, and confidence score for every detected object. It has three classes:

Output label	Meaning	Typical use
frame	A manga panel/frame. The mask follows the visible panel area or border.	Split a page into panels before reading-order or per-panel analysis.
character (person)	A visible person or character. The exported checkpoint label is character; this is the model's person class.	Find people for cropping, composition analysis, or character-aware editing.
speech_bubble	A dialogue/speech balloon.	Send dialogue regions to OCR or exclude them from artwork processing.

character and person refer to the same class in this card. Use the literal label character when reading result.names from Ultralytics.

The repository contains two YOLO11 checkpoints. manga-yolo11m-seg-validation-best.pt is the recommended checkpoint: it has the strongest validation result among the files currently published.

Checkpoints

Checkpoint	Base model	Training snapshot	Mask mAP50	Mask mAP50-95
snapshots/yolo11m_1024_b2/manga-yolo11m-seg-validation-best.pt	yolo11m-seg.pt	epoch 15	0.8742	0.6078
snapshots/yolo11s_1024_b8/manga-yolo11s-seg-validation-best.pt	yolo11s-seg.pt	epoch 18	0.8649	0.5895

Metrics were recorded by Ultralytics on the held-out validation split used during each run, at 1024 px. They are useful for comparing these snapshots, not as a claim of performance on every manga style, scan quality, language, or page layout.

Quick start

pip install ultralytics huggingface_hub

from huggingface_hub import hf_hub_download
from ultralytics import YOLO

checkpoint = hf_hub_download(
    repo_id="anonimkaq4/manga-page-element-segmentation",
    filename="snapshots/yolo11m_1024_b2/manga-yolo11m-seg-validation-best.pt",
)

model = YOLO(checkpoint)
results = model("manga_page.png", imgsz=1024, conf=0.25)

for result in results:
    print(result.names)  # {0: 'frame', 1: 'speech_bubble', 2: 'character'}
    result.save(filename="prediction.png")

For a smaller model, change the filename to snapshots/yolo11s_1024_b8/manga-yolo11s-seg-validation-best.pt.

Training details

• Task: instance segmentation
• Architecture: Ultralytics YOLO11 segmentation
• Input resolution: 1024 × 1024
• Classes: frame, speech_bubble, character (person)
• Data: Manga109-s supplied the manga-page images. MangaSeg / MS92-MangaSegmentation supplied the pixel-accurate instance annotations in COCO RLE format. Those annotations were converted to YOLO segmentation polygons for training.
• Source-label mapping: MangaSeg frame → frame; balloon → speech_bubble; body → character. The model was not trained to predict MangaSeg text, face, or onomatopoeia labels.
• Augmentation: Ultralytics default training augmentation; horizontal flip enabled

Exact run settings and learning curves are kept alongside every checkpoint in args.yaml, dataset.yaml, and results.csv.

Intended use

These weights are intended as a research/prototyping component for manga-page analysis: proposing panel frames, people/characters, and dialogue balloons before downstream OCR, layout analysis, or editing workflows. Run a visual check on your own pages before using predictions in production.

Limitations

• This is an experimental snapshot, not a finished general-purpose manga detector.
• Dense action scenes, stylized or partial characters, overlapping balloons, double-page spreads, borders without frames, and low-quality scans can reduce quality.
• The model recognizes only the three listed classes. Text, sound effects, backgrounds, and reading order are out of scope.
• Training data and validation split details should be considered when comparing these numbers with other work.

Usage and license

This repository does not grant rights to the underlying manga artwork or annotations. Manga109-s and MangaSeg have separate licenses and conditions; review both before use or redistribution. In particular, MangaSeg requires the credit "Copyrighted by Minshan Xie" in publications, reproductions, redistributions, or derivative works that use its images. When publishing results derived from Manga109-s, clearly acknowledge use of Manga109-s and follow its author-credit and page-publication restrictions. Also comply with the license of the Ultralytics base model/software. This experimental checkpoint is shared without warranty; contact the repository owner before redistributing it or using it in a commercial product.

Citation

This model was trained using MangaSeg annotations. If you use the model, its training data, or derived results, cite the MangaSeg paper:

@inproceedings{xie2025advancing,
  title={Advancing Manga Analysis: Comprehensive Segmentation Annotations for the Manga109 Dataset},
  author={Xie, Minshan and Lin, Jian and Liu, Hanyuan and Li, Chengze and Wong, Tien-Tsin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2025}
}

For the source manga pages and their original layout annotations, also cite Manga109-s:

@article{multimedia_aizawa_2020,
  title={Building a Manga Dataset ``Manga109'' with Annotations for Multimedia Applications},
  author={Aizawa, Kiyoharu and Fujimoto, Azuma and Otsubo, Atsushi and Ogawa, Toru and Matsui, Yusuke and Tsubota, Kota and Ikuta, Hikaru},
  journal={IEEE MultiMedia},
  volume={27},
  number={2},
  pages={8--18},
  year={2020},
  doi={10.1109/MMUL.2020.2987895}
}

Files

Each checkpoint directory contains:

• manga-yolo11*-seg-validation-best.pt — checkpoint selected by validation metric;
• results.csv — per-epoch training and validation metrics;
• args.yaml and dataset.yaml — run configuration;
• snapshot_metadata.json — compact summary of the training run.