forkger/manga-panel-rfdetr

Manga Panel Detector (RF-DETR-Medium, Core ML)

An on-device object detector for manga pages with three classes: panel, text (speech and caption regions) and body (character figures). It is a fine-tune of RF-DETR Medium, trained on Manga109-s and converted to native Core ML (FP16) for iPhone and iPad.

It powers the panel-by-panel reading mode of KMReader, an open-source reader for Komga and KMServer on Apple platforms. The model only outputs boxes, so other apps can use it too.

Files

File	Format	Size
manga-panel-rfdetr-med-576-fp16.mlpackage	Core ML ML Program (FP16), iOS 16+ / macOS 13+	58 MB

Input

• One image input named image: 576×576 RGB.
• Resize the whole page to 576×576 directly (stretch, no letterbox).
• Scaling (1/255) and ImageNet normalization are built into the model. Pass plain RGB pixels.

Output

Two FP16 arrays. The converter named them automatically, so match them by shape:

Shape	Meaning
[1, 300, 4]	300 candidate boxes as (cx, cy, w, h), normalized to [0, 1] of the page
[1, 300, 5]	Class logits per candidate

Class channels: 1 = panel, 2 = text, 3 = body. Channels 0 and 4 are unused.

Decoding: apply a sigmoid to the logits and keep a candidate for a class when its score passes the threshold. No non-maximum suppression is needed (DETR-style set prediction). Thresholds used in KMReader:

Class	Threshold
panel	0.40
text	0.50
body	0.50

Reading order, overlap cleanup and zoom framing are not part of the model. KMReader's panel mode is one open-source implementation of them.

Compute units

On A14 and newer, load with MLComputeUnits.all to use the Neural Engine. Some older chips (A12) cannot build the model for the Neural Engine or GPU and fail to load with error -5; fall back to .cpuAndGPU, then .cpuOnly. In the iOS Simulator use .cpuOnly, since other settings return invalid scores there.

The first load on a device builds and caches a device-specific version: a few seconds on A18, a minute or two on A14. Later loads are fast.

Performance

Accuracy on 4 Manga109-s books (724 pages) held out from training and validation, measured on the PyTorch model before conversion:

Class	AP@[.5:.95]	AP@50
panel	0.911	0.977
text	0.866	0.976
body	0.667	0.901
all	0.815	0.951

Speed per page on device (Core ML, FP16):

Device	Compute units	Time per page
iPhone 12 mini (A14)	Neural Engine (.all)	~85–230 ms
iPhone XR (A12)	CPU fallback (.cpuOnly)	~730 ms
iPhone 16 (A18)	Neural Engine (.all)	~30–170 ms (typically ~50 ms)

Training

• Base: RF-DETR-Medium, COCO-pretrained checkpoint.
• Data: Manga109-s (v2026), 87 volumes (15,588 single pages; two-page spreads were split). Panels come from the frame annotations, text from text, bodies from body.
• Split by book, with no book in more than one split: 70 train, 13 validation, 4 test.
• Resolution 576, 50 epochs, checkpoint chosen by best EMA validation mAP@[.5:.95].

Limitations

• Borderless panels are detected less reliably. Manga109 annotates bordered frames best, and the model inherits that. A reader should treat "no panels found" as "show the whole page".
• Manga109-s contains only Japanese manga, so expect lower accuracy on webtoons, Western comics and very different art styles.
• text is a region detector without text recognition. body boxes mark figures, not who they are.

License and attribution

• The model weights are released under Apache-2.0, fine-tuned from RF-DETR by Roboflow (Apache-2.0).
• The model was trained on the Manga109-s dataset. Its terms require that the use of Manga109-s is indicated clearly wherever results such as pre-trained models are published. If you use or republish this model, keep this attribution with it.
• This repository contains no Manga109-s images or annotations.

Citation

If you use this model, please cite Manga109 and RF-DETR:

@inproceedings{baek2026mangav26,
  author    = {Baek, Jeonghun and Miyai, Atsuyuki and Onohara, Shota and Ikuta, Hikaru and Aizawa, Kiyoharu},
  title     = {Manga109-v2026: Revisiting Manga109 Annotations for Modern Manga Understanding},
  booktitle = {Culture {\times} AI Workshop at ICML 2026},
  year      = {2026}
}

@article{multimedia_aizawa_2020,
  author  = {Aizawa, Kiyoharu and Fujimoto, Azuma and Otsubo, Atsushi and Ogawa, Toru and Matsui, Yusuke and Tsubota, Koki and Ikuta, Hikaru},
  title   = {Building a Manga Dataset ``Manga109'' with Annotations for Multimedia Applications},
  journal = {IEEE MultiMedia},
  volume  = {27},
  number  = {2},
  pages   = {8--18},
  year    = {2020},
  doi     = {10.1109/mmul.2020.2987895}
}

@article{mtap_matsui_2017,
  author  = {Matsui, Yusuke and Ito, Kota and Aramaki, Yuji and Fujimoto, Azuma and Ogawa, Toru and Yamasaki, Toshihiko and Aizawa, Kiyoharu},
  title   = {Sketch-based Manga Retrieval using Manga109 Dataset},
  journal = {Multimedia Tools and Applications},
  volume  = {76},
  number  = {20},
  pages   = {21811--21838},
  year    = {2017},
  doi     = {10.1007/s11042-016-4020-z}
}

@inproceedings{robinson2026rfdetr,
  author    = {Robinson, Isaac and Robicheaux, Peter and Popov, Matvei and Ramanan, Deva and Peri, Neehar},
  title     = {RF-DETR: Real-Time Detection Transformer},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2511.09554}
}