README.md

metadata

license: mit
tags:
  - object-detection
  - open-vocabulary
  - dental
  - medical-imaging

OralDetect-Family

Weights for OralDetect, an open-vocabulary dental object detector, and OralCLIP, the dental image–text model its towers come from.

Code, configs and launchers: https://github.com/OralGPT/OralDetect · Data: OralDetect-Training · OralDetect-Bench

Files

	size	what
OralDetect/oraldetect.pth	0.83 G	the detector. ConvNeXt-B vision + dental BERT text tower + modality calibration, 1024×1024, 87 classes
OralDetect/oraldetect_init.pth	0.41 G	training init — towers loaded, neck/head still random. Start here to train from scratch
OralDetect/class_names_oraldetect.json	3 K	the 87-class vocabulary (list[str])
OralDetect/class_texts_oraldetect.json	3 K	the prompts, list[list[str]], same order
OralCLIP/oralclip.pt	0.75 G	full dental CLIP — vision.* (ConvNeXt-B) + text.* + logit_scale
OralCLIP/vision_tower.pt	0.34 G	the vision tower alone, keys prefixed vision.
OralCLIP/oralbert/	0.41 G	dental BERT text tower, HF format — AutoModel.from_pretrained

The vocabulary files are not optional. In an open-vocabulary detector the class name is the classifier: the text tower embeds the string and that embedding is the class prototype.

Results

Test bench, per-modality COCO bbox mAP:

intraoral	panoramic	periapical	cytology	histology	MACRO-5
0.611	0.802	0.528	0.341	0.600	0.577

MACRO-5 is the equal-weight mean of the five per-modality mAPs, and it is the headline number. Checkpoint selection uses MACRO-4 (histology excluded — its val split is 35 images and swings ±0.13 between adjacent epochs).

⚠️ Do not report standard COCO bbox_mAP on this benchmark. It is a macro over classes, not modalities, and panoramic owns 48 of the 87 class slots — 36 of them pure tooth/quadrant numbering. It largely measures tooth-counting on panoramic radiographs. (For reference only, this checkpoint scores 0.644 there.)

Usage

import torch
ck = torch.load("OralDetect/oraldetect.pth", map_location="cpu")
classes = ck["meta"]["dataset_meta"]["classes"]      # the 87 names

To run detection, use the repo's launcher and point the yaml at these files:

paths:
  init:       /path/to/oraldetect.pth
  text_tower: /path/to/OralCLIP/oralbert      # HF-format directory
data:
  class_names: /path/to/class_names_oraldetect.json
  class_texts: /path/to/class_texts_oraldetect.json

Adding a class it was never trained on is a one-line edit to the vocabulary files — no retraining.

⚠️ The detector's architecture must match the checkpoint or load_from silently drops the mismatched keys and you get a partly random model. The repo's run_finetune.py / run_eval.py diff the state_dict up front and refuse to start on any mismatch. Three things must not change: the dental BERT text tower, the modality calibration module, and 1024×1024 input.

Provenance and licensing

⚠️ These are derived weights. They build on, and inherit the terms of:

• WeDetect (WeChatCV/WeDetect) — the detector architecture and its base checkpoint, which oraldetect_init.pth is built on top of.
• BiomedCLIP — the starting point OralCLIP was contrastively finetuned from.
• BiomedBERT / PubMedBERT — the starting point the dental BERT tower was MLM-finetuned from.
• The 13 source detection datasets behind OralDetect-Training, whose licences are mixed and individually attributed in the dataset card.

The license: mit tag above covers our contribution only. Check each upstream licence before redistribution or commercial use — we cannot relicense them. Research use.

Optimizer state, training configs and internal paths have been stripped from every file; what remains is weights, the class vocabulary, and the epoch they came from.