README.md

---
license: mit
tags:
  - object-detection
  - open-vocabulary
  - dental
  - medical-imaging
---

# OralDetect-Family

Weights for **OralDetect**, an open-vocabulary dental object detector, and **OralCLIP**, the dental
image–text model its towers come from.

Code, configs and launchers: <https://github.com/OralGPT/OralDetect> ·
Data: [OralDetect-Training](https://huggingface.co/datasets/OralGPT/OralDetect-Training) ·
[OralDetect-Bench](https://huggingface.co/datasets/OralGPT/OralDetect-Bench)

## Files

| | size | what |
|---|--:|---|
| `OralDetect/oraldetect.pth` | 0.83 G | **the detector.** ConvNeXt-B vision + dental BERT text tower + modality calibration, 1024×1024, 87 classes |
| `OralDetect/oraldetect_init.pth` | 0.41 G | training init — towers loaded, neck/head still random. Start here to train from scratch |
| `OralDetect/class_names_oraldetect.json` | 3 K | the 87-class vocabulary (`list[str]`) |
| `OralDetect/class_texts_oraldetect.json` | 3 K | the prompts, `list[list[str]]`, same order |
| `OralCLIP/oralclip.pt` | 0.75 G | full dental CLIP — `vision.*` (ConvNeXt-B) + `text.*` + `logit_scale` |
| `OralCLIP/vision_tower.pt` | 0.34 G | the vision tower alone, keys prefixed `vision.` |
| `OralCLIP/oralbert/` | 0.41 G | dental BERT text tower, HF format — `AutoModel.from_pretrained` |

The vocabulary files are not optional. In an open-vocabulary detector **the class name is the
classifier**: the text tower embeds the string and that embedding *is* the class prototype.

## Results

Test bench, per-modality COCO bbox mAP:

| intraoral | panoramic | periapical | cytology | histology | **MACRO-5** |
|--:|--:|--:|--:|--:|--:|
| 0.611 | 0.802 | 0.528 | 0.341 | 0.600 | **0.577** |

**MACRO-5** is the equal-weight mean of the five per-modality mAPs, and it is the headline number.
Checkpoint selection uses MACRO-4 (histology excluded — its val split is 35 images and swings ±0.13
between adjacent epochs).

> ⚠️ **Do not report standard COCO `bbox_mAP` on this benchmark.** It is a macro over *classes*, not
> modalities, and panoramic owns 48 of the 87 class slots — 36 of them pure tooth/quadrant
> numbering. It largely measures tooth-counting on panoramic radiographs. (For reference only, this
> checkpoint scores 0.644 there.)

## Usage

```python
import torch
ck = torch.load("OralDetect/oraldetect.pth", map_location="cpu")
classes = ck["meta"]["dataset_meta"]["classes"]      # the 87 names
```

To run detection, use the repo's launcher and point the yaml at these files:

```yaml
paths:
  init:       /path/to/oraldetect.pth
  text_tower: /path/to/OralCLIP/oralbert      # HF-format directory
data:
  class_names: /path/to/class_names_oraldetect.json
  class_texts: /path/to/class_texts_oraldetect.json
```

Adding a class it was never trained on is a one-line edit to the vocabulary files — no retraining.

⚠️ The detector's architecture must match the checkpoint or `load_from` silently drops the
mismatched keys and you get a partly random model. The repo's `run_finetune.py` / `run_eval.py`
diff the state_dict up front and refuse to start on any mismatch. Three things must not change: the
dental BERT text tower, the modality calibration module, and 1024×1024 input.

## Provenance and licensing

⚠️ **These are derived weights.** They build on, and inherit the terms of:

- **WeDetect** ([WeChatCV/WeDetect](https://github.com/WeChatCV/WeDetect)) — the detector
  architecture and its base checkpoint, which `oraldetect_init.pth` is built on top of.
- **BiomedCLIP** — the starting point OralCLIP was contrastively finetuned from.
- **BiomedBERT / PubMedBERT** — the starting point the dental BERT tower was MLM-finetuned from.
- The 13 source detection datasets behind OralDetect-Training, whose licences are mixed and
  individually attributed in the dataset card.

The `license: mit` tag above covers our contribution only. **Check each upstream licence before
redistribution or commercial use** — we cannot relicense them. Research use.

Optimizer state, training configs and internal paths have been stripped from every file; what
remains is weights, the class vocabulary, and the epoch they came from.