dchen0/font-classifier

Font Classifier

A DINOv2 Vision Transformer fine-tuned with LoRA for font classification across 394 font variants from 32 Google Fonts families.

How it was made

1. Base model: facebook/dinov2-base-imagenet1k-1-layer (87.2M parameters, frozen).
2. Fine-tuning: LoRA (rank 8, alpha 16) applied to the query and value projections in each ViT attention block, plus a trainable classification head. ~900K trainable parameters (1% of total).
3. Promotion: This model was promoted from the lora_r8/result_model adapter in dchen0/font-model-results using promote_model.py. That script loads the base DINOv2 model, merges the LoRA adapter weights into it (merge_and_unload()), and uploads the result as a standalone checkpoint. No adapter or PEFT library needed at inference time.

Performance

• 99.0% top-1 accuracy on 394 font classes (held-out test set)
• 99.8% family-level accuracy (collapsing weight variants into parent families)
• Errors are overwhelmingly within-family weight confusions (e.g. Roboto-400 vs Roboto-500), not cross-family misidentifications

Method	Trainable Params	Top-1 Acc
LoRA r=8 (this model)	900K	99.0%
ResNet-50	25.6M	98.8%
LoRA r=16	1.2M	98.9%
LoRA r=4	753K	97.9%
Full Fine-Tuning	87.2M	95.9%

Training data

dchen0/font_crops_v5 — ~225K synthetic images generated by rendering random text in each font variant. ~575 training images and 40 test images per class. Images include color augmentation, layout variation (left/center/right alignment, multi-line), and Gaussian noise.

Font families (32)

BigShouldersText, BricolageGrotesque, CrimsonPro, DMSans, Geist, HedvigLettersSerif, InstrumentSans, InstrumentSerif, Inter, JetBrainsMono, LexendDeca, Lora, Merriweather, Montserrat, Newsreader, NunitoSans, Onest, OpenSans, Petrona, PlayfairDisplay, PlusJakartaSans, Poppins, PT Serif Caption, RethinkSans, Roboto, RobotoSerif, ShipporiMincho, Sora, SpaceGrotesk, Ultra, Urbanist, WorkSans

Training details

Hyperparameter	Value
Optimizer	AdamW
Learning rate	1e-4
Batch size	64
Epochs	100
LR scheduler	Linear decay
Precision	FP16
LoRA rank	8
LoRA alpha	16
LoRA dropout	0.1
LoRA targets	query, value
GPU	NVIDIA RTX 3090 (24 GB)
Training time	~33 hours

Preprocessing

Preprocessing is built into handler.py and must match at inference time:

1. Convert to RGB
2. Pad to square (black fill, centered)
3. Resize to 224x224
4. Normalize with ImageNet stats (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

Usage

from transformers import Dinov2ForImageClassification, AutoImageProcessor
from handler import get_inference_transform
from PIL import Image
import torch

model = Dinov2ForImageClassification.from_pretrained("dchen0/font-classifier")
processor = AutoImageProcessor.from_pretrained("dchen0/font-classifier")
model.eval()

transform = get_inference_transform(processor, processor.size["shortest_edge"])
image = Image.open("font_sample.png").convert("RGB")
pixel_values = transform(image).unsqueeze(0)

with torch.no_grad():
    logits = model(pixel_values=pixel_values).logits

predicted_class = logits.argmax(-1).item()
print(model.config.id2label[predicted_class])

Source

• Training code: github.com/Create-Inc/font-model
• Results repo (checkpoints, logs): dchen0/font-model-results
• Dataset: dchen0/font_crops_v5