File size: 3,833 Bytes
b528527 8f92600 b528527 8f92600 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | ---
license: mit
library_name: onnx
pipeline_tag: image-feature-extraction
language:
- mul
inference: false
tags:
- onnx
- computer-vision
- font-recognition
- typography
- image-embeddings
- document-analysis
- multi-task
---
# FontDNA
**FontDNA** is an ONNX model for recognizing typography from **word-image crops**.
It returns a font embedding plus word-local typography predictions. Font family is not a fixed classifier: compare the embedding against embeddings of your own reference fonts.
## Model I/O
**Inputs**
- `img` — `float32`, shape `(B, 1, 40, W)`
- `cols` — `int64`, shape `(B,)`; real crop width divided by `8`
Word crops are grayscale, resized to height `40` while preserving aspect ratio. Width is aligned to multiples of `8` (max `320`). Scale pixels to `[0,1]`, then normalize each crop with:
```python
x = (x - x.mean()) / (x.std() + 1e-4)
```
**Outputs, in order**
| # | Output | Decode as |
|---|---|---|
| 0 | font embedding | L2-normalize, then use cosine similarity |
| 1 | style logits `[4]` | `sigmoid` → `bold, italic, underline, strikethrough` |
| 2 | category logits `[5]` | `softmax` → `serif, sans-serif, display, handwriting, monospace` |
| 3 | confidence logit | `sigmoid` |
| 4 | script logits `[14]` | `softmax` → mapping below |
Script indices:
```text
0 latin 5 hangul 10 tamil
1 cyrillic 6 arabic 11 thai
2 greek 7 hebrew 12 armenian
3 han 8 devanagari 13 georgian
4 kana 9 bengali
```
Styles are **per word**. For example, one underlined word can still share the same font-family embedding/section as the surrounding regular words.
## Raw ONNX Runtime example
```python
import numpy as np
import onnxruntime as ort
session = ort.InferenceSession("glyphdna.onnx")
# img: preprocessed float32 array shaped (B, 1, 40, W)
# cols: int64 array shaped (B,), where cols[i] = real_width // 8
embedding, style_logits, category_logits, confidence_logits, script_logits = (
session.run(None, {"img": img, "cols": cols})
)
sigmoid = lambda x: 1 / (1 + np.exp(-np.clip(x, -60, 60)))
softmax = lambda x: np.exp(x - x.max(-1, keepdims=True)) / np.exp(
x - x.max(-1, keepdims=True)
).sum(-1, keepdims=True)
embedding = embedding / np.maximum(
np.linalg.norm(embedding, axis=1, keepdims=True), 1e-8
)
styles = sigmoid(style_logits)
categories = softmax(category_logits)
confidence = sigmoid(confidence_logits).reshape(-1)
scripts = softmax(script_logits)
# Example: first word
print("bold:", styles[0, 0])
print("italic:", styles[0, 1])
print("underline:", styles[0, 2])
print("strikethrough:", styles[0, 3])
print("category index:", categories[0].argmax())
print("script index:", scripts[0].argmax())
```
If your exported ONNX uses different tensor names, inspect them with:
```python
print([x.name for x in session.get_inputs()])
print([x.name for x in session.get_outputs()])
```
## Font matching
To recognize a font family, embed reference word renders from each candidate font with the same model and preprocessing. L2-normalize the vectors and rank candidates by cosine similarity. Combining several neighboring word embeddings is usually more reliable than trusting a single word, because some words contain few font-discriminative glyphs.
A standalone Python library for indexing fonts, segmenting images, clustering same-font sections, and preserving per-word formatting is planned separately and is expected to be released in a few weeks.
## Notes
- This model is **not OCR** and does not transcribe text.
- Exact font names come from your own reference/index collection.
- Bold, italic, underline, strikethrough, category, and script are predicted per word.
- Precompiled font indexes must use the same model weights and preprocessing.
## License
MIT.
|