README.md

---
license: mit
library_name: transformers
pipeline_tag: image-to-text
base_model: breezedeus/pix2text-mfr-1.5
base_model_relation: finetune
tags:
  - typst
  - math-ocr
  - formula-recognition
  - onnx
  - vision-encoder-decoder
  - browser
  - grayscale
---

# TypLens V1.1

TypLens V1.1 converts an image of a printed mathematical formula directly into
Typst math source. Its 29.2-million-parameter model can run locally in a browser
using the supplied full-precision or compact ONNX graphs.

Repository name: **TypLens**. Model version: **V1.1**. Release tag: **v1.1**.

[Hugging Face model repository](https://huggingface.co/dbcccc/TypLens) · [GitHub repository](https://github.com/dbccccccc/TypLens)

## What changed from V1

- A newly trained **single-channel grayscale** model replaces V1's RGB encoder
  input. Training ran for eight epochs (33,544 updates) on the original 268,346
  training pairs, starting from the upstream-derived native-vocabulary
  initializer rather than continuing the released V1 checkpoint.
- Training and inference use the same luminance conversion, background polarity
  normalization, content bounding box, 2% margin and 384×384 bicubic resize.
- Full FP32 and compact dynamic INT8 ONNX exports come from the final checkpoint.
  The vocabulary and native Typst output format remain unchanged.

**Integration change:** V1.1 requires [1, 1, 384, 384] tensors and the custom
preprocessing described in [INFERENCE.md](INFERENCE.md). Replace the preprocessing
and both model graphs together. V1's RGB input pipeline is incompatible.

## Files and sizes

| Format | Files in the Hugging Face pack | Weight bytes | Download size, decimal MB |
|---|---|---:|---:|
| Native FP32 | model.safetensors | 116,870,000 | 116.9 |
| Browser FP32 | onnx/fp32/encoder.onnx + decoder.onnx | 117,177,351 | 117.2 |
| Browser compact | onnx/int8/encoder.onnx + decoder.onnx | 33,120,766 | 33.1 |

The compact ONNX weights are 71.7% smaller than the FP32 ONNX weights. These
numbers exclude tokenizer metadata and the application's runtime. Compact means
dynamic INT8 linear weights; some operations and the caches remain FP32.
The GitHub release provides a separate ZIP for each ONNX variant.

This distribution contains weights, tokenizer/configuration, documentation and
license notices. It contains no training code, optimizer state, training images
or inference runtime binaries. See [release-manifest.json](release-manifest.json)
for file hashes and [INFERENCE.md](INFERENCE.md) for integration.

## Recorded evaluation

The final server FP32 checkpoint was reviewed on **135 reused IBEM development
images** with greedy generation and a maximum of 1,023 new tokens:

| Measure | Result |
|---|---:|
| Natural end of generation | 135/135 |
| Typst compilation | 135/135 |
| Content accepted by source-image review | 127/135 (94.1%) |
| Exact match to one reference source string | 76/135 (56.3%) |
| Accepted despite a different reference source string | 51 |

Equivalent Typst spellings and nonessential spacing differences were accepted;
visible symbols, indices, accents and matrix structure still had to match.
The reviewer was an AI assistant, without an independent second annotator.
38 cases were reviewed anew; 97 judgments were inherited only when both the
source image hash and complete prediction matched the earlier source review.
This reused development set also informed checkpoint selection, so **94.1% is
not an independent test accuracy**. It is a server FP32 result, not a measured
compact-model accuracy. No controlled V1 versus V1.1 accuracy comparison is claimed.

Both ONNX variants completed three real desktop Edge/WASM smoke-test images
each. Input tensors matched the training preprocessing and generated tokens
matched their CPU ONNX references; on these three images both variants also
matched server FP32 outputs. One image retained a known accent error. These
checks establish basic execution/parity, not full accuracy or absence of
quantization loss. See [evaluation-summary.json](evaluation-summary.json).

## Intended use and limitations

Use one printed formula crop from a PDF, paper or screenshot per request.
This model does not convert full documents. Handwriting, mobile inference and
WebGPU performance have not been evaluated for this release. Small accents,
similar symbols, missing indices and complex block matrices remain failure cases.
Inspect the generated formula before use. Applications can show partial or
validation-failed output with a warning so that it remains editable.

## Provenance and license

The model derives from [Pix2Text-MFR-1.5](https://huggingface.co/breezedeus/pix2text-mfr-1.5)
(pinned revision `1cef9f0bdcd6a4c63df7de1311fb0894593340cc`), with a native Typst
vocabulary and a one-channel image projection. It has 29,206,656 parameters and
uses 1,199 tokens. Source checkpoint SHA-256:
`d42cfd2da242a11ecdb1fe8727ee01e6896a116cd46f74357c4fdce90eebe788`.

The release retains **MIT** for project contributions, with the Pix2Text and
Microsoft MIT notices. Training data came from UniMER and IBEM. Dataset attribution
and component-level provenance limits are retained in [NOTICE.md](NOTICE.md).
The model license does not relicense those datasets. Keep [LICENSE](LICENSE),
NOTICE.md and licenses/ when redistributing the weights.