cstr/pix2tex-mfr-gguf

pix2tex-mfr GGUF — Math Formula Recognition

GGUF-format models for on-device math OCR, converted from breezedeus/pix2text-mfr (MIT license).

Recognizes printed mathematical expressions in images and outputs LaTeX.

Architecture

• Encoder: DeiT (Data-efficient Image Transformer) — 12 layers, 6 heads, hidden=384
• Decoder: TrOCR (Transformer OCR) — 6 layers, 8 heads, d_model=256, vocab=1200
• Input: 384×384 RGB image, patch size 16
• Output: LaTeX token sequence (BPE, 1200 tokens)
• Parameters: 29.3M

Available Variants

File	Format	Size	Quality	Use case
pix2tex-mfr-f32.gguf	FP32	112 MB	Reference	Development/debugging
pix2tex-mfr-f16.gguf	FP16	56 MB	~F32	Desktop, full quality
pix2tex-mfr-q8_0.gguf	Q8_0	31 MB	≥99.5%	Desktop, balanced
pix2tex-mfr-q4_k.gguf	Q4_K	17 MB	≥98%	Mobile, smallest

Usage with CrispEmbed

#include "math_ocr.h"

math_ocr_context *ctx = math_ocr_init("pix2tex-mfr-q8_0.gguf", 4);
int len;
const char *latex = math_ocr_recognize(ctx, grayscale_pixels, width, height, &len);
printf("LaTeX: %s\n", latex);  // e.g. "\frac{x^{2}+1}{x-1}"
math_ocr_free(ctx);

Usage with CrispCalc

CrispCalc (Flutter CAS calculator) integrates this model via its OCR feature:

1. Tap the 📷 button on the Calculator or Notepad screen
2. Take a photo or select from gallery
3. The model runs on-device via CrispEmbed FFI
4. Review the recognized expression and insert

Conversion

Converted from ONNX using:

python models/convert-pix2tex-to-gguf.py \
    --model-dir /path/to/pix2text-mfr \
    --output pix2tex-mfr-f16.gguf --fp16

Quantized using:

crispembed-quantize pix2tex-mfr-f16.gguf pix2tex-mfr-q8_0.gguf q8_0
crispembed-quantize pix2tex-mfr-f16.gguf pix2tex-mfr-q4_k.gguf q4_k

Accuracy

Verified against the ONNX reference model:

• Encoder output: cosine similarity 0.999983 (F32)
• Decoder logits: match to 3 decimal places
• End-to-end: produces identical LaTeX tokens for test images

License

• Model weights: MIT (from breezedeus/pix2text-mfr)
• GGUF converter: AGPL-3.0 (CrispEmbed)
• CrispCalc integration: AGPL-3.0

Part of the Crisp ecosystem

Project	Role
CrispCalc	Flutter CAS calculator with OCR
CrispEmbed	ggml inference engine (text + vision + math OCR)
CrispASR	Speech recognition engine