README.md

metadata

tags:
  - texformer
  - ocr
  - latex
  - seq2seq
  - pytorch
library_name: pytorch
pipeline_tag: text2text-generation

texformer-576m-fp32

This repository contains a TeXformer 576m checkpoint in fp32 precision for OCR-to-LaTeX generation.

This is a custom TeXformer architecture checkpoint (model.pt) plus tokenizer assets. It is not a standard transformers AutoModel checkpoint. This export is derived from a checkpoint trained in bf16.

Files

• model.pt: TeXformer checkpoint
• tokenizer/pdf_tokenizer.json: PDF-side tokenizer
• tokenizer/latex_tokenizer.json: LaTeX-side tokenizer
• tokenizer/pdf_tags.json: frequent PDF tag metadata
• tokenizer/latex_commands.json: frequent LaTeX command metadata

Architecture

• Parameters (deduplicated): 576,538,624
• Parameters (state_dict entries): 625,690,624
• Encoder layers: 16
• Decoder layers: 16
• Hidden size (d_model): 1024
• Attention heads: 16
• Feed-forward size (d_ff): 4224
• Max encoder length: 2560
• Max decoder length: 2560
• Stored precision: fp32

Quantization

• Quantization method: fp32
• Checkpoint payload key: model_state_dict
• Original model training precision: bf16
• Sample tensor dtype: torch.float32

Usage

from pathlib import Path
import torch
from huggingface_hub import snapshot_download
from texformer.models.checkpoint_loader import load_texformer_model
from texformer.tokenization.tokenizer import TeXFormerTokenizer

repo_id = "aamingem/texformer-576m-fp32"
local_dir = Path(snapshot_download(repo_id=repo_id))
tokenizer_dir = local_dir / "tokenizer"

tokenizer = TeXFormerTokenizer(tokenizer_dir)
if torch.cuda.is_available():
    device = torch.device("cuda")
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
    device = torch.device("mps")
else:
    device = torch.device("cpu")
model, global_step, epoch, missing, unexpected = load_texformer_model(
    checkpoint_path=local_dir / "model.pt",
    tokenizer=tokenizer,
    device=device,
)
print("Loaded model:", model.__class__.__name__)
print("Missing keys:", len(missing), "Unexpected keys:", len(unexpected))

Intended Use

• OCR-to-LaTeX / PDF-text-to-LaTeX sequence generation
• Research and experimentation on scientific document conversion

Limitations

• May produce incorrect or non-compiling LaTeX.
• Performance depends on input extraction quality.
• Not intended for high-stakes use without human verification.