cstr/lilt-base-GGUF

LiLT Base — GGUF

GGUF conversion of SCUT-DLVCLab/lilt-roberta-en-base for use with CrispEmbed.

LiLT (Language-independent Layout Transformer) is a dual-stream encoder that combines RoBERTa (768d text) with a parallel layout transformer (192d) via BiACM (bidirectional attention complementation). This is the base model (pre-trained, no task-specific head) — use it as a starting point for fine-tuning on your own document understanding tasks.

For a ready-to-use model fine-tuned on form understanding, see cstr/lilt-funsd-GGUF.

Model Details

Property	Value
Architecture	LiLT (RoBERTa + Layout Transformer + BiACM)
Parameters	130.7M
Hidden size	768 (text) / 192 (layout)
Layers	12
Heads	12
Vocab	50,265 (RoBERTa BPE)
License	MIT

Available Formats

File	Format	Size
	Float32	498 MB
	Q8_0	134 MB
	Q4_K	90 MB

Architecture

LiLT's key innovation is BiACM (Bidirectional Attention Complementation):

1. Text and layout streams each compute separate Q/K/V projections
2. Attention scores from both streams are summed before softmax
3. Each stream applies the combined attention to its own values
4. Separate FFN layers process each stream independently

This allows layout information to guide text attention patterns (and vice versa) without requiring pixel-level image features.

Layout Embeddings

Each token's bounding box [x0, y0, x1, y1] is encoded via 6 learned position embeddings (x, y, h, w) concatenated to 768d, projected to 192d, and combined with sequential position embeddings.

Parity

Verified against HuggingFace transformers:

• 25/25 encoder stages: cos_min = 1.000000
• max_abs < 1.6e-03 across all layers

Citation

Provenance and EU AI Act Art. 53 note

• Upstream model: SCUT-DLVCLab/lilt-roberta-en-base — published by SCUT-DLVCLab.
• Upstream licence: mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.