gemma-4-12b-it-structure-lora-peft

Give it a policy document one page image at a time, and it returns the whole document as one nested JSON tree, text verbatim, nested as printed. LoRA adapter for google/gemma-4-12B-it. A llama.cpp version of the same weights is at webAI-Official/gemma-4-12b-it-structure-lora-GGUF.

How it works

The model reads one page per request and returns that page as a JSON tree: sections, subsections, numbered clauses, lettered items and definitions, text copied verbatim. A node that continues from an earlier page is written as a stub ({"cont": true, "type": "clause", "label": "12."}) and the merge code joins it to the tree built so far, so content that runs across a page break comes back as one node. Each page's prompt carries the open path (the chain of nodes still open at the end of the previous page, from the model's own output).

Serving recipe (required)

Callers must follow this input recipe; the adapter was trained on it and nothing else.

1. Render each PDF page at 200 DPI.
2. Send each page as two full-width strips, top half then bottom half, each reaching 4% of the page height into the other (extract.py, page_strips).
3. Serve with an image budget of 560 tokens per image (--mm-processor-kwargs '{"max_soft_tokens": 560}', or "mm_processor_kwargs": {"max_soft_tokens": 560} per request). Do not use 1120: vLLM 0.28 garbles a 1,120-token Gemma 4 image (Transformers reads it fine), and the base model and the adapter both fall into repetition loops.
4. User turn: the two strips, then the text The page is shown as 2 overlapping strips, top to bottom., a blank line, and the page prompt (Open path: [] (this is the first page) + Extract this page. for page 1; the open path of the tree merged so far for later pages).
5. System prompt system_prompt.txt; thinking off (chat_template_kwargs: {"enable_thinking": false}); greedy (temperature: 0); max_tokens: 4096; no presence or repetition penalty (they corrupt verbatim text).
6. Pages of one document in order; parse each reply with page_merge.parse_page and attach it with page_merge.merge_lenient (extract.py does all of this).

Getting started (vLLM)

vllm serve google/gemma-4-12B-it --served-model-name gemma-4-12b-it \
  --enable-lora --max-lora-rank 16 \
  --lora-modules structure=webAI-Official/gemma-4-12b-it-structure-lora-peft \
  --mm-processor-kwargs '{"max_soft_tokens": 560}' --max-model-len 32768

pip install openai pillow pypdfium2
hf download webAI-Official/gemma-4-12b-it-structure-lora-peft extract.py page_merge.py system_prompt.txt --local-dir .
python extract.py policy.pdf > policy.json

With Transformers + PEFT (as it was trained): PeftModel.from_pretrained on AutoModelForCausalLM.from_pretrained("google/gemma-4-12B-it"), the chat template rendered with enable_thinking=False, the two strips passed as two images and max_soft_tokens=560 to the processor (AutoProcessor).

Evaluation

The page loop run on documents held out from training, for the adapter and for the base model with the same system prompt, strips and decoding; each page's prompt built from that model's own earlier pages. On the original adapter's test (the first table) the original Qwen3.5-9B adapter scores node F1 0.89 with this pipeline.

Held-out insurance policies (GEICO auto, Encompass home; the original adapter's test)

2 documents, 34 pages, scored against their reference trees.

Metric	google/gemma-4-12B-it	This adapter
Pages attached	34/34	34/34
Node F1	0.758	0.870
Precision	0.798	0.860
Recall	0.723	0.881
Right parent, relaxed	0.633	0.771
Right parent, strict	0.472	0.607
Node type accuracy	0.479	0.973
Text exact (matched nodes)	0.806	0.877
Whole-document word similarity	0.953	0.992
Words captured	0.927	0.991

Held-out public-domain documents (flood insurance policy forms, Regulation E; two layouts each)

8 documents, 138 pages, scored against their reference trees.

Metric	google/gemma-4-12B-it	This adapter
Pages attached	137/138	138/138
Node F1	0.772	0.942
Precision	0.788	0.946
Recall	0.760	0.939
Right parent, relaxed	0.668	0.741
Right parent, strict	0.569	0.729
Node type accuracy	0.462	0.765
Text exact (matched nodes)	0.913	0.965
Whole-document word similarity	0.921	0.969
Words captured	0.918	0.972

Behaviour kept

Check	google/gemma-4-12B-it	This adapter
Ordinary questions about a page answered with extraction JSON	0/85	0/85
Held-out chart and table questions (exact answers)	40/40	40/40
Thinking prompts that still produce reasoning	10/10	10/10

Quoting one provision from a page without the extraction prompt is weaker than the base model (12/23 against 16/23); use the extraction loop for verbatim text.

Adapter details

• Type: LoRA, rank 16, alpha 32, dropout 0.05, on down_proj, gate_proj, k_proj, o_proj, q_proj, up_proj, v_proj of the language model; the image embedder is frozen
• The k_proj of the eight full-attention layers (5, 11, ..., 47) is left out: their single projection serves keys and values, and vLLM 0.28 would apply a LoRA there to the keys only
• Training: 4836 conversations (2665 extraction, 721 contrast, 478 retention text, 972 retention image), 3 epochs, lr 0.0002, cosine, 8 conversations per step, max length 8192; final train loss 0.1139, validation loss by epoch 0.03079, 0.03599, 0.01924
• Images: 560 tokens each; an extraction page as two strips
• Data: the 484-conversation mix of Ashish-Ranjan/Policy-Structure-Pages (its contrast and retention prompts answered again by Gemma 4 itself) plus pages typeset from public-domain eCFR regulations in sampled one- and two-column layouts, with exact page trees
• Recipe after Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA (same task, prompts, page loop and merge code)

Limitations

• Trained on clean renders and digital PDFs at 200 DPI; scanned or photographed pages were not tested.
• Mostly US personal-lines insurance and regulation-style drafting; declarations pages, contents pages and data tables are not in the training data.
• Pages run in order, so an early nesting mistake carries into later prompts.
• The reference-scored test is small (2 policies and 4 public-domain documents in 2 layouts).

Files

• adapter_model.safetensors, adapter_config.json: LoRA weights and config
• tokenizer*.json, chat_template.jinja, processor_config.json: as used in training
• extract.py: PDF to JSON tree with the serving recipe; page_merge.py, system_prompt.txt: its parts
• train_metrics.json: training metrics; eval/report.json: every evaluation number above

License and provenance

The task design, system prompt, prompts, page loop and page_merge.py come from Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA (Apache-2.0; LICENSE-policy-structure). The eCFR pages are U.S. government text in the public domain. Gemma 4 is Apache-2.0.