Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA

Qwen3.5-9B · Policy Structure (LoRA)

Give it a policy document one page image at a time, and it returns the whole document as one nested JSON tree. Text is copied verbatim and nested the way the document is printed: sections, subsections, numbered clauses, lettered items and definitions. Content that runs across a page break is joined back up.

• Live demo (side-by-side with the original model): policy-structure-extractor.vercel.app
• Base model: Qwen/Qwen3.5-9B
• Code: extract.py (PDF → JSON tree) · page_merge.py (page reassembly)
• Results: results/ (every number in this card, as CSV)
• Author: Ashish Ranjan

Model summary

Developed by	Ashish Ranjan
Model type	LoRA adapter for a vision-language model (image + text in, text out)
Fine-tuned from	Qwen/Qwen3.5-9B (9.45 B parameters)
Task	Page-by-page extraction of a document's nested structure into JSON
Input	One page image (rendered at 200 DPI) + a short text describing where the previous page left off ("open path") + the extraction system prompt
Output	One JSON object per page (typically ~1,200 tokens, at most 4,096). The included code merges the pages into one tree per document
Language	English
Adapter	rank 16, 43.3 M trainable parameters (0.46%), 173 MB
License	Apache-2.0
GPU memory	about 21.5 GB with the default settings: works on 48 GB and 80 GB GPUs, not on a 24 GB L4 (details)
Release	v1, 2026-10-16. A static model trained on an offline dataset

Highlights

• Structure: node F1 of 0.89, against 0.36 for the original model, on two insurance policies held out from training (34 pages).
• Nesting: 87% of nodes sit under the right parent, against 59% for the original model, and 95% have the right type.
• Verbatim text: 99.8% whole-document word similarity. Only 4 of the 1,301 reference nodes are missing.
• Beyond the training domain: on six unlabelled documents it had never seen (88 pages, including commercial forms, a card agreement and a federal regulation), it captured 98% of the printed words, against 43% for the original model.
• Behaves normally without the extraction prompt: no ordinary question was answered with JSON, visual-QA accuracy is 32/40 against 34/40, and thinking mode is intact.
• Fast and cheap: 8.6 s per page on one H100. One training run took 33 minutes; the whole project cost an estimated $12 of cloud GPU time.

Contents

How it works · Getting started · Evaluation · Uses · Bias, risks and limitations · Training details · Technical specifications · Environmental impact · Troubleshooting · Files · License and data provenance · Citation · Glossary

How it works

A policy is too long for one generation, and its nesting runs across pages. The model therefore reads one page per request. Each request also carries a short description of where the previous page left off, called the open path.

page 1 ─► model ─► page tree ─► merge ─► document tree ─► open path ─┐
                                                                     │
page 2 + open path ◄─────────────────────────────────────────────────┘ ─► model ─► page tree ─► merge ─► …

1. The open path is the chain of nodes still open at the end of the previous page, from the document root down through the last child at each level. Each entry holds the node's type and its label, or its title, or the first words of its text. The deepest entry also gets a tail (its last 150 characters), so the model can tell whether the new page continues that sentence.
2. The page tree. The model writes only what is printed on the current page. A node that continues from an earlier page is written as a stub along the left edge, for example {"cont": true, "type": "clause", "label": "12."}. A stub never repeats earlier text. If the page starts mid-sentence, the deepest stub's text holds just the continuation.
3. Merging is done by deterministic Python code, not by the model (page_merge.py):
   • Each stub is matched to the open node at the same level.
   • Continuation text is appended to that node, and new nodes are attached after it.
   • The next page's open path is then read off the merged tree.
   • Slicing the training documents into pages and merging them back reproduces every original tree exactly.
4. Order matters. Pages of one document are processed in order, because each prompt is built from the model's own output for the earlier pages. Several documents can still be processed side by side in one batch.

Example: held-out GEICO auto policy, page 5

Page 5 of GEICO's Maine Family Automobile Insurance Policy (form A-30ME), a publicly filed policy form, shown here for illustration only (not affiliated with or endorsed by GEICO). This policy was held out from training.

The open path given to the model was built from its own output for pages 3–4:

[{"type": "document"},
 {"type": "section", "title": "Section I — Liability Coverages"},
 {"type": "subsection", "title": "Exclusions — When Section I Does Not Apply"},
 {"type": "clause", "label": "12.", "tail": "of a non-owned auto or temporary substitute auto that is designed for use principally off public roads that is not registered for use on public roads."}]

The start of the model's output, after the bracket repair:

{"cont": true, "type": "document", "children": [
 {"cont": true, "type": "section", "title": "Section I — Liability Coverages", "children": [
  {"cont": true, "type": "subsection", "title": "Exclusions — When Section I Does Not Apply", "children": [
   {"type": "clause", "label": "13.", "text": "We do not cover bodily injury or property damage that results from nuclear exposure or explosion including resulting fire, radiation or contamination is not covered."},
   {"type": "clause", "label": "14.", "text": "We do not cover bodily injury or property damage that results from bio-chemical attack or exposure to bio-chemical agents is not covered."},
   {"type": "clause", "label": "15.", "text": "Section I does not apply:", "children": [
    {"type": "item", "label": "(a)", "text": "To bodily injury or property damage caused by an auto driven in or preparing for any racing, speed or demolition contest or stunting activity of any nature, whether or not prearranged or organized."},
    {"type": "item", "label": "(b)", "text": "To the operation or use of a motor vehicle on a track designed primarily for racing or high speed driving. ..."}]},
   …

How to read this:

• The page continues the open Exclusions subsection, so it starts with three stubs and then adds exclusions 13 onwards.
• The repeated "…is not covered." in exclusions 13 and 14 is printed that way in the policy; the model copies text exactly as printed.
• The raw output had a closing-bracket miscount, which repair_json fixed. The raw and parsed outputs are both in examples/geico_p5.json.

Getting started

Requirements

• Hardware: a Linux machine with an NVIDIA GPU with more than 24 GB of memory. With the default settings the model needs about 21.5 GB: 17.8 GB of weights once loaded, plus vLLM's working memory and the cache for one page.
• Software: vLLM 0.30.0, pypdfium2 and Pillow (see requirements.txt).
• Tested end to end from this repository (2026-10-16, fresh install on one H100): download, then extract.py on pages 3–8 of the held-out GEICO policy.
  • All 6 pages attached, and 4 of the 6 outputs were byte-identical to the evaluation run.
  • Node F1 on those pages was 0.868, against 0.861 in the evaluation run.
  • Greedy decoding on GPUs isn't bit-exact across library versions and batch sizes, so expect small differences like these.

Which GPU? Measured 2026-10-16 with the published extract.py on held-out GEICO pages:

GPU	Usable memory	Result	Speed, one document	GPU cost per page*
NVIDIA H100	80 GB	✅ works	8.6 s/page	≈ $0.009
NVIDIA L40S	44 GB	✅ works	32 s/page	≈ $0.018
NVIDIA L4	22 GB	❌ out of memory at start-up, also at --gpu-memory-utilization 0.95	–	–

*At Modal's on-demand prices. The H100 is the cheapest per page because it generates almost 4× faster.

Other GPUs (calculated, not tested): any card with 32 GB or more should work. 24 GB consumer cards (RTX 3090 / 4090) have slightly more usable memory than the L4, so they're borderline. 16 GB cards can't hold the weights.

Run on a PDF

pip install -U "huggingface_hub[cli]"
hf download Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA --local-dir policy-structure
cd policy-structure
pip install -r requirements.txt
python extract.py my_policy.pdf --out-dir out

What it does:

• Renders each page at 200 DPI, the resolution the model was trained on.
• Runs the page loop.
• Writes out/my_policy.json, the merged document tree.
• Writes out/my_policy.pages.json, which holds for every page the raw model output, whether its brackets were repaired, merge repairs, output tokens and timing.

Options:

• --pages 3-18 processes only some pages.
• Passing several PDFs batches them together.
• --adapter <folder> loads the adapter from a local copy.

Use from Python

from extract import render_pdf, extract, vllm_generator

generate = vllm_generator("Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA")              # loads Qwen3.5-9B + this adapter in vLLM
docs = {"policy": render_pdf("policy.pdf", pages=range(3, 19))}
result = extract(docs, generate)                   # pages in order; documents batched together
tree = result["policy"]["tree"]

Prompt format (for your own loop)

Everything the evaluation used is in this repo.

Part	Content
System message	system_prompt.txt, verbatim (599 tokens). It switches on extraction mode; without it the model behaves like the original.
User message, first page	the page image, then Open path: [] (this is the first page)\n\nExtract this page.
User message, later pages	the page image, then Open path at the end of the previous page:\n{path}\n\nExtract this page., where {path} is page_merge.path_text(open_path(tree))
Thinking	off: chat_template_kwargs={"enable_thinking": False}
Decoding	greedy (temperature=0), max_tokens=4096, no presence or repetition penalty, because penalties corrupt verbatim transcription
Image	one page per request, rendered at 200 DPI (US letter = 1700×2200 px ≈ 3,700 image tokens)
After each page	repair_json(raw) → strip_cont(out) for the first page, merge_lenient(tree, out) afterwards → open_path(tree) for the next prompt

Keep the bracket repair. With free decoding the model sometimes miscounts closing brackets at the end of deeply nested pages (9 of 16 GEICO pages and 7 of 18 Encompass pages needed it).

Output format

Every node is {type, label, title, text, children}, and empty fields are left out. There are no ids; assign them downstream if you need them.

Field	Meaning
type	document, part, section, subsection, clause, definition, subclause, item, paragraph
label	the enumerator exactly as printed: "A.", "12.", "(a)", "(i)"
title	the heading, including a bold run-in heading at the start of a paragraph
text	the node's own prose, verbatim
children	child nodes in reading order

Conventions the model learned:

• Headings of equal visual weight are siblings.
• A list is nested under the heading or lead-in sentence (ending in ":") that introduces it.
• The first enumerated level under a section or subsection is a clause; deeper levels are items.
• Page furniture is left out: running headers and footers, page numbers, form numbers, copyright lines, insurer address blocks and endorsement banners.

Evaluation

Testing data, factors and metrics

Testing data:

Set	What	How it's scored
Held-out documents	2 complete insurance policies never seen in training: GEICO Family Auto (single column, pages 3–18) and Encompass Special Home (two columns, pages 3–20). 34 pages, 1,301 reference nodes	Against hand-authored reference trees
Unseen documents	6 more documents, every page including covers and contents pages (88 pages): Liberty Mutual homeowners and auto, ISO commercial general liability and commercial property forms, a Capital One card agreement, and Regulation E (12 CFR 1005)	Label-free checks against each page's PDF text layer (no reference trees exist)
Behaviour checks	18 ordinary questions about held-out policy pages, 40 held-out visual-QA questions from the_cauldron, 10 thinking-mode text prompts	Against the original model

Factors:

• Layout: one, two or three columns.
• Document type: personal-lines insurance, commercial insurance, a card agreement, a federal regulation.
• Page kind: body pages, covers, contents pages.
• Decoding and merge settings, for the ablation.

Metrics:

Metric	What it measures
Node precision / recall / F1	Are the document's units found, as separate nodes? Nodes are matched in order by label plus the opening of their title and text.
Right parent (relaxed, strict)	Is each matched node nested under the right parent?
Node type accuracy	Section, clause, item and so on.
Text exact, word similarity	Is the text verbatim?
Pages attached	Did every page's output parse and join the tree?
Words captured, words not on page	Label-free completeness and invented text, against the PDF text layer with headers and footers removed.

All runs went page by page, exactly as in How it works. Each page's prompt carried the open path built from the model's own earlier pages, never from the reference.

Results: held-out documents

Metric	GEICO original	GEICO fine-tuned	Encompass original	Encompass fine-tuned
Pages attached	16/16	16/16	18/18	18/18
Nodes (reference / predicted)	585 / 220	585 / 614	716 / 343	716 / 722
Precision	0.650	0.845	0.551	0.917
Recall	0.244	0.887	0.264	0.925
Node F1	0.355	0.866	0.357	0.921
Right parent, relaxed	0.531	0.902	0.640	0.837
Right parent, strict	0.455	0.680	0.312	0.702
Node type accuracy	0.503	0.936	0.556	0.968
Text exact (matched nodes)	0.895	0.990	0.862	0.992
Whole-document word similarity	0.763	0.999	0.805	0.997

• Original model: the same prompts with JSON-constrained decoding, its best setting. With free decoding it scores lower (F1 0.22 / 0.36), because most of its pages don't parse.
• Fine-tuned model: the released pipeline: free decoding, then bracket repair, then the tolerant merge.

More on the metrics

• Right parent, strict: the matched node's parent is the matched counterpart of its reference parent. This check cascades: if a parent is written slightly differently (for example "4" instead of "4."), all of its children count as wrong.
• Right parent, relaxed: the parent is the same node, ignoring label punctuation, or has the same heading or text start. Only 1.5% (GEICO) and 4% (Encompass) of nodes are genuinely under the wrong node.
• Unmatched reference nodes (66 GEICO, 54 Encompass): almost all are present but merged into a neighbouring node's text, or split differently. Only 4 of 1,301 reference nodes are missing outright.

Results: unseen documents

On GEICO, where the right answer is known, the reference tree scores 99.8% captured / 0.1% not on page, and the fine-tuned model 99.7% / 0.1%. So these two checks track quality closely.

Document (not in training)	Pages attached, fine-tuned vs original	Words captured	Words not on page
Liberty Mutual homeowners (three columns)	18/18 vs 13/18	99.6% vs 46%	2.3% vs 6.5%
Liberty Mutual auto	12/13 vs 5/13	95.7% vs 27%	0.3% vs 7.0%
ISO commercial general liability, CG 00 01 (watermarked)	20/20 vs 4/20	96.4% vs 18%	1.8% vs 4.2%
ISO commercial property, CP 00 17 + a second form	16/16 vs 16/16	99.7% vs 78%	0.2% vs 7.2%
Capital One REI card agreement	6/6 vs 3/6	99.6% vs 33%	0.0% vs 3.2%
Regulation E, 12 CFR 1005 (a document type absent from training)	15/15 vs 10/15	97.1% vs 48%	0.5% vs 2.6%
All 88 pages	87/88 vs 51/88	98% vs 43%	1.0% vs ≈5%

What we found:

• Text capture generalises. 79 of 88 pages scored ≥ 95%, across a three-column layout, a watermark, the small print of a federal regulation and a pricing table.
• Nesting is the weak spot on unfamiliar layouts. On some pages the model closes one bracket level too few in the middle of the page. Every later node on that page then lands one level too deep, for example Part B nested inside Part A. It happened most on Regulation E (9 of 15 pages) and the commercial general liability form (8 of 20).
• Contents pages: one was lost, because the model copied the dot leaders ("Agreement . . . .") until it hit the 4,096-token cap. Another came out as empty headings.
• Regulation-style run-in labels such as "(b) Electronic fund transfer—(1) Definition." are nested incorrectly.
• A second form inside the same PDF is attached beside the first form's sections instead of under a new node.

The live demo shows these outputs page by page under "More documents"; per-document numbers are in results/unseen_docs.csv.

Ablation: what each part of the pipeline adds

Decoding and merge (fine-tuned adapter)	GEICO F1	Encompass F1	Pages attached
Free decoding, strict merge	0.761	0.808	28 / 34
JSON-constrained decoding, strict merge	0.866	0.818	33 / 34
Free decoding + bracket repair + lenient merge (released)	0.866	0.921	34 / 34

Constrained decoding guarantees valid JSON, but on dense two-column pages it stops at a premature close and loses the right-hand column. The repair-based pipeline keeps that column.

Behaviour outside extraction

Check	Original	Fine-tuned
Ordinary questions about held-out policy pages answered with extraction JSON (should be 0)	0 / 18	0 / 18
Held-out visual QA from the_cauldron, relaxed match	34 / 40	32 / 40
Thinking-mode prompts that still produce reasoning	10 / 10	10 / 10

Learning-rate sweep and checkpoint selection

Learning rate	Validation loss, epoch 1 / 2 / 3	Full-document F1 at epoch 3 (free decoding, GEICO / Encompass)
5e-5	0.0171 / 0.0137 / 0.0132	0.546 / 0.360
1e-4	0.0150 / 0.0121 / 0.0116	0.685 / 0.752
2e-4 (released, epoch 3)	0.0141 / 0.0117 / 0.0116	0.761 / 0.808

• Validation loss is teacher-forced on the 16 GEICO pages. It was 0.0623 for the original model.
• F1 in this table is plain free decoding before bracket repair. It is dominated by pages lost to bracket errors, which the released pipeline recovers.
• Selection: lowest validation loss together with the best full-document F1. Because the checkpoint was chosen on the same two held-out documents, their scores are slightly optimistic; the six unseen documents were not used for any choice.

Uses

Direct use

Turning born-digital, English insurance policies and similarly nested contracts into one JSON tree per document, with extract.py or your own loop (see Prompt format).

Downstream use

• Structure-aware chunking for retrieval (RAG): one chunk per section or clause, with its heading path.
• Clause lookup and side-by-side comparison of policies.
• A starting point for fine-tuning on other nested document types with the same page-by-page format.

Out-of-scope use

• Automated coverage, claims, underwriting or legal decisions without human review.
• Legal or insurance advice.
• Scanned, handwritten or non-English documents, and table-heavy documents: grids are not supported, and the schema has no table type.
• General chat or image questions. These still work, but use the base model for them.

Bias, risks and limitations

• Small reference-scored test. Structure is scored against reference trees on only two documents (34 pages), both US insurance policies. The six unseen documents (88 pages) were checked with label-free measures only.
• Insurance-centred training. Training used 15 documents, mostly US personal-lines insurance plus one credit-card agreement. Layouts and drafting styles from insurers or document types outside that set nest less reliably. Flat or lightly structured documents may come out over-nested.
• Pages the model hasn't seen. Cover pages, contents pages, declarations pages and data tables were not in training. Contents pages with dot leaders can make the model copy the leaders until the token cap, losing the page. Key-value tables are flattened into items.
• Granularity and nesting. The model sometimes:
  • merges two paragraphs into one node;
  • folds several heading levels into one title;
  • attaches a lead-in sentence at the wrong level;
  • drops a label's punctuation ("4" vs "4."), because the training labels were inconsistent about this.
• Brackets. repair_json fixes missing or extra closers at the end of a page. It can't fix a pair missing in the middle of a page, which pushes the rest of that page one level too deep.
• Error propagation. Pages are processed in order, so an early nesting mistake carries into later prompts.
• Downstream risk. A clause attached to the wrong parent can make a RAG system cite the wrong section or apply an exclusion to the wrong coverage.
• Personal data. Run on a real policyholder's document, the output reproduces any personal data on the page verbatim (names, addresses, policy numbers). Handle the outputs like the source document.

Recommendations

• Always keep the bracket repair and the lenient merge (extract.py does).
• Skip contents pages, or stop generation on a long run of . . ., and split PDFs that bundle several forms into one document per form.
• Check outputs cheaply without labels:
  • compare the output's words with the PDF text layer (the "words captured" check);
  • flag numbering gaps or restarts among siblings ("1, 2, 4" or a list starting at 3), which reliably point to nesting slips.
• Before relying on it for a new document type, score a few documents against hand-checked trees.
• Keep a human in the loop for any decision that depends on the extracted structure.

Training details

Training data

Part	Conversations	What
Extraction	265 (55%)	One page image + open path → that page's JSON, from 15 documents (below)
Contrast	71 (15%)	Pages from the same documents with ordinary prompts (summarise, answer a question, list the headings, transcribe, generic JSON request) → normal answers, so JSON comes out only when the extraction prompt is used
Retention, images	99 (20%)	General image questions (natural images, charts, documents, infographics, OCR, diagrams, screens) sampled from the_cauldron
Retention, text	49 (10%)	English chat prompts from oasst2
Total	484

Training documents: 15 policy forms and agreements: forms from US insurers (plus one Canadian policy), some of them ISO standard forms, and one bank credit-card agreement. Declarations pages and other policyholder-specific pages were excluded. The documents and their reference trees aren't released.

Kind	Documents	Pages
Homeowners	6	131
Personal auto	5	39
Renters, with endorsements	1	39
Dwelling fire	1	27
Manufactured home	1	22
Credit-card agreement	1	7
Total	15	265

13 of the 15 documents are two-column layouts. Drafting styles range from ISO standard forms to insurers' own plain-language and narrative styles. Enumeration styles include A. / 1. / a. / (1) / (a) / 1) / a).

• Data collection method: hybrid. The extraction data comes from public policy documents, mostly from a US state insurance regulator's policy-comparison filings, and one bank's published agreement. The retention prompts and images come from public datasets.
• Labelling method:
  • Extraction targets: human. The author wrote a reference tree for each document from its page images, using the PDF text layer for exact wording.
  • Contrast and retention answers: synthetic. The original Qwen3.5-9B generated them itself (self-distillation), about 75% with thinking on, so the adapter doesn't unlearn reasoning.

Training procedure

Preprocessing:

1. Each reference tree was aligned to the page breaks and sliced into per-page targets with continuation stubs. Merging the slices reproduces every tree exactly.
2. Node types were normalised with one rule across documents.
3. Five near-duplicate pages were dropped (4-gram Jaccard ≥ 0.7).
4. Pages were rendered at 200 DPI. A page image is 3,657 tokens and is never downscaled.

Hyperparameters (ms-swift 4.5.3; full arguments in training/swift_args.json):

Setting	Value
Method	LoRA on the language model only; vision encoder and projector frozen
Target modules	all linear layers: attention q/k/v/o, Gated DeltaNet in_proj_qkv/z/b/a + out_proj, MLP gate/up/down
Rank / alpha / dropout	16 / 32 / 0.05
Learning rate	2e-4, cosine schedule, 5% warmup (swept over 5e-5, 1e-4, 2e-4)
Optimizer	AdamW (fused), betas 0.9 / 0.95, weight decay 0.1, gradient clip 1.0
Batch	1 per device × 8 gradient accumulation = 8
Epochs / steps	3 / 183
Max length	8,192 tokens, with up to 4,096 tokens per image
Loss	assistant tokens only; the empty <think></think> block of non-thinking answers is masked (loss_scale default+ignore_empty_think)
Training regime	bf16 mixed precision, gradient checkpointing, SDPA attention
Seed	42

Speeds, sizes, times

Training time	33 minutes per run on one H100 (183 steps); the three-run sweep ran in parallel
Peak training memory	68.5 GiB
Adapter size	173 MB (43.3 M parameters)
Sequence lengths	extraction targets median 1,073 tokens (max 2,316); extraction conversations median 5,455; longest conversation 8,058
Inference, one document	8.6 s per page on one H100 (≈ 1,170 output tokens per page at 108 tokens/s); 32 s per page on one L40S (≈ 39 tokens/s)
Inference, batched	≈ 2.6 s per page of throughput with several documents in flight
Model load	≈ 3.6 minutes on a fresh machine

The training log is in training/train_log.jsonl.

Technical specifications

Model architecture and objective

• Base model: Qwen3.5-9B, a natively multimodal model. Its language model mixes Gated DeltaNet (linear attention) layers with full-attention layers and supports a 262K-token context.
• What the adapter changes: low-rank updates to every linear layer of the language model. The vision encoder and projector are unchanged.
• Objective: supervised fine-tuning with next-token cross-entropy on the assistant's tokens.

Compute infrastructure

• Hardware: NVIDIA H100 80 GB on Modal, one GPU per training run. Self-distillation answers were generated on an L40S.
• Software:
  • training: ms-swift 4.5.3, transformers 5.16.1, peft 0.20.0, flash-linear-attention 0.5.2;
  • inference and evaluation: vLLM 0.30.0.

Environmental impact

Estimated with the Machine Learning CO₂ Impact method (Lacoste et al., 2019): GPU hours × power draw × grid carbon intensity.

Hardware type	NVIDIA H100 80 GB (plus an L40S for data generation)
Hours used	≈ 1.7 GPU-hours for the released learning-rate sweep (3 runs × 33 minutes). ≈ 3.5 GPU-hours for the whole project, including data generation and every evaluation (estimated from the cloud bill)
Cloud provider	Modal
Compute region	not recorded
Carbon emitted	≈ 0.5 kg CO₂eq for the sweep, ≈ 1 kg CO₂eq for the whole project (assuming 0.7 kW per GPU and 0.4 kg CO₂eq/kWh)

Troubleshooting

• vLLM fails while compiling a FlashInfer sampler (no nvcc on the machine): set VLLM_USE_FLASHINFER_SAMPLER=0. The evaluation ran this way.
• Out of memory while vLLM starts: the GPU is too small. The model needs about 21.5 GB, so a 24 GB L4 fails; use a 32 GB or larger card. On a borderline card, --gpu-memory-utilization 0.95 gives vLLM a little more room. max_model_len is 12,288 tokens: one page image plus prompt plus up to 4,096 output tokens.
• A page is missing from the tree: look at its entry in <name>.pages.json. An error means the output couldn't be parsed even after repair. output_tokens = 4096 means generation hit the cap, usually on a contents page.
• Output is prose instead of JSON: make sure the system message is exactly system_prompt.txt, the adapter is loaded and thinking is off.

Files in this repository

File	What
adapter_model.safetensors, adapter_config.json	The LoRA adapter (PEFT format)
extract.py	PDF → nested JSON with vLLM: renders pages, runs the page loop, writes the tree plus per-page logs
page_merge.py	repair_json, merge_lenient, strip_cont, open_path, path_text: the exact reassembly code used in training and evaluation (standard library only)
system_prompt.txt	The extraction system prompt
requirements.txt	Tested versions
examples/geico_p5.json	The example above: open path, raw output, parsed output
results/heldout_scores.csv	Every evaluated setting × document
results/unseen_docs.csv	Label-free checks on the 6 unseen documents, fine-tuned vs original
results/sweep.csv	Validation loss and full-document F1 for all 9 checkpoints
results/guardrails.csv	Behaviour checks for the original model and all checkpoints
training/swift_args.json, training/train_log.jsonl	Exact training arguments and log of the released run
assets/	Figures and the example page
SHA256SUMS, LICENSE	Checksums of every file; Apache-2.0 licence text

License and data provenance

• Licence: the adapter weights and the code in this repository are released under Apache-2.0, the same licence as the base model, Qwen3.5-9B.
• Training documents: insurance policy forms (most from a US state insurance regulator's public policy-comparison filings) and a public card agreement. Their copyrights stay with their owners, and some include ISO copyrighted material. The documents and the hand-authored reference trees are not distributed here. The single example page above is shown for illustration.
• Trademarks and affiliation: this project is not affiliated with or endorsed by any insurer or other company named in this card. Names are used only to identify the publicly filed documents the model was tested on.
• Retention data: prompts and images come from HuggingFaceM4/the_cauldron, whose subsets carry their own licences, and from OpenAssistant/oasst2 (Apache-2.0). The answers were generated by Qwen3.5-9B. None of that data is redistributed here.

Citation

BibTeX:

@misc{ranjan2026policystructure,
  title        = {Qwen3.5-9B Policy Structure: page-by-page nested JSON extraction of insurance policies with a LoRA adapter},
  author       = {Ranjan, Ashish},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA}}
}

APA: Ranjan, A. (2026). Qwen3.5-9B Policy Structure: page-by-page nested JSON extraction of insurance policies with a LoRA adapter [Computer software]. Hugging Face. https://huggingface.co/Ashish-Ranjan/Qwen3.5-9B-Policy-Structure-LoRA

Glossary

Term	Meaning
Node	One unit of the document: a section, clause, item, paragraph and so on
Reference tree	A hand-authored, correct tree for a whole document, used for training targets and scoring
Open path	The chain of nodes still open at the end of the previous page; it goes into the next page's prompt
Stub	A {"cont": true, ...} node in a page's output that names an open node the page continues, instead of repeating it
Bracket repair	repair_json: fixes miscounted closing brackets in a page's output before merging
Lenient merge	merge_lenient: attaches a page to the tree; a stub that doesn't match becomes a new node instead of dropping the page
Self-distillation	Training on answers the original model generated itself, to keep its general behaviour

Model card authors and contact

Written by Ashish Ranjan. For questions, problems or results on other document types, open a discussion in this repository's Community tab.

Acknowledgements

• Qwen team: Qwen3.5-9B.
• ModelScope: ms-swift.
• The vLLM project.
• Modal: the GPUs.
• HuggingFaceM4 (the_cauldron) and OpenAssistant (oasst2): retention data.
• Thinking Machines' "LoRA Without Regret": the LoRA hyperparameter choices.