--- license: apache-2.0 base_model: Qwen/Qwen3-VL-4B-Instruct pipeline_tag: image-text-to-text library_name: transformers tags: - ocr - document-parsing - document-ai - layout - vision-language-model - qwen3_vl --- # Hanji Parse 4B **Hanji Parse 4B** is a document-parsing vision-language model fine-tuned from [`Qwen/Qwen3-VL-4B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct). Given a page image, it emits a JSON array of layout-grounded content blocks — each block a semantic section of the page (a paragraph, a heading with its content, a key-value panel, or a whole table) with a bounding box and its transcribed text. Tables are transcribed as GitHub-Flavored Markdown inside a single block. Figures, photos, and signatures are returned as image blocks. ## Output contract The model returns **JSON only**: an array of records ```json [{"bbox_2d": [x1, y1, x2, y2], "text_content": "..."}] ``` - `bbox_2d` is `[left, top, right, bottom]` in **normalized 0–1000 page coordinates** (divide by 1000 and multiply by the page width/height to recover pixel boxes). - Blocks are **semantic sections**, typically 5–30 per page — not one record per line, cell, or field. - Tables come back as **one block** containing a GitHub-Flavored Markdown table; every cell rides inside the markdown. - Non-text graphics (photos, charts, stamps, signatures) come back with `text_content` = `""`. - Checkboxes are transcribed inline as `[x]` / `[ ]` before their label. - An empty page returns exactly `[]`. ## Usage — read this before running the model ### 1. Image preprocessing - Downscale so the image is at most **2,000,000 pixels** (2 MP), preserving aspect ratio. **Never upscale.** - Floor each dimension to a **multiple of 32** (Qwen3-VL uses 16-px patches with 2×2 spatial merge → 32 px per visual token). - Use LANCZOS resampling; feed the result as a PNG. ```python from PIL import Image MAX_PIXELS, PATCH = 2_000_000, 32 def preprocess(img: Image.Image) -> Image.Image: w, h = img.size scale = min(1.0, (MAX_PIXELS / (w * h)) ** 0.5) w, h = int(w * scale) // PATCH * PATCH, int(h * scale) // PATCH * PATCH return img.convert("RGB").resize((w, h), Image.LANCZOS) ``` ### 2. The prompt Send the page image followed by exactly this text as the user turn. Do not paraphrase, extend, or reformat it. ````text Detect every BLOCK in this document and return a JSON array. Schema: [{"bbox_2d":[x1,y1,x2,y2], "text_content":"..."}] Coordinates: normalized 0-1000 page coordinates; [x1,y1,x2,y2] = [left,top,right,bottom]. EMPTY PAGE: If the page has no legible content, return exactly []. Otherwise, transcribe every legible content block; a page with only one legible item is not empty. Your DEFAULT is to GROUP. Most such pages form 5-30 records; sparse pages may form only 1-4. More than 30 remains unusual. A "block" is a semantic SECTION (a panel, a heading + its content, a key-value group, or a whole table), NOT a single line, cell, or field. If you are emitting one record per line, per cell, or per form field, STOP - that is WRONG. When a region is not a clean table, you must STILL group it into section blocks; never fall back to one-record-per-element. Block categories: - Text records: ONE record per block - a heading TOGETHER WITH the lines beneath it, a paragraph, a list, or a key-value field group. text_content = the block's text, with "\n" between its lines. DO NOT emit one record per line. - Table records: a table or dense grid of cells, rendered as GitHub-Flavored Markdown (| col | col |\n|---|---|\n| cell | cell |). Never emit one record per cell or per row - the markdown carries every cell. A repeated item|amount list (receipt lines, menu items) IS a table. HEADERLESS TABLES: if a table has no visible column headings, DO NOT invent any. Render only the visible rows/cells in their observed order. If Markdown syntax needs a separator row, use empty header cells rather than synthetic names like "Column 1". TABLE CELL TEXT: cell contents must be plain visible text. Do NOT add Markdown emphasis or formatting inside cells (no **bold**, _italics_, backticks, or headings) unless that formatting is the only way to preserve information that is visible on the page. TALL TABLES: A logical table on one page is ONE block regardless of row count. Include every visible row in one GFM table. Its bbox_2d must tightly enclose the full table from the first row through the last row. - Image records: ONE record per photo, figure, chart, scan, or non-text graphic. text_content = "". Handwritten signatures, cursive e-signatures, initials, signature scribbles, and signature marks are ALWAYS images - do NOT transcribe or guess them, even if partly readable. Printed labels such as "Signature:" remain text. Do NOT emit for logos < 40 px wide. CRITICAL - transcribe MEANING, not layout glyphs: - Fill-in / blank lines: emit ONLY the label, NOT the blank. Write "Name:" - never "Name:________________". For signature fields, keep the printed label as text and emit the actual signature mark itself as an image record with text_content="". - NEVER reproduce decorative rules or separators - rows of *, -, _, =, ., or any repeated glyph. Omit them entirely; they are not content. - Checkboxes / Y-N / selection fields: write the field and its options on one line with the marks inline - " Y [x] N [ ]", "