Update README.md

README.md

@@ -58,6 +58,17 @@ learning rates. A full Phase 1→4 retrain with the corrected collator would be
 The collator fix lives in [`train_phase1.py`](https://github.com/Praxisyn/newtype_cognition/blob/main/train_phase1.py)
 on the `main` branch; this checkpoint was trained before that fix took effect end-to-end.
 
+**`<OCR_WITH_REGION>` does not produce region coordinates.** Despite its name, the token
+outputs plain text identical to `<OCR>` — bounding box coordinates are not generated.
+The fine-tuning process appears to have collapsed the two tokens to the same behavior,
+likely because the region format (`<loc_N>` tokens) was not well represented in the
+training data or was discarded by the collator.
+
+**OCR extracts dominant text only.** Both `<OCR>` and `<OCR_WITH_REGION>` return the
+most visually prominent text in the document (e.g., the largest number or title), not
+a full transcription of all text regions. On template documents with placeholder data,
+this may produce a single value such as `$30` or `$0.00`.
+
 ## Evaluation
 
 Evaluated on a 50-sample slice of `HuggingFaceM4/DocumentVQA` validation:
@@ -72,6 +83,31 @@ Most of the 10% exact match comes from samples where the expected answer is the
 visible text on the document (e.g., the company title is the answer to "What is the
 company name?"). It is **not** evidence of question understanding.
 
+## Token behavior and recommendations
+
+Based on empirical testing, the three most useful task tokens behave as follows:
+
+| Token | Behavior | Recommended for |
+|---|---|---|
+| `<OCR>` | Extracts the single most visually dominant text | Quick salience extraction, document indexing |
+| `<OCR_WITH_REGION>` | Identical output to `<OCR>` — region coordinates not generated | Avoid; use `<OCR>` directly |
+| `<MORE_DETAILED_CAPTION>` | Produces a structured natural-language description of the document layout | Thumbnail descriptions, alt-text generation, layout understanding |
+
+**`<MORE_DETAILED_CAPTION>` is the most informative token** for document understanding tasks.
+On a standard service invoice it correctly identified the document type, described the
+column structure, and extracted footer text — without being asked any question. Output is
+capped by `max_new_tokens`; increase to 256 or higher to avoid truncation on dense documents.
+
+Example output on a service invoice template (`<MORE_DETAILED_CAPTION>`):
+```
+The image is a service invoice template with hourly rate. It has a white background
+and black text. The template is divided into two columns, with the left column containing
+the company name, address, phone number, and email address. The right column contains
+the total amount of the service invoice, which includes the total cost, subtotal,
+discount, and other details. At the bottom of the template, there is a note that reads
+"Thank you" and "Please make check payable to your company name."
+```
+
 ## Use Cases
 
 This model is best suited for tasks that do **not** require understanding a specific question about the document. Given its question-blind behavior, it works well as a document-aware OCR captioner in the following scenarios:
@@ -91,7 +127,6 @@ Using the model as a first stage to extract text regions, then passing those reg
 **Fine-tuning checkpoint**
 Starting a domain-specific fine-tune from this checkpoint rather than from `microsoft/Florence-2-large` vanilla, particularly for document-heavy domains.
 
-
 ## When Not to Use This Model
 
 - **Document Question Answering (DocQA):** The model is question-blind and will ignore any natural language question you provide. Do not use it in any pipeline where the output must depend on what the user asks.
@@ -169,11 +204,9 @@ All datasets streamed; no full local copies retained.
 - A late-stage low-lr "polish" phase **cannot** fix a behavioral bug introduced in
   earlier phases. Sanity-check inference behavior at the end of Phase 1, not at Phase 4.
 
-
-
 ## Demo
-Try it live: [Space](https://huggingface.co/spaces/d3p4rt/newtype-cognition-demo)
 
+Try it live: [Space](https://huggingface.co/spaces/d3p4rt/newtype-cognition-demo)
 
 ## License
 
@@ -188,4 +221,4 @@ MIT (inherited from base model).
   year         = {2026},
   howpublished = {\url{https://huggingface.co/d3p4rt/newtype-cognition}},
 }
-```
+```