update_colab

README.md

@@ -6,21 +6,18 @@ language:
 base_model:
 - Qwen/Qwen2.5-VL-7B-Instruct
 tags:
-- OCR
+  - OCR
+  - vision-language
+  - document-understanding
+  - multilingual
+  - thai
 ---
 
 
 
-**Typhoon-OCR-7B**: A bilingual document parsing model built specifically for real-world documents in Thai and English inspired by models like olmOCR.
+**Typhoon-OCR-7B**: A bilingual document parsing model built specifically for real-world documents in Thai and English inspired by models like olmOCR based on Qwen2.5-VL-Instruction.
 
 
-## **Model Description**
-
-- **Model type**: A 7B Vision-Language Models (VLMs) model based on Qwen2.5-VL-Instruction.
-- **Requirement**: transformers 4.50.0 or newer.
-- **Primary Language(s)**: Thai 🇹🇭 and English 🇬🇧
-- **License**: 
-
 
 ## **Real-World Document Support**
 
@@ -51,6 +48,123 @@ However, in the Thai books benchmark, performance slightly declined due to the h
 For this version, our primary focus has been on achieving high-quality OCR for both English and Thai text. Future releases may extend support to more advanced image analysis and figure interpretation.
 
 ## Usage Example
+**(Recommended!!!): Full inference code available on [Colab](https://colab.research.google.com/drive/1z4Fm2BZnKcFIoWuyxzzIIIn8oI2GKl3r?usp=sharing)**
+
+Below is a partial snippet. You can run inference using either the API or a local model.
+
+**API**:
+```python
+from typing import Callable
+
+PROMPTS_SYS = {
+    "default": lambda base_text: (f"Below is an image of a document page along with its dimensions. "
+        f"Simply return the markdown representation of this document, presenting tables in markdown format as they naturally appear.\n"
+        f"If the document contains images, use a placeholder like dummy.png for each image.\n"
+        f"Your final output must be in JSON format with a single key `natural_text` containing the response.\n"
+        f"RAW_TEXT_START\n{base_text}\nRAW_TEXT_END"),
+    "structure": lambda base_text: (
+        f"Below is an image of a document page, along with its dimensions and possibly some raw textual content previously extracted from it. "
+        f"Note that the text extraction may be incomplete or partially missing. Carefully consider both the layout and any available text to reconstruct the document accurately.\n"
+        f"Your task is to return the markdown representation of this document, presenting tables in HTML format as they naturally appear.\n"
+        f"If the document contains images or figures, analyze them and include the tag <figure>IMAGE_ANALYSIS</figure> in the appropriate location.\n"
+        f"Your final output must be in JSON format with a single key `natural_text` containing the response.\n"
+        f"RAW_TEXT_START\n{base_text}\nRAW_TEXT_END"
+    ),
+}
+
+def get_prompt(prompt_name: str) -> Callable[[str], str]:
+    """
+    Fetches the system prompt based on the provided PROMPT_NAME.
+
+    :param prompt_name: The identifier for the desired prompt.
+    :return: The system prompt as a string.
+    """
+    return PROMPTS_SYS.get(prompt_name, lambda x: "Invalid PROMPT_NAME provided.")
+
+
+
+# Render the first page to base64 PNG and then load it into a PIL image.
+image_base64 = render_pdf_to_base64png(filename, page_num, target_longest_image_dim=1800)
+image_pil = Image.open(BytesIO(base64.b64decode(image_base64)))
+
+# Extract anchor text from the PDF (first page)
+anchor_text = get_anchor_text(filename, page_num, pdf_engine="pdfreport", target_length=8000)
+
+# Retrieve and fill in the prompt template with the anchor_text
+prompt_template_fn = get_prompt(task_type)
+PROMPT = prompt_template_fn(anchor_text)
+
+messages = [{
+        "role": "user",
+        "content": [
+            {"type": "text", "text": PROMPT},
+            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_base64}"}},
+        ],
+    }]
+# send messages to openai compatible api
+openai = OpenAI(base_url="https://api.opentyphoon.ai/v1", api_key="TYPHOON_API_KEY")
+response = openai.chat.completions.create(
+          model="typhoon-ocr-preview",
+          messages=messages,
+          max_tokens=16384,
+          extra_body={
+              "repetition_penalty": 1.2,
+              "temperature": 0.1,
+              "top_p": 0.6,
+          },
+      )
+text_output = response.choices[0].message.content
+print(text_output)
+```
+**Local Model (GPU Required)**:
+```python
+# Initialize the model
+model = Qwen2_5_VLForConditionalGeneration.from_pretrained("scb10x/typhoon-ocr-7b", torch_dtype=torch.bfloat16 ).eval()
+processor = AutoProcessor.from_pretrained("scb10x/typhoon-ocr-7b")
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+model.to(device)
+# Apply the chat template and processor
+text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+main_image = Image.open(BytesIO(base64.b64decode(image_base64)))
+
+inputs = processor(
+          text=[text],
+          images=[main_image],
+          padding=True,
+          return_tensors="pt",
+      )
+inputs = {key: value.to(device) for (key, value) in inputs.items()}
+
+# Generate the output
+output = model.generate(
+                  **inputs,
+                  temperature=0.1,
+                  max_new_tokens=12000,
+                  num_return_sequences=1,
+                  repetition_penalty=1.2,
+                  do_sample=True,
+              )
+# Decode the output
+prompt_length = inputs["input_ids"].shape[1]
+new_tokens = output[:, prompt_length:]
+text_output = processor.tokenizer.batch_decode(
+          new_tokens, skip_special_tokens=True
+      )
+print(text_output[0])
+```
+
+## **Intended Uses & Limitations**
+
+This model is an instructional model. However, it’s still undergoing development. It incorporates some level of guardrails, but it still may produce answers that are inaccurate, biased, or otherwise objectionable in response to user prompts. We recommend that developers assess these risks in the context of their use case.
+
+## **Follow us**
+
+**https://twitter.com/opentyphoon**
+
+## **Support**
+
+**https://discord.gg/us5gAYmrxw**
 
 
 ## **Citation**