Update README.md

README.md

@@ -78,6 +78,22 @@ from typhoon_ocr import ocr_document
 markdown = ocr_document("test.png")
 print(markdown)
 ```
+
+**(Recommended): Local Model via vllm (GPU Required)**:
+
+```bash
+pip install vllm
+vllm serve scb10x/typhoon-ocr-7b --max-model-len 32000 --served-model-name typhoon-ocr-preview # OpenAI Compatible at http://localhost:8000 (or other port)
+# then you can supply base_url in to ocr_document
+```
+
+```python
+from typhoon_ocr import ocr_document
+markdown = ocr_document('image.png', base_url='http://localhost:8000/v1', api_key='anything-is-ok')
+print(markdown)
+```
+To read more about [vllm](https://docs.vllm.ai/en/latest/getting_started/quickstart.html)
+
 **Run Manually**
 
 Below is a partial snippet. You can run inference using either the API or a local model.
@@ -149,7 +165,8 @@ response = openai.chat.completions.create(
 text_output = response.choices[0].message.content
 print(text_output)
 ```
-*Local Model (GPU Required)*:
+
+*(Not Recommended): Local Model - Transformers (GPU Required)*:
 ```python
 # Initialize the model
 model = Qwen2_5_VLForConditionalGeneration.from_pretrained("scb10x/typhoon-ocr-7b", torch_dtype=torch.bfloat16 ).eval()
@@ -191,7 +208,7 @@ print(text_output[0])
 
 This model only works with the specific prompts defined below, where `{base_text}` refers to information extracted from the PDF metadata using the `get_anchor_text` function from the `typhoon-ocr` package. It will not function correctly with any other prompts.
 
-```
+```python
 PROMPTS_SYS = {
     "default": lambda base_text: (f"Below is an image of a document page along with its dimensions. "
         f"Simply return the markdown representation of this document, presenting tables in markdown format as they naturally appear.\n"
@@ -212,16 +229,23 @@ PROMPTS_SYS = {
 ### Generation Parameters
 
 We suggest using the following generation parameters. Since this is an OCR model, we do not recommend using a high temperature. Make sure the temperature is set to 0 or 0.1, not higher.
-```
+```python
 temperature=0.1,
 top_p=0.6,
 repetition_penalty: 1.2
 ```
 
 ## Hosting
-```
+
+We recommend to inference typhoon-ocr using [vllm](https://github.com/vllm-project/vllm) instead of huggingface transformers, and using typhoon-ocr library to ocr documents. To read more about [vllm](https://docs.vllm.ai/en/latest/getting_started/quickstart.html)
+```bash
 vllm serve scb10x/typhoon-ocr-7b --max-model-len 32000 # OpenAI Compatible at http://localhost:8000
-# then you can supply base_url in to ocr_document('image.jpg', base_url='http://localhost:8000/v1')
+# then you can supply base_url in to ocr_document
+```
+
+```python
+from typhoon_ocr import ocr_document
+ocr_document('image.jpg', base_url='http://localhost:8000/v1')
 ```
 
 ## **Intended Uses & Limitations**