Image-Text-to-Text
Transformers
Safetensors
English
Chinese
multilingual
glm_ocr
ocr
vision-language-model
document-understanding
conversational
Eval Results
Instructions to use XCurOS/XCurOS-OCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use XCurOS/XCurOS-OCR with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="XCurOS/XCurOS-OCR") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("XCurOS/XCurOS-OCR") model = AutoModelForMultimodalLM.from_pretrained("XCurOS/XCurOS-OCR") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use XCurOS/XCurOS-OCR with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "XCurOS/XCurOS-OCR" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XCurOS/XCurOS-OCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/XCurOS/XCurOS-OCR
- SGLang
How to use XCurOS/XCurOS-OCR with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "XCurOS/XCurOS-OCR" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XCurOS/XCurOS-OCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "XCurOS/XCurOS-OCR" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XCurOS/XCurOS-OCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use XCurOS/XCurOS-OCR with Docker Model Runner:
docker model run hf.co/XCurOS/XCurOS-OCR
Add XCurOS-OCR model weights
Browse files- .eval_results/mdpbench.yaml +198 -0
- .eval_results/olmocrbench.yaml +73 -0
- LICENSE +9 -0
- chat_template.jinja +140 -0
- config.json +65 -0
- generation_config.json +10 -0
- model.safetensors +3 -0
- preprocessor_config.json +11 -0
- tokenizer.json +0 -0
- tokenizer_config.json +49 -0
.eval_results/mdpbench.yaml
ADDED
|
@@ -0,0 +1,198 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
- dataset:
|
| 2 |
+
id: Delores-Lin/MDPBench
|
| 3 |
+
task_id: overall
|
| 4 |
+
value: 67.3
|
| 5 |
+
date: "2026-04-14"
|
| 6 |
+
source:
|
| 7 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 8 |
+
name: MDPBench leaderboard
|
| 9 |
+
user: Delores-Lin
|
| 10 |
+
- dataset:
|
| 11 |
+
id: Delores-Lin/MDPBench
|
| 12 |
+
task_id: digital
|
| 13 |
+
value: 77.9
|
| 14 |
+
date: "2026-04-14"
|
| 15 |
+
source:
|
| 16 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 17 |
+
name: MDPBench leaderboard
|
| 18 |
+
user: Delores-Lin
|
| 19 |
+
- dataset:
|
| 20 |
+
id: Delores-Lin/MDPBench
|
| 21 |
+
task_id: photographed
|
| 22 |
+
value: 63.7
|
| 23 |
+
date: "2026-04-14"
|
| 24 |
+
source:
|
| 25 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 26 |
+
name: MDPBench leaderboard
|
| 27 |
+
user: Delores-Lin
|
| 28 |
+
- dataset:
|
| 29 |
+
id: Delores-Lin/MDPBench
|
| 30 |
+
task_id: latin
|
| 31 |
+
value: 78.7
|
| 32 |
+
date: "2026-04-14"
|
| 33 |
+
source:
|
| 34 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 35 |
+
name: MDPBench leaderboard
|
| 36 |
+
user: Delores-Lin
|
| 37 |
+
- dataset:
|
| 38 |
+
id: Delores-Lin/MDPBench
|
| 39 |
+
task_id: de
|
| 40 |
+
value: 82.7
|
| 41 |
+
date: "2026-04-14"
|
| 42 |
+
source:
|
| 43 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 44 |
+
name: MDPBench leaderboard
|
| 45 |
+
user: Delores-Lin
|
| 46 |
+
- dataset:
|
| 47 |
+
id: Delores-Lin/MDPBench
|
| 48 |
+
task_id: en
|
| 49 |
+
value: 84.5
|
| 50 |
+
date: "2026-04-14"
|
| 51 |
+
source:
|
| 52 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 53 |
+
name: MDPBench leaderboard
|
| 54 |
+
user: Delores-Lin
|
| 55 |
+
- dataset:
|
| 56 |
+
id: Delores-Lin/MDPBench
|
| 57 |
+
task_id: es
|
| 58 |
+
value: 75.8
|
| 59 |
+
date: "2026-04-14"
|
| 60 |
+
source:
|
| 61 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 62 |
+
name: MDPBench leaderboard
|
| 63 |
+
user: Delores-Lin
|
| 64 |
+
- dataset:
|
| 65 |
+
id: Delores-Lin/MDPBench
|
| 66 |
+
task_id: fr
|
| 67 |
+
value: 76.2
|
| 68 |
+
date: "2026-04-14"
|
| 69 |
+
source:
|
| 70 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 71 |
+
name: MDPBench leaderboard
|
| 72 |
+
user: Delores-Lin
|
| 73 |
+
- dataset:
|
| 74 |
+
id: Delores-Lin/MDPBench
|
| 75 |
+
task_id: id
|
| 76 |
+
value: 79.7
|
| 77 |
+
date: "2026-04-14"
|
| 78 |
+
source:
|
| 79 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 80 |
+
name: MDPBench leaderboard
|
| 81 |
+
user: Delores-Lin
|
| 82 |
+
- dataset:
|
| 83 |
+
id: Delores-Lin/MDPBench
|
| 84 |
+
task_id: it
|
| 85 |
+
value: 82.8
|
| 86 |
+
date: "2026-04-14"
|
| 87 |
+
source:
|
| 88 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 89 |
+
name: MDPBench leaderboard
|
| 90 |
+
user: Delores-Lin
|
| 91 |
+
- dataset:
|
| 92 |
+
id: Delores-Lin/MDPBench
|
| 93 |
+
task_id: nl
|
| 94 |
+
value: 80.2
|
| 95 |
+
date: "2026-04-14"
|
| 96 |
+
source:
|
| 97 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 98 |
+
name: MDPBench leaderboard
|
| 99 |
+
user: Delores-Lin
|
| 100 |
+
- dataset:
|
| 101 |
+
id: Delores-Lin/MDPBench
|
| 102 |
+
task_id: pt
|
| 103 |
+
value: 77.4
|
| 104 |
+
date: "2026-04-14"
|
| 105 |
+
source:
|
| 106 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 107 |
+
name: MDPBench leaderboard
|
| 108 |
+
user: Delores-Lin
|
| 109 |
+
- dataset:
|
| 110 |
+
id: Delores-Lin/MDPBench
|
| 111 |
+
task_id: vi
|
| 112 |
+
value: 69.2
|
| 113 |
+
date: "2026-04-14"
|
| 114 |
+
source:
|
| 115 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 116 |
+
name: MDPBench leaderboard
|
| 117 |
+
user: Delores-Lin
|
| 118 |
+
- dataset:
|
| 119 |
+
id: Delores-Lin/MDPBench
|
| 120 |
+
task_id: non_latin
|
| 121 |
+
value: 54.3
|
| 122 |
+
date: "2026-04-14"
|
| 123 |
+
source:
|
| 124 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 125 |
+
name: MDPBench leaderboard
|
| 126 |
+
user: Delores-Lin
|
| 127 |
+
- dataset:
|
| 128 |
+
id: Delores-Lin/MDPBench
|
| 129 |
+
task_id: ar
|
| 130 |
+
value: 21.7
|
| 131 |
+
date: "2026-04-14"
|
| 132 |
+
source:
|
| 133 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 134 |
+
name: MDPBench leaderboard
|
| 135 |
+
user: Delores-Lin
|
| 136 |
+
- dataset:
|
| 137 |
+
id: Delores-Lin/MDPBench
|
| 138 |
+
task_id: hi
|
| 139 |
+
value: 39.6
|
| 140 |
+
date: "2026-04-14"
|
| 141 |
+
source:
|
| 142 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 143 |
+
name: MDPBench leaderboard
|
| 144 |
+
user: Delores-Lin
|
| 145 |
+
- dataset:
|
| 146 |
+
id: Delores-Lin/MDPBench
|
| 147 |
+
task_id: jp
|
| 148 |
+
value: 65.5
|
| 149 |
+
date: "2026-04-14"
|
| 150 |
+
source:
|
| 151 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 152 |
+
name: MDPBench leaderboard
|
| 153 |
+
user: Delores-Lin
|
| 154 |
+
- dataset:
|
| 155 |
+
id: Delores-Lin/MDPBench
|
| 156 |
+
task_id: ko
|
| 157 |
+
value: 61.2
|
| 158 |
+
date: "2026-04-14"
|
| 159 |
+
source:
|
| 160 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 161 |
+
name: MDPBench leaderboard
|
| 162 |
+
user: Delores-Lin
|
| 163 |
+
- dataset:
|
| 164 |
+
id: Delores-Lin/MDPBench
|
| 165 |
+
task_id: ru
|
| 166 |
+
value: 64.2
|
| 167 |
+
date: "2026-04-14"
|
| 168 |
+
source:
|
| 169 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 170 |
+
name: MDPBench leaderboard
|
| 171 |
+
user: Delores-Lin
|
| 172 |
+
- dataset:
|
| 173 |
+
id: Delores-Lin/MDPBench
|
| 174 |
+
task_id: th
|
| 175 |
+
value: 27.4
|
| 176 |
+
date: "2026-04-14"
|
| 177 |
+
source:
|
| 178 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 179 |
+
name: MDPBench leaderboard
|
| 180 |
+
user: Delores-Lin
|
| 181 |
+
- dataset:
|
| 182 |
+
id: Delores-Lin/MDPBench
|
| 183 |
+
task_id: zh
|
| 184 |
+
value: 78.5
|
| 185 |
+
date: "2026-04-14"
|
| 186 |
+
source:
|
| 187 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 188 |
+
name: MDPBench leaderboard
|
| 189 |
+
user: Delores-Lin
|
| 190 |
+
- dataset:
|
| 191 |
+
id: Delores-Lin/MDPBench
|
| 192 |
+
task_id: zh_t
|
| 193 |
+
value: 76.7
|
| 194 |
+
date: "2026-04-14"
|
| 195 |
+
source:
|
| 196 |
+
url: https://huggingface.co/datasets/Delores-Lin/MDPBench
|
| 197 |
+
name: MDPBench leaderboard
|
| 198 |
+
user: Delores-Lin
|
.eval_results/olmocrbench.yaml
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
- dataset:
|
| 2 |
+
id: allenai/olmOCR-bench
|
| 3 |
+
task_id: overall
|
| 4 |
+
value: 75.2
|
| 5 |
+
notes: "Excluding Headers & Footers category. Using ZAI API."
|
| 6 |
+
source:
|
| 7 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 8 |
+
name: GLM-OCR API evaluation
|
| 9 |
+
user: staghado
|
| 10 |
+
- dataset:
|
| 11 |
+
id: allenai/olmOCR-bench
|
| 12 |
+
task_id: arxiv_math
|
| 13 |
+
value: 80.7
|
| 14 |
+
source:
|
| 15 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 16 |
+
name: GLM-OCR API evaluation
|
| 17 |
+
user: staghado
|
| 18 |
+
- dataset:
|
| 19 |
+
id: allenai/olmOCR-bench
|
| 20 |
+
task_id: old_scans_math
|
| 21 |
+
value: 68.3
|
| 22 |
+
source:
|
| 23 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 24 |
+
name: GLM-OCR API evaluation
|
| 25 |
+
user: staghado
|
| 26 |
+
- dataset:
|
| 27 |
+
id: allenai/olmOCR-bench
|
| 28 |
+
task_id: table_tests
|
| 29 |
+
value: 77.6
|
| 30 |
+
source:
|
| 31 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 32 |
+
name: GLM-OCR API evaluation
|
| 33 |
+
user: staghado
|
| 34 |
+
- dataset:
|
| 35 |
+
id: allenai/olmOCR-bench
|
| 36 |
+
task_id: old_scans
|
| 37 |
+
value: 37.6
|
| 38 |
+
source:
|
| 39 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 40 |
+
name: GLM-OCR API evaluation
|
| 41 |
+
user: staghado
|
| 42 |
+
- dataset:
|
| 43 |
+
id: allenai/olmOCR-bench
|
| 44 |
+
task_id: multi_column
|
| 45 |
+
value: 76.7
|
| 46 |
+
source:
|
| 47 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 48 |
+
name: GLM-OCR API evaluation
|
| 49 |
+
user: staghado
|
| 50 |
+
- dataset:
|
| 51 |
+
id: allenai/olmOCR-bench
|
| 52 |
+
task_id: long_tiny_text
|
| 53 |
+
value: 86.9
|
| 54 |
+
source:
|
| 55 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 56 |
+
name: GLM-OCR API evaluation
|
| 57 |
+
user: staghado
|
| 58 |
+
- dataset:
|
| 59 |
+
id: allenai/olmOCR-bench
|
| 60 |
+
task_id: headers_footers
|
| 61 |
+
value: 95.8
|
| 62 |
+
source:
|
| 63 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 64 |
+
name: GLM-OCR API evaluation
|
| 65 |
+
user: staghado
|
| 66 |
+
- dataset:
|
| 67 |
+
id: allenai/olmOCR-bench
|
| 68 |
+
task_id: baseline
|
| 69 |
+
value: 98.8
|
| 70 |
+
source:
|
| 71 |
+
url: https://huggingface.co/zai-org/GLM-OCR
|
| 72 |
+
name: GLM-OCR API evaluation
|
| 73 |
+
user: staghado
|
LICENSE
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2025 the original authors
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
| 6 |
+
|
| 7 |
+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
| 8 |
+
|
| 9 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[gMASK]<sop>
|
| 2 |
+
{%- if tools -%}
|
| 3 |
+
<|system|>
|
| 4 |
+
# Tools
|
| 5 |
+
|
| 6 |
+
You may call one or more functions to assist with the user query.
|
| 7 |
+
|
| 8 |
+
You are provided with function signatures within <tools></tools> XML tags:
|
| 9 |
+
<tools>
|
| 10 |
+
{% for tool in tools %}
|
| 11 |
+
{{ tool | tojson(ensure_ascii=False) }}
|
| 12 |
+
{% endfor %}
|
| 13 |
+
</tools>
|
| 14 |
+
|
| 15 |
+
For each function call, output the function name and arguments within the following XML format:
|
| 16 |
+
<tool_call>{function-name}
|
| 17 |
+
<arg_key>{arg-key-1}</arg_key>
|
| 18 |
+
<arg_value>{arg-value-1}</arg_value>
|
| 19 |
+
<arg_key>{arg-key-2}</arg_key>
|
| 20 |
+
<arg_value>{arg-value-2}</arg_value>
|
| 21 |
+
...
|
| 22 |
+
</tool_call>{%- endif -%}
|
| 23 |
+
{%- macro visible_text(content) -%}
|
| 24 |
+
{%- if content is string -%}
|
| 25 |
+
{{- content }}
|
| 26 |
+
{%- elif content is iterable and content is not mapping -%}
|
| 27 |
+
{%- for item in content -%}
|
| 28 |
+
{%- if item is mapping and item.type == 'text' -%}
|
| 29 |
+
{{- item.text }}
|
| 30 |
+
{%- elif item is mapping and (item.type == 'image' or 'image' in item) -%}
|
| 31 |
+
<|begin_of_image|><|image|><|end_of_image|>
|
| 32 |
+
{%- elif item is mapping and (item.type == 'video' or 'video' in item) -%}
|
| 33 |
+
<|begin_of_video|><|video|><|end_of_video|>
|
| 34 |
+
{%- elif item is string -%}
|
| 35 |
+
{{- item }}
|
| 36 |
+
{%- endif -%}
|
| 37 |
+
{%- endfor -%}
|
| 38 |
+
{%- else -%}
|
| 39 |
+
{{- content }}
|
| 40 |
+
{%- endif -%}
|
| 41 |
+
{%- endmacro -%}
|
| 42 |
+
{%- set ns = namespace(last_user_index=-1) %}
|
| 43 |
+
{%- for m in messages %}
|
| 44 |
+
{%- if m.role == 'user' %}
|
| 45 |
+
{% set ns.last_user_index = loop.index0 -%}
|
| 46 |
+
{%- endif %}
|
| 47 |
+
{%- endfor %}
|
| 48 |
+
{% for m in messages %}
|
| 49 |
+
{%- if m.role == 'user' -%}<|user|>
|
| 50 |
+
{% if m.content is string %}
|
| 51 |
+
{{ m.content }}
|
| 52 |
+
{%- else %}
|
| 53 |
+
{%- for item in m.content %}
|
| 54 |
+
{% if item.type == 'video' or 'video' in item %}
|
| 55 |
+
<|begin_of_video|><|video|><|end_of_video|>{% elif item.type == 'image' or 'image' in item %}
|
| 56 |
+
<|begin_of_image|><|image|><|end_of_image|>{% elif item.type == 'text' %}
|
| 57 |
+
{{ item.text }}
|
| 58 |
+
{%- endif %}
|
| 59 |
+
{%- endfor %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}}
|
| 62 |
+
{%- elif m.role == 'assistant' -%}
|
| 63 |
+
<|assistant|>
|
| 64 |
+
{%- set reasoning_content = '' %}
|
| 65 |
+
{%- set content = visible_text(m.content) %}
|
| 66 |
+
{%- if m.reasoning_content is string %}
|
| 67 |
+
{%- set reasoning_content = m.reasoning_content %}
|
| 68 |
+
{%- else %}
|
| 69 |
+
{%- if '</think>' in content %}
|
| 70 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 71 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 72 |
+
{%- endif %}
|
| 73 |
+
{%- endif %}
|
| 74 |
+
{%- if loop.index0 > ns.last_user_index and reasoning_content -%}
|
| 75 |
+
{{ '\n<think>' + reasoning_content.strip() + '</think>'}}
|
| 76 |
+
{%- else -%}
|
| 77 |
+
{{ '\n<think></think>' }}
|
| 78 |
+
{%- endif -%}
|
| 79 |
+
{%- if content.strip() -%}
|
| 80 |
+
{{ '\n' + content.strip() }}
|
| 81 |
+
{%- endif -%}
|
| 82 |
+
{% if m.tool_calls %}
|
| 83 |
+
{% for tc in m.tool_calls %}
|
| 84 |
+
{%- if tc.function %}
|
| 85 |
+
{%- set tc = tc.function %}
|
| 86 |
+
{%- endif %}
|
| 87 |
+
{{ '\n<tool_call>' + tc.name }}
|
| 88 |
+
{% set _args = tc.arguments %}
|
| 89 |
+
{% for k, v in _args.items() %}
|
| 90 |
+
<arg_key>{{ k }}</arg_key>
|
| 91 |
+
<arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>
|
| 92 |
+
{% endfor %}
|
| 93 |
+
</tool_call>{% endfor %}
|
| 94 |
+
{% endif %}
|
| 95 |
+
{%- elif m.role == 'tool' -%}
|
| 96 |
+
{%- if m.content is string -%}
|
| 97 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 98 |
+
{{- '<|observation|>' }}
|
| 99 |
+
{%- endif %}
|
| 100 |
+
{{- '\n<tool_response>\n' }}
|
| 101 |
+
{{- m.content }}
|
| 102 |
+
{{- '\n</tool_response>' }}
|
| 103 |
+
{% elif m.content is iterable and m.content is not mapping %}
|
| 104 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 105 |
+
{{- '<|observation|>' }}
|
| 106 |
+
{%- endif %}
|
| 107 |
+
{{- '\n<tool_response>\n' }}
|
| 108 |
+
{%- for tr in m.content -%}
|
| 109 |
+
{%- if tr is mapping and tr.type is defined -%}
|
| 110 |
+
{%- set t = tr.type | lower -%}
|
| 111 |
+
{%- if t == 'text' and tr.text is defined -%}
|
| 112 |
+
{{ tr.text }}
|
| 113 |
+
{%- elif t in ['image', 'image_url'] -%}
|
| 114 |
+
<|begin_of_image|><|image|><|end_of_image|>
|
| 115 |
+
{%- elif t in ['video', 'video_url'] -%}
|
| 116 |
+
<|begin_of_video|><|video|><|end_of_video|>
|
| 117 |
+
{%- else -%}
|
| 118 |
+
{{ tr | tojson(ensure_ascii=False) }}
|
| 119 |
+
{%- endif -%}
|
| 120 |
+
{%- else -%}
|
| 121 |
+
{{ tr.output if tr.output is defined else tr }}
|
| 122 |
+
{%- endif -%}
|
| 123 |
+
{%- endfor -%}
|
| 124 |
+
{{- '\n</tool_response>' }}
|
| 125 |
+
{%- else -%}
|
| 126 |
+
<|observation|>{% for tr in m.content %}
|
| 127 |
+
|
| 128 |
+
<tool_response>
|
| 129 |
+
{{ tr.output if tr.output is defined else tr }}
|
| 130 |
+
</tool_response>{% endfor -%}
|
| 131 |
+
{% endif -%}
|
| 132 |
+
{%- elif m.role == 'system' -%}
|
| 133 |
+
<|system|>
|
| 134 |
+
{{ visible_text(m.content) }}
|
| 135 |
+
{%- endif -%}
|
| 136 |
+
{%- endfor -%}
|
| 137 |
+
{%- if add_generation_prompt -%}
|
| 138 |
+
<|assistant|>
|
| 139 |
+
{{'<think></think>\n' if (enable_thinking is defined and not enable_thinking) else ''}}
|
| 140 |
+
{%- endif -%}
|
config.json
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"GlmOcrForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"model_type": "glm_ocr",
|
| 6 |
+
"text_config": {
|
| 7 |
+
"model_type": "glm_ocr_text",
|
| 8 |
+
"pad_token_id": 59246,
|
| 9 |
+
"vocab_size": 59392,
|
| 10 |
+
"eos_token_id": [
|
| 11 |
+
59246,
|
| 12 |
+
59253
|
| 13 |
+
],
|
| 14 |
+
"attention_bias": false,
|
| 15 |
+
"attention_dropout": 0.0,
|
| 16 |
+
"head_dim": 128,
|
| 17 |
+
"hidden_act": "silu",
|
| 18 |
+
"hidden_size": 1536,
|
| 19 |
+
"initializer_range": 0.02,
|
| 20 |
+
"intermediate_size": 4608,
|
| 21 |
+
"max_position_embeddings": 131072,
|
| 22 |
+
"num_attention_heads": 16,
|
| 23 |
+
"num_hidden_layers": 16,
|
| 24 |
+
"num_nextn_predict_layers": 1,
|
| 25 |
+
"num_key_value_heads": 8,
|
| 26 |
+
"rms_norm_eps": 1e-05,
|
| 27 |
+
"dtype": "bfloat16",
|
| 28 |
+
"rope_parameters": {
|
| 29 |
+
"rope_type": "default",
|
| 30 |
+
"mrope_section": [
|
| 31 |
+
16,
|
| 32 |
+
24,
|
| 33 |
+
24
|
| 34 |
+
],
|
| 35 |
+
"partial_rotary_factor": 1.0,
|
| 36 |
+
"rope_theta": 10000
|
| 37 |
+
},
|
| 38 |
+
"tie_word_embeddings": false,
|
| 39 |
+
"use_cache": true
|
| 40 |
+
},
|
| 41 |
+
"vision_config": {
|
| 42 |
+
"model_type": "glm_ocr_vision",
|
| 43 |
+
"hidden_size": 1024,
|
| 44 |
+
"depth": 24,
|
| 45 |
+
"num_heads": 16,
|
| 46 |
+
"attention_bias": true,
|
| 47 |
+
"intermediate_size": 4096,
|
| 48 |
+
"hidden_act": "silu",
|
| 49 |
+
"hidden_dropout_prob": 0.0,
|
| 50 |
+
"initializer_range": 0.02,
|
| 51 |
+
"image_size": 336,
|
| 52 |
+
"patch_size": 14,
|
| 53 |
+
"out_hidden_size": 1536,
|
| 54 |
+
"rms_norm_eps": 1e-05,
|
| 55 |
+
"spatial_merge_size": 2,
|
| 56 |
+
"temporal_patch_size": 2
|
| 57 |
+
},
|
| 58 |
+
"image_start_token_id": 59256,
|
| 59 |
+
"image_end_token_id": 59257,
|
| 60 |
+
"video_start_token_id": 59258,
|
| 61 |
+
"video_end_token_id": 59259,
|
| 62 |
+
"image_token_id": 59280,
|
| 63 |
+
"video_token_id": 59281,
|
| 64 |
+
"transformers_version": "5.0.1dev0"
|
| 65 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"do_sample": false,
|
| 4 |
+
"eos_token_id": [
|
| 5 |
+
59246,
|
| 6 |
+
59253
|
| 7 |
+
],
|
| 8 |
+
"pad_token_id": 59246,
|
| 9 |
+
"transformers_version": "5.0.1dev0"
|
| 10 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a16eb0de98d199293371c560f95f83130d2a2c9612449df16839f08ff9498815
|
| 3 |
+
size 2650579464
|
preprocessor_config.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"size": {"shortest_edge": 12544, "longest_edge": 9633792},
|
| 3 |
+
"do_rescale": true,
|
| 4 |
+
"patch_size": 14,
|
| 5 |
+
"temporal_patch_size": 2,
|
| 6 |
+
"merge_size": 2,
|
| 7 |
+
"image_mean": [0.48145466, 0.4578275, 0.40821073],
|
| 8 |
+
"image_std": [0.26862954, 0.26130258, 0.27577711],
|
| 9 |
+
"image_processor_type": "Glm46VImageProcessor",
|
| 10 |
+
"processor_class": "Glm46VProcessor"
|
| 11 |
+
}
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"backend": "tokenizers",
|
| 3 |
+
"clean_up_tokenization_spaces": false,
|
| 4 |
+
"eos_token": "<|endoftext|>",
|
| 5 |
+
"extra_special_tokens": [
|
| 6 |
+
"<|endoftext|>",
|
| 7 |
+
"[MASK]",
|
| 8 |
+
"[gMASK]",
|
| 9 |
+
"[sMASK]",
|
| 10 |
+
"<sop>",
|
| 11 |
+
"<eop>",
|
| 12 |
+
"<|system|>",
|
| 13 |
+
"<|user|>",
|
| 14 |
+
"<|assistant|>",
|
| 15 |
+
"<|observation|>",
|
| 16 |
+
"<|begin_of_image|>",
|
| 17 |
+
"<|end_of_image|>",
|
| 18 |
+
"<|begin_of_video|>",
|
| 19 |
+
"<|end_of_video|>",
|
| 20 |
+
"<|begin_of_audio|>",
|
| 21 |
+
"<|end_of_audio|>",
|
| 22 |
+
"<|begin_of_transcription|>",
|
| 23 |
+
"<|end_of_transcription|>",
|
| 24 |
+
"<|code_prefix|>",
|
| 25 |
+
"<|code_middle|>",
|
| 26 |
+
"<|code_suffix|>",
|
| 27 |
+
"<think>",
|
| 28 |
+
"</think>",
|
| 29 |
+
"<tool_call>",
|
| 30 |
+
"</tool_call>",
|
| 31 |
+
"<tool_response>",
|
| 32 |
+
"</tool_response>",
|
| 33 |
+
"<arg_key>",
|
| 34 |
+
"</arg_key>",
|
| 35 |
+
"<arg_value>",
|
| 36 |
+
"</arg_value>",
|
| 37 |
+
"/nothink",
|
| 38 |
+
"<|begin_of_box|>",
|
| 39 |
+
"<|end_of_box|>",
|
| 40 |
+
"<|image|>",
|
| 41 |
+
"<|video|>"
|
| 42 |
+
],
|
| 43 |
+
"is_local": true,
|
| 44 |
+
"model_max_length": 655380,
|
| 45 |
+
"pad_token": "<|endoftext|>",
|
| 46 |
+
"padding_side": "left",
|
| 47 |
+
"processor_class": "Glm46VProcessor",
|
| 48 |
+
"tokenizer_class": "TokenizersBackend"
|
| 49 |
+
}
|