Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,76 @@
+---
+language:
+- en
+- zh
+tags:
+- fp8
+- quantization
+- dynamic
+- vision-language
+- multimodal
+- vllm
+- llm-compressor
+- skywork_chat
+- Skywork+R1V
+pipeline_tag: image-text-to-text
+inference: false
+license: mit
+---
+# 🔥 Skywork-R1V3-38B-FP8-Dynamic: Optimized Vision-Language Model 🔥
+This is a **FP8 dynamic quantized** version of [Skywork/Skywork-R1V3-38B](https://huggingface.co/Skywork/Skywork-R1V3-38B), optimized for high-performance inference with vLLM. 
+The model utilizes **dynamic FP8 quantization** for optimal ease of use and deployment, achieving significant speedup with minimal accuracy degradation on vision-language tasks.
+## 🚀 Key Features
+- **FP8 Dynamic Quantization**: No calibration required, ready to use immediately
+- **Vision-Language Optimized**: Specialized quantization recipe that preserves visual understanding
+- **vLLM Ready**: Seamless integration with vLLM for production deployment  
+- **Memory Efficient**: ~50% memory reduction compared to FP16 original
+- **Performance Boost**: Significant faster inference on H100/L40S GPUs
+## 📊 Model Details
+- **Original Model**: [Skywork/Skywork-R1V3-38B](https://huggingface.co/Skywork/Skywork-R1V3-38B)
+- **Source Model**: Skywork/Skywork-R1V3-38B
+- **Quantized Model**: Skywork-R1V3-38B-FP8-Dynamic
+- **Quantization Method**: FP8 Dynamic (W8A8) 
+- **Quantization Library**: [LLM Compressor](https://github.com/vllm-project/llm-compressor) v0.6.1a20250708
+- **Quantized by**: [brandonbeiler](https://huggingface.co/brandonbeiler)
+## 🔧 Usage
+### With vLLM (Recommended)
+```python
+from vllm import LLM, SamplingParams
+
+# Load the quantized model
+
+model = LLM(
+    model="brandonbeiler/Skywork-R1V3-38B-FP8-Dynamic",
+    tensor_parallel_size=1, # Adjust based on your GPU setup
+    limit_mm_per_prompt={"image": 20},
+    trust_remote_code=True, # required for older versions of vLLM
+    max_model_len=32768,  # Decrease if you run into memory issues
+    gpu_memory_utilization=0.8,  # Adjust based on your GPU memory
+)
+
+# Generate response
+sampling_params = SamplingParams(temperature=0.0, max_tokens=8000) # adjust temperature as desired
+response = model.generate("Describe this image: <image>", sampling_params)
+print(response[0].outputs[0].text)
+```
+
+## 🏗️ Technical Specifications
+### Hardware Requirements
+- **Inference**: ? VRAM (+ VRAM for context)
+- **Supported GPUs**: H100, L40S, A100 (80GB), RTX 4090 (2x for tensor parallelism)
+- **GPU Architecture**: Ada Lovelace, Hopper (for optimal FP8 performance)
+### Quantization Details
+- **Weights**: FP8 E4M3 with dynamic per-tensor scales
+- **Activations**: FP8 E4M3 with dynamic per-tensor scales  
+- **Preserved Components**: Vision tower, embeddings, normalization layers, mlp1
+## 🔬 Package Versions
+This model was created using:
+```
+llmcompressor==0.6.1a20250708
+compressed-tensors==latest
+transformers==4.52.4
+torch==2.7.0
+vllm==0.9.2
+```
+
+*Quantized with ❤️ using LLM Compressor for the open-source community*

added_tokens.json

@@ -0,0 +1,33 @@
+{
+  "</box>": 151673,
+  "</img>": 151666,
+  "</quad>": 151669,
+  "</ref>": 151671,
+  "</tool_call>": 151658,
+  "<IMG_CONTEXT>": 151667,
+  "<box>": 151672,
+  "<img>": 151665,
+  "<quad>": 151668,
+  "<ref>": 151670,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'FIRST, think through the problem step-by-step. Explain each step clearly, including any relevant concepts or formulas. Reflect on why each step is necessary and check for potential errors. Consider alternative approaches and justify the chosen method. Enclose this entire reasoning process within <think></think> tags. THEN, provide the final answer enclosed in \boxed{}.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n<think>\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,308 @@
+{
+  "architectures": [
+    "SkyworkR1VChatModel"
+  ],
+  "auto_map": {
+    "AutoConfig": "Skywork/Skywork-R1V3-38B--configuration_skywork_chat.SkyworkChatConfig",
+    "AutoModel": "Skywork/Skywork-R1V3-38B--modeling_skywork_chat.SkyworkChatModel",
+    "AutoModelForCausalLM": "Skywork/Skywork-R1V3-38B--modeling_skywork_chat.SkyworkChatModel"
+  },
+  "downsample_ratio": 0.5,
+  "dynamic_image_size": true,
+  "force_image_size": 448,
+  "freeze_adapter": false,
+  "freeze_llm": false,
+  "freeze_vision": false,
+  "hidden_size": 5120,
+  "llm_config": {
+    "architectures": [
+      "Qwen2ForCausalLM"
+    ],
+    "attention_dropout": 0.0,
+    "attn_implementation": "flash_attention_2",
+    "bos_token_id": 151643,
+    "eos_token_id": 151645,
+    "hidden_act": "silu",
+    "hidden_size": 5120,
+    "initializer_range": 0.02,
+    "intermediate_size": 27648,
+    "max_position_embeddings": 32768,
+    "max_window_layers": 70,
+    "model_type": "qwen2",
+    "num_attention_heads": 40,
+    "num_hidden_layers": 64,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-06,
+    "rope_scaling": null,
+    "rope_theta": 1000000.0,
+    "sliding_window": 131072,
+    "torch_dtype": "bfloat16",
+    "use_bfloat16": true,
+    "use_cache": false,
+    "use_sliding_window": false,
+    "vocab_size": 151674
+  },
+  "max_dynamic_patch": 6,
+  "min_dynamic_patch": 1,
+  "model_type": "skywork_chat",
+  "pad2square": false,
+  "ps_version": "v2",
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": true,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": null,
+          "observer_kwargs": {},
+          "strategy": "token",
+          "symmetric": true,
+          "type": "float"
+        },
+        "output_activations": null,
+        "targets": [
+          "Linear"
+        ],
+        "weights": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": false,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": "minmax",
+          "observer_kwargs": {},
+          "strategy": "channel",
+          "symmetric": true,
+          "type": "float"
+        }
+      }
+    },
+    "format": "float-quantized",
+    "global_compression_ratio": null,
+    "ignore": [
+      "vision_model.encoder.layers.0.attn.qkv",
+      "vision_model.encoder.layers.0.attn.proj",
+      "vision_model.encoder.layers.0.mlp.fc1",
+      "vision_model.encoder.layers.0.mlp.fc2",
+      "vision_model.encoder.layers.1.attn.qkv",
+      "vision_model.encoder.layers.1.attn.proj",
+      "vision_model.encoder.layers.1.mlp.fc1",
+      "vision_model.encoder.layers.1.mlp.fc2",
+      "vision_model.encoder.layers.2.attn.qkv",
+      "vision_model.encoder.layers.2.attn.proj",
+      "vision_model.encoder.layers.2.mlp.fc1",
+      "vision_model.encoder.layers.2.mlp.fc2",
+      "vision_model.encoder.layers.3.attn.qkv",
+      "vision_model.encoder.layers.3.attn.proj",
+      "vision_model.encoder.layers.3.mlp.fc1",
+      "vision_model.encoder.layers.3.mlp.fc2",
+      "vision_model.encoder.layers.4.attn.qkv",
+      "vision_model.encoder.layers.4.attn.proj",
+      "vision_model.encoder.layers.4.mlp.fc1",
+      "vision_model.encoder.layers.4.mlp.fc2",
+      "vision_model.encoder.layers.5.attn.qkv",
+      "vision_model.encoder.layers.5.attn.proj",
+      "vision_model.encoder.layers.5.mlp.fc1",
+      "vision_model.encoder.layers.5.mlp.fc2",
+      "vision_model.encoder.layers.6.attn.qkv",
+      "vision_model.encoder.layers.6.attn.proj",
+      "vision_model.encoder.layers.6.mlp.fc1",
+      "vision_model.encoder.layers.6.mlp.fc2",
+      "vision_model.encoder.layers.7.attn.qkv",
+      "vision_model.encoder.layers.7.attn.proj",
+      "vision_model.encoder.layers.7.mlp.fc1",
+      "vision_model.encoder.layers.7.mlp.fc2",
+      "vision_model.encoder.layers.8.attn.qkv",
+      "vision_model.encoder.layers.8.attn.proj",
+      "vision_model.encoder.layers.8.mlp.fc1",
+      "vision_model.encoder.layers.8.mlp.fc2",
+      "vision_model.encoder.layers.9.attn.qkv",
+      "vision_model.encoder.layers.9.attn.proj",
+      "vision_model.encoder.layers.9.mlp.fc1",
+      "vision_model.encoder.layers.9.mlp.fc2",
+      "vision_model.encoder.layers.10.attn.qkv",
+      "vision_model.encoder.layers.10.attn.proj",
+      "vision_model.encoder.layers.10.mlp.fc1",
+      "vision_model.encoder.layers.10.mlp.fc2",
+      "vision_model.encoder.layers.11.attn.qkv",
+      "vision_model.encoder.layers.11.attn.proj",
+      "vision_model.encoder.layers.11.mlp.fc1",
+      "vision_model.encoder.layers.11.mlp.fc2",
+      "vision_model.encoder.layers.12.attn.qkv",
+      "vision_model.encoder.layers.12.attn.proj",
+      "vision_model.encoder.layers.12.mlp.fc1",
+      "vision_model.encoder.layers.12.mlp.fc2",
+      "vision_model.encoder.layers.13.attn.qkv",
+      "vision_model.encoder.layers.13.attn.proj",
+      "vision_model.encoder.layers.13.mlp.fc1",
+      "vision_model.encoder.layers.13.mlp.fc2",
+      "vision_model.encoder.layers.14.attn.qkv",
+      "vision_model.encoder.layers.14.attn.proj",
+      "vision_model.encoder.layers.14.mlp.fc1",
+      "vision_model.encoder.layers.14.mlp.fc2",
+      "vision_model.encoder.layers.15.attn.qkv",
+      "vision_model.encoder.layers.15.attn.proj",
+      "vision_model.encoder.layers.15.mlp.fc1",
+      "vision_model.encoder.layers.15.mlp.fc2",
+      "vision_model.encoder.layers.16.attn.qkv",
+      "vision_model.encoder.layers.16.attn.proj",
+      "vision_model.encoder.layers.16.mlp.fc1",
+      "vision_model.encoder.layers.16.mlp.fc2",
+      "vision_model.encoder.layers.17.attn.qkv",
+      "vision_model.encoder.layers.17.attn.proj",
+      "vision_model.encoder.layers.17.mlp.fc1",
+      "vision_model.encoder.layers.17.mlp.fc2",
+      "vision_model.encoder.layers.18.attn.qkv",
+      "vision_model.encoder.layers.18.attn.proj",
+      "vision_model.encoder.layers.18.mlp.fc1",
+      "vision_model.encoder.layers.18.mlp.fc2",
+      "vision_model.encoder.layers.19.attn.qkv",
+      "vision_model.encoder.layers.19.attn.proj",
+      "vision_model.encoder.layers.19.mlp.fc1",
+      "vision_model.encoder.layers.19.mlp.fc2",
+      "vision_model.encoder.layers.20.attn.qkv",
+      "vision_model.encoder.layers.20.attn.proj",
+      "vision_model.encoder.layers.20.mlp.fc1",
+      "vision_model.encoder.layers.20.mlp.fc2",
+      "vision_model.encoder.layers.21.attn.qkv",
+      "vision_model.encoder.layers.21.attn.proj",
+      "vision_model.encoder.layers.21.mlp.fc1",
+      "vision_model.encoder.layers.21.mlp.fc2",
+      "vision_model.encoder.layers.22.attn.qkv",
+      "vision_model.encoder.layers.22.attn.proj",
+      "vision_model.encoder.layers.22.mlp.fc1",
+      "vision_model.encoder.layers.22.mlp.fc2",
+      "vision_model.encoder.layers.23.attn.qkv",
+      "vision_model.encoder.layers.23.attn.proj",
+      "vision_model.encoder.layers.23.mlp.fc1",
+      "vision_model.encoder.layers.23.mlp.fc2",
+      "vision_model.encoder.layers.24.attn.qkv",
+      "vision_model.encoder.layers.24.attn.proj",
+      "vision_model.encoder.layers.24.mlp.fc1",
+      "vision_model.encoder.layers.24.mlp.fc2",
+      "vision_model.encoder.layers.25.attn.qkv",
+      "vision_model.encoder.layers.25.attn.proj",
+      "vision_model.encoder.layers.25.mlp.fc1",
+      "vision_model.encoder.layers.25.mlp.fc2",
+      "vision_model.encoder.layers.26.attn.qkv",
+      "vision_model.encoder.layers.26.attn.proj",
+      "vision_model.encoder.layers.26.mlp.fc1",
+      "vision_model.encoder.layers.26.mlp.fc2",
+      "vision_model.encoder.layers.27.attn.qkv",
+      "vision_model.encoder.layers.27.attn.proj",
+      "vision_model.encoder.layers.27.mlp.fc1",
+      "vision_model.encoder.layers.27.mlp.fc2",
+      "vision_model.encoder.layers.28.attn.qkv",
+      "vision_model.encoder.layers.28.attn.proj",
+      "vision_model.encoder.layers.28.mlp.fc1",
+      "vision_model.encoder.layers.28.mlp.fc2",
+      "vision_model.encoder.layers.29.attn.qkv",
+      "vision_model.encoder.layers.29.attn.proj",
+      "vision_model.encoder.layers.29.mlp.fc1",
+      "vision_model.encoder.layers.29.mlp.fc2",
+      "vision_model.encoder.layers.30.attn.qkv",
+      "vision_model.encoder.layers.30.attn.proj",
+      "vision_model.encoder.layers.30.mlp.fc1",
+      "vision_model.encoder.layers.30.mlp.fc2",
+      "vision_model.encoder.layers.31.attn.qkv",
+      "vision_model.encoder.layers.31.attn.proj",
+      "vision_model.encoder.layers.31.mlp.fc1",
+      "vision_model.encoder.layers.31.mlp.fc2",
+      "vision_model.encoder.layers.32.attn.qkv",
+      "vision_model.encoder.layers.32.attn.proj",
+      "vision_model.encoder.layers.32.mlp.fc1",
+      "vision_model.encoder.layers.32.mlp.fc2",
+      "vision_model.encoder.layers.33.attn.qkv",
+      "vision_model.encoder.layers.33.attn.proj",
+      "vision_model.encoder.layers.33.mlp.fc1",
+      "vision_model.encoder.layers.33.mlp.fc2",
+      "vision_model.encoder.layers.34.attn.qkv",
+      "vision_model.encoder.layers.34.attn.proj",
+      "vision_model.encoder.layers.34.mlp.fc1",
+      "vision_model.encoder.layers.34.mlp.fc2",
+      "vision_model.encoder.layers.35.attn.qkv",
+      "vision_model.encoder.layers.35.attn.proj",
+      "vision_model.encoder.layers.35.mlp.fc1",
+      "vision_model.encoder.layers.35.mlp.fc2",
+      "vision_model.encoder.layers.36.attn.qkv",
+      "vision_model.encoder.layers.36.attn.proj",
+      "vision_model.encoder.layers.36.mlp.fc1",
+      "vision_model.encoder.layers.36.mlp.fc2",
+      "vision_model.encoder.layers.37.attn.qkv",
+      "vision_model.encoder.layers.37.attn.proj",
+      "vision_model.encoder.layers.37.mlp.fc1",
+      "vision_model.encoder.layers.37.mlp.fc2",
+      "vision_model.encoder.layers.38.attn.qkv",
+      "vision_model.encoder.layers.38.attn.proj",
+      "vision_model.encoder.layers.38.mlp.fc1",
+      "vision_model.encoder.layers.38.mlp.fc2",
+      "vision_model.encoder.layers.39.attn.qkv",
+      "vision_model.encoder.layers.39.attn.proj",
+      "vision_model.encoder.layers.39.mlp.fc1",
+      "vision_model.encoder.layers.39.mlp.fc2",
+      "vision_model.encoder.layers.40.attn.qkv",
+      "vision_model.encoder.layers.40.attn.proj",
+      "vision_model.encoder.layers.40.mlp.fc1",
+      "vision_model.encoder.layers.40.mlp.fc2",
+      "vision_model.encoder.layers.41.attn.qkv",
+      "vision_model.encoder.layers.41.attn.proj",
+      "vision_model.encoder.layers.41.mlp.fc1",
+      "vision_model.encoder.layers.41.mlp.fc2",
+      "vision_model.encoder.layers.42.attn.qkv",
+      "vision_model.encoder.layers.42.attn.proj",
+      "vision_model.encoder.layers.42.mlp.fc1",
+      "vision_model.encoder.layers.42.mlp.fc2",
+      "vision_model.encoder.layers.43.attn.qkv",
+      "vision_model.encoder.layers.43.attn.proj",
+      "vision_model.encoder.layers.43.mlp.fc1",
+      "vision_model.encoder.layers.43.mlp.fc2",
+      "vision_model.encoder.layers.44.attn.qkv",
+      "vision_model.encoder.layers.44.attn.proj",
+      "vision_model.encoder.layers.44.mlp.fc1",
+      "vision_model.encoder.layers.44.mlp.fc2",
+      "language_model.lm_head",
+      "mlp1.1",
+      "mlp1.3"
+    ],
+    "kv_cache_scheme": null,
+    "quant_method": "compressed-tensors",
+    "quantization_status": "compressed"
+  },
+  "select_layer": -1,
+  "template": "skywork-r1v-chat",
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": null,
+  "use_backbone_lora": 0,
+  "use_llm_lora": 0,
+  "use_thumbnail": true,
+  "vision_config": {
+    "architectures": [
+      "InternVisionModel"
+    ],
+    "attention_dropout": 0.0,
+    "drop_path_rate": 0.1,
+    "dropout": 0.0,
+    "hidden_act": "gelu",
+    "hidden_size": 3200,
+    "image_size": 448,
+    "initializer_factor": 0.1,
+    "initializer_range": 1e-10,
+    "intermediate_size": 12800,
+    "layer_norm_eps": 1e-06,
+    "model_type": "",
+    "norm_type": "rms_norm",
+    "num_attention_heads": 25,
+    "num_channels": 3,
+    "num_hidden_layers": 45,
+    "patch_size": 14,
+    "qk_normalization": true,
+    "qkv_bias": false,
+    "torch_dtype": "bfloat16",
+    "use_bfloat16": true,
+    "use_flash_attn": true
+  }
+}

configuration_skywork_chat.py

@@ -0,0 +1,91 @@
+import copy
+
+from transformers import AutoConfig, LlamaConfig
+from transformers.configuration_utils import PretrainedConfig
+from transformers.utils import logging
+
+from .configuration_skywork_vit import SkyworkVisionConfig
+from .configuration_skywork_lm2 import SkyworkLM2Config
+from transformers import Qwen2Config, Qwen2ForCausalLM
+
+logger = logging.get_logger(__name__)
+
+
+class SkyworkChatConfig(PretrainedConfig):
+    model_type = 'skywork_chat'
+    is_composition = True
+
+    def __init__(
+            self,
+            vision_config=None,
+            llm_config=None,
+            use_backbone_lora=0,
+            use_llm_lora=0,
+            select_layer=-1,
+            force_image_size=None,
+            downsample_ratio=0.5,
+            template=None,
+            dynamic_image_size=False,
+            use_thumbnail=False,
+            ps_version='v1',
+            min_dynamic_patch=1,
+            max_dynamic_patch=6,
+            **kwargs):
+        super().__init__(**kwargs)
+        if vision_config is None:
+            vision_config = {'architectures': ['SkyworkVisionModel']}
+            logger.info('vision_config is None. Initializing the SkyworkVisionConfig with default values.')
+
+        if llm_config is None:
+            llm_config = {'architectures': ['Qwen2ForCausalLM']}
+            logger.info('llm_config is None. Initializing the LlamaConfig config with default values (`LlamaConfig`).')
+
+        self.vision_config = SkyworkVisionConfig(**vision_config)
+        if llm_config.get('architectures')[0] == 'LlamaForCausalLM':
+            self.llm_config = LlamaConfig(**llm_config)
+        elif llm_config.get('architectures')[0] == 'Qwen2ForCausalLM':
+            self.llm_config = Qwen2Config(**llm_config)
+        else:
+            raise ValueError('Unsupported architecture: {}'.format(llm_config.get('architectures')[0]))
+
+
+        self.use_backbone_lora = use_backbone_lora
+        self.use_llm_lora = use_llm_lora
+        self.select_layer = select_layer
+        self.force_image_size = force_image_size
+        self.downsample_ratio = downsample_ratio
+        self.template = template
+        self.dynamic_image_size = dynamic_image_size
+        self.use_thumbnail = use_thumbnail
+        self.ps_version = ps_version  # pixel shuffle version
+        self.min_dynamic_patch = min_dynamic_patch
+        self.max_dynamic_patch = max_dynamic_patch
+
+        logger.info(f'vision_select_layer: {self.select_layer}')
+        logger.info(f'ps_version: {self.ps_version}')
+        logger.info(f'min_dynamic_patch: {self.min_dynamic_patch}')
+        logger.info(f'max_dynamic_patch: {self.max_dynamic_patch}')
+
+    def to_dict(self):
+        """
+        Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`].
+        Returns:
+            `Dict[str, any]`: Dictionary of all the attributes that make up this configuration instance,
+        """
+        output = copy.deepcopy(self.__dict__)
+        output['vision_config'] = self.vision_config.to_dict()
+        output['llm_config'] = self.llm_config.to_dict()
+        output['model_type'] = self.__class__.model_type
+        output['use_backbone_lora'] = self.use_backbone_lora
+        output['use_llm_lora'] = self.use_llm_lora
+        output['select_layer'] = self.select_layer
+        output['force_image_size'] = self.force_image_size
+        output['downsample_ratio'] = self.downsample_ratio
+        output['template'] = self.template
+        output['dynamic_image_size'] = self.dynamic_image_size
+        output['use_thumbnail'] = self.use_thumbnail
+        output['ps_version'] = self.ps_version
+        output['min_dynamic_patch'] = self.min_dynamic_patch
+        output['max_dynamic_patch'] = self.max_dynamic_patch
+
+        return output

generation_config.json

@@ -0,0 +1,4 @@
+{
+  "_from_model_config": true,
+  "transformers_version": "4.52.4"
+}

model-00001-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f5eb4f056eedb2060883a1fd8c6c98a94241370888ee26485a477b4beb5f011
+size 4988569440

model-00002-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37c5d0d80a6bc831595b1b15a8c2200dcab17247bdde8abd634b4bf4a455257b
+size 4937253584

model-00003-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9077f7ab6996897805edf72ff0cc599dd57906a37902421038951189c9d49cb
+size 4997644696

model-00004-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1807f64a9d47085f8191976aafee778c367266a7f52ea1995f9dd85451a4af77
+size 4877704976

model-00005-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8779cc0d2aacab68944be6178173ac739458d00b892645dedc7d2473b562aaad
+size 4877705072

model-00006-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:523708a187a2b5d13d98f4718cd542ed03033217ee2bf9fef5168e9dd33a58ec
+size 4877705072

model-00007-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10db9a2a255a2b21084722111486e4c4cd7857822a3ba3613aaad80e016ea317
+size 4877705072

model-00008-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30ef9ecc7f802559c0f85ea2df195edaf5cfada57fa1d77b698d0775d9d4863c
+size 4877705072

model-00009-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0644533ba86abada73df8c3635a3112787b7d3458a0f7a5cd6cb0d6de8095e0c
+size 4531533888

model-00010-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e87b670c1730f4876b97ae2fc99c0212b2a555e00f3a54ad8298ef6192f98fe7
+size 1736714912

modeling_skywork_chat.py

@@ -0,0 +1,357 @@
+import warnings
+import re
+from typing import List, Optional, Tuple, Union
+
+import torch.utils.checkpoint
+import transformers
+from torch import nn
+from torch.nn import CrossEntropyLoss
+from transformers import (AutoModel, GenerationConfig, LlamaForCausalLM,
+                          LlamaTokenizer)
+from transformers.modeling_outputs import CausalLMOutputWithPast
+from transformers.modeling_utils import PreTrainedModel
+from transformers.utils import ModelOutput, logging
+
+from .configuration_skywork_chat import SkyworkChatConfig
+from .conversation import get_conv_template
+from .modeling_skywork_vit import SkyworkVisionModel, has_flash_attn
+from .modeling_skywork_lm2 import SkyworkLM2ForCausalLM
+
+from transformers import Qwen2Config, Qwen2ForCausalLM
+
+logger = logging.get_logger(__name__)
+
+
+def version_cmp(v1, v2, op='eq'):
+    import operator
+
+    from packaging import version
+    op_func = getattr(operator, op)
+    return op_func(version.parse(v1), version.parse(v2))
+
+
+class SkyworkChatModel(PreTrainedModel):
+    config_class = SkyworkChatConfig
+    main_input_name = 'pixel_values'
+    base_model_prefix = 'language_model'
+    _supports_flash_attn_2 = True
+    _no_split_modules = ['SkyworkVisionModel', 'LlamaDecoderLayer', 'SkyworkLM2DecoderLayer']
+
+    def __init__(self, config: SkyworkChatConfig, vision_model=None, language_model=None, use_flash_attn=True):
+        super().__init__(config)
+
+        assert version_cmp(transformers.__version__, '4.36.2', 'ge')
+        image_size = config.force_image_size or config.vision_config.image_size
+        patch_size = config.vision_config.patch_size
+        self.patch_size = patch_size
+        self.select_layer = config.select_layer
+        self.template = config.template
+        self.num_image_token = int((image_size // patch_size) ** 2 * (config.downsample_ratio ** 2))
+        self.downsample_ratio = config.downsample_ratio
+        self.ps_version = config.ps_version
+        use_flash_attn = use_flash_attn if has_flash_attn else False
+        config.vision_config.use_flash_attn = True if use_flash_attn else False
+        config.llm_config.attn_implementation = 'flash_attention_2' if use_flash_attn else 'eager'
+
+        logger.info(f'num_image_token: {self.num_image_token}')
+        logger.info(f'ps_version: {self.ps_version}')
+        if vision_model is not None:
+            self.vision_model = vision_model
+        else:
+            self.vision_model = SkyworkVisionModel(config.vision_config)
+        if language_model is not None:
+            self.language_model = language_model
+        else:
+            if config.llm_config.architectures[0] == 'LlamaForCausalLM':
+                self.language_model = LlamaForCausalLM(config.llm_config)
+            elif config.llm_config.architectures[0] == 'SkyworkLM2ForCausalLM':
+                self.language_model = SkyworkLM2ForCausalLM(config.llm_config)
+            elif config.llm_config.architectures[0] == 'Qwen2ForCausalLM':
+                self.language_model = Qwen2ForCausalLM(config.llm_config)
+            else:
+                raise NotImplementedError(f'{config.llm_config.architectures[0]} is not implemented.')
+
+        vit_hidden_size = config.vision_config.hidden_size
+        llm_hidden_size = config.llm_config.hidden_size
+
+        self.mlp1 = nn.Sequential(
+            nn.LayerNorm(vit_hidden_size * int(1 / self.downsample_ratio) ** 2),
+            nn.Linear(vit_hidden_size * int(1 / self.downsample_ratio) ** 2, llm_hidden_size),
+            nn.GELU(),
+            nn.Linear(llm_hidden_size, llm_hidden_size)
+        )
+
+        self.img_context_token_id = None
+        self.conv_template = get_conv_template(self.template)
+        self.system_message = self.conv_template.system_message
+
+    def forward(
+            self,
+            pixel_values: torch.FloatTensor,
+            input_ids: torch.LongTensor = None,
+            attention_mask: Optional[torch.Tensor] = None,
+            position_ids: Optional[torch.LongTensor] = None,
+            image_flags: Optional[torch.LongTensor] = None,
+            past_key_values: Optional[List[torch.FloatTensor]] = None,
+            labels: Optional[torch.LongTensor] = None,
+            use_cache: Optional[bool] = None,
+            output_attentions: Optional[bool] = None,
+            output_hidden_states: Optional[bool] = None,
+            return_dict: Optional[bool] = None,
+    ) -> Union[Tuple, CausalLMOutputWithPast]:
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        image_flags = image_flags.squeeze(-1)
+        input_embeds = self.language_model.get_input_embeddings()(input_ids).clone()
+
+        vit_embeds = self.extract_feature(pixel_values)
+        vit_embeds = vit_embeds[image_flags == 1]
+        vit_batch_size = pixel_values.shape[0]
+
+        B, N, C = input_embeds.shape
+        input_embeds = input_embeds.reshape(B * N, C)
+
+        if torch.distributed.get_rank() == 0:
+            print(f'dynamic ViT batch size: {vit_batch_size}, images per sample: {vit_batch_size / B}, dynamic token length: {N}')
+
+        input_ids = input_ids.reshape(B * N)
+        selected = (input_ids == self.img_context_token_id)
+        try:
+            input_embeds[selected] = input_embeds[selected] * 0.0 + vit_embeds.reshape(-1, C)
+        except Exception as e:
+            vit_embeds = vit_embeds.reshape(-1, C)
+            print(f'warning: {e}, input_embeds[selected].shape={input_embeds[selected].shape}, '
+                  f'vit_embeds.shape={vit_embeds.shape}')
+            n_token = selected.sum()
+            input_embeds[selected] = input_embeds[selected] * 0.0 + vit_embeds[:n_token]
+
+        input_embeds = input_embeds.reshape(B, N, C)
+
+        outputs = self.language_model(
+            inputs_embeds=input_embeds,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            use_cache=use_cache,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+        logits = outputs.logits
+
+        loss = None
+        if labels is not None:
+            # Shift so that tokens < n predict n
+            shift_logits = logits[..., :-1, :].contiguous()
+            shift_labels = labels[..., 1:].contiguous()
+            # Flatten the tokens
+            loss_fct = CrossEntropyLoss()
+            shift_logits = shift_logits.view(-1, self.language_model.config.vocab_size)
+            shift_labels = shift_labels.view(-1)
+            # Enable model parallelism
+            shift_labels = shift_labels.to(shift_logits.device)
+            loss = loss_fct(shift_logits, shift_labels)
+
+        if not return_dict:
+            output = (logits,) + outputs[1:]
+            return (loss,) + output if loss is not None else output
+
+        return CausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+        )
+
+    def pixel_shuffle(self, x, scale_factor=0.5):
+        n, w, h, c = x.size()
+        # N, W, H, C --> N, W, H * scale, C // scale
+        x = x.view(n, w, int(h * scale_factor), int(c / scale_factor))
+        # N, W, H * scale, C // scale --> N, H * scale, W, C // scale
+        x = x.permute(0, 2, 1, 3).contiguous()
+        # N, H * scale, W, C // scale --> N, H * scale, W * scale, C // (scale ** 2)
+        x = x.view(n, int(h * scale_factor), int(w * scale_factor),
+                   int(c / (scale_factor * scale_factor)))
+        if self.ps_version == 'v1':
+            warnings.warn("In ps_version 'v1', the height and width have not been swapped back, "
+                          'which results in a transposed image.')
+        else:
+            x = x.permute(0, 2, 1, 3).contiguous()
+        return x
+
+    def extract_feature(self, pixel_values):
+        if self.select_layer == -1:
+            vit_embeds = self.vision_model(
+                pixel_values=pixel_values,
+                output_hidden_states=False,
+                return_dict=True).last_hidden_state
+        else:
+            vit_embeds = self.vision_model(
+                pixel_values=pixel_values,
+                output_hidden_states=True,
+                return_dict=True).hidden_states[self.select_layer]
+        vit_embeds = vit_embeds[:, 1:, :]
+
+        h = w = int(vit_embeds.shape[1] ** 0.5)
+        vit_embeds = vit_embeds.reshape(vit_embeds.shape[0], h, w, -1)
+        vit_embeds = self.pixel_shuffle(vit_embeds, scale_factor=self.downsample_ratio)
+        vit_embeds = vit_embeds.reshape(vit_embeds.shape[0], -1, vit_embeds.shape[-1])
+        vit_embeds = self.mlp1(vit_embeds)
+        return vit_embeds
+
+    def batch_chat(self, tokenizer, pixel_values, questions, generation_config, num_patches_list=None,
+                   history=None, return_history=False, IMG_START_TOKEN='<img>', IMG_END_TOKEN='</img>',
+                   IMG_CONTEXT_TOKEN='<IMG_CONTEXT>', verbose=False, image_counts=None):
+        if history is not None or return_history:
+            print('Now multi-turn chat is not supported in batch_chat.')
+            raise NotImplementedError
+
+        if image_counts is not None:
+            num_patches_list = image_counts
+            print('Warning: `image_counts` is deprecated. Please use `num_patches_list` instead.')
+
+        img_context_token_id = tokenizer.convert_tokens_to_ids(IMG_CONTEXT_TOKEN)
+        self.img_context_token_id = img_context_token_id
+      
+
+        if verbose and pixel_values is not None:
+            image_bs = pixel_values.shape[0]
+            print(f'dynamic ViT batch size: {image_bs}')
+
+        queries = []
+        for idx, num_patches in enumerate(num_patches_list):
+            question = questions[idx]
+            if pixel_values is not None and '<image>' not in question:
+                question = '<image>\n' + question
+            template = get_conv_template(self.template)
+            template.system_message = self.system_message
+            template.append_message(template.roles[0], question)
+            template.append_message(template.roles[1], None)
+            query = template.get_prompt()
+
+            image_tokens = IMG_START_TOKEN + IMG_CONTEXT_TOKEN * self.num_image_token * num_patches + IMG_END_TOKEN
+            query = query.replace('<image>', image_tokens, 1)
+            queries.append(query)
+
+        tokenizer.padding_side = 'left'
+        model_inputs = tokenizer(queries, return_tensors='pt', padding=True)
+        input_ids = model_inputs['input_ids'].to(self.device)
+        attention_mask = model_inputs['attention_mask'].to(self.device)
+        eos_token_id = tokenizer.convert_tokens_to_ids(template.sep.strip())
+        generation_config['eos_token_id'] = eos_token_id
+        generation_output = self.generate(
+            pixel_values=pixel_values,
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            **generation_config
+        )
+        responses = tokenizer.batch_decode(generation_output, skip_special_tokens=True)
+        responses = [response.split(template.sep.strip())[0].strip() for response in responses]
+        return responses
+
+    def chat(self, tokenizer, pixel_values, question, generation_config, history=None, return_history=False,
+             num_patches_list=None, IMG_START_TOKEN='<img>', IMG_END_TOKEN='</img>', IMG_CONTEXT_TOKEN='<IMG_CONTEXT>',
+             verbose=False, mode="think"):
+
+        if history is None and pixel_values is not None and '<image>' not in question:
+            question = '<image>\n' + question
+
+        if num_patches_list is None:
+            num_patches_list = [pixel_values.shape[0]] if pixel_values is not None else []
+        assert pixel_values is None or len(pixel_values) == sum(num_patches_list)
+
+        img_context_token_id = tokenizer.convert_tokens_to_ids(IMG_CONTEXT_TOKEN)
+        self.img_context_token_id = img_context_token_id
+
+        template = get_conv_template(self.template)
+        template.system_message = self.system_message
+        eos_token_id = tokenizer.convert_tokens_to_ids(template.sep.strip())
+       
+
+        history = [] if history is None else history
+        for (old_question, old_answer) in history:
+            template.append_message(template.roles[0], old_question)
+            template.append_message(template.roles[1], old_answer)
+        template.append_message(template.roles[0], question)
+        template.append_message(template.roles[1], None)
+        query = template.get_prompt()
+        if mode != "think":
+            query = re.sub(r'\n<think>', '', query, count=1)
+   
+
+        if verbose and pixel_values is not None:
+            image_bs = pixel_values.shape[0]
+            print(f'dynamic ViT batch size: {image_bs}')
+
+        for num_patches in num_patches_list:
+            image_tokens = IMG_START_TOKEN + IMG_CONTEXT_TOKEN * self.num_image_token * num_patches + IMG_END_TOKEN
+            query = query.replace('<image>', image_tokens, 1)
+
+
+        model_inputs = tokenizer(query, return_tensors='pt')
+        input_ids = model_inputs['input_ids'].to(self.device)
+        attention_mask = model_inputs['attention_mask'].to(self.device)
+        generation_config['eos_token_id'] = eos_token_id
+        generation_output = self.generate(
+            pixel_values=pixel_values,
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            **generation_config
+        )
+        response = tokenizer.batch_decode(generation_output, skip_special_tokens=True)[0]
+        response = response.split(template.sep.strip())[0].strip()
+        history.append((question, response))
+        
+        if return_history:
+            return response, history
+        else:
+            query_to_print = query.replace(IMG_CONTEXT_TOKEN, '')
+            query_to_print = query_to_print.replace(f'{IMG_START_TOKEN}{IMG_END_TOKEN}', '<image>')
+            if verbose:
+                print(query_to_print, response)
+            return response
+
+    @torch.no_grad()
+    def generate(
+            self,
+            pixel_values: Optional[torch.FloatTensor] = None,
+            input_ids: Optional[torch.FloatTensor] = None,
+            attention_mask: Optional[torch.LongTensor] = None,
+            visual_features: Optional[torch.FloatTensor] = None,
+            generation_config: Optional[GenerationConfig] = None,
+            output_hidden_states: Optional[bool] = None,
+            **generate_kwargs,
+    ) -> torch.LongTensor:
+
+        assert self.img_context_token_id is not None
+        if pixel_values is not None:
+            if visual_features is not None:
+                vit_embeds = visual_features
+            else:
+                vit_embeds = self.extract_feature(pixel_values)
+            input_embeds = self.language_model.get_input_embeddings()(input_ids)
+            B, N, C = input_embeds.shape
+            input_embeds = input_embeds.reshape(B * N, C)
+
+            input_ids = input_ids.reshape(B * N)
+            selected = (input_ids == self.img_context_token_id)
+  
+            assert selected.sum() != 0
+            input_embeds[selected] = vit_embeds.reshape(-1, C).to(input_embeds.device)
+
+            input_embeds = input_embeds.reshape(B, N, C)
+        else:
+            input_embeds = self.language_model.get_input_embeddings()(input_ids)
+
+
+        outputs = self.language_model.generate(
+            inputs_embeds=input_embeds,
+            attention_mask=attention_mask,
+            generation_config=generation_config,
+            output_hidden_states=output_hidden_states,
+            use_cache=True,
+            **generate_kwargs,
+        )
+
+        return outputs

recipe.yaml

@@ -0,0 +1,6 @@
+default_stage:
+  default_modifiers:
+    QuantizationModifier:
+      targets: [Linear]
+      ignore: ['re:.*lm_head', 're:.*vision.*', 're:mlp1.*']
+      scheme: FP8_DYNAMIC

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f9ba4b4a6625b5047a1356f6081b641c3e4e6a4a198facbd4bef217747d1685
+size 11423548

tokenizer_config.json

@@ -0,0 +1,280 @@
+{
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<img>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151666": {
+      "content": "</img>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151667": {
+      "content": "<IMG_CONTEXT>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151668": {
+      "content": "<quad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151669": {
+      "content": "</quad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151670": {
+      "content": "<ref>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151671": {
+      "content": "</ref>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151672": {
+      "content": "<box>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151673": {
+      "content": "</box>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 16384,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}