Add files using upload-large-folder tool

LLM/Florence-2-base/.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+*bin filter=lfs diff=lfs merge=lfs -text

LLM/Florence-2-base/CODE_OF_CONDUCT.md

@@ -0,0 +1,9 @@
+# Microsoft Open Source Code of Conduct
+
+This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
+
+Resources:
+
+- [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
+- [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
+- Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns

LLM/Florence-2-base/LICENSE

@@ -0,0 +1,21 @@
+    MIT License
+
+    Copyright (c) Microsoft Corporation.
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy
+    of this software and associated documentation files (the "Software"), to deal
+    in the Software without restriction, including without limitation the rights
+    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+    copies of the Software, and to permit persons to whom the Software is
+    furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all
+    copies or substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+    SOFTWARE

LLM/Florence-2-base/README.md

@@ -0,0 +1,259 @@
+---
+license: mit
+license_link: https://huggingface.co/microsoft/Florence-2-base/resolve/main/LICENSE
+pipeline_tag: image-text-to-text
+tags:
+- vision
+---
+
+# Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
+
+## Model Summary
+
+This Hub repository contains a HuggingFace's `transformers` implementation of Florence-2 model from Microsoft.
+
+Florence-2 is an advanced vision foundation model that uses a prompt-based approach to handle a wide range of vision and vision-language tasks.  Florence-2 can interpret simple text prompts to perform tasks like captioning, object detection, and segmentation. It leverages our FLD-5B dataset, containing 5.4 billion annotations across 126 million images, to master multi-task learning. The model's sequence-to-sequence architecture enables it to excel in both zero-shot and fine-tuned settings, proving to be a competitive vision foundation model. 
+
+Resources and Technical Documentation:
++ [Florence-2 technical report](https://arxiv.org/abs/2311.06242). 
++ [Jupyter Notebook for inference and visualization of Florence-2-large model](https://huggingface.co/microsoft/Florence-2-large/blob/main/sample_inference.ipynb)
+
+| Model   | Model size | Model Description | 
+| ------- | ------------- |   ------------- |  
+| Florence-2-base[[HF]](https://huggingface.co/microsoft/Florence-2-base) | 0.23B  | Pretrained model with FLD-5B  
+| Florence-2-large[[HF]](https://huggingface.co/microsoft/Florence-2-large) | 0.77B  | Pretrained model with FLD-5B  
+| Florence-2-base-ft[[HF]](https://huggingface.co/microsoft/Florence-2-base-ft) | 0.23B  | Finetuned model on a colletion of downstream tasks
+| Florence-2-large-ft[[HF]](https://huggingface.co/microsoft/Florence-2-large-ft) | 0.77B | Finetuned model on a colletion of downstream tasks
+ 
+## How to Get Started with the Model
+
+Use the code below to get started with the model.  All models are trained with float16. 
+
+```python
+import requests
+
+from PIL import Image
+from transformers import AutoProcessor, AutoModelForCausalLM 
+
+device = "cuda:0" if torch.cuda.is_available() else "cpu"
+torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
+
+model = AutoModelForCausalLM.from_pretrained("microsoft/Florence-2-base", torch_dtype=torch_dtype, trust_remote_code=True).to(device)
+processor = AutoProcessor.from_pretrained("microsoft/Florence-2-base", trust_remote_code=True)
+
+prompt = "<OD>"
+
+url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
+image = Image.open(requests.get(url, stream=True).raw)
+
+inputs = processor(text=prompt, images=image, return_tensors="pt").to(device, torch_dtype)
+
+generated_ids = model.generate(
+    input_ids=inputs["input_ids"],
+    pixel_values=inputs["pixel_values"],
+    max_new_tokens=1024,
+    do_sample=False,
+    num_beams=3,
+)
+generated_text = processor.batch_decode(generated_ids, skip_special_tokens=False)[0]
+
+parsed_answer = processor.post_process_generation(generated_text, task="<OD>", image_size=(image.width, image.height))
+
+print(parsed_answer)
+
+```
+
+
+## Tasks
+
+This model is capable of performing different tasks through changing the prompts.
+
+First, let's define a function to run a prompt.
+
+<details>
+<summary> Click to expand </summary>
+
+```python
+import requests
+
+from PIL import Image
+from transformers import AutoProcessor, AutoModelForCausalLM 
+
+device = "cuda:0" if torch.cuda.is_available() else "cpu"
+torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
+
+model = AutoModelForCausalLM.from_pretrained("microsoft/Florence-2-base", torch_dtype=torch_dtype, trust_remote_code=True).to(device)
+processor = AutoProcessor.from_pretrained("microsoft/Florence-2-base", trust_remote_code=True)
+
+url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
+image = Image.open(requests.get(url, stream=True).raw)
+
+def run_example(task_prompt, text_input=None):
+    if text_input is None:
+        prompt = task_prompt
+    else:
+        prompt = task_prompt + text_input
+    inputs = processor(text=prompt, images=image, return_tensors="pt").to(device, torch_dtype)
+    generated_ids = model.generate(
+      input_ids=inputs["input_ids"],
+      pixel_values=inputs["pixel_values"],
+      max_new_tokens=1024,
+      num_beams=3
+    )
+    generated_text = processor.batch_decode(generated_ids, skip_special_tokens=False)[0]
+
+    parsed_answer = processor.post_process_generation(generated_text, task=task_prompt, image_size=(image.width, image.height))
+
+    print(parsed_answer)
+```
+</details>
+
+Here are the tasks `Florence-2` could perform:
+
+<details>
+<summary> Click to expand </summary>
+
+
+
+### Caption
+```python
+prompt = "<CAPTION>"
+run_example(prompt)
+```
+
+### Detailed Caption
+```python
+prompt = "<DETAILED_CAPTION>"
+run_example(prompt)
+```
+
+### More Detailed Caption
+```python
+prompt = "<MORE_DETAILED_CAPTION>"
+run_example(prompt)
+```
+
+### Caption to Phrase Grounding 
+caption to phrase grounding task requires additional text input, i.e. caption. 
+
+Caption to phrase grounding results format: 
+{'\<CAPTION_TO_PHRASE_GROUNDING>': {'bboxes': [[x1, y1, x2, y2], ...], 'labels': ['', '', ...]}}
+```python
+task_prompt = "<CAPTION_TO_PHRASE_GROUNDING>"
+results = run_example(task_prompt, text_input="A green car parked in front of a yellow building.")
+```
+
+### Object Detection
+
+OD results format: 
+{'\<OD>': {'bboxes': [[x1, y1, x2, y2], ...], 
+'labels': ['label1', 'label2', ...]} }
+
+```python
+prompt = "<OD>"
+run_example(prompt)
+```
+
+### Dense Region Caption
+Dense region caption results format: 
+{'\<DENSE_REGION_CAPTION>' : {'bboxes': [[x1, y1, x2, y2], ...], 
+'labels': ['label1', 'label2', ...]} }
+```python
+prompt = "<DENSE_REGION_CAPTION>"
+run_example(prompt)
+```
+
+### Region proposal
+Dense region caption results format: 
+{'\<REGION_PROPOSAL>': {'bboxes': [[x1, y1, x2, y2], ...], 
+'labels': ['', '', ...]}}
+```python
+prompt = "<REGION_PROPOSAL>"
+run_example(prompt)
+```
+
+### OCR 
+
+```python
+prompt = "<OCR>"
+run_example(prompt)
+```
+
+### OCR with Region
+OCR with region output format:
+{'\<OCR_WITH_REGION>': {'quad_boxes': [[x1, y1, x2, y2, x3, y3, x4, y4], ...], 'labels': ['text1', ...]}}
+```python
+prompt = "<OCR_WITH_REGION>"
+run_example(prompt)
+```
+
+for More detailed examples, please refer to [notebook](https://huggingface.co/microsoft/Florence-2-large/blob/main/sample_inference.ipynb)
+</details>
+
+# Benchmarks
+
+## Florence-2 Zero-shot performance
+  
+The following table presents the zero-shot performance of generalist vision foundation models on image captioning and object detection evaluation tasks. These models have not been exposed to the training data of the evaluation tasks during their training phase.  
+  
+| Method | #params | COCO Cap. test CIDEr | NoCaps val CIDEr | TextCaps val CIDEr | COCO Det. val2017 mAP |  
+|--------|---------|----------------------|------------------|--------------------|-----------------------|
+| Flamingo | 80B | 84.3 | - | - | - | 
+| Florence-2-base| 0.23B | 133.0 | 118.7 | 70.1 | 34.7 | 
+| Florence-2-large| 0.77B | 135.6 | 120.8 | 72.8 | 37.5 |
+
+  
+The following table continues the comparison with performance on other vision-language evaluation tasks.  
+  
+| Method | Flickr30k test R@1 | Refcoco val Accuracy | Refcoco test-A Accuracy | Refcoco test-B Accuracy | Refcoco+ val Accuracy | Refcoco+ test-A Accuracy | Refcoco+ test-B Accuracy | Refcocog val Accuracy | Refcocog test Accuracy | Refcoco RES val mIoU |  
+|--------|----------------------|----------------------|-------------------------|-------------------------|-----------------------|--------------------------|--------------------------|-----------------------|------------------------|----------------------|  
+| Kosmos-2 | 78.7 | 52.3 | 57.4 | 47.3 | 45.5 | 50.7 | 42.2 | 60.6 | 61.7 | - |  
+| Florence-2-base | 83.6 | 53.9 | 58.4 | 49.7 | 51.5 | 56.4 | 47.9 | 66.3 | 65.1 | 34.6 |  
+| Florence-2-large | 84.4 | 56.3 | 61.6 | 51.4 | 53.6 | 57.9 | 49.9 | 68.0 | 67.0 | 35.8 |  
+
+
+
+## Florence-2 finetuned performance 
+
+We finetune Florence-2 models with a collection of downstream tasks, resulting two generalist models *Florence-2-base-ft* and *Florence-2-large-ft* that can conduct a wide range of downstream tasks. 
+  
+The table below compares the performance of specialist and generalist models on various captioning and Visual Question Answering (VQA) tasks. Specialist models are fine-tuned specifically for each task, whereas generalist models are fine-tuned in a task-agnostic manner across all tasks. The symbol "▲" indicates the usage of external OCR as input.  
+  
+| Method         | # Params | COCO Caption Karpathy test CIDEr | NoCaps val CIDEr | TextCaps val CIDEr | VQAv2 test-dev Acc | TextVQA test-dev Acc | VizWiz VQA test-dev Acc |  
+|----------------|----------|-----------------------------------|------------------|--------------------|--------------------|----------------------|-------------------------|  
+| **Specialist Models**   |          |                                   |                  |                    |                    |                      |                         |  
+| CoCa           | 2.1B     | 143.6                              | 122.4            | -                  | 82.3               | -                    | -                       |  
+| BLIP-2         | 7.8B     | 144.5                              | 121.6            | -                  | 82.2               | -                    | -                       |  
+| GIT2           | 5.1B     | 145.0                              | 126.9            | 148.6              | 81.7               | 67.3                 | 71.0                    |  
+| Flamingo       | 80B      | 138.1                              | -                | -                  | 82.0               | 54.1                 | 65.7                    |  
+| PaLI           | 17B      | 149.1                              | 127.0            | 160.0▲             | 84.3               | 58.8 / 73.1▲         | 71.6 / 74.4▲            |  
+| PaLI-X         | 55B      | 149.2                              | 126.3            | 147.0 / 163.7▲     | 86.0               | 71.4 / 80.8▲         | 70.9 / 74.6▲            |  
+| **Generalist Models**   |          |                                   |                  |                    |                    |                      |                         |  
+| Unified-IO     | 2.9B     | -                                  | 100.0            | -                  | 77.9               | -                    | 57.4                    |  
+| Florence-2-base-ft | 0.23B  | 140.0                              | 116.7            | 143.9              | 79.7               | 63.6                 | 63.6                    |  
+| Florence-2-large-ft | 0.77B  | 143.3                              | 124.9            | 151.1              | 81.7               | 73.5                 | 72.6                    |  
+  
+  
+| Method               | # Params | COCO Det. val2017 mAP | Flickr30k test R@1 | RefCOCO val Accuracy | RefCOCO test-A Accuracy | RefCOCO test-B Accuracy | RefCOCO+ val Accuracy | RefCOCO+ test-A Accuracy | RefCOCO+ test-B Accuracy | RefCOCOg val Accuracy | RefCOCOg test Accuracy | RefCOCO RES val mIoU |  
+|----------------------|----------|-----------------------|--------------------|----------------------|-------------------------|-------------------------|------------------------|---------------------------|---------------------------|------------------------|-----------------------|------------------------|  
+| **Specialist Models** |          |                       |                    |                      |                         |                         |                        |                           |                           |                        |                       |                        |  
+| SeqTR                | -        | -                     | -                  | 83.7                 | 86.5                    | 81.2                    | 71.5                   | 76.3                      | 64.9                      | 74.9                   | 74.2                  | -                      |  
+| PolyFormer           | -        | -                     | -                  | 90.4                 | 92.9                    | 87.2                    | 85.0                   | 89.8                      | 78.0                      | 85.8                   | 85.9                  | 76.9                   |  
+| UNINEXT              | 0.74B    | 60.6                  | -                  | 92.6                 | 94.3                    | 91.5                    | 85.2                   | 89.6                      | 79.8                      | 88.7                   | 89.4                  | -                      |  
+| Ferret               | 13B      | -                     | -                  | 89.5                 | 92.4                    | 84.4                    | 82.8                   | 88.1                      | 75.2                      | 85.8                   | 86.3                  | -                      |  
+| **Generalist Models** |          |                       |                    |                      |                         |                         |                        |                           |                           |                        |                       |                        |  
+| UniTAB               | -        | -                     | -                  | 88.6                 | 91.1                    | 83.8                    | 81.0                   | 85.4                      | 71.6                      | 84.6                   | 84.7                  | -                      |  
+| Florence-2-base-ft | 0.23B    | 41.4                  | 84.0                | 92.6                 | 94.8                    | 91.5                   | 86.8                   | 91.7                      | 82.2                      | 89.8                   | 82.2                  | 78.0                  |  
+| Florence-2-large-ft| 0.77B    | 43.4                  | 85.2               | 93.4                 | 95.3                    | 92.0                    | 88.3                   | 92.9                      | 83.6                      | 91.2                   | 91.7                  | 80.5                   |  
+  
+
+## BibTex and citation info
+
+```
+@article{xiao2023florence,
+  title={Florence-2: Advancing a unified representation for a variety of vision tasks},
+  author={Xiao, Bin and Wu, Haiping and Xu, Weijian and Dai, Xiyang and Hu, Houdong and Lu, Yumao and Zeng, Michael and Liu, Ce and Yuan, Lu},
+  journal={arXiv preprint arXiv:2311.06242},
+  year={2023}
+}
+```

LLM/Florence-2-base/SECURITY.md

@@ -0,0 +1,41 @@
+<!-- BEGIN MICROSOFT SECURITY.MD V0.0.9 BLOCK -->
+
+## Security
+
+Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet) and [Xamarin](https://github.com/xamarin).
+
+If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/security.md/definition), please report it to us as described below.
+
+## Reporting Security Issues
+
+**Please do not report security vulnerabilities through public GitHub issues.**
+
+Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/security.md/msrc/create-report).
+
+If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com).  If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/security.md/msrc/pgp).
+
+You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). 
+
+Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
+
+  * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
+  * Full paths of source file(s) related to the manifestation of the issue
+  * The location of the affected source code (tag/branch/commit or direct URL)
+  * Any special configuration required to reproduce the issue
+  * Step-by-step instructions to reproduce the issue
+  * Proof-of-concept or exploit code (if possible)
+  * Impact of the issue, including how an attacker might exploit the issue
+
+This information will help us triage your report more quickly.
+
+If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/security.md/msrc/bounty) page for more details about our active programs.
+
+## Preferred Languages
+
+We prefer all communications to be in English.
+
+## Policy
+
+Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/security.md/cvd).
+
+<!-- END MICROSOFT SECURITY.MD BLOCK -->

LLM/Florence-2-base/SUPPORT.md

@@ -0,0 +1,25 @@
+# TODO: The maintainer of this repo has not yet edited this file
+
+**REPO OWNER**: Do you want Customer Service & Support (CSS) support for this product/project?
+
+- **No CSS support:** Fill out this template with information about how to file issues and get help.
+- **Yes CSS support:** Fill out an intake form at [aka.ms/onboardsupport](https://aka.ms/onboardsupport). CSS will work with/help you to determine next steps.
+- **Not sure?** Fill out an intake as though the answer were "Yes". CSS will help you decide.
+
+*Then remove this first heading from this SUPPORT.MD file before publishing your repo.*
+
+# Support
+
+## How to file issues and get help  
+
+This project uses GitHub Issues to track bugs and feature requests. Please search the existing 
+issues before filing new issues to avoid duplicates.  For new issues, file your bug or 
+feature request as a new Issue.
+
+For help and questions about using this project, please **REPO MAINTAINER: INSERT INSTRUCTIONS HERE 
+FOR HOW TO ENGAGE REPO OWNERS OR COMMUNITY FOR HELP. COULD BE A STACK OVERFLOW TAG OR OTHER
+CHANNEL. WHERE WILL YOU HELP PEOPLE?**.
+
+## Microsoft Support Policy  
+
+Support for this **PROJECT or PRODUCT** is limited to the resources listed above.

LLM/Florence-2-base/config.json

@@ -0,0 +1,85 @@
+{
+  "_name_or_path": "florence2",
+  "architectures": [
+    "Florence2ForConditionalGeneration"
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration_florence2.Florence2Config",
+    "AutoModelForCausalLM": "modeling_florence2.Florence2ForConditionalGeneration"
+  },
+  "bos_token_id": 0,
+  "eos_token_id": 2,
+  "ignore_index": -100,
+  "model_type": "florence2",
+  "pad_token_id": 1,
+  "projection_dim": 768,
+  "text_config": {
+      "vocab_size": 51289,
+      "activation_dropout": 0.1,
+      "activation_function": "gelu",
+      "add_bias_logits": false,
+      "add_final_layer_norm": false,
+      "attention_dropout": 0.1,
+      "bos_token_id": 0,
+      "classif_dropout": 0.1,
+      "classifier_dropout": 0.0,
+      "d_model": 768,
+      "decoder_attention_heads": 12,
+      "decoder_ffn_dim": 3072,
+      "decoder_layerdrop": 0.0,
+      "decoder_layers": 6,
+      "decoder_start_token_id": 2,
+      "dropout": 0.1,
+      "early_stopping": true,
+      "encoder_attention_heads": 12,
+      "encoder_ffn_dim": 3072,
+      "encoder_layerdrop": 0.0,
+      "encoder_layers": 6,
+      "eos_token_id": 2,
+      "forced_eos_token_id": 2,
+      "forced_bos_token_id": 0,
+      "gradient_checkpointing": false,
+      "init_std": 0.02,
+      "is_encoder_decoder": true,
+      "label2id": {
+        "LABEL_0": 0,
+        "LABEL_1": 1,
+        "LABEL_2": 2
+      },
+      "max_position_embeddings": 1024,
+      "no_repeat_ngram_size": 3,
+      "normalize_before": false,
+      "num_hidden_layers": 6,
+      "pad_token_id": 1,
+      "scale_embedding": false,
+      "num_beams": 3
+  },
+  "vision_config": {
+    "model_type": "davit",
+    "drop_path_rate": 0.1,  
+    "patch_size": [7, 3, 3, 3],  
+    "patch_stride": [4, 2, 2, 2],  
+    "patch_padding": [3, 1, 1, 1],  
+    "patch_prenorm": [false, true, true, true],  
+    "enable_checkpoint": false,  
+    "dim_embed": [128, 256, 512, 1024],
+    "num_heads": [4, 8, 16, 32],
+    "num_groups": [4, 8, 16, 32],
+    "depths": [1, 1, 9, 1],  
+    "window_size": 12,
+    "projection_dim": 768,
+    "visual_temporal_embedding": {
+        "type": "COSINE",
+        "max_temporal_embeddings": 100
+    },
+    "image_pos_embed": {
+        "type": "learned_abs_2d",
+        "max_pos_embeddings": 50
+    },
+    "image_feature_source": ["spatial_avg_pool", "temporal_avg_pool"]
+  },
+  "vocab_size": 51289,
+  "torch_dtype": "float16",
+  "transformers_version": "4.41.0.dev0",
+  "is_encoder_decoder": true
+}

LLM/Florence-2-base/configuration_florence2.py

@@ -0,0 +1,340 @@
+# coding=utf-8
+# Copyright 2024 Microsoft and the HuggingFace Inc. team. All rights reserved.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import warnings
+""" Florence-2 configuration"""
+
+from typing import Optional
+
+from transformers import AutoConfig
+from transformers.configuration_utils import PretrainedConfig
+from transformers.utils import logging
+
+logger = logging.get_logger(__name__)
+
+class Florence2VisionConfig(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`Florence2VisionModel`]. It is used to instantiate a Florence2VisionModel
+    according to the specified arguments, defining the model architecture. Instantiating a configuration with the 
+    defaults will yield a similar configuration to that of the Florence2VisionModel architecture.
+
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
+    documentation from [`PretrainedConfig`] for more information.
+
+    Args:
+        drop_path_rate (`float`, *optional*, defaults to 0.1):
+            The dropout rate of the drop path layer.
+        patch_size (`List[int]`, *optional*, defaults to [7, 3, 3, 3]):
+            The patch size of the image.
+        patch_stride (`List[int]`, *optional*, defaults to [4, 2, 2, 2]):
+            The patch stride of the image.
+        patch_padding (`List[int]`, *optional*, defaults to [3, 1, 1, 1]):
+            The patch padding of the image.
+        patch_prenorm (`List[bool]`, *optional*, defaults to [false, true, true, true]):
+            Whether to apply layer normalization before the patch embedding layer.
+        enable_checkpoint (`bool`, *optional*, defaults to False):
+            Whether to enable checkpointing.
+        dim_embed (`List[int]`, *optional*, defaults to [256, 512, 1024, 2048]):
+            The dimension of the embedding layer.
+        num_heads (`List[int]`, *optional*, defaults to [8, 16, 32, 64]):
+            The number of attention heads.
+        num_groups (`List[int]`, *optional*, defaults to [8, 16, 32, 64]):
+            The number of groups.
+        depths (`List[int]`, *optional*, defaults to [1, 1, 9, 1]):
+            The depth of the model.
+        window_size (`int`, *optional*, defaults to 12):
+            The window size of the model.
+        projection_dim (`int`, *optional*, defaults to 1024):
+            The dimension of the projection layer.
+        visual_temporal_embedding (`dict`, *optional*):
+            The configuration of the visual temporal embedding.
+        image_pos_embed (`dict`, *optional*):
+            The configuration of the image position embedding.
+        image_feature_source (`List[str]`, *optional*, defaults to ["spatial_avg_pool", "temporal_avg_pool"]):
+            The source of the image feature.
+    Example:
+
+    ```python
+    >>> from transformers import Florence2VisionConfig, Florence2VisionModel
+
+    >>> # Initializing a Florence2 Vision style configuration
+    >>> configuration = Florence2VisionConfig()
+
+    >>> # Initializing a model (with random weights)
+    >>> model = Florence2VisionModel(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+
+    model_type = "davit"
+    keys_to_ignore_at_inference = ["past_key_values"]
+
+    def __init__(
+        self,
+        drop_path_rate=0.1,
+        patch_size=[7, 3, 3, 3],
+        patch_stride=[4, 2, 2, 2],
+        patch_padding=[3, 1, 1, 1],
+        patch_prenorm=[False, True, True, True],
+        enable_checkpoint=False,
+        dim_embed=[256, 512, 1024, 2048],
+        num_heads=[8, 16, 32, 64],
+        num_groups=[8, 16, 32, 64],
+        depths=[1, 1, 9, 1],
+        window_size=12,
+        projection_dim=1024,
+        visual_temporal_embedding=None,
+        image_pos_embed=None,
+        image_feature_source=["spatial_avg_pool", "temporal_avg_pool"],
+        **kwargs,
+    ):
+        self.drop_path_rate = drop_path_rate
+        self.patch_size = patch_size
+        self.patch_stride = patch_stride
+        self.patch_padding = patch_padding
+        self.patch_prenorm = patch_prenorm
+        self.enable_checkpoint = enable_checkpoint
+        self.dim_embed = dim_embed
+        self.num_heads = num_heads
+        self.num_groups = num_groups
+        self.depths = depths
+        self.window_size = window_size
+        self.projection_dim = projection_dim
+        self.visual_temporal_embedding = visual_temporal_embedding
+        self.image_pos_embed = image_pos_embed
+        self.image_feature_source = image_feature_source
+
+        super().__init__(**kwargs)
+
+
+
+class Florence2LanguageConfig(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`Florence2LanguagePreTrainedModel`]. It is used to instantiate a BART
+    model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
+    defaults will yield a similar configuration to that of the BART
+    [facebook/bart-large](https://huggingface.co/facebook/bart-large) architecture.
+
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
+    documentation from [`PretrainedConfig`] for more information.
+
+
+    Args:
+        vocab_size (`int`, *optional*, defaults to 51289):
+            Vocabulary size of the Florence2Language model. Defines the number of different tokens that can be represented by the
+            `inputs_ids` passed when calling [`Florence2LanguageModel`].
+        d_model (`int`, *optional*, defaults to 1024):
+            Dimensionality of the layers and the pooler layer.
+        encoder_layers (`int`, *optional*, defaults to 12):
+            Number of encoder layers.
+        decoder_layers (`int`, *optional*, defaults to 12):
+            Number of decoder layers.
+        encoder_attention_heads (`int`, *optional*, defaults to 16):
+            Number of attention heads for each attention layer in the Transformer encoder.
+        decoder_attention_heads (`int`, *optional*, defaults to 16):
+            Number of attention heads for each attention layer in the Transformer decoder.
+        decoder_ffn_dim (`int`, *optional*, defaults to 4096):
+            Dimensionality of the "intermediate" (often named feed-forward) layer in decoder.
+        encoder_ffn_dim (`int`, *optional*, defaults to 4096):
+            Dimensionality of the "intermediate" (often named feed-forward) layer in decoder.
+        activation_function (`str` or `function`, *optional*, defaults to `"gelu"`):
+            The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,
+            `"relu"`, `"silu"` and `"gelu_new"` are supported.
+        dropout (`float`, *optional*, defaults to 0.1):
+            The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
+        attention_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for the attention probabilities.
+        activation_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for activations inside the fully connected layer.
+        classifier_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for classifier.
+        max_position_embeddings (`int`, *optional*, defaults to 1024):
+            The maximum sequence length that this model might ever be used with. Typically set this to something large
+            just in case (e.g., 512 or 1024 or 2048).
+        init_std (`float`, *optional*, defaults to 0.02):
+            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
+        encoder_layerdrop (`float`, *optional*, defaults to 0.0):
+            The LayerDrop probability for the encoder. See the [LayerDrop paper](see https://arxiv.org/abs/1909.11556)
+            for more details.
+        decoder_layerdrop (`float`, *optional*, defaults to 0.0):
+            The LayerDrop probability for the decoder. See the [LayerDrop paper](see https://arxiv.org/abs/1909.11556)
+            for more details.
+        scale_embedding (`bool`, *optional*, defaults to `False`):
+            Scale embeddings by diving by sqrt(d_model).
+        use_cache (`bool`, *optional*, defaults to `True`):
+            Whether or not the model should return the last key/values attentions (not used by all models).
+        num_labels (`int`, *optional*, defaults to 3):
+            The number of labels to use in [`Florence2LanguageForSequenceClassification`].
+        forced_eos_token_id (`int`, *optional*, defaults to 2):
+            The id of the token to force as the last generated token when `max_length` is reached. Usually set to
+            `eos_token_id`.
+
+    Example:
+
+    ```python
+    >>> from transformers import Florence2LanguageConfig, Florence2LanguageModel
+
+    >>> # Initializing a Florence2 Language style configuration
+    >>> configuration = Florence2LanguageConfig()
+
+    >>> # Initializing a model (with random weights)
+    >>> model = Florence2LangaugeModel(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+
+    model_type = "florence2_language"
+    keys_to_ignore_at_inference = ["past_key_values"]
+    attribute_map = {"num_attention_heads": "encoder_attention_heads", "hidden_size": "d_model"}
+
+    def __init__(
+        self,
+        vocab_size=51289,
+        max_position_embeddings=1024,
+        encoder_layers=12,
+        encoder_ffn_dim=4096,
+        encoder_attention_heads=16,
+        decoder_layers=12,
+        decoder_ffn_dim=4096,
+        decoder_attention_heads=16,
+        encoder_layerdrop=0.0,
+        decoder_layerdrop=0.0,
+        activation_function="gelu",
+        d_model=1024,
+        dropout=0.1,
+        attention_dropout=0.0,
+        activation_dropout=0.0,
+        init_std=0.02,
+        classifier_dropout=0.0,
+        scale_embedding=False,
+        use_cache=True,
+        num_labels=3,
+        pad_token_id=1,
+        bos_token_id=0,
+        eos_token_id=2,
+        is_encoder_decoder=True,
+        decoder_start_token_id=2,
+        forced_eos_token_id=2,
+        **kwargs,
+    ):
+        self.vocab_size = vocab_size
+        self.max_position_embeddings = max_position_embeddings
+        self.d_model = d_model
+        self.encoder_ffn_dim = encoder_ffn_dim
+        self.encoder_layers = encoder_layers
+        self.encoder_attention_heads = encoder_attention_heads
+        self.decoder_ffn_dim = decoder_ffn_dim
+        self.decoder_layers = decoder_layers
+        self.decoder_attention_heads = decoder_attention_heads
+        self.dropout = dropout
+        self.attention_dropout = attention_dropout
+        self.activation_dropout = activation_dropout
+        self.activation_function = activation_function
+        self.init_std = init_std
+        self.encoder_layerdrop = encoder_layerdrop
+        self.decoder_layerdrop = decoder_layerdrop
+        self.classifier_dropout = classifier_dropout
+        self.use_cache = use_cache
+        self.num_hidden_layers = encoder_layers
+        self.scale_embedding = scale_embedding  # scale factor will be sqrt(d_model) if True
+
+        super().__init__(
+            num_labels=num_labels,
+            pad_token_id=pad_token_id,
+            bos_token_id=bos_token_id,
+            eos_token_id=eos_token_id,
+            is_encoder_decoder=is_encoder_decoder,
+            decoder_start_token_id=decoder_start_token_id,
+            forced_eos_token_id=forced_eos_token_id,
+            **kwargs,
+        )
+
+        # ensure backward compatibility for BART CNN models
+        if self.forced_bos_token_id is None and kwargs.get("force_bos_token_to_be_generated", False):
+            self.forced_bos_token_id = self.bos_token_id
+            warnings.warn(
+                f"Please make sure the config includes `forced_bos_token_id={self.bos_token_id}` in future versions. "
+                "The config can simply be saved and uploaded again to be fixed."
+            )
+
+class Florence2Config(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`Florence2ForConditionalGeneration`]. It is used to instantiate an
+    Florence-2 model according to the specified arguments, defining the model architecture. 
+
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
+    documentation from [`PretrainedConfig`] for more information.
+
+    Args:
+        vision_config (`Florence2VisionConfig`,  *optional*):
+            Custom vision config or dict
+        text_config (`Union[AutoConfig, dict]`, *optional*):
+            The config object of the text backbone. 
+        ignore_index (`int`, *optional*, defaults to -100):
+            The ignore index for the loss function.
+        vocab_size (`int`, *optional*, defaults to 51289):
+            Vocabulary size of the Florence2model. Defines the number of different tokens that can be represented by the
+            `inputs_ids` passed when calling [`~Florence2ForConditionalGeneration`]
+        projection_dim (`int`, *optional*, defaults to 1024):
+            Dimension of the multimodal projection space.
+
+    Example:
+
+    ```python
+    >>> from transformers import Florence2ForConditionalGeneration, Florence2Config, CLIPVisionConfig, BartConfig
+
+    >>> # Initializing a clip-like vision config
+    >>> vision_config = CLIPVisionConfig()
+
+    >>> # Initializing a Bart config
+    >>> text_config = BartConfig()
+
+    >>> # Initializing a Florence-2 configuration
+    >>> configuration = Florence2Config(vision_config, text_config)
+
+    >>> # Initializing a model from the florence-2 configuration
+    >>> model = Florence2ForConditionalGeneration(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+
+    model_type = "florence2"
+    is_composition = False
+
+    def __init__(
+        self,
+        vision_config=None,
+        text_config=None,
+        ignore_index=-100,
+        vocab_size=51289,
+        projection_dim=1024,
+        **kwargs,
+    ):
+        self.ignore_index = ignore_index
+        self.vocab_size = vocab_size
+        self.projection_dim = projection_dim
+        if vision_config is not None:
+            vision_config = Florence2VisionConfig(**vision_config)
+        self.vision_config = vision_config
+        self.vocab_size = self.vocab_size
+
+        self.text_config = text_config
+        if text_config is not None:
+            self.text_config = Florence2LanguageConfig(**text_config)
+
+
+        super().__init__(**kwargs)
+

LLM/Florence-2-base/preprocessor_config.json

@@ -0,0 +1,39 @@
+{
+  "auto_map": {
+    "AutoProcessor": "processing_florence2.Florence2Processor"
+   },
+  "_valid_processor_keys": [
+    "images",
+    "do_resize",
+    "size",
+    "resample",
+    "do_rescale",
+    "rescale_factor",
+    "do_normalize",
+    "image_mean",
+    "image_std",
+    "return_tensors",
+    "data_format",
+    "input_data_format",
+    "do_convert_rgb"
+  ],
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "do_center_crop": false,
+  "image_processor_type": "CLIPImageProcessor",
+  "image_seq_length": 577,
+  "image_mean": [0.485, 0.456, 0.406],
+  "image_std":  [0.229, 0.224, 0.225],
+  "processor_class": "Florence2Processor",
+  "resample": 3,
+  "size": {
+    "height": 768,
+    "width":768 
+  },
+  "crop_size": {
+    "height": 768,
+    "width": 768
+  }
+}

LLM/Florence-2-base/processing_florence2.py

@@ -0,0 +1,1148 @@
+# coding=utf-8
+# Copyright 2024 Microsoft and The HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Processor class for Florence-2.
+"""
+
+import re
+import logging
+from typing import List, Optional, Union
+import numpy as np
+import math
+
+import torch
+
+from transformers.feature_extraction_utils import BatchFeature
+from transformers.image_utils import ImageInput, is_valid_image
+from transformers.processing_utils import ProcessorMixin
+from transformers.tokenization_utils_base import (
+    PaddingStrategy,
+    PreTokenizedInput,
+    TextInput,
+    TruncationStrategy,
+)
+from transformers import BartTokenizer, BartTokenizerFast
+from transformers.utils import TensorType
+
+
+logger = logging.getLogger(__name__)
+
+# Copied from transformers.models.idefics2.processing_idefics2.is_url
+def is_url(val) -> bool:
+    return isinstance(val, str) and val.startswith("http")
+
+# Copied from transformers.models.idefics2.processing_idefics2.is_image_or_image_url
+def is_image_or_image_url(elem):
+    return is_url(elem) or is_valid_image(elem)
+
+
+def _is_str_or_image(elem):
+    return isinstance(elem, (str)) or is_image_or_image_url(elem)
+
+
+class Florence2Processor(ProcessorMixin):
+    r"""
+    Constructs a Florence2 processor which wraps a Florence2 image processor and a Florence2 tokenizer into a single processor.
+
+    [`Florence2Processor`] offers all the functionalities of [`CLIPImageProcessor`] and [`BartTokenizerFast`]. See the
+    [`~Florence2Processor.__call__`] and [`~Florence2Processor.decode`] for more information.
+
+    Args:
+        image_processor ([`CLIPImageProcessor`], *optional*):
+            The image processor is a required input.
+        tokenizer ([`BartTokenizerFast`], *optional*):
+            The tokenizer is a required input.
+    """
+
+    attributes = ["image_processor", "tokenizer"]
+    image_processor_class = "CLIPImageProcessor"
+    tokenizer_class = ("BartTokenizer", "BartTokenizerFast")
+
+    def __init__(
+        self,
+        image_processor=None,
+        tokenizer=None,
+    ):
+        if image_processor is None:
+            raise ValueError("You need to specify an `image_processor`.")
+        if tokenizer is None:
+            raise ValueError("You need to specify a `tokenizer`.")
+        if not hasattr(image_processor, "image_seq_length"):
+            raise ValueError("Image processor is missing an `image_seq_length` attribute.")
+
+        self.image_seq_length = image_processor.image_seq_length
+
+        tokens_to_add = {
+                'additional_special_tokens': \
+                    tokenizer.additional_special_tokens + \
+                    ['<od>', '</od>', '<ocr>', '</ocr>'] + \
+                    [f'<loc_{x}>' for x in range(1000)] + \
+                    ['<cap>', '</cap>', '<ncap>', '</ncap>','<dcap>', '</dcap>', '<grounding>', '</grounding>', '<seg>', '</seg>', '<sep>', '<region_cap>', '</region_cap>', '<region_to_desciption>', '</region_to_desciption>', '<proposal>', '</proposal>', '<poly>', '</poly>', '<and>']
+            }
+        tokenizer.add_special_tokens(tokens_to_add)
+
+        self.tasks_answer_post_processing_type = {
+            '<OCR>': 'pure_text',
+            '<OCR_WITH_REGION>': 'ocr',
+            '<CAPTION>': 'pure_text',
+            '<DETAILED_CAPTION>': 'pure_text',
+            '<MORE_DETAILED_CAPTION>': 'pure_text',
+            '<OD>': 'description_with_bboxes',
+            '<DENSE_REGION_CAPTION>': 'description_with_bboxes',
+            '<CAPTION_TO_PHRASE_GROUNDING>': "phrase_grounding",
+            '<REFERRING_EXPRESSION_SEGMENTATION>': 'polygons',
+            '<REGION_TO_SEGMENTATION>': 'polygons',
+            '<OPEN_VOCABULARY_DETECTION>': 'description_with_bboxes_or_polygons',
+            '<REGION_TO_CATEGORY>': 'pure_text',
+            '<REGION_TO_DESCRIPTION>': 'pure_text',
+            '<REGION_TO_OCR>': 'pure_text',
+            '<REGION_PROPOSAL>': 'bboxes'
+        }
+
+        self.task_prompts_without_inputs = {
+            '<OCR>': 'What is the text in the image?',
+            '<OCR_WITH_REGION>': 'What is the text in the image, with regions?',
+            '<CAPTION>': 'What does the image describe?',
+            '<DETAILED_CAPTION>': 'Describe in detail what is shown in the image.',
+            '<MORE_DETAILED_CAPTION>': 'Describe with a paragraph what is shown in the image.',
+            '<OD>': 'Locate the objects with category name in the image.',
+            '<DENSE_REGION_CAPTION>': 'Locate the objects in the image, with their descriptions.',
+            '<REGION_PROPOSAL>': 'Locate the region proposals in the image.'
+        }
+
+        self.task_prompts_with_input = {
+            '<CAPTION_TO_PHRASE_GROUNDING>': "Locate the phrases in the caption: {input}",
+            '<REFERRING_EXPRESSION_SEGMENTATION>': 'Locate {input} in the image with mask',
+            '<REGION_TO_SEGMENTATION>': 'What is the polygon mask of region {input}',
+            '<OPEN_VOCABULARY_DETECTION>': 'Locate {input} in the image.',
+            '<REGION_TO_CATEGORY>': 'What is the region {input}?',
+            '<REGION_TO_DESCRIPTION>': 'What does the region {input} describe?',
+            '<REGION_TO_OCR>': 'What text is in the region {input}?',
+        }
+
+        self.post_processor = Florence2PostProcesser(tokenizer=tokenizer)
+
+
+        super().__init__(image_processor, tokenizer)
+    
+    def _construct_prompts(self, text):
+        # replace the task tokens with the task prompts if task token is in the text
+        prompts = []
+        for _text in text:
+            # 1. fixed task prompts without additional inputs
+            for task_token, task_prompt in self.task_prompts_without_inputs.items():
+                if task_token in _text:
+                    assert _text == task_token, f"Task token {task_token} should be the only token in the text."
+                    _text = task_prompt
+                    break
+            # 2. task prompts with additional inputs 
+            for task_token, task_prompt in self.task_prompts_with_input.items():
+                if task_token in _text:
+                    _text = task_prompt.format(input=_text.replace(task_token, ''))
+                    break
+            prompts.append(_text)
+        return prompts
+
+    def __call__(
+        self,
+        text: Union[TextInput, PreTokenizedInput, List[TextInput], List[PreTokenizedInput]] = None,
+        images: ImageInput = None,
+        tokenize_newline_separately: bool = True,
+        padding: Union[bool, str, PaddingStrategy] = False,
+        truncation: Union[bool, str, TruncationStrategy] = None,
+        max_length=None,
+        return_tensors: Optional[Union[str, TensorType]] = TensorType.PYTORCH,
+        do_resize: bool = None,
+        do_normalize: bool = None,
+        image_mean: Optional[Union[float, List[float]]] = None,
+        image_std: Optional[Union[float, List[float]]] = None,
+        data_format: Optional["ChannelDimension"] = "channels_first",  # noqa: F821
+        input_data_format: Optional[
+            Union[str, "ChannelDimension"]  # noqa: F821
+        ] = None,
+        resample: "PILImageResampling" = None,  # noqa: F821
+        do_convert_rgb: bool = None,
+        do_thumbnail: bool = None,
+        do_align_long_axis: bool = None,
+        do_rescale: bool = None,
+    ) -> BatchFeature:
+        """
+        Main method to prepare for the model one or several sequences(s) and image(s). This method forwards the `text`
+        and `kwargs` arguments to BartTokenizerFast's [`~BartTokenizerFast.__call__`] if `text` is not `None` to encode
+        the text. To prepare the image(s), this method forwards the `images` and `kwrags` arguments to
+        CLIPImageProcessor's [`~CLIPImageProcessor.__call__`] if `images` is not `None`. Please refer to the doctsring
+        of the above two methods for more information.
+
+        Args:
+            text (`str`, `List[str]`, `List[List[str]]`):
+                The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings
+                (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set
+                `is_split_into_words=True` (to lift the ambiguity with a batch of sequences).
+            images (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `List[PIL.Image.Image]`, `List[np.ndarray]`, `List[torch.Tensor]`):
+                The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch
+                tensor. In case of a NumPy array/PyTorch tensor, each image should be of shape (C, H, W), where C is a
+                number of channels, H and W are image height and width.
+            tokenize_newline_separately (`bool`, defaults to `True`):
+                Adds a separately tokenized '\n' at the end of the prompt.
+            padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `False`):
+                Select a strategy to pad the returned sequences (according to the model's padding side and padding
+                index) among:
+                - `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single
+                  sequence if provided).
+                - `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum
+                  acceptable input length for the model if that argument is not provided.
+                - `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different
+                  lengths).
+            max_length (`int`, *optional*):
+                Maximum length of the returned list and optionally padding length (see above).
+            truncation (`bool`, *optional*):
+                Activates truncation to cut input sequences longer than `max_length` to `max_length`.
+            return_tensors (`str` or [`~utils.TensorType`], *optional*):
+                If set, will return tensors of a particular framework. Acceptable values are:
+
+                - `'tf'`: Return TensorFlow `tf.constant` objects.
+                - `'pt'`: Return PyTorch `torch.Tensor` objects.
+                - `'np'`: Return NumPy `np.ndarray` objects.
+                - `'jax'`: Return JAX `jnp.ndarray` objects.
+
+        Returns:
+            [`BatchFeature`]: A [`BatchFeature`] with the following fields:
+
+            - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. If `suffix`
+              is provided, the `input_ids` will also contain the suffix input ids.
+            - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when
+              `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not
+              `None`).
+            - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`.
+            - **labels** -- Labels compatible with training if `suffix` is not None
+        """
+
+        return_token_type_ids = False
+
+        if images is None:
+            raise ValueError("`images` are expected as arguments to a `Florence2Processor` instance.")
+        if text is None:
+            logger.warning_once(
+                "You are using Florence-2 without a text prompt."
+            )
+            text = ""
+
+        if isinstance(text, List) and isinstance(images, List):
+            if len(images) < len(text):
+                raise ValueError(
+                    f"Received {len(images)} images for {len(text)} prompts. Each prompt should be associated with an image."
+                )
+        if _is_str_or_image(text):
+            text = [text]
+        elif isinstance(text, list) and _is_str_or_image(text[0]):
+            pass
+
+        pixel_values = self.image_processor(
+            images,
+            do_resize=do_resize,
+            do_normalize=do_normalize,
+            return_tensors=return_tensors,
+            image_mean=image_mean,
+            image_std=image_std,
+            input_data_format=input_data_format,
+            data_format=data_format,
+            resample=resample,
+            do_convert_rgb=do_convert_rgb,
+        )["pixel_values"]
+
+        if max_length is not None:
+            max_length -= self.image_seq_length  # max_length has to account for the image tokens
+
+        text = self._construct_prompts(text)
+
+        inputs = self.tokenizer(
+            text,
+            return_tensors=return_tensors,
+            padding=padding,
+            max_length=max_length,
+            truncation=truncation,
+            return_token_type_ids=return_token_type_ids,
+        )
+
+        return_data = {**inputs, "pixel_values": pixel_values}
+
+        if return_token_type_ids:
+            labels = inputs["input_ids"].masked_fill(inputs["token_type_ids"] == 0, -100)
+            return_data.update({"labels": labels})
+        return BatchFeature(data=return_data)
+
+    # Copied from transformers.models.clip.processing_clip.CLIPProcessor.batch_decode with CLIP->Florence2
+    def batch_decode(self, *args, **kwargs):
+        """
+        This method forwards all its arguments to BartTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please
+        refer to the docstring of this method for more information.
+        """
+        return self.tokenizer.batch_decode(*args, **kwargs)
+
+    # Copied from transformers.models.clip.processing_clip.CLIPProcessor.decode with CLIP->Florence2
+    def decode(self, *args, **kwargs):
+        """
+        This method forwards all its arguments to BartTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to
+        the docstring of this method for more information.
+        """
+        return self.tokenizer.decode(*args, **kwargs)
+
+    @property
+    # Copied from transformers.models.clip.processing_clip.CLIPProcessor.model_input_names with CLIP->Florence2
+    def model_input_names(self):
+        tokenizer_input_names = self.tokenizer.model_input_names
+        image_processor_input_names = self.image_processor.model_input_names
+        return list(dict.fromkeys(tokenizer_input_names + image_processor_input_names))
+
+    def post_process_generation(self, text=None, sequence=None, transition_beam_score=None, task=None, image_size=None):
+        """
+        Post-process the output of the model to each of the task outputs.
+
+        Args:
+            text (`str`): The text to post-process.
+            task (`str`): The task to post-process the text for.
+            image_size (`Tuple[int, int]`): The size of the image. height x width.
+        """
+
+        task_answer_post_processing_type = self.tasks_answer_post_processing_type.get(task, 'pure_text')
+        task_answer = self.post_processor(
+            text=text,
+            sequence=sequence,
+            transition_beam_score=transition_beam_score,
+            image_size=image_size,
+            parse_tasks=task_answer_post_processing_type,
+        )[task_answer_post_processing_type]
+
+        if task_answer_post_processing_type == 'pure_text':
+            final_answer = task_answer
+            # remove the special tokens
+            final_answer = final_answer.replace('<s>', '').replace('</s>', '')
+        elif task_answer_post_processing_type in ['od', 'description_with_bboxes', 'bboxes']:
+            od_instances = task_answer
+            bboxes_od = [_od_instance['bbox'] for _od_instance in od_instances]
+            labels_od = [str(_od_instance['cat_name']) for _od_instance in od_instances]
+            final_answer = {'bboxes': bboxes_od, 'labels': labels_od}
+            if len(od_instances) and 'score' in od_instances[0]:
+                scores_od = [_od_instance['score'] for _od_instance in od_instances]
+                final_answer['scores'] = scores_od
+        elif task_answer_post_processing_type in ['ocr']:
+            bboxes = [_od_instance['quad_box'] for _od_instance in task_answer]
+            labels = [str(_od_instance['text']) for _od_instance in task_answer]
+            final_answer = {'quad_boxes': bboxes, 'labels': labels}
+        elif task_answer_post_processing_type in ['phrase_grounding']:
+            bboxes = []
+            labels = []
+            for _grounded_phrase in task_answer:
+                for _bbox in _grounded_phrase['bbox']:
+                    bboxes.append(_bbox)
+                    labels.append(_grounded_phrase['cat_name'])
+            final_answer = {'bboxes': bboxes, 'labels': labels}
+        elif task_answer_post_processing_type in ['description_with_polygons', 'polygons']:
+            labels = []
+            polygons = []
+            for result in task_answer:
+                label = result['cat_name']
+                _polygons = result['polygons']
+                labels.append(label)
+                polygons.append(_polygons)
+            final_answer = {'polygons': polygons, 'labels': labels}
+        elif task_answer_post_processing_type in ['description_with_bboxes_or_polygons']:
+            bboxes = []
+            bboxes_labels = []
+            polygons = []
+            polygons_labels = []
+            for result in task_answer:
+                label = result['cat_name']
+                if 'polygons' in result:
+                    _polygons = result['polygons']
+                    polygons.append(_polygons)
+                    polygons_labels.append(label)
+                else:
+                    _bbox = result['bbox']
+                    bboxes.append(_bbox)
+                    bboxes_labels.append(label)
+            final_answer = {'bboxes': bboxes, 'bboxes_labels': bboxes_labels, 'polygons': polygons, 'polygons_labels': polygons_labels}
+        else:
+            raise ValueError('Unknown task answer post processing type: {}'.format(task_answer_post_processing_type))
+
+        final_answer = {
+            task: final_answer}
+        return final_answer 
+
+class BoxQuantizer(object):
+    def __init__(self, mode, bins):
+        self.mode = mode
+        self.bins = bins
+
+    def quantize(self, boxes: torch.Tensor, size):
+        bins_w, bins_h = self.bins  # Quantization bins.
+        size_w, size_h = size       # Original image size.
+        size_per_bin_w = size_w / bins_w
+        size_per_bin_h = size_h / bins_h
+        xmin, ymin, xmax, ymax = boxes.split(1, dim=-1)  # Shape: 4 * [N, 1].
+
+        if self.mode == 'floor':
+            quantized_xmin = (
+                xmin / size_per_bin_w).floor().clamp(0, bins_w - 1)
+            quantized_ymin = (
+                ymin / size_per_bin_h).floor().clamp(0, bins_h - 1)
+            quantized_xmax = (
+                xmax / size_per_bin_w).floor().clamp(0, bins_w - 1)
+            quantized_ymax = (
+                ymax / size_per_bin_h).floor().clamp(0, bins_h - 1)
+
+        elif self.mode == 'round':
+            raise NotImplementedError()
+
+        else:
+            raise ValueError('Incorrect quantization type.')
+
+        quantized_boxes = torch.cat(
+            (quantized_xmin, quantized_ymin, quantized_xmax, quantized_ymax), dim=-1
+        ).int()
+
+        return quantized_boxes
+
+    def dequantize(self, boxes: torch.Tensor, size):
+        bins_w, bins_h = self.bins  # Quantization bins.
+        size_w, size_h = size       # Original image size.
+        size_per_bin_w = size_w / bins_w
+        size_per_bin_h = size_h / bins_h
+        xmin, ymin, xmax, ymax = boxes.split(1, dim=-1)  # Shape: 4 * [N, 1].
+
+        if self.mode == 'floor':
+            # Add 0.5 to use the center position of the bin as the coordinate.
+            dequantized_xmin = (xmin + 0.5) * size_per_bin_w
+            dequantized_ymin = (ymin + 0.5) * size_per_bin_h
+            dequantized_xmax = (xmax + 0.5) * size_per_bin_w
+            dequantized_ymax = (ymax + 0.5) * size_per_bin_h
+
+        elif self.mode == 'round':
+            raise NotImplementedError()
+
+        else:
+            raise ValueError('Incorrect quantization type.')
+
+        dequantized_boxes = torch.cat(
+            (dequantized_xmin, dequantized_ymin,
+             dequantized_xmax, dequantized_ymax), dim=-1
+        )
+
+        return dequantized_boxes
+
+
+class CoordinatesQuantizer(object):
+    """
+    Quantize coornidates (Nx2)
+    """
+
+    def __init__(self, mode, bins):
+        self.mode = mode
+        self.bins = bins
+
+    def quantize(self, coordinates: torch.Tensor, size):
+        bins_w, bins_h = self.bins  # Quantization bins.
+        size_w, size_h = size       # Original image size.
+        size_per_bin_w = size_w / bins_w
+        size_per_bin_h = size_h / bins_h
+        assert coordinates.shape[-1] == 2, 'coordinates should be shape (N, 2)'
+        x, y = coordinates.split(1, dim=-1)  # Shape: 4 * [N, 1].
+
+        if self.mode == 'floor':
+            quantized_x = (x / size_per_bin_w).floor().clamp(0, bins_w - 1)
+            quantized_y = (y / size_per_bin_h).floor().clamp(0, bins_h - 1)
+
+        elif self.mode == 'round':
+            raise NotImplementedError()
+
+        else:
+            raise ValueError('Incorrect quantization type.')
+
+        quantized_coordinates = torch.cat(
+            (quantized_x, quantized_y), dim=-1
+        ).int()
+
+        return quantized_coordinates
+
+    def dequantize(self, coordinates: torch.Tensor, size):
+        bins_w, bins_h = self.bins  # Quantization bins.
+        size_w, size_h = size       # Original image size.
+        size_per_bin_w = size_w / bins_w
+        size_per_bin_h = size_h / bins_h
+        assert coordinates.shape[-1] == 2, 'coordinates should be shape (N, 2)'
+        x, y = coordinates.split(1, dim=-1)  # Shape: 4 * [N, 1].
+
+        if self.mode == 'floor':
+            # Add 0.5 to use the center position of the bin as the coordinate.
+            dequantized_x = (x + 0.5) * size_per_bin_w
+            dequantized_y = (y + 0.5) * size_per_bin_h
+
+        elif self.mode == 'round':
+            raise NotImplementedError()
+
+        else:
+            raise ValueError('Incorrect quantization type.')
+
+        dequantized_coordinates = torch.cat(
+            (dequantized_x, dequantized_y), dim=-1
+        )
+
+        return dequantized_coordinates
+
+
+class Florence2PostProcesser(object):
+    r"""
+    Florence-2 post process for converting text prediction to various tasks results. 
+
+    Args:
+        config: A dict of configs.
+        tokenizer: A tokenizer for decoding text to spans.
+        sample config:
+            UNIFIED_POST_PROCESS:
+                # commom configs
+                NUM_BBOX_HEIGHT_BINS: 1000
+                NUM_BBOX_WIDTH_BINS: 1000
+                COORDINATES_HEIGHT_BINS: 1000
+                COORDINATES_WIDTH_BINS: 1000
+                # task specific configs, override the common configs
+                PRASE_TASKS:
+                    - TASK_NAME: 'video_dense_caption'
+                      PATTERN: 'r<time_(\d+)><time_(\d+)>([a-zA-Z0-9 ]+)'
+                      SCORE_MODE: 'avg_cat_name_scores'
+                      NUM_BINS: 100
+                    - TASK_NAME: 'od'
+                      PATTERN: 'r<loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)>([a-zA-Z0-9 ]+)'
+                      SCORE_MODE: 'avg_cat_name_scores'
+
+    Returns:
+        parsed_dict (dict): A dict of parsed results.
+    """
+    def __init__(
+        self,
+        tokenizer=None
+    ):
+        parse_tasks = []
+        parse_task_configs = {}
+        config = self._create_default_config()
+        for task in config['PARSE_TASKS']:
+            parse_tasks.append(task['TASK_NAME'])
+            parse_task_configs[task['TASK_NAME']] = task
+
+        self.config = config
+        self.parse_tasks = parse_tasks
+        self.parse_tasks_configs = parse_task_configs
+
+        self.tokenizer =  tokenizer
+        if self.tokenizer is not None:
+            self.all_special_tokens = set(self.tokenizer.all_special_tokens)
+
+        self.init_quantizers()
+        self.black_list_of_phrase_grounding = self._create_black_list_of_phrase_grounding()
+
+    def _create_black_list_of_phrase_grounding(self):
+        black_list = {}
+
+        if 'phrase_grounding' in self.parse_tasks and self.parse_tasks_configs['phrase_grounding']['FILTER_BY_BLACK_LIST']:
+            black_list =  set(
+                ['it', 'I', 'me', 'mine',
+                 'you', 'your', 'yours',
+                 'he', 'him', 'his',
+                 'she', 'her', 'hers',
+                 'they', 'them', 'their', 'theirs',
+                 'one', 'oneself',
+                 'we', 'us', 'our', 'ours',
+                 'you', 'your', 'yours',
+                 'they', 'them', 'their', 'theirs',
+                 'mine', 'yours', 'his', 'hers', 'its',
+                 'ours', 'yours', 'theirs',
+                 'myself', 'yourself', 'himself', 'herself', 'itself',
+                 'ourselves', 'yourselves', 'themselves',
+                 'this', 'that',
+                 'these', 'those',
+                 'who', 'whom', 'whose', 'which', 'what',
+                 'who', 'whom', 'whose', 'which', 'that',
+                 'all', 'another', 'any', 'anybody', 'anyone', 'anything',
+                 'each', 'everybody', 'everyone', 'everything',
+                 'few', 'many', 'nobody', 'none', 'one', 'several',
+                 'some', 'somebody', 'someone', 'something',
+                 'each other', 'one another',
+                 'myself', 'yourself', 'himself', 'herself', 'itself',
+                 'ourselves', 'yourselves', 'themselves',
+                 'the image', 'image', 'images', 'the', 'a', 'an', 'a group',
+                 'other objects', 'lots', 'a set',
+                 ]
+            )
+
+        return black_list
+    
+    def _create_default_config(self):
+        config = {
+            'NUM_BBOX_HEIGHT_BINS': 1000,
+            'NUM_BBOX_WIDTH_BINS': 1000,
+            'BOX_QUANTIZATION_MODE': 'floor',
+            'COORDINATES_HEIGHT_BINS': 1000,
+            'COORDINATES_WIDTH_BINS': 1000,
+            'COORDINATES_QUANTIZATION_MODE': 'floor',
+            'PARSE_TASKS': [
+                {
+                    'TASK_NAME': 'od',
+                    'PATTERN': r'([a-zA-Z0-9 ]+)<loc_(\\d+)><loc_(\\d+)><loc_(\\d+)><loc_(\\d+)>',
+                    'SCORE_MODE': 'avg_loc_scores'
+                },
+                {
+                    'TASK_NAME': 'ocr',
+                    'PATTERN':  r'(.+?)<loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)>',
+                    'AREA_THRESHOLD': 0.00
+                },
+                {
+                    'TASK_NAME': 'phrase_grounding',
+                    'FILTER_BY_BLACK_LIST': True
+                },
+                {
+                    'TASK_NAME': 'pure_text',
+                },
+                {
+                    'TASK_NAME': 'description_with_bboxes',
+                    'SCORE_MODE': 'avg_loc_scores'
+                },
+                {
+                    'TASK_NAME': 'description_with_polygons',
+                },
+                {
+                    'TASK_NAME': 'polygons',
+                },
+                {
+                    'TASK_NAME': 'bboxes',
+                },
+                {
+                    'TASK_NAME': 'description_with_bboxes_or_polygons',
+                }
+            ]
+        }
+
+        return config
+
+    def init_quantizers(self):
+        # we have box_quantizer (od, grounding) and coordinates_quantizer (ocr, referring_segmentation)
+        num_bbox_height_bins = self.config.get('NUM_BBOX_HEIGHT_BINS', 1000)
+        num_bbox_width_bins = self.config.get('NUM_BBOX_WIDTH_BINS', 1000)
+        box_quantization_mode = self.config.get('BOX_QUANTIZATION_MODE', 'floor')
+        self.box_quantizer = BoxQuantizer(
+            box_quantization_mode,
+            (num_bbox_width_bins, num_bbox_height_bins),
+        )
+        
+        num_bbox_height_bins = self.config['COORDINATES_HEIGHT_BINS'] if 'COORDINATES_HEIGHT_BINS' in self.config else self.config.get('NUM_BBOX_HEIGHT_BINS', 1000)
+        num_bbox_width_bins = self.config['COORDINATES_WIDTH_BINS'] if 'COORDINATES_WIDTH_BINS' in self.config else self.config.get('NUM_BBOX_WIDTH_BINS', 1000)
+        box_quantization_mode = self.config.get('COORDINATES_QUANTIZATION_MODE') if 'COORDINATES_QUANTIZATION_MODE' in self.config else self.config.get('BOX_QUANTIZATION_MODE', 'floor')
+        self.coordinates_quantizer = CoordinatesQuantizer(
+            box_quantization_mode,
+            (num_bbox_width_bins, num_bbox_height_bins),
+        )
+
+    def decode_with_spans(self, tokenizer, token_ids):
+        filtered_tokens = tokenizer.convert_ids_to_tokens(
+            token_ids, skip_special_tokens=False)
+        assert len(filtered_tokens) == len(token_ids)
+
+        sub_texts = []
+        for token in filtered_tokens:
+            if token in self.all_special_tokens:
+                sub_texts.append(token)
+            else:
+                if isinstance(tokenizer, (BartTokenizer, BartTokenizerFast)):
+                    sub_text = tokenizer.convert_tokens_to_string([token])
+                else:
+                    raise ValueError(f'type {type(tokenizer)} not supported')
+                sub_texts.append(sub_text)
+
+        text = ''
+        spans = []
+        for sub_text in sub_texts:
+            span = (len(text), len(text) + len(sub_text))  # [start index, end index).
+            text += sub_text
+            spans.append(span)
+
+        return text, spans
+
+    def parse_od_from_text_and_spans(
+        self,
+        text,
+        pattern,
+        image_size,
+        phrase_centric=False
+    ):
+        parsed = list(re.finditer(pattern, text))
+
+        instances = []
+        for i in range(len(parsed)):
+            # Prepare instance.
+            instance = {}
+
+            if phrase_centric:
+                bbox_bins = [int(parsed[i].group(j)) for j in range(2, 6)]
+            else:
+                bbox_bins = [int(parsed[i].group(j)) for j in range(1, 5)]
+            instance['bbox'] = self.box_quantizer.dequantize(
+                boxes=torch.tensor(bbox_bins),
+                size=image_size
+            ).tolist()  
+
+            if phrase_centric:
+                instance['cat_name'] = parsed[i].group(1).lower().strip()
+            else:
+                instance['cat_name'] = parsed[i].group(5).lower().strip()
+            instances.append(instance)
+
+        return instances
+
+    def parse_ocr_from_text_and_spans(self, 
+                                     text, 
+                                     pattern, 
+                                     image_size,
+                                     area_threshold=-1.0,
+        ):
+        bboxes = []
+        labels = []
+        text = text.replace('<s>', '')
+        # ocr with regions
+        parsed = re.findall(pattern, text)
+        instances = []
+        image_width, image_height = image_size
+
+        for ocr_line in parsed:
+            ocr_content = ocr_line[0]
+            quad_box = ocr_line[1:]
+            quad_box = [int(i) for i in quad_box]
+            quad_box = self.coordinates_quantizer.dequantize(
+                torch.tensor(np.array(quad_box).reshape(-1, 2)),
+                size=image_size
+            ).reshape(-1).tolist()
+
+            if area_threshold > 0:
+                x_coords = [i for i in quad_box[0::2]]
+                y_coords = [i for i in quad_box[1::2]]
+
+                # apply the Shoelace formula
+                area = 0.5 * abs(sum(x_coords[i] * y_coords[i + 1] - x_coords[i + 1] * y_coords[i] for i in range(4 - 1)))
+
+                if area < (image_width * image_height) * area_threshold:
+                    continue
+
+            bboxes.append(quad_box)
+            labels.append(ocr_content)
+            instances.append({
+                'quad_box': quad_box,
+                'text': ocr_content,
+            })
+        return instances
+
+    def parse_phrase_grounding_from_text_and_spans(self, text, pattern, image_size):
+        # ignore <s> </s> and <pad>
+        cur_span = 0
+        if text.startswith('<s>'):   
+            cur_span += 3
+
+        text = text.replace('<s>', '')
+        text = text.replace('</s>', '')
+        text = text.replace('<pad>', '')
+
+        pattern = r"([^<]+(?:<loc_\d+>){4,})"
+        phrases = re.findall(pattern, text)
+    
+        # pattern should be text pattern and od pattern
+        pattern = r'^\s*(.*?)(?=<od>|</od>|<box>|</box>|<bbox>|</bbox>|<loc_)'
+        box_pattern = r'<loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)>'
+
+        instances = []
+        for pharse_text in phrases:
+            phrase_text_strip = pharse_text.replace('<ground>', '', 1)
+            phrase_text_strip = pharse_text.replace('<obj>', '', 1)
+
+            if phrase_text_strip == '':
+                cur_span += len(pharse_text)
+                continue
+
+            # Prepare instance.
+            instance = {}
+
+            # parse phrase, get string 
+            phrase = re.search(pattern, phrase_text_strip)
+            if phrase is None:
+                cur_span += len(pharse_text)
+                continue
+
+            # parse bboxes by box_pattern
+            bboxes_parsed = list(re.finditer(box_pattern, pharse_text))
+            if len(bboxes_parsed) == 0:
+                cur_span += len(pharse_text)
+                continue
+
+            phrase = phrase.group()
+            # remove leading and trailing spaces
+            phrase = phrase.strip()
+
+            if phrase in self.black_list_of_phrase_grounding:
+                cur_span += len(pharse_text)
+                continue
+
+            # a list of list 
+            bbox_bins = [[int(_bboxes_parsed.group(j)) for j in range(1, 5)] for _bboxes_parsed in bboxes_parsed]
+            instance['bbox'] = self.box_quantizer.dequantize(
+                boxes=torch.tensor(bbox_bins),
+                size=image_size
+            ).tolist()  
+
+            # exclude non-ascii characters
+            phrase = phrase.encode('ascii',errors='ignore').decode('ascii')
+            instance['cat_name'] = phrase
+
+            instances.append(instance)
+
+        return instances
+
+    def parse_description_with_bboxes_from_text_and_spans(
+            self, 
+            text, 
+            spans=None,
+            scores=None,
+            score_mode=None,
+            pattern=None, 
+            image_size=None, 
+            allow_empty_phrase=False
+        ):
+        def find_matched_token_indices(cur_span, token_spans):
+            inds = []
+            for i, token_span in enumerate(token_spans):
+                if not (token_span[1] <= cur_span[0] or token_span[0] >= cur_span[1]):
+                    inds.append(i)
+            return inds
+
+        cur_span = 0
+        if text.startswith('<s>'):   
+            cur_span += 3
+
+        text = text.replace('<s>', '')
+        text = text.replace('</s>', '')
+        text = text.replace('<pad>', '')
+
+        if allow_empty_phrase:
+            pattern = rf"(?:(?:<loc_\d+>){{4,}})"
+        else:
+            pattern = r"([^<]+(?:<loc_\d+>){4,})"
+        phrases = re.findall(pattern, text)
+    
+        # pattern should be text pattern and od pattern
+        pattern = r'^\s*(.*?)(?=<od>|</od>|<box>|</box>|<bbox>|</bbox>|<loc_)'
+        box_pattern = r'<loc_(\d+)><loc_(\d+)><loc_(\d+)><loc_(\d+)>'
+
+        instances = []
+        for pharse_text in phrases:
+            phrase_text_strip = pharse_text.replace('<ground>', '', 1)
+            phrase_text_strip = pharse_text.replace('<obj>', '', 1)
+
+            if phrase_text_strip == '' and not allow_empty_phrase:
+                cur_span += len(pharse_text)
+                continue
+
+            # parse phrase, get string 
+            phrase = re.search(pattern, phrase_text_strip)
+            if phrase is None:
+                cur_span += len(pharse_text)
+                continue
+
+            phrase_span = phrase.span()
+            phrase = phrase.group()
+            # remove leading and trailing spaces
+            phrase = phrase.strip()
+
+            # parse bboxes by box_pattern
+            bboxes_parsed = list(re.finditer(box_pattern, pharse_text))
+            if len(bboxes_parsed) == 0:
+                cur_span += len(pharse_text)
+                continue
+
+            # a list of list 
+            bbox_bins = [[int(_bboxes_parsed.group(j)) for j in range(1, 5)] for _bboxes_parsed in bboxes_parsed]
+
+            bboxes = self.box_quantizer.dequantize(
+                boxes=torch.tensor(bbox_bins),
+                size=image_size
+            ).tolist()  
+
+            if score_mode == 'avg_loc_scores':
+                if spans is None or scores is None:
+                    all_scores = None
+                else:
+                    bbox_end_spans = [_bboxes_parsed.span(0) for _bboxes_parsed in bboxes_parsed]
+                    all_scores = []
+                    for _spans in bbox_end_spans:
+                        token_inds = find_matched_token_indices((_spans[0] + cur_span, _spans[1]+ cur_span), spans)
+                        loc_scores = [scores[token_i] for token_i in token_inds]
+                        score = sum(loc_scores) / len(loc_scores)
+                        all_scores.append(score)
+            elif score_mode == 'avg_cat_name_scores': 
+                if spans is None or scores is None:
+                    all_scores = None
+                else:
+                    cat_name_token_inds = find_matched_token_indices((phrase_span[0] + cur_span, phrase_span[1]+cur_span), spans)
+                    cat_name_scores = [scores[token_i] for token_i in cat_name_token_inds]
+                    score = sum(cat_name_scores) / len(cat_name_scores)
+                    all_scores = [score] * len(bboxes)
+            elif score_mode is None:
+                all_scores = None
+            else:
+                raise ValueError('Unknown score mode: {}'.format(score_mode))
+
+            phrase = phrase.encode('ascii',errors='ignore').decode('ascii')
+            for _idx, _bboxes in enumerate(bboxes):
+                # Prepare instance.
+                instance = {}
+                instance['bbox'] = _bboxes
+                # exclude non-ascii characters
+                instance['cat_name'] = phrase
+                if all_scores is not None:
+                    instance['score'] = math.exp(all_scores[_idx])
+                instances.append(instance)
+            
+            cur_span += len(pharse_text)
+
+        return instances
+
+    def parse_description_with_polygons_from_text_and_spans(self, text, pattern, image_size, 
+                                                            allow_empty_phrase=False,
+                                                            polygon_sep_token='<sep>',
+                                                            polygon_start_token='<poly>',
+                                                            polygon_end_token='</poly>',
+                                                            with_box_at_start=False,
+                                                            ):
+        
+        # ref_seg format: '<expression><x1><y1><x2><y2><><><sep><><><><>'
+        # ignore <s> </s> and <pad>
+
+        text = text.replace('<s>', '')
+        text = text.replace('</s>', '')
+        text = text.replace('<pad>', '')
+
+        if allow_empty_phrase:
+            pattern = rf"(?:(?:<loc_\d+>|{re.escape(polygon_sep_token)}|{re.escape(polygon_start_token)}|{re.escape(polygon_end_token)}){{4,}})"
+        else:
+            # [^<]+: This part matches one or more characters that are not the < symbol. 
+            # The ^ inside the square brackets [] is a negation, meaning it matches anything except <.
+            #
+            pattern = rf"([^<]+(?:<loc_\d+>|{re.escape(polygon_sep_token)}|{re.escape(polygon_start_token)}|{re.escape(polygon_end_token)}){{4,}})"
+        phrases = re.findall(pattern, text)
+
+        phrase_string_pattern = r'^\s*(.*?)(?=<od>|</od>|<box>|</box>|<bbox>|</bbox>|<loc_|<poly>)'
+        box_pattern =  rf'((?:<loc_\d+>)+)(?:{re.escape(polygon_sep_token)}|$)'
+
+        # one polygons instance is separated by polygon_start_token and polygon_end_token
+        polygons_instance_pattern = rf'{re.escape(polygon_start_token)}(.*?){re.escape(polygon_end_token)}'
+    
+        instances = []
+        for phrase_text in phrases:
+
+            # exclude loc_\d+>
+            # need to get span if want to include category score
+            phrase_text_strip = re.sub(r'^loc_\d+>', '', phrase_text, count=1)
+
+            # phrase = phrase.replace('<poly>', '')
+            # phrase = phrase.replace('poly>', '')
+
+            if phrase_text_strip == '' and not allow_empty_phrase:
+                continue
+
+
+            # parse phrase, get string 
+            phrase = re.search(phrase_string_pattern, phrase_text_strip)
+            if phrase is None:
+                continue
+            phrase = phrase.group()
+            # remove leading and trailing spaces
+            phrase = phrase.strip()
+
+            # parse bboxes by box_pattern
+
+            # split by polygon_start_token and polygon_end_token first using polygons_instance_pattern
+            if polygon_start_token in phrase_text and polygon_end_token in phrase_text:
+                polygons_instances_parsed = list(re.finditer(polygons_instance_pattern, phrase_text))
+            else:
+                polygons_instances_parsed = [phrase_text]
+
+            for _polygons_instances_parsed in polygons_instances_parsed:
+                # Prepare instance.
+                instance = {}
+
+                # polygons_parsed= list(re.finditer(box_pattern, phrase_text))
+                if isinstance(_polygons_instances_parsed, str): 
+                    polygons_parsed= list(re.finditer(box_pattern, _polygons_instances_parsed))
+                else:
+                    polygons_parsed= list(re.finditer(box_pattern, _polygons_instances_parsed.group(1)))
+                if len(polygons_parsed) == 0:
+                    continue
+
+                # a list of list (polygon)
+                bbox = []
+                polygons = []
+                for _polygon_parsed in polygons_parsed:
+                    # group 1: whole <loc_\d+>...</loc_\d+>
+                    _polygon = _polygon_parsed.group(1)
+                    # parse into list of int
+                    _polygon = [int(_loc_parsed.group(1)) for _loc_parsed in re.finditer(r'<loc_(\d+)>', _polygon)]
+                    if with_box_at_start and len(bbox) == 0:
+                        if len(_polygon) > 4:
+                            # no valid bbox prediction
+                            bbox = _polygon[:4]
+                            _polygon = _polygon[4:]
+                        else:
+                            bbox = [0, 0, 0, 0]
+                    # abandon last element if is not paired 
+                    if len(_polygon) % 2 == 1:
+                        _polygon = _polygon[:-1]
+                    
+                    # reshape into (n, 2)
+                    _polygon = self.coordinates_quantizer.dequantize(
+                        torch.tensor(np.array(_polygon).reshape(-1, 2)),
+                        size=image_size
+                    ).reshape(-1).tolist()
+                    # reshape back
+                    polygons.append(_polygon)
+
+                instance['cat_name'] = phrase
+                instance['polygons'] = polygons
+                if len(bbox) != 0:
+                    instance['bbox'] = self.box_quantizer.dequantize(
+                        boxes=torch.tensor([bbox]),
+                        size=image_size
+                    ).tolist()[0]  
+
+                instances.append(instance)
+
+        return instances
+
+    def __call__(
+        self,
+        text=None,
+        sequence=None,
+        transition_beam_score=None,
+        image_size=None,
+        parse_tasks=None,
+    ):
+        """
+        Args:
+            text: model outputs
+            image_size: (width, height)
+            parse_tasks: a list of tasks to parse, if None, parse all tasks.
+        """
+        if parse_tasks is not None:
+            if isinstance(parse_tasks, str):
+                parse_tasks = [parse_tasks]
+            for _parse_task in parse_tasks:
+                assert _parse_task in self.parse_tasks, f'parse task {_parse_task} not supported'
+        
+        # sequence or text should be provided 
+        assert sequence is not None or text is not None, 'sequence or text should be provided'
+        assert sequence is None or text is None, 'only one of sequence and text should be provided'
+
+        if sequence is not None:
+            sequence = sequence.tolist()[1:]
+            text, spans = self.decode_with_spans(self.tokenizer, sequence)
+            if transition_beam_score is not None:
+                transition_beam_score = transition_beam_score.tolist()
+                assert len(sequence) == len(transition_beam_score)
+        else:
+            spans = None
+            transition_beam_score = None
+
+        parsed_dict = {
+            'text': text
+        }
+
+        for task in self.parse_tasks:
+            if parse_tasks is not None and task not in parse_tasks:
+                continue
+
+            pattern = self.parse_tasks_configs[task].get('PATTERN', None)
+            score_mode = self.parse_tasks_configs[task].get('SCORE_MODE', None)
+
+            if task == 'ocr':
+                instances = self.parse_ocr_from_text_and_spans(
+                    text,
+                    pattern=pattern,
+                    image_size=image_size,
+                    area_threshold=self.parse_tasks_configs[task].get('AREA_THRESHOLD', 0.0),
+                )
+                parsed_dict['ocr'] = instances
+            elif task == 'phrase_grounding':
+                instances = self.parse_phrase_grounding_from_text_and_spans( 
+                    text,
+                    pattern=pattern,
+                    image_size=image_size,
+                )
+                parsed_dict['phrase_grounding'] = instances
+            elif task == 'pure_text':
+                parsed_dict['pure_text'] = text 
+            elif task == 'description_with_bboxes':
+                instances = self.parse_description_with_bboxes_from_text_and_spans( 
+                    text,
+                    spans=spans,
+                    scores=transition_beam_score,
+                    score_mode=score_mode,
+                    pattern=pattern,
+                    image_size=image_size,
+                )
+                parsed_dict['description_with_bboxes'] = instances
+            elif task == 'description_with_polygons':
+                instances = self.parse_description_with_polygons_from_text_and_spans( 
+                    text,
+                    pattern=pattern,
+                    image_size=image_size,
+                )
+                parsed_dict['description_with_polygons'] = instances
+            elif task == 'polygons':
+                instances = self.parse_description_with_polygons_from_text_and_spans( 
+                    text,
+                    pattern=pattern,
+                    image_size=image_size,
+                    allow_empty_phrase=True,
+                )
+                parsed_dict['polygons'] = instances
+            elif task == 'bboxes':
+                instances = self.parse_description_with_bboxes_from_text_and_spans( 
+                    text,
+                    pattern=pattern,
+                    image_size=image_size,
+                    allow_empty_phrase=True,
+                )
+                parsed_dict['bboxes'] = instances
+            elif task == 'description_with_bboxes_or_polygons':
+                if '<poly>' in text:
+                    # only support either polygons or bboxes, not both at the same time
+                    instances = self.parse_description_with_polygons_from_text_and_spans( 
+                        text,
+                        pattern=pattern,
+                        image_size=image_size,
+                    )
+                else:
+                    instances = self.parse_description_with_bboxes_from_text_and_spans( 
+                        text,
+                        pattern=pattern,
+                        image_size=image_size,
+                    )
+                parsed_dict['description_with_bboxes_or_polygons'] = instances
+            else:
+                raise ValueError("task {} is not supported".format(task))
+
+        return parsed_dict

LLM/Florence-2-base/tokenizer_config.json

@@ -0,0 +1,4 @@
+{
+    "model_max_length": 1024
+}
+

LLM/Qwen-VL/Qwen3-VL-2B-Thinking-FP8/config.json

@@ -0,0 +1,319 @@
+{
+  "architectures": [
+    "Qwen3VLForConditionalGeneration"
+  ],
+  "image_token_id": 151655,
+  "model_type": "qwen3_vl",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "intermediate_size": 6144,
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_vl_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 28,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-06,
+    "rope_scaling": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        24,
+        20,
+        20
+      ],
+      "rope_type": "default"
+    },
+    "rope_theta": 5000000,
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.0.dev0",
+  "video_token_id": 151656,
+  "vision_config": {
+    "deepstack_visual_indexes": [
+      5,
+      11,
+      17
+    ],
+    "depth": 24,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1024,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "qwen3_vl",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2048,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652,
+  "quantization_config": {
+    "activation_scheme": "dynamic",
+    "fmt": "e4m3",
+    "quant_method": "fp8",
+    "ignored_layers": [
+      "lm_head",
+      "model.visual.merger.linear_fc1",
+      "model.visual.merger.linear_fc2",
+      "model.visual.merger.norm",
+      "model.visual.patch_embed.proj",
+      "model.visual.pos_embed",
+      "visual.merger.linear_fc1",
+      "visual.merger.linear_fc2",
+      "visual.merger.norm",
+      "visual.patch_embed.proj",
+      "visual.pos_embed",
+      "model.visual.blocks.0.attn.proj",
+      "model.visual.blocks.0.attn.qkv",
+      "model.visual.blocks.0.mlp.linear_fc1",
+      "model.visual.blocks.0.mlp.linear_fc2",
+      "visual.blocks.0.attn.proj",
+      "visual.blocks.0.attn.qkv_proj",
+      "visual.blocks.0.mlp.linear_fc1",
+      "visual.blocks.0.mlp.linear_fc2",
+      "model.visual.blocks.1.attn.proj",
+      "model.visual.blocks.1.attn.qkv",
+      "model.visual.blocks.1.mlp.linear_fc1",
+      "model.visual.blocks.1.mlp.linear_fc2",
+      "visual.blocks.1.attn.proj",
+      "visual.blocks.1.attn.qkv_proj",
+      "visual.blocks.1.mlp.linear_fc1",
+      "visual.blocks.1.mlp.linear_fc2",
+      "model.visual.blocks.2.attn.proj",
+      "model.visual.blocks.2.attn.qkv",
+      "model.visual.blocks.2.mlp.linear_fc1",
+      "model.visual.blocks.2.mlp.linear_fc2",
+      "visual.blocks.2.attn.proj",
+      "visual.blocks.2.attn.qkv_proj",
+      "visual.blocks.2.mlp.linear_fc1",
+      "visual.blocks.2.mlp.linear_fc2",
+      "model.visual.blocks.3.attn.proj",
+      "model.visual.blocks.3.attn.qkv",
+      "model.visual.blocks.3.mlp.linear_fc1",
+      "model.visual.blocks.3.mlp.linear_fc2",
+      "visual.blocks.3.attn.proj",
+      "visual.blocks.3.attn.qkv_proj",
+      "visual.blocks.3.mlp.linear_fc1",
+      "visual.blocks.3.mlp.linear_fc2",
+      "model.visual.blocks.4.attn.proj",
+      "model.visual.blocks.4.attn.qkv",
+      "model.visual.blocks.4.mlp.linear_fc1",
+      "model.visual.blocks.4.mlp.linear_fc2",
+      "visual.blocks.4.attn.proj",
+      "visual.blocks.4.attn.qkv_proj",
+      "visual.blocks.4.mlp.linear_fc1",
+      "visual.blocks.4.mlp.linear_fc2",
+      "model.visual.blocks.5.attn.proj",
+      "model.visual.blocks.5.attn.qkv",
+      "model.visual.blocks.5.mlp.linear_fc1",
+      "model.visual.blocks.5.mlp.linear_fc2",
+      "visual.blocks.5.attn.proj",
+      "visual.blocks.5.attn.qkv_proj",
+      "visual.blocks.5.mlp.linear_fc1",
+      "visual.blocks.5.mlp.linear_fc2",
+      "model.visual.blocks.6.attn.proj",
+      "model.visual.blocks.6.attn.qkv",
+      "model.visual.blocks.6.mlp.linear_fc1",
+      "model.visual.blocks.6.mlp.linear_fc2",
+      "visual.blocks.6.attn.proj",
+      "visual.blocks.6.attn.qkv_proj",
+      "visual.blocks.6.mlp.linear_fc1",
+      "visual.blocks.6.mlp.linear_fc2",
+      "model.visual.blocks.7.attn.proj",
+      "model.visual.blocks.7.attn.qkv",
+      "model.visual.blocks.7.mlp.linear_fc1",
+      "model.visual.blocks.7.mlp.linear_fc2",
+      "visual.blocks.7.attn.proj",
+      "visual.blocks.7.attn.qkv_proj",
+      "visual.blocks.7.mlp.linear_fc1",
+      "visual.blocks.7.mlp.linear_fc2",
+      "model.visual.blocks.8.attn.proj",
+      "model.visual.blocks.8.attn.qkv",
+      "model.visual.blocks.8.mlp.linear_fc1",
+      "model.visual.blocks.8.mlp.linear_fc2",
+      "visual.blocks.8.attn.proj",
+      "visual.blocks.8.attn.qkv_proj",
+      "visual.blocks.8.mlp.linear_fc1",
+      "visual.blocks.8.mlp.linear_fc2",
+      "model.visual.blocks.9.attn.proj",
+      "model.visual.blocks.9.attn.qkv",
+      "model.visual.blocks.9.mlp.linear_fc1",
+      "model.visual.blocks.9.mlp.linear_fc2",
+      "visual.blocks.9.attn.proj",
+      "visual.blocks.9.attn.qkv_proj",
+      "visual.blocks.9.mlp.linear_fc1",
+      "visual.blocks.9.mlp.linear_fc2",
+      "model.visual.blocks.10.attn.proj",
+      "model.visual.blocks.10.attn.qkv",
+      "model.visual.blocks.10.mlp.linear_fc1",
+      "model.visual.blocks.10.mlp.linear_fc2",
+      "visual.blocks.10.attn.proj",
+      "visual.blocks.10.attn.qkv_proj",
+      "visual.blocks.10.mlp.linear_fc1",
+      "visual.blocks.10.mlp.linear_fc2",
+      "model.visual.blocks.11.attn.proj",
+      "model.visual.blocks.11.attn.qkv",
+      "model.visual.blocks.11.mlp.linear_fc1",
+      "model.visual.blocks.11.mlp.linear_fc2",
+      "visual.blocks.11.attn.proj",
+      "visual.blocks.11.attn.qkv_proj",
+      "visual.blocks.11.mlp.linear_fc1",
+      "visual.blocks.11.mlp.linear_fc2",
+      "model.visual.blocks.12.attn.proj",
+      "model.visual.blocks.12.attn.qkv",
+      "model.visual.blocks.12.mlp.linear_fc1",
+      "model.visual.blocks.12.mlp.linear_fc2",
+      "visual.blocks.12.attn.proj",
+      "visual.blocks.12.attn.qkv_proj",
+      "visual.blocks.12.mlp.linear_fc1",
+      "visual.blocks.12.mlp.linear_fc2",
+      "model.visual.blocks.13.attn.proj",
+      "model.visual.blocks.13.attn.qkv",
+      "model.visual.blocks.13.mlp.linear_fc1",
+      "model.visual.blocks.13.mlp.linear_fc2",
+      "visual.blocks.13.attn.proj",
+      "visual.blocks.13.attn.qkv_proj",
+      "visual.blocks.13.mlp.linear_fc1",
+      "visual.blocks.13.mlp.linear_fc2",
+      "model.visual.blocks.14.attn.proj",
+      "model.visual.blocks.14.attn.qkv",
+      "model.visual.blocks.14.mlp.linear_fc1",
+      "model.visual.blocks.14.mlp.linear_fc2",
+      "visual.blocks.14.attn.proj",
+      "visual.blocks.14.attn.qkv_proj",
+      "visual.blocks.14.mlp.linear_fc1",
+      "visual.blocks.14.mlp.linear_fc2",
+      "model.visual.blocks.15.attn.proj",
+      "model.visual.blocks.15.attn.qkv",
+      "model.visual.blocks.15.mlp.linear_fc1",
+      "model.visual.blocks.15.mlp.linear_fc2",
+      "visual.blocks.15.attn.proj",
+      "visual.blocks.15.attn.qkv_proj",
+      "visual.blocks.15.mlp.linear_fc1",
+      "visual.blocks.15.mlp.linear_fc2",
+      "model.visual.blocks.16.attn.proj",
+      "model.visual.blocks.16.attn.qkv",
+      "model.visual.blocks.16.mlp.linear_fc1",
+      "model.visual.blocks.16.mlp.linear_fc2",
+      "visual.blocks.16.attn.proj",
+      "visual.blocks.16.attn.qkv_proj",
+      "visual.blocks.16.mlp.linear_fc1",
+      "visual.blocks.16.mlp.linear_fc2",
+      "model.visual.blocks.17.attn.proj",
+      "model.visual.blocks.17.attn.qkv",
+      "model.visual.blocks.17.mlp.linear_fc1",
+      "model.visual.blocks.17.mlp.linear_fc2",
+      "visual.blocks.17.attn.proj",
+      "visual.blocks.17.attn.qkv_proj",
+      "visual.blocks.17.mlp.linear_fc1",
+      "visual.blocks.17.mlp.linear_fc2",
+      "model.visual.blocks.18.attn.proj",
+      "model.visual.blocks.18.attn.qkv",
+      "model.visual.blocks.18.mlp.linear_fc1",
+      "model.visual.blocks.18.mlp.linear_fc2",
+      "visual.blocks.18.attn.proj",
+      "visual.blocks.18.attn.qkv_proj",
+      "visual.blocks.18.mlp.linear_fc1",
+      "visual.blocks.18.mlp.linear_fc2",
+      "model.visual.blocks.19.attn.proj",
+      "model.visual.blocks.19.attn.qkv",
+      "model.visual.blocks.19.mlp.linear_fc1",
+      "model.visual.blocks.19.mlp.linear_fc2",
+      "visual.blocks.19.attn.proj",
+      "visual.blocks.19.attn.qkv_proj",
+      "visual.blocks.19.mlp.linear_fc1",
+      "visual.blocks.19.mlp.linear_fc2",
+      "model.visual.blocks.20.attn.proj",
+      "model.visual.blocks.20.attn.qkv",
+      "model.visual.blocks.20.mlp.linear_fc1",
+      "model.visual.blocks.20.mlp.linear_fc2",
+      "visual.blocks.20.attn.proj",
+      "visual.blocks.20.attn.qkv_proj",
+      "visual.blocks.20.mlp.linear_fc1",
+      "visual.blocks.20.mlp.linear_fc2",
+      "model.visual.blocks.21.attn.proj",
+      "model.visual.blocks.21.attn.qkv",
+      "model.visual.blocks.21.mlp.linear_fc1",
+      "model.visual.blocks.21.mlp.linear_fc2",
+      "visual.blocks.21.attn.proj",
+      "visual.blocks.21.attn.qkv_proj",
+      "visual.blocks.21.mlp.linear_fc1",
+      "visual.blocks.21.mlp.linear_fc2",
+      "model.visual.blocks.22.attn.proj",
+      "model.visual.blocks.22.attn.qkv",
+      "model.visual.blocks.22.mlp.linear_fc1",
+      "model.visual.blocks.22.mlp.linear_fc2",
+      "visual.blocks.22.attn.proj",
+      "visual.blocks.22.attn.qkv_proj",
+      "visual.blocks.22.mlp.linear_fc1",
+      "visual.blocks.22.mlp.linear_fc2",
+      "model.visual.blocks.23.attn.proj",
+      "model.visual.blocks.23.attn.qkv",
+      "model.visual.blocks.23.mlp.linear_fc1",
+      "model.visual.blocks.23.mlp.linear_fc2",
+      "visual.blocks.23.attn.proj",
+      "visual.blocks.23.attn.qkv_proj",
+      "visual.blocks.23.mlp.linear_fc1",
+      "visual.blocks.23.mlp.linear_fc2",
+      "model.visual.blocks.24.attn.proj",
+      "model.visual.blocks.24.attn.qkv",
+      "model.visual.blocks.24.mlp.linear_fc1",
+      "model.visual.blocks.24.mlp.linear_fc2",
+      "visual.blocks.24.attn.proj",
+      "visual.blocks.24.attn.qkv_proj",
+      "visual.blocks.24.mlp.linear_fc1",
+      "visual.blocks.24.mlp.linear_fc2",
+      "model.visual.blocks.25.attn.proj",
+      "model.visual.blocks.25.attn.qkv",
+      "model.visual.blocks.25.mlp.linear_fc1",
+      "model.visual.blocks.25.mlp.linear_fc2",
+      "visual.blocks.25.attn.proj",
+      "visual.blocks.25.attn.qkv_proj",
+      "visual.blocks.25.mlp.linear_fc1",
+      "visual.blocks.25.mlp.linear_fc2",
+      "model.visual.blocks.26.attn.proj",
+      "model.visual.blocks.26.attn.qkv",
+      "model.visual.blocks.26.mlp.linear_fc1",
+      "model.visual.blocks.26.mlp.linear_fc2",
+      "visual.blocks.26.attn.proj",
+      "visual.blocks.26.attn.qkv_proj",
+      "visual.blocks.26.mlp.linear_fc1",
+      "visual.blocks.26.mlp.linear_fc2",
+      "model.visual.deepstack_merger_list.0.linear_fc1",
+      "model.visual.deepstack_merger_list.0.linear_fc2",
+      "model.visual.deepstack_merger_list.0.norm",
+      "visual.deepstack_merger_list.0.linear_fc1",
+      "visual.deepstack_merger_list.0.linear_fc2",
+      "visual.deepstack_merger_list.0.norm",
+      "model.visual.deepstack_merger_list.1.linear_fc1",
+      "model.visual.deepstack_merger_list.1.linear_fc2",
+      "model.visual.deepstack_merger_list.1.norm",
+      "visual.deepstack_merger_list.1.linear_fc1",
+      "visual.deepstack_merger_list.1.linear_fc2",
+      "visual.deepstack_merger_list.1.norm",
+      "model.visual.deepstack_merger_list.2.linear_fc1",
+      "model.visual.deepstack_merger_list.2.linear_fc2",
+      "model.visual.deepstack_merger_list.2.norm",
+      "visual.deepstack_merger_list.2.linear_fc1",
+      "visual.deepstack_merger_list.2.linear_fc2",
+      "visual.deepstack_merger_list.2.norm"
+    ],
+    "weight_block_size": [
+      128,
+      128
+    ]
+  }
+}

LLM/Qwen-VL/Qwen3-VL-2B-Thinking-FP8/model.safetensors.index.json

@@ -0,0 +1,827 @@
+{
+  "metadata": {},
+  "weight_map": {
+    "model.language_model.layers.4.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.4.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.0.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.17.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.24.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.1.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.7.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.23.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.13.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.20.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.27.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.14.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.10.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.11.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.21.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.12.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.6.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.2.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "lm_head.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.embed_tokens.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.norm.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.0.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.1.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.10.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.11.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.12.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.13.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.14.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.15.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.16.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.17.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.18.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.19.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.2.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.20.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.21.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.22.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.23.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.3.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.4.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.5.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.6.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.7.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.8.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.attn.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.attn.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.attn.qkv.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.attn.qkv.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.norm1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.norm1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.norm2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.blocks.9.norm2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.0.norm.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.0.norm.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.1.norm.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.1.norm.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.2.norm.bias": "model-00001-of-00001.safetensors",
+    "model.visual.deepstack_merger_list.2.norm.weight": "model-00001-of-00001.safetensors",
+    "model.visual.merger.linear_fc1.bias": "model-00001-of-00001.safetensors",
+    "model.visual.merger.linear_fc1.weight": "model-00001-of-00001.safetensors",
+    "model.visual.merger.linear_fc2.bias": "model-00001-of-00001.safetensors",
+    "model.visual.merger.linear_fc2.weight": "model-00001-of-00001.safetensors",
+    "model.visual.merger.norm.bias": "model-00001-of-00001.safetensors",
+    "model.visual.merger.norm.weight": "model-00001-of-00001.safetensors",
+    "model.visual.patch_embed.proj.bias": "model-00001-of-00001.safetensors",
+    "model.visual.patch_embed.proj.weight": "model-00001-of-00001.safetensors",
+    "model.visual.pos_embed.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.8.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.15.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.5.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.25.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.18.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.9.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.22.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.3.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.19.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.16.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.input_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.mlp.down_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.mlp.down_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.mlp.gate_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.mlp.gate_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.mlp.up_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.mlp.up_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.post_attention_layernorm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.k_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.k_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.k_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.o_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.o_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.q_norm.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.q_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.q_proj.weight_scale_inv": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.v_proj.weight": "model-00001-of-00001.safetensors",
+    "model.language_model.layers.26.self_attn.v_proj.weight_scale_inv": "model-00001-of-00001.safetensors"
+  }
+}

LLM/Qwen-VL/Qwen3-VL-2B-Thinking-FP8/preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "patch_size": 16,
+  "temporal_patch_size": 2,
+  "merge_size": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "Qwen3VLProcessor",
+  "image_processor_type": "Qwen2VLImageProcessorFast"
+}

LLM/Qwen-VL/Qwen3-VL-2B-Thinking-FP8/tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- macro render_content(content, do_vision_count) %}\n    {%- if content is string %}\n        {{- content }}\n    {%- else %}\n        {%- for item in content %}\n            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}\n                {%- if do_vision_count %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                {%- endif %}\n                {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                <|vision_start|><|image_pad|><|vision_end|>\n            {%- elif 'video' in item or item.type == 'video' %}\n                {%- if do_vision_count %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                {%- endif %}\n                {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                <|vision_start|><|video_pad|><|vision_end|>\n            {%- elif 'text' in item %}\n                {{- item.text }}\n            {%- endif %}\n        {%- endfor %}\n    {%- endif %}\n{%- endmacro %}\n{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- render_content(messages[0].content, false) + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + render_content(messages[0].content, false) + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n    {%- set index = (messages|length - 1) - loop.index0 %}\n    {%- if ns.multi_step_tool and message.role == \"user\" %}\n        {%- set content = render_content(message.content, false) %}\n        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}\n            {%- set ns.multi_step_tool = false %}\n            {%- set ns.last_query_index = index %}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- set content = render_content(message.content, True) %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is string %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in content %}\n                {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n                {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n<think>\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

LLM/Qwen-VL/Qwen3-VL-2B-Thinking-FP8/video_preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+  "size": {
+    "longest_edge": 25165824,
+    "shortest_edge": 4096
+  },
+  "patch_size": 16,
+  "temporal_patch_size": 2,
+  "merge_size": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor_type": "Qwen3VLVideoProcessor"
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/README.md

@@ -0,0 +1,117 @@
+---
+license: apache-2.0
+base_model:
+- Qwen/Qwen3-VL-4B-Instruct
+pipeline_tag: image-text-to-text
+tags:
+- abliterated
+- uncensored
+library_name: transformers
+---
+
+# huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated-FP8
+
+
+This is an uncensored version of [Qwen/Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) created with abliteration (see [remove-refusals-with-transformers](https://github.com/Sumandora/remove-refusals-with-transformers) to know more about it).  
+
+It was only the text part that was processed, not the image part.
+
+The abliterated model will no longer say "I can’t describe or analyze this image."
+
+This FP8 version was converted from [huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated). For the forwarding method, refer to [finegrained_fp8](https://huggingface.co/docs/transformers/quantization/finegrained_fp8).
+
+## Chat with Image
+
+```
+from transformers import Qwen3VLForConditionalGeneration, AutoProcessor, BitsAndBytesConfig
+import os
+import torch
+
+cpu_count = os.cpu_count()
+print(f"Number of CPU cores in the system: {cpu_count}")
+half_cpu_count = cpu_count // 2
+os.environ["MKL_NUM_THREADS"] = str(half_cpu_count)
+os.environ["OMP_NUM_THREADS"] = str(half_cpu_count)
+torch.set_num_threads(half_cpu_count)
+
+MODEL_ID = "huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated-FP8"
+
+# default: Load the model on the available device(s)
+model = Qwen3VLForConditionalGeneration.from_pretrained(
+    MODEL_ID, 
+    device_map="auto", 
+    trust_remote_code=True,
+    dtype=torch.bfloat16,
+    low_cpu_mem_usage=True,
+)
+# We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.
+# model = Qwen3VLForConditionalGeneration.from_pretrained(
+#     "Qwen/Qwen3-VL-235B-A22B-Instruct",
+#     dtype=torch.bfloat16,
+#     attn_implementation="flash_attention_2",
+#     device_map="auto",
+# )
+
+processor = AutoProcessor.from_pretrained(MODEL_ID)
+
+
+image_path = "/png/cars.jpg"
+
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image", "image": f"{image_path}",
+            },
+            {"type": "text", "text": "Describe this image."},
+        ],
+    }
+]
+
+# Preparation for inference
+inputs = processor.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_dict=True,
+    return_tensors="pt"
+).to(model.device)
+
+# Inference: Generation of the output
+generated_ids = model.generate(**inputs, max_new_tokens=128)
+generated_ids_trimmed = [
+    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+]
+output_text = processor.batch_decode(
+    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
+)
+print(output_text)
+
+```
+
+
+### Usage Warnings
+
+
+ - **Risk of Sensitive or Controversial Outputs**: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.
+
+ - **Not Suitable for All Audiences**: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.
+
+ - **Legal and Ethical Responsibilities**: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.
+
+ - **Research and Experimental Use**: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.
+
+ - **Monitoring and Review Recommendations**: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.
+
+ - **No Default Safety Guarantees**: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.
+
+
+### Donation
+##### Your donation helps us continue our further development and improvement, a cup of coffee can do it.
+- bitcoin:
+```
+  bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge
+```
+- Support our work on [Ko-fi](https://ko-fi.com/huihuiai)!
+

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/chat_template.jinja

@@ -0,0 +1,120 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' }}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- for message in messages %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content_item in message.content %}
+                {%- if 'text' in content_item %}
+                    {{- content_item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and message.content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/config.json

@@ -0,0 +1,320 @@
+{
+  "architectures": [
+    "Qwen3VLForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "image_token_id": 151655,
+  "model_type": "qwen3_vl",
+  "quantization_config": {
+    "activation_scheme": "dynamic",
+    "modules_to_not_convert": [
+      "lm_head",
+      "model.visual.merger.linear_fc1",
+      "model.visual.merger.linear_fc2",
+      "model.visual.merger.norm",
+      "model.visual.patch_embed.proj",
+      "model.visual.pos_embed",
+      "visual.merger.linear_fc1",
+      "visual.merger.linear_fc2",
+      "visual.merger.norm",
+      "visual.patch_embed.proj",
+      "visual.pos_embed",
+      "model.visual.blocks.0.attn.proj",
+      "model.visual.blocks.0.attn.qkv",
+      "model.visual.blocks.0.mlp.linear_fc1",
+      "model.visual.blocks.0.mlp.linear_fc2",
+      "visual.blocks.0.attn.proj",
+      "visual.blocks.0.attn.qkv_proj",
+      "visual.blocks.0.mlp.linear_fc1",
+      "visual.blocks.0.mlp.linear_fc2",
+      "model.visual.blocks.1.attn.proj",
+      "model.visual.blocks.1.attn.qkv",
+      "model.visual.blocks.1.mlp.linear_fc1",
+      "model.visual.blocks.1.mlp.linear_fc2",
+      "visual.blocks.1.attn.proj",
+      "visual.blocks.1.attn.qkv_proj",
+      "visual.blocks.1.mlp.linear_fc1",
+      "visual.blocks.1.mlp.linear_fc2",
+      "model.visual.blocks.2.attn.proj",
+      "model.visual.blocks.2.attn.qkv",
+      "model.visual.blocks.2.mlp.linear_fc1",
+      "model.visual.blocks.2.mlp.linear_fc2",
+      "visual.blocks.2.attn.proj",
+      "visual.blocks.2.attn.qkv_proj",
+      "visual.blocks.2.mlp.linear_fc1",
+      "visual.blocks.2.mlp.linear_fc2",
+      "model.visual.blocks.3.attn.proj",
+      "model.visual.blocks.3.attn.qkv",
+      "model.visual.blocks.3.mlp.linear_fc1",
+      "model.visual.blocks.3.mlp.linear_fc2",
+      "visual.blocks.3.attn.proj",
+      "visual.blocks.3.attn.qkv_proj",
+      "visual.blocks.3.mlp.linear_fc1",
+      "visual.blocks.3.mlp.linear_fc2",
+      "model.visual.blocks.4.attn.proj",
+      "model.visual.blocks.4.attn.qkv",
+      "model.visual.blocks.4.mlp.linear_fc1",
+      "model.visual.blocks.4.mlp.linear_fc2",
+      "visual.blocks.4.attn.proj",
+      "visual.blocks.4.attn.qkv_proj",
+      "visual.blocks.4.mlp.linear_fc1",
+      "visual.blocks.4.mlp.linear_fc2",
+      "model.visual.blocks.5.attn.proj",
+      "model.visual.blocks.5.attn.qkv",
+      "model.visual.blocks.5.mlp.linear_fc1",
+      "model.visual.blocks.5.mlp.linear_fc2",
+      "visual.blocks.5.attn.proj",
+      "visual.blocks.5.attn.qkv_proj",
+      "visual.blocks.5.mlp.linear_fc1",
+      "visual.blocks.5.mlp.linear_fc2",
+      "model.visual.blocks.6.attn.proj",
+      "model.visual.blocks.6.attn.qkv",
+      "model.visual.blocks.6.mlp.linear_fc1",
+      "model.visual.blocks.6.mlp.linear_fc2",
+      "visual.blocks.6.attn.proj",
+      "visual.blocks.6.attn.qkv_proj",
+      "visual.blocks.6.mlp.linear_fc1",
+      "visual.blocks.6.mlp.linear_fc2",
+      "model.visual.blocks.7.attn.proj",
+      "model.visual.blocks.7.attn.qkv",
+      "model.visual.blocks.7.mlp.linear_fc1",
+      "model.visual.blocks.7.mlp.linear_fc2",
+      "visual.blocks.7.attn.proj",
+      "visual.blocks.7.attn.qkv_proj",
+      "visual.blocks.7.mlp.linear_fc1",
+      "visual.blocks.7.mlp.linear_fc2",
+      "model.visual.blocks.8.attn.proj",
+      "model.visual.blocks.8.attn.qkv",
+      "model.visual.blocks.8.mlp.linear_fc1",
+      "model.visual.blocks.8.mlp.linear_fc2",
+      "visual.blocks.8.attn.proj",
+      "visual.blocks.8.attn.qkv_proj",
+      "visual.blocks.8.mlp.linear_fc1",
+      "visual.blocks.8.mlp.linear_fc2",
+      "model.visual.blocks.9.attn.proj",
+      "model.visual.blocks.9.attn.qkv",
+      "model.visual.blocks.9.mlp.linear_fc1",
+      "model.visual.blocks.9.mlp.linear_fc2",
+      "visual.blocks.9.attn.proj",
+      "visual.blocks.9.attn.qkv_proj",
+      "visual.blocks.9.mlp.linear_fc1",
+      "visual.blocks.9.mlp.linear_fc2",
+      "model.visual.blocks.10.attn.proj",
+      "model.visual.blocks.10.attn.qkv",
+      "model.visual.blocks.10.mlp.linear_fc1",
+      "model.visual.blocks.10.mlp.linear_fc2",
+      "visual.blocks.10.attn.proj",
+      "visual.blocks.10.attn.qkv_proj",
+      "visual.blocks.10.mlp.linear_fc1",
+      "visual.blocks.10.mlp.linear_fc2",
+      "model.visual.blocks.11.attn.proj",
+      "model.visual.blocks.11.attn.qkv",
+      "model.visual.blocks.11.mlp.linear_fc1",
+      "model.visual.blocks.11.mlp.linear_fc2",
+      "visual.blocks.11.attn.proj",
+      "visual.blocks.11.attn.qkv_proj",
+      "visual.blocks.11.mlp.linear_fc1",
+      "visual.blocks.11.mlp.linear_fc2",
+      "model.visual.blocks.12.attn.proj",
+      "model.visual.blocks.12.attn.qkv",
+      "model.visual.blocks.12.mlp.linear_fc1",
+      "model.visual.blocks.12.mlp.linear_fc2",
+      "visual.blocks.12.attn.proj",
+      "visual.blocks.12.attn.qkv_proj",
+      "visual.blocks.12.mlp.linear_fc1",
+      "visual.blocks.12.mlp.linear_fc2",
+      "model.visual.blocks.13.attn.proj",
+      "model.visual.blocks.13.attn.qkv",
+      "model.visual.blocks.13.mlp.linear_fc1",
+      "model.visual.blocks.13.mlp.linear_fc2",
+      "visual.blocks.13.attn.proj",
+      "visual.blocks.13.attn.qkv_proj",
+      "visual.blocks.13.mlp.linear_fc1",
+      "visual.blocks.13.mlp.linear_fc2",
+      "model.visual.blocks.14.attn.proj",
+      "model.visual.blocks.14.attn.qkv",
+      "model.visual.blocks.14.mlp.linear_fc1",
+      "model.visual.blocks.14.mlp.linear_fc2",
+      "visual.blocks.14.attn.proj",
+      "visual.blocks.14.attn.qkv_proj",
+      "visual.blocks.14.mlp.linear_fc1",
+      "visual.blocks.14.mlp.linear_fc2",
+      "model.visual.blocks.15.attn.proj",
+      "model.visual.blocks.15.attn.qkv",
+      "model.visual.blocks.15.mlp.linear_fc1",
+      "model.visual.blocks.15.mlp.linear_fc2",
+      "visual.blocks.15.attn.proj",
+      "visual.blocks.15.attn.qkv_proj",
+      "visual.blocks.15.mlp.linear_fc1",
+      "visual.blocks.15.mlp.linear_fc2",
+      "model.visual.blocks.16.attn.proj",
+      "model.visual.blocks.16.attn.qkv",
+      "model.visual.blocks.16.mlp.linear_fc1",
+      "model.visual.blocks.16.mlp.linear_fc2",
+      "visual.blocks.16.attn.proj",
+      "visual.blocks.16.attn.qkv_proj",
+      "visual.blocks.16.mlp.linear_fc1",
+      "visual.blocks.16.mlp.linear_fc2",
+      "model.visual.blocks.17.attn.proj",
+      "model.visual.blocks.17.attn.qkv",
+      "model.visual.blocks.17.mlp.linear_fc1",
+      "model.visual.blocks.17.mlp.linear_fc2",
+      "visual.blocks.17.attn.proj",
+      "visual.blocks.17.attn.qkv_proj",
+      "visual.blocks.17.mlp.linear_fc1",
+      "visual.blocks.17.mlp.linear_fc2",
+      "model.visual.blocks.18.attn.proj",
+      "model.visual.blocks.18.attn.qkv",
+      "model.visual.blocks.18.mlp.linear_fc1",
+      "model.visual.blocks.18.mlp.linear_fc2",
+      "visual.blocks.18.attn.proj",
+      "visual.blocks.18.attn.qkv_proj",
+      "visual.blocks.18.mlp.linear_fc1",
+      "visual.blocks.18.mlp.linear_fc2",
+      "model.visual.blocks.19.attn.proj",
+      "model.visual.blocks.19.attn.qkv",
+      "model.visual.blocks.19.mlp.linear_fc1",
+      "model.visual.blocks.19.mlp.linear_fc2",
+      "visual.blocks.19.attn.proj",
+      "visual.blocks.19.attn.qkv_proj",
+      "visual.blocks.19.mlp.linear_fc1",
+      "visual.blocks.19.mlp.linear_fc2",
+      "model.visual.blocks.20.attn.proj",
+      "model.visual.blocks.20.attn.qkv",
+      "model.visual.blocks.20.mlp.linear_fc1",
+      "model.visual.blocks.20.mlp.linear_fc2",
+      "visual.blocks.20.attn.proj",
+      "visual.blocks.20.attn.qkv_proj",
+      "visual.blocks.20.mlp.linear_fc1",
+      "visual.blocks.20.mlp.linear_fc2",
+      "model.visual.blocks.21.attn.proj",
+      "model.visual.blocks.21.attn.qkv",
+      "model.visual.blocks.21.mlp.linear_fc1",
+      "model.visual.blocks.21.mlp.linear_fc2",
+      "visual.blocks.21.attn.proj",
+      "visual.blocks.21.attn.qkv_proj",
+      "visual.blocks.21.mlp.linear_fc1",
+      "visual.blocks.21.mlp.linear_fc2",
+      "model.visual.blocks.22.attn.proj",
+      "model.visual.blocks.22.attn.qkv",
+      "model.visual.blocks.22.mlp.linear_fc1",
+      "model.visual.blocks.22.mlp.linear_fc2",
+      "visual.blocks.22.attn.proj",
+      "visual.blocks.22.attn.qkv_proj",
+      "visual.blocks.22.mlp.linear_fc1",
+      "visual.blocks.22.mlp.linear_fc2",
+      "model.visual.blocks.23.attn.proj",
+      "model.visual.blocks.23.attn.qkv",
+      "model.visual.blocks.23.mlp.linear_fc1",
+      "model.visual.blocks.23.mlp.linear_fc2",
+      "visual.blocks.23.attn.proj",
+      "visual.blocks.23.attn.qkv_proj",
+      "visual.blocks.23.mlp.linear_fc1",
+      "visual.blocks.23.mlp.linear_fc2",
+      "model.visual.blocks.24.attn.proj",
+      "model.visual.blocks.24.attn.qkv",
+      "model.visual.blocks.24.mlp.linear_fc1",
+      "model.visual.blocks.24.mlp.linear_fc2",
+      "visual.blocks.24.attn.proj",
+      "visual.blocks.24.attn.qkv_proj",
+      "visual.blocks.24.mlp.linear_fc1",
+      "visual.blocks.24.mlp.linear_fc2",
+      "model.visual.blocks.25.attn.proj",
+      "model.visual.blocks.25.attn.qkv",
+      "model.visual.blocks.25.mlp.linear_fc1",
+      "model.visual.blocks.25.mlp.linear_fc2",
+      "visual.blocks.25.attn.proj",
+      "visual.blocks.25.attn.qkv_proj",
+      "visual.blocks.25.mlp.linear_fc1",
+      "visual.blocks.25.mlp.linear_fc2",
+      "model.visual.blocks.26.attn.proj",
+      "model.visual.blocks.26.attn.qkv",
+      "model.visual.blocks.26.mlp.linear_fc1",
+      "model.visual.blocks.26.mlp.linear_fc2",
+      "visual.blocks.26.attn.proj",
+      "visual.blocks.26.attn.qkv_proj",
+      "visual.blocks.26.mlp.linear_fc1",
+      "visual.blocks.26.mlp.linear_fc2",
+      "model.visual.deepstack_merger_list.0.linear_fc1",
+      "model.visual.deepstack_merger_list.0.linear_fc2",
+      "model.visual.deepstack_merger_list.0.norm",
+      "visual.deepstack_merger_list.0.linear_fc1",
+      "visual.deepstack_merger_list.0.linear_fc2",
+      "visual.deepstack_merger_list.0.norm",
+      "model.visual.deepstack_merger_list.1.linear_fc1",
+      "model.visual.deepstack_merger_list.1.linear_fc2",
+      "model.visual.deepstack_merger_list.1.norm",
+      "visual.deepstack_merger_list.1.linear_fc1",
+      "visual.deepstack_merger_list.1.linear_fc2",
+      "visual.deepstack_merger_list.1.norm",
+      "model.visual.deepstack_merger_list.2.linear_fc1",
+      "model.visual.deepstack_merger_list.2.linear_fc2",
+      "model.visual.deepstack_merger_list.2.norm",
+      "visual.deepstack_merger_list.2.linear_fc1",
+      "visual.deepstack_merger_list.2.linear_fc2",
+      "visual.deepstack_merger_list.2.norm"
+    ],
+    "quant_method": "fp8",
+    "weight_block_size": [
+      128,
+      128
+    ]
+  },
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 2560,
+    "initializer_range": 0.02,
+    "intermediate_size": 9728,
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_vl_text",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 36,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-06,
+    "rope_scaling": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        24,
+        20,
+        20
+      ],
+      "rope_type": "default"
+    },
+    "rope_theta": 5000000,
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.0.dev0",
+  "video_token_id": 151656,
+  "vision_config": {
+    "deepstack_visual_indexes": [
+      5,
+      11,
+      17
+    ],
+    "depth": 24,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1024,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "qwen3_vl",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 2560,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.57.0.dev0"
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/model.safetensors.index.json

@@ -0,0 +1,973 @@
+{
+  "metadata": {
+    "total_parameters": 4438037568,
+    "total_size": 5242759296
+  },
+  "weight_map": {
+    "model.language_model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.0.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.1.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.10.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.11.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.12.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.13.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.14.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.15.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.16.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.17.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.18.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.19.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.2.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.20.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.21.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.22.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.23.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.24.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.25.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.26.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.27.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.28.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.29.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.3.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.30.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.31.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.32.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.33.mlp.down_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.33.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.33.mlp.up_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.33.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.mlp.down_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.mlp.gate_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.mlp.up_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.k_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.o_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.q_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.34.self_attn.v_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.mlp.down_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.mlp.gate_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.mlp.up_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.k_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.o_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.q_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.35.self_attn.v_proj.weight_scale_inv": "model-00002-of-00002.safetensors",
+    "model.language_model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.4.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.5.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.6.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.7.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.8.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.mlp.down_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.mlp.gate_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.mlp.up_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.k_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.o_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.q_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.language_model.layers.9.self_attn.v_proj.weight_scale_inv": "model-00001-of-00002.safetensors",
+    "model.language_model.norm.weight": "model-00002-of-00002.safetensors",
+    "model.visual.blocks.0.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.0.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.1.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.10.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.11.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.12.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.13.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.14.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.15.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.16.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.17.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.18.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.19.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.2.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.20.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.21.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.22.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.23.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.3.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.4.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.5.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.6.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.7.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.8.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.attn.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.attn.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.attn.qkv.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.attn.qkv.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.mlp.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.norm1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.norm1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.norm2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.blocks.9.norm2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.0.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.0.norm.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.0.norm.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.1.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.1.norm.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.1.norm.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.2.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.2.norm.bias": "model-00001-of-00002.safetensors",
+    "model.visual.deepstack_merger_list.2.norm.weight": "model-00001-of-00002.safetensors",
+    "model.visual.merger.linear_fc1.bias": "model-00001-of-00002.safetensors",
+    "model.visual.merger.linear_fc1.weight": "model-00001-of-00002.safetensors",
+    "model.visual.merger.linear_fc2.bias": "model-00001-of-00002.safetensors",
+    "model.visual.merger.linear_fc2.weight": "model-00001-of-00002.safetensors",
+    "model.visual.merger.norm.bias": "model-00001-of-00002.safetensors",
+    "model.visual.merger.norm.weight": "model-00001-of-00002.safetensors",
+    "model.visual.patch_embed.proj.bias": "model-00001-of-00002.safetensors",
+    "model.visual.patch_embed.proj.weight": "model-00001-of-00002.safetensors",
+    "model.visual.pos_embed.weight": "model-00001-of-00002.safetensors"
+  }
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/preprocessor_config.json

@@ -0,0 +1,39 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_pad": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_pixels": null,
+  "merge_size": 2,
+  "min_pixels": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "temporal_patch_size": 2
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/tokenizer_config.json

@@ -0,0 +1,240 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

LLM/Qwen3-VL-4B-Instruct-abliterated-FP8/video_preprocessor_config.json

@@ -0,0 +1,41 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "do_sample_frames": true,
+  "fps": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_frames": 768,
+  "merge_size": 2,
+  "min_frames": 4,
+  "num_frames": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_metadata": false,
+  "size": {
+    "longest_edge": 25165824,
+    "shortest_edge": 4096
+  },
+  "temporal_patch_size": 2,
+  "video_metadata": null,
+  "video_processor_type": "Qwen3VLVideoProcessor"
+}

RMBG/RMBG-2.0/BiRefNet_config.py

@@ -0,0 +1,11 @@
+from transformers import PretrainedConfig
+
+class BiRefNetConfig(PretrainedConfig):
+    model_type = "SegformerForSemanticSegmentation"
+    def __init__(
+        self,
+        bb_pretrained=False,
+        **kwargs
+    ):
+        self.bb_pretrained = bb_pretrained
+        super().__init__(**kwargs)

RMBG/RMBG-2.0/birefnet.py

@@ -0,0 +1,2244 @@
+### config.py
+
+import os
+import math
+
+
+class Config():
+    def __init__(self) -> None:
+        # PATH settings
+        self.sys_home_dir = os.path.expanduser('~')    # Make up your file system as: SYS_HOME_DIR/codes/dis/BiRefNet, SYS_HOME_DIR/datasets/dis/xx, SYS_HOME_DIR/weights/xx
+
+        # TASK settings
+        self.task = ['DIS5K', 'COD', 'HRSOD', 'DIS5K+HRSOD+HRS10K', 'P3M-10k'][0]
+        self.training_set = {
+            'DIS5K': ['DIS-TR', 'DIS-TR+DIS-TE1+DIS-TE2+DIS-TE3+DIS-TE4'][0],
+            'COD': 'TR-COD10K+TR-CAMO',
+            'HRSOD': ['TR-DUTS', 'TR-HRSOD', 'TR-UHRSD', 'TR-DUTS+TR-HRSOD', 'TR-DUTS+TR-UHRSD', 'TR-HRSOD+TR-UHRSD', 'TR-DUTS+TR-HRSOD+TR-UHRSD'][5],
+            'DIS5K+HRSOD+HRS10K': 'DIS-TE1+DIS-TE2+DIS-TE3+DIS-TE4+DIS-TR+TE-HRS10K+TE-HRSOD+TE-UHRSD+TR-HRS10K+TR-HRSOD+TR-UHRSD',     # leave DIS-VD for evaluation.
+            'P3M-10k': 'TR-P3M-10k',
+        }[self.task]
+        self.prompt4loc = ['dense', 'sparse'][0]
+
+        # Faster-Training settings
+        self.load_all = True
+        self.compile = True     # 1. Trigger CPU memory leak in some extend, which is an inherent problem of PyTorch.
+                                #   Machines with > 70GB CPU memory can run the whole training on DIS5K with default setting.
+                                # 2. Higher PyTorch version may fix it: https://github.com/pytorch/pytorch/issues/119607.
+                                # 3. But compile in Pytorch > 2.0.1 seems to bring no acceleration for training.
+        self.precisionHigh = True
+
+        # MODEL settings
+        self.ms_supervision = True
+        self.out_ref = self.ms_supervision and True
+        self.dec_ipt = True
+        self.dec_ipt_split = True
+        self.cxt_num = [0, 3][1]    # multi-scale skip connections from encoder
+        self.mul_scl_ipt = ['', 'add', 'cat'][2]
+        self.dec_att = ['', 'ASPP', 'ASPPDeformable'][2]
+        self.squeeze_block = ['', 'BasicDecBlk_x1', 'ResBlk_x4', 'ASPP_x3', 'ASPPDeformable_x3'][1]
+        self.dec_blk = ['BasicDecBlk', 'ResBlk', 'HierarAttDecBlk'][0]
+
+        # TRAINING settings
+        self.batch_size = 4
+        self.IoU_finetune_last_epochs = [
+            0,
+            {
+                'DIS5K': -50,
+                'COD': -20,
+                'HRSOD': -20,
+                'DIS5K+HRSOD+HRS10K': -20,
+                'P3M-10k': -20,
+            }[self.task]
+        ][1]    # choose 0 to skip
+        self.lr = (1e-4 if 'DIS5K' in self.task else 1e-5) * math.sqrt(self.batch_size / 4)     # DIS needs high lr to converge faster. Adapt the lr linearly
+        self.size = 1024
+        self.num_workers = max(4, self.batch_size)          # will be decrease to min(it, batch_size) at the initialization of the data_loader
+
+        # Backbone settings
+        self.bb = [
+            'vgg16', 'vgg16bn', 'resnet50',         # 0, 1, 2
+            'swin_v1_t', 'swin_v1_s',               # 3, 4
+            'swin_v1_b', 'swin_v1_l',               # 5-bs9, 6-bs4
+            'pvt_v2_b0', 'pvt_v2_b1',               # 7, 8
+            'pvt_v2_b2', 'pvt_v2_b5',               # 9-bs10, 10-bs5
+        ][6]
+        self.lateral_channels_in_collection = {
+            'vgg16': [512, 256, 128, 64], 'vgg16bn': [512, 256, 128, 64], 'resnet50': [1024, 512, 256, 64],
+            'pvt_v2_b2': [512, 320, 128, 64], 'pvt_v2_b5': [512, 320, 128, 64],
+            'swin_v1_b': [1024, 512, 256, 128], 'swin_v1_l': [1536, 768, 384, 192],
+            'swin_v1_t': [768, 384, 192, 96], 'swin_v1_s': [768, 384, 192, 96],
+            'pvt_v2_b0': [256, 160, 64, 32], 'pvt_v2_b1': [512, 320, 128, 64],
+        }[self.bb]
+        if self.mul_scl_ipt == 'cat':
+            self.lateral_channels_in_collection = [channel * 2 for channel in self.lateral_channels_in_collection]
+        self.cxt = self.lateral_channels_in_collection[1:][::-1][-self.cxt_num:] if self.cxt_num else []
+
+        # MODEL settings - inactive
+        self.lat_blk = ['BasicLatBlk'][0]
+        self.dec_channels_inter = ['fixed', 'adap'][0]
+        self.refine = ['', 'itself', 'RefUNet', 'Refiner', 'RefinerPVTInChannels4'][0]
+        self.progressive_ref = self.refine and True
+        self.ender = self.progressive_ref and False
+        self.scale = self.progressive_ref and 2
+        self.auxiliary_classification = False       # Only for DIS5K, where class labels are saved in `dataset.py`.
+        self.refine_iteration = 1
+        self.freeze_bb = False
+        self.model = [
+            'BiRefNet',
+        ][0]
+        if self.dec_blk == 'HierarAttDecBlk':
+            self.batch_size = 2 ** [0, 1, 2, 3, 4][2]
+
+        # TRAINING settings - inactive
+        self.preproc_methods = ['flip', 'enhance', 'rotate', 'pepper', 'crop'][:4]
+        self.optimizer = ['Adam', 'AdamW'][1]
+        self.lr_decay_epochs = [1e5]    # Set to negative N to decay the lr in the last N-th epoch.
+        self.lr_decay_rate = 0.5
+        # Loss
+        self.lambdas_pix_last = {
+            # not 0 means opening this loss
+            # original rate -- 1 : 30 : 1.5 : 0.2, bce x 30
+            'bce': 30 * 1,          # high performance
+            'iou': 0.5 * 1,         # 0 / 255
+            'iou_patch': 0.5 * 0,   # 0 / 255, win_size = (64, 64)
+            'mse': 150 * 0,         # can smooth the saliency map
+            'triplet': 3 * 0,
+            'reg': 100 * 0,
+            'ssim': 10 * 1,          # help contours,
+            'cnt': 5 * 0,          # help contours
+            'structure': 5 * 0,    # structure loss from codes of MVANet. A little improvement on DIS-TE[1,2,3], a bit more decrease on DIS-TE4.
+        }
+        self.lambdas_cls = {
+            'ce': 5.0
+        }
+        # Adv
+        self.lambda_adv_g = 10. * 0        # turn to 0 to avoid adv training
+        self.lambda_adv_d = 3. * (self.lambda_adv_g > 0)
+
+        # PATH settings - inactive
+        self.data_root_dir = os.path.join(self.sys_home_dir, 'datasets/dis')
+        self.weights_root_dir = os.path.join(self.sys_home_dir, 'weights')
+        self.weights = {
+            'pvt_v2_b2': os.path.join(self.weights_root_dir, 'pvt_v2_b2.pth'),
+            'pvt_v2_b5': os.path.join(self.weights_root_dir, ['pvt_v2_b5.pth', 'pvt_v2_b5_22k.pth'][0]),
+            'swin_v1_b': os.path.join(self.weights_root_dir, ['swin_base_patch4_window12_384_22kto1k.pth', 'swin_base_patch4_window12_384_22k.pth'][0]),
+            'swin_v1_l': os.path.join(self.weights_root_dir, ['swin_large_patch4_window12_384_22kto1k.pth', 'swin_large_patch4_window12_384_22k.pth'][0]),
+            'swin_v1_t': os.path.join(self.weights_root_dir, ['swin_tiny_patch4_window7_224_22kto1k_finetune.pth'][0]),
+            'swin_v1_s': os.path.join(self.weights_root_dir, ['swin_small_patch4_window7_224_22kto1k_finetune.pth'][0]),
+            'pvt_v2_b0': os.path.join(self.weights_root_dir, ['pvt_v2_b0.pth'][0]),
+            'pvt_v2_b1': os.path.join(self.weights_root_dir, ['pvt_v2_b1.pth'][0]),
+        }
+
+        # Callbacks - inactive
+        self.verbose_eval = True
+        self.only_S_MAE = False
+        self.use_fp16 = False   # Bugs. It may cause nan in training.
+        self.SDPA_enabled = False    # Bugs. Slower and errors occur in multi-GPUs
+
+        # others
+        self.device = [0, 'cpu'][0]     # .to(0) == .to('cuda:0')
+
+        self.batch_size_valid = 1
+        self.rand_seed = 7
+        # run_sh_file = [f for f in os.listdir('.') if 'train.sh' == f] + [os.path.join('..', f) for f in os.listdir('..') if 'train.sh' == f]
+        # with open(run_sh_file[0], 'r') as f:
+        #     lines = f.readlines()
+        #     self.save_last = int([l.strip() for l in lines if '"{}")'.format(self.task) in l and 'val_last=' in l][0].split('val_last=')[-1].split()[0])
+        #     self.save_step = int([l.strip() for l in lines if '"{}")'.format(self.task) in l and 'step=' in l][0].split('step=')[-1].split()[0])
+        # self.val_step = [0, self.save_step][0]
+
+    def print_task(self) -> None:
+        # Return task for choosing settings in shell scripts.
+        print(self.task)
+
+
+
+### models/backbones/pvt_v2.py
+
+import torch
+import torch.nn as nn
+from functools import partial
+
+from timm.models.layers import DropPath, to_2tuple, trunc_normal_
+from timm.models.registry import register_model
+
+import math
+
+# from config import Config
+
+# config = Config()
+
+class Mlp(nn.Module):
+    def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.):
+        super().__init__()
+        out_features = out_features or in_features
+        hidden_features = hidden_features or in_features
+        self.fc1 = nn.Linear(in_features, hidden_features)
+        self.dwconv = DWConv(hidden_features)
+        self.act = act_layer()
+        self.fc2 = nn.Linear(hidden_features, out_features)
+        self.drop = nn.Dropout(drop)
+
+        self.apply(self._init_weights)
+
+    def _init_weights(self, m):
+        if isinstance(m, nn.Linear):
+            trunc_normal_(m.weight, std=.02)
+            if isinstance(m, nn.Linear) and m.bias is not None:
+                nn.init.constant_(m.bias, 0)
+        elif isinstance(m, nn.LayerNorm):
+            nn.init.constant_(m.bias, 0)
+            nn.init.constant_(m.weight, 1.0)
+        elif isinstance(m, nn.Conv2d):
+            fan_out = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
+            fan_out //= m.groups
+            m.weight.data.normal_(0, math.sqrt(2.0 / fan_out))
+            if m.bias is not None:
+                m.bias.data.zero_()
+
+    def forward(self, x, H, W):
+        x = self.fc1(x)
+        x = self.dwconv(x, H, W)
+        x = self.act(x)
+        x = self.drop(x)
+        x = self.fc2(x)
+        x = self.drop(x)
+        return x
+
+
+class Attention(nn.Module):
+    def __init__(self, dim, num_heads=8, qkv_bias=False, qk_scale=None, attn_drop=0., proj_drop=0., sr_ratio=1):
+        super().__init__()
+        assert dim % num_heads == 0, f"dim {dim} should be divided by num_heads {num_heads}."
+
+        self.dim = dim
+        self.num_heads = num_heads
+        head_dim = dim // num_heads
+        self.scale = qk_scale or head_dim ** -0.5
+
+        self.q = nn.Linear(dim, dim, bias=qkv_bias)
+        self.kv = nn.Linear(dim, dim * 2, bias=qkv_bias)
+        self.attn_drop_prob = attn_drop
+        self.attn_drop = nn.Dropout(attn_drop)
+        self.proj = nn.Linear(dim, dim)
+        self.proj_drop = nn.Dropout(proj_drop)
+
+        self.sr_ratio = sr_ratio
+        if sr_ratio > 1:
+            self.sr = nn.Conv2d(dim, dim, kernel_size=sr_ratio, stride=sr_ratio)
+            self.norm = nn.LayerNorm(dim)
+
+        self.apply(self._init_weights)
+
+    def _init_weights(self, m):
+        if isinstance(m, nn.Linear):
+            trunc_normal_(m.weight, std=.02)
+            if isinstance(m, nn.Linear) and m.bias is not None:
+                nn.init.constant_(m.bias, 0)
+        elif isinstance(m, nn.LayerNorm):
+            nn.init.constant_(m.bias, 0)
+            nn.init.constant_(m.weight, 1.0)
+        elif isinstance(m, nn.Conv2d):
+            fan_out = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
+            fan_out //= m.groups
+            m.weight.data.normal_(0, math.sqrt(2.0 / fan_out))
+            if m.bias is not None:
+                m.bias.data.zero_()
+
+    def forward(self, x, H, W):
+        B, N, C = x.shape
+        q = self.q(x).reshape(B, N, self.num_heads, C // self.num_heads).permute(0, 2, 1, 3)
+
+        if self.sr_ratio > 1:
+            x_ = x.permute(0, 2, 1).reshape(B, C, H, W)
+            x_ = self.sr(x_).reshape(B, C, -1).permute(0, 2, 1)
+            x_ = self.norm(x_)
+            kv = self.kv(x_).reshape(B, -1, 2, self.num_heads, C // self.num_heads).permute(2, 0, 3, 1, 4)
+        else:
+            kv = self.kv(x).reshape(B, -1, 2, self.num_heads, C // self.num_heads).permute(2, 0, 3, 1, 4)
+        k, v = kv[0], kv[1]
+
+        if config.SDPA_enabled:
+            x = torch.nn.functional.scaled_dot_product_attention(
+                q, k, v,
+                attn_mask=None, dropout_p=self.attn_drop_prob, is_causal=False
+            ).transpose(1, 2).reshape(B, N, C)
+        else:
+            attn = (q @ k.transpose(-2, -1)) * self.scale
+            attn = attn.softmax(dim=-1)
+            attn = self.attn_drop(attn)
+
+            x = (attn @ v).transpose(1, 2).reshape(B, N, C)
+        x = self.proj(x)
+        x = self.proj_drop(x)
+
+        return x
+
+
+class Block(nn.Module):
+
+    def __init__(self, dim, num_heads, mlp_ratio=4., qkv_bias=False, qk_scale=None, drop=0., attn_drop=0.,
+                 drop_path=0., act_layer=nn.GELU, norm_layer=nn.LayerNorm, sr_ratio=1):
+        super().__init__()
+        self.norm1 = norm_layer(dim)
+        self.attn = Attention(
+            dim,
+            num_heads=num_heads, qkv_bias=qkv_bias, qk_scale=qk_scale,
+            attn_drop=attn_drop, proj_drop=drop, sr_ratio=sr_ratio)
+        # NOTE: drop path for stochastic depth, we shall see if this is better than dropout here
+        self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
+        self.norm2 = norm_layer(dim)
+        mlp_hidden_dim = int(dim * mlp_ratio)
+        self.mlp = Mlp(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)
+
+        self.apply(self._init_weights)
+
+    def _init_weights(self, m):
+        if isinstance(m, nn.Linear):
+            trunc_normal_(m.weight, std=.02)
+            if isinstance(m, nn.Linear) and m.bias is not None:
+                nn.init.constant_(m.bias, 0)
+        elif isinstance(m, nn.LayerNorm):
+            nn.init.constant_(m.bias, 0)
+            nn.init.constant_(m.weight, 1.0)
+        elif isinstance(m, nn.Conv2d):
+            fan_out = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
+            fan_out //= m.groups
+            m.weight.data.normal_(0, math.sqrt(2.0 / fan_out))
+            if m.bias is not None:
+                m.bias.data.zero_()
+
+    def forward(self, x, H, W):
+        x = x + self.drop_path(self.attn(self.norm1(x), H, W))
+        x = x + self.drop_path(self.mlp(self.norm2(x), H, W))
+
+        return x
+
+
+class OverlapPatchEmbed(nn.Module):
+    """ Image to Patch Embedding
+    """
+
+    def __init__(self, img_size=224, patch_size=7, stride=4, in_channels=3, embed_dim=768):
+        super().__init__()
+        img_size = to_2tuple(img_size)
+        patch_size = to_2tuple(patch_size)
+
+        self.img_size = img_size
+        self.patch_size = patch_size
+        self.H, self.W = img_size[0] // patch_size[0], img_size[1] // patch_size[1]
+        self.num_patches = self.H * self.W
+        self.proj = nn.Conv2d(in_channels, embed_dim, kernel_size=patch_size, stride=stride,
+                              padding=(patch_size[0] // 2, patch_size[1] // 2))
+        self.norm = nn.LayerNorm(embed_dim)
+
+        self.apply(self._init_weights)
+
+    def _init_weights(self, m):
+        if isinstance(m, nn.Linear):
+            trunc_normal_(m.weight, std=.02)
+            if isinstance(m, nn.Linear) and m.bias is not None:
+                nn.init.constant_(m.bias, 0)
+        elif isinstance(m, nn.LayerNorm):
+            nn.init.constant_(m.bias, 0)
+            nn.init.constant_(m.weight, 1.0)
+        elif isinstance(m, nn.Conv2d):
+            fan_out = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
+            fan_out //= m.groups
+            m.weight.data.normal_(0, math.sqrt(2.0 / fan_out))
+            if m.bias is not None:
+                m.bias.data.zero_()
+
+    def forward(self, x):
+        x = self.proj(x)
+        _, _, H, W = x.shape
+        x = x.flatten(2).transpose(1, 2)
+        x = self.norm(x)
+
+        return x, H, W
+
+
+class PyramidVisionTransformerImpr(nn.Module):
+    def __init__(self, img_size=224, patch_size=16, in_channels=3, num_classes=1000, embed_dims=[64, 128, 256, 512],
+                 num_heads=[1, 2, 4, 8], mlp_ratios=[4, 4, 4, 4], qkv_bias=False, qk_scale=None, drop_rate=0.,
+                 attn_drop_rate=0., drop_path_rate=0., norm_layer=nn.LayerNorm,
+                 depths=[3, 4, 6, 3], sr_ratios=[8, 4, 2, 1]):
+        super().__init__()
+        self.num_classes = num_classes
+        self.depths = depths
+
+        # patch_embed
+        self.patch_embed1 = OverlapPatchEmbed(img_size=img_size, patch_size=7, stride=4, in_channels=in_channels,
+                                              embed_dim=embed_dims[0])
+        self.patch_embed2 = OverlapPatchEmbed(img_size=img_size // 4, patch_size=3, stride=2, in_channels=embed_dims[0],
+                                              embed_dim=embed_dims[1])
+        self.patch_embed3 = OverlapPatchEmbed(img_size=img_size // 8, patch_size=3, stride=2, in_channels=embed_dims[1],
+                                              embed_dim=embed_dims[2])
+        self.patch_embed4 = OverlapPatchEmbed(img_size=img_size // 16, patch_size=3, stride=2, in_channels=embed_dims[2],
+                                              embed_dim=embed_dims[3])
+
+        # transformer encoder
+        dpr = [x.item() for x in torch.linspace(0, drop_path_rate, sum(depths))]  # stochastic depth decay rule
+        cur = 0
+        self.block1 = nn.ModuleList([Block(
+            dim=embed_dims[0], num_heads=num_heads[0], mlp_ratio=mlp_ratios[0], qkv_bias=qkv_bias, qk_scale=qk_scale,
+            drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[cur + i], norm_layer=norm_layer,
+            sr_ratio=sr_ratios[0])
+            for i in range(depths[0])])
+        self.norm1 = norm_layer(embed_dims[0])
+
+        cur += depths[0]
+        self.block2 = nn.ModuleList([Block(
+            dim=embed_dims[1], num_heads=num_heads[1], mlp_ratio=mlp_ratios[1], qkv_bias=qkv_bias, qk_scale=qk_scale,
+            drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[cur + i], norm_layer=norm_layer,
+            sr_ratio=sr_ratios[1])
+            for i in range(depths[1])])
+        self.norm2 = norm_layer(embed_dims[1])
+
+        cur += depths[1]
+        self.block3 = nn.ModuleList([Block(
+            dim=embed_dims[2], num_heads=num_heads[2], mlp_ratio=mlp_ratios[2], qkv_bias=qkv_bias, qk_scale=qk_scale,
+            drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[cur + i], norm_layer=norm_layer,
+            sr_ratio=sr_ratios[2])
+            for i in range(depths[2])])
+        self.norm3 = norm_layer(embed_dims[2])
+
+        cur += depths[2]
+        self.block4 = nn.ModuleList([Block(
+            dim=embed_dims[3], num_heads=num_heads[3], mlp_ratio=mlp_ratios[3], qkv_bias=qkv_bias, qk_scale=qk_scale,
+            drop=drop_rate, attn_drop=attn_drop_rate, drop_path=dpr[cur + i], norm_layer=norm_layer,
+            sr_ratio=sr_ratios[3])
+            for i in range(depths[3])])
+        self.norm4 = norm_layer(embed_dims[3])
+
+        # classification head
+        # self.head = nn.Linear(embed_dims[3], num_classes) if num_classes > 0 else nn.Identity()
+
+        self.apply(self._init_weights)
+
+    def _init_weights(self, m):
+        if isinstance(m, nn.Linear):
+            trunc_normal_(m.weight, std=.02)
+            if isinstance(m, nn.Linear) and m.bias is not None:
+                nn.init.constant_(m.bias, 0)
+        elif isinstance(m, nn.LayerNorm):
+            nn.init.constant_(m.bias, 0)
+            nn.init.constant_(m.weight, 1.0)
+        elif isinstance(m, nn.Conv2d):
+            fan_out = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
+            fan_out //= m.groups
+            m.weight.data.normal_(0, math.sqrt(2.0 / fan_out))
+            if m.bias is not None:
+                m.bias.data.zero_()
+
+    def init_weights(self, pretrained=None):
+        if isinstance(pretrained, str):
+            logger = 1
+            #load_checkpoint(self, pretrained, map_location='cpu', strict=False, logger=logger)
+
+    def reset_drop_path(self, drop_path_rate):
+        dpr = [x.item() for x in torch.linspace(0, drop_path_rate, sum(self.depths))]
+        cur = 0
+        for i in range(self.depths[0]):
+            self.block1[i].drop_path.drop_prob = dpr[cur + i]
+
+        cur += self.depths[0]
+        for i in range(self.depths[1]):
+            self.block2[i].drop_path.drop_prob = dpr[cur + i]
+
+        cur += self.depths[1]
+        for i in range(self.depths[2]):
+            self.block3[i].drop_path.drop_prob = dpr[cur + i]
+
+        cur += self.depths[2]
+        for i in range(self.depths[3]):
+            self.block4[i].drop_path.drop_prob = dpr[cur + i]
+
+    def freeze_patch_emb(self):
+        self.patch_embed1.requires_grad = False
+
+    @torch.jit.ignore
+    def no_weight_decay(self):
+        return {'pos_embed1', 'pos_embed2', 'pos_embed3', 'pos_embed4', 'cls_token'}  # has pos_embed may be better
+
+    def get_classifier(self):
+        return self.head
+
+    def reset_classifier(self, num_classes, global_pool=''):
+        self.num_classes = num_classes
+        self.head = nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()
+
+    def forward_features(self, x):
+        B = x.shape[0]
+        outs = []
+
+        # stage 1
+        x, H, W = self.patch_embed1(x)
+        for i, blk in enumerate(self.block1):
+            x = blk(x, H, W)
+        x = self.norm1(x)
+        x = x.reshape(B, H, W, -1).permute(0, 3, 1, 2).contiguous()
+        outs.append(x)
+
+        # stage 2
+        x, H, W = self.patch_embed2(x)
+        for i, blk in enumerate(self.block2):
+            x = blk(x, H, W)
+        x = self.norm2(x)
+        x = x.reshape(B, H, W, -1).permute(0, 3, 1, 2).contiguous()
+        outs.append(x)
+
+        # stage 3
+        x, H, W = self.patch_embed3(x)
+        for i, blk in enumerate(self.block3):
+            x = blk(x, H, W)
+        x = self.norm3(x)
+        x = x.reshape(B, H, W, -1).permute(0, 3, 1, 2).contiguous()
+        outs.append(x)
+
+        # stage 4
+        x, H, W = self.patch_embed4(x)
+        for i, blk in enumerate(self.block4):
+            x = blk(x, H, W)
+        x = self.norm4(x)
+        x = x.reshape(B, H, W, -1).permute(0, 3, 1, 2).contiguous()
+        outs.append(x)
+
+        return outs
+
+        # return x.mean(dim=1)
+
+    def forward(self, x):
+        x = self.forward_features(x)
+        # x = self.head(x)
+
+        return x
+
+
+class DWConv(nn.Module):
+    def __init__(self, dim=768):
+        super(DWConv, self).__init__()
+        self.dwconv = nn.Conv2d(dim, dim, 3, 1, 1, bias=True, groups=dim)
+
+    def forward(self, x, H, W):
+        B, N, C = x.shape
+        x = x.transpose(1, 2).view(B, C, H, W).contiguous()
+        x = self.dwconv(x)
+        x = x.flatten(2).transpose(1, 2)
+
+        return x
+
+
+def _conv_filter(state_dict, patch_size=16):
+    """ convert patch embedding weight from manual patchify + linear proj to conv"""
+    out_dict = {}
+    for k, v in state_dict.items():
+        if 'patch_embed.proj.weight' in k:
+            v = v.reshape((v.shape[0], 3, patch_size, patch_size))
+        out_dict[k] = v
+
+    return out_dict
+
+
+## @register_model
+class pvt_v2_b0(PyramidVisionTransformerImpr):
+    def __init__(self, **kwargs):
+        super(pvt_v2_b0, self).__init__(
+            patch_size=4, embed_dims=[32, 64, 160, 256], num_heads=[1, 2, 5, 8], mlp_ratios=[8, 8, 4, 4],
+            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[2, 2, 2, 2], sr_ratios=[8, 4, 2, 1],
+            drop_rate=0.0, drop_path_rate=0.1)
+
+
+
+## @register_model
+class pvt_v2_b1(PyramidVisionTransformerImpr):
+    def __init__(self, **kwargs):
+        super(pvt_v2_b1, self).__init__(
+            patch_size=4, embed_dims=[64, 128, 320, 512], num_heads=[1, 2, 5, 8], mlp_ratios=[8, 8, 4, 4],
+            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[2, 2, 2, 2], sr_ratios=[8, 4, 2, 1],
+            drop_rate=0.0, drop_path_rate=0.1)
+
+## @register_model
+class pvt_v2_b2(PyramidVisionTransformerImpr):
+    def __init__(self, in_channels=3, **kwargs):
+        super(pvt_v2_b2, self).__init__(
+            patch_size=4, embed_dims=[64, 128, 320, 512], num_heads=[1, 2, 5, 8], mlp_ratios=[8, 8, 4, 4],
+            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[3, 4, 6, 3], sr_ratios=[8, 4, 2, 1],
+            drop_rate=0.0, drop_path_rate=0.1, in_channels=in_channels)
+
+## @register_model
+class pvt_v2_b3(PyramidVisionTransformerImpr):
+    def __init__(self, **kwargs):
+        super(pvt_v2_b3, self).__init__(
+            patch_size=4, embed_dims=[64, 128, 320, 512], num_heads=[1, 2, 5, 8], mlp_ratios=[8, 8, 4, 4],
+            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[3, 4, 18, 3], sr_ratios=[8, 4, 2, 1],
+            drop_rate=0.0, drop_path_rate=0.1)
+
+## @register_model
+class pvt_v2_b4(PyramidVisionTransformerImpr):
+    def __init__(self, **kwargs):
+        super(pvt_v2_b4, self).__init__(
+            patch_size=4, embed_dims=[64, 128, 320, 512], num_heads=[1, 2, 5, 8], mlp_ratios=[8, 8, 4, 4],
+            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[3, 8, 27, 3], sr_ratios=[8, 4, 2, 1],
+            drop_rate=0.0, drop_path_rate=0.1)
+
+
+## @register_model
+class pvt_v2_b5(PyramidVisionTransformerImpr):
+    def __init__(self, **kwargs):
+        super(pvt_v2_b5, self).__init__(
+            patch_size=4, embed_dims=[64, 128, 320, 512], num_heads=[1, 2, 5, 8], mlp_ratios=[4, 4, 4, 4],
+            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[3, 6, 40, 3], sr_ratios=[8, 4, 2, 1],
+            drop_rate=0.0, drop_path_rate=0.1)
+
+
+
+### models/backbones/swin_v1.py
+
+# --------------------------------------------------------
+# Swin Transformer
+# Copyright (c) 2021 Microsoft
+# Licensed under The MIT License [see LICENSE for details]
+# Written by Ze Liu, Yutong Lin, Yixuan Wei
+# --------------------------------------------------------
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.utils.checkpoint as checkpoint
+import numpy as np
+from timm.models.layers import DropPath, to_2tuple, trunc_normal_
+
+# from config import Config
+
+
+# config = Config()
+
+class Mlp(nn.Module):
+    """ Multilayer perceptron."""
+
+    def __init__(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.):
+        super().__init__()
+        out_features = out_features or in_features
+        hidden_features = hidden_features or in_features
+        self.fc1 = nn.Linear(in_features, hidden_features)
+        self.act = act_layer()
+        self.fc2 = nn.Linear(hidden_features, out_features)
+        self.drop = nn.Dropout(drop)
+
+    def forward(self, x):
+        x = self.fc1(x)
+        x = self.act(x)
+        x = self.drop(x)
+        x = self.fc2(x)
+        x = self.drop(x)
+        return x
+
+
+def window_partition(x, window_size):
+    """
+    Args:
+        x: (B, H, W, C)
+        window_size (int): window size
+
+    Returns:
+        windows: (num_windows*B, window_size, window_size, C)
+    """
+    B, H, W, C = x.shape
+    x = x.view(B, H // window_size, window_size, W // window_size, window_size, C)
+    windows = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(-1, window_size, window_size, C)
+    return windows
+
+
+def window_reverse(windows, window_size, H, W):
+    """
+    Args:
+        windows: (num_windows*B, window_size, window_size, C)
+        window_size (int): Window size
+        H (int): Height of image
+        W (int): Width of image
+
+    Returns:
+        x: (B, H, W, C)
+    """
+    B = int(windows.shape[0] / (H * W / window_size / window_size))
+    x = windows.view(B, H // window_size, W // window_size, window_size, window_size, -1)
+    x = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(B, H, W, -1)
+    return x
+
+
+class WindowAttention(nn.Module):
+    """ Window based multi-head self attention (W-MSA) module with relative position bias.
+    It supports both of shifted and non-shifted window.
+
+    Args:
+        dim (int): Number of input channels.
+        window_size (tuple[int]): The height and width of the window.
+        num_heads (int): Number of attention heads.
+        qkv_bias (bool, optional):  If True, add a learnable bias to query, key, value. Default: True
+        qk_scale (float | None, optional): Override default qk scale of head_dim ** -0.5 if set
+        attn_drop (float, optional): Dropout ratio of attention weight. Default: 0.0
+        proj_drop (float, optional): Dropout ratio of output. Default: 0.0
+    """
+
+    def __init__(self, dim, window_size, num_heads, qkv_bias=True, qk_scale=None, attn_drop=0., proj_drop=0.):
+
+        super().__init__()
+        self.dim = dim
+        self.window_size = window_size  # Wh, Ww
+        self.num_heads = num_heads
+        head_dim = dim // num_heads
+        self.scale = qk_scale or head_dim ** -0.5
+
+        # define a parameter table of relative position bias
+        self.relative_position_bias_table = nn.Parameter(
+            torch.zeros((2 * window_size[0] - 1) * (2 * window_size[1] - 1), num_heads))  # 2*Wh-1 * 2*Ww-1, nH
+
+        # get pair-wise relative position index for each token inside the window
+        coords_h = torch.arange(self.window_size[0])
+        coords_w = torch.arange(self.window_size[1])
+        coords = torch.stack(torch.meshgrid([coords_h, coords_w], indexing='ij'))  # 2, Wh, Ww
+        coords_flatten = torch.flatten(coords, 1)  # 2, Wh*Ww
+        relative_coords = coords_flatten[:, :, None] - coords_flatten[:, None, :]  # 2, Wh*Ww, Wh*Ww
+        relative_coords = relative_coords.permute(1, 2, 0).contiguous()  # Wh*Ww, Wh*Ww, 2
+        relative_coords[:, :, 0] += self.window_size[0] - 1  # shift to start from 0
+        relative_coords[:, :, 1] += self.window_size[1] - 1
+        relative_coords[:, :, 0] *= 2 * self.window_size[1] - 1
+        relative_position_index = relative_coords.sum(-1)  # Wh*Ww, Wh*Ww
+        self.register_buffer("relative_position_index", relative_position_index)
+
+        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
+        self.attn_drop_prob = attn_drop
+        self.attn_drop = nn.Dropout(attn_drop)
+        self.proj = nn.Linear(dim, dim)
+        self.proj_drop = nn.Dropout(proj_drop)
+
+        trunc_normal_(self.relative_position_bias_table, std=.02)
+        self.softmax = nn.Softmax(dim=-1)
+
+    def forward(self, x, mask=None):
+        """ Forward function.
+
+        Args:
+            x: input features with shape of (num_windows*B, N, C)
+            mask: (0/-inf) mask with shape of (num_windows, Wh*Ww, Wh*Ww) or None
+        """
+        B_, N, C = x.shape
+        qkv = self.qkv(x).reshape(B_, N, 3, self.num_heads, C // self.num_heads).permute(2, 0, 3, 1, 4)
+        q, k, v = qkv[0], qkv[1], qkv[2]  # make torchscript happy (cannot use tensor as tuple)
+
+        q = q * self.scale
+
+        if config.SDPA_enabled:
+            x = torch.nn.functional.scaled_dot_product_attention(
+                q, k, v,
+                attn_mask=None, dropout_p=self.attn_drop_prob, is_causal=False
+            ).transpose(1, 2).reshape(B_, N, C)
+        else:
+            attn = (q @ k.transpose(-2, -1))
+
+            relative_position_bias = self.relative_position_bias_table[self.relative_position_index.view(-1)].view(
+                self.window_size[0] * self.window_size[1], self.window_size[0] * self.window_size[1], -1)  # Wh*Ww,Wh*Ww,nH
+            relative_position_bias = relative_position_bias.permute(2, 0, 1).contiguous()  # nH, Wh*Ww, Wh*Ww
+            attn = attn + relative_position_bias.unsqueeze(0)
+
+            if mask is not None:
+                nW = mask.shape[0]
+                attn = attn.view(B_ // nW, nW, self.num_heads, N, N) + mask.unsqueeze(1).unsqueeze(0)
+                attn = attn.view(-1, self.num_heads, N, N)
+                attn = self.softmax(attn)
+            else:
+                attn = self.softmax(attn)
+
+            attn = self.attn_drop(attn)
+
+            x = (attn @ v).transpose(1, 2).reshape(B_, N, C)
+        x = self.proj(x)
+        x = self.proj_drop(x)
+        return x
+
+
+class SwinTransformerBlock(nn.Module):
+    """ Swin Transformer Block.
+
+    Args:
+        dim (int): Number of input channels.
+        num_heads (int): Number of attention heads.
+        window_size (int): Window size.
+        shift_size (int): Shift size for SW-MSA.
+        mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.
+        qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True
+        qk_scale (float | None, optional): Override default qk scale of head_dim ** -0.5 if set.
+        drop (float, optional): Dropout rate. Default: 0.0
+        attn_drop (float, optional): Attention dropout rate. Default: 0.0
+        drop_path (float, optional): Stochastic depth rate. Default: 0.0
+        act_layer (nn.Module, optional): Activation layer. Default: nn.GELU
+        norm_layer (nn.Module, optional): Normalization layer.  Default: nn.LayerNorm
+    """
+
+    def __init__(self, dim, num_heads, window_size=7, shift_size=0,
+                 mlp_ratio=4., qkv_bias=True, qk_scale=None, drop=0., attn_drop=0., drop_path=0.,
+                 act_layer=nn.GELU, norm_layer=nn.LayerNorm):
+        super().__init__()
+        self.dim = dim
+        self.num_heads = num_heads
+        self.window_size = window_size
+        self.shift_size = shift_size
+        self.mlp_ratio = mlp_ratio
+        assert 0 <= self.shift_size < self.window_size, "shift_size must in 0-window_size"
+
+        self.norm1 = norm_layer(dim)
+        self.attn = WindowAttention(
+            dim, window_size=to_2tuple(self.window_size), num_heads=num_heads,
+            qkv_bias=qkv_bias, qk_scale=qk_scale, attn_drop=attn_drop, proj_drop=drop)
+
+        self.drop_path = DropPath(drop_path) if drop_path > 0. else nn.Identity()
+        self.norm2 = norm_layer(dim)
+        mlp_hidden_dim = int(dim * mlp_ratio)
+        self.mlp = Mlp(in_features=dim, hidden_features=mlp_hidden_dim, act_layer=act_layer, drop=drop)
+
+        self.H = None
+        self.W = None
+
+    def forward(self, x, mask_matrix):
+        """ Forward function.
+
+        Args:
+            x: Input feature, tensor size (B, H*W, C).
+            H, W: Spatial resolution of the input feature.
+            mask_matrix: Attention mask for cyclic shift.
+        """
+        B, L, C = x.shape
+        H, W = self.H, self.W
+        assert L == H * W, "input feature has wrong size"
+
+        shortcut = x
+        x = self.norm1(x)
+        x = x.view(B, H, W, C)
+
+        # pad feature maps to multiples of window size
+        pad_l = pad_t = 0
+        pad_r = (self.window_size - W % self.window_size) % self.window_size
+        pad_b = (self.window_size - H % self.window_size) % self.window_size
+        x = F.pad(x, (0, 0, pad_l, pad_r, pad_t, pad_b))
+        _, Hp, Wp, _ = x.shape
+
+        # cyclic shift
+        if self.shift_size > 0:
+            shifted_x = torch.roll(x, shifts=(-self.shift_size, -self.shift_size), dims=(1, 2))
+            attn_mask = mask_matrix
+        else:
+            shifted_x = x
+            attn_mask = None
+
+        # partition windows
+        x_windows = window_partition(shifted_x, self.window_size)  # nW*B, window_size, window_size, C
+        x_windows = x_windows.view(-1, self.window_size * self.window_size, C)  # nW*B, window_size*window_size, C
+
+        # W-MSA/SW-MSA
+        attn_windows = self.attn(x_windows, mask=attn_mask)  # nW*B, window_size*window_size, C
+
+        # merge windows
+        attn_windows = attn_windows.view(-1, self.window_size, self.window_size, C)
+        shifted_x = window_reverse(attn_windows, self.window_size, Hp, Wp)  # B H' W' C
+
+        # reverse cyclic shift
+        if self.shift_size > 0:
+            x = torch.roll(shifted_x, shifts=(self.shift_size, self.shift_size), dims=(1, 2))
+        else:
+            x = shifted_x
+
+        if pad_r > 0 or pad_b > 0:
+            x = x[:, :H, :W, :].contiguous()
+
+        x = x.view(B, H * W, C)
+
+        # FFN
+        x = shortcut + self.drop_path(x)
+        x = x + self.drop_path(self.mlp(self.norm2(x)))
+
+        return x
+
+
+class PatchMerging(nn.Module):
+    """ Patch Merging Layer
+
+    Args:
+        dim (int): Number of input channels.
+        norm_layer (nn.Module, optional): Normalization layer.  Default: nn.LayerNorm
+    """
+    def __init__(self, dim, norm_layer=nn.LayerNorm):
+        super().__init__()
+        self.dim = dim
+        self.reduction = nn.Linear(4 * dim, 2 * dim, bias=False)
+        self.norm = norm_layer(4 * dim)
+
+    def forward(self, x, H, W):
+        """ Forward function.
+
+        Args:
+            x: Input feature, tensor size (B, H*W, C).
+            H, W: Spatial resolution of the input feature.
+        """
+        B, L, C = x.shape
+        assert L == H * W, "input feature has wrong size"
+
+        x = x.view(B, H, W, C)
+
+        # padding
+        pad_input = (H % 2 == 1) or (W % 2 == 1)
+        if pad_input:
+            x = F.pad(x, (0, 0, 0, W % 2, 0, H % 2))
+
+        x0 = x[:, 0::2, 0::2, :]  # B H/2 W/2 C
+        x1 = x[:, 1::2, 0::2, :]  # B H/2 W/2 C
+        x2 = x[:, 0::2, 1::2, :]  # B H/2 W/2 C
+        x3 = x[:, 1::2, 1::2, :]  # B H/2 W/2 C
+        x = torch.cat([x0, x1, x2, x3], -1)  # B H/2 W/2 4*C
+        x = x.view(B, -1, 4 * C)  # B H/2*W/2 4*C
+
+        x = self.norm(x)
+        x = self.reduction(x)
+
+        return x
+
+
+class BasicLayer(nn.Module):
+    """ A basic Swin Transformer layer for one stage.
+
+    Args:
+        dim (int): Number of feature channels
+        depth (int): Depths of this stage.
+        num_heads (int): Number of attention head.
+        window_size (int): Local window size. Default: 7.
+        mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.
+        qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True
+        qk_scale (float | None, optional): Override default qk scale of head_dim ** -0.5 if set.
+        drop (float, optional): Dropout rate. Default: 0.0
+        attn_drop (float, optional): Attention dropout rate. Default: 0.0
+        drop_path (float | tuple[float], optional): Stochastic depth rate. Default: 0.0
+        norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm
+        downsample (nn.Module | None, optional): Downsample layer at the end of the layer. Default: None
+        use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False.
+    """
+
+    def __init__(self,
+                 dim,
+                 depth,
+                 num_heads,
+                 window_size=7,
+                 mlp_ratio=4.,
+                 qkv_bias=True,
+                 qk_scale=None,
+                 drop=0.,
+                 attn_drop=0.,
+                 drop_path=0.,
+                 norm_layer=nn.LayerNorm,
+                 downsample=None,
+                 use_checkpoint=False):
+        super().__init__()
+        self.window_size = window_size
+        self.shift_size = window_size // 2
+        self.depth = depth
+        self.use_checkpoint = use_checkpoint
+
+        # build blocks
+        self.blocks = nn.ModuleList([
+            SwinTransformerBlock(
+                dim=dim,
+                num_heads=num_heads,
+                window_size=window_size,
+                shift_size=0 if (i % 2 == 0) else window_size // 2,
+                mlp_ratio=mlp_ratio,
+                qkv_bias=qkv_bias,
+                qk_scale=qk_scale,
+                drop=drop,
+                attn_drop=attn_drop,
+                drop_path=drop_path[i] if isinstance(drop_path, list) else drop_path,
+                norm_layer=norm_layer)
+            for i in range(depth)])
+
+        # patch merging layer
+        if downsample is not None:
+            self.downsample = downsample(dim=dim, norm_layer=norm_layer)
+        else:
+            self.downsample = None
+
+    def forward(self, x, H, W):
+        """ Forward function.
+
+        Args:
+            x: Input feature, tensor size (B, H*W, C).
+            H, W: Spatial resolution of the input feature.
+        """
+
+        # calculate attention mask for SW-MSA
+        Hp = int(np.ceil(H / self.window_size)) * self.window_size
+        Wp = int(np.ceil(W / self.window_size)) * self.window_size
+        img_mask = torch.zeros((1, Hp, Wp, 1), device=x.device)  # 1 Hp Wp 1
+        h_slices = (slice(0, -self.window_size),
+                    slice(-self.window_size, -self.shift_size),
+                    slice(-self.shift_size, None))
+        w_slices = (slice(0, -self.window_size),
+                    slice(-self.window_size, -self.shift_size),
+                    slice(-self.shift_size, None))
+        cnt = 0
+        for h in h_slices:
+            for w in w_slices:
+                img_mask[:, h, w, :] = cnt
+                cnt += 1
+
+        mask_windows = window_partition(img_mask, self.window_size)  # nW, window_size, window_size, 1
+        mask_windows = mask_windows.view(-1, self.window_size * self.window_size)
+        attn_mask = mask_windows.unsqueeze(1) - mask_windows.unsqueeze(2)
+        attn_mask = attn_mask.masked_fill(attn_mask != 0, float(-100.0)).masked_fill(attn_mask == 0, float(0.0))
+
+        for blk in self.blocks:
+            blk.H, blk.W = H, W
+            if self.use_checkpoint:
+                x = checkpoint.checkpoint(blk, x, attn_mask)
+            else:
+                x = blk(x, attn_mask)
+        if self.downsample is not None:
+            x_down = self.downsample(x, H, W)
+            Wh, Ww = (H + 1) // 2, (W + 1) // 2
+            return x, H, W, x_down, Wh, Ww
+        else:
+            return x, H, W, x, H, W
+
+
+class PatchEmbed(nn.Module):
+    """ Image to Patch Embedding
+
+    Args:
+        patch_size (int): Patch token size. Default: 4.
+        in_channels (int): Number of input image channels. Default: 3.
+        embed_dim (int): Number of linear projection output channels. Default: 96.
+        norm_layer (nn.Module, optional): Normalization layer. Default: None
+    """
+
+    def __init__(self, patch_size=4, in_channels=3, embed_dim=96, norm_layer=None):
+        super().__init__()
+        patch_size = to_2tuple(patch_size)
+        self.patch_size = patch_size
+
+        self.in_channels = in_channels
+        self.embed_dim = embed_dim
+
+        self.proj = nn.Conv2d(in_channels, embed_dim, kernel_size=patch_size, stride=patch_size)
+        if norm_layer is not None:
+            self.norm = norm_layer(embed_dim)
+        else:
+            self.norm = None
+
+    def forward(self, x):
+        """Forward function."""
+        # padding
+        _, _, H, W = x.size()
+        if W % self.patch_size[1] != 0:
+            x = F.pad(x, (0, self.patch_size[1] - W % self.patch_size[1]))
+        if H % self.patch_size[0] != 0:
+            x = F.pad(x, (0, 0, 0, self.patch_size[0] - H % self.patch_size[0]))
+
+        x = self.proj(x)  # B C Wh Ww
+        if self.norm is not None:
+            Wh, Ww = x.size(2), x.size(3)
+            x = x.flatten(2).transpose(1, 2)
+            x = self.norm(x)
+            x = x.transpose(1, 2).view(-1, self.embed_dim, Wh, Ww)
+
+        return x
+
+
+class SwinTransformer(nn.Module):
+    """ Swin Transformer backbone.
+        A PyTorch impl of : `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows`  -
+          https://arxiv.org/pdf/2103.14030
+
+    Args:
+        pretrain_img_size (int): Input image size for training the pretrained model,
+            used in absolute postion embedding. Default 224.
+        patch_size (int | tuple(int)): Patch size. Default: 4.
+        in_channels (int): Number of input image channels. Default: 3.
+        embed_dim (int): Number of linear projection output channels. Default: 96.
+        depths (tuple[int]): Depths of each Swin Transformer stage.
+        num_heads (tuple[int]): Number of attention head of each stage.
+        window_size (int): Window size. Default: 7.
+        mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.
+        qkv_bias (bool): If True, add a learnable bias to query, key, value. Default: True
+        qk_scale (float): Override default qk scale of head_dim ** -0.5 if set.
+        drop_rate (float): Dropout rate.
+        attn_drop_rate (float): Attention dropout rate. Default: 0.
+        drop_path_rate (float): Stochastic depth rate. Default: 0.2.
+        norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm.
+        ape (bool): If True, add absolute position embedding to the patch embedding. Default: False.
+        patch_norm (bool): If True, add normalization after patch embedding. Default: True.
+        out_indices (Sequence[int]): Output from which stages.
+        frozen_stages (int): Stages to be frozen (stop grad and set eval mode).
+            -1 means not freezing any parameters.
+        use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False.
+    """
+
+    def __init__(self,
+                 pretrain_img_size=224,
+                 patch_size=4,
+                 in_channels=3,
+                 embed_dim=96,
+                 depths=[2, 2, 6, 2],
+                 num_heads=[3, 6, 12, 24],
+                 window_size=7,
+                 mlp_ratio=4.,
+                 qkv_bias=True,
+                 qk_scale=None,
+                 drop_rate=0.,
+                 attn_drop_rate=0.,
+                 drop_path_rate=0.2,
+                 norm_layer=nn.LayerNorm,
+                 ape=False,
+                 patch_norm=True,
+                 out_indices=(0, 1, 2, 3),
+                 frozen_stages=-1,
+                 use_checkpoint=False):
+        super().__init__()
+
+        self.pretrain_img_size = pretrain_img_size
+        self.num_layers = len(depths)
+        self.embed_dim = embed_dim
+        self.ape = ape
+        self.patch_norm = patch_norm
+        self.out_indices = out_indices
+        self.frozen_stages = frozen_stages
+
+        # split image into non-overlapping patches
+        self.patch_embed = PatchEmbed(
+            patch_size=patch_size, in_channels=in_channels, embed_dim=embed_dim,
+            norm_layer=norm_layer if self.patch_norm else None)
+
+        # absolute position embedding
+        if self.ape:
+            pretrain_img_size = to_2tuple(pretrain_img_size)
+            patch_size = to_2tuple(patch_size)
+            patches_resolution = [pretrain_img_size[0] // patch_size[0], pretrain_img_size[1] // patch_size[1]]
+
+            self.absolute_pos_embed = nn.Parameter(torch.zeros(1, embed_dim, patches_resolution[0], patches_resolution[1]))
+            trunc_normal_(self.absolute_pos_embed, std=.02)
+
+        self.pos_drop = nn.Dropout(p=drop_rate)
+
+        # stochastic depth
+        dpr = [x.item() for x in torch.linspace(0, drop_path_rate, sum(depths))]  # stochastic depth decay rule
+
+        # build layers
+        self.layers = nn.ModuleList()
+        for i_layer in range(self.num_layers):
+            layer = BasicLayer(
+                dim=int(embed_dim * 2 ** i_layer),
+                depth=depths[i_layer],
+                num_heads=num_heads[i_layer],
+                window_size=window_size,
+                mlp_ratio=mlp_ratio,
+                qkv_bias=qkv_bias,
+                qk_scale=qk_scale,
+                drop=drop_rate,
+                attn_drop=attn_drop_rate,
+                drop_path=dpr[sum(depths[:i_layer]):sum(depths[:i_layer + 1])],
+                norm_layer=norm_layer,
+                downsample=PatchMerging if (i_layer < self.num_layers - 1) else None,
+                use_checkpoint=use_checkpoint)
+            self.layers.append(layer)
+
+        num_features = [int(embed_dim * 2 ** i) for i in range(self.num_layers)]
+        self.num_features = num_features
+
+        # add a norm layer for each output
+        for i_layer in out_indices:
+            layer = norm_layer(num_features[i_layer])
+            layer_name = f'norm{i_layer}'
+            self.add_module(layer_name, layer)
+
+        self._freeze_stages()
+
+    def _freeze_stages(self):
+        if self.frozen_stages >= 0:
+            self.patch_embed.eval()
+            for param in self.patch_embed.parameters():
+                param.requires_grad = False
+
+        if self.frozen_stages >= 1 and self.ape:
+            self.absolute_pos_embed.requires_grad = False
+
+        if self.frozen_stages >= 2:
+            self.pos_drop.eval()
+            for i in range(0, self.frozen_stages - 1):
+                m = self.layers[i]
+                m.eval()
+                for param in m.parameters():
+                    param.requires_grad = False
+
+
+    def forward(self, x):
+        """Forward function."""
+        x = self.patch_embed(x)
+
+        Wh, Ww = x.size(2), x.size(3)
+        if self.ape:
+            # interpolate the position embedding to the corresponding size
+            absolute_pos_embed = F.interpolate(self.absolute_pos_embed, size=(Wh, Ww), mode='bicubic')
+            x = (x + absolute_pos_embed) # B Wh*Ww C
+            
+        outs = []#x.contiguous()]
+        x = x.flatten(2).transpose(1, 2)
+        x = self.pos_drop(x)
+        for i in range(self.num_layers):
+            layer = self.layers[i]
+            x_out, H, W, x, Wh, Ww = layer(x, Wh, Ww)
+
+            if i in self.out_indices:
+                norm_layer = getattr(self, f'norm{i}')
+                x_out = norm_layer(x_out)
+
+                out = x_out.view(-1, H, W, self.num_features[i]).permute(0, 3, 1, 2).contiguous()
+                outs.append(out)
+
+        return tuple(outs)
+
+    def train(self, mode=True):
+        """Convert the model into training mode while keep layers freezed."""
+        super(SwinTransformer, self).train(mode)
+        self._freeze_stages()
+
+def swin_v1_t():
+    model = SwinTransformer(embed_dim=96, depths=[2, 2, 6, 2], num_heads=[3, 6, 12, 24], window_size=7)
+    return model
+
+def swin_v1_s():
+    model = SwinTransformer(embed_dim=96, depths=[2, 2, 18, 2], num_heads=[3, 6, 12, 24], window_size=7)
+    return model
+
+def swin_v1_b():
+    model = SwinTransformer(embed_dim=128, depths=[2, 2, 18, 2], num_heads=[4, 8, 16, 32], window_size=12)
+    return model
+
+def swin_v1_l():
+    model = SwinTransformer(embed_dim=192, depths=[2, 2, 18, 2], num_heads=[6, 12, 24, 48], window_size=12)
+    return model
+
+
+
+### models/modules/deform_conv.py
+
+import torch
+import torch.nn as nn
+from torchvision.ops import deform_conv2d
+
+
+class DeformableConv2d(nn.Module):
+    def __init__(self,
+                 in_channels,
+                 out_channels,
+                 kernel_size=3,
+                 stride=1,
+                 padding=1,
+                 bias=False):
+
+        super(DeformableConv2d, self).__init__()
+        
+        assert type(kernel_size) == tuple or type(kernel_size) == int
+
+        kernel_size = kernel_size if type(kernel_size) == tuple else (kernel_size, kernel_size)
+        self.stride = stride if type(stride) == tuple else (stride, stride)
+        self.padding = padding
+        
+        self.offset_conv = nn.Conv2d(in_channels,
+                                     2 * kernel_size[0] * kernel_size[1],
+                                     kernel_size=kernel_size,
+                                     stride=stride,
+                                     padding=self.padding,
+                                     bias=True)
+
+        nn.init.constant_(self.offset_conv.weight, 0.)
+        nn.init.constant_(self.offset_conv.bias, 0.)
+        
+        self.modulator_conv = nn.Conv2d(in_channels,
+                                     1 * kernel_size[0] * kernel_size[1],
+                                     kernel_size=kernel_size,
+                                     stride=stride,
+                                     padding=self.padding,
+                                     bias=True)
+
+        nn.init.constant_(self.modulator_conv.weight, 0.)
+        nn.init.constant_(self.modulator_conv.bias, 0.)
+
+        self.regular_conv = nn.Conv2d(in_channels,
+                                      out_channels=out_channels,
+                                      kernel_size=kernel_size,
+                                      stride=stride,
+                                      padding=self.padding,
+                                      bias=bias)
+
+    def forward(self, x):
+        #h, w = x.shape[2:]
+        #max_offset = max(h, w)/4.
+
+        offset = self.offset_conv(x)#.clamp(-max_offset, max_offset)
+        modulator = 2. * torch.sigmoid(self.modulator_conv(x))
+        
+        x = deform_conv2d(
+            input=x,
+            offset=offset,
+            weight=self.regular_conv.weight,
+            bias=self.regular_conv.bias,
+            padding=self.padding,
+            mask=modulator,
+            stride=self.stride,
+        )
+        return x
+
+
+
+
+### utils.py
+
+import torch.nn as nn
+
+
+def build_act_layer(act_layer):
+    if act_layer == 'ReLU':
+        return nn.ReLU(inplace=True)
+    elif act_layer == 'SiLU':
+        return nn.SiLU(inplace=True)
+    elif act_layer == 'GELU':
+        return nn.GELU()
+
+    raise NotImplementedError(f'build_act_layer does not support {act_layer}')
+
+
+def build_norm_layer(dim,
+                     norm_layer,
+                     in_format='channels_last',
+                     out_format='channels_last',
+                     eps=1e-6):
+    layers = []
+    if norm_layer == 'BN':
+        if in_format == 'channels_last':
+            layers.append(to_channels_first())
+        layers.append(nn.BatchNorm2d(dim))
+        if out_format == 'channels_last':
+            layers.append(to_channels_last())
+    elif norm_layer == 'LN':
+        if in_format == 'channels_first':
+            layers.append(to_channels_last())
+        layers.append(nn.LayerNorm(dim, eps=eps))
+        if out_format == 'channels_first':
+            layers.append(to_channels_first())
+    else:
+        raise NotImplementedError(
+            f'build_norm_layer does not support {norm_layer}')
+    return nn.Sequential(*layers)
+
+
+class to_channels_first(nn.Module):
+
+    def __init__(self):
+        super().__init__()
+
+    def forward(self, x):
+        return x.permute(0, 3, 1, 2)
+
+
+class to_channels_last(nn.Module):
+
+    def __init__(self):
+        super().__init__()
+
+    def forward(self, x):
+        return x.permute(0, 2, 3, 1)
+
+
+
+### dataset.py
+
+_class_labels_TR_sorted = (
+    'Airplane, Ant, Antenna, Archery, Axe, BabyCarriage, Bag, BalanceBeam, Balcony, Balloon, Basket, BasketballHoop, Beatle, Bed, Bee, Bench, Bicycle, '
+    'BicycleFrame, BicycleStand, Boat, Bonsai, BoomLift, Bridge, BunkBed, Butterfly, Button, Cable, CableLift, Cage, Camcorder, Cannon, Canoe, Car, '
+    'CarParkDropArm, Carriage, Cart, Caterpillar, CeilingLamp, Centipede, Chair, Clip, Clock, Clothes, CoatHanger, Comb, ConcretePumpTruck, Crack, Crane, '
+    'Cup, DentalChair, Desk, DeskChair, Diagram, DishRack, DoorHandle, Dragonfish, Dragonfly, Drum, Earphone, Easel, ElectricIron, Excavator, Eyeglasses, '
+    'Fan, Fence, Fencing, FerrisWheel, FireExtinguisher, Fishing, Flag, FloorLamp, Forklift, GasStation, Gate, Gear, Goal, Golf, GymEquipment, Hammock, '
+    'Handcart, Handcraft, Handrail, HangGlider, Harp, Harvester, Headset, Helicopter, Helmet, Hook, HorizontalBar, Hydrovalve, IroningTable, Jewelry, Key, '
+    'KidsPlayground, Kitchenware, Kite, Knife, Ladder, LaundryRack, Lightning, Lobster, Locust, Machine, MachineGun, MagazineRack, Mantis, Medal, MemorialArchway, '
+    'Microphone, Missile, MobileHolder, Monitor, Mosquito, Motorcycle, MovingTrolley, Mower, MusicPlayer, MusicStand, ObservationTower, Octopus, OilWell, '
+    'OlympicLogo, OperatingTable, OutdoorFitnessEquipment, Parachute, Pavilion, Piano, Pipe, PlowHarrow, PoleVault, Punchbag, Rack, Racket, Rifle, Ring, Robot, '
+    'RockClimbing, Rope, Sailboat, Satellite, Scaffold, Scale, Scissor, Scooter, Sculpture, Seadragon, Seahorse, Seal, SewingMachine, Ship, Shoe, ShoppingCart, '
+    'ShoppingTrolley, Shower, Shrimp, Signboard, Skateboarding, Skeleton, Skiing, Spade, SpeedBoat, Spider, Spoon, Stair, Stand, Stationary, SteeringWheel, '
+    'Stethoscope, Stool, Stove, StreetLamp, SweetStand, Swing, Sword, TV, Table, TableChair, TableLamp, TableTennis, Tank, Tapeline, Teapot, Telescope, Tent, '
+    'TobaccoPipe, Toy, Tractor, TrafficLight, TrafficSign, Trampoline, TransmissionTower, Tree, Tricycle, TrimmerCover, Tripod, Trombone, Truck, Trumpet, Tuba, '
+    'UAV, Umbrella, UnevenBars, UtilityPole, VacuumCleaner, Violin, Wakesurfing, Watch, WaterTower, WateringPot, Well, WellLid, Wheel, Wheelchair, WindTurbine, Windmill, WineGlass, WireWhisk, Yacht'
+)
+class_labels_TR_sorted = _class_labels_TR_sorted.split(', ')
+
+
+### models/backbones/build_backbones.py
+
+import torch
+import torch.nn as nn
+from collections import OrderedDict
+from torchvision.models import vgg16, vgg16_bn, VGG16_Weights, VGG16_BN_Weights, resnet50, ResNet50_Weights
+# from models.pvt_v2 import pvt_v2_b0, pvt_v2_b1, pvt_v2_b2, pvt_v2_b5
+# from models.swin_v1 import swin_v1_t, swin_v1_s, swin_v1_b, swin_v1_l
+# from config import Config
+
+
+config = Config()
+
+def build_backbone(bb_name, pretrained=True, params_settings=''):
+    if bb_name == 'vgg16':
+        bb_net = list(vgg16(pretrained=VGG16_Weights.DEFAULT if pretrained else None).children())[0]
+        bb = nn.Sequential(OrderedDict({'conv1': bb_net[:4], 'conv2': bb_net[4:9], 'conv3': bb_net[9:16], 'conv4': bb_net[16:23]}))
+    elif bb_name == 'vgg16bn':
+        bb_net = list(vgg16_bn(pretrained=VGG16_BN_Weights.DEFAULT if pretrained else None).children())[0]
+        bb = nn.Sequential(OrderedDict({'conv1': bb_net[:6], 'conv2': bb_net[6:13], 'conv3': bb_net[13:23], 'conv4': bb_net[23:33]}))
+    elif bb_name == 'resnet50':
+        bb_net = list(resnet50(pretrained=ResNet50_Weights.DEFAULT if pretrained else None).children())
+        bb = nn.Sequential(OrderedDict({'conv1': nn.Sequential(*bb_net[0:3]), 'conv2': bb_net[4], 'conv3': bb_net[5], 'conv4': bb_net[6]}))
+    else:
+        bb = eval('{}({})'.format(bb_name, params_settings))
+        if pretrained:
+            bb = load_weights(bb, bb_name)
+    return bb
+
+def load_weights(model, model_name):
+    save_model = torch.load(config.weights[model_name], map_location='cpu')
+    model_dict = model.state_dict()
+    state_dict = {k: v if v.size() == model_dict[k].size() else model_dict[k] for k, v in save_model.items() if k in model_dict.keys()}
+    # to ignore the weights with mismatched size when I modify the backbone itself.
+    if not state_dict:
+        save_model_keys = list(save_model.keys())
+        sub_item = save_model_keys[0] if len(save_model_keys) == 1 else None
+        state_dict = {k: v if v.size() == model_dict[k].size() else model_dict[k] for k, v in save_model[sub_item].items() if k in model_dict.keys()}
+        if not state_dict or not sub_item:
+            print('Weights are not successully loaded. Check the state dict of weights file.')
+            return None
+        else:
+            print('Found correct weights in the "{}" item of loaded state_dict.'.format(sub_item))
+    model_dict.update(state_dict)
+    model.load_state_dict(model_dict)
+    return model
+
+
+
+### models/modules/decoder_blocks.py
+
+import torch
+import torch.nn as nn
+# from models.aspp import ASPP, ASPPDeformable
+# from config import Config
+
+
+# config = Config()
+
+
+class BasicDecBlk(nn.Module):
+    def __init__(self, in_channels=64, out_channels=64, inter_channels=64):
+        super(BasicDecBlk, self).__init__()
+        inter_channels = in_channels // 4 if config.dec_channels_inter == 'adap' else 64
+        self.conv_in = nn.Conv2d(in_channels, inter_channels, 3, 1, padding=1)
+        self.relu_in = nn.ReLU(inplace=True)
+        if config.dec_att == 'ASPP':
+            self.dec_att = ASPP(in_channels=inter_channels)
+        elif config.dec_att == 'ASPPDeformable':
+            self.dec_att = ASPPDeformable(in_channels=inter_channels)
+        self.conv_out = nn.Conv2d(inter_channels, out_channels, 3, 1, padding=1)
+        self.bn_in = nn.BatchNorm2d(inter_channels) if config.batch_size > 1 else nn.Identity()
+        self.bn_out = nn.BatchNorm2d(out_channels) if config.batch_size > 1 else nn.Identity()
+
+    def forward(self, x):
+        x = self.conv_in(x)
+        x = self.bn_in(x)
+        x = self.relu_in(x)
+        if hasattr(self, 'dec_att'):
+            x = self.dec_att(x)
+        x = self.conv_out(x)
+        x = self.bn_out(x)
+        return x
+
+
+class ResBlk(nn.Module):
+    def __init__(self, in_channels=64, out_channels=None, inter_channels=64):
+        super(ResBlk, self).__init__()
+        if out_channels is None:
+            out_channels = in_channels
+        inter_channels = in_channels // 4 if config.dec_channels_inter == 'adap' else 64
+
+        self.conv_in = nn.Conv2d(in_channels, inter_channels, 3, 1, padding=1)
+        self.bn_in = nn.BatchNorm2d(inter_channels) if config.batch_size > 1 else nn.Identity()
+        self.relu_in = nn.ReLU(inplace=True)
+
+        if config.dec_att == 'ASPP':
+            self.dec_att = ASPP(in_channels=inter_channels)
+        elif config.dec_att == 'ASPPDeformable':
+            self.dec_att = ASPPDeformable(in_channels=inter_channels)
+
+        self.conv_out = nn.Conv2d(inter_channels, out_channels, 3, 1, padding=1)
+        self.bn_out = nn.BatchNorm2d(out_channels) if config.batch_size > 1 else nn.Identity()
+        
+        self.conv_resi = nn.Conv2d(in_channels, out_channels, 1, 1, 0)
+
+    def forward(self, x):
+        _x = self.conv_resi(x)
+        x = self.conv_in(x)
+        x = self.bn_in(x)
+        x = self.relu_in(x)
+        if hasattr(self, 'dec_att'):
+            x = self.dec_att(x)
+        x = self.conv_out(x)
+        x = self.bn_out(x)
+        return x + _x
+
+
+
+### models/modules/lateral_blocks.py
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from functools import partial
+
+# from config import Config
+
+
+# config = Config()
+
+
+class BasicLatBlk(nn.Module):
+    def __init__(self, in_channels=64, out_channels=64, inter_channels=64):
+        super(BasicLatBlk, self).__init__()
+        inter_channels = in_channels // 4 if config.dec_channels_inter == 'adap' else 64
+        self.conv = nn.Conv2d(in_channels, out_channels, 1, 1, 0)
+
+    def forward(self, x):
+        x = self.conv(x)
+        return x
+
+
+
+### models/modules/aspp.py
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+# from models.deform_conv import DeformableConv2d
+# from config import Config
+
+
+# config = Config()
+
+
+class _ASPPModule(nn.Module):
+    def __init__(self, in_channels, planes, kernel_size, padding, dilation):
+        super(_ASPPModule, self).__init__()
+        self.atrous_conv = nn.Conv2d(in_channels, planes, kernel_size=kernel_size,
+                                            stride=1, padding=padding, dilation=dilation, bias=False)
+        self.bn = nn.BatchNorm2d(planes) if config.batch_size > 1 else nn.Identity()
+        self.relu = nn.ReLU(inplace=True)
+
+    def forward(self, x):
+        x = self.atrous_conv(x)
+        x = self.bn(x)
+
+        return self.relu(x)
+
+
+class ASPP(nn.Module):
+    def __init__(self, in_channels=64, out_channels=None, output_stride=16):
+        super(ASPP, self).__init__()
+        self.down_scale = 1
+        if out_channels is None:
+            out_channels = in_channels
+        self.in_channelster = 256 // self.down_scale
+        if output_stride == 16:
+            dilations = [1, 6, 12, 18]
+        elif output_stride == 8:
+            dilations = [1, 12, 24, 36]
+        else:
+            raise NotImplementedError
+
+        self.aspp1 = _ASPPModule(in_channels, self.in_channelster, 1, padding=0, dilation=dilations[0])
+        self.aspp2 = _ASPPModule(in_channels, self.in_channelster, 3, padding=dilations[1], dilation=dilations[1])
+        self.aspp3 = _ASPPModule(in_channels, self.in_channelster, 3, padding=dilations[2], dilation=dilations[2])
+        self.aspp4 = _ASPPModule(in_channels, self.in_channelster, 3, padding=dilations[3], dilation=dilations[3])
+
+        self.global_avg_pool = nn.Sequential(nn.AdaptiveAvgPool2d((1, 1)),
+                                             nn.Conv2d(in_channels, self.in_channelster, 1, stride=1, bias=False),
+                                             nn.BatchNorm2d(self.in_channelster) if config.batch_size > 1 else nn.Identity(),
+                                             nn.ReLU(inplace=True))
+        self.conv1 = nn.Conv2d(self.in_channelster * 5, out_channels, 1, bias=False)
+        self.bn1 = nn.BatchNorm2d(out_channels) if config.batch_size > 1 else nn.Identity()
+        self.relu = nn.ReLU(inplace=True)
+        self.dropout = nn.Dropout(0.5)
+
+    def forward(self, x):
+        x1 = self.aspp1(x)
+        x2 = self.aspp2(x)
+        x3 = self.aspp3(x)
+        x4 = self.aspp4(x)
+        x5 = self.global_avg_pool(x)
+        x5 = F.interpolate(x5, size=x1.size()[2:], mode='bilinear', align_corners=True)
+        x = torch.cat((x1, x2, x3, x4, x5), dim=1)
+
+        x = self.conv1(x)
+        x = self.bn1(x)
+        x = self.relu(x)
+
+        return self.dropout(x)
+
+
+##################### Deformable
+class _ASPPModuleDeformable(nn.Module):
+    def __init__(self, in_channels, planes, kernel_size, padding):
+        super(_ASPPModuleDeformable, self).__init__()
+        self.atrous_conv = DeformableConv2d(in_channels, planes, kernel_size=kernel_size,
+                                            stride=1, padding=padding, bias=False)
+        self.bn = nn.BatchNorm2d(planes) if config.batch_size > 1 else nn.Identity()
+        self.relu = nn.ReLU(inplace=True)
+
+    def forward(self, x):
+        x = self.atrous_conv(x)
+        x = self.bn(x)
+
+        return self.relu(x)
+
+
+class ASPPDeformable(nn.Module):
+    def __init__(self, in_channels, out_channels=None, parallel_block_sizes=[1, 3, 7]):
+        super(ASPPDeformable, self).__init__()
+        self.down_scale = 1
+        if out_channels is None:
+            out_channels = in_channels
+        self.in_channelster = 256 // self.down_scale
+
+        self.aspp1 = _ASPPModuleDeformable(in_channels, self.in_channelster, 1, padding=0)
+        self.aspp_deforms = nn.ModuleList([
+            _ASPPModuleDeformable(in_channels, self.in_channelster, conv_size, padding=int(conv_size//2)) for conv_size in parallel_block_sizes
+        ])
+
+        self.global_avg_pool = nn.Sequential(nn.AdaptiveAvgPool2d((1, 1)),
+                                             nn.Conv2d(in_channels, self.in_channelster, 1, stride=1, bias=False),
+                                             nn.BatchNorm2d(self.in_channelster) if config.batch_size > 1 else nn.Identity(),
+                                             nn.ReLU(inplace=True))
+        self.conv1 = nn.Conv2d(self.in_channelster * (2 + len(self.aspp_deforms)), out_channels, 1, bias=False)
+        self.bn1 = nn.BatchNorm2d(out_channels) if config.batch_size > 1 else nn.Identity()
+        self.relu = nn.ReLU(inplace=True)
+        self.dropout = nn.Dropout(0.5)
+
+    def forward(self, x):
+        x1 = self.aspp1(x)
+        x_aspp_deforms = [aspp_deform(x) for aspp_deform in self.aspp_deforms]
+        x5 = self.global_avg_pool(x)
+        x5 = F.interpolate(x5, size=x1.size()[2:], mode='bilinear', align_corners=True)
+        x = torch.cat((x1, *x_aspp_deforms, x5), dim=1)
+
+        x = self.conv1(x)
+        x = self.bn1(x)
+        x = self.relu(x)
+
+        return self.dropout(x)
+
+
+
+### models/refinement/refiner.py
+
+import torch
+import torch.nn as nn
+from collections import OrderedDict
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torchvision.models import vgg16, vgg16_bn
+from torchvision.models import resnet50
+
+# from config import Config
+# from dataset import class_labels_TR_sorted
+# from models.build_backbone import build_backbone
+# from models.decoder_blocks import BasicDecBlk
+# from models.lateral_blocks import BasicLatBlk
+# from models.ing import *
+# from models.stem_layer import StemLayer
+
+
+class RefinerPVTInChannels4(nn.Module):
+    def __init__(self, in_channels=3+1):
+        super(RefinerPVTInChannels4, self).__init__()
+        self.config = Config()
+        self.epoch = 1
+        self.bb = build_backbone(self.config.bb, params_settings='in_channels=4')
+
+        lateral_channels_in_collection = {
+            'vgg16': [512, 256, 128, 64], 'vgg16bn': [512, 256, 128, 64], 'resnet50': [1024, 512, 256, 64],
+            'pvt_v2_b2': [512, 320, 128, 64], 'pvt_v2_b5': [512, 320, 128, 64],
+            'swin_v1_b': [1024, 512, 256, 128], 'swin_v1_l': [1536, 768, 384, 192],
+        }
+        channels = lateral_channels_in_collection[self.config.bb]
+        self.squeeze_module = BasicDecBlk(channels[0], channels[0])
+
+        self.decoder = Decoder(channels)
+
+        if 0:
+            for key, value in self.named_parameters():
+                if 'bb.' in key:
+                    value.requires_grad = False
+
+    def forward(self, x):
+        if isinstance(x, list):
+            x = torch.cat(x, dim=1)
+        ########## Encoder ##########
+        if self.config.bb in ['vgg16', 'vgg16bn', 'resnet50']:
+            x1 = self.bb.conv1(x)
+            x2 = self.bb.conv2(x1)
+            x3 = self.bb.conv3(x2)
+            x4 = self.bb.conv4(x3)
+        else:
+            x1, x2, x3, x4 = self.bb(x)
+
+        x4 = self.squeeze_module(x4)
+
+        ########## Decoder ##########
+
+        features = [x, x1, x2, x3, x4]
+        scaled_preds = self.decoder(features)
+
+        return scaled_preds
+
+
+class Refiner(nn.Module):
+    def __init__(self, in_channels=3+1):
+        super(Refiner, self).__init__()
+        self.config = Config()
+        self.epoch = 1
+        self.stem_layer = StemLayer(in_channels=in_channels, inter_channels=48, out_channels=3, norm_layer='BN' if self.config.batch_size > 1 else 'LN')
+        self.bb = build_backbone(self.config.bb)
+
+        lateral_channels_in_collection = {
+            'vgg16': [512, 256, 128, 64], 'vgg16bn': [512, 256, 128, 64], 'resnet50': [1024, 512, 256, 64],
+            'pvt_v2_b2': [512, 320, 128, 64], 'pvt_v2_b5': [512, 320, 128, 64],
+            'swin_v1_b': [1024, 512, 256, 128], 'swin_v1_l': [1536, 768, 384, 192],
+        }
+        channels = lateral_channels_in_collection[self.config.bb]
+        self.squeeze_module = BasicDecBlk(channels[0], channels[0])
+
+        self.decoder = Decoder(channels)
+
+        if 0:
+            for key, value in self.named_parameters():
+                if 'bb.' in key:
+                    value.requires_grad = False
+
+    def forward(self, x):
+        if isinstance(x, list):
+            x = torch.cat(x, dim=1)
+        x = self.stem_layer(x)
+        ########## Encoder ##########
+        if self.config.bb in ['vgg16', 'vgg16bn', 'resnet50']:
+            x1 = self.bb.conv1(x)
+            x2 = self.bb.conv2(x1)
+            x3 = self.bb.conv3(x2)
+            x4 = self.bb.conv4(x3)
+        else:
+            x1, x2, x3, x4 = self.bb(x)
+
+        x4 = self.squeeze_module(x4)
+
+        ########## Decoder ##########
+
+        features = [x, x1, x2, x3, x4]
+        scaled_preds = self.decoder(features)
+
+        return scaled_preds
+
+
+class Decoder(nn.Module):
+    def __init__(self, channels):
+        super(Decoder, self).__init__()
+        self.config = Config()
+        DecoderBlock = eval('BasicDecBlk')
+        LateralBlock = eval('BasicLatBlk')
+
+        self.decoder_block4 = DecoderBlock(channels[0], channels[1])
+        self.decoder_block3 = DecoderBlock(channels[1], channels[2])
+        self.decoder_block2 = DecoderBlock(channels[2], channels[3])
+        self.decoder_block1 = DecoderBlock(channels[3], channels[3]//2)
+
+        self.lateral_block4 = LateralBlock(channels[1], channels[1])
+        self.lateral_block3 = LateralBlock(channels[2], channels[2])
+        self.lateral_block2 = LateralBlock(channels[3], channels[3])
+
+        if self.config.ms_supervision:
+            self.conv_ms_spvn_4 = nn.Conv2d(channels[1], 1, 1, 1, 0)
+            self.conv_ms_spvn_3 = nn.Conv2d(channels[2], 1, 1, 1, 0)
+            self.conv_ms_spvn_2 = nn.Conv2d(channels[3], 1, 1, 1, 0)
+        self.conv_out1 = nn.Sequential(nn.Conv2d(channels[3]//2, 1, 1, 1, 0))
+
+    def forward(self, features):
+        x, x1, x2, x3, x4 = features
+        outs = []
+        p4 = self.decoder_block4(x4)
+        _p4 = F.interpolate(p4, size=x3.shape[2:], mode='bilinear', align_corners=True)
+        _p3 = _p4 + self.lateral_block4(x3)
+
+        p3 = self.decoder_block3(_p3)
+        _p3 = F.interpolate(p3, size=x2.shape[2:], mode='bilinear', align_corners=True)
+        _p2 = _p3 + self.lateral_block3(x2)
+
+        p2 = self.decoder_block2(_p2)
+        _p2 = F.interpolate(p2, size=x1.shape[2:], mode='bilinear', align_corners=True)
+        _p1 = _p2 + self.lateral_block2(x1)
+
+        _p1 = self.decoder_block1(_p1)
+        _p1 = F.interpolate(_p1, size=x.shape[2:], mode='bilinear', align_corners=True)
+        p1_out = self.conv_out1(_p1)
+
+        if self.config.ms_supervision:
+            outs.append(self.conv_ms_spvn_4(p4))
+            outs.append(self.conv_ms_spvn_3(p3))
+            outs.append(self.conv_ms_spvn_2(p2))
+        outs.append(p1_out)
+        return outs
+
+
+class RefUNet(nn.Module):
+    # Refinement
+    def __init__(self, in_channels=3+1):
+        super(RefUNet, self).__init__()
+        self.encoder_1 = nn.Sequential(
+            nn.Conv2d(in_channels, 64, 3, 1, 1),
+            nn.Conv2d(64, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.encoder_2 = nn.Sequential(
+            nn.MaxPool2d(2, 2, ceil_mode=True),
+            nn.Conv2d(64, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.encoder_3 = nn.Sequential(
+            nn.MaxPool2d(2, 2, ceil_mode=True),
+            nn.Conv2d(64, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.encoder_4 = nn.Sequential(
+            nn.MaxPool2d(2, 2, ceil_mode=True),
+            nn.Conv2d(64, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.pool4 = nn.MaxPool2d(2, 2, ceil_mode=True)
+        #####
+        self.decoder_5 = nn.Sequential(
+            nn.Conv2d(64, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+        #####
+        self.decoder_4 = nn.Sequential(
+            nn.Conv2d(128, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.decoder_3 = nn.Sequential(
+            nn.Conv2d(128, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.decoder_2 = nn.Sequential(
+            nn.Conv2d(128, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.decoder_1 = nn.Sequential(
+            nn.Conv2d(128, 64, 3, 1, 1),
+            nn.BatchNorm2d(64),
+            nn.ReLU(inplace=True)
+        )
+
+        self.conv_d0 = nn.Conv2d(64, 1, 3, 1, 1)
+
+        self.upscore2 = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)
+
+    def forward(self, x):
+        outs = []
+        if isinstance(x, list):
+            x = torch.cat(x, dim=1)
+        hx = x
+
+        hx1 = self.encoder_1(hx)
+        hx2 = self.encoder_2(hx1)
+        hx3 = self.encoder_3(hx2)
+        hx4 = self.encoder_4(hx3)
+
+        hx = self.decoder_5(self.pool4(hx4))
+        hx = torch.cat((self.upscore2(hx), hx4), 1)
+
+        d4 = self.decoder_4(hx)
+        hx = torch.cat((self.upscore2(d4), hx3), 1)
+
+        d3 = self.decoder_3(hx)
+        hx = torch.cat((self.upscore2(d3), hx2), 1)
+
+        d2 = self.decoder_2(hx)
+        hx = torch.cat((self.upscore2(d2), hx1), 1)
+
+        d1 = self.decoder_1(hx)
+
+        x = self.conv_d0(d1)
+        outs.append(x)
+        return outs
+
+
+
+### models/stem_layer.py
+
+import torch.nn as nn
+# from utils import build_act_layer, build_norm_layer
+
+
+class StemLayer(nn.Module):
+    r""" Stem layer of InternImage
+    Args:
+        in_channels (int): number of input channels
+        out_channels (int): number of output channels
+        act_layer (str): activation layer
+        norm_layer (str): normalization layer
+    """
+
+    def __init__(self,
+                 in_channels=3+1,
+                 inter_channels=48,
+                 out_channels=96,
+                 act_layer='GELU',
+                 norm_layer='BN'):
+        super().__init__()
+        self.conv1 = nn.Conv2d(in_channels,
+                               inter_channels,
+                               kernel_size=3,
+                               stride=1,
+                               padding=1)
+        self.norm1 = build_norm_layer(
+            inter_channels, norm_layer, 'channels_first', 'channels_first'
+        )
+        self.act = build_act_layer(act_layer)
+        self.conv2 = nn.Conv2d(inter_channels,
+                               out_channels,
+                               kernel_size=3,
+                               stride=1,
+                               padding=1)
+        self.norm2 = build_norm_layer(
+            out_channels, norm_layer, 'channels_first', 'channels_first'
+        )
+
+    def forward(self, x):
+        x = self.conv1(x)
+        x = self.norm1(x)
+        x = self.act(x)
+        x = self.conv2(x)
+        x = self.norm2(x)
+        return x
+
+
+### models/birefnet.py
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from kornia.filters import laplacian
+from transformers import PreTrainedModel
+
+# from config import Config
+# from dataset import class_labels_TR_sorted
+# from models.build_backbone import build_backbone
+# from models.decoder_blocks import BasicDecBlk, ResBlk, HierarAttDecBlk
+# from models.lateral_blocks import BasicLatBlk
+# from models.aspp import ASPP, ASPPDeformable
+# from models.ing import *
+# from models.refiner import Refiner, RefinerPVTInChannels4, RefUNet
+# from models.stem_layer import StemLayer
+from .BiRefNet_config import BiRefNetConfig
+
+
+class BiRefNet(
+    PreTrainedModel
+):
+    config_class = BiRefNetConfig
+    def __init__(self, bb_pretrained=True, config=BiRefNetConfig()):
+        super(BiRefNet, self).__init__(config)
+        bb_pretrained = config.bb_pretrained
+        self.config = Config()
+        self.epoch = 1
+        self.bb = build_backbone(self.config.bb, pretrained=bb_pretrained)
+
+        channels = self.config.lateral_channels_in_collection
+
+        if self.config.auxiliary_classification:
+            self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
+            self.cls_head = nn.Sequential(
+                nn.Linear(channels[0], len(class_labels_TR_sorted))
+            )
+
+        if self.config.squeeze_block:
+            self.squeeze_module = nn.Sequential(*[
+                eval(self.config.squeeze_block.split('_x')[0])(channels[0]+sum(self.config.cxt), channels[0])
+                for _ in range(eval(self.config.squeeze_block.split('_x')[1]))
+            ])
+
+        self.decoder = Decoder(channels)
+
+        if self.config.ender:
+            self.dec_end = nn.Sequential(
+                nn.Conv2d(1, 16, 3, 1, 1),
+                nn.Conv2d(16, 1, 3, 1, 1),
+                nn.ReLU(inplace=True),
+            )
+
+        # refine patch-level segmentation
+        if self.config.refine:
+            if self.config.refine == 'itself':
+                self.stem_layer = StemLayer(in_channels=3+1, inter_channels=48, out_channels=3, norm_layer='BN' if self.config.batch_size > 1 else 'LN')
+            else:
+                self.refiner = eval('{}({})'.format(self.config.refine, 'in_channels=3+1'))
+
+        if self.config.freeze_bb:
+            # Freeze the backbone...
+            print(self.named_parameters())
+            for key, value in self.named_parameters():
+                if 'bb.' in key and 'refiner.' not in key:
+                    value.requires_grad = False
+
+    def forward_enc(self, x):
+        if self.config.bb in ['vgg16', 'vgg16bn', 'resnet50']:
+            x1 = self.bb.conv1(x); x2 = self.bb.conv2(x1); x3 = self.bb.conv3(x2); x4 = self.bb.conv4(x3)
+        else:
+            x1, x2, x3, x4 = self.bb(x)
+            if self.config.mul_scl_ipt == 'cat':
+                B, C, H, W = x.shape
+                x1_, x2_, x3_, x4_ = self.bb(F.interpolate(x, size=(H//2, W//2), mode='bilinear', align_corners=True))
+                x1 = torch.cat([x1, F.interpolate(x1_, size=x1.shape[2:], mode='bilinear', align_corners=True)], dim=1)
+                x2 = torch.cat([x2, F.interpolate(x2_, size=x2.shape[2:], mode='bilinear', align_corners=True)], dim=1)
+                x3 = torch.cat([x3, F.interpolate(x3_, size=x3.shape[2:], mode='bilinear', align_corners=True)], dim=1)
+                x4 = torch.cat([x4, F.interpolate(x4_, size=x4.shape[2:], mode='bilinear', align_corners=True)], dim=1)
+            elif self.config.mul_scl_ipt == 'add':
+                B, C, H, W = x.shape
+                x1_, x2_, x3_, x4_ = self.bb(F.interpolate(x, size=(H//2, W//2), mode='bilinear', align_corners=True))
+                x1 = x1 + F.interpolate(x1_, size=x1.shape[2:], mode='bilinear', align_corners=True)
+                x2 = x2 + F.interpolate(x2_, size=x2.shape[2:], mode='bilinear', align_corners=True)
+                x3 = x3 + F.interpolate(x3_, size=x3.shape[2:], mode='bilinear', align_corners=True)
+                x4 = x4 + F.interpolate(x4_, size=x4.shape[2:], mode='bilinear', align_corners=True)
+        class_preds = self.cls_head(self.avgpool(x4).view(x4.shape[0], -1)) if self.training and self.config.auxiliary_classification else None
+        if self.config.cxt:
+            x4 = torch.cat(
+                (
+                    *[
+                        F.interpolate(x1, size=x4.shape[2:], mode='bilinear', align_corners=True),
+                        F.interpolate(x2, size=x4.shape[2:], mode='bilinear', align_corners=True),
+                        F.interpolate(x3, size=x4.shape[2:], mode='bilinear', align_corners=True),
+                    ][-len(self.config.cxt):],
+                    x4
+                ),
+                dim=1
+            )
+        return (x1, x2, x3, x4), class_preds
+
+    def forward_ori(self, x):
+        ########## Encoder ##########
+        (x1, x2, x3, x4), class_preds = self.forward_enc(x)
+        if self.config.squeeze_block:
+            x4 = self.squeeze_module(x4)
+        ########## Decoder ##########
+        features = [x, x1, x2, x3, x4]
+        if self.training and self.config.out_ref:
+            features.append(laplacian(torch.mean(x, dim=1).unsqueeze(1), kernel_size=5))
+        scaled_preds = self.decoder(features)
+        return scaled_preds, class_preds
+
+    def forward(self, x):
+        scaled_preds, class_preds = self.forward_ori(x)
+        class_preds_lst = [class_preds]
+        return [scaled_preds, class_preds_lst] if self.training else scaled_preds
+
+
+class Decoder(nn.Module):
+    def __init__(self, channels):
+        super(Decoder, self).__init__()
+        self.config = Config()
+        DecoderBlock = eval(self.config.dec_blk)
+        LateralBlock = eval(self.config.lat_blk)
+
+        if self.config.dec_ipt:
+            self.split = self.config.dec_ipt_split
+            N_dec_ipt = 64
+            DBlock = SimpleConvs
+            ic = 64
+            ipt_cha_opt = 1
+            self.ipt_blk5 = DBlock(2**10*3 if self.split else 3, [N_dec_ipt, channels[0]//8][ipt_cha_opt], inter_channels=ic)
+            self.ipt_blk4 = DBlock(2**8*3 if self.split else 3, [N_dec_ipt, channels[0]//8][ipt_cha_opt], inter_channels=ic)
+            self.ipt_blk3 = DBlock(2**6*3 if self.split else 3, [N_dec_ipt, channels[1]//8][ipt_cha_opt], inter_channels=ic)
+            self.ipt_blk2 = DBlock(2**4*3 if self.split else 3, [N_dec_ipt, channels[2]//8][ipt_cha_opt], inter_channels=ic)
+            self.ipt_blk1 = DBlock(2**0*3 if self.split else 3, [N_dec_ipt, channels[3]//8][ipt_cha_opt], inter_channels=ic)
+        else:
+            self.split = None
+
+        self.decoder_block4 = DecoderBlock(channels[0]+([N_dec_ipt, channels[0]//8][ipt_cha_opt] if self.config.dec_ipt else 0), channels[1])
+        self.decoder_block3 = DecoderBlock(channels[1]+([N_dec_ipt, channels[0]//8][ipt_cha_opt] if self.config.dec_ipt else 0), channels[2])
+        self.decoder_block2 = DecoderBlock(channels[2]+([N_dec_ipt, channels[1]//8][ipt_cha_opt] if self.config.dec_ipt else 0), channels[3])
+        self.decoder_block1 = DecoderBlock(channels[3]+([N_dec_ipt, channels[2]//8][ipt_cha_opt] if self.config.dec_ipt else 0), channels[3]//2)
+        self.conv_out1 = nn.Sequential(nn.Conv2d(channels[3]//2+([N_dec_ipt, channels[3]//8][ipt_cha_opt] if self.config.dec_ipt else 0), 1, 1, 1, 0))
+
+        self.lateral_block4 = LateralBlock(channels[1], channels[1])
+        self.lateral_block3 = LateralBlock(channels[2], channels[2])
+        self.lateral_block2 = LateralBlock(channels[3], channels[3])
+
+        if self.config.ms_supervision:
+            self.conv_ms_spvn_4 = nn.Conv2d(channels[1], 1, 1, 1, 0)
+            self.conv_ms_spvn_3 = nn.Conv2d(channels[2], 1, 1, 1, 0)
+            self.conv_ms_spvn_2 = nn.Conv2d(channels[3], 1, 1, 1, 0)
+
+            if self.config.out_ref:
+                _N = 16
+                self.gdt_convs_4 = nn.Sequential(nn.Conv2d(channels[1], _N, 3, 1, 1), nn.BatchNorm2d(_N) if self.config.batch_size > 1 else nn.Identity(), nn.ReLU(inplace=True))
+                self.gdt_convs_3 = nn.Sequential(nn.Conv2d(channels[2], _N, 3, 1, 1), nn.BatchNorm2d(_N) if self.config.batch_size > 1 else nn.Identity(), nn.ReLU(inplace=True))
+                self.gdt_convs_2 = nn.Sequential(nn.Conv2d(channels[3], _N, 3, 1, 1), nn.BatchNorm2d(_N) if self.config.batch_size > 1 else nn.Identity(), nn.ReLU(inplace=True))
+
+                self.gdt_convs_pred_4 = nn.Sequential(nn.Conv2d(_N, 1, 1, 1, 0))
+                self.gdt_convs_pred_3 = nn.Sequential(nn.Conv2d(_N, 1, 1, 1, 0))
+                self.gdt_convs_pred_2 = nn.Sequential(nn.Conv2d(_N, 1, 1, 1, 0))
+                
+                self.gdt_convs_attn_4 = nn.Sequential(nn.Conv2d(_N, 1, 1, 1, 0))
+                self.gdt_convs_attn_3 = nn.Sequential(nn.Conv2d(_N, 1, 1, 1, 0))
+                self.gdt_convs_attn_2 = nn.Sequential(nn.Conv2d(_N, 1, 1, 1, 0))
+
+    def get_patches_batch(self, x, p):
+        _size_h, _size_w = p.shape[2:]
+        patches_batch = []
+        for idx in range(x.shape[0]):
+            columns_x = torch.split(x[idx], split_size_or_sections=_size_w, dim=-1)
+            patches_x = []
+            for column_x in columns_x:
+                patches_x += [p.unsqueeze(0) for p in torch.split(column_x, split_size_or_sections=_size_h, dim=-2)]
+            patch_sample = torch.cat(patches_x, dim=1)
+            patches_batch.append(patch_sample)
+        return torch.cat(patches_batch, dim=0)
+
+    def forward(self, features):
+        if self.training and self.config.out_ref:
+            outs_gdt_pred = []
+            outs_gdt_label = []
+            x, x1, x2, x3, x4, gdt_gt = features
+        else:
+            x, x1, x2, x3, x4 = features
+        outs = []
+
+        if self.config.dec_ipt:
+            patches_batch = self.get_patches_batch(x, x4) if self.split else x
+            x4 = torch.cat((x4, self.ipt_blk5(F.interpolate(patches_batch, size=x4.shape[2:], mode='bilinear', align_corners=True))), 1)
+        p4 = self.decoder_block4(x4)
+        m4 = self.conv_ms_spvn_4(p4) if self.config.ms_supervision else None
+        if self.config.out_ref:
+            p4_gdt = self.gdt_convs_4(p4)
+            if self.training:
+                # >> GT:
+                m4_dia = m4
+                gdt_label_main_4 = gdt_gt * F.interpolate(m4_dia, size=gdt_gt.shape[2:], mode='bilinear', align_corners=True)
+                outs_gdt_label.append(gdt_label_main_4)
+                # >> Pred:
+                gdt_pred_4 = self.gdt_convs_pred_4(p4_gdt)
+                outs_gdt_pred.append(gdt_pred_4)
+            gdt_attn_4 = self.gdt_convs_attn_4(p4_gdt).sigmoid()
+            # >> Finally:
+            p4 = p4 * gdt_attn_4
+        _p4 = F.interpolate(p4, size=x3.shape[2:], mode='bilinear', align_corners=True)
+        _p3 = _p4 + self.lateral_block4(x3)
+
+        if self.config.dec_ipt:
+            patches_batch = self.get_patches_batch(x, _p3) if self.split else x
+            _p3 = torch.cat((_p3, self.ipt_blk4(F.interpolate(patches_batch, size=x3.shape[2:], mode='bilinear', align_corners=True))), 1)
+        p3 = self.decoder_block3(_p3)
+        m3 = self.conv_ms_spvn_3(p3) if self.config.ms_supervision else None
+        if self.config.out_ref:
+            p3_gdt = self.gdt_convs_3(p3)
+            if self.training:
+                # >> GT:
+                # m3 --dilation--> m3_dia
+                # G_3^gt * m3_dia --> G_3^m, which is the label of gradient
+                m3_dia = m3
+                gdt_label_main_3 = gdt_gt * F.interpolate(m3_dia, size=gdt_gt.shape[2:], mode='bilinear', align_corners=True)
+                outs_gdt_label.append(gdt_label_main_3)
+                # >> Pred:
+                # p3 --conv--BN--> F_3^G, where F_3^G predicts the \hat{G_3} with xx
+                # F_3^G --sigmoid--> A_3^G
+                gdt_pred_3 = self.gdt_convs_pred_3(p3_gdt)
+                outs_gdt_pred.append(gdt_pred_3)
+            gdt_attn_3 = self.gdt_convs_attn_3(p3_gdt).sigmoid()
+            # >> Finally:
+            # p3 = p3 * A_3^G
+            p3 = p3 * gdt_attn_3
+        _p3 = F.interpolate(p3, size=x2.shape[2:], mode='bilinear', align_corners=True)
+        _p2 = _p3 + self.lateral_block3(x2)
+
+        if self.config.dec_ipt:
+            patches_batch = self.get_patches_batch(x, _p2) if self.split else x
+            _p2 = torch.cat((_p2, self.ipt_blk3(F.interpolate(patches_batch, size=x2.shape[2:], mode='bilinear', align_corners=True))), 1)
+        p2 = self.decoder_block2(_p2)
+        m2 = self.conv_ms_spvn_2(p2) if self.config.ms_supervision else None
+        if self.config.out_ref:
+            p2_gdt = self.gdt_convs_2(p2)
+            if self.training:
+                # >> GT:
+                m2_dia = m2
+                gdt_label_main_2 = gdt_gt * F.interpolate(m2_dia, size=gdt_gt.shape[2:], mode='bilinear', align_corners=True)
+                outs_gdt_label.append(gdt_label_main_2)
+                # >> Pred:
+                gdt_pred_2 = self.gdt_convs_pred_2(p2_gdt)
+                outs_gdt_pred.append(gdt_pred_2)
+            gdt_attn_2 = self.gdt_convs_attn_2(p2_gdt).sigmoid()
+            # >> Finally:
+            p2 = p2 * gdt_attn_2
+        _p2 = F.interpolate(p2, size=x1.shape[2:], mode='bilinear', align_corners=True)
+        _p1 = _p2 + self.lateral_block2(x1)
+
+        if self.config.dec_ipt:
+            patches_batch = self.get_patches_batch(x, _p1) if self.split else x
+            _p1 = torch.cat((_p1, self.ipt_blk2(F.interpolate(patches_batch, size=x1.shape[2:], mode='bilinear', align_corners=True))), 1)
+        _p1 = self.decoder_block1(_p1)
+        _p1 = F.interpolate(_p1, size=x.shape[2:], mode='bilinear', align_corners=True)
+
+        if self.config.dec_ipt:
+            patches_batch = self.get_patches_batch(x, _p1) if self.split else x
+            _p1 = torch.cat((_p1, self.ipt_blk1(F.interpolate(patches_batch, size=x.shape[2:], mode='bilinear', align_corners=True))), 1)
+        p1_out = self.conv_out1(_p1)
+
+        if self.config.ms_supervision:
+            outs.append(m4)
+            outs.append(m3)
+            outs.append(m2)
+        outs.append(p1_out)
+        return outs if not (self.config.out_ref and self.training) else ([outs_gdt_pred, outs_gdt_label], outs)
+
+
+class SimpleConvs(nn.Module):
+    def __init__(
+        self, in_channels: int, out_channels: int, inter_channels=64
+    ) -> None:
+        super().__init__()
+        self.conv1 = nn.Conv2d(in_channels, inter_channels, 3, 1, 1)
+        self.conv_out = nn.Conv2d(inter_channels, out_channels, 3, 1, 1)
+
+    def forward(self, x):
+        return self.conv_out(self.conv1(x))

RMBG/RMBG-2.0/config.json

@@ -0,0 +1,20 @@
+{
+  "_name_or_path": "ZhengPeng7/BiRefNet",
+  "architectures": [
+    "BiRefNet"
+  ],
+  "auto_map": {
+    "AutoConfig": "BiRefNet_config.BiRefNetConfig",
+    "AutoModelForImageSegmentation": "birefnet.BiRefNet"
+  },
+  "custom_pipelines": {
+    "image-segmentation": {
+      "pt": [
+        "AutoModelForImageSegmentation"
+      ],
+      "tf": [],
+      "type": "image"
+    }
+  },
+  "bb_pretrained": false
+}

checkpoints/Qwen-Rapid-AIO-NSFW-v18.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "Qwen-Rapid-AIO-NSFW-v18",
+  "model_name": "Qwen-Rapid-AIO-NSFW-v18",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/Qwen-Rapid-AIO-NSFW-v18.safetensors",
+  "size": 28431844103,
+  "modified": 1772965292.050338,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/SUPIR-v0F.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "SUPIR-v0F",
+  "model_name": "SUPIR-v0F",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/SUPIR-v0F.ckpt",
+  "size": 5329719950,
+  "modified": 1773182934.338468,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/analogMadnessSDXL_xl5.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "analogMadnessSDXL_xl5",
+  "model_name": "analogMadnessSDXL_xl5",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/analogMadnessSDXL_xl5.safetensors",
+  "size": 6938040682,
+  "modified": 1773170478.150429,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/epicrealismXL_pureFix.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "epicrealismXL_pureFix",
+  "model_name": "epicrealismXL_pureFix",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/epicrealismXL_pureFix.safetensors",
+  "size": 6938041144,
+  "modified": 1777644086.893859,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/gonzalomoXLFluxPony_v30FluxDAIO.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "gonzalomoXLFluxPony_v30FluxDAIO",
+  "model_name": "gonzalomoXLFluxPony_v30FluxDAIO",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/gonzalomoXLFluxPony_v30FluxDAIO.safetensors",
+  "size": 17068674198,
+  "modified": 1777201626.156541,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/illustriousRealismBy_v10.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "illustriousRealismBy_v10",
+  "model_name": "illustriousRealismBy_v10",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/illustriousRealismBy_v10.safetensors",
+  "size": 6938062210,
+  "modified": 1772965292.082859,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/illustriousRealismBy_v10VAE.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "illustriousRealismBy_v10VAE",
+  "model_name": "illustriousRealismBy_v10VAE",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/illustriousRealismBy_v10VAE.safetensors",
+  "size": 0,
+  "modified": 1772965292.112515,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/intorealismUltra_v10.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "intorealismUltra_v10",
+  "model_name": "intorealismUltra_v10",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/intorealismUltra_v10.safetensors",
+  "size": 6938040706,
+  "modified": 1777300584.216715,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}

checkpoints/juggernautXL_ragnarokBy.metadata.json

@@ -0,0 +1,25 @@
+{
+  "file_name": "juggernautXL_ragnarokBy",
+  "model_name": "juggernautXL_ragnarokBy",
+  "file_path": "/workspace/runpod-slim/ComfyUI/models/checkpoints/juggernautXL_ragnarokBy.safetensors",
+  "size": 7105350162,
+  "modified": 1777660870.143452,
+  "sha256": "",
+  "base_model": "Unknown",
+  "preview_url": "",
+  "preview_nsfw_level": 0,
+  "notes": "",
+  "from_civitai": false,
+  "civitai": {},
+  "tags": [],
+  "modelDescription": "",
+  "civitai_deleted": false,
+  "favorite": false,
+  "exclude": false,
+  "db_checked": false,
+  "skip_metadata_refresh": false,
+  "metadata_source": null,
+  "last_checked_at": 0,
+  "hash_status": "pending",
+  "sub_type": "checkpoint"
+}