diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/.readthedocs.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/.readthedocs.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..c6cf8e2a075ea15f39dc7aba8faa98f464f52fe6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/.readthedocs.yaml
@@ -0,0 +1,17 @@
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.8"
+
+formats:
+    - epub
+
+sphinx:
+  configuration: docs/en/conf.py
+
+python:
+  install:
+    - requirements: requirements/docs.txt
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/ConfigSystem.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/ConfigSystem.md
new file mode 100644
index 0000000000000000000000000000000000000000..120e0cb05a36ed9d911811c71d6201b319dda7c1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/ConfigSystem.md
@@ -0,0 +1,67 @@
+# Config System
+
+By default, VLMEvalKit launches the evaluation by setting the model name(s) (defined in `/vlmeval/config.py`) and dataset name(s) (defined in `vlmeval/dataset/__init__.py` or `vlmeval/dataset/video_dataset_config.py`) in the `run.py` script with the `--model` and `--data` arguments. Such approach is simple and efficient in most scenarios, however, it may not be flexible enough when the user wants to evaluate multiple models / datasets with different settings.
+
+To address this, VLMEvalKit provides a more flexible config system. The user can specify the model and dataset settings in a json file, and pass the path to the config file to the `run.py` script with the `--config` argument. Here is a sample config json:
+
+```json
+{
+    "model": {
+        "GPT4o_20240806_T00_HIGH": {
+            "class": "GPT4V",
+            "model": "gpt-4o-2024-08-06",
+            "temperature": 0,
+            "img_detail": "high"
+        },
+        "GPT4o_20240806_T10_Low": {
+            "class": "GPT4V",
+            "model": "gpt-4o-2024-08-06",
+            "temperature": 1.0,
+            "img_detail": "low"
+        },
+        "GPT4o_20241120": {}
+    },
+    "data": {
+        "MME-RealWorld-Lite": {
+            "class": "MMERealWorld",
+            "dataset": "MME-RealWorld-Lite"
+        },
+        "MMBench_DEV_EN_V11": {
+            "class": "ImageMCQDataset",
+            "dataset": "MMBench_DEV_EN_V11"
+        },
+        "MMBench_Video_8frame_nopack":{},
+        "Video-MME_16frame_subs": {
+            "class": "VideoMME",
+            "dataset": "Video-MME",
+            "nframe": 16,
+            "use_subtitle": true
+        }
+    }
+}
+```
+
+Explanation of the config json:
+
+1. Now we support two fields: `model` and `data`, each of which is a dictionary. The key of the dictionary is the name of the model / dataset (set by the user), and the value is the setting of the model / dataset.
+2. For items in `model`, the value is a dictionary containing the following keys:
+    - `class`: The class name of the model, which should be a class name defined in `vlmeval/vlm/__init__.py` (open-source models) or `vlmeval/api/__init__.py` (API models).
+    - Other kwargs: Other kwargs are model-specific parameters, please refer to the definition of the model class for detailed usage. For example, `model`, `temperature`, `img_detail` are arguments of the `GPT4V` class. It's noteworthy that the `model` argument is required by most model classes.
+    - Tip: The defined model in the `supported_VLM` of `vlmeval/config.py` can be used as a shortcut, for example, `GPT4o_20241120: {}` is equivalent to `GPT4o_20241120: {'class': 'GPT4V', 'model': 'gpt-4o-2024-11-20', 'temperature': 0, 'img_size': -1, 'img_detail': 'high', 'retry': 10, 'verbose': False}`
+3. For the dictionary `data`, we suggest users to use the official dataset name as the key (or part of the key), since we frequently determine the post-processing / judging settings based on the dataset name. For items in `data`, the value is a dictionary containing the following keys:
+    - `class`: The class name of the dataset, which should be a class name defined in `vlmeval/dataset/__init__.py`.
+    - Other kwargs: Other kwargs are dataset-specific parameters, please refer to the definition of the dataset class for detailed usage. Typically, the `dataset` argument is required by most dataset classes. It's noteworthy that the `nframe` argument or `fps` argument is required by most video dataset classes.
+    - Tip: The defined dataset in the `supported_video_datasets` of `vlmeval/dataset/video_dataset_config.py` can be used as a shortcut, for example, `MMBench_Video_8frame_nopack: {}` is equivalent to `MMBench_Video_8frame_nopack: {'class': 'MMBenchVideo', 'dataset': 'MMBench-Video', 'nframe': 8, 'pack': False}`.
+Saving the example config json to `config.json`, you can launch the evaluation by:
+
+```bash
+python run.py --config config.json
+```
+
+That will generate the following output files under the working directory `$WORK_DIR` (Following the format `{$WORK_DIR}/{$MODEL_NAME}/{$MODEL_NAME}_{$DATASET_NAME}_*`):
+
+- `$WORK_DIR/GPT4o_20240806_T00_HIGH/GPT4o_20240806_T00_HIGH_MME-RealWorld-Lite*`
+- `$WORK_DIR/GPT4o_20240806_T10_Low/GPT4o_20240806_T10_Low_MME-RealWorld-Lite*`
+- `$WORK_DIR/GPT4o_20240806_T00_HIGH/GPT4o_20240806_T00_HIGH_MMBench_DEV_EN_V11*`
+- `$WORK_DIR/GPT4o_20240806_T10_Low/GPT4o_20240806_T10_Low_MMBench_DEV_EN_V11*`
+...
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Contributors.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Contributors.md
new file mode 100644
index 0000000000000000000000000000000000000000..ddf50c6c4eb7caf352fe29069e65a93a2d4cac49
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Contributors.md
@@ -0,0 +1,21 @@
+# Contributors
+
+## Contributors w. 3+ Major Contributions
+
+> In this section, we list all the contributors who have made significant contributions (3+) to the development of VLMEvalKit.
+
+New Qualified Contributors (2024.09):
+
+1. [amitbcp](https://github.com/amitbcp): The contributor helped support MUIRBench, Phi-3.5, Idefics3, VILA, and xGen-MM
+2. [czczup](https://github.com/czczup): The contributor helped support the InternVL Series (V1.5, Mini-InternVL, V2, etc.)
+3. [DseidLi](https://github.com/DseidLi): The contributor helped support LLaVA-OneVision, GQA, and developed the readthedocs site for VLMEvalKit
+4. [mayubo2333](https://github.com/mayubo2333): The contributor helped support MMLongBench, SlideVQA, and DUDE
+5. [sun-hailong](https://github.com/sun-hailong): The contributor helped support A-OKVQA, Parrot, MMMB, and MTL-MMBench
+6. [PhoenixZ810](https://github.com/PhoenixZ810): The contributor helped support Video-ChatGPT, Chat-UniVI, and Llama-VID
+7. [Cuiunbo](https://github.com/Cuiunbo): The contributor helped support OmniLMM-12B, MiniCPM-V Series (V1, V2, V2.5)
+
+## Full Contributor List
+
+> In this section, we list all the contributors as well as their corresponding contributions to the development of VLMEvalKit.
+
+TBD.
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Development.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Development.md
new file mode 100644
index 0000000000000000000000000000000000000000..0fe5a60e22252a2098ed0edd47e6f219d51a0a4d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Development.md
@@ -0,0 +1,145 @@
+# Develop new Benchmark / MLLM
+
+>  🛠️ How to implement a new Benchmark / VLM in VLMEvalKit?
+
+## Implement a new benchmark
+
+Example PR: **Math-Vision Benchmark** ([#292](https://github.com/open-compass/VLMEvalKit/pull/292/files))
+
+In VLMEvalKit, benchmarks are organized as dataset classes. When you try to implement a new benchmark, you can either reuse existing dataset classes (*e.g.*, You can reuse `ImageMCQDataset` when implementing a new multi-choice benchmark), or support a new dataset class. Each dataset must have the following two member functions (either reuse the one of the parent class or implement your own):
+
+- `build_prompt(self, line)`: The function input `line` is an integer (the sample index) or a `pd.Series` object (the raw record of the sample). The function outputs a `multi-modal message`, serving as the input of an MLLM. The `multi-modal message` is an interleaved list of multi-modal messages adopting the following format (the example includes an image and a text message): `[dict(type='image', value=IMAGE_PTH), dict(type='text', value=prompt)]`.
+- `evaluate(self, eval_file,  **judge_kwargs)`: The function input `eval_file` is the MLLM prediction (typically in `.xlsx` format). If the benchmark requires an external LLM (typically GPT) for evaluation, then `judge_kwargs` can pass the arguments for the LLM. The function outputs the benchmark evaluation results (metrics) in the form of `dict` or `pd.DataFrame`.
+
+We then brief the typical steps to implement a new benchmark under VLMEvalKit:
+
+### 1. Prepare your benchmark tsv file
+
+Currently, we organize a benchmark as one single TSV file. During inference, the data file will be automatically downloaded from the definited `DATASET_URL` link to `$LMUData` file (default path is `$HOME/LMUData`, if not set explicitly). You can upload the prepared TSV file to a downloadable address (e.g., Huggingface) or send it to us at <opencompass@pjlab.org.cn>. We will assist in uploading the dataset to the server. You can also customize `LMUData` path in the environment variable `LMUData=/path/to/your/data`.
+
+The contents of the TSV file consist of:
+
+| Dataset Name \ Fields                   | index | image | image_path | question | hint | multi-choice<br>options | answer | category | l2-category | split |
+| --------------------------------------- | ----- | ----- | ---------- | -------- | ---- | ----------------------- | ------ | -------- | ----------- | ----- |
+| MMBench_DEV_[CN/EN]                     | ✅     | ✅     |            | ✅        | ✅    | ✅                       | ✅      | ✅        | ✅           | ✅     |
+| MMBench_TEST_[CN/EN]                    | ✅     | ✅     |            | ✅        | ✅    | ✅                       |        | ✅        | ✅           | ✅     |
+| CCBench                                 | ✅     | ✅     |            | ✅        |      | ✅                       | ✅      | ✅        |             |       |
+| SEEDBench_IMG                           | ✅     | ✅     |            | ✅        |      | ✅                       | ✅      | ✅        |             |       |
+| MME                                     | ✅     | ✅     |            | ✅        |      |                         | ✅      | ✅        |             |       |
+| MMVet                                   | ✅     | ✅     |            | ✅        |      |                         | ✅      | ✅        |             |       |
+| MMMU_DEV_VAL                            | ✅     | ✅     | ✅          | ✅        |      | ✅                       | ✅      | ✅        | ✅           | ✅     |
+| COCO_VAL                                | ✅     | ✅     |            |          |      |                         | ✅      |          |             |       |
+| OCRVQA_[TEST/TESTCORE]                  | ✅     | ✅     |            | ✅        |      |                         | ✅      |          |             |       |
+| TextVQA_VAL                             | ✅     | ✅     |            | ✅        |      |                         | ✅      |          |             |       |
+| VCR_[EN/ZH]\_[EASY/HARD]\_[ALL/500/100] | ✅     | ✅     |            | ✅        |      |                         | ✅      |          |             |       |
+| MMMB_[en/cn/pt/ar/tr/ru] | ✅     | ✅     |            | ✅        | ✅     | ✅     | ✅      | ✅         |             |✅       |
+| MMBench_dev_[en/cn/pt/ar/tr/ru] | ✅     | ✅     |            | ✅        | ✅     | ✅     | ✅      | ✅         | ✅            |✅       |
+
+<div align="center"><b>Table 1. TSV fields of supported datasets.</b></div>
+
+**Intro to mandatory fields in the `TSV` file:**
+
+- **index:** Integer, Unique for each line in `tsv`
+- **image:** The base64 of the image, you can use APIs implemented in `vlmeval/smp/vlm.py` for encoding and decoding:
+  - Encoding: `encode_image_to_base64 `(for PIL Image) / `encode_image_file_to_base64` (for image file path)
+  - Decoding: `decode_base64_to_image`(for PIL Image) / `decode_base64_to_image_file` (for image file path)
+- **question**: The question corresponding to the image, a string
+- **answer**: The answer to the question, a string. The `test` split does not need this field
+
+### 2. Cutomize your benchmark prompt
+
+`ImageBaseDataset` defines the default prompt format. If you need to add prompts specific to the dataset or input data in the `Interleave` format to the model, you can implement this through the `build_prompt(line)` function. This function takes a line from a TSV file as input, containing fields such as index, image, question, etc. The function returns a dictionary list of multimodal messages `msg` in the format `[dict(type='image', value=IMAGE_PTH), dict(type='text', value=prompt)]`, including the image path and the text prompt to be input into VLMs. For interleave type inputs, you can directly place the dictionary of the image path at the image token position.
+
+### 3. Cutomize your benchmark metrics
+
+To add evaluation for a new benchmark, you need to customize a class object to implement the dataset’s metrics calculation. Multimodal datasets inherit from the `ImageBaseDataset` object in `vlmeval/dataset/image_base.py`. The TYPE defines the type of dataset, `DATASET_URL` is the download address of the dataset, and `DATASET_MD5` is the MD5 checksum for consistency checking of the dataset file.
+
+In this class, **you need to implement** the `evaluate(eval_file, **judge_kwargs)` class function to calculate metrics and output results for the custom dataset. The function input `eval_file` is the path to the model prediction results file `{model_name}_{dataset}.xlsx`. This file can be read as a pandas.DataFrame using the `load(eval_file)` method, containing fields such as index, question, answer, category, prediction, etc. The judge_kwargs will pass a dictionary related to evaluation, such as the name of the `judge model`, the number of API request threads, etc. **The return value** of the function is the calculated accuracy and other metrics, formatted as a dictionary composed of lists, organized into a pandas.DataFrame.
+
+## Implement a new model
+
+Example PR: **Support LLaVA-Next-Interleave** ([#294](https://github.com/open-compass/VLMEvalKit/pull/294))
+
+**1. Support `generate_inner` API (mandatory).**
+
+All existing models are implemented in `vlmeval/vlm`. For a minimal model, your model class **must implement the method** `generate_inner(msgs, dataset=None)`. In this function, you feed a multi-modal message to your VLM and return the VLM prediction (which is a string). The optional argument `dataset` can be used as the flag for the model to switch among various inference strategies.
+
+The multi-modal messages `msgs` is a list of dictionaries, each dictionary has two keys: type and value:
+- `type`: We currently support two types, choices are ["image", "text"].
+- `value`: When type=='text' , the value is the text message (a single string); when type=='image', the value can be the local path of an image file, or the image URL.
+
+Currently a multi-modal message may contain arbitrarily interleaved images and texts. If your model do not support that, a practice can be taking the 1st image and concatenated text messages as the input. You can set the `INTERLEAVE = False` in your model class and use `self.message_to_promptimg(message, dataset=dataset)` to build your prompt and the first image's path.
+
+Here are some examples of multi-modal messages:
+
+```python
+IMAGE_PTH = 'assets/apple.jpg'
+IMAGE_URL = 'https://raw.githubusercontent.com/open-compass/VLMEvalKit/main/assets/apple.jpg'
+msg1 = [
+    dict(type='image', value=IMAGE_PTH),
+    dict(type='text', value='What is in this image?')
+]
+msg2 = [
+    dict(type='image', value=IMAGE_URL),
+    dict(type='image', value=IMAGE_URL),
+    dict(type='text', value='How many apples are there in these images?')
+]
+response = model.generate(msg1)
+```
+
+For convenience sake, we also support to take a list of string as inputs. In that case, we will check if a string is an image path or image URL and automatically convert it to the list[dict] format:
+
+```python
+IMAGE_PTH = 'assets/apple.jpg'
+IMAGE_URL = 'https://raw.githubusercontent.com/open-compass/VLMEvalKit/main/assets/apple.jpg'
+msg1 = [IMAGE_PTH, 'What is in this image?']
+msg2 = [IMAGE_URL, IMAGE_URL,  'How many apples are there in these images?']
+response = model.generate(msg1)
+```
+
+**Support Custom Prompt (optional).**
+
+Besides, your model can support **custom prompt building** by implementing two optional methods: `use_custom_prompt(dataset)` and `build_prompt(line, dataset=None)`.
+
+Both functions take the dataset name as the input：
+
+-  `use_custom_prompt(dataset)` returns a boolean flag, indicating whether the model should use the custom prompt building strategy.
+- If `use_custom_prompt(dataset)` returns True, `build_prompt(line, dataset)` should return a customly bulit multimodal message for the corresponding `dataset`, given `line`, which is a dictionary that includes the necessary information of a data sample. If `use_custom_prompt(dataset)` returns False, the default prompt building strategy will be used.
+
+**Support multi-turn chatting (optional).**
+
+You can also support the multi-turn chatting and evaluation with your VLM by supporting the `chat_inner(message, dataset)` function. The function outputs a single string response, and the `message` is a list of chat history, following the below format.
+
+```python
+# Assume msg1, msg2, msg3, ... are multi-modal messages following the previously described format
+# `chat_inner` take the following chat history list as input:
+message = [
+    dict(role='user', content=msg1),
+    dict(role='assistant', content=msg2),
+    dict(role='user', content=msg3),
+    dict(role='assistant', content=msg4),
+	......
+    dict(role='user', content=msgn),
+]
+# `message` should contain an odd number of chat utterances, the role of utterances should be interleaved "user" and "assistant", with the role of the last utterance to be "user".
+# The chat function will call `chat_inner`
+response = model.chat(message)
+```
+
+### Example PRs:
+
+- VLM that doesn't support interleaved images and texts, and does not use custom prompts: [[Model] Support glm-4v-9b](https://github.com/open-compass/VLMEvalKit/pull/221)
+- VLM that supports interleaved images and texts and custom prompts: [Add MiniCPM-Llama3-V-2.5](https://github.com/open-compass/VLMEvalKit/pull/205)
+- VLM API: [Feature add glmv](https://github.com/open-compass/VLMEvalKit/pull/201)
+
+## Contribute to VLMEvalKit
+
+If you want to contribute codes to **VLMEvalKit**, please do the pre-commit check before you submit a PR. That helps to keep the code tidy.
+
+```bash
+# Under the directory of VLMEvalKit, install the pre-commit hook:
+pip install pre-commit
+pre-commit install
+pre-commit run --all-files
+# Then you can commit your code.
+```
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/EvalByLMDeploy.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/EvalByLMDeploy.md
new file mode 100644
index 0000000000000000000000000000000000000000..fc0a8c38c26542eb44acdc74b28aaca9755735ba
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/EvalByLMDeploy.md
@@ -0,0 +1,27 @@
+# Using LMDeploy to Accelerate Evaluation and Inference
+
+VLMEvalKit supports testing VLM models deployed by LMDeploy. Below, we use InternVL2-8B as an example to show how to test the model.
+
+## Step 0: Install LMDeploy
+
+```bash
+pip install lmdeploy
+```
+For other installation methods, you can refer to LMDeploy's [documentation](https://github.com/InternLM/lmdeploy).
+
+## Step 1: Start the Inference Service
+
+```bash
+lmdeploy serve api_server OpenGVLab/InternVL2-8B --model-name InternVL2-8B
+```
+> [!IMPORTANT]
+> Since models in VLMEvalKit may have custom behaviors when building prompts for different datasets, such as InternVL2's handling of HallusionBench, it is necessary to specify `--model-name` when starting the server. This allows the VLMEvalKit to select appropriate prompt construction strategy based on the name when using the LMDeploy API.
+>
+> If `--server-port`, is specified, the corresponding environment variable `LMDEPLOY_API_BASE` needs to be set.
+
+
+## Step 2: Evaluation
+
+```bash
+python run.py --data MMStar --model lmdeploy --verbose --api-nproc 64
+```
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Makefile b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Makefile
new file mode 100644
index 0000000000000000000000000000000000000000..d4bb2cbb9eddb1bb1b4f366623044af8e4830919
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Quickstart.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Quickstart.md
new file mode 100644
index 0000000000000000000000000000000000000000..264d2cddfdcd149cbd52843e6097a5958548d479
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/Quickstart.md
@@ -0,0 +1,236 @@
+# Quickstart
+
+Before running the evaluation script, you need to **configure** the VLMs and set the model_paths properly.
+
+After that, you can use a single script `run.py` to inference and evaluate multiple VLMs and benchmarks at a same time.
+
+## Step 0. Installation & Setup essential keys
+
+**Installation.**
+
+```bash
+git clone https://github.com/open-compass/VLMEvalKit.git
+cd VLMEvalKit
+pip install -e .
+```
+
+**Setup Keys.**
+
+To infer with API models (GPT-4v, Gemini-Pro-V, etc.) or use LLM APIs as the **judge or choice extractor**, you need to first setup API keys. VLMEvalKit will use an judge **LLM** to extract answer from the output if you set the key, otherwise it uses the **exact matching** mode (find "Yes", "No", "A", "B", "C"... in the output strings). **The exact matching can only be applied to the Yes-or-No tasks and the Multi-choice tasks.**
+- You can place the required keys in `$VLMEvalKit/.env` or directly set them as the environment variable. If you choose to create a `.env` file, its content will look like:
+
+  ```bash
+  # The .env file, place it under $VLMEvalKit
+  # API Keys of Proprietary VLMs
+  # QwenVL APIs
+  DASHSCOPE_API_KEY=
+  # Gemini w. Google Cloud Backends
+  GOOGLE_API_KEY=
+  # OpenAI API
+  OPENAI_API_KEY=
+  OPENAI_API_BASE=
+  # StepAI API
+  STEPAI_API_KEY=
+  # REKA API
+  REKA_API_KEY=
+  # GLMV API
+  GLMV_API_KEY=
+  # CongRong API
+  CW_API_BASE=
+  CW_API_KEY=
+  # SenseNova API
+  SENSENOVA_API_KEY=
+  # Hunyuan-Vision API
+  HUNYUAN_SECRET_KEY=
+  HUNYUAN_SECRET_ID=
+  # LMDeploy API
+  LMDEPLOY_API_BASE=
+  # MiniMax API
+  MINIMAX_API_KEY=
+  # You can also set a proxy for calling api models during the evaluation stage
+  EVAL_PROXY=
+  ```
+
+- Fill the blanks with your API keys (if necessary). Those API keys will be automatically loaded when doing the inference and evaluation.
+## Step 1. Configuration
+
+**VLM Configuration**: All VLMs are configured in `vlmeval/config.py`. Few legacy VLMs (like MiniGPT-4, LLaVA-v1-7B) requires additional configuration (configuring the code / model_weight root in the config file). During evaluation, you should use the model name specified in `supported_VLM` in `vlmeval/config.py` to select the VLM. Make sure you can successfully infer with the VLM before starting the evaluation with the following command `vlmutil check {MODEL_NAME}`.
+
+Note: For the Qwen-VL series models (Qwen-VL, Qwen2-VL, Qwen2.5-VL), the upper and lower bounds of the number of pixels specified in vlmeval/config.py are as follows:
+
+```
+min_pixels=1280 * 28 * 28,
+max_pixels=16384 * 28 * 28,
+```
+Where 1280 is the maximum value recommended by Qwen for balancing performance, computational resources, and memory, and 16384 is the theoretical maximum value for model input. This setting has a positive effect on some vision tasks that require high resolution (such as document understanding). However, there is no practical basis for considering this setting. If you need to align with the official settings, you can remove these two values, or set them to the following values from the official Qwen demo:
+
+```
+min_pixels=256 * 28 * 28,
+max_pixels=1280 * 28 * 28,
+```
+
+## Step 2. Evaluation
+
+**New!!!**  We integrated a new config system to enable more flexible evaluation settings. Check the [Document](/docs/en/ConfigSystem.md) or run `python run.py --help` for more details 🔥🔥🔥
+
+We use `run.py` for evaluation. To use the script, you can use `$VLMEvalKit/run.py` or create a soft-link of the script (to use the script anywhere):
+
+**Arguments**
+
+- `--data (list[str])`: Set the dataset names that are supported in VLMEvalKit (names can be found in the codebase README).
+- `--model (list[str])`: Set the VLM names that are supported in VLMEvalKit (defined in `supported_VLM` in `vlmeval/config.py`).
+- `--mode (str, default to 'all', choices are ['all', 'infer'])`: When `mode` set to "all", will perform both inference and evaluation; when set to "infer", will only perform the inference.
+- `--api-nproc (int, default to 4)`: The number of threads for OpenAI API calling.
+- `--work-dir (str, default to '.')`: The directory to save evaluation results.
+
+**Command for Evaluating Image Benchmarks **
+
+You can run the script with `python` or `torchrun`:
+
+```bash
+# When running with `python`, only one VLM instance is instantiated, and it might use multiple GPUs (depending on its default behavior).
+# That is recommended for evaluating very large VLMs (like IDEFICS-80B-Instruct).
+
+# IDEFICS-80B-Instruct on MMBench_DEV_EN, MME, and SEEDBench_IMG, Inference and Evalution
+python run.py --data MMBench_DEV_EN MME SEEDBench_IMG --model idefics_80b_instruct --verbose
+# IDEFICS-80B-Instruct on MMBench_DEV_EN, MME, and SEEDBench_IMG, Inference only
+python run.py --data MMBench_DEV_EN MME SEEDBench_IMG --model idefics_80b_instruct --verbose --mode infer
+
+# When running with `torchrun`, one VLM instance is instantiated on each GPU. It can speed up the inference.
+# However, that is only suitable for VLMs that consume small amounts of GPU memory.
+
+# IDEFICS-9B-Instruct, Qwen-VL-Chat, mPLUG-Owl2 on MMBench_DEV_EN, MME, and SEEDBench_IMG. On a node with 8 GPU. Inference and Evaluation.
+torchrun --nproc-per-node=8 run.py --data MMBench_DEV_EN MME SEEDBench_IMG --model idefics_80b_instruct qwen_chat mPLUG-Owl2 --verbose
+# Qwen-VL-Chat on MME. On a node with 2 GPU. Inference and Evaluation.
+torchrun --nproc-per-node=2 run.py --data MME --model qwen_chat --verbose
+```
+
+**Command for Evaluating Video Benchmarks**
+
+```bash
+# When running with `python`, only one VLM instance is instantiated, and it might use multiple GPUs (depending on its default behavior).
+# That is recommended for evaluating very large VLMs (like IDEFICS-80B-Instruct).
+
+# IDEFICS2-8B on MMBench-Video, with 8 frames as inputs and vanilla evaluation. On a node with 8 GPUs. MMBench_Video_8frame_nopack is a defined dataset setting in `vlmeval/dataset/video_dataset_config.py`.
+torchrun --nproc-per-node=8 run.py --data MMBench_Video_8frame_nopack --model idefics2_8
+# GPT-4o (API model) on MMBench-Video, with 1 frame per second as inputs and pack evaluation (all questions of a video in a single query).
+python run.py --data MMBench_Video_1fps_pack --model GPT4o
+```
+
+The evaluation results will be printed as logs, besides. **Result Files** will also be generated in the directory `$YOUR_WORKING_DIRECTORY/{model_name}`. Files ending with `.csv` contain the evaluated metrics.
+
+### Frequently Asked Questions
+
+#### Constructing Input Prompt: The `build_prompt()` Function
+If you find that the model's output does not match the expected results when evaluating a specific benchmark, it could be due to the model not constructing the input prompt correctly.
+
+In VLMEvalKit, each `dataset` class includes a function named `build_prompt()`, which is responsible for formatting input questions. Different benchmarks can either customize their own `build_prompt()` function or use the default implementation.
+
+For instance, when handling the default [Multiple-Choice QA](https://github.com/open-compass/VLMEvalKit/blob/43af13e052de6805a8b08cd04aed5e0d74f82ff5/vlmeval/dataset/image_mcq.py#L164), the `ImageMCQDataset.build_prompt()` method combines elements such as `hint`, `question`, and `options` (if present in the dataset) into a complete question format, as shown below:
+
+```
+HINT
+QUESTION
+Options:
+A. Option A
+B. Option B
+···
+Please select the correct answer from the options above.
+```
+
+Additionally, since different models may have varying evaluation requirements, VLMEvalKit also supports customizing the prompt construction method at the model level through `model.build_prompt()`. For an example, you can refer to [InternVL](https://github.com/open-compass/VLMEvalKit/blob/43af13e052de6805a8b08cd04aed5e0d74f82ff5/vlmeval/vlm/internvl_chat.py#L324).
+
+**Note: If both `model.build_prompt()` and `dataset.build_prompt()` are defined, `model.build_prompt()` will take precedence over `dataset.build_prompt()`, effectively overriding it.**
+
+Some models, such as Qwen2VL and InternVL, define extensive prompt-building methods for various types of benchmarks. To provide more flexibility in adapting to different benchmarks, VLMEvalKit allows users to customize the `model.use_custom_prompt()` function within the model. By adding or modifying the `use_custom_prompt()` function, you can decide which benchmarks should utilize the model's custom prompt logic. Below is an example:
+
+```python
+def use_custom_prompt(self, dataset: str) -> bool:
+    from vlmeval.dataset import DATASET_TYPE, DATASET_MODALITY
+    dataset_type = DATASET_TYPE(dataset, default=None)
+    if not self._use_custom_prompt:
+        return False
+    if listinstr(['MMVet'], dataset):
+        return True
+    if dataset_type == 'MCQ':
+        return True
+    if DATASET_MODALITY(dataset) == 'VIDEO':
+        return False
+    return False
+```
+Only when the `use_custom_prompt()` function returns `True` will VLMEvalKit call the model's `build_prompt()` function for the current benchmark.
+With this approach, you can flexibly control which benchmarks use the model's custom prompt logic based on your specific needs, thereby better adapting to different models and tasks.
+
+#### Model Splitting
+
+Currently, VLMEvalKit automatically supports GPU resource allocation and model splitting between processes on the same machine. This feature is supported when the inference backend is `lmdeploy` or `transformers`, with the following behaviors:
+
+- When launching with `python` command, the model is by default allocated to all available GPUs. If you want to specify which GPUs to use, you can use `CUDA_VISIBLE_DEVICES` environment variable.
+- When starting with `torchrun` command, each model instance will be allocated to `N_GPU // N_PROC` GPUs, where `N_PROC` is the number of processes specified by the `--nproc-per-node` parameter in the torchrun command. The value of `N_GPU` is determined as follows:
+  - If `CUDA_VISIBLE_DEVICES` environment variable is not set, `N_GPU` will be the total number of available GPUs.
+  - If `CUDA_VISIBLE_DEVICES` environment variable is set, `N_GPU` will be the number of GPUs specified by the `CUDA_VISIBLE_DEVICES` environment variable, and only the specified GPUs will be utilized.
+Below are specific examples of running evaluation tasks on a machine equipped with 8 GPUs:
+
+```bash
+<!-- Launch two model instances in data parallel, each instance using 4 GPUs -->
+torchrun --nproc-per-node=2 run.py --data MMBench_DEV_EN --model InternVL3-78B
+<!-- Launch one model instance, using all 8 GPUs -->
+python run.py --data MMBench_DEV_EN --model InternVL3-78B
+<!-- Launch three model instances, each instance using 2 GPUs, GPU 0 and 7 are not used -->
+CUDA_VISIBLE_DEVICES=1,2,3,4,5,6 torchrun --nproc-per-node=3 run.py --data MMBench_DEV_EN --model InternVL3-38B
+```
+
+PS: The feature is not compatible with `vllm` backend. When you evaluate a model with `vllm` backend, please use `python` to launch, and all visible GPU devices will be used.
+
+#### Performance Discrepancies
+
+Model performance may vary across different environments. As a result, you might observe discrepancies between your evaluation results and those listed on the official VLMEvalKit leaderboard. These differences could be attributed to variations in versions of libraries such as `transformers`, `cuda`, and `torch`.
+
+Besides, if you encounter unexpected performance, we recommend first reviewing the local generation records (`{model}_{dataset}.xlsx`) or the evaluation records (`{model}_{dataset}_{judge_model}.xlsx`). This may help you better understand the evaluation outcomes and identify potential issues.
+
+## Deploy a local language model as the judge / choice extractor
+The default setting mentioned above uses OpenAI's GPT as the judge LLM. However, you can also deploy a local judge LLM with [LMDeploy](https://github.com/InternLM/lmdeploy).
+
+First install:
+```
+pip install lmdeploy openai
+```
+
+And then deploy a local judge LLM with the single line of code. LMDeploy will automatically download the model from Huggingface. Assuming we use internlm2-chat-1_8b as the judge, port 23333, and the key sk-123456 (the key must start with "sk-" and follow with any number you like):
+```
+lmdeploy serve api_server internlm/internlm2-chat-1_8b --server-port 23333
+```
+
+You need to get the model name registered by LMDeploy with the following python code:
+```
+from openai import OpenAI
+client = OpenAI(
+    api_key='sk-123456',
+    base_url="http://0.0.0.0:23333/v1"
+)
+model_name = client.models.list().data[0].id
+```
+
+Now set some environment variables to tell VLMEvalKit how to use the local judge LLM. As mentioned above, you can also set them in `$VLMEvalKit/.env` file:
+```
+OPENAI_API_KEY=sk-123456
+OPENAI_API_BASE=http://0.0.0.0:23333/v1/chat/completions
+LOCAL_LLM=<model_name you get>
+```
+
+Finally, you can run the commands in step 2 to evaluate your VLM with the local judge LLM.
+
+Note that
+
+- If you hope to deploy the judge LLM in a single GPU and evaluate your VLM on other GPUs because of limited GPU memory, try `CUDA_VISIBLE_DEVICES=x` like
+```
+CUDA_VISIBLE_DEVICES=0 lmdeploy serve api_server internlm/internlm2-chat-1_8b --server-port 23333
+CUDA_VISIBLE_DEVICES=1,2,3 torchrun --nproc-per-node=3 run.py --data HallusionBench  --model qwen_chat --verbose
+```
+- If the local judge LLM is not good enough in following the instructions, the evaluation may fail. Please report such failures (e.g., by issues).
+- It's possible to deploy the judge LLM in different ways, e.g., use a private LLM (not from HuggingFace) or use a quantized LLM. Please refer to the [LMDeploy doc](https://lmdeploy.readthedocs.io/en/latest/serving/api_server.html). You can use any other deployment framework if they support OpenAI API.
+
+
+### Using LMDeploy to Accelerate Evaluation and Inference
+
+You can refer this [doc](/docs/en/EvalByLMDeploy.md)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/_templates/404.html b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/_templates/404.html
new file mode 100644
index 0000000000000000000000000000000000000000..64910175d5d69946845b04d5e6a378de205e8388
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/_templates/404.html
@@ -0,0 +1,18 @@
+{% extends "layout.html" %}
+
+{% block body %}
+
+<h1>Page Not Found</h1>
+<p>
+  The page you are looking for cannot be found.
+</p>
+<p>
+  If you just switched documentation versions, it is likely that the page you were on is moved. You can look for it in
+  the content table left, or go to <a href="{{ pathto(root_doc) }}">the homepage</a>.
+</p>
+<!-- <p>
+  If you cannot find documentation you want, please <a
+    href="">open an issue</a> to tell us!
+</p> -->
+
+{% endblock %}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/conf.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/conf.py
new file mode 100644
index 0000000000000000000000000000000000000000..e1d103e156699646e7dda5e17e7b8b12c5b7989c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/conf.py
@@ -0,0 +1,234 @@
+# flake8: noqa
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+
+import ast
+import os
+import subprocess
+import sys
+
+import pytorch_sphinx_theme
+from sphinx.builders.html import StandaloneHTMLBuilder
+
+sys.path.insert(0, os.path.abspath('../../'))
+
+# -- Project information -----------------------------------------------------
+
+project = 'VLMEvalKit'
+copyright = '2023, VLMEvalKit'
+author = 'VLMEvalKit Authors'
+
+# The full version, including alpha/beta/rc tags
+version_file = '../../vlmeval/__init__.py'
+
+
+def get_version():
+    with open(version_file, 'r') as f:
+        file_content = f.read()
+    # Parse the file content into an abstract syntax tree (AST)
+    tree = ast.parse(file_content, filename=version_file)
+
+    # Iterate through the body of the AST, looking for an assignment to __version__
+    for node in tree.body:
+        if isinstance(node, ast.Assign):
+            for target in node.targets:
+                if isinstance(target, ast.Name) and target.id == '__version__':
+                    return node.value.s
+    raise ValueError('__version__ not found')
+
+
+release = get_version()
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.napoleon',
+    'sphinx.ext.viewcode',
+    'myst_parser',
+    'sphinx_copybutton',
+    'sphinx_tabs.tabs',
+    'notfound.extension',
+    'sphinxcontrib.jquery',
+    'sphinx_design',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+source_suffix = {
+    '.rst': 'restructuredtext',
+    '.md': 'markdown',
+}
+
+language = 'en'
+
+# The master toctree document.
+root_doc = 'index'
+html_context = {
+    'github_version': 'latest',
+}
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'pytorch_sphinx_theme'
+html_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+# yapf: disable
+html_theme_options = {
+    'menu': [
+        {
+            'name': 'GitHub',
+            'url': 'https://github.com/open-compass/VLMEvalKit'
+        },
+    ],
+    # Specify the language of shared menu
+    'menu_lang': 'en',
+    # Disable the default edit on GitHub
+    'default_edit_on_github': False,
+}
+# yapf: enable
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+html_css_files = [
+    'https://cdn.datatables.net/v/bs4/dt-1.12.1/datatables.min.css',
+    'css/readthedocs.css'
+]
+html_js_files = [
+    'https://cdn.datatables.net/v/bs4/dt-1.12.1/datatables.min.js',
+    'js/custom.js'
+]
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'vlmevalkitdoc'
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (root_doc, 'vlmevalkit.tex', 'VLMEvalKit Documentation', author,
+     'manual'),
+]
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [(root_doc, 'vlmevalkit', 'VLMEvalKit Documentation', [author],
+              1)]
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (root_doc, 'vlmevalkit', 'VLMEvalKit Documentation', author,
+     'VLMEvalKit Authors', 'AGI evaluation toolbox and benchmark.',
+     'Miscellaneous'),
+]
+
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+# set priority when building html
+StandaloneHTMLBuilder.supported_image_types = [
+    'image/svg+xml', 'image/gif', 'image/png', 'image/jpeg'
+]
+
+# -- Extension configuration -------------------------------------------------
+# Ignore >>> when copying code
+copybutton_prompt_text = r'>>> |\.\.\. '
+copybutton_prompt_is_regexp = True
+
+# Auto-generated header anchors
+myst_heading_anchors = 3
+# Enable "colon_fence" extension of myst.
+myst_enable_extensions = ['colon_fence', 'dollarmath']
+
+# Configuration for intersphinx
+intersphinx_mapping = {
+    'python': ('https://docs.python.org/3', None),
+    'numpy': ('https://numpy.org/doc/stable', None),
+    'torch': ('https://pytorch.org/docs/stable/', None),
+    'mmengine': ('https://mmengine.readthedocs.io/en/latest/', None),
+    'transformers':
+    ('https://huggingface.co/docs/transformers/main/en/', None),
+}
+napoleon_custom_sections = [
+    # Custom sections for data elements.
+    ('Meta fields', 'params_style'),
+    ('Data fields', 'params_style'),
+]
+
+# Disable docstring inheritance
+autodoc_inherit_docstrings = False
+# Mock some imports during generate API docs.
+autodoc_mock_imports = ['rich', 'attr', 'einops']
+# Disable displaying type annotations, these can be very verbose
+autodoc_typehints = 'none'
+
+# The not found page
+notfound_template = '404.html'
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/docutils.conf b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/docutils.conf
new file mode 100644
index 0000000000000000000000000000000000000000..0c00c84688701117f231fd0c8ec295fb747b7d8f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/docutils.conf
@@ -0,0 +1,2 @@
+[html writers]
+table_style: colwidths-auto
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/index.rst b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/index.rst
new file mode 100644
index 0000000000000000000000000000000000000000..425c7de4de85670f8fd7a64d65fb786a9006f7e1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/en/index.rst
@@ -0,0 +1,41 @@
+Welcome to the VLMEvalKit Tutorial!
+==========================================
+
+VLMEvalKit Getting Started Guide
+-------------------------------
+
+To help users get started quickly, we recommend the following process:
+
+- For users who want to use VLMEvalKit, we recommend reading the "Start Your First Step" section to set up the environment and start a mini-experiment to familiarize yourself with the process.
+
+- If you want to customize more modules, such as adding datasets and models, we provide an "Advanced Tutorial."
+
+We always welcome users' PRs (Pull Requests) and Issues to improve VLMEvalKit!
+
+.. _Start Your First Step:
+.. toctree::
+   :maxdepth: 1
+   :caption: Start Your First Step
+
+   Quickstart.md
+
+.. _Advanced Tutorial:
+.. toctree::
+   :maxdepth: 1
+   :caption: Advanced Tutorial
+
+   Development.md
+   ConfigSystem.md
+
+.. _Other Notes:
+.. toctree::
+   :maxdepth: 1
+   :caption: Other Notes
+
+   Contributors.md
+
+Index and Tables
+==================
+
+* :ref:`genindex`
+* :ref:`search`
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/ja/README_ja.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/ja/README_ja.md
new file mode 100644
index 0000000000000000000000000000000000000000..5bf9564b098bec3748712b150d555ef963c400b9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/ja/README_ja.md
@@ -0,0 +1,117 @@
+<div align="center">
+
+![LOGO](http://opencompass.openxlab.space/utils/MMLB.jpg)
+
+<b>VLMEvalKit: 大規模視覚言語モデルの評価ツールキット</b>
+
+[![][github-contributors-shield]][github-contributors-link] • [![][github-forks-shield]][github-forks-link] • [![][github-stars-shield]][github-stars-link] • [![][github-issues-shield]][github-issues-link] • [![][github-license-shield]][github-license-link]
+
+[English](/README.md) | [简体中文](/docs/zh-CN/README_zh-CN.md) | 日本語
+
+<a href="https://rank.opencompass.org.cn/leaderboard-multimodal">🏆 OpenCompass Learderboard </a> •
+<a href="#-datasets-models-and-evaluation-results">📊Datasets & Models </a> •
+<a href="#%EF%B8%8F-quickstart">🏗️Quickstart </a> •
+<a href="#%EF%B8%8F-development-guide">🛠️Development </a> •
+<a href="#-the-goal-of-vlmevalkit">🎯Goal </a> •
+<a href="#%EF%B8%8F-citation">🖊️Citation </a>
+
+<a href="https://huggingface.co/spaces/opencompass/open_vlm_leaderboard">🤗 HF Leaderboard</a> •
+<a href="https://huggingface.co/datasets/VLMEval/OpenVLMRecords">🤗 Evaluation Records</a> •
+<a href="https://discord.gg/evDT4GZmxN">🔊 Discord Channel</a> •
+<a href="https://www.arxiv.org/abs/2407.11691">📝 Technical Report</a>
+</div>
+
+**VLMEvalKit**（pythonパッケージ名は**vlmeval**）は、**大規模視覚言語モデル（LVLMs）**の**オープンソース評価ツールキット**です。このツールキットは、複数のリポジトリでのデータ準備という重労働なしに、さまざまなベンチマークでLVLMsの**ワンコマンド評価**を可能にします。VLMEvalKitでは、すべてのLVLMsに対して**生成ベースの評価**を採用し、**正確なマッチング**と**LLMベースの回答抽出**の両方で得られた評価結果を提供します。
+
+PS: 日本語の README には最新のアップデートがすべて含まれていない場合があります。英語版をご確認ください。
+
+## 📊 データセット、モデル、および評価結果
+
+**公式のマルチモーダルリーダーボードでのパフォーマンス数値は、ここからダウンロードできます！**
+
+[**OpenVLM Leaderboard**](https://huggingface.co/spaces/opencompass/open_vlm_leaderboard): [すべての詳細な結果をダウンロード](http://opencompass.openxlab.space/assets/OpenVLM.json)。
+
+**Supported Benchmarks** in [**VLMEvalKit Features**](https://aicarrier.feishu.cn/wiki/Qp7wwSzQ9iK1Y6kNUJVcr6zTnPe?table=tblsdEpLieDoCxtb) を確認して、すべてのサポートされているベンチマーク（70以上）を表示してください。
+
+**Supported LMMs** in [**VLMEvalKit Features**](https://aicarrier.feishu.cn/wiki/Qp7wwSzQ9iK1Y6kNUJVcr6zTnPe?table=tblsdEpLieDoCxtb) を確認して、すべてのサポートされている LMMs（200以上）を表示してください。
+
+**Transformersバージョンの推奨事項:**
+
+特定のtransformerバージョンで一部のVLMが実行できない可能性があることに注意してください。各VLMを評価するために、以下の設定を推奨します：
+
+- **`transformers==4.33.0`を使用してください**: `Qwenシリーズ`, `Monkeyシリーズ`, `InternLM-XComposerシリーズ`, `mPLUG-Owl2`, `OpenFlamingo v2`, `IDEFICSシリーズ`, `VisualGLM`, `MMAlaya`, `ShareCaptioner`, `MiniGPT-4シリーズ`, `InstructBLIPシリーズ`, `PandaGPT`, `VXVERSE`, `GLM-4v-9B`.
+- **`transformers==4.37.0`を使用してください**: `LLaVAシリーズ`, `ShareGPT4Vシリーズ`, `TransCore-M`, `LLaVA (XTuner)`, `CogVLMシリーズ`, `EMU2シリーズ`, `Yi-VLシリーズ`, `MiniCPM-[V1/V2]`, `OmniLMM-12B`, `DeepSeek-VLシリーズ`, `InternVLシリーズ`, `Cambrianシリーズ`, `VILA-VLシリーズ`.
+- **`transformers==4.40.0`を使用してください**: `IDEFICS2`, `Bunny-Llama3`, `MiniCPM-Llama3-V2.5`, `360VL-70B`, `Phi-3-Vision`, `WeMM`.
+- **`transformers==4.42.0`を使用してください**: `AKI`.
+- **`transformers==latest`を使用してください**: `LLaVA-Nextシリーズ`, `PaliGemma-3B`, `Chameleon-VLシリーズ`, `Video-LLaVA-7B-HF`, `Ovis1.5シリーズ`, `Mantisシリーズ`, `MiniCPM-V2.6`.
+
+```python
+# デモ
+from vlmeval.config import supported_VLM
+model = supported_VLM['idefics_9b_instruct']()
+# 単一画像のフォワード
+ret = model.generate(['assets/apple.jpg', 'この画像には何がありますか？'])
+print(ret)  # この画像には葉がついた赤いリンゴがあります。
+# 複数画像のフォワード
+ret = model.generate(['assets/apple.jpg', 'assets/apple.jpg', '提供された画像にはリンゴが何個ありますか？'])
+print(ret)  # 提供された画像にはリンゴが2個あります。
+```
+
+## 🏗️ クイックスタート
+
+クイックスタートガイドについては、[クイックスタート](/docs/en/Quickstart.md)を参照してください。
+
+## 🛠️ 開発ガイド
+
+カスタムベンチマーク、VLMsを開発するか、単に**VLMEvalKit**に他のコードを貢献する場合は、[開発ガイド](/docs/en/Development.md)を参照してください。
+
+コミュニティからの共有を奨励し、それに応じたクレジットを共有するために、次回のレポート更新では以下のことを実施します：
+
+- 全ての貢献に対して感謝の意を示します
+- 新しいモデル、評価セット、または主要な機能への3つ以上の主要な貢献を持つ貢献者は、テクニカルレポートの著者リストに加わることができます。適格な貢献者は、issueを作成するか、または[VLM評価キット ディスコードチャンネル](https://discord.com/invite/evDT4GZmxN)で kennyutc にDMを送ることができます。私たちはそれに応じてフォローアップします。
+
+## 🎯 VLMEvalKitの目標
+
+**このコードベースは以下を目的として設計されています：**
+
+1. 研究者や開発者が既存のLVLMsを評価し、評価結果を**簡単に再現できるようにする**ための**使いやすい**、**オープンソースの評価ツールキット**を提供します。
+2. VLMの開発者が自分のモデルを簡単に評価できるようにします。複数のサポートされているベンチマークでVLMを評価するには、単一の`generate_inner()`関数を**実装するだけで**、他のすべてのワークロード（データのダウンロード、データの前処理、予測の推論、メトリックの計算）はコードベースによって処理されます。
+
+**このコードベースは以下を目的として設計されていません：**
+
+1. すべての**第三者ベンチマーク**の元の論文で報告された正確な精度数値を再現すること。その理由は2つあります：
+   1. VLMEvalKitは、すべてのVLMに対して**生成ベースの評価**を使用します（オプションで**LLMベースの回答抽出**を使用）。一方、一部のベンチマークは異なるアプローチを使用する場合があります（SEEDBenchはPPLベースの評価を使用します）。これらのベンチマークについては、対応する結果で両方のスコアを比較します。開発者には、コードベースで他の評価パラダイムをサポートすることをお勧めします。
+   2. デフォルトでは、すべてのVLMに対して同じプロンプトテンプレートを使用してベンチマークを評価します。一方、**一部のVLMには特定のプロンプトテンプレートがある**場合があります（現時点ではコードベースでカバーされていない場合があります）。VLMの開発者には、現在カバーされていない場合でも、VLMEvalKitで独自のプロンプトテンプレートを実装することをお勧めします。これにより、再現性が向上します。
+
+## 🖊️ 引用
+
+この作業が役立つ場合は、このリポジトリに**スター🌟**を付けてください。サポートありがとうございます！
+
+[![Stargazers repo roster for @open-compass/VLMEvalKit](https://reporoster.com/stars/open-compass/VLMEvalKit)](https://github.com/open-compass/VLMEvalKit/stargazers)
+
+研究でVLMEvalKitを使用する場合、または公開されたオープンソースの評価結果を参照する場合は、以下のBibTeXエントリと、使用した特定のVLM/ベンチマークに対応するBibTexエントリを使用してください。
+
+```bib
+@misc{duan2024vlmevalkit,
+      title={VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models},
+      author={Haodong Duan and Junming Yang and Yuxuan Qiao and Xinyu Fang and Lin Chen and Yuan Liu and Xiaoyi Dong and Yuhang Zang and Pan Zhang and Jiaqi Wang and Dahua Lin and Kai Chen},
+      year={2024},
+      eprint={2407.11691},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2407.11691},
+}
+```
+
+<p align="right"><a href="#top">🔝Top に戻る</a></p>
+
+[github-contributors-link]: https://github.com/open-compass/VLMEvalKit/graphs/contributors
+[github-contributors-shield]: https://img.shields.io/github/contributors/open-compass/VLMEvalKit?color=c4f042&labelColor=black&style=flat-square
+[github-forks-link]: https://github.com/open-compass/VLMEvalKit/network/members
+[github-forks-shield]: https://img.shields.io/github/forks/open-compass/VLMEvalKit?color=8ae8ff&labelColor=black&style=flat-square
+[github-issues-link]: https://github.com/open-compass/VLMEvalKit/issues
+[github-issues-shield]: https://img.shields.io/github/issues/open-compass/VLMEvalKit?color=ff80eb&labelColor=black&style=flat-square
+[github-license-link]: https://github.com/open-compass/VLMEvalKit/blob/main/LICENSE
+[github-license-shield]: https://img.shields.io/github/license/open-compass/VLMEvalKit?color=white&labelColor=black&style=flat-square
+[github-stars-link]: https://github.com/open-compass/VLMEvalKit/stargazers
+[github-stars-shield]: https://img.shields.io/github/stars/open-compass/VLMEvalKit?color=ffcb47&labelColor=black&style=flat-square
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/.readthedocs.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/.readthedocs.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..b7e46fe34090d74f165034fb5bed93f2f112f42b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/.readthedocs.yaml
@@ -0,0 +1,17 @@
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.8"
+
+formats:
+    - epub
+
+sphinx:
+  configuration: docs/zh-CN/conf.py
+
+python:
+  install:
+    - requirements: requirements/docs.txt
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/ConfigSystem.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/ConfigSystem.md
new file mode 100644
index 0000000000000000000000000000000000000000..14e8d49564ec5956bb6b31b3bab161be2cee402b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/ConfigSystem.md
@@ -0,0 +1,69 @@
+
+# 配置系统
+
+默认情况下，VLMEvalKit通过在`run.py`脚本中使用`--model`和`--data`参数设置模型名称（在`/vlmeval/config.py`中定义）和数据集名称（在`vlmeval/dataset/__init__.py` 或 `vlmeval/dataset/video_dataset_config.py` 中定义）来启动评估。这种方法在大多数情况下简单且高效，但当用户希望使用不同设置评估多个模型/数据集时，可能不够灵活。
+
+为了解决这个问题，VLMEvalKit提供了一个更灵活的配置系统。用户可以在json文件中指定模型和数据集设置，并通过`--config`参数将配置文件的路径传递给`run.py`脚本。以下是一个示例配置json：
+
+```json
+{
+    "model": {
+        "GPT4o_20240806_T00_HIGH": {
+            "class": "GPT4V",
+            "model": "gpt-4o-2024-08-06",
+            "temperature": 0,
+            "img_detail": "high"
+        },
+        "GPT4o_20240806_T10_Low": {
+            "class": "GPT4V",
+            "model": "gpt-4o-2024-08-06",
+            "temperature": 1.0,
+            "img_detail": "low"
+        },
+        "GPT4o_20241120": {}
+    },
+    "data": {
+        "MME-RealWorld-Lite": {
+            "class": "MMERealWorld",
+            "dataset": "MME-RealWorld-Lite"
+        },
+        "MMBench_DEV_EN_V11": {
+            "class": "ImageMCQDataset",
+            "dataset": "MMBench_DEV_EN_V11"
+        },
+        "MMBench_Video_8frame_nopack":{},
+        "Video-MME_16frame_subs": {
+            "class": "VideoMME",
+            "dataset": "Video-MME",
+            "nframe": 16,
+            "use_subtitle": true
+        }
+    }
+}
+```
+
+配置json的解释：
+
+1. 现在我们支持两个字段：`model`和`data`，每个字段都是一个字典。字典的键是模型/数据集的名称（由用户设置），值是模型/数据集的设置。
+2. 对于`model`中的项目，值是一个包含以下键的字典：
+    - `class`：模型的类名，应该是`vlmeval/vlm/__init__.py`（开源模型）或`vlmeval/api/__init__.py`（API模型）中定义的类名。
+    - 其他kwargs：其他kwargs是模型特定的参数，请参考模型类的定义以获取详细用法。例如，`model`、`temperature`、`img_detail`是`GPT4V`类的参数。值得注意的是，大多数模型类都需要`model`参数。
+    - Tip：在位于`vlmeval/config.py`的变量`supported_VLM`中的已经被定义的模型可以作为`model`的键，而不需要填对应的值即可启动。例如，`GPT4o_20240806_T00_HIGH: {}`是等价于`GPT4o_20240806_T00_HIGH: {'class': 'GPT4V', 'model': 'gpt-4o-2024-08-06', 'temperature': 0, 'img_size': -1, 'img_detail': 'high', 'retry': 10, 'verbose': False}`。
+3. 对于字典`data`，我们建议用户使用官方数据集名称作为键（或键的一部分），因为我们经常根据数据集名称确定后处理/判断设置。对于`data`中的项目，值是一个包含以下键的字典：
+    - `class`：数据集的类名，应该是`vlmeval/dataset/__init__.py`中定义的类名。
+    - 其他kwargs：其他kwargs是数据集特定的参数，请参考数据集类的定义以获取详细用法。通常，大多数数据集类都需要`dataset`参数。大多数视频数据集类都需要 `nframe` 或 `fps` 参数。
+    - Tip：在位于`vlmeval/dataset/video_dataset_config.py`的变量`supported_video_dataset`中的已经被定义的数据集可以作为`data`的键，而不需要填对应的值即可启动。例如，`MMBench_Video_8frame_nopack: {}`是等价于`MMBench_Video_8frame_nopack: {'class': 'MMBenchVideo', 'dataset': 'MMBench-Video', 'nframe': 8, 'pack': False}`。
+
+将示例配置json保存为`config.json`，您可以通过以下命令启动评估：
+
+```bash
+python run.py --config config.json
+```
+
+这将在工作目录`$WORK_DIR`下生成以下输出文件（格式为`{$WORK_DIR}/{$MODEL_NAME}/{$MODEL_NAME}_{$DATASET_NAME}_*`）：
+
+- `$WORK_DIR/GPT4o_20240806_T00_HIGH/GPT4o_20240806_T00_HIGH_MME-RealWorld-Lite*`
+- `$WORK_DIR/GPT4o_20240806_T10_Low/GPT4o_20240806_T10_Low_MME-RealWorld-Lite*`
+- `$WORK_DIR/GPT4o_20240806_T00_HIGH/GPT4o_20240806_T00_HIGH_MMBench_DEV_EN_V11*`
+- `$WORK_DIR/GPT4o_20240806_T10_Low/GPT4o_20240806_T10_Low_MMBench_DEV_EN_V11*`
+......
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Development.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Development.md
new file mode 100644
index 0000000000000000000000000000000000000000..69db06498d30354aac0720063589069587a89301
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Development.md
@@ -0,0 +1,139 @@
+# 🛠️ 如何在 VLMEvalKit 中实现一个新的 Benchmark 或多模态模型（VLM）
+
+## 实现一个新的 benchmark
+
+示例 PR: **添加 Math-Vision Benchmark** ([#292](https://github.com/open-compass/VLMEvalKit/pull/292/files))
+
+目前在 VLMEvalKit 中，benchmark 以数据集类的形式呈现，当你新增一个 benchmark 时，你可以选择复用现有的数据集类 (如单选题 benchmark 可复用 `ImageMCQDataset`)，或是实现新的数据集类。你的数据集类必须支持以下两种方法 (复用父类或自行实现):
+
+- `build_prompt(self, line)`: 方法输入 `line` 类型为 int (对应数据 index) 或 `pd.Series` (对应数据原始 record)。方法输出一条 `multi-modal message` 作为多模态模型输入，`multi-modal message` 是一个图文交错的列表，如以下格式 (一图一文): `[dict(type='image', value=IMAGE_PTH), dict(type='text', value=prompt)]`。
+- `evaluate(self, eval_file, **judge_kwargs)`: 方法输入 `eval_file` 为多模态模型的预测结果 (多以 `.xlsx` 格式存在)，如 benchmark evaluation 需要大语言模型 (一般为 GPT) 辅助，则 `judge_kwargs` 传入大语言模型的参数。方法输出 benchmark 的评测结果，以 `dict` 或 `pd.DataFrame` 的形式。
+
+以下，我们简述新增数据集的通常步骤：
+
+### 1. TSV 数据文件准备 (图文评测集)
+
+目前，我们将每一个 benchmark 数据集设置为一个单独的 TSV 文件。在推理过程中，数据文件将从数据集定义的 `DATASET_URL` 链接地址自动下载到 `$LMUData` 中（如果没有明确设置的话，默认路径是 `$HOME/LMUData`）。你可以将准备好的 TSV 文件上传到一个可下载的地址（如：huggingface），或发送给我们 <opencompass@pjlab.org.cn>，我们将帮助上传数据集到服务器中。此外，你也可以在环境变量中自定义设置下载路径 `LMUData=/path/to/your/data`。
+
+TSV 文件中的内容组成为：
+
+| 数据集名称 \ 字段  | index | image | image_path | question | hint | multi-choice<br>options | answer | category | l2-category | split |
+| ---------------------- | ----- | ----- | ---------- | -------- | ---- | ----------------------- | ------ | -------- | ----------- | ----- |
+| MMBench_DEV_[CN/EN]    | ✅     | ✅     |            | ✅        | ✅    | ✅                       | ✅      | ✅        | ✅           | ✅     |
+| MMBench_TEST_[CN/EN]   | ✅     | ✅     |            | ✅        | ✅    | ✅                       |        | ✅        | ✅           | ✅     |
+| CCBench                | ✅     | ✅     |            | ✅        |      | ✅                       | ✅      | ✅        |             |       |
+| SEEDBench_IMG          | ✅     | ✅     |            | ✅        |      | ✅                       | ✅      | ✅        |             |       |
+| MME                    | ✅     | ✅     |            | ✅        |      |                         | ✅      | ✅        |             |       |
+| MMVet                  | ✅     | ✅     |            | ✅        |      |                         | ✅      | ✅        |             |       |
+| MMMU_DEV_VAL           | ✅     | ✅     | ✅          | ✅        |      | ✅                       | ✅      | ✅        | ✅           | ✅     |
+| COCO_VAL               | ✅     | ✅     |            |          |      |                         | ✅      |          |             |       |
+| OCRVQA_[TEST/TESTCORE] | ✅     | ✅     |            | ✅        |      |                         | ✅      |          |             |       |
+| TextVQA_VAL            | ✅     | ✅     |            | ✅        |      |                         | ✅      |          |             |       |
+| VCR_[EN/ZH]\_[EASY/HARD]_[ALL/500/100]            | ✅     | ✅     |            | ✅        |      |                         | ✅      |          |             |       |
+
+<div align="center"><b>表 1. 支持的数据集的 TSV 字段。</b></div>
+
+**TSV 中必须字段的介绍：**
+
+- **index:** 一个整数，`tsv` 中每一行的唯一标识
+- **image:** 图片的 base64 编码，你可以使用 `vlmeval/smp/vlm.py` 中实现的API进行编码和解码：
+    - 编码：`encode_image_to_base64`（对于PIL Image）/ `encode_image_file_to_base64`（对于图片文件路径）
+    - 解码：`decode_base64_to_image`（对于PIL Image）/ `decode_base64_to_image_file`（对于图片文件路径）
+- **question:** 针对图像所提取出的问题，类型为字符串
+- **answer:** 问题的答案，类型为字符串，Test 集可缺失这一字段
+
+### 2. 自定义数据集的 prompt 构建
+
+`ImageBaseDataset` 定义了默认的 prompt 格式。如果需要针对数据集添加 prompt，或给模型输入 `Interleave` 的数据格式，可以通过 `build_prompt(line)` 函数实现。该函数输入为，每次给定 TSV 文件中的一行，包含 index, image, question 等内容作为 line。该函数将返回一个多模态消息 `msg` 的字典列表 `[dict(type='image', value=IMAGE_PTH), dict(type='text', value=prompt)]`，包括图片路径和将被输入到 VLMs 的文本 prompt。对于 interleave 类型输入，可以直接将图片路径的字典放置到 image token 位置。
+
+### 3. 自定义数据集的指标实现
+
+增加对 benchmark 的评测需要自定义一个该数据集的 class 对象，从而实现数据集的指标计算。图文多模态数据集均继承自 `vlmeval/dataset/image_base.py` 中的 `ImageBaseDataset` 对象。其中 `TYPE` 定义了数据集的类型；`DATASET_URL` 为数据集的下载地址；`DATASET_MD5` 为数据集文件的 md5 一致性编码检查。
+
+在 class 中**需要实现** `evaluate(eval_file, **judge_kwargs)` 类函数，对自定义的数据集结果进行指标计算和结果输出。函数输入 `eval_file` 为模型预测结果 `{model_name}_{dataset}.xlsx` 的路径。可以通过 `load(eval_file)` 文件将其读取为 panda.DataFrames 类型，其中包含 index, question, answer, category, prediction 等字段。`judge_kwargs` 参数将传递一个评测相关的字典，如：judge 模型的名称，api 请求线程数等。**函数的返回值**为评估完成的准确度等指标，其格式为由 list 组成的字典，并组织成 panda.DataFrames 类型。
+
+## 实现一个新的模型
+
+示例 PR: **支持 LLaVA-Next-Interleave** ([#294](https://github.com/open-compass/VLMEvalKit/pull/294))
+
+**1. 支持 `generate_inner` API (必须)**
+
+现有所有的模型都在 `vlmeval/vlm` 中实现。对于一个最基本的模型，你的模型类**应该实现方法** `generate_inner(msgs, dataset=None)`。这个函数将向 VLM 输入一个多模态数据，并返回 VLM 的预测（一个字符串）。可选参数 `dataset` 可以用作模型在不同推理策略之间切换的标志。
+
+其中多模态消息 `msgs` 是一个字典列表，每个字典有两个键：类型和值：
+- `type`：我们目前支持两种类型，选项是 ["image", "text"]。
+- `value`：当类型为 `text` 时，值是文本消息（一个字符串）；当类型为 `image` 时，值可以是图像文件的本地路径，或者是图像的URL。
+
+> 目前，一个多模态消息可能包含任意交错的图像和文本。如果你的模型不支持这一点，我们推荐的做法是取第一张图像和连接的文本消息作为模型的输入。你可以在模型的 class 中设置 `INTERLEAVE = False` 并调用 `self.message_to_promptimg(message, dataset=dataset)` 函数来获取你的 prompt 和第一张图片的地址。
+
+一些多模态消息的例子:
+
+```python
+IMAGE_PTH = 'assets/apple.jpg'
+IMAGE_URL = 'https://raw.githubusercontent.com/open-compass/VLMEvalKit/main/assets/apple.jpg'
+msg1 = [
+    dict(type='image', value=IMAGE_PTH),
+    dict(type='text', value='What is in this image?')
+]
+msg2 = [
+    dict(type='image', value=IMAGE_URL),
+    dict(type='image', value=IMAGE_URL),
+    dict(type='text', value='How many apples are there in these images?')
+]
+response = model.generate(msg1)
+```
+
+为了方便起见，我们还支持接受字符串列表作为输入。在这种情况下，我们将检查一个字符串是图像路径还是图像 URL，并自动将其转换为 `list[dict]` 格式：
+
+```python
+IMAGE_PTH = 'assets/apple.jpg'
+IMAGE_URL = 'https://raw.githubusercontent.com/open-compass/VLMEvalKit/main/assets/apple.jpg'
+msg1 = [IMAGE_PTH, 'What is in this image?']
+msg2 = [IMAGE_URL, IMAGE_URL,  'How many apples are there in these images?']
+response = model.generate(msg1)
+```
+
+**2. 支持自定义提示词构建 (可选)**
+
+此外，你的模型可以通过实现两个可选方法来支持自定义提示构建：`use_custom_prompt(dataset)` 和 `build_prompt(line, dataset=None)`。
+
+- `use_custom_prompt(dataset)` 将返回一个布尔值，指示模型是否应使用自定义提示构建策略。
+- 如果`use_custom_prompt(dataset)`返回 True，`build_prompt(line, dataset)` 应该为相应的数据集返回一个自定义构建的多模态消息，line 数据是一个包含数据样本所需信息的字典。如果`use_custom_prompt(dataset)` 返回False，则将使用默认的 prompt 构建策略。
+
+**3. 支持多轮对话 (可选)**
+
+你可以通过支持 `chat_inner(message, dataset)` API 为你的模型新增多轮对话功能并兼容多轮对话评测。这个 API 输出一个字符串型回复，`message` 包含一个聊天记录的列表，格式如下：
+
+```python
+# Assume msg1, msg2, msg3, ... are multi-modal messages following the previously described format
+# `chat_inner` take the following chat history list as input:
+message = [
+    dict(role='user', content=msg1),
+    dict(role='assistant', content=msg2),
+    dict(role='user', content=msg3),
+    dict(role='assistant', content=msg4),
+	......
+    dict(role='user', content=msgn),
+]
+# `message` should contain an odd number of chat utterances, the role of utterances should be interleaved "user" and "assistant", with the role of the last utterance to be "user".
+# The chat function will call `chat_inner`
+response = model.chat(message)
+```
+
+### 示例 PRs：
+
+- 不支持交错的图像和文本，且不使用自定义提示的VLM：[[模型] 支持 glm-4v-9b](https://github.com/open-compass/VLMEvalKit/pull/221)
+- 支持交错的图像和文本及自定义提示的VLM：[添加 MiniCPM-Llama3-V-2.5](https://github.com/open-compass/VLMEvalKit/pull/205)
+- VLM API：[特征添加 glmv](https://github.com/open-compass/VLMEvalKit/pull/201)
+
+## 为 VLMEvalKit 贡献代码
+
+如果你想为 **VLMEvalKit** 贡献代码，请在提交PR之前进行预提交检查。这有助于保持代码整洁。
+
+```bash
+# 在VLMEvalKit的目录下，安装预提交 hook:
+pip install pre-commit
+pre-commit install
+pre-commit run --all-files
+# 然后提交你的代码。
+```
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/EvalByLMDeploy.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/EvalByLMDeploy.md
new file mode 100644
index 0000000000000000000000000000000000000000..cdb46c70f0cc9d2620e0a98471f0c9b354472518
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/EvalByLMDeploy.md
@@ -0,0 +1,28 @@
+# 使用 LMDeploy 加速评测推理
+
+VLMEvalKit 支持测试由 LMDeploy 部署的 VLM 模型，下面以 InternVL2-8B 为例，展示如何测试模型
+
+## 第0步 安装 LMDeploy
+
+```bash
+pip install lmdeploy
+```
+
+其他安装方式可以参考 LMDeploy 的[文档](https://github.com/InternLM/lmdeploy)
+
+## 第1步 启动推理服务
+
+```bash
+lmdeploy serve api_server OpenGVLab/InternVL2-8B --model-name InternVL2-8B
+```
+> [!IMPORTANT]
+> 因为 VLMEvalKit 中的模型对于不同数据集在构建 prompt 时可能有自定义行为，如 InternVL2 对于 HallusionBench 的处理，所以，server 端在启动的时候需要指定 `--model-name`，这样在使用 LMDEploy api 时可以根据名字选择合适的 prompt 构建策略。
+>
+> 如果指定了 `--server-port`，需要设置对应的环境变量 `LMDEPLOY_API_BASE`
+
+
+## 第2步 评测
+
+```bash
+python run.py --data MMStar --model InternVL2-8B --verbose --api-nproc 64
+```
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Makefile b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Makefile
new file mode 100644
index 0000000000000000000000000000000000000000..d4bb2cbb9eddb1bb1b4f366623044af8e4830919
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Quickstart.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Quickstart.md
new file mode 100644
index 0000000000000000000000000000000000000000..1245235b071adad507e871e1cfb1ca1145ae7e4b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/Quickstart.md
@@ -0,0 +1,231 @@
+# 快速开始
+
+在运行评测脚本之前，你需要先**配置** VLMs，并正确设置模型路径。然后你可以使用脚本 `run.py` 进行多个VLMs和基准测试的推理和评估。
+
+## 第0步 安装和设置必要的密钥
+
+**安装**
+
+```bash
+git clone https://github.com/open-compass/VLMEvalKit.git
+cd VLMEvalKit
+pip install -e .
+```
+
+**设置密钥**
+
+要使用 API 模型（如 GPT-4v, Gemini-Pro-V 等）进行推理，或使用 LLM API 作为**评判者或选择提取器**，你需要首先设置 API 密钥。如果你设置了密钥，VLMEvalKit 将使用一个评判 LLM 从输出中提取答案，否则它将使用**精确匹配模式**（在输出字符串中查找 "Yes", "No", "A", "B", "C"...）。**精确匹配模式只能应用于是或否任务和多项选择任务。**
+
+- 你可以将所需的密钥放在 `$VLMEvalKit/.env` 中，或直接将它们设置为环境变量。如果你选择创建 `.env` 文件，其内容将如下所示：
+
+  ```bash
+  # .env 文件，将其放置在 $VLMEvalKit 下
+  # 专有 VLMs 的 API 密钥
+  # QwenVL APIs
+  DASHSCOPE_API_KEY=
+  # Gemini w. Google Cloud Backends
+  GOOGLE_API_KEY=
+  # OpenAI API
+  OPENAI_API_KEY=
+  OPENAI_API_BASE=
+  # StepAI API
+  STEPAI_API_KEY=
+  # REKA API
+  REKA_API_KEY=
+  # GLMV API
+  GLMV_API_KEY=
+  # CongRong API
+  CW_API_BASE=
+  CW_API_KEY=
+  # SenseNova API
+  SENSENOVA_API_KEY=
+  # Hunyuan-Vision API
+  HUNYUAN_SECRET_KEY=
+  HUNYUAN_SECRET_ID=
+  # LMDeploy API
+  LMDEPLOY_API_BASE=
+  # MiniMax API
+  MINIMAX_API_KEY=
+  # 你可以设置一个评估时代理，评估阶段产生的 API 调用将通过这个代理进行
+  EVAL_PROXY=
+  ```
+
+- 如果需要使用 API 在对应键值空白处填写上你的密钥。这些 API 密钥将在进行推理和评估时自动加载。
+## 第1步 配置
+
+**VLM 配置**：所有 VLMs 都在 `vlmeval/config.py` 中配置。对于某些 VLMs（如 MiniGPT-4、LLaVA-v1-7B），需要额外的配置（在配置文件中配置代码 / 模型权重根目录）。在评估时，你应该使用 `vlmeval/config.py` 中 `supported_VLM` 指定的模型名称来选择 VLM。确保在开始评估之前，你可以成功使用 VLM 进行推理，使用以下命令 `vlmutil check {MODEL_NAME}`。
+
+注：对于Qwen-VL系列模型（Qwen-VL, Qwen2-VL, Qwen2.5-VL），vlmeval/config.py 中所指定的像素数量上下界如下：
+
+```
+min_pixels=1280 * 28 * 28,
+max_pixels=16384 * 28 * 28,
+```
+其中，1280为Qwen官方为平衡性能、计算资源与内存的推荐最大值，而16384为模型输入的理论最大值。这种设定对于部分需要高分辨率的视觉任务（如文档理解）有着积极的作用。但考虑这一设定并没有实际的依据，如果需要与官方的设定对齐，可以去掉这两个数值，或是设置为以下来自Qwen官方demo的数值：
+
+```
+min_pixels=256 * 28 * 28,
+max_pixels=1280 * 28 * 28,
+```
+
+## 第2步 评测
+
+**新功能!!!** 我们集成了一个新的配置系统，以实现更灵活的评估设置。查看[文档](/docs/zh-CN/ConfigSystem.md)或运行`python run.py --help`了解更多详情 🔥🔥🔥
+
+我们使用 `run.py` 进行评估。你可以使用 `$VLMEvalKit/run.py` 或创建脚本的软链接运行（以便在任何地方使用该脚本）：
+
+**参数**
+
+- `--data (list[str])`: 设置在 VLMEvalKit 中支持的数据集名称（可以在代码库首页的 README 中找到支持的数据集列表）
+- `--model (list[str])`: 设置在 VLMEvalKit 中支持的 VLM 名称（在 `vlmeval/config.py` 中的 `supported_VLM` 中定义）
+- `--mode (str, 默认值为 'all', 可选值为 ['all', 'infer'])`：当 mode 设置为 "all" 时，将执行推理和评估；当设置为 "infer" 时，只执行推理
+- `--api-nproc (int, 默认值为 4)`: 调用 API 的线程数
+- `--work-dir (str, default to '.')`: 存放测试结果的目录
+
+**用于评测图像多模态评测集的命令**
+
+你可以使用 `python` 或 `torchrun` 来运行脚本:
+
+```bash
+# 使用 `python` 运行时，只实例化一个 VLM，并且它可能使用多个 GPU。
+# 这推荐用于评估参数量非常大的 VLMs（如 IDEFICS-80B-Instruct）。
+
+# 在 MMBench_DEV_EN、MME 和 SEEDBench_IMG 上使用 IDEFICS-80B-Instruct 进行推理和评估
+python run.py --data MMBench_DEV_EN MME SEEDBench_IMG --model idefics_80b_instruct --verbose
+# 在 MMBench_DEV_EN、MME 和 SEEDBench_IMG 上使用 IDEFICS-80B-Instruct 仅进行推理
+python run.py --data MMBench_DEV_EN MME SEEDBench_IMG --model idefics_80b_instruct --verbose --mode infer
+
+# 使用 `torchrun` 运行时，每个 GPU 上实例化一个 VLM 实例。这可以加快推理速度。
+# 但是，这仅适用于消耗少量 GPU 内存的 VLMs。
+
+# 在 MMBench_DEV_EN、MME 和 SEEDBench_IMG 上使用 IDEFICS-9B-Instruct、Qwen-VL-Chat、mPLUG-Owl2。在具有 8 个 GPU 的节点上进行推理和评估。
+torchrun --nproc-per-node=8 run.py --data MMBench_DEV_EN MME SEEDBench_IMG --model idefics_80b_instruct qwen_chat mPLUG-Owl2 --verbose
+# 在 MME 上使用 Qwen-VL-Chat。在具有 2 个 GPU 的节点上进行推理和评估。
+torchrun --nproc-per-node=2 run.py --data MME --model qwen_chat --verbose
+```
+
+**用于评测视频多模态评测集的命令**
+
+```bash
+# 使用 `python` 运行时，只实例化一个 VLM，并且它可能使用多个 GPU。
+# 这推荐用于评估参数量非常大的 VLMs（如 IDEFICS-80B-Instruct）。
+
+# 在 MMBench-Video 上评测 IDEFCIS2-8B, 视频采样 8 帧作为输入，不采用 pack 模式评测. MMBench_Video_8frame_nopack 是一个定义在 `vlmeval/dataset/video_dataset_config.py` 的数据集设定.
+torchrun --nproc-per-node=8 run.py --data MMBench_Video_8frame_nopack --model idefics2_8
+# 在 MMBench-Video 上评测 GPT-4o (API 模型), 视频采样每秒一帧作为输入，采用 pack 模式评测
+python run.py --data MMBench_Video_1fps_pack --model GPT4o
+```
+
+评估结果将作为日志打印出来。此外，**结果文件**也会在目录 `$YOUR_WORKING_DIRECTORY/{model_name}` 中生成。以 `.csv` 结尾的文件包含评估的指标。
+### 常见问题
+#### 构建输入prompt：`build_prompt()`函数
+如果您在评测某个benchmark时，发现模型输出的结果与预期不符，可能是因为您使用的模型没有正确构建输入prompt。
+
+在VLMEvalkit中，每个`dataset`类都包含一个名为`build_prompt()`的函数，用于构建输入问题的格式。不同的benchmark可以选择自定义`build_prompt()`函数，也可以使用默认的实现。
+
+例如，在处理默认的[多选题/Multi-Choice QA]([vlmeval/dataset/image_mcq.py](https://github.com/open-compass/VLMEvalKit/blob/43af13e052de6805a8b08cd04aed5e0d74f82ff5/vlmeval/dataset/image_mcq.py#L164))时，`ImageMCQDataset.build_prompt()`类会将`hint`、`question`、`options`等元素（若数据集中包含）组合成一个完整的问题格式，如下所示：
+```
+HINT
+QUESTION
+Options:
+A. Option A
+B. Option B
+···
+Please select the correct answer from the options above.
+```
+
+此外，由于不同模型对评测的需求可能有所不同，VLMEvalkit也支持在模型层面自定义对不同benchmark构建prompt的方法，即`model.build_prompt()`，具体示例可以参考[InternVL](https://github.com/open-compass/VLMEvalKit/blob/43af13e052de6805a8b08cd04aed5e0d74f82ff5/vlmeval/vlm/internvl_chat.py#L324)。
+
+**注意：当同时定义了`model.build_prompt()`以及`dataset.build_prompt()`时，`model.build_prompt()`将优先于`dataset.build_prompt()`，即前者会覆盖后者。**
+
+由于部分模型（如Qwen2VL，InternVL等）对于不同类型的benchmark定义了广泛的prompt构建方法，为了更灵活地适应不同的benchmark，VLMEvalkit支持在模型中自定义`model.use_custom_prompt()`函数。通过添加或者修改`use_custom_prompt()`函数，您可以决定对于哪些benchmark使用模型自定义的`use_custom_prompt()`方法，示例如下：
+```
+def use_custom_prompt(self, dataset: str) -> bool:
+    from vlmeval.dataset import DATASET_TYPE, DATASET_MODALITY
+    dataset_type = DATASET_TYPE(dataset, default=None)
+    if not self._use_custom_prompt:
+        return False
+    if listinstr(['MMVet'], dataset):
+        return True
+    if dataset_type == 'MCQ':
+        return True
+    if DATASET_MODALITY(dataset) == 'VIDEO':
+        return False
+    return False
+```
+仅当`use_custom_prompt()`函数返回`True`时，VLMEvalkit才会对当前benchmark调用模型的`build_prompt()`函数。
+通过这种方式，您可以根据具体需求灵活地控制哪些benchmark使用模型自定义的prompt构建逻辑，从而更好地适配不同模型和任务的需求。
+
+#### 模型切分
+
+目前 VLMEvalKit 的启动方式自动支持同机上进程间 GPU 资源的划分与模型切分。该功能在推理后端为 `lmdeploy` 或 `transformers` 时被支持，具体行为如下：
+
+- 基于 `python` 命令启动时，模型默认分配到所有可用的 GPU 上，如想指定使用哪些 GPU，可以使用 `CUDA_VISIBLE_DEVICES` 环境变量。
+- 基于 `torchrun` 命令启动时，每个模型实例会被分配到 `N_GPU // N_PROC` 个 GPU 上，`N_PROC` 为 torchrun 命令中的 `--nproc-per-node` 参数所指定的进程数。`N_GPU` 的取值为：
+    - 如 `CUDA_VISIBLE_DEVICES` 环境变量未设置，`N_GPU` 为全部可用 GPU 数量。
+    - 如 `CUDA_VISIBLE_DEVICES` 环境变量被设置，`N_GPU` 为 `CUDA_VISIBLE_DEVICES` 环境变量所指定的 GPU 数量，并且，仅有指定的 GPU 会被利用。
+
+下面提供了，在一台配备 8 块 GPU 的机器上运行评测任务的具体示例：
+```bash
+# <!-- 起两个模型实例数据并行，每个实例用 4 GPU -->
+torchrun --nproc-per-node=2 run.py --data MMBench_DEV_EN --model InternVL3-78B
+# <!-- 起一个模型实例，每个实例用 8 GPU -->
+python run.py --data MMBench_DEV_EN --model InternVL3-78B
+# <!-- 起三个模型实例，每个实例用 2 GPU，0 号、7 号 GPU 未被使用 -->
+CUDA_VISIBLE_DEVICES=1,2,3,4,5,6 torchrun --nproc-per-node=3 run.py --data MMBench_DEV_EN --model InternVL3-38B
+```
+
+注：此方式不支持 `vllm` 后端，基于 `vllm` 后端起评测任务时，请用 `python` 命令启动，默认调用所有可见的 GPU。
+
+#### 性能差距
+在不同的运行环境中，模型的性能表现可能会有所差异。因此，在评估过程中，您可能会发现自己的评测结果与VLMEvalKit官方榜单上的结果存在差距。这种差异可能与`transformers`, `cuda`, `torch`等版本的变化有关。
+
+此外，对于异常的表现，我们建议您优先查看运行完成后的本地生成记录`{model}_{dataset}.xlsx`或者评估记录`{model}_{dataset}_{judge_model}.xlsx`，这可能会帮助您更好地理解评估结果并发现问题。
+
+
+
+### 部署本地语言模型作为评判 / 选择提取器
+上述默认设置使用 OpenAI 的 GPT 作为评判 LLM。你也可以使用 [LMDeploy](https://github.com/InternLM/lmdeploy) 部署本地评判 LLM。
+
+首先进行安装:
+```
+pip install lmdeploy openai
+```
+
+然后可以通过一行代码部署本地评判 LLM。LMDeploy 将自动从 Huggingface 下载模型。假设我们使用 internlm2-chat-1_8b 作为评判，端口为 23333，密钥为 sk-123456（密钥必须以 "sk-" 开头，后跟任意数字）：
+```
+lmdeploy serve api_server internlm/internlm2-chat-1_8b --server-port 23333
+```
+
+使用以下 Python 代码获取由 LMDeploy 注册的模型名称：
+```
+from openai import OpenAI
+client = OpenAI(
+    api_key='sk-123456',
+    base_url="http://0.0.0.0:23333/v1"
+)
+model_name = client.models.list().data[0].id
+```
+
+配置对应环境变量，以告诉 VLMEvalKit 如何使用本地评判 LLM。正如上面提到的，也可以在  `$VLMEvalKit/.env` 文件中设置：
+```
+OPENAI_API_KEY=sk-123456
+OPENAI_API_BASE=http://0.0.0.0:23333/v1/chat/completions
+LOCAL_LLM=<model_name you get>
+```
+
+最后，你可以运行第2步中的命令，使用本地评判 LLM 来评估你的 VLM。
+
+**请注意：**
+
+- 如果你希望将评判 LLM 部署在单独的一个 GPU 上，并且由于 GPU 内存有限而希望在其他 GPU 上评估你的 VLM，可以使用 `CUDA_VISIBLE_DEVICES=x` 这样的方法，例如：
+```
+CUDA_VISIBLE_DEVICES=0 lmdeploy serve api_server internlm/internlm2-chat-1_8b --server-port 23333
+CUDA_VISIBLE_DEVICES=1,2,3 torchrun --nproc-per-node=3 run.py --data HallusionBench  --model qwen_chat --verbose
+```
+- 如果本地评判 LLM 在遵循指令方面不够好，评估过程可能会失败。请通过 issues 报告此类失败情况。
+- 可以以不同的方式部署评判 LLM，例如使用私有 LLM（而非来自 HuggingFace）或使用量化 LLM。请参考 [LMDeploy doc](https://lmdeploy.readthedocs.io/en/latest/serving/api_server.html) 文档。也可以使用其他支持 OpenAI API 框架的方法。
+
+### 使用 LMDeploy 加速模型推理
+
+可参考[文档](/docs/zh-CN/EvalByLMDeploy.md)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/README_zh-CN.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/README_zh-CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..92c526fcb3d2d5766469773c9eaa51196df53b4a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/README_zh-CN.md
@@ -0,0 +1,131 @@
+<div align="center">
+
+![LOGO](http://opencompass.openxlab.space/utils/MMLB.jpg)
+
+<b>VLMEvalKit: 一种多模态大模型评测工具 </b>
+
+[![][github-contributors-shield]][github-contributors-link] • [![][github-forks-shield]][github-forks-link] • [![][github-stars-shield]][github-stars-link] • [![][github-issues-shield]][github-issues-link] • [![][github-license-shield]][github-license-link]
+
+[English](/README.md) | 简体中文 | [日本語](/docs/ja/README_ja.md)
+
+<a href="https://rank.opencompass.org.cn/leaderboard-multimodal">🏆 OpenCompass 排行榜 </a> •
+<a href="#%EF%B8%8F-quickstart">🏗️ 快速开始 </a> •
+<a href="#-datasets-models-and-evaluation-results">📊 数据集和模型 </a> •
+<a href="#%EF%B8%8F-development-guide">🛠️ 开发指南 </a> •
+<a href="#-the-goal-of-vlmevalkit">🎯 我们的目标 </a> •
+<a href="#%EF%B8%8F-citation">🖊️ 引用 </a>
+
+<a href="https://huggingface.co/spaces/opencompass/open_vlm_leaderboard">🤗 HuggingFace 排行榜 (存档全部性能) </a> •
+<a href="https://huggingface.co/datasets/VLMEval/OpenVLMRecords">🤗 原始评测记录</a> •
+<a href="https://discord.gg/evDT4GZmxN">🔊 Discord</a> •
+<a href="https://www.arxiv.org/abs/2407.11691">📝 技术报告 </a>
+</div>
+
+**VLMEvalKit** (python 包名为 **vlmeval**) 是一款专为大型视觉语言模型 (Large Vision-Language Models， LVLMs) 评测而设计的开源工具包。该工具支持在各种基准测试上对大型视觉语言模型进行**一键评估**，无需进行繁重的数据准备工作，让评估过程更加简便。在 VLMEvalKit 中，我们对所有大型视觉语言模型生成的结果进行评测，并提供基于**精确匹配**与基于 **LLM 的答案提取**两种评测结果。
+
+## 🆕 更新
+
+- **[2025-04-29]** 优化 `torchrun` 启动逻辑：目前 `torchrun` 启动时，若进程数为 M，机器 GPU 卡数为 N，将会自动调整每个进程分配的 GPU 数量为 `N // M`。目前此分配方式适用于 `transformers`, `lmdeploy` 推理后端，`vllm` 推理后端仅支持使用 python 启动 🔥🔥🔥
+- **[2025-02-20]** 支持新模型：**InternVL2.5 series, QwenVL2.5 series, QVQ-72B, Doubao-VL, Janus-Pro-7B, MiniCPM-o-2.6, InternVL2-MPO, LLaVA-CoT, Hunyuan-Standard-Vision, Ovis2, Valley, SAIL-VL, Ross, Long-VITA, EMU3, SmolVLM**。支持新基准：**MMMU-Pro, WeMath, 3DSRBench, LogicVista, VL-RewardBench, CC-OCR, CG-Bench, CMMMU, WorldSense**。请参考[**VLMEvalKit Features**](https://aicarrier.feishu.cn/wiki/Qp7wwSzQ9iK1Y6kNUJVcr6zTnPe?table=tblsdEpLieDoCxtb)以获取更多信息。感谢社区的各位贡献者 🔥🔥🔥
+- **[2024-11-21]** 集成了一个新的配置系统，以实现更灵活的评估设置。查看[文档](/docs/zh-CN/ConfigSystem.md)或运行`python run.py --help`了解更多详情 🔥🔥🔥
+- **[2024-11-21]** 支持 **[QSpatial](https://andrewliao11.github.io/spatial_prompt/)**，一个用于定量空间推理的多模态基准（例如，确定大小/距离），感谢 **[andrewliao11](https://github.com/andrewliao11)** 提供官方支持 🔥🔥🔥
+- **[2024-11-21]** 支持 **[MM-Math](https://github.com/kge-sun/mm-math)**，一个包含约6K初中多模态推理数学问题的新多模态数学基准。GPT-4o-20240806在该基准上达到了22.5%的准确率 🔥🔥🔥
+- **[2024-11-16]** 支持 **[OlympiadBench](https://github.com/OpenBMB/OlympiadBench)**，一个多模态基准，包含奥林匹克级别的数学和物理问题 🔥🔥🔥
+- **[2024-11-16]** 支持 **[WildVision](https://huggingface.co/datasets/WildVision/wildvision-bench)**，一个基于多模态竞技场数据的主观多模态基准 🔥🔥🔥
+- **[2024-11-13]** 支持 **[MIA-Bench](https://arxiv.org/abs/2407.01509)**，一个多模态指令跟随基准 🔥🔥🔥
+- **[2024-11-08]** 支持 **[Aria](https://arxiv.org/abs/2410.05993)**，一个多模态原生 MoE 模型，感谢 **[teowu](https://github.com/teowu)** 🔥🔥🔥
+- **[2024-11-04]** 支持 **[WorldMedQA-V](https://www.arxiv.org/abs/2410.12722)**，该基准包含 1000 多个医学 VQA 问题，涵盖巴西、以色列、日本、西班牙等四个国家的语言，以及它们的英文翻译 🔥🔥🔥
+
+## 🏗️ 快速开始 <a id="quickstart"></a>
+
+请参阅[**快速开始**](/docs/zh-CN/Quickstart.md)获取入门指南。
+
+## 📊 评测结果，支持的数据集和模型 <a id="data-model-results"></a>
+
+### 评测结果
+
+**[OpenVLM Leaderboard](https://huggingface.co/spaces/opencompass/open_vlm_leaderboard)**: **[下载全部细粒度测试结果](http://opencompass.openxlab.space/assets/OpenVLM.json)**.
+
+请查看[**VLMEvalKit Features**](https://aicarrier.feishu.cn/wiki/Qp7wwSzQ9iK1Y6kNUJVcr6zTnPe?table=tblsdEpLieDoCxtb)中的 **Supported Benchmarks** 标签，以查看所有支持的图像和视频基准（70+）。
+
+请查看[**VLMEvalKit Features**](https://aicarrier.feishu.cn/wiki/Qp7wwSzQ9iK1Y6kNUJVcr6zTnPe?table=tblsdEpLieDoCxtb)中的 **Supported LMMs** 标签，以查看所有支持的 LMMs，包括商业 API、开源模型等（200+）。
+
+### 其他
+
+**Transformers 的版本推荐:**
+
+**请注意**，某些 VLM 可能无法在某些特定的 transformers 版本下运行，我们建议使用以下设置来评估对应的VLM:
+
+- **请用** `transformers==4.33.0` **来运行**: `Qwen series`, `Monkey series`, `InternLM-XComposer Series`, `mPLUG-Owl2`, `OpenFlamingo v2`, `IDEFICS series`, `VisualGLM`, `MMAlaya`, `ShareCaptioner`, `MiniGPT-4 series`, `InstructBLIP series`, `PandaGPT`, `VXVERSE`.
+- **请用** `transformers==4.37.0 ` **来运行**: `LLaVA series`, `ShareGPT4V series`, `TransCore-M`, `LLaVA (XTuner)`, `CogVLM Series`, `EMU2 Series`, `Yi-VL Series`, `MiniCPM-[V1/V2]`, `OmniLMM-12B`, `DeepSeek-VL series`, `InternVL series`, `Cambrian Series`, `VILA Series`, `Llama-3-MixSenseV1_1`, `Parrot-7B`, `PLLaVA Series`.
+- **请用** `transformers==4.40.0 ` **来运行**: `IDEFICS2`, `Bunny-Llama3`, `MiniCPM-Llama3-V2.5`, `360VL-70B`, `Phi-3-Vision`, `WeMM`.
+- **请用** `transformers==4.42.0 ` **来运行**: `AKI`.
+- **请用** `transformers==latest` **来运行**: `LLaVA-Next series`, `PaliGemma-3B`, `Chameleon series`, `Video-LLaVA-7B-HF`, `Ovis series`, `Mantis series`, `MiniCPM-V2.6`, `OmChat-v2.0-13B-sinlge-beta`, `Idefics-3`, `GLM-4v-9B`, `VideoChat2-HD`.
+
+**如何测试一个 VLM 是否可以正常运行:**
+
+```python
+from vlmeval.config import supported_VLM
+model = supported_VLM['idefics_9b_instruct']()
+# 前向单张图片
+ret = model.generate(['assets/apple.jpg', 'What is in this image?'])
+print(ret)  # 这张图片上有一个带叶子的红苹果
+# 前向多张图片
+ret = model.generate(['assets/apple.jpg', 'assets/apple.jpg', 'How many apples are there in the provided images? '])
+print(ret)  # 提供的图片中有两个苹果
+```
+
+## 🛠️ 开发指南 <a id="development"></a>
+
+要开发自定义评测数据集，支持其他 VLMs，或为 VLMEvalKit 贡献代码，请参阅[**开发指南**](/docs/zh-CN/Development_zh-CN.md)。
+
+为激励来自社区的共享并分享相应的 credit，在下一次 report 更新中，我们将：
+
+- 致谢所有的 contribution
+- 具备三个或以上主要贡献 (支持新模型、评测集、或是主要特性) 的贡献者将可以加入技术报告的作者列表 。合条件的贡献者可以创建 issue 或是在 [VLMEvalKit Discord Channel](https://discord.com/invite/evDT4GZmxN) 私信 kennyutc，我们将进行跟进
+
+## 🎯 VLMEvalKit 的目标 <a id="goal-of-vlmevalkit"></a>
+
+**该代码库的设计目标是：**
+
+1. 提供一个**易于使用**的**开源评估工具包**，方便研究人员和开发人员评测现有的多模态大模型，并使评测结果**易于复现**。
+2. 使 VLM 开发人员能够轻松地评测自己的模型。在多个支持的基准测试上评估 VLM，只需实现一个 `generate_inner()` 函数，所有其他工作负载（数据下载、数据预处理、预测推理、度量计算）都由代码库处理。
+
+**该代码库的设计目标不是:**
+
+复现所有**第三方基准测试**原始论文中报告的准确数字。有两个相关的原因:
+1. VLMEvalKit 对所有 VLMs 使用基于生成的评估（可选使用基于 LLM 的答案提取）。同时，一些基准测试可能官方使用不同的方法（*例如，SEEDBench 使用基于 PPL 的评估*）。对于这些基准测试，我们在相应的结果中比较两个得分。我们鼓励开发人员在代码库中支持其他评估范式。
+2. 默认情况下，我们对所有多模态模型使用相同的提示模板来评估基准测试。同时，**一些多模态模型可能有他们特定的提示模板**（目前可能未在代码库中涵盖）。我们鼓励 VLM 的开发人员在 VLMEvalKit 中实现自己的提示模板，如果目前未覆盖。这将有助于提高可复现性。
+
+## 🖊️ 引用 <a id="citation"></a>
+
+如果我们的工作对您有所帮助，请考虑 **star🌟** VLMEvalKit。感谢支持！
+
+[![Stargazers repo roster for @open-compass/VLMEvalKit](https://reporoster.com/stars/open-compass/VLMEvalKit)](https://github.com/open-compass/VLMEvalKit/stargazers)
+
+如果您在研究中使用了 VLMEvalKit，或希望参考已发布的开源评估结果，请使用以下 BibTeX 条目以及与您使用的特定 VLM / 基准测试相对应的 BibTex 条目。
+
+```bib
+@misc{duan2024vlmevalkit,
+      title={VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models},
+      author={Haodong Duan and Junming Yang and Yuxuan Qiao and Xinyu Fang and Lin Chen and Yuan Liu and Xiaoyi Dong and Yuhang Zang and Pan Zhang and Jiaqi Wang and Dahua Lin and Kai Chen},
+      year={2024},
+      eprint={2407.11691},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2407.11691},
+}
+```
+
+<p align="right"><a href="#top">🔝回到顶部</a></p>
+
+[github-contributors-link]: https://github.com/open-compass/VLMEvalKit/graphs/contributors
+[github-contributors-shield]: https://img.shields.io/github/contributors/open-compass/VLMEvalKit?color=c4f042&labelColor=black&style=flat-square
+[github-forks-link]: https://github.com/open-compass/VLMEvalKit/network/members
+[github-forks-shield]: https://img.shields.io/github/forks/open-compass/VLMEvalKit?color=8ae8ff&labelColor=black&style=flat-square
+[github-issues-link]: https://github.com/open-compass/VLMEvalKit/issues
+[github-issues-shield]: https://img.shields.io/github/issues/open-compass/VLMEvalKit?color=ff80eb&labelColor=black&style=flat-square
+[github-license-link]: https://github.com/open-compass/VLMEvalKit/blob/main/LICENSE
+[github-license-shield]: https://img.shields.io/github/license/open-compass/VLMEvalKit?color=white&labelColor=black&style=flat-square
+[github-stars-link]: https://github.com/open-compass/VLMEvalKit/stargazers
+[github-stars-shield]: https://img.shields.io/github/stars/open-compass/VLMEvalKit?color=ffcb47&labelColor=black&style=flat-square
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/css/readthedocs.css b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/css/readthedocs.css
new file mode 100644
index 0000000000000000000000000000000000000000..c83beffd261d9d7cb79dc499aec7187474639d89
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/css/readthedocs.css
@@ -0,0 +1,63 @@
+.header-logo {
+  background-image: url("../image/logo.svg");
+  background-size: 275px 80px;
+  height: 80px;
+  width: 275px;
+}
+
+
+@media screen and (min-width: 1100px) {
+  .header-logo {
+    top: -25px;
+  }
+}
+
+pre {
+    white-space: pre;
+}
+
+@media screen and (min-width: 2000px) {
+  .pytorch-content-left {
+    width: 1200px;
+    margin-left: 30px;
+  }
+  article.pytorch-article {
+    max-width: 1200px;
+  }
+  .pytorch-breadcrumbs-wrapper {
+    width: 1200px;
+  }
+  .pytorch-right-menu.scrolling-fixed {
+    position: fixed;
+    top: 45px;
+    left: 1580px;
+  }
+}
+
+
+article.pytorch-article section code {
+  padding: .2em .4em;
+  background-color: #f3f4f7;
+  border-radius: 5px;
+}
+
+/* Disable the change in tables */
+article.pytorch-article section table code {
+  padding: unset;
+  background-color: unset;
+  border-radius: unset;
+}
+
+table.autosummary td {
+  width: 50%
+}
+
+img.align-center {
+  display: block;
+  margin-left: auto;
+  margin-right: auto;
+}
+
+article.pytorch-article p.rubric {
+  font-weight: bold;
+}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/image/logo.svg b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/image/logo.svg
new file mode 100644
index 0000000000000000000000000000000000000000..043530572afb48d0eac26b4b53d448aae6e9a9af
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/image/logo.svg
@@ -0,0 +1,24 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" version="1.1" width="210" height="36" viewBox="0 0 1920 480" xml:space="preserve">
+<desc>Created with Fabric.js 5.3.0</desc>
+<defs>
+</defs>
+<g transform="matrix(8.6096948697 0 0 8.6096948697 283.4607944077 326.096948697)" id="2rbaYufxVtcDPCyBcyhLL"  >
+<path style="stroke: none; stroke-width: 1; stroke-dasharray: none; stroke-linecap: butt; stroke-dashoffset: 0; stroke-linejoin: miter; stroke-miterlimit: 4; fill: rgb(88,120,180); fill-rule: nonzero; opacity: 1;"  transform=" translate(-19.1, -28)" d="M 16.5 22.6 L 10.1 25.700000000000003 L 15.399999999999999 25.500000000000004 L 16.5 22.6 z M 12.3 33.6 L 13.4 30.700000000000003 L 8.100000000000001 30.900000000000002 L 12.3 33.6 z M 21.6 33.3 L 28 30.199999999999996 L 22.7 30.399999999999995 L 21.6 33.3 z M 25.8 22.4 L 24.7 25.299999999999997 L 30 25.099999999999998 L 25.8 22.4 z M 31.5 26.2 L 24.4 26.4 L 22.7 25.299999999999997 L 24.2 21.299999999999997 L 22.2 20 L 19 21.5 L 17.5 25.4 L 14.8 26.7 L 7.700000000000001 26.9 L 4.500000000000001 28.4 L 6.600000000000001 29.799999999999997 L 13.700000000000001 29.599999999999998 L 13.700000000000001 29.599999999999998 L 15.4 30.7 L 13.9 34.7 L 16 36 L 19.2 34.5 L 20.7 30.6 L 20.7 30.6 L 23.3 29.400000000000002 L 23.3 29.400000000000002 L 30.5 29.200000000000003 L 33.7 27.700000000000003 L 31.5 26.2 z M 20.2 28.7 C 19.2 29.2 17.9 29.2 17.2 28.8 C 16.599999999999998 28.400000000000002 16.8 27.6 17.8 27.2 C 18.8 26.7 20.1 26.7 20.8 27.099999999999998 C 21.5 27.5 21.2 28.2 20.2 28.7 z" stroke-linecap="round" />
+</g>
+<g transform="matrix(8.6096948697 0 0 8.6096948697 351.9078686218 195.2295866776)" id="Gj0NH9qgtM_hm6hdZ_Tf1"  >
+<path style="stroke: none; stroke-width: 1; stroke-dasharray: none; stroke-linecap: butt; stroke-dashoffset: 0; stroke-linejoin: miter; stroke-miterlimit: 4; fill: rgb(54,86,155); fill-rule: nonzero; opacity: 1;"  transform=" translate(-27.05, -12.8)" d="M 33.5 19.8 L 32.2 13.3 L 30.700000000000003 15.200000000000001 L 33.5 19.8 z M 27.5 5.1 L 23.3 2.3999999999999995 L 26 7 L 27.5 5.1 z M 20.7 5.7 L 22 12.2 L 23.5 10.299999999999999 L 20.7 5.7 z M 26.8 20.4 L 31 23.099999999999998 L 28.3 18.5 L 26.8 20.4 z M 34 22.3 L 30.4 16.1 L 30.4 16.1 L 29.9 13.400000000000002 L 31.9 10.800000000000002 L 31.299999999999997 7.600000000000002 L 29.199999999999996 6.200000000000003 L 27.199999999999996 8.800000000000002 L 25.499999999999996 7.700000000000003 L 21.799999999999997 1.400000000000003 L 19.6 0 L 20.200000000000003 3.2 L 23.900000000000002 9.5 L 23.900000000000002 9.5 L 24.400000000000002 12.1 L 24.400000000000002 12.1 L 22.400000000000002 14.7 L 23.000000000000004 17.9 L 25.100000000000005 19.299999999999997 L 27.000000000000004 16.799999999999997 L 28.700000000000003 17.9 L 32.400000000000006 24.2 L 34.50000000000001 25.599999999999998 L 34 22.3 z M 27.5 14.6 C 26.9 14.2 26.2 13 26 12 C 25.8 11 26.2 10.5 26.8 10.9 C 27.400000000000002 11.3 28.1 12.5 28.3 13.5 C 28.5 14.6 28.1 15.1 27.5 14.6 z" stroke-linecap="round" />
+</g>
+<g transform="matrix(8.6096948697 0 0 8.6096948697 209.8479032717 200.3954035994)" id="lmxdEhRpowSXVKC1LScVi"  >
+<path style="stroke: none; stroke-width: 1; stroke-dasharray: none; stroke-linecap: butt; stroke-dashoffset: 0; stroke-linejoin: miter; stroke-miterlimit: 4; fill: rgb(27,56,130); fill-rule: nonzero; opacity: 1;"  transform=" translate(-10.55, -13.4)" d="M 12 2.8 L 5.6 5.9 L 9.399999999999999 7.6000000000000005 L 12 2.8 z M 1.1 14.4 L 2.4000000000000004 20.9 L 5 16.099999999999998 L 1.1 14.4 z M 9.1 24 L 15.5 20.9 L 11.7 19.2 L 9.1 24 z M 20 12.4 L 18.7 5.9 L 16.099999999999998 10.7 L 20 12.4 z M 20.4 14.9 L 15.299999999999999 12.600000000000001 L 15.299999999999999 12.600000000000001 L 14.799999999999999 9.900000000000002 L 18.299999999999997 3.400000000000002 L 17.699999999999996 0.20000000000000195 L 14.499999999999996 1.700000000000002 L 11 8.1 L 8.3 9.4 L 8.3 9.4 L 3.2 7.1 L 0 8.6 L 0.6 11.8 L 5.8 14.100000000000001 L 6.3 16.8 L 6.3 16.8 L 2.8 23.4 L 3.4 26.599999999999998 L 6.6 25.099999999999998 L 10.1 18.599999999999998 L 12.7 17.4 L 12.7 17.4 L 17.9 19.799999999999997 L 21.099999999999998 18.299999999999997 L 20.4 14.9 z M 10.9 15.2 C 9.9 15.7 9 15.2 8.8 14.2 C 8.600000000000001 13.2 9.200000000000001 12 10.200000000000001 11.5 C 11.200000000000001 11 12.100000000000001 11.5 12.3 12.5 C 12.5 13.5 11.9 14.7 10.9 15.2 z" stroke-linecap="round" />
+</g>
+<g transform="matrix(1 0 0 1 1109.6072390741 240)" style="" id="IiDed4ej1oioRNDwSqAwf">
+		<text xml:space="preserve" font-family="Amethysta" font-size="225" font-style="normal" font-weight="normal" line-height="1" style="stroke: none; stroke-width: 1; stroke-dasharray: none; stroke-linecap: butt; stroke-dashoffset: 0; stroke-linejoin: miter; stroke-miterlimit: 4; fill: rgb(27,56,130); fill-rule: nonzero; opacity: 1; white-space: pre;"><tspan x="-714.0188713264" y="72.3065" style="white-space: pre; "> VLMEvalKit</tspan></text>
+<style>
+          @font-face {
+          font-family: "Amethysta";
+          font-weight: normal;
+          src: url("data:font/ttf;base64,AAEAAAAQAQAABAAAT1MvMkNd22wAALD0AAAAYFZETVjvltn1AACxVAAAC7pjbWFwzY9v1AAA4BgAAACsY3Z0IAeEKKwAAO3QAAAARmZwZ23wG6vaAADgxAAAC8RnYXNwAAcAHwAA9oQAAAAMZ2x5ZkxCsh8AAAEMAACqNGhkbXh305zTAAC9EAAAIwhoZWFkDGirbQAArSAAAAA2aGhlYRTzDDYAALDQAAAAJGhtdHjJk0PLAACtWAAAA3hsb2NhicReVQAAq2AAAAG+bWF4cANfDf0AAKtAAAAAIG5hbWWgJ7bsAADuGAAABiZwb3N0oRA6+AAA9EAAAAJCcHJlcG+p1t8AAOyIAAABSAACAEH/nAeSBR0AFwC2AapAGLa0sbCtq5+egX5WVFJRREM9OiUjIB8LBytLsCZQWEB7swEICa+uAgoIGgEHCotXHBMSEA4NCwcACwUHiQEABVshAgEASzMCBgF2Ri8pBAMCCBVZAQABFHRubGllQTcHAxIACQgJKwAICggrAAoHCisABwUHKwAFAAUrBAEAAQArAAYBAgEGAikAAQECAQAbAAICCBYAAwMIAxcNG0uwPlBYQHmzAQgJr64CCggaAQcKi1ccExIQDg0LBwALBQeJAQAFWyECAQBLMwIGAXZGLykEAwIIFVkBAAEUdG5saWVBNwcDEgAJCAkrAAgKCCsACgcKKwAHBQcrAAUABSsEAQABACsABgECAQYCKQABAAIDAQIBAB0AAwMIAxcMG0CEswEICa+uAgoIGgEHCotXHBMSEA4NCwcACwUHiQEABVshAgEASzMCBgF2Ri8pBAMCCBVZAQABFHRubGllQTcHAxIACQgJKwAICggrAAoHCisABwUHKwAFAAUrBAEAAQArAAYBAgEGAikAAwIDLAABBgIBAQAaAAEBAgEAGwACAQIBABgOWVmwLysBFz4BJy4BJz4CNyYnByYnBgcXBx4BNwEHJwIBFA4BIwcWFzI2HgIXHAEOAgcuAgcWFxYXBgcmByIOAwcuASImJz4DNy4CNTQ3IiYnIyInBicGBx4DFx4DFxQOAQcuAScWFwYnJicmByY1PgcyNjMuAjU0PgE3JjcuAycmJy4IJyY2MwU+Bzc+ATMyFzU3Mh0BBzcyBI2/CDABDn8rGkMtCy0fl4JFKQy6kwE9CAOhgwIF/hgFEg4ZdU4FLBgpJhICAQQBGRosGDETBAQDEEQ/DRMXCCMGAgUDAgEKFQ4hCiVcMRcPFQM3PTQUIxUKCzshMRQJMxwrEAIGAhsiJykOBAw1OD9QBgUKCg0IEAcSBhUCG1IsBRAFEwVSjGpCJAUDCkwYQh04HyohDg8IEwIMSqxndUNHKSMLPYFYNSuYBjOPBwIUkwI8CA1rJyNYOxEnDr5nLi8hk74KMAIDKYcC/hz+swkUFBkoJgICBhENAwQDAQMBDAsHAxwXBQYJA0QCAQcDEAICAQQFCw4FCgMUIhEHDRIcCwg2BwsKByUVIxACCQgbFgUDBAIYEQYbNQoCVQsJEwkHAwYEBAICAQIBFjUfCgcHCwQULi12hGI8CgQDGggXDBYQFhUMDSEuDyITGBEVFhoRbF0PAjYEAo4GAAIAmv/lAZYGAAAKAA4AYbgAMCu4AAkvQQMAEAAJAAFduAAD3EEHAFAAAwBgAAMAcAADAANdugANAAkAAxESObgADS+4AA7QALgADS+4ABNFWLgABi8buQAGABk+WbgAANy4AAzcQQMAIAANAAFdMDElMhYVFAYjIiY0NjcjAzMBGTRJSTQ2SUl1cyK44Uk0NUpKakhxBK4AAgBoA8kCLQXbAAMABwBFuAAwK7gAAS+4AALQuAABELgABdxBAwAvAAUAAV24AAbQALgAE0VYuAABLxu5AAEAHT5ZuAAA3LgABNC4AAEQuAAF0DAxEwMzAzMDMwONJaYpxyWmKQPJAhL97gIS/e4AAAIAZv/VBUgGFwAbAB8AjLgAMCsAuAAKL7gAE0VYuAAALxu5AAAAGT5ZugAdAAoAABESObgAHS+5ABsAFvS4AALQuAAdELgABdC6AAwACgAAERI5uAAML7kAHAAW9LgABtC4AAwQuAAJ0LgAChC4AA7QuAAMELgAENC4ABwQuAAT0LgAHRC4ABTQuAAbELgAF9C4AAAQuAAY0DAxBScTIzUzEyM1IRMXAyETFwMhFSEDIRUhAycTIRMDIRMBNYeU3Pxp/gEho4ieATOkh54BDf7TaQEv/q6Zh5P+zYtoATFoKxcB424BVG8CFxf+AAIXF/4Ab/6sbv4GFwHjAcL+rAFUAAMAhf9/A/4GRgAmAC0ANAE8uAAwK7oAJwANADMrQQMA/wANAAFdQQMAEAANAAFdQQMAEAAnAAFdQQMAMAAnAAFduAAnELgAINC6AAEADQAgERI5uAABL7oABQANACAREjm4AAUvuAAH0LgAARC4AAnQuAAx0LgAEtC4AAEQuAAl0EEFABgAJQAoACUAAl24ACvQuAAd0LgAFdC6ABcAIAANERI5uAAXL7gAGtC4AA0QuAAu0AC4ABNFWLgAFS8buQAVAB0+WbgAE0VYuAABLxu5AAEAGT5ZuAAA3LgAARC4AAbcuAABELkACQAW9LoACgABABUREjm4ABUQuAAS0LgAFRC4ABTcuAAVELgAGdy4ABUQuQAcABb0ugAxABUAARESObgAMRC4AB3QuAABELgAJdC4AAoQuAAq0LgACRC4ACvQuAAcELgAMtAwMQU1LgEnETMTFjMRLgE1ND4CNzUzFRYXESMnJiMRHgEVFA4CBxUTNCYnET4BARQWFxEOAQIUbrJpMV+Mbdi3Pm+OVF2Tridoa0froj9wi1PIVHRZb/4PU3lha4F/AR0fAV/+/CcCL2HJjFSHWTUIgX0DLP6y+BT97nGxlV+TXTUJgwHbW30//gYRbgOPTXtGAdsQZwAABQCa/9EGdQXDABMAHwArADcAOwCcuAAwK7gAJi+4ADLcQQMAEAAyAAFduAAA0LgAMhC4ACzcuAAK0LgAJhC4ACDcuAAU0LgAJhC4ABrQugA6ACYALBESOQC4ACkvuAA5L7gAE0VYuAA7Lxu5ADsAHT5ZuAATRVi4AC8vG7kALwAZPlm5AAUAFPS4AC8QuAA13LkADwAU9LgAKRC5ABcAFPS4ACkQuAAj3LkAHQAU9DAxARQeAjMyPgI1NC4CIyIOAgE0JiMiBhUUFjMyNjcUBiMiJjU0NjMyFgEUBiMiJjU0NjMyFgkBIwEEWh01RyoqSjcfHzVIKilJNh/+UnJVVHJwUlV2jceNkbq+j5S+AzzHjZG7vpCTv/60/UaGArsBMStOOyMkPVAsLFA8IyM9UgMKW4GEXleAg1qLvrCLkbuw/D+Kv7CLkbuwA/76DgXyAAADAGr/1wXlBdkAKgAyAD4AzLgAMCu4ABIvuAAw0LgAM9AAuAATRVi4ACEvG7kAIQAdPlm4ABNFWLgADS8buQANABk+WbgAE0VYuAAKLxu5AAoAGT5ZugABACEADRESObgAAS+5ACoAFvS4AATQugALAA0AARESOboAKAABAA0REjm6AAYACwAoERI5uAAKELkABwAW9LoANgAhAA0REjm6AC4ADQAhERI5ugAZADYALhESOboAJwA2AC4REjm4AA0QuQArABb0ugAtACgACxESObgAIRC5ADwAFPQwMQE1IRUHBgcfARUhJwYhIi4CNTQ+BDcnJjU0PgIzMhYVFAYHATY3ATI3AQYVFBYDFB8BPgE1NCYjIgYEGwGbi1Bso9P+gX/I/vhXmHhGECguVlFDYGg4YpVWrciLqQFzTUT9nsCM/mC+kDI9in1abVZhegLRKSlC1piyRimJsixUi1c1VEo6QjUoa3SeSIdpP8GHe69l/mtzqf3FgQHEi7SMegRKakKYU5FVbnyIAAABAGgDyQEOBdsAAwAkuAAwK7gAAS+4AALQALgAE0VYuAABLxu5AAEAHT5ZuAAA3DAxEwMzA40lpikDyQIS/e4AAQCo/lQCdQZ5AA0AKbgAMCu4AAMvQQMAfwADAAFdQQMAwAADAAFduAAK0AC4AAYvuAAALzAxASYCERASNxcGAhEQEhcCPbfe37Y2go2Pgv5UnQIcAVkBVwIdnzG0/fX+3f7c/fe0AAABAB7+VAHrBnkADQApuAAwK7gACy9BAwAvAAsAAV1BAwDAAAsAAV24AATQALgACC+4AAAvMDETJzYSERACJzcWEhEQAlY4go+Ngja2397+VDG0AgkBJAEjAgu0MZ/94/6p/qf95AAFABQDqANEBrAABQALABEAFwAdAEm4ADAruAABL0EDABAAAQABXbgABNC4AAfQuAABELgAENC4AAQQuAAV0LgAARC4ABrQALgAAi+4AA3cQQMAEAANAAFduAAK0DAxAQM3MxcDHwEPAScDIwMjLwE/ASUfAQcFJyUnPwEFAY05KV4pOSHpEEwzjB6KM0wO6HYBGykdEf7DlP7BDx0rARsFVgE7Hx/+xWjeMTcCASH+3Tcx4FSZH1gxKwIpL1odlwABAGYAjQSPBLYACwA4uAAwK7gABi+4AAPcuAAA0LgABhC4AAnQALgABi+4AAncQQMAAAAJAAFxuAAA0LgABhC4AAPQMDEBIRUhESMRITUhETMCwwHM/jSM/i8B0YwC44P+LQHTgwHTAAEAcf64AboA4QAQAFa4ADAruAAGL0EDABAABgABXbgAANxBCQBPAAAAXwAAAG8AAAB/AAAABF24AArQuAAGELgADtwAuAATRVi4AA4vG7kADgAZPlm4AAPcuAAOELgACdwwMTc0NjMyFhUUBgcnPgE1JyImuko1Qz6qfyBVdQI1SmQ0SV9JhtckPih0QRJKAAEAZgIpAuECpAADADi4ADAruAADL0EDAA8AAwABcUEFAEAAAwBQAAMAAl1BAwCgAAMAAV24AALcALgAAy+5AAAAFfQwMRMhFSFmAnv9hQKkewABAHH/5QFtAOEACgA+uAAwK7gACS9BAwAQAAkAAV24AAPcQQcAUAADAGAAAwBwAAMAA10AuAATRVi4AAYvG7kABgAZPlm4AADcMDE3MhYVFAYjIiY0NvA0SUk0NklJ4Uk0NUpKakgAAAH/mv6HA74GAAADAA+4ADArALgAAi+4AAAvMDETIwEzN50Dh53+hwd5AAIAcf/XBIUF7AAbADAAlLgAMCu6ABUAIQAzK0EDAA8AIQABcUEDAC8AIQABcUEDAD8AIQABXUEDAE8AIQABcbgAIRC4AAfQQQMAgAAVAAFdQQMAYAAVAAFduAAVELgALNAAuAATRVi4ACYvG7kAJgAdPlm4ABNFWLgAHC8buQAcABk+WbgAJhC5AAAAFvS4ABwQuQAOABb0QQMARQAhAAFxMDEBIg4EFRQeBDMyPgQ1NC4EAyIuAQI1NBI+ATMyHgMVFAIOAQJ9RmVILRkJCRktSGZFRGRGLRkJCRksR2RQicl2OECA0YltrHNNIT9/zwV7M1t9ladXVqWTfVszMlp9lKdXVqaVfVsz+lx01QESq6oBGNZ3TorG5omp/urUdQAAAQBIAAAC/gXbAAoAVbgAMCu4AAkvQQMAXwAJAAFdQQMAEAAJAAFduAAE0AC4ABNFWLgAAi8buQACAB0+WbgAE0VYuAAHLxu5AAcAGT5ZuQAJABb0uAAE0LgAAhC4AArQMDETJyUzERcVITUlEV4WAZtOzf1sAQsExVi++pwpTk4pBI8AAQBcAAAEEgXsAB4A7LgAMCu6AAYAAwAzK0EDAJAABgABXUEDAPAABgABXUEDAMAABgABXbgABhC4ABXQuAAA0LgAAC9BAwBAAAAAAV1BAwBvAAMAAV1BAwA/AAMAAV26AA4AAwAVERI5uAAOL0EFAC8ADgA/AA4AAnG4AAvQuAADELgAHNBBAwC2ABwAAV0AuAATRVi4ABAvG7kAEAAdPlm4ABNFWLgAAS8buQABABk+WbgAANy4AAEQuAAd3EEDALAAHQABXboAAwAdAAEREjm6AAQAEAABERI5uAAQELkACQAW9LgAEBC4AAzcugAbAAEAEBESOTAxAREhNQEAETQmIyIHAyMRNjMyHgIVFA4DBwEhNwQS/EoBmgEQnZZLX2s9n8lmrJBSCSVJgWD+sAJSRAGJ/neDAaABFAEbjpsV/tcBcT4nVZhnQ1Z4c5xf/rDnAAEAcf/XA8kF7AAvAPy4ADArugAIAAAAMytBAwBvAAAAAXFBAwA/AAAAAV1BAwA/AAAAAXFBAwBAAAAAAV24AAAQuAAD0EEDAGAACAABXUEDAEAACAABXboADAAAAAgREjm4AAwvugASAAAACBESObgAEi+4AAAQuAAZ0LgAGS+4ABbQuAASELgAIdC6ACQADAAhERI5uAAIELgAKtAAuAATRVi4ABsvG7kAGwAdPlm4ABNFWLgALi8buQAuABk+WbgAAty4AC4QuQAFABb0ugANABsALhESObgADS9BAwA/AA0AAV25AAwAFfS4ABsQuQAUABb0uAAbELgAF9y6ACQADQAMERI5MDE3ETMTFjMyNjU0JisBNTMyPgE1ECEiBwMjETYzMh4DFRQGBxUeAxUUDgEjInEzbGJZkKeogWhmSm8z/u9BUms9oa1IfnZUMoh0T2xYK4vpmasbAX3+1yuSn4+yiV6ERwEREf7XAXE8FjZTg1NtzTEIGjxchFmHvVoAAgAKAAAEIQXsAAoADQCyuAAwK7oACAAJADMrQQMAMAAIAAFdQQMALwAIAAFdQQMAwAAIAAFdQQMAkAAIAAFduAAIELgADNC4AAgQuAAF0LgAAtC6AAAADAACERI5QQMALwAJAAFduAAJELgAC9AAuAATRVi4AAAvG7kAAAAdPlm4ABNFWLgABi8buQAGABk+WboABQAAAAYREjm4AAUvuAAC3LgABRC4AAjQuAACELgADNC4AArQuAAAELgADdAwMQEzETMVIxEjESE1FyERAtGLxcW4/WaaAgAF7Pvrnv7HATmiBAL6AAEAXP/XA9UFwwAlANu4ADArugAKAAAAMytBAwA/AAAAAV1BAwBvAAAAAV24AAAQuAAD0EEDAPAACgABXUEDAJAACgABXUEDAMAACgABXbgAChC4AB/QugASAAAAHxESObgAEi9BBQAPABIAHwASAAJxugAVAB8AABESObgAFS+4ABIQuAAX0AC4ABNFWLgAFC8buQAUAB0+WbgAE0VYuAAkLxu5ACQAGT5ZuAAC3LgAJBC5AAUAFvS6ABkAFAAkERI5uAAZL0EDADAAGQABcUEDAMAAGQABXbkAEAAV9LgAFBC4ABXcMDE3ETMTFjMyPgI1NC4DIyIHESEVIRE2MzIeAxUUDgIjIlwzbWBaRHhhORI0U49dbJYDFf12fmZtq2lFG16ez3KgGwF9/tcrJkt9UT5qb04xIQLLnv5iGTtdfnxBdMB9RQACAHH/1wR1BewAEwAtANq4ADArugAFACMAMytBAwBgAAUAAV1BAwAgAAUAAXFBAwCwAAUAAV1BAwBAAAUAAV1BAwAQAAUAAV1BAwAPACMAAXFBAwBfACMAAXFBAwAgACMAAXG4ACMQuAAP0LgAF9C4AAUQuAAc0LoAKwAcACMREjkAuAATRVi4ACkvG7kAKQAdPlm4ABNFWLgAIC8buQAgABk+WbkAAAAW9LgAIBC4ABncQQMAAAAZAAFxQQMAQAAZAAFxQQMAYAAZAAFxuQAKABX0uAApELkAFAAU9LoAFwAZACAREjkwMSUyPgI1NC4CIyIHDgEVFB4CASIAAzYzMgAVFA4BIyAAETQ+AzMyFwcmAnU8cFYzNlp4QoKaAgMgSHcBbeX+zCa7nccBCovnjv8A/vw7d6Pgf3FyCjFEMGKUY2aPWSlIETUjhtqbVAVH/s7+64H+89+a7XsBawFag/jWol0bTggAAAEAHwAAA8sFwwAIAG24ADAruAAHL0EDAC8ABwABXUELAC8ABwA/AAcATwAHAF8ABwBvAAcABXG4AATQALgAE0VYuAAHLxu5AAcAHT5ZuAATRVi4AAEvG7kAAQAZPlm4AAcQuAAE3LoAAAAHAAQREjm4AAcQuAAF3DAxCQEjASEHIxEhA8v+GNMCD/2VK2oDrAV5+ocFI9MBcwADAFz/1wQhBewAIAAvADwBH7gAMCu6ACsAEwAzK0EDAKAAKwABXUEDAC8AKwABXUEDAE8AKwABcUEDADAAKwABXUEDABAAKwABXUEDAH8AEwABXUEDAC8AEwABXUEDABAAEwABXUEDADAAEwABXboANwArABMREjm4ADcvuAAA0LgAKxC4AAnQugAZABMAKxESObgAGS+4ADDQugADAAkAMBESOboAFgAZACsREjm6ACIAKwAZERI5uAATELgAJdC6ADQAMAAJERI5ALgAE0VYuAAeLxu5AB4AHT5ZuAATRVi4AA4vG7kADgAZPlm6ADQAHgAOERI5ugAiAA4AHhESOboAAwA0ACIREjm6ABYAIgA0ERI5uQAoABb0QQMARwArAAFxuAAeELkAOgAW9DAxARQGBx4EFRQOAiMiLgI1NDY3LgE1ND4CMzIWAScOARUUFjMyNjU0LgIBFBYfAT4BNTQmIyIGA+GNeUNVWzMgRXq7bm24dkKTkoNrSXiVUMnf/khUY1eegH+cFz17/tV2oRxIUnl1ZXoEcV+zSCIySUpoP1CafExJeZpSetM8Q6xtW5ZeM8/9mSs2noKQt7NzK0lWVwH4X4dRDzqcR4OghwAAAgBI/9cETAXsABkALADiuAAwK7oAKAAIADMrQQMAEAAoAAFdQQMALwAoAAFdQQMAUAAoAAFxQQMAAAAoAAFxuAAoELgAA9BBAwBPAAgAAV1BAwAvAAgAAV1BAwBvAAgAAV1BAwC/AAgAAV24ACgQuAAP0LoAFwAIAA8REjm4AAgQuAAf0AC4ABNFWLgADC8buQAMAB0+WbgAE0VYuAAVLxu5ABUAGT5ZuQAAABT0uAAMELgABdxBAwAPAAUAAXFBAwBvAAUAAXG6AAMABQAMERI5QQMAZQAIAAFduAAMELkAGgAW9LgABRC5ACQAFfQwMSUyABMGIyIANTQ+ATMgABEUDgMjIic3FgEiDgIVFB4CMzI3NjU0LgIBMeUBNCe6nsf+9orojgEAAQQ7d6Pgf3hsCjEBWTxtWTQ1XXNFg5oEIEZ6NwEyARaBAQzfm+17/pX+poP41qJdG00IBUgwXJtiY5RWKkgiRonSoVQAAgCa/+cBlgQMABMAHgBjuAAwK7gADy9BAwAQAA8AAV24AAXcQQcAUAAFAGAABQBwAAUAA124ABfQuAAPELgAHNAAuAATRVi4ABQvG7kAFAAfPlm4ABNFWLgACi8buQAKABk+WbgAANy4ABQQuAAa3DAxJTIeAhUUDgIjIi4CNTQ+AhMyFhUUBiMiJjQ2ARkZLiIUEyIuGhsuIhQUIi4bNElJNDZJSeMUIi4ZGi4jFBQiLhsbLSITAylJNDVKSmpIAAACAJr+uAHnBAwACQAaAIO4ADAruAAQL7gAA9C4AAfcQQcAXwAHAG8ABwB/AAcAA124ABAQuAAK3EEHAF8ACgBvAAoAfwAKAANduAAU0LgAEBC4ABjcALgAE0VYuAAALxu5AAAAHz5ZuAATRVi4ABgvG7kAGAAZPlm4AAAQuAAF3LgAGBC4AA3cuAAYELgAE9wwMQEyFhQGIyImNDYDNDYzMhYVFAYHJz4BNSciJgFqNElJNDVJSVJKNUM+qn8gVXUCNUoEDEhqSkpqSPxYNElfSYbXJD4odEESSgAAAQBcAHkEMwTBAAYAM7gAMCu4AAEvuAAC3LgAARC4AATQuAACELgABdAAuAACL7gAA9y4AAIQuAAG3LgABdwwMRM1ARUJARVcA9f84QMfAmprAeym/oX+faQAAgBmAZwEkQN5AAMABwAvuAAwK7gABC+4AAXcuAAC0LgABBC4AAPQALgABy+4AAPcuAAA3LgABxC4AATcMDETIRUhFSEVIWYEK/vVBCv71QN5e+d7AAABAJoAeQRxBMEABgA4uAAwK7gAAC9BAwAQAAAAAV24AAbcuAAD0LgAABC4AATQALgABi+4AALcuAAD3LgABhC4AAXcMDEBFQE1CQE1BHH8KQMe/OIC1Wv+D6QBgwF7pgACAHH/5QO5BewAFgAhALi4ADArugAKABEAMytBAwBwAAoAAV24AAoQuAAA0LoABgARAAAREjm4AAYvuAAD0LgAERC4AA/QuAAGELgAGtC4ABovQQUAfwAaAI8AGgACXbgAH9xBBwBQAB8AYAAfAHAAHwADXQC4ABNFWLgAFC8buQAUAB0+WbgAE0VYuAAXLxu5ABcAGT5ZuAAd3LgABdy6AAIABQAUERI5ugAHABQABRESObgAFBC5AA0AFvS4ABQQuAAQ3DAxAQYPARUjNTc+ATU0JgciBwMjETYzMgQBIiY1NDYzMhYUBgO5A8biZsU7Obx7S0ZrPae45QEB/iU0SUk0NklJBIT0p8i99NNHnlWTjQsU/tcBcj2++rdJNDVKSmpIAAACAHv/HwcABbgAQgBPAQm4ADAruAA0L0EDABAANAABXbgAPdy6AAsANAA9ERI5uAALL7gAR9y4AAHQuABHELgAEtC4AD0QuAAZ0LgANBC4ACLQugArADQAPRESObgACxC4AE3QALgAE0VYuAA5Lxu5ADkAHT5ZuAAu3EEDABAALgABXboABgA5AC4REjm4AAYvQQMA0AAGAAFduAAS3EEDACAAEgABXboAAQASAAYREjm4AAYQuABA0LgAQC+4ABPcQQMA8AATAAFdQQMAAAATAAFxuAA5ELgAHNxBAwAPABwAAXG4AC4QuAAo3LgABhC4AEPcQQcA0ABDAOAAQwDwAEMAA11BAwAAAEMAAXG4ABIQuABH3DAxATUnDgIjIi4CNTQ+BBcRMj4DNTQAISIOAxUUHgMzMjcXDgEjIi4CAjU0EjYkMzIEEhUQACEiJiUyNjcRIg4DFRQWBHUGRlJkRkFfNRkgRHWb4IZIdE41F/6s/vF12baHTEyGttJztMshcvhvgfLaol+L8AFSwOABW73+yf8AJi7+80J1SUJ5WUQiRgEKgQJaUis1XXBCRoqOd1svBfznOWGHl1XcARNEhLTshYnkoG80VkBGSz2DuwEQoroBRueFpP7cuf7k/pc2YktfAfM8Y3yGPldnAAL//AAABcsFwwAPABMAibgAMCu4ABQvuAAH0EEDAIUABwABXUEDABMABwABXbgAEdAAuAATRVi4AAkvG7kACQAdPlm4ABNFWLgABS8buQAFABk+WboAEQAJAAUREjm4ABEvuQABABb0uAAFELkABwAW9LgAAtC4AAcQuAAK0LgABRC4AA3QuAAKELgAD9C4AAkQuAAT0DAxASEDFxUhNTcBMwEXFSE1NwEDIQMD0/3wkqj+I64CClwCF6T+AKL+VtcBvt8B5f55NSkpSAVS+q5IKSk1BDP9vwJBAAADAHEAAAUABcMAEwAfACgAxLgAMCu6ABoADQAzK0EDAEAAGgABXUEDABAAGgABXbgAGhC4ACTQuAAkL7gAANBBAwAvAA0AAV1BAwAQAA0AAV1BAwCQAA0AAV24AA0QuAAV0LgAINC6AAMAIAAAERI5uAAaELgAB9AAuAATRVi4ABAvG7kAEAAdPlm4ABNFWLgACi8buQAKABk+WbkADQAW9LgAEBC5AA4AFvS6ACAAEAAKERI5uAAgL7kAFAAW9LgAChC5ABYAFvS4ABAQuQAnABb0MDEBFAYHFR4BFRQEKQE1NxEnNSEgFgERITI+ATU0LgIjJTMyNjU0JisBBKqAbJer/ub++f2SuLgCJwEY+v1CAQ5jj0EoSnhJ/vLfhoqJnsgEaHekMAstwonB2SlIBOFIKbT9yv2RXYlOOm5bOGudd3uFAAEAe//nBOcF4wAjAL24ADArugAAAAgAMytBAwBgAAAAAV1BAwBgAAAAAXFBAwCwAAAAAV1BAwBAAAAAAV1BAwAQAAAAAV1BAwAvAAgAAV1BAwAQAAgAAV24AAAQuAAP0LgADy+4ABLQuAAIELgAGtC4AAAQuAAh0AC4ABNFWLgADS8buQANAB0+WbgAE0VYuAACLxu5AAIAGT5ZuAANELgAEdy4AA0QuQAUABb0uAACELkAHwAW9LgAAhC4ACLcQQMAMAAiAAFdMDElBiMiLgM1NBI2JDMyFwMjAyYjIg4DFRQeAjMyNxMzBOevt3zhxZFTetMBGKGqrgIpfVSJUI6DXzhZmstzZltuKSU+N3is+Za8AS/BZjf+gwEdJSpileSPmPmhVx0BSQACAHEAAAWoBcMADwAaAIC4ADArugAYAA0AMytBAwCwABgAAV24ABgQuAAG0EEDAC8ADQABXUEDAE8ADQABXbgADRC4ABLQALgAE0VYuAAALxu5AAAAHT5ZuAATRVi4AAovG7kACgAZPlm5AA0AFvS4AAAQuQAOABb0uAAAELkAEAAW9LgAChC5ABMAFvQwMRMhMgQWEhUUAgQjITU3EScFIxEzMj4CNRAAcQJsnQEEv2u2/rrR/Za4uAJe4+llt49V/vIFw1+2/uW02P6xuClIBOFIQvsSTpTskwE7AVIAAAEAcQAABI8F3QAWAPi4ADArugAWAAIAMytBAwA/AAIAAV1BAwAfAAIAAXFBAwCvAAIAAV1BAwA/ABYAAV1BAwBQABYAAV24ABYQuAAI0LgACC+4AAvQuAACELgAEtC4AA3QugAPAA0ACBESObgAFhC4ABPQALgAE0VYuAAFLxu5AAUAHT5ZuAATRVi4AAAvG7kAAAAZPlm5AAIAFvRBAwCnAAIAAV24AAUQuQADABb0uAAFELgACty4AAUQuQAMABb0ugANAAUAABESObgADS9BAwA/AA0AAV1BAwBfAA0AAV1BAwBPAA0AAXG5ABEAFvS4AAAQuQASABb0uAAAELgAFNwwMTM1NxEnNSE3FxEjAyERJRcHJREhEzMRcbi4Ay2+Hyl//hkBnycU/k4B+38pKUgE4UgpGhr+bgEn/c0UDoUQ/bQBJ/5vAAEAcQAABD0F3QAUALq4ADArugALAAUAMytBAwAvAAUAAV1BAwAfAAUAAXFBAwAQAAUAAV24AAUQuAAA0EEDAC8ACwABXbgACxC4AA7QuAAAELgAENC6ABIACwAQERI5ALgAE0VYuAAILxu5AAgAHT5ZuAATRVi4AAMvG7kAAwAZPlm5AAUAFvS4AADQuAAIELkABgAW9LgACBC4AA3cuAAIELkADwAW9LoAEAAIAAMREjm4ABAvQQMAPwAQAAFduQAUABb0MDElFxUhNTcRJzUhNxcRIwMhESUXByUB7PX9kLi4Au+/Hit//lkBnycU/k5xSCkpSAThSCkaGv5sASn9uBUOhhEAAAEAe//lBVIF4wAsALS4ADArugAnAAwAMytBAwAPAAwAAXFBAwAQAAwAAV1BBQAQACcAIAAnAAJdQQMAQAAnAAFxQQMAIAAnAAFxuAAnELgALNC6ABUALAAMERI5uAAVL7gAGNC4AAwQuAAf0AC4ABNFWLgAEy8buQATAB0+WbgAE0VYuAAFLxu5AAUAGT5ZuAATELgAF9y4ABMQuQAaABb0uAAFELkAJAAW9LoAKQATAAUREjm4ACkvuQAnABb0MDElDgMjIi4ENTQ+BDMyFwMjAyYjIg4CFRQeAjMyNxElNSEXBwUpOmtpazllv6qOZzk+bpm4z223pQIpe2Ntb86eX1yf1HdoY/78AbwpKUocJhgLLFV/p8x5gtqvg1gsN/6DARsgS574rKHzpFMdAeUzPil/AAABAHEAAAZQBcMAGwDXuAAwK7oAAAAIADMrQQMAsAAAAAFdQQMALwAIAAFdQQMATwAIAAFduAAIELgAA9C4AA/QuAAAELgAENC4AAAQuAAX0AC4ABNFWLgACy8buQALAB0+WbgAE0VYuAAGLxu5AAYAGT5ZuQAIABb0uAAD0LgAANC6AA8ACwAGERI5uAAPL0EDAE8ADwABcUEDAF8ADwABXUEDAD8ADwABXbkAAgAW9LgACxC5AAkAFvS4AA7QuAAR0LgACxC4ABPQuAARELgAFtC4AAAQuAAX0LgABhC4ABrQMDElESERFxUhNTcRJzUhFQcRIREnNSEVBxEXFSE1BNX9F7j91664AimuAum4AhSZuP3XcQJH/blIKSlIBOFIKSlI/dECL0gpJUz7H0gpKQABAHEAAAKkBcMACwBWuAAwK7gABi9BAwAvAAYAAV24AAHQALgAE0VYuAAJLxu5AAkAHT5ZuAATRVi4AAQvG7kABAAZPlm4AAkQuQAHABb0uAAA0LgABBC5AAYAFvS4AAHQMDEBERcVITU3ESc1IRUB7Lj91664AikFUvsfSCkpSAThSCkpAAEAEP7yAnUFwwANAES4ADAruAAIL0EDAD8ACAABXUEFAC8ACAA/AAgAAnG4AAHQALgABC+4ABNFWLgACy8buQALAB0+WbkACQAW9LgAANAwMQERFAYHJz4BNREnNSEVAcfU3AeOZrgCKQVS+5PB9D5BUb+zBFxIKSkAAQBxAAAFiwXDABoAybgAMCu4AA8vQQMALwAPAAFduAAE0EEDACMABAABXbgAAtC4AA8QuAAK0LgAFtC6AAMAFgACERI5uAAEELgAB9C4AAIQuAAY0AC4ABNFWLgAEi8buQASAB0+WbgAE0VYuAANLxu5AA0AGT5ZuAASELkAEAAW9LgAGNC4AALQugAJABIADRESObgACS+4ABfQugADABcACRESObgADRC5AA8AFvS4AATQuAANELgAB9C4AA8QuAAK0LgAEBC4ABXQuAASELgAGtAwMQEVBwkBFxUhASMRFxUhNTcRJzUhFQcRNwEnNQV7xf5EAdW8/qr+JXm5/eGuuAIerpgBppoFwzBJ/dP9XkwvAsP9rkgpKUgE4UgpKUj9zRwCHTswAAEAcQAABJoFwwANAIK4ADArugAAAAMAMytBAwA/AAAAAV1BAwA/AAMAAV1BAwCvAAMAAV24AAMQuAAK0LgAABC4AAvQALgAE0VYuAAGLxu5AAYAHT5ZuAATRVi4AAEvG7kAAQAZPlm5AAMAFvS4AAYQuQAEABb0uAAJ0LgAARC5AAoAFvS4AAEQuAAM3DAxKQE1NxEnNSEVBxEhEzMEmvvXuLgCM64B/H8pKUgE4UgpKUj7IwEnAAABAFwAAAcEBcMAHAEpuAAwK7oACQAVADMrQQMALwAJAAFxQQMA4AAJAAFdQQMAwAAJAAFduAAJELgABNBBDwCYAAQAqAAEALgABADIAAQA2AAEAOgABAD4AAQAB124AADQQQMALwAVAAFxQQMAEAAVAAFxugANABUABBESObgADRC4AAzQuAAVELgAENC4ABUQuAAZ0LoAHAAVAAQREjkAuAATRVi4ABkvG7kAGQAdPlm4ABNFWLgAAS8buQABAB0+WbgAE0VYuAATLxu5ABMAGT5ZuAATRVi4AAcvG7kABwAZPlm4AAEQuQADABb0uAAHELkACQAW9LgABNC4AAEQuAAK0LgAExC4AA3QuAAKELgADtC4ABMQuQAVABb0uAAQ0LgAGRC5ABYAFvS4AA0QuAAc0DAxASEVBxEXFSE1NxEjASMBIxEXFSE1NxEnNSEXATMFagGQrrj+CqAM/ill/j4Ko/4prrgB3QQBbwgFwylI+x9IKSlIBHv7FATw+4tSKSlSBNdIKV/71wABAEgAAAXsBcMAGgC9uAAwK7oAFgAMADMrQQMAPwAWAAFdQQMAYAAWAAFduAAWELgAANC4ABYQuAAC0EEHAC8ADAA/AAwATwAMAANdQQMAvwAMAAFduAAMELgAB9C4AAwQuAAQ0AC4ABNFWLgADy8buQAPAB0+WbgAE0VYuAAKLxu5AAoAGT5ZuAAPELkADQAW9LgAFtC4AADQuAAKELgAAtC4AA8QuAAF0LgAChC5AAwAFfS4AAfQuAACELgAFNC4AA8QuAAY0DAxAREjAScjFxEXFSE1NxEnNSEXARMzJxEnNSEVBT1e/Sl9DAak/imuuAG4FAIDnw0LpAHYBUj6uAQx4ff8YFIpKVIE10gpY/0R/vnwAu5SKSkAAAIAe//XBcsF8AASAB8AnbgAMCu6AB0ABAAzK0EDAC8ABAABXUEDAC8ABAABcUEDABAABAABXUEDAJAAHQABXUEDAGAAHQABcUEDAEAAHQABcUEDAEAAHQABXUEDABAAHQABXbgAHRC4AA7QuAAEELgAF9AAuAATRVi4AAkvG7kACQAdPlm4ABNFWLgAAC8buQAAABk+WbgACRC5ABMAFvS4AAAQuQAaABb0MDEFIiQCNTQSPgEzMh4BEhUUAg4BAyIGAhUQADMyEhEQAAMZwf7Rrm29/pCP9bBkbbr9gozVcgEA09D//wApuwFi6a4BKcpyccr+2a+u/trGbgWorf7PxP7W/pEBcgEpASoBdgAAAgBxAAAErAXDABUAIgCsuAAwK7oAIAAGADMrQQMAPwAGAAFdQQMAHwAGAAFxQQMAgAAGAAFduAAGELgAAdBBAwA/ACAAAV1BAwCAACAAAV24ACAQuAAP0LgAARC4ABjQALgAE0VYuAAJLxu5AAkAHT5ZuAATRVi4AAQvG7kABAAZPlm5AAYAFvS4AAHQuAAJELkABwAW9LoAFAAJAAQREjm4ABQvuAAJELkAFwAW9LgAFBC5ABsAFvQwMQERFxUhNTcRJzUhMh4CFRQOAiMiEyMRHgEzMj4CNTQmAeHi/a64uAIcgsqLSEKM2JdDfskmRB9wkVUhoAJY/hlIKSlIBOFIKTtsl1xfrIJNAwv9aQUFOGCASKGgAAIAe/5WBd8F3wATAC4BFLgAMCu6AA8AJQAzK0EDABAAJQABcUEDAC8AJQABXUEDAA8AJQABcUEDAN8AJQABXUEDAC8AJQABcUEDAOAAJQABXUEDABAAJQABXbgAJRC4AAXQQQMAQAAPAAFdQQMA4AAPAAFdQQMAQAAPAAFxQQMA3wAPAAFdQQMAEAAPAAFxQQMAkAAPAAFdQQMAEAAPAAFdQQMAYAAPAAFxuAAPELgAFNC6ACAAJQAUERI5uAAgELgAGdBBAwAgADAAAXFBAwBAADAAAV0AuAAeL7gAE0VYuAAqLxu5ACoAHT5ZuAATRVi4ACAvG7kAIAAZPlm4ACoQuQAAABb0uAAgELkACgAW9LgAIBC4ABnQuAAeELgAG9AwMQEiDgIVFB4CMzI+AjU0LgIBFAIOAQcWFzcXByYDLgICNTQSNiQzMh4BEgMva615QkJ5rWtqq3lBQnqqAkdWmct3S5CcFPHlZ4rrq2FwwQEBkI/3tWcFb2Cu8pKT76pdXqzvkpPxrV/9cJj++rp7FqNqFSWDiQEKB26/AReprgEkxGxsw/7eAAIAcQAABWYFwwAcACYAxrgAMCu6ACQADQAzK0EDAC8ADQABXUEDALAAJAABXUEDABAAJAABcboAAQANACQREjm4AA0QuAAI0LgAJBC4ABbQuAABELgAGdC4AAgQuAAf0AC4ABNFWLgAEC8buQAQAB0+WbgAE0VYuAALLxu5AAsAGT5ZuAAA0LoABAAQAAsREjm4AAQvuAALELkADQAW9LgACNC4ABAQuQAOABb0ugAZAAQAEBESObgADRC4ABrQuAAQELkAHgAW9LgABBC5ACIAFvQwMSEBDgEjIiYnERcVITU3ESc1ITIeAhUUBgcBFxUBIxEeATMgETQmBDH+xR09ICVNKeL9rri4AjOCyotIi4gBI5P9W+ApSSMBgqACagQEBgX+BEgpKUgE4UgpO2yXXIfRN/3IOSkFWv19BQUBTKGgAAEAZv/nA/AF3wAzALu4ADArugAHABEAMytBAwA/ABEAAV1BAwAvABEAAXFBAwAQABEAAV24ABEQuAAx0LgAMS+4AADQQQMAEAAHAAFduAAHELgAJ9C6ABUAJwARERI5uAAVL7gAGNC4ABEQuAAd0AC4ABNFWLgAEy8buQATAB0+WbgAE0VYuAAsLxu5ACwAGT5ZuQACABb0ugAMACwAExESObgAExC4ABfcuAATELkAGgAW9LoAIgATACwREjm4ACwQuAAz3DAxNxYzMj4CNTQuAicuAzUQITIXESMnJiMiBhUUHgIXHgMVFA4CIyIuAicRM/ydaTZpUzMaRXZcZJFfLQHXp7kpbGpVj6AWRHxlZJViMVCHtGM6Y2BkOzSDJxc4XUYuUVBTMjZhaHhNAXkv/qb+Fm1yKEhOWDg2XGFyS3ihYyoHDxkRAWgAAQAKAAAE2QXdABMAr7gAMCu4ABAvuAAA0LgAAC9BAwCvAAAAAV1BAwAvAAAAAV24ABAQuAAJ0LgABdC4AAUvQQMAbwAFAAFxQQMAvwAFAAFdQQMAkAAFAAFdQQMAEAAFAAFxuAAI0LgAABC4ABHQALgAE0VYuAACLxu5AAIAHT5ZuAATRVi4AA0vG7kADQAZPlm4AAIQuAAT3LgAB9C4AAIQuQARABb0uAAI0LgADRC5AA8AFvS4AArQMDETNxchNxcRIwMlERcVITU3EQUDIwofvgMVvh8pff6kuP3irv6XgykFwxoaGhr+YgExDvsNSCkpSATzEP7RAAEAXv/XBhIFwwAdAJ+4ADArugAFABUAMytBAwCAAAUAAV24AAUQuAAM0EEDAE8AFQABXUEDAM8AFQABXUEDAK8AFQABXUEDAC8AFQABXUEDAIAAFQABXbgAFRC4ABzQALgAE0VYuAAYLxu5ABgAHT5ZuAATRVi4AA8vG7kADwAZPlm5AAAAFvS4ABgQuQAWABb0uAAb0LgABtC4ABgQuAAI0LgABhC4AAvQMDElMj4CNREnNSEVBxEQACEiLgM1ESc1IRUHERADO2WWdDu4AeWo/uz+6WKdkGA5uQIprkQuaryHAylSKSlS/OX+z/7bH1GBzokDM0gpKUj85/4LAAAB/83/7AWeBcMADwBTuAAwKwC4ABNFWLgADy8buQAPAB0+WbgAE0VYuAAMLxu5AAwAGT5ZuAAPELkADQAW9LgAAtC4AAwQuAAD0LgADRC4AArQuAAF0LgADxC4AAfQMDEBFQcBMwEnNSEVBwEjASc1AdGkAZwIAZmVAcmu/eVO/eymBcMpPPvRBC88KSlI+poFZkgpAAAB/83/7Af+BcMAFwBzuAAwKwC4ABNFWLgAFy8buQAXAB0+WbgAE0VYuAAULxu5ABQAGT5ZuAAXELkAFQAW9LgAAtC4ABQQuAAE0LgAFxC4AAXQuAAEELgACNC4ABUQuAAO0LgACdC4AAUQuAAL0LgAFBC4ABDQuAAFELgAEdAwMQEVBwEzATMBMwEnNSEVBwEjASMBIwEnNQICsAEWCQFqgQFQCAEZrgHfrv6FXv6FBv6bbP5guAXDKUT7zwSe+3gEGUYpKUj6mgSB+38FZkgpAAABAA4AAAW2BcMAGwCiuAAwK0EDABQAEwABXQC4ABNFWLgAFi8buQAWAB0+WbgAE0VYuAAQLxu5ABAAGT5ZuAAWELgAAdC4ABYQuQAUABX0uAAb0LgABNC6AAwAEAAWERI5ugAaABYAEBESOboABQAMABoREjm4ABAQuQASABX0uAAG0LgAEBC4AAnQuAAGELgAC9C4ABIQuAAN0LoAEwAaAAwREjm4ABQQuAAZ0DAxATUhFQcJARcVITU3CQEXFSE1NwkBJzUhFQcJAQORAgLG/nUBwLT9v7T+qv6arv3zyQGu/mC4AjGgATgBQwWHPDxU/df9d0Y7O0IB8v4KPjs7TgJWAl5KPDxK/j4BxAAB/80AAAUnBcMAFQBruAAwK7gAEi+4AAvQALgAE0VYuAAVLxu5ABUAHT5ZuAATRVi4AA8vG7kADwAZPlm4ABUQuQATABb0uAAC0LoAAwAVAA8REjm4ABMQuAAK0LgABdC4ABUQuAAH0LgADxC5ABEAFvS4AAzQMDEBFQcBMwEnNSEVBwERFxUhNTcRASc1Ab6DAVgJAW6HAaSa/ky4/eKu/lCkBcMpNP1YAqY2KSlI/OP+PEgpKUgBwgMfSCkAAQA9/+UElgXdABMAlrgAMCu6AAoAEwAzK7gAChC4AATQQQMAPwATAAFduAATELgADtC4AAbQuAAKELgAB9C4AAQQuAAQ0LgAExC4ABHQALgAE0VYuAACLxu5AAIAHT5ZuAATRVi4AAwvG7kADAAZPlm4AAIQuQARABb0uAAE0LgADBC5AAcAFvS4AAwQuAAI3LgABxC4AA7QuAACELgAEtwwMRM3FyEXBwEhEzMRBychJzcBIQMjbx6/AzUVef0iAqR/KR6//KQVeQLT/Yt8KQXDGhpbkfueASf+ZBsbWpIEYv7ZAAABAM3+iQJkBnEABwAruAAwK7gAAS+4AAbQALgAAy+4AAAvuAADELkABAAW9LgAABC5AAcAFvQwMQEhESEVIxEzAmT+aQGX6en+iQfocfj6AAH/mv6HA74GAAADAA+4ADArALgAAy+4AAEvMDEBIwEzA76d/Hmd/ocHeQAAAQAf/okBtgZxAAcAMLgAMCu4AAIvQQMALwACAAFduAAF0AC4AAAvuAADL7kABAAW9LgAABC5AAcAFvQwMRMhESE1MxEjHwGX/mnp6QZx+BhxBwYAAQCYBAAD6QXLAAYAMLgAMCu4AAYvuAAC3AC4ABNFWLgAAC8buQAAAB0+WbgABdy4AAPQuAAAELgABNAwMQEzAQcJAScCCm0Bcj3+k/6WPQXL/nI9AUb+uj0AAf/6/wgEGf+DAAMAOLgAMCu4AAQvuAAFL7gABBC4AADQuAAFELgAAdAAuAATRVi4AAQvG7kABAAZPlm4AADcuAAD3DAxByEVIQYEH/vhfXsAAAH/gQSPAOUGFAAEAFq4ADAruAADL0EDAG8AAwABXbgAAdwAuAACL0EDAF8AAgABXUEDAB8AAgABcUEFAD8AAgBPAAIAAnFBAwD/AAIAAV1BAwA/AAIAAV1BAwAfAAIAAV24AADcMDEbAQcBNyHEN/7TDAYU/qInAS8pAAACAFz/4gQfBBkAIAAqAOe4ADArugAlAAoAMytBAwCwACUAAV1BAwCAACUAAV24ACUQuAAF0EEDAG8ACgABcUEFADAACgBAAAoAAl24ACUQuAAQ0LgAJRC4AB/QugAZAAoAHxESObgAGS+4ABbQuAAKELgAKNBBAwAAACwAAXEAuAATRVi4ABwvG7kAHAAfPlm4ABNFWLgABy8buQAHABk+WbgAE0VYuAACLxu5AAIAGT5ZuAAA3LoAEAAcAAcREjm4ABAvugAFABAABxESObgAHBC5ABQAFPS4ABwQuAAX3LgABxC5ACEAFvS4ABAQuQAlABT0MDElFQcGJjUGIyImNTQ+Azc1NCYjIg8BIxE+ATMyFhURBTI2NxEOARUUFgQf2UBYgq+RkEx2pJhUWGZNaGoxWshMpbP+azl4LMjSblglQw45RXmWcVWDTjQUAmCMbhjoAR0gJIWY/Y8vNzABSQmRcFZQAAACAAr/0wQ/BhkAFQAiAKq4ADArugAgAAEAMytBAwAQAAEAAV1BAwAwAAEAAV24AAEQuAAZ0LgAB9BBAwAQACAAAV1BAwAwACAAAV24ACAQuAAO0LoAFQAZAAEREjkAuAATRVi4AAUvG7kABQAhPlm4ABNFWLgACS8buQAJAB8+WbgAE0VYuAATLxu5ABMAGT5ZuAAA0LgAAC+4AAUQuQACABf0uAAJELkAFgAU9LgAExC5ABwAFPQwMQUjESc1JRcRNjMyHgIVFA4CIyInASIHER4BMzI+ATU0JgEAPbkBXBVoqFOdekpVkLdmg4EBCXZRMmM4XYlBpS0FjUgpSBX9ultLh855gNCDRjwDj179WDklfsFzv/MAAQBc/+cDnAQbAB0Ax7gAMCu6AAwABQAzK0EDABAABQABXUEDAC8ABQABXUEDAG8ABQABcUEDAFAABQABXUEDADAABQABXUEDAFAADAABXUEDAJAADAABXUEDAHAADAABXUEDADAADAABXUEDABAADAABXbgADBC4AA/QuAAFELgAFNC4AAwQuAAb0AC4ABNFWLgACi8buQAKAB8+WbgAE0VYuAAALxu5AAAAGT5ZuAAKELgADty4AAoQuQARABT0uAAAELkAFwAW9LgAABC4ABrcMDEFIi4CNTQ+AjMyFxEjAyYjIgYVFBYzMjY3Fw4BAjVeq4JOTomyZ5OOKW88O52Wxq07X0UfWaYZTYnMd4HQhEZC/ssBDA7o1szaHydARjMAAAIAXP/nBJEGGQAYACMA1rgAMCu6ABkACwAzK0EDAI8AGQABXUEDAEAAGQABXbgAGRC4AATQQQMAjwALAAFdQQMAXwALAAFdQQMAQAALAAFduAAZELgAEtC4ABkQuAAY0LgACxC4AB/QALgAE0VYuAAWLxu5ABYAIT5ZuAATRVi4ABAvG7kAEAAfPlm4ABNFWLgABi8buQAGABk+WbgAE0VYuAACLxu5AAIAGT5ZuAAA3LoABAAGABAREjm6ABIAEAAGERI5uAAWELkAEwAX9LgAEBC5ABwAFPS4AAYQuQAiABb0MDElFQUnNQYjIi4CNTQ+AjMyFxEnNSUXEScRJiMiBhUUFjMyBJH+pBR4o1OadkdRirxpX2a5AV0UuF1TjbeXiH1YJUwVYHVLh816d8qJTSEBakgpSBX6hz4CtjP5ucLsAAIAXP/nA88EGQAcACMA77gAMCu6ACEAEQAzK0EDACAAEQABXUEDAG8AEQABXUEDAG8AEQABcUEDAEAAEQABXUEDAKAAEQABXbgAERC4AAHQQQMA5wABAAFdQQMAQAAhAAFdQQMAwAAhAAFdQQMAbwAhAAFxQQUAkAAhAKAAIQACXUEDACAAIQABXUEDACAAIQABcbgAIRC4ABzQugAJABwAERESObgAARC4ACDQALgAE0VYuAAXLxu5ABcAHz5ZuAATRVi4AAwvG7kADAAZPlm6AAEAFwAMERI5uAABL7gADBC5AAYAFvS4ABcQuQAdABT0uAABELkAIAAU9DAxASEeAzMyNxcOASMiLgI1ND4DMzIeAhUBIgYHITQmA7L9eQE+an9LaHcfVKhrXayCTjlefYNEVZFxQf5sc4kQAd1sAfpwpl4sRjxKM0+LynVvt3pUJT97yIABobaeoLQAAQBIAAADbQYZAB4A3LgAMCu6ABMABwAzK0EDADAABwABXUEDAKAABwABXUEDAFAABwABXbgABxC4AADQuAAHELgACtBBAwAwABMAAV1BAwBQABMAAV1BAwCgABMAAV24ABMQuAAW0LgAABC4ABvQugAdABMABxESOQC4ABNFWLgAHC8buQAcAB8+WbgAE0VYuAAQLxu5ABAAIT5ZuAATRVi4AAQvG7kABAAZPlm4ABwQuQAAABX0uAAEELkABgAW9LgAAdC4AAAQuAAH0LgAHBC4AArQuAAQELgAFdy4ABAQuQAWABT0MDEBEQUVITU3ESc1NzU0PgIzMhYXESMnIg4BHQEhFxUBwwEA/Y+4wsJMf59XH2gbKY5Xby0BGiMDdfz8SCkpSAMANykpZmusajgQC/7P71iNYnsMPAAAAwBm/ecEewQZABMAQgBOAZm4ADArugBJADIAMytBAwAQADIAAV1BAwAwADIAAV24ADIQuAAp0LgAKS+4AALQuAApELgADNy4ADIQuAAt0LgALS+4ABjQuAAMELgAINBBAwAQAEkAAV1BAwAAAEkAAXFBAwAwAEkAAV24AEkQuAA+0LoAOQAyAD4REjm6ADwAPgAyERI5QQMA9QA8AAFdQQMABQA8AAFxQQUAFAA8ACQAPAACcUEDADMAPAABcbgAMhC4AEPQALgAE0VYuAA3Lxu5ADcAHz5ZuAATRVi4ADovG7kAOgAfPlm4ABNFWLgAJi8buQAmABs+WbkABwAU9LgANxC4ABTcQQMAzwAUAAFdugARACYAFBESOUEPAJoAEQCqABEAugARAMoAEQDaABEA6gARAPoAEQAHXUEJAAoAEQAaABEAKgARADoAEQAEcUEHAGkAEQB5ABEAiQARAANdugAWABQANxESOboAGwAUACYREjm6ACsAGwARERI5ugAvADcAFBESObgAOhC5ADwAFvS4ABQQuQBGABT0uAA3ELkATAAU9DAxBQYVFB4CMzI+AjU0LgInLgETIicGFRQeBRUUDgMjIiY1NDcmNTQ3LgE1ND4CMzIXIRUHFhUUDgIBFBYzMjY1NCYjIgYBf2I6Wm4zNWtVNj9sklMjN7VnVj9ViKSkiFVCa5CVTdfyv3ecV106a6hlWEkBkttaO2un/rl4aml4eGlqeCc8gjtPLxQVKT0pP0wtFgkECAFCICpAJSwTECI7d1ZFckkyFpV4rU8obYBVNapfSYltQhkpTmmISYltQgGBiZKTiIiSkgABAD0AAAT+BhkAHQC2uAAwK7oAFgAEADMrQQMAbwAEAAFdQQMAbwAEAAFxQQMAUAAEAAFduAAEELgAHdC4AArQQQMAUAAWAAFduAAWELgAEdAAuAATRVi4AAgvG7kACAAhPlm4ABNFWLgADS8buQANAB8+WbgAE0VYuAACLxu5AAIAGT5ZuQAEABb0uAAIELkABQAX9LoACgANAAIREjm4AAQQuAAd0LgAFtC4ABHQuAACELgAFNC4AA0QuQAaABX0MDElFSE1NxEnNSUXET4BMzIWFREXFSE1NxE0JiMiBxECXP3hubkBXRRGvlyHsbj94a5mfZVnKSkpSATvSClIFf2HQkykov2eSCkpSAIlkHBZ/TQAAAIAUgAAAnsFwwAKABYAzLgAMCu4AAQvQQUAQAAEAFAABAACXbgACtC4ABHQuAARL0EDAI8AEQABXbgAC9xBBQBvAAsAfwALAAJdQQMAXgALAAFdQQMAQAALAAFxQQMAEAAYAAFxALgAFC+4ABNFWLgACC8buQAIAB8+WbgAE0VYuAACLxu5AAIAGT5ZuQAEABb0uAAIELkABQAX9LgABBC4AArQQQMAvwAUAAFdQQMAHwAUAAFdQQMAIAAUAAFxQQMAgAAUAAFduAAUELgADtxBAwBPAA4AAXEwMSUVITU3ESc1JRcRAzQ2MzIWFRQGIyImAnv917i4AVwV/Ek2NElJNDZJKSkpSALvSClIFfxtBNU0SUk0NklJAAIAJf3nAbwFwwAUAB8Ay7gAMCu4ABMvQQMAUAATAAFduAAE0LgAG9C4ABsvQQcAEAAbACAAGwAwABsAA3G4ABXcQQUAbwAVAH8AFQACXUEDAF4AFQABXUEDAEAAFQABcUEDAJ8AIQABXUEDAB8AIQABXQC4AB0vuAATRVi4AAIvG7kAAgAfPlm4ABNFWLgACi8buQAKABs+WbgAAhC5ABQAF/RBAwAfAB0AAV1BAwC/AB0AAV1BAwAgAB0AAXFBAwCAAB0AAV24AB0QuAAY3EEDAE8AGAABcTAxEzUlFxEUDgMHJz4GNREDNDYzMhYVFAYiJjcBXBUTMVaFXgYoPisdEggDL0k1NElIakkDqClIFfw+bZ2LYEwaQhY3OlNOdWdOAuUB5jRJSTQ2SUkAAQA9AAAEngYZABoA4bgAMCu4AAQvQQMAzwAEAAFdQQMAUAAEAAFduAAa0LgACtC4AAQQuAAT0EEDAMUAEwABXUEDAGUAEwABXUEDAGMAEwABcbgAEdC4AAzQugASAAoAERESObgAExC4ABfQALgAE0VYuAAILxu5AAgAIT5ZuAATRVi4AA8vG7kADwAfPlm4ABNFWLgAAi8buQACABk+WbkABAAW9LgACBC5AAUAF/S6ABkADwACERI5uAAZL7gAC9y4AA8QuQARABb0uAAM0LgACxC4ABLQuAAEELgAGtC4ABPQuAACELgAF9AwMSUVITU3ESc1JRcRNwEnNSEVBwkBNRcVIQEjEQJS/eu5uQFdFFoBK4MBtKz+2wFdrv7P/olIKSkpSATvSClIFfwrFQFgMykpRP6o/jQCSCkB4/6OAAEAPQAAAmYGGQAKAGS4ADAruAAEL0EDAM8ABAABXUEDAFAABAABXbgACtBBAwAQAAwAAXEAuAATRVi4AAgvG7kACAAhPlm4ABNFWLgAAi8buQACABk+WbkABAAW9LgACBC5AAUAF/S4AAQQuAAK0DAxJRUhNTcRJzUlFxECZv3XubkBXRQpKSlIBO9IKUgV+m0AAQBIAAAHdwQZADMBG7gAMCu6ACwABAAzK0EDADAABAABXUEDAKAABAABXUEDAFAABAABXbgABBC4ADPQuAAK0EEDADAALAABXUEDAFAALAABXUEDAKAALAABXbgALBC4ACfQugAQACcALBESObgALBC4AB7cuAAZ0AC4ABNFWLgACC8buQAIAB8+WbgAE0VYuAANLxu5AA0AHz5ZuAATRVi4ABMvG7kAEwAfPlm4ABNFWLgAAi8buQACABk+WbkABAAW9LgACBC5AAUAF/S6AAoADQACERI5uAACELgAKtC6ABAAEwAqERI5uAAEELgALNC4AB7QuAAZ0LgAKhC4ABzQuAATELkAIgAV9LgALBC4ACfQuAANELkAMAAV9LgABBC4ADPQMDElFSE1NxEnNSUXFT4BMzIWFz4BMzIeAhURFxUhNTcRNCYjIgcWFREXFSE1NxE0JiMiBxECZv3iuLgBXBRFtVxakiVFx2Y8alUyuf3hrmN2lWcKrv3srmN2jmQpKSlIAu9IKUgVd0JKVVVPWyZNgFP9nkgpKUgCJY9xYTIw/Z5IKSlIAiWPcVn9NAABAEgAAAUIBBkAHQDDuAAwK7oAFgAEADMrQQMAUAAEAAFdQQMAQAAEAAFxuAAEELgAHdC4AArQQQUAUAAWAGAAFgACXUEDAEAAFgABcUEDAGAAFgABcbgAFhC4ABHQALgAE0VYuAAILxu5AAgAHz5ZuAATRVi4AA0vG7kADQAfPlm4ABNFWLgAAi8buQACABk+WbkABAAW9LgACBC5AAUAF/S6AAoADQACERI5uAAEELgAHdC4ABbQuAAR0LgAAhC4ABTQuAANELkAGgAV9DAxJRUhNTcRJzUlFxU+ATMyFhURFxUhNTcRNCYjIgcRAmb94ri4AVwUR75chrG4/eGvZ32VZykpKUgC70gpSBV5Qkykov2eSCkpSAIlkHBZ/TQAAAIAXP/nBC8EGQATACYA3LgAMCu6ACAADwAzK0EDADAAIAABXUEFAJAAIACgACAAAl1BAwAPACAAAXFBAwDAACAAAV1BAwBQACAAAV1BAwAQACAAAV1BAwDgACAAAV24ACAQuAAF0EEDAJgABQABXUEDADAADwABXUEDAC8ADwABXUEDAG8ADwABcUEDAA8ADwABcUEDAFAADwABXUEDABAADwABXUEDAKAADwABXbgADxC4ABbQALgAE0VYuAAALxu5AAAAHz5ZuAATRVi4AAovG7kACgAZPlm5ABsAFPS4AAAQuQAlABT0MDEBMh4CFRQOAiMiLgI1ND4CBwYVFB4CMzI+AjU0LgIjIgJGcrZ+Q0J+tnN0tn5CQ362Z00nS25HR3BNKSlNcEeOBBlOjcZ4d8aOTk6OxXh4xo1O1H3IY6V3QkJ3pWNipXdDAAACAAr+AAQ/BBkAGgAqAM+4ADArugAoAAQAMytBAwAQAAQAAV1BAwAwAAQAAV24AAQQuAAa0LgAH9C4AArQQQMAMAAoAAFdQQMAEAAoAAFduAAoELgAEtAAuAATRVi4AAgvG7kACAAfPlm4ABNFWLgADS8buQANAB8+WbgAE0VYuAAXLxu5ABcAGT5ZuAATRVi4AAIvG7kAAgAbPlm5AAQAFvS4AAgQuQAFABf0ugAKAA0AFxESOboAGQAXAA0REjm4AAQQuAAa0LgADRC5ABsAFPS4ABcQuQAiABT0MDEBFSE1NxEnNSUXFT4BMzIeAhUUDgIjIicREyIGBxEeATMyPgM1NCYCXP2uubkBXBU4gFZTnXtLVI2qXndk4z96Kh13RytOSjYgkP4pKSlIBO9IKUgVZkU2S4fOeYnUfT8r/l8FQTo0/YU5QhxCYpVdwvAAAAIAXP4ABJEENwAYACUAyrgAMCu6ABsACwAzK0EDAEAAGwABXbgAGxC4AAPQQQMAXwALAAFdQQMAjwALAAFdQQMAQAALAAFduAAbELgAFdC6ABMAGwAVERI5uAALELgAI9AAuAATRVi4ABEvG7kAEQAfPlm4ABNFWLgABi8buQAGABk+WbgAE0VYuAAALxu5AAAAGz5ZuQACABb0ugADAAYAERESOboAEwARAAYREjm4ABEQuAAU0LgAFC+4AAIQuAAW0LgABhC5ABkAFvS4ABEQuQAfABT0MDEBNTcRDgEjIi4CNTQ+AzMyFzczERcVATI3ES4BIyIOARUUFgIr9juWSlOadkc4YYKXUIR6QD24/bt4XTJjOF6JQJf+AClIAek5OkuHzXpnsH1aKzpY+jpIKQJSdwKLOSV+wXPC7AAAAQBIAAADSAQZABMAtrgAMCu6AAwABAAzK0EDAM8ABAABXUEDAEAABAABXUEDALAABAABXbgABBC4ABPQuAAK0EEDAEAADAABXUEDALAADAABXUEDAGAADAABcbgADBC4AA/QALgAE0VYuAAILxu5AAgAHz5ZuAATRVi4AAwvG7kADAAfPlm4ABNFWLgAAi8buQACABk+WbkABAAW9LgACBC5AAUAF/S6AAoADAACERI5uAAMELgAD9C4AAQQuAAT0DAxJRUhNTcRJzUlFxU2MxUHJyIGBxECpP2kuLgBXBSY+CmSLXQ0KSkpSALvSClIFXuQ3g5QJyX9QAAAAQB7/+MDVAQZADsA8bgAMCu6AAsAFAAzK0EDADAAFAABXUEDAO8AFAABXUEDAC8AFAABXUEDABAAFAABXUEDAKAAFAABXbgAFBC4ADrQuAA6L7gAAdBBAwAQAAsAAV1BAwAgAAsAAXFBAwAwAAsAAV1BAwCgAAsAAV24AAsQuAAw0LoAGwAwABQREjm4ABsvuAAe0LgAFBC4ACbQALgAE0VYuAAZLxu5ABkAHz5ZuAATRVi4ADUvG7kANQAZPlm4AADcQQMAEAAAAAFduAA1ELkABgAU9LoADwA1ABkREjm4ABkQuAAd3LgAGRC5ACEAFPS6ACsAGQA1ERI5MDETFx4DMzI+AjU0Ji8BLgM1ND4CMzIXAyMnLgEjIg4CFRQeAhceARcWFRQOAiMiLgInEa5rDSYsLhYzUzwgTUZrOWNKKzZkjVaEhAQpaiY3EjZPMxkZMUoxRlsUnUdzkEkmWFhRHwFS4wgNCgYVJzkkL1UoPB5ATFw5RWZDISv+3+sFBhkpNBwhNTAvGiYwDmmJSm9LJQoSGQ8BKwABAB3/5wL2BQ4AGQCIuAAwK7gADy9BAwBvAA8AAV1BAwDvAA8AAV24AAHQuAAPELgAE9C4AAEQuAAW0EEDANAAGwABXQC4ABNFWLgAFi8buQAWAB8+WbgAE0VYuAAMLxu5AAwAGT5ZuAAWELkAAAAV9LgADBC5AAYAFfS4AAAQuAAQ0LgAFhC4ABPQuAAWELgAFdwwMQERFB4CMzI2NxcGIyImNREnNTcTMxEhFxUBjxUpLiA8US8fjahiiLrCSmYBJSMDdf3TRFwtERUdQHWbjAJnMykpART+7Aw8AAABAB//5wTLBBkAHQDRuAAwK7oACgAdADMrQQMADwAdAAFxQQMA7wAdAAFduAAdELgAA9BBAwDvAAoAAV1BAwA/AAoAAV1BAwCgAAoAAV24AAoQuAAO0LgAChC4ABTQQQMAwAAfAAFdALgAE0VYuAACLxu5AAIAHz5ZuAATRVi4AA0vG7kADQAfPlm4ABNFWLgAFy8buQAXABk+WbgAE0VYuAASLxu5ABIAGT5ZuAAXELkABwAV9LgADRC5AAoAF/S4ABIQuAAQ3LoAFAANABcREjm4AAIQuQAdABf0MDETNSUXERQWMzI3ESc1JRcRFxUFJzUOASMiLgI1ER8BXBRkdYpouAFcFLn+pBVFtls8alUyA6gpSBX9Zo9xVwKfSClIFfyHMyVMFXdCSiZNgFMCMwAB/+H/5wRcBAAADwBguAAwK0EDAEYABgABXQC4ABNFWLgACC8buQAIAB8+WbgAE0VYuAAFLxu5AAUAGT5ZuAAIELkABgAW9LgAAtC4AAYQuAAL0LgABRC4AAzQuAACELgADdC4AAgQuAAP0DAxARUHNQEHASc1IRUHARMnNQRcj/6bbv51jgHqnAEh+JQEAClIB/xoFwOoSCkpPf1NArU7KQAB/+H/5wYzBAAAFABwuAAwK0EDAD8AFgABXQC4ABNFWLgABi8buQAGAB8+WbgAE0VYuAADLxu5AAMAGT5ZuAAA0LgABhC4AAvQuAAB0LgABhC5AAQAFvS4AAnQuAADELgACtC4AA3QuAAEELgAE9C4AA7QuAALELgAENAwMQUBAwcBJzUhFQcTATMBEyc1IRUHAwQ//tj8b/64gwHqptsBBm0BGLSnAZuH/hkDB/0QFwOvQSkpQf1eAwz9CAKOQSkpRPxrAAAB/98AAARiBAAAHACduAAwKwC4ABNFWLgAFy8buQAXAB8+WbgAE0VYuAAQLxu5ABAAGT5ZuAAXELgAAdC4ABcQuQAVABb0uAAE0LoADAAQABcREjm6ABsAFwAQERI5ugAFAAwAGxESObgAEBC5ABIAFvS4AAbQuAAQELgACdC4AAYQuAAL0LgAEhC4AA3QugATABsADBESObgAFRC4ABrQuAAEELgAHNAwMQE1IRUHCQEXFSE1NwsBFxUhNTcBMwEnNSEVBxsBAmgBpJf+/AE9tP3yj+/ymP5FtAEjAv7msAH+f8bLA9cpKTv+l/48RikpLQFW/qwvKSlGAZMBlEEpKS/+4wEbAAAB/9f95wSHBAAAEgBXuAAwKwC4ABAvuAATRVi4AAEvG7kAAQAfPlm4ABNFWLgADS8buQANABs+WbgAARC5ABEAFvS4AATQuAAQELgABdC4ABEQuAAL0LgABtC4AAEQuAAI0DAxAzUhFQcBEyc1IRUHAQMvAQkBFSkCEpsBJ/GTAbS4/rjhqhMBPv5oA9cpKT39YgKiOSkpSPxx/ecfKQH0A3MHAAEAXAAAA3sEAAANAOq4ADArugAJAAgAMytBAwBjAAkAAV1BAwD0AAkAAV1BAwAkAAkAAXFBBQCkAAkAtAAJAAJdQQUAMAAJAEAACQACXUEDAIAACQABXbgACRC4AAHQQQMALwAIAAFdQQMAawAIAAFxQQMAMAAIAAFduAAIELgAAtC4AAEQuAAG0LgABi+4AAPQuAAIELgADdC4AA0vuAAK0AC4ABNFWLgADS8buQANAB8+WbgAE0VYuAAGLxu5AAYAGT5ZuAANELkACgAU9LgAAdC4AAYQuQADABT0uAAGELgABdy4AAMQuAAI0LgADRC4AAvcMDEBFQEhNzMRITUBIQcjEQN7/aoBrnUp/OsCVv5SdSkEAGb8zPb+pGYDNPYBXAABAJr+iQMABk4AJQBDuAAwK7gABS+4AA/QALgACS+4AB4vugABAAkAHhESObgAAS+4AADcuAAJELgACty6ABUAAQAAERI5uAAeELgAHdwwMRM1MjY9ATQ2OwEVDgMdARQOAgcWERUUHgIXFSMiJjURNCaaXma1zh9QXDcRFChGLrARN1xQH861YwI/Ttyr8K+bTgQYO1RJxFibhFsQPP6E40lUOxgETpyuAQ6muAABALj+BgE7Bh8AAwAguAAwK7gAAS9BAwAQAAEAAV24AADcALgAAy+4AAEvMDEBBxEzATuDg/4IAggZAAABAEj+iQKuBk4AJQBMuAAwK7gAIi9BAwAvACIAAV24ABjQALgAHi+4AAkvugAAAB4ACRESObgAAC+4AAHcuAAJELgACty6ABIAAAABERI5uAAeELgAHdwwMQEVIgYVERQGKwE1PgM9ARA3LgM9ATQuAic1MzIWHQEUFgKuYWO1zh9QXDcRsC5GKBQRN1xQH861ZgKNTrim/vKunE4EGDtUSeMBfDwQW4SbWMRJVDsYBE6br/Cr3AAAAQCaAaoF5QMlAB4ARLgAMCu4ABMvQQMAEAATAAFduAAE3AC4AAovuAAA3LgAChC4ABncuAAD0LgAAy+4ABkQuAAP3LgAChC4ABLQuAASLzAxATI2NxcOBCMiLgIjIgYHJz4EMzIeAwRaQdNVIgkhZWWPQUKXcYczSqF4IAoiaGWHODdvXF5wAlBaQiUMJmJLPkNPQ0ZJJwskXUg6LD8+LAD//wAAAAAAAAAAAgYABAAAAAIAmf9qAZUFhQAKAA4AXLgAMCu4AAMvQQMAEAADAAFduAAI3EEHAFAACABgAAgAcAAIAANdugAOAAMACBESObgADi9BBQBfAA4AbwAOAAJxuAAN0AC4AAYvuAAOL7gABhC4AADcuAAL3DAxASImNTQ2MzIWFAYHMxMjARY0SUk0NklJdXMiuASJSTQ1SkpqSHH7UgACAHH/zQOwBRQAGwAhANe4ADArugAMAAQAMytBAwBPAAQAAXFBAwAvAAQAAV1BAwBPAAQAAV1BAwAQAAQAAV1BAwAvAAwAAV26AAEABAAMERI5uAABL7gAH9C4AAfQuAABELgAGtBBBQAYABoAKAAaAAJduAAS0LgACtC4AAwQuAAP0LgADBC4ABjQuAAEELgAHNAAuAAaL7gAAdC4ABoQuAAK3LgAB9C4AAoQuAAJ3LgAChC4AA7cuAAKELkAEQAU9LgAGhC5ABQAFvS4ABoQuAAb3LgAFBC4AB/QuAARELgAINAwMQU1JgI1NBI3NTMVFhcRIwMmJxEWMzI2NxcGBxUBFBYXEQYCKbn/98FcgnopbzA0Dh87X0UfiqH+untv6jOVDwEo4u4BFxGDgQY5/ssBDAsD/J4CHihAbQqXAq6eziYDSDUAAQBS/+EEpAXJAEABObgAMCu6AB8AGAAzK7gAHxC4AAHQQQMAfwAYAAFduAAYELgAD9C4ABgQuAAT0LoAFgATABgREjm4AB8QuAAi0LgAGBC4ACnQuAAz0LoALwApADMREjm6ADEAHwAzERI5ALgAE0VYuAAdLxu5AB0AHT5ZuAATRVi4AAYvG7kABgAZPlm4ABNFWLgADi8buQAOABk+WbgABhC4AADcuAAGELgAOtxBAwBgADoAAXFBCwCQADoAoAA6ALAAOgDAADoA0AA6AAVduAAK3EEHAM8ACgDfAAoA7wAKAANduAAOELgAD9y6AC8AHQAGERI5uAAvL7kAMwAV9LgAE9C4AC8QuAAW0LgAHRC4ACHcuAAdELkAJAAW9LoANwA6AA8REjm4AAYQuAA+3EEHAMAAPgDQAD4A4AA+AANdMDElFw4DIyIuASMiDgEHJzY1NC8BNTcmNTQ+AjMyFxEjJyYjIg4CFRQeAxclFwclFhUUBxU2MzIeATMyNgRkQB9GS0csOqyxQC2Acw8p2xl+bCFFe5xduZYoik41N11UMAQJBhACAV4nFf6gDCcxLT+zoSolO+cOTGYzE05OO0UFRWz9dowEZgW7fmKTWSsp/sTqDBMsVDogQlgwdBAQDoUIgD2CWwQTTk4tAAIAcQHHBAAFUgAbACcAQLgAMCu4AAQvuAAS3LgABBC4AB/cuAASELgAJdwAuAALL7gAGdxBAwAvABkAAV24AAsQuAAc3LgAGRC4ACLcMDETJzcmNTQ3JzcXNjMyFzcXBxYVFAcXBycGIyInEyIGFRQWMzI2NTQmuEemPj6mS6Jee4JZokymQD6kSKhae4Fa12N8fWZlfX4Bx0umWH94W6RMokZGokyiWXx9XKRLoUFDAg2CZmiDg2hngQAAAf/NAAAFJwXDACUAsLgAMCu4ABovuAAT0AC4ABNFWLgAJS8buQAlAB0+WbgAE0VYuAAXLxu5ABcAGT5ZuAAlELkAIwAW9LgAAtC6ACEAJQAXERI5uAAhL7gAA9C4ACMQuAAK0LgABdC4ACUQuAAH0LgAIRC4AAvQuAAhELkAIAAV9LgADtC4ACEQuAAc3EEFAFAAHABgABwAAl24ABDQuAAcELkAGwAV9LgAE9C4ABcQuQAZABb0uAAU0DAxARUHATMBJzUhFQcBMxUhBxUhFSERFxUhNTcRITUhNSchNTMBJzUBwYQBWwIBZIcBsJr+lN3+3wQBJf7buP3irv7lARsG/uvR/pqkBcMpNP1UAqo2KSlI/Wh7Ckd7/v5IKSlIAQJ7RQx7AphIKQAAAgC4/qABOwWPAAMABwAsuAAwK7gABC9BAwAQAAQAAV24AAXcuAAA0LgABBC4AAHQALgAAy+4AAYvMDEBIxEzAzMRBwE7g4ODg4MCpALr/Af9DAIAAgDb/tkEAgYCADYAQgEBuAAwK7oAQAAPADMrQQMAEABAAAFduABAELgAJ9C6ADQADwAnERI5uAA0L7gAANC6AC0AJwAPERI5uAAtL7gABdC6ABQADwAnERI5uAAUL7oAEQAUAEAREjm6ABkAJwAPERI5uAAZL0EDAB8AGQABXbgAHNC4ABQQuAAh0LgADxC4ADrQugAqAC0AOhESOboAOABAABQREjm6AD4AOgAtERI5ALgAFy+4ADIvuAAC3LoAPgAyABcREjm4AD4QuAAH0LoAOAAXADIREjm4ADgQuAAj0LoAEQAjADgREjm4ABcQuAAb3LgAFxC4AB7cugAqAD4ABxESObgAMhC4ADbcMDEFFhcWNjU0JicuBTU0Ny4BNTQ2MzIXESMnJiMiBhUUFhceARUUBgceARUUDgIjIicRMwEnBhUUFh8BNjU0JgF/fVplbGfALTBRLzEX21hgzLGEnTIxbmNacGedy5Vka0tFMF2VX4rCLwEfMKNtiG99i5geAgNZVkpfbRoeODNGUy/dfziBWYGeKf7owCtVUThjX3mhc2eiZjdxTUJyWjQ7AUwDIRlklE9wUEOPeVeLAAIAbQTJAqgFrgATACcAergAMCu4AAAvQQUAIAAAADAAAAACXbgACtxBBQBgAAoAcAAKAAJduAAAELgAFNy4AB7cQQUAYAAeAHAAHgACXQC4AA8vQQMAvwAPAAFdQQMAHwAPAAFdQQMAgAAPAAFdQQMAIAAPAAFxuAAF3LgAGdC4AA8QuAAj0DAxEzQ+AjMyHgIVFA4CIyIuAiU0PgIzMh4CFRQOAiMiLgJtEh8rGBgqHhERHioYGCsfEgFWEh4qGBgqHxISHyoYGCofEQU9GCkeEhIeKRgYKx8SEh8rGBcpHxISHykXGCsfEhIfKwADAHH/5wYrBb4AFQAoAEUA2rgAMCu4AB8vQQMAEAAfAAFduAAW3EEDAB8AFgABXbgAANC4AB8QuAAM0LoAMAAfABYREjm4ADAvuAA13EEDAHAANQABXbgAKdC4ACkvuAA1ELgAONC4ADAQuAA+0LgAKRC4AEPQALgAE0VYuAAbLxu5ABsAGT5ZuAAk3LgAB9y4ABsQuAAR3EEDAHAAEQABcroAKwAkABsREjm4ACsvQQMAfwArAAFduAAz3EEHABAAMwAgADMAMAAzAANduAA33LgAMxC4ADncuAArELgAQdy4ACsQuABE3DAxATQuBCMiDgIVFB4CMzI+AjcUAgYEIyIkAjU0EjYkMzIEFhIBBiMiLgI1NDYzMhcVIycmDgMVFBYzMj8BMwWwJklphp9bguGnYFmg4YiC36Nce3PC/u2Xyf6vwXTGARSZmgEPvW3+IWRjVpZ4RvO3YV4rSDpnVz8jj4ctKzorAttVoIx0VC5hq+qJgeewZ2Wv7IeV/urNfMgBWMyZARXIdXTG/vD93iMyYaJnzeUf0YoIAy1Lf1Sxkw2XAAIAXAHwA9sF2wAhACoArLgAMCu6ACYACwAzK7gAJhC4AAXQQQMAXwALAAFxQQUALwALAD8ACwACXbgAJhC4ABHQuAAmELgAINC6ABkACwAgERI5uAAZL7gAF9C4AAsQuAAp0AC4ABNFWLgAHS8buQAdAB0+WbgACNy4AAPQuAADL7gAANy6ABEAHQAIERI5uAARL7gAHRC5ABUAFvS4AB0QuAAY3LgACBC5ACIAFfS4ABEQuQAmABT0MDEBFQcGJj0BBiMiJjU0PgM3NTQmIyIPASMRPgEzMhYVEQUyNjcRDgEVFAPbyz1ieZCGhkNokotNP19KXlI9UbpJmab+fTRpKLGsAl4jPww4QAJ0imlNeUkyFQNYglEXwwEJHyB7jf26HCwoASQKfmWLAAIAcQB1A4EELwAGAA0AO7gAMCu4AAQvQQMAEAAEAAFdQQUAMAAEAEAABAACXbgAANC4AAQQuAAL3LgAB9AAGbgAAC8YuAAH0DAxCQEHATUBFxsBBwE1ARcBTAErIf4bAeUhN9Mh/owBdCECUv49GgHGLQHHG/4+/n8bAYcpAYgbAAABAGYBOQSPArYABQBGuAAwK7gAAC+4AAHcQQUAEAABACAAAQACcbgAABC4AAPcALgAAy+4AAHcuAADELgABNxBAwDwAAQAAV1BAwAAAAQAAXEwMQEjNSE1IQSPcvxJBCkBOfSJAAABAGYCKQMzAqQAAwAiuAAwK7gAAy9BAwAPAAMAAXG4AALcALgAAy+5AAAAFfQwMRMhFSFmAs39MwKkewAEAHH/5wYrBb4AFQAoAD8ASAD6uAAwK7gAHy9BAwAQAB8AAV24ABbcQQMAHwAWAAFduAAA0LgAHxC4AAzQugAyAB8AFhESObgAMi+4AEbcuAA50LoAKgAyADkREjm4ADIQuAAt0LgAKhC4ADzQuAAtELgAQtAAuAATRVi4ABsvG7kAGwAZPlm4ACTcuAAH3LgAGxC4ABHcQQMAcAARAAFyugAwACQAGxESObgAMC+4ACnQuAAwELgANtxBBQAQADYAIAA2AAJdugAsADYAMBESObgALC+4ADAQuAAy3LgALdC4ADYQuAAz3LoAPAAsADYREjm4ADIQuAA90LgANhC4AEDcuAAsELgAQtwwMQE0LgQjIg4CFRQeAjMyPgI3FAIGBCMiJAI1NBI2JDMyBBYSAQMGJxEXFSE1NxEnNSEyFhUUBgcTFxUBIxEWPgE1NCYFsCZJaYafW4Lhp2BZoOGIgt+jXHtzwv7tl8n+r8F0xgEUmZoBD71t/fK7OUx5/qhkZAFIjplNSaZe/nFvaHgvTALbVaCMdFQuYavqiYHnsGdlr+yHlf7qzXzIAVjMmQEVyHV0xv7w/dQBTgcL/vUpHh4pApIpHnlkSHMg/tkjHgLM/toQH0U9VEEAAAH/+gUdBBkFmAADABm4ADAruAADL7gAAtwAuAADL7kAAAAV9DAxAyEVIQYEH/vhBZh7AAACAE4DmgKwBfwAEgAmAE64ADAruAACL7gADNy4ABjQuAACELgAItAAuAATRVi4AAcvG7kABwAdPlm4ABHcuAAT3EEDADAAEwABcbgABxC4AB3cQQMAPwAdAAFxMDETJjU0PgIzMh4CFRQOAiMiNzI+AjU0LgIjIg4CFRQeAqhaMVNvPj5vUzExU28+fX0oRTMeHjRFJylFMx0cM0YD9Fp9Pm9TMTFTbz4+b1Mxch0zRiknRTQeHjNGJypGMxwAAgBmAD0EjwTLAAsADwBtuAAwK7gABi+4AAPcuAAA0LgAAxC4AALcuAAGELgAB9y4AAYQuAAJ0LgABxC4AAzQuAACELgADdAAuAAPL7gAB9y4AAjcQQMAAAAIAAFxuAAA0LgABxC4AAPQuAAPELgADNxBAwAAAAwAAXEwMQEhFSERIxEhNSERMwEhFSECwwHM/jSM/i8B0Yz9owQp+9cDSoP+qAFYgwGB+/aEAAEATgKJAo8F7wAZAEi4ADAruAATL7gAB9AAuAATRVi4AA8vG7kADwAdPlm4AAHcuAAA3LgAARC5ABgAFfS4AAPQuAAPELkACgAU9LgADxC4AAvcMDEBByE1AT4BNTQmDwEjNT4BHgIVFAYPASE3Ao8C/cEBEEswjnxCNUiSkW9FU2zGAUMtA4f+bwEnTmA9VkAZruwWFAotY0lHiWnDdwABAHECcwKRBfAAJgDCuAAwK7gAFi+4ACPQALgAE0VYuAANLxu5AA0AHT5ZuAAZ3EEDAH8AGQABXUEDAO8AGQABXUEDAF8AGQABcUEDAD8AGQABcboAAAANABkREjm4AAAvQQUAHwAAAC8AAAACXUEHAF8AAABvAAAAfwAAAANdQQMAbwAAAAFxQQUArwAAAL8AAAACXbgADRC5AAYAFPS4AA0QuAAJ3LgAABC5ACYAFPS6ABMAAAAmERI5uAAZELgAHdy4ABkQuQAgABT0MDEBMjY1NCYjIg8BIzU2MzIWFRQGBxUWFRQGIyInNTMXFjMyNjU0JicBJ1JqTEk9MDk1l2N2jFlHwrV4fnUnOz5PSFhdfARgZEZJPgqu7Ctza0J7EQcax2x9L+udG0ZJV0UGAAABAHkEjwHdBhQABABeuAAwK7gAAC9BBQAgAAAAMAAAAAJduAAD3AC4AAQvQQMAXwAEAAFdQQMAHwAEAAFxQQUAPwAEAE8ABAACcUEDAP8ABAABXUEDAD8ABAABXUEDAB8ABAABXbgAAdwwMRsBHwEBecSUDP7TBLYBXigu/tEAAAEAGf4ABNkEAAAcANS4ADArugAaABAAMyu4ABoQuAAE0EEDAA8AEAABcUEDAJ8AEAABXUEDAG8AEAABXbgAEBC4ABPQugAJABAAExESObgAC9C4ABAQuAAO0LgAGhC4ABvQALgAE0VYuAATLxu5ABMAHz5ZuAATRVi4AAIvG7kAAgAZPlm4ABNFWLgADS8buQANABs+WbgAE0VYuAAHLxu5AAcAGT5ZuAACELgAANy4ABMQuAAa0LoABAAHABoREjm6AAkAEwAHERI5uAATELkAEAAW9LgABxC5ABcAFfQwMSUVBSc1DgEjIicWFw8BJwMRJzUhERQWMzI3ETMRBNn+pBVHvF1xVSaEFcAfJ7gBcGd9kmm5WClIFXlDSzz13CkpKQMEAmJIKf1qkHBZAz38oAAAAQBx/1oDsAVeAA8AVrgAMCu4AAgvuAAE3EEDAD8ABAABXbgAA9C4AAgQuAAH0LgACBC4AAzcALgAAC+4AAcvuAAAELkAAQAV9LgABxC4AATQuAABELgABdC4AAAQuAAJ3DAxARUjESMRIxEjES4BNTQ2MwOwanuSe5K7yJgFXnv6dwWJ+ncDTAjAlJnDAAEAdQF7AXECdwALACC4ADAruAAAL0EDABAAAAABXbgABtwAuAAJL7gAA9wwMRM0NjMyFhUUBiMiJnVJNjRJSTQ2SQH6NElJNDZJSQABAI/+FAI3AAAAEwAsuAAwK7gABS+4ABHQALgACi+4ABNFWLgAAS8buQABABk+WbgAChC4AA/cMDEhMwceARUUDgIjIic1FjMyNTQnASNHFGl4LU9eOExKPR2tklIEak48WTIXE0wJhnoXAAABAEgCiQJ/BdsACgBjuAAwK7gAAy+4AAnQALgAE0VYuAAHLxu5AAcAHT5ZuAAA3EEDAKAAAAABXUEDABAAAAABcUEDAMAAAAABXUEDAFAAAAABXUEDAGAAAAABcbkACQAW9LgAA9C4AAcQuAAE0DAxASE1NxEHJyUzERcCf/3fz8kcASVctgKJTikCNzlUif0lKQAAAgBcAfYDwQXbABMAJwBouAAwK7oAHgAAADMrQQMAHwAAAAFdQQMAXwAAAAFxQQMAPwAAAAFdQQMAwAAeAAFduAAeELgACtC4AAAQuAAU0AC4ABNFWLgABS8buQAFAB0+WbgAD9y5ABkAFPS4AAUQuQAjABT0MDETND4CMzIeAhUUDgIjIi4CNxQeAjMyPgI1NC4CIyIOAlw7cKFmZqFxOztxoWZmom87tCFBXj49YUMjI0NhPT1eQSID53C4hEhIhLhwb7iCSEiCuG9blms7O2uWW1yXbDs6bJcAAAIASAB1A1gELwAGAA0ALrgAMCu4AAQvQQMAEAAEAAFduAAB0LgABBC4AAvcuAAI0AAZuAAILxi4AAHQMDElCQE3ARUBJRMDNwEVAQFSASv+1SEB5f4b/tXT0yABdf6LjwHDAcIb/jkt/jpcAYEBgRv+eCn+ef//AIX/0Qb4BdsAJgB8PQAAJwDVAqwAAAEHANYEM/14AGq4ADArQQcAbwADAH8AAwCPAAMAA11BAwAVAAwAAV1BAwAQABYAAV1BAwAQABoAAV0AuAATRVi4AAgvG7kACAAdPlm4ABNFWLgADi8buQAOAB0+WbgAE0VYuAAULxu5ABQAGT5ZuAAa3DAx//8Ahf/RBykF2wAnAHUEmv14ACYAfD0AAQcA1QKsAAAAb7gAMCtBAwBwABMAAV1BAwAvAB0AAV1BBQBvAB0AfwAdAAJdQQUAoAAdALAAHQACXUEDACoAJgABXQC4ABNFWLgAIi8buQAiAB0+WbgAE0VYuAAnLxu5ACcAHT5ZuAATRVi4AAEvG7kAAQAZPlkwMQD//wCF/9EGpgXwACcA1gPh/XgAJwDVAloAAAEGAHYUAABHuAAwK0EDAG8ALQABXQC4ABNFWLgAHy8buQAfAB0+WbgAE0VYuAAQLxu5ABAAHT5ZuAATRVi4AAUvG7kABQAZPlm4AAvcMDEAAAIAPf8MA4UFEwAWACEAmbgAMCu6ABIAAAAzK7oABQASAAAREjm4AAUvuAAE0LgAABC4AArQuAASELgAD9C4AAUQuAAa0LgAGi9BBQBgABoAcAAaAAJduAAf3EEHAF8AHwBvAB8AfwAfAANdALgAFy+4ABQvuAAXELgAHdy4AATcugACAAQAFBESOboABwAUAAQREjm4ABQQuQANABb0uAAUELgAENwwMTc2PwE1MxUHDgEVFBY3MjcTMxEGIyIkATIWFRQGIyImNDY9A8biZsU7Obx7S0ZrPae45f7/Ads0SUk0NklJdPSnyL3000eeVZONCxQBKf6OPb4FSUk0NUpKakj////8AAAFywdWAiYAJQAAAQcA2QGxAAAAGLgAMCtBAwAQABcAAV1BAwBAABcAAV0wMf////wAAAXLB1YCJgAlAAABBwDaAjwAAAAluAAwK0EDAP8AGAABXUEFAF8AGABvABgAAnFBAwAQABgAAV0wMQD////8AAAFywdvAiYAJQAAAQcA2wFyAAAAN7gAMCtBBQBfAB0AbwAdAAJxQQMAvwAdAAFdQQMA/wAdAAFdQQMADwAdAAFxQQMAEAAdAAFdMDEA/////AAABcsHIgImACUAAAEHAN0BWQAAABy4ADArQQMAEAAmAAFdQQUA0AAmAOAAJgACXTAx/////AAABcsHTAImACUAAAEHANwBVwAAADK4ADAruAAUL0EDABAAFAABXUEDAG8AFAABcUEDAIAAFAABXUEDADAAFAABXbgAKNAwMf////wAAAXLB6ACJgAlAAABBwDIAd4BXgBsuAAwK7gAFC9BAwBPABQAAV1BAwAvABQAAV1BAwBvABQAAV1BAwCgABQAAV1BAwBAABQAAXG4ADfQALgAIy9BAwBvACMAAV1BAwBPACMAAV1BAwBwACMAAV1BBQAwACMAQAAjAAJxuAAo0DAxAAIAH//lB1oF3QAcAB8A9bgAMCu6ABoAAAAzK0EDAGAAGgABcbgAGhC4AAzQuAAML7gAD9C4AAAQuAAW0LgAEdC6ABMADAARERI5uAAaELgAF9C4AAAQuAAf0AC4ABNFWLgACS8buQAJAB0+WbgAE0VYuAAGLxu5AAYAGT5ZuAATRVi4AAAvG7kAAAAZPlm6AB8AAAAJERI5uAAfL7kAAQAW9LgABhC5AAgAFvS4AAPQuAAJELgADty4AAkQuQAQABb0ugARAAkAABESObgAES9BAwBfABEAAV1BAwA/ABEAAV25ABUAFvS4AAAQuQAWABb0uAAAELgAGNy4AAkQuAAd0DAxIREhAxcVITU3ASE3FxEjAyERJRcHJREhEzMRBycJASED8v471dv97K4C5wK0vx8pf/4WAaAnFf5OAf5/KR++/XX+dAGMAeX+ezcpKUgFUhoa/m4BJ/3NFA6FEP20ASf+bxsbBSH9LwABAHv+FATnBeMANwD3uAAwK7oAAAAcADMrQQMAsAAAAAFdQQMAEAAAAAFdQQMAYAAAAAFxQQMAYAAAAAFdQQMAQAAAAAFdQQMALwAcAAFdQQMAEAAcAAFdugAGABwAABESObgABi+4ABXQuAAC0LgABhC4AA3cuAAGELgAEtC4AAAQuAAj0LgAIy+4ACbQuAAcELgALtC4AAAQuAA10AC4AAsvuAATRVi4ACEvG7kAIQAdPlm4ABNFWLgAAi8buQACABk+WbgACxC4ABDcuAACELgAFdC4ACEQuAAl3LgAIRC5ACgAFvS4AAIQuQAzABb0uAACELgANtxBAwAwADYAAV0wMSUGDwEeARUUDgIjIic1FjMyNTQnNyYnLgM1NBI2JDMyFwMjAyYjIg4DFRQeAjMyNxMzBOeutg5peC1PXjhMSj0drJEZV1FwxZFTetMBGKGqrgIpfVSJUI6DXzhZmstzZltuKSU9ATkEak48WTIXE0wJhnoXaAYTHHis+Za8AS/BZjf+gwEdJSpileSPmPmhVx0BSf//AHEAAASPB1YCJgApAAABBwDZAZcAAAAPuAAwK0EDAJ8AGgABXTAxAP//AHEAAASPB1YCJgApAAABBwDaAdwAAAAcuAAwK0EDAC8AGwABXUEFAG8AGwB/ABsAAl0wMf//AHEAAASPB28CJgApAAABBwDbAUUAAAAPuAAwK0EDAC8AIAABcTAxAP//AHEAAASPB0wCJgApAAABBwDcATMAAAAOuAAwK7gAFy+4ACvQMDH//wBxAAACpAdWAiYALQAAAQcA2QDJAAAAMbgAMCtBAwAQAA8AAV1BCwAgAA8AMAAPAEAADwBQAA8AYAAPAAVxQQMAgAAPAAFdMDEA//8AcQAAAqQHVgImAC0AAAEHANoAiwAAAE24ADArQQMALwAQAAFdQQ0ArwAQAL8AEADPABAA3wAQAO8AEAD/ABAABl1BDwAPABAAHwAQAC8AEAA/ABAATwAQAF8AEABvABAAB3EwMQD//wBxAAADCAdvACYALTEAAQYA21AAAE64ADArQQsAHwAVAC8AFQA/ABUATwAVAF8AFQAFcUEHAC8AFQA/ABUATwAVAANdQQMA3wAVAAFdQQkAYAAVAHAAFQCAABUAkAAVAARdMDH//wBwAAAC0QdMACYALRcAAQYA3A4AABe4ADAruAAML0EDAC8ADAABXbgAINAwMQAAAgBxAAAFzQXDABMAIgDRuAAwK7oAIAANADMrQQMAQAAgAAFdQQMAEAAgAAFdQQMAsAAgAAFduAAgELgABtBBAwAvAA0AAV1BAwAQAA0AAV24AA0QuAAR0LgADRC4ABrQuAAW0LoAFwAWACAREjkAuAATRVi4AAAvG7kAAAAdPlm4ABNFWLgACi8buQAKABk+WbkADQAW9LoAFgAAAAoREjm4ABYvQQMAPwAWAAFduQAZABX0uAAO0LgAFhC4ABHQuAAAELkAEgAW9LgAABC5ABQAFvS4AAoQuQAbABb0MDETITIEFhIVFAIEIyE1NxEjNTMRJwUjESEVIREzMj4CNRAAlgJsnQEEv2u2/rrR/Za43d24Al7kAWv+leplt49V/vIFw1+2/uW02P6xuClIAjV7AjFIQv3Je/3ETpTskwE7AVL//wBIAAAF7AciAiYAMgAAAQcA3QGFAAAAF7gAMCtBBwA/AC0ATwAtAF8ALQADXTAxAP//AHv/1wXLB1YCJgAzAAABBwDZAkQAAAAhuAAwK0EDABAAIwABXUEDAGAAIwABcUEDAPAAIwABXTAxAP//AHv/1wXLB1YCJgAzAAABBwDaAiUAAAAYuAAwK0EDAA8AJAABcUEDAD8AJAABcTAx//8Ae//XBcsHbwImADMAAAEHANsBtgAAADK4ADArQQMAXwApAAFxQQUALwApAD8AKQACXUEFAL8AKQDPACkAAl1BAwCPACkAAV0wMf//AHv/1wXLByICJgAzAAABBwDdAXsAAAAguAAwK0EHAD8AMgBPADIAXwAyAANdQQMADwAyAAFxMDH//wB7/9cFywdMAiYAMwAAAQcA3AGQAAAANrgAMCu4ACAvQQMAbwAgAAFxQQMALwAgAAFdQQUAPwAgAE8AIAACcUEDAPAAIAABXbgANNAwMQABANMA+gQnBE4ACwBfuAAwK7gACC+4AATcuAAC0LgAAi+6AAMABAAIERI5ugAJAAgABBESObgACBC4AArQALgABy+4AAvcugAAAAsABxESObgAAdC4AAEvuAAHELgABdC6AAYABwALERI5MDEJARcJAQcJAScJATcCfwFGXP66AUxi/rT+uFwBSP62YgMCAUZc/rr+tmIBSv64XAFIAUxiAAADAHv/kwXfBgAAGQAhACkBFrgAMCu6ABoABQAzK0EDABAABQABXUEDAA8ABQABcUEDAC8ABQABXUEDAC8ABQABcUEDAN8ABQABXUEDABAABQABcUEDAOAABQABXUEDAJAAGgABXUEDABAAGgABcUEDAGAAGgABcUEDAN8AGgABXUEDAEAAGgABcUEDAOAAGgABXUEDAEAAGgABXUEDABAAGgABXbgAGhC4ABLQuAAFELgAJdC6AB0AJQAaERI5ugAoABoAJRESOUEDAEAAKwABXUEDACAAKwABcQC4ABNFWLgACi8buQAKAB0+WbgAE0VYuAAXLxu5ABcAGT5ZuAAKELkAIgAW9LgAFxC5AB8AFvS6ABwAIgAfERI5ugAnAB8AIhESOTAxBSM3JgI1NBI2JDMyFzczBxYSFRQCDgEjIicBECcBFjMyEgEiAhEQFwEmAWaZlW55cMEBAZC6kkqaeW98cL/+j8KUAzF1/Z5vmdL9/jHV/m8CZHBt+GUBLcCuASTEbFh5x2T+y8Gu/t/AaVYCogEcsfwGXgFjA7j+lv7Y/uutA/ddAP//AF7/1wYSB1YCJgA5AAABBwDZAkgAAAAYuAAwK0EDAIAAIQABXUEDAGAAIQABcTAx//8AXv/XBhIHVgImADkAAAAHANoCtAAA//8AXv/XBhIHbwImADkAAAEHANsCBAAAAA+4ADArQQMAgAAnAAFdMDEA//8AXv/XBhIHTAImADkAAAEHANwB6QAAAC24ADAruAAeL0EDAC8AHgABXUEFAHAAHgCAAB4AAl1BAwAAAB4AAXG4ADLQMDEA////zQAABScHVgImAD0AAAEHANoB1QAAACW4ADArQQMALwAaAAFdQQUArwAaAL8AGgACXUEDAG8AGgABXTAxAAACAHEAAASsBcMAFgAjAMK4ADArugAhABEAMytBAwA/ABEAAV1BAwAfABEAAXFBAwCAABEAAV24ABEQuAAM0LgAGdC4AAHQQQMAPwAhAAFdQQMAgAAhAAFduAAhELgABdAAuAATRVi4ABQvG7kAFAAdPlm4ABNFWLgADy8buQAPABk+WbgAFBC5ABIAFvS4AADQugABABQADxESObgAAS+6AAoADwAUERI5uAAKL7gADxC5ABEAFvS4AAzQuAABELkAGAAW9LgAChC5ABoAFvQwMQEVMyAEFRQOAycRFxUhNTcRJzUhFRMjERY+BTU0JgHhrAEEARs0banql7n94a64Ah4byViMa081IA2hBVJx37pWl4FSIxL++kgpKUgE4UgpKf7f/WgNCBc0O1JSMaCiAAABACn/1wRcBhkAPAEJuAAwK7oAJQA7ADMruAA7ELgAAtC4ADsQuAA40EEDAC8AJQABXboALwA4ACUREjm4AC8vQQUAIAAvADAALwACXbgAC9C6ACwAOAAlERI5uAAsL7gADtC4ACUQuAAW0LoAHQA4ACUREjm4AB0vALgAE0VYuAACLxu5AAIAHz5ZuAATRVi4AAgvG7kACAAhPlm4ABNFWLgAGy8buQAbABk+WbgAE0VYuAA4Lxu5ADgAGT5ZugALAAgAGxESOboAEQAIABsREjm4ABsQuAAf3LgAGxC5ACIAFPS6ACgAGwAIERI5uAALELgAL9C4AAgQuQA0ABT0uAA4ELkAOwAW9LgAAhC5ADwAFfQwMRM1NzU0PgIzMhYVDgEVFB4FFRQOAiMiJzUzFxYzMjY1NC4ENTQ2NzQuAiMiBhURITU3ESnHSXeRTLC3rn0sSFZVSCw3W288k5JQOS9AZlo6VmVWOqCTITxELG6D/pW5A3VON4tfnmI1p7RSl3EsVEdJUlh0QEZwRSQxxYkTU2wuXEhaWX9Hg+VCQFksEp6G+2grSAMC//8AXP/iBB8GFAImAEUAAAEHAEQBaAAAAA+4ADArQQMArwAuAAFdMDEA//8AXP/iBB8GFAImAEUAAAEHAHcBPgAAAA+4ADArQQMAQAArAAFdMDEA//8AXP/iBB8GEgImAEUAAAEHAMYAwQAAADK4ADArQQMAgAAzAAFdQQUAbwAzAH8AMwACXUEFADAAMwBAADMAAnFBAwDAADMAAV0wMf//AFz/4gQfBaoCJgBFAAABBgDJdQAAF7gAMCtBBwBgADsAcAA7AIAAOwADXTAxAP//AFz/4gQfBa4CJgBFAAABBgBreQAAG7gAMCu4ACsvQQUAQAArAFAAKwACXbgAP9AwMQD//wBc/+IEHwZCAiYARQAAAQcAyAEGAAAAMrgAMCu4ACsvQQMAnwArAAFdQQMAPwArAAFdQQMA4AArAAFdQQMAQAArAAFxuABO0DAxAAMAXP/nBiEEGQAyADwAQwGTuAAwK7oANgAVADMrQQUAoAA2ALAANgACXUEDADAANgABXUEDABAANgABXbgANhC4AAHQuAA2ELgAQdxBAwBgAEEAAV1BAwDAAEEAAV1BAwDwAEEAAV1BAwCQAEEAAV1BAwAgAEEAAXFBAwBQAEEAAXG4ADLQugAJADIANhESOboADwA2AAEREjlBAwAvABUAAV1BAwCgABUAAV1BAwAwABUAAV24ADYQuAAb0LoAJAAVAAEREjm4ACQvuAAh0LgAARC4AEDQugAqABsAQBESObgAFRC4ADrQALgAE0VYuAAnLxu5ACcAHz5ZuAATRVi4AC0vG7kALQAfPlm4ABNFWLgAEi8buQASABk+WbgAE0VYuAAMLxu5AAwAGT5ZugABAC0ADBESObgAAS+4AAwQuQAGABb0ugAPAAwALRESOboAGwAnABIREjm4ABsvuAAnELkAHwAU9LgAJxC4ACLQugAqAC0ADBESObgAEhC5ADMAFvS4ABsQuQA4ABT0uAAtELkAPQAU9LgAARC5AEAAFPQwMQEhHgMzMjcXDgEjIiYnDgEjIiY1ND4DNzU0JiMiDwEjET4BMzIWFz4BMzIeAhUBMjY9AgQVFBYBIgYHITQmBgT9eQE+an9LaHcfVKhraLxEPtiMkZBMdqSYVFhmTWhqMVrITGSeK0q4TFWRcUH7sGR5/nViAwhziRAB3W0B+nCmXixGPEozYFdcW5ZxVYNONBQCYIxuGOgBHSAkWkpOVj97yID+P6KHaCUQ+lhUA2K2nqC0AAEAXP4UA5wEGwAwAQG4ADArugAiABsAMytBAwAwACIAAV1BAwBwACIAAV1BAwBQACIAAV1BAwAQACIAAV1BAwCQACIAAV24ACIQuAAA0EEDAFAAGwABXUEDAG8AGwABcUEDAC8AGwABXUEDADAAGwABXUEDABAAGwABXboABwAbACIREjm4AAcvuAAW0LgAA9C4AAcQuAAO3LgABxC4ABPQuAAiELgAJdC4ABsQuAAq0AC4AAwvuAATRVi4ACAvG7kAIAAfPlm4ABNFWLgAAy8buQADABk+WbgADBC4ABHcuAADELgAFtC4ACAQuAAk3LgAIBC5ACcAFPS4AAMQuQAtABb0uAADELgAMNwwMSUOAQ8BHgEVFA4CIyInNRYzMjU0JzcuAzU0PgIzMhcRIwMmIyIGFRQWMzI2NwOcU5ZbDml4LU9eOExKPR2skRhYnXhHTomyZ5OOKW88O52Wxq07X0VgQTQEOQRqTjxZMhcTTAmGehdoB1OHxXGB0IRGQv7LAQwO6NbM2h8n//8AXP/nA88GFAAmAEkAAAEHAEQBkQAAAA+4ADArQQMAoAAnAAFdMDEA//8AXP/nA88GFAAmAEkAAAEHAHcBcQAAADy4ADArQQMAEAAkAAFdQQMAvwAkAAFdQQMA/wAkAAFdQQMADwAkAAFxQQMAIAAkAAFxQQMAYAAkAAFdMDH//wBc/+cDzwYSACYASQAAAQcAxgDpAAAAR7gAMCtBAwC/ACwAAV1BAwBfACwAAV1BAwDAACwAAV1BBQDgACwA8AAsAAJdQQsAAAAsABAALAAgACwAMAAsAEAALAAFcTAxAP//AFz/5wPPBa4AJgBJAAABBwBrAKIAAAAXuAAwK7gAJC9BAwBAACQAAV24ADjQMDEA//8AUgAAAqgGFAAmAMMtAAEHAEQA0QAAABy4ADArQQMAXwAOAAFdQQUA0AAOAOAADgACXTAx//8AUgAAAnsGFAImAMMAAAEHAHcAgwAAACq4ADArQQMAEAALAAFdQQMAzwALAAFdQQMAIAALAAFxQQMAQAALAAFdMDH//wBSAAACngYSACYAwyMAAQYAxh8AACm4ADArQQMAXwATAAFxQQMA8AATAAFdQQcAAAATABAAEwAgABMAA3EwMQD//wBTAAACrAWuACYAwzEAAQYAa+YAADG4ADAruAALL0EFAN8ACwDvAAsAAl1BAwBAAAsAAV1BBQCgAAsAsAALAAJduAAf0DAxAAACAJz/0QQ3BfYAHgAuAKe4ADArugAfABYAMytBAwAQABYAAV1BAwCgABYAAV1BAwAwABYAAV1BAwAwAB8AAXFBAwCgAB8AAV1BBwAQAB8AIAAfADAAHwADXbgAHxC4AA3QugAEABYADRESOboAHAAfAAQREjm4ABYQuAAm0AC4ABovuAAFL7gAE0VYuAASLxu5ABIAGT5ZQQMAXwAFAAFduAAaELkAIwAW9LgAEhC5ACkAFPQwMQEnNyYnNxYXNxcHFgARFA4CIyIuATU0PgEzMhcmJwE0JyYjIgYVFBYzMj4DAaRKYkVeQXoqXkhU5wEAOXO+f4LIaGPRlGdgWskBbSNeg4WMjGpIaj0kDASoN2stKzEvFWc4XIf+R/7YfMaWUYzqkI7bhTTqmfzBp5JIqb3i3zBMc3AA//8ASAAABQgFqgImAFIAAAEHAMkBPwAAACS4ADArQQMAYAAuAAFdQQkAgAAuAJAALgCgAC4AsAAuAARdMDH//wBc/+cELwYUAiYAUwAAAQcARAGiAAAAGLgAMCtBAwBPACoAAV1BAwCgACoAAV0wMf//AFz/5wQvBhQCJgBTAAABBwB3AYEAAAAcuAAwK0EDABAAJwABXUEFACAAJwAwACcAAnEwMf//AFz/5wQvBhICJgBTAAABBwDGAPoAAABIuAAwK0EFAFAALwBgAC8AAnFBAwC/AC8AAV1BAwB/AC8AAV1BAwBQAC8AAV1BAwAwAC8AAXFBBwDQAC8A4AAvAPAALwADXTAx//8AXP/nBC8FqgImAFMAAAEHAMkArgAAABy4ADArQQMALwA3AAFdQQUAoAA3ALAANwACXTAx//8AXP/nBC8FrgImAFMAAAEHAGsAsgAAADa4ADAruAAnL0EDAA8AJwABcUEFAG8AJwB/ACcAAl1BAwCgACcAAV1BAwBQACcAAXG4ADvQMDEAAwBmAIUEjwSPAAMADwAaAGa4ADAruAAEL0EDAC8ABAABXbgACty4AALcuAAEELgAA9y4AAoQuAAT0LgABBC4ABjQALgAAy+4AADcQQMAAAAAAAFxuAADELgADdxBAwA/AA0AAV24AAfcuAAAELgAENy4ABbcMDETIRUhATQ2MzIWFRQGIyImEzIWFRQGIyImNDZmBCn71wGYSTY1SEg1Nkl/NUhJNDZJSQLjg/6kNUhJNDZJSQPBSTQ1SkpqSAAAAwBc/6QELwRUABQAHAAkAPu4ADArugAdAAAAMytBAwCgAAAAAV1BAwAPAAAAAXFBAwAvAAAAAV1BAwBvAAAAAXFBAwAQAAAAAV1BAwBQAAAAAV1BAwAwAAAAAV1BAwBQAB0AAV1BAwDAAB0AAV1BAwAPAB0AAXFBAwDgAB0AAV1BBQCQAB0AoAAdAAJdQQMAMAAdAAFdQQMAEAAdAAFduAAdELgACtC4AAAQuAAV0LoAGAAVAB0REjm6ACAAHQAVERI5ALgAE0VYuAADLxu5AAMAHz5ZuAATRVi4AA0vG7kADQAZPlm5ACIAFPS4AAMQuQAaABT0ugAXACIAGhESOboAHwAaACIREjkwMRM0ADMyFzczBxYVFAAjIicHIzcuATcUFwEmIyIGBTQnARYzMjZcAQTmlnZKk3t7/vzlZFc5lFtcYcNUAYtOao2aAlQr/o00PY2gAgDyASdIg9mR6vL+2R9ioEXkk9h4AsNO+MmNb/1cGfkA//8AH//nBMsGFAImAFkAAAEHAEQBwQAAAA+4ADArQQMAHwAhAAFxMDEA//8AH//nBMsGFAImAFkAAAEHAHcBoAAAACG4ADArQQMAPwAeAAFdQQMA7wAeAAFdQQMAUAAeAAFdMDEA//8AH//nBMsGEgImAFkAAAEHAMYBGQAAACq4ADArQQMAUAAmAAFdQQMArwAmAAFdQQMAMAAmAAFxQQMAwAAmAAFdMDH//wAf/+cEywWuAiYAWQAAAQcAawDRAAAAILgAMCu4AB4vQQMAbwAeAAFdQQMAcAAeAAFduAAy0DAx////1/3nBIcGFAImAF0AAAEHAHcBjQAAAA+4ADArQQMAPwATAAFdMDEAAAIACv4ABD8GGQAaACsA07gAMCu6ACcABAAzK0EDABAABAABXUEDADAABAABXbgABBC4ABrQuAAf0LgACtBBAwAwACcAAV1BAwAQACcAAV24ACcQuAAS0AC4ABNFWLgACC8buQAIACE+WbgAE0VYuAANLxu5AA0AHz5ZuAATRVi4AAIvG7kAAgAbPlm4ABNFWLgAFy8buQAXABk+WbgAAhC5AAQAFvS4AAgQuQAFABf0ugAKAA0AFxESOboAGQAXAA0REjm4AAQQuAAa0LgADRC5ABsAFPS4ABcQuQAiABT0MDEBFSE1NxEnNSUXET4BMzIeAhUUDgIjIicREyIGBxEeATMyPgI1NC4CAlz9rrm5AVwVOIBWVJ56SlKJsF54Y+M/eiodd0czZVAxJUZl/ikpKUgG70gpSBX9mkU2S4zHe4vKhEAr/l8FQTo0/YU5Qi5npXhfoHNAAP///9f95wSHBa4CJgBdAAABBwBrAL4AAAAguAAwK7gAEy9BAwA/ABMAAV1BAwBvABMAAV24ACfQMDEAAQBSAAACewQZAAoAW7gAMCu4AAQvQQMAQAAEAAFduAAK0EEDABAADAABcQC4ABNFWLgACC8buQAIAB8+WbgAE0VYuAACLxu5AAIAGT5ZuQAEABb0uAAIELkABQAX9LgABBC4AArQMDElFSE1NxEnNSUXEQJ7/de4uAFcFSkpKUgC70gpSBX8bQAAAgB7/+UILwXfABMANwF+uAAwK7oAGQAcADMrQQMAgAAcAAFduAAcELgAJNC4ACQvQQUAPwAkAE8AJAACXUEDAG8AJAABXUEDAI8AJAABXbgABdC4ABwQuAAP0LgAHBC4ABXQQQMAgAAZAAFdQQMAoAAZAAFdQQMAYAAZAAFxuAAZELgAFtC4ABwQuAAr0LgAGRC4AC/QuAAvL7gAMtC4ABUQuAA00LoANgAvADQREjkAuAATRVi4ACkvG7kAKQAdPlm4ABNFWLgALC8buQAsAB0+WbgAE0VYuAAcLxu5ABwAGT5ZuAATRVi4AB8vG7kAHwAZPlm4ACkQuQAAABb0uAAfELkACgAW9EEDAIgAHAABXboANAAsABwREjm4ADQvQQMATwA0AAFxQQMAXwA0AAFdQQMAPwA0AAFduQAUABb0uAAcELkAFQAW9LgAHBC4ABfcQQMAqwAZAAFdugAdABwALBESOUEDAIYAJAABXboAKwAsABwREjm4ACwQuAAx3LgALBC5ADMAFvQwMQEiDgIVFB4CMzI+AjU0LgIBESETMxEHJyE1BiEiLgECNTQSNiQzMhc1ITcXESMDIRElFwcDL2uteUJCea1rYJxuPDxvmwH8Afx/KR++/YWn/vOR+bZocMEBAZD/mwJmvx8pf/4YAaAnFQVvYK7ykpPvql1eq/CSkvKtX/1H/bQBJ/5vGxuqw2jAAR+vrgEkxGy4nBoa/m4BJ/3NFA6FAAMAXP/nBtMEGQASADwARwGduAAwK7oADAA4ADMrQQMAoAA4AAFdQQMAzwA4AAFdQQMAbwA4AAFxQQMADwA4AAFxQQMALwA4AAFdQQMAMAA4AAFdQQMAEAA4AAFduAA4ELgAAtBBAwBwAAwAAV1BAwDgAAwAAV1BAwAPAAwAAXFBBQCQAAwAoAAMAAJdQQMAMAAMAAFdQQMAEAAMAAFduAAMELgAINC4AELQugAWAAwAQhESObgADBC4AEPcQQMAwABDAAFdQQMAYABDAAFdQQMAkABDAAFdQQMA8ABDAAFdQQMAAABDAAFxQQMAIABDAAFxuAAe0LoAKAAMAB4REjm6ADAADAAgERI5ALgAE0VYuAATLxu5ABMAHz5ZuAATRVi4ABkvG7kAGQAfPlm4ABNFWLgAMy8buQAzABk+WbgAE0VYuAAtLxu5AC0AGT5ZuAAzELkABwAU9EEDAJsADAABXbgAExC5ABEAFPS6ABYAGQAtERI5ugAgABkALRESObgAIC+4AC0QuQAlABb0ugAwAC0AGRESObgAGRC5AD0AFPS4ACAQuQBCABT0MDEBBhUUHgIzMj4CNTQuAiMiNzIWFz4BMzIeAhUHIR4DMzI3Fw4DIyImJw4BIyIuAjU0PgIFIg4CByE0LgIBbE0nS25HR3BNKSlNcEeOjorOPTm3cFSUb0Ed/XkDQGaDR2l2HytSVl42crE3P86MdLZ+QkN+tgNsOV1DKwgB3Rs1TgNFfchjpXdCQnelY2Kld0NYb2RqaT+AwYIdd6BgKUY8JjAcC3BnZ3BOjsV4eMaNTmEvWH5PUH5YLgABADMEjQJ9BhIACQBvuAAwK7gACC+4AAHcALgABi9BAwBeAAYAAV1BAwAfAAYAAXFBAwD/AAYAAV1BBQA/AAYATwAGAAJxQQMAPgAGAAFdQQMAHgAGAAFduAAD0LgABhC4AAncQQMAsAAJAAFduAAF0EEDAFkABQABXTAxARMPAScjBy8BEwGF+AxSxQjHTAz0BhL+phYV4uIVGAFYAAEASATLApEGUAAIACe4ADAruAABL7gAB9wAuAAAL7gAA9y4AAAQuAAE0LgAAxC4AAXQMDEBAz8BFzcfAQMBP/cMUsnKTAzzBMsBWhYV4eEVGP6oAAIAOQSaAd8GQgATACcAdrgAMCu4AAAvuAAK3LgAGdC4AAAQuAAj0AC4AA8vQQMAAAAPAAFxQQMAXwAPAAFdQQMAHwAPAAFdQQMATwAPAAFxQQMAPwAPAAFdQQMA4AAPAAFdQQMAwAAPAAFduAAF3LgADxC5ABQAFPS4AAUQuQAeABT0MDETND4CMzIeAhUUDgIjIi4CFzI+AjU0LgIjIg4CFRQeAjkiOk0qKk06IiI5TSsqTToi0RgpHxERHykYFygeEREeKAVvK0w6IiI6TSorTTsiIjpNRREeKBgXKB4RER4oFxcoHhIAAAEASASgAwgFqgAaAKi4ADAruAAQL7gAA9wAuAAJL0EDAN8ACQABXUEDAB8ACQABXUEDAKAACQABXUEDAIAACQABXbgAANxBBQDgAAAA8AAAAAJdQQUAAAAAABAAAAACcbgACRC4ABbcQQkATwAWAF8AFgBvABYAfwAWAARduAAC0LgAAi+4ABYQuAAM3EEFAO8ADAD/AAwAAl1BBQAPAAwAHwAMAAJxuAAJELgAD9C4AA8vMDEBNjcXDgQHBiYHDgEHJz4ENzYeAgJCVkUrBA0tLkciN8knKEEwKwQPLy9CHiNRPVMFTg9NJQcXPjIvBwpQBwcqLysHGD4xLQUHFSEVAAEAZgIrBIcCpgADACa4ADAruAADL0EFAEAAAwBQAAMAAl24AALcALgAAy+5AAAAFfQwMRMhFSFmBCH73wKmewABAGYCKQc/AqQAAwAmuAAwK7gAAy9BBQBAAAMAUAADAAJduAAC3AC4AAMvuQAAABX0MDETIRUhZgbZ+ScCpHsAAQCaA8MB4wXsABAASbgAMCu4AAYvuAAA3EEJAEAAAABQAAAAYAAAAHAAAAAEXbgACtC4AAYQuAAO3AC4ABNFWLgACS8buQAJAB0+WbgADty4AAPcMDEBFAYjIiY1NDY3Fw4BFRcyFgGaSjVDPqp/IFV1AjVKBD80SF5Jh9YlPih0QRNJAAABAEgDwwGRBewAEABJuAAwK7gABi+4AADcQQkATwAAAF8AAABvAAAAfwAAAARduAAK0LgABhC4AA7cALgAE0VYuAADLxu5AAMAHT5ZuAAO3LgACdwwMRM0NjMyFhUUBgcnPgE1JyImkUo1Qz6qfyBVdQI1SgVvNElfSYfWJD0odEETSQABAHH+uAG6AOEAEABWuAAwK7gABi9BAwAQAAYAAV24AADcQQkATwAAAF8AAABvAAAAfwAAAARduAAK0LgABhC4AA7cALgAE0VYuAAOLxu5AA4AGT5ZuAAD3LgADhC4AAncMDE3NDYzMhYVFAYHJz4BNSciJrpKNUM+qn8gVXUCNUpkNElfSYbXJD4odEESSgACAJoDwwN9BewAEAAhAIq4ADAruAAGL7gAANxBCQBAAAAAUAAAAGAAAABwAAAABF24AArQuAAGELgADty4AAYQuAAX3LgAEdxBCQBAABEAUAARAGAAEQBwABEABF24ABvQuAAXELgAH9wAuAATRVi4AAkvG7kACQAdPlm4AA7cuAAD3LgAFNC4AAkQuAAa0LgADhC4AB/QMDEBFAYjIiY1NDY3Fw4BFRcyFgUUBiMiJjU0NjcXDgEVFzIWAZpKNUM+qn8gVXUCNUoBmUk2Qj+qfyFVdgI2SQQ/NEheSYfWJT4odEETSTY0SF5Jh9YlPih0QRNJAAACAEgDwwMrBewAEAAhAIq4ADAruAAGL7gAANxBCQBPAAAAXwAAAG8AAAB/AAAABF24AArQuAAGELgADty4AAYQuAAX3LgAEdxBCQBPABEAXwARAG8AEQB/ABEABF24ABvQuAAXELgAH9wAuAATRVi4ABQvG7kAFAAdPlm4AAPQuAAUELgAH9y4ABrcuAAJ0LgAHxC4AA7QMDEBNDYzMhYVFAYHJz4BNSciJiU0NjMyFhUUBgcnPgE1JyImAitJNkI/qn8hVnUCNkn+Zko1Qz6qfyBVdQI1SgVvNElfSYfWJD0odEETSTY0SV9Jh9YkPSh0QRNJAAACAHH+tgNUAN8ADwAgAJO4ADAruAADL0EDABAAAwABXbgADdxBCQBPAA0AXwANAG8ADQB/AA0ABF24AAfQuAADELgAC9y4AAMQuAAW3LgAENxBCQBPABAAXwAQAG8AEAB/ABAABF24ABrQuAAWELgAHtwAuAATRVi4AB4vG7kAHgAZPlm4ABPcuAAA0LgAHhC4ABncuAAG0LgAHhC4AAvQMDElMhYVFAYHJz4BNSciJjQ2BTQ2MzIWFRQGByc+ATUnIiYC00I/qn8hVnUCNklJ/h1KNUM+qn8gVXUCNUrfX0mG1yQ+KHRBEkpqSH00SV9JhtckPih0QRJKAAABAM0CAALTBAYAEwAXuAAwK7gAAC+4AArcALgADy+4AAXcMDETND4CMzIeAhUUDgIjIi4CzSpHXjU0XkcpKkZeNDVeRyoDAjRfRyoqR180NF5GKipHXgAAAQBxAHUCdwQvAAYALrgAMCu4AAQvQQsAEAAEACAABAAwAAQAQAAEAFAABAAFXbgAANAAGbgAAC8YMDEJAQcBNQEXAUwBKyH+GwHlIQJS/j0aAcYtAccbAAEASAB1Ak4ELwAGACe4ADAruAAEL0EDABAABAABXUEDAFAABAABXbgAAdAAGbgAAS8YMDE3CQE3ARUBSAEr/tUgAeb+Go8BwwHCG/45Lf46AAAB/4n/0QLJBcMAAwAcuAAwKwC4AAEvuAATRVi4AAIvG7kAAgAdPlkwMRcjATMOhQK7hS8F8gACAAoCiQLFBgAACgANAJq4ADAruAAHL7gABNC4AAHQuAAHELgAC9AAuAAAL0EHAAAAAAAQAAAAIAAAAANxQQMAYAAAAAFxQQMA4AAAAAFdQQMAoAAAAAFduAAF3EEDAF8ABQABcUEDAD8ABQABcUEDAO8ABQABXboABAAAAAUREjm4AAQvuQABABb0uAAEELgAB9C4AAEQuAAL0LgACdC4AAAQuAAM0DAxAREXFSMVIzUhNQEDEQECUHV1nP5WAc8l/tEGAP22FF67u3QCSP22AXv+hQAAAf/w/+cExQXfACkA/7gAMCu6AAwAAgAzK0EDAA8AAgABcbgAAhC4AAfQQQMAUAAMAAFdQQMAMAAMAAFduAAMELgAD9C4AAIQuAAY0LgAFNC6ABUAGAAMERI5ugAaABgADBESObgAGBC4ABzQuAAMELgAItC4AAIQuAAo0AC4ABNFWLgACi8buQAKAB0+WbgAE0VYuAAlLxu5ACUAGT5ZugAVAAoAJRESObgAFS+4ABrcQQcAMAAaAEAAGgBQABoAA124AAHQuAAVELkAFgAV9LgABNC4ABUQuAAH0LgAChC4AA7cuAAKELkAEQAW9LgAGhC5ABsAFfS4ACUQuQAfABX0uAAbELgAKNAwMQMzNTQ3IzUzEgAzMhcDIxEmIyICByEHIR0BIQchHgEzMjcXDgEjIgADIxCLAo2bMwF19r7Dbz1igafcIQHrFP4cAbsV/mch8LqsvhlT6nr//qctmQKmNyMSewELAUcz/osBESn+99t7NTd74+RjNk5cATkBCwABAGYCYASPAuMAAwAguAAwK7gAAi+4AAHcALgAAi+4AAPcQQMAAAADAAFxMDEBFSE1BI/71wLjg4MAAf/jBjEBngdWAAQAc7gAMCu4AAMvQQMAkAADAAFduAAA3AC4AAEvQQUAjgABAJ4AAQACXUEDAF8AAQABcUEDAB8AAQABXUEFAM8AAQDfAAEAAl1BBwBOAAEAXgABAG4AAQADXUEDAAAAAQABcbgABNxBBQAfAAQALwAEAAJdMDEBByUnNwGeI/5uBnUGbTyWKWYAAAEAIQYxAdsHVgAEAHO4ADAruAAEL0EDAD8ABAABXbgAAdwAuAADL0EHAE8AAwBfAAMAbwADAANdQQUAzwADAN8AAwACXUEDAF8AAwABcUEFAI8AAwCfAAMAAl1BAwAfAAMAAV1BAwAAAAMAAXG4AADcQQUAHwAAAC8AAAACXTAxARcHBScBZnUG/m8jB1ZmKZY8AAABACEGJQK4B28ACQBxuAAwK7gACC+4AAPcALgABy9BBQCPAAcAnwAHAAJdQQMAXwAHAAFxQQUAzwAHAN8ABwACXUEJAD8ABwBPAAcAXwAHAG8ABwAEXUEDAB8ABwABXbgAANxBAwAfAAAAAV24AAcQuAAE0LgAABC4AAbQMDEBMwEVByUjBSc1ATNzARJD/vwJ/vxDB2/++xwpzc0pHAAAAgBiBmACwwdMABMAJwB5uAAwK7gAAC+4AArcQQUAYAAKAHAACgACXbgAABC4ABTcuAAe3EEHAFAAHgBgAB4AcAAeAANdALgADy9BBQBPAA8AXwAPAAJxQQMAHwAPAAFdQQUAXwAPAG8ADwACXUEDAOAADwABXbgABdy4ABnQuAAPELgAI9AwMRM0PgIzMh4CFRQOAiMiLgIlND4CMzIeAhUUDgIjIi4CYhEgKhoYKx8TEx8rGBoqIBEBdRMgKxkZKx8SEh8rGRkrIBMG1xkqIBISICoZGisgEhIgKxoZKiASEiAqGRorIBISICsAAAEAQgY+Aw4HIgAcAKC4ADAruAASL0EDAIAAEgABXbgABNwAuAAJL0EHAE8ACQBfAAkAbwAJAANdQQMA3wAJAAFdQQUAjwAJAJ8ACQACXUEDAB8ACQABXUEFAAAACQAQAAkAAnG4AADcQQMA4AAAAAFduAAJELgAF9xBBQBNABcAXQAXAAJduAAD0LgAAy+4ABcQuAAO3EEDAO8ADgABXbgACRC4ABHQuAARLzAxAT4BNxcOAyMiLgIjIgYHJz4DMzIeAjICRCNVKycTMj5IKiNEQT4dLU02JBU1PUMkIz4+PykGywMhKikcPzQjHCEcKSovGz01IhwhGwABAAAA3gGcAAcAUwAEAAIAOABHADoAAAIAC8QAAwABAAAB1QHVAdUB1QHVAiECWQLaA8sEdAU6BVoFjgXBBiEGVQadBsYG+gcPB6AH4giMCU0JwQplCxoLZwxQDQcNaA3WDgMOLg5eDvEP6BBWEPkRjhH+EqMTJxPCFFsUnhTbFXAVzBaSFyAXqBg0GQkZqxpRGs4bThuZHAIciRzoHVsdgx2ZHcMd8B4ZHlcfCx+XICkgyyF6IhkjUiPdJGolASWiJesmxCdVJ/wopSlFKcMqkCr/K5gr6SxMLNAtIi20LgsuKS6FLtYu3i8oL8wwxDEiMbgx4jLCMzk0DDSiNOM1FjU0NiI2PDabNvE3QTfaOBo4tDj7OSE5VzmhOg86SjqQOtk7DjuQO6k7yTvyPA08Mzx2PSw9+D4NPig+PT5RPnc+qz7ePvY/mD+xP88/6EAOQCtAU0CkQXlBkkGeQbNB10H3QpBDZkN7Q5BDtkPOQ+hEDkU6RgBGFUZARnFGikalRsdG6EcNR6pHyUfiR/1ILkhJSHFI0UmNSaJJwEniSf9KFErBSt5LI0w5TW1NvE3mTltO3E78TxxPX0+hT+lQZFDfUV1RiVG1Ud5R+VJkUyVTQlOMU9ZUJlSdVRoAAAABAAAAAQDFerZfHF8PPPUAGwgAAAAAAMsDk3oAAAAA1SvMxP+B/c0NiQegAAAACAACAAAAAAAAB7gAQQAAAAAAAAAAAAAAAAK4AAACLwCaArAAaAV9AGYEgwCFBuUAmgYEAGoBkQBoApMAqAKTAB4DVAAUBPYAZgIrAHEDSABmAd0AcQNY/5oFCABxA28ASASFAFwEOQBxBH0ACgQdAFwEvABxBDsAHwR9AFwEvABIAi8AmgJ9AJoEzQBcBPgAZgTNAJoD9gBxB2YAewWs//wFSABxBUQAewYOAHEFAABxBK4AcQW4AHsGwQBxAxQAcQKoABAFlgBxBOEAcQdgAFwGMwBIBkYAewT0AHEGWgB7BaQAcQRgAGYE4wAKBjEAXgVq/80H5f/NBcUADgT0/80E/AA9ApgAzQNY/5oCgwAfBFQAmAQS//oBxf+BBD0AXAScAAoD7gBcBM8AXAQhAFwDJQBIBJoAZgUdAD0CuABSAhAAJQSoAD0CuAA9B5YASAUnAEgEiwBcBJwACgScAFwDZgBIA7oAewNCAB0FCAAfBD3/4QYU/+EEHf/fBGj/1wOkAFwDSACaAfQAuANIAEgGfwCaArgAAAIvAJkEIQBxBHEAUgRxAHEEy//NAfQAuARmANsDMwBtBpwAcQP6AFwDyQBxBSkAZgOaAGYGnABxBBL/+gL+AE4E9gBmAtcATgMbAHEB3wB5BQwAGQR9AHEB5wB1Ar4AjwLHAEgEHQBcA8kASAd9AIUHrgCFBysAhQP2AD0FrP/8Baz//AWs//wFrP/8Baz//AWs//wHyQAfBUQAewUAAHEFAABxBQAAcQUAAHEDFABxAxQAcQN5AHEDQgBwBj0AcQYzAEgGRgB7BkYAewZGAHsGRgB7BkYAewT2ANMGWgB7BjEAXgYxAF4GMQBeBjEAXgT0/80FLQBxBKQAKQQ9AFwEPQBcBD0AXAQ9AFwEPQBcBD0AXAZoAFwD4wBcBBcAXAQXAFwEFwBcBBcAXALlAFICuABSAtsAUgLpAFMEqACcBScASASLAFwEiwBcBIsAXASLAFwEiwBcBPgAZgSLAFwFCAAfBQgAHwUIAB8FCAAfBGj/1wScAAoEaP/XArgAUgigAHsHGwBcArAAMwLRAEgCEAA5A1QASATuAGYHpgBmAisAmgIrAEgCKwBxA8UAmgPFAEgDxQBxA4EAzQK+AHECvgBIAtv/iQM3AAoFAv/wBPYAZgIE/+MCBAAhAtcAIQMCAGIDEABCAAEAAAfm/c0AAA3w/4H/mg2JAAEAAAAAAAAAAAAAAAAAAADeAAMEdgGQAAUAAAUzBMwAAACZBTMEzAAAAswAZgJmAAACAAAAAAAAAAAAgAAAJwAAAAIAAAAAAAAAACAgICAAQAAgIhIH5v3NAAAH5gIzAAAAAQAAAAAEAAXDAAAAIAAAAAEAAgACAQEBAQEAAAAAEgXmAPgI/wAIAAj//gAJAAr//gAKAAr//QALAAv//QAMAAz//QANAA3//AAOAA7//AAPAA7//AAQABD//AARABH/+wASABH/+wATABL/+wAUABP/+gAVABT/+gAWABX/+gAXABb/+gAYABb/+QAZABf/+QAaABj/+QAbABr/+QAcABv/+AAdABz/+AAeABz/+AAfAB3/9wAgAB7/9wAhACD/9wAiACH/9wAjACH/9gAkACL/9gAlACP/9gAmACX/9gAnACX/9QAoACb/9QApACf/9QAqACj/9AArACn/9AAsACr/9AAtACv/9AAuACz/8wAvAC3/8wAwAC7/8wAxAC7/8wAyADD/8gAzADH/8gA0ADL/8gA1ADL/8QA2ADP/8QA3ADT/8QA4ADb/8QA5ADf/8AA6ADf/8AA7ADj/8AA8ADn/7wA9ADr/7wA+ADz/7wA/ADz/7wBAAD3/7gBBAD3/7gBCAD//7gBDAED/7gBEAEH/7QBFAEL/7QBGAEP/7QBHAET/7ABIAET/7ABJAEX/7ABKAEf/7ABLAEj/6wBMAEn/6wBNAEn/6wBOAEr/6wBPAEv/6gBQAE3/6gBRAE7/6gBSAE7/6QBTAE//6QBUAE//6QBVAFL/6QBWAFL/6ABXAFP/6ABYAFT/6ABZAFT/6ABaAFb/5wBbAFf/5wBcAFj/5wBdAFn/5gBeAFn/5gBfAFv/5gBgAFv/5gBhAF3/5QBiAF3/5QBjAF//5QBkAGD/5QBlAGD/5ABmAGH/5ABnAGL/5ABoAGT/4wBpAGT/4wBqAGX/4wBrAGb/4wBsAGb/4gBtAGn/4gBuAGn/4gBvAGr/4QBwAGr/4QBxAGv/4QByAG3/4QBzAG7/4AB0AG//4AB1AG//4AB2AHD/4AB3AHH/3wB4AHP/3wB5AHT/3wB6AHT/3gB7AHX/3gB8AHb/3gB9AHf/3gB+AHj/3QB/AHn/3QCAAHv/3QCBAHv/3QCCAHz/3ACDAHz/3ACEAH7/3ACFAID/2wCGAID/2wCHAIH/2wCIAIH/2wCJAIL/2gCKAIT/2gCLAIX/2gCMAIb/2gCNAIb/2QCOAIf/2QCPAIj/2QCQAIr/2ACRAIr/2ACSAIv/2ACTAIz/2ACUAI3/1wCVAI7/1wCWAI//1wCXAJD/1gCYAJD/1gCZAJL/1gCaAJP/1gCbAJP/1QCcAJX/1QCdAJb/1QCeAJf/1QCfAJf/1ACgAJj/1AChAJr/1ACiAJv/0wCjAJz/0wCkAJz/0wClAJ3/0wCmAJ3/0gCnAKD/0gCoAKH/0gCpAKH/0gCqAKL/0QCrAKL/0QCsAKT/0QCtAKb/0ACuAKb/0ACvAKf/0ACwAKf/0ACxAKn/zwCyAKn/zwCzAKv/zwC0AKz/zwC1AKz/zgC2AK7/zgC3AK7/zgC4AK//zQC5ALD/zQC6ALL/zQC7ALP/zQC8ALP/zAC9ALT/zAC+ALT/zAC/ALf/ywDAALf/ywDBALj/ywDCALn/ywDDALn/ygDEALz/ygDFALz/ygDGAL3/ygDHAL3/yQDIAL7/yQDJAMD/yQDKAMH/yADLAML/yADMAML/yADNAMP/yADOAMT/xwDPAMX/xwDQAMf/xwDRAMf/xwDSAMj/xgDTAMn/xgDUAMr/xgDVAMv/xQDWAMz/xQDXAM7/xQDYAM7/xQDZAM//xADaAM//xADbAND/xADcANP/xADdANP/wwDeANT/wwDfANT/wwDgANX/wgDhANb/wgDiANj/wgDjANn/wgDkANn/wQDlANr/wQDmANv/wQDnANz/wADoAN3/wADpAN7/wADqAN//wADrAOD/vwDsAOH/vwDtAOL/vwDuAOP/vwDvAOP/vgDwAOX/vgDxAOb/vgDyAOb/vQDzAOj/vQD0AOn/vQD1AOr/vQD2AOv/vAD3AOv/vAD4AOz/vAD5AO7/vAD6AO//uwD7AO//uwD8APD/uwD9APH/ugD+APL/ugD/APT/ugD4CP8ACAAI//4ACQAK//4ACgAK//0ACwAL//0ADAAM//0ADQAN//wADgAO//wADwAO//wAEAAQ//wAEQAR//sAEgAR//sAEwAS//sAFAAT//oAFQAU//oAFgAV//oAFwAW//oAGAAW//kAGQAX//kAGgAY//kAGwAa//kAHAAb//gAHQAc//gAHgAc//gAHwAd//cAIAAe//cAIQAg//cAIgAh//cAIwAh//YAJAAi//YAJQAj//YAJgAl//YAJwAl//UAKAAm//UAKQAn//UAKgAo//QAKwAp//QALAAq//QALQAr//QALgAs//MALwAt//MAMAAu//MAMQAu//MAMgAw//IAMwAx//IANAAy//IANQAy//EANgAz//EANwA0//EAOAA2//EAOQA3//AAOgA3//AAOwA4//AAPAA5/+8APQA6/+8APgA8/+8APwA8/+8AQAA9/+4AQQA9/+4AQgA//+4AQwBA/+4ARABB/+0ARQBC/+0ARgBD/+0ARwBE/+wASABE/+wASQBF/+wASgBH/+wASwBI/+sATABJ/+sATQBJ/+sATgBK/+sATwBL/+oAUABN/+oAUQBO/+oAUgBO/+kAUwBP/+kAVABP/+kAVQBS/+kAVgBS/+gAVwBT/+gAWABU/+gAWQBU/+gAWgBW/+cAWwBX/+cAXABY/+cAXQBZ/+YAXgBZ/+YAXwBb/+YAYABb/+YAYQBd/+UAYgBd/+UAYwBf/+UAZABg/+UAZQBg/+QAZgBh/+QAZwBi/+QAaABk/+MAaQBk/+MAagBl/+MAawBm/+MAbABm/+IAbQBp/+IAbgBp/+IAbwBq/+EAcABq/+EAcQBr/+EAcgBt/+EAcwBu/+AAdABv/+AAdQBv/+AAdgBw/+AAdwBx/98AeABz/98AeQB0/98AegB0/94AewB1/94AfAB2/94AfQB3/94AfgB4/90AfwB5/90AgAB7/90AgQB7/90AggB8/9wAgwB8/9wAhAB+/9wAhQCA/9sAhgCA/9sAhwCB/9sAiACB/9sAiQCC/9oAigCE/9oAiwCF/9oAjACG/9oAjQCG/9kAjgCH/9kAjwCI/9kAkACK/9gAkQCK/9gAkgCL/9gAkwCM/9gAlACN/9cAlQCO/9cAlgCP/9cAlwCQ/9YAmACQ/9YAmQCS/9YAmgCT/9YAmwCT/9UAnACV/9UAnQCW/9UAngCX/9UAnwCX/9QAoACY/9QAoQCa/9QAogCb/9MAowCc/9MApACc/9MApQCd/9MApgCd/9IApwCg/9IAqACh/9IAqQCh/9IAqgCi/9EAqwCi/9EArACk/9EArQCm/9AArgCm/9AArwCn/9AAsACn/9AAsQCp/88AsgCp/88AswCr/88AtACs/88AtQCs/84AtgCu/84AtwCu/84AuACv/80AuQCw/80AugCy/80AuwCz/80AvACz/8wAvQC0/8wAvgC0/8wAvwC3/8sAwAC3/8sAwQC4/8sAwgC5/8sAwwC5/8oAxAC8/8oAxQC8/8oAxgC9/8oAxwC9/8kAyAC+/8kAyQDA/8kAygDB/8gAywDC/8gAzADC/8gAzQDD/8gAzgDE/8cAzwDF/8cA0ADH/8cA0QDH/8cA0gDI/8YA0wDJ/8YA1ADK/8YA1QDL/8UA1gDM/8UA1wDO/8UA2ADO/8UA2QDP/8QA2gDP/8QA2wDQ/8QA3ADT/8QA3QDT/8MA3gDU/8MA3wDU/8MA4ADV/8IA4QDW/8IA4gDY/8IA4wDZ/8IA5ADZ/8EA5QDa/8EA5gDb/8EA5wDc/8AA6ADd/8AA6QDe/8AA6gDf/8AA6wDg/78A7ADh/78A7QDi/78A7gDj/78A7wDj/74A8ADl/74A8QDm/74A8gDm/70A8wDo/70A9ADp/70A9QDq/70A9gDr/7wA9wDr/7wA+ADs/7wA+QDu/7wA+gDv/7sA+wDv/7sA/ADw/7sA/QDx/7oA/gDy/7oA/wD0/7oAAAAAACgAAADgCQoJAAAAAwIDBgUIBwIDAwQGAgQCBAYEBQUFBQUFBQUCAwUGBQQIBgYGBwYFBggDAwYFCAcHBgcGBQYHBgkGBgYDBAMFBQIFBQQFBQQFBgMDBQMJBgUFBQQEBAYFBwUFBAQCBAcDAgUFBQUCBQQHBAQGBAcFAwYDBAIGBQIDAwUECAkIBAYGBgYGBgkGBgYGBgMDBAQHBwcHBwcHBgcHBwcHBgYFBQUFBQUFBwQFBQUFAwMDAwUGBQUFBQUGBQYGBgYFBQUDCggDAwIEBgkCAgIEBAQEAwMDBAYGAgIDAwMKCwoAAAADAwMHBgkIAgMDBAYDBAIEBgQGBQYFBgUGBgMDBgYGBQkHBwcIBgYHCAQDBwYJCAgGCAcFBggHCgcGBgMEAwUFAgUGBQYFBAYGAwMGAwkGBgYGBAUEBgUIBQYFBAIECAMDBQYGBgIGBAgFBQYFCAUEBgQEAgYGAgMDBQUJCgkFBwcHBwcHCgcGBgYGBAQEBAgICAgICAgGCAgICAgGBgYFBQUFBQUIBQUFBQUEAwQEBgYGBgYGBgYGBgYGBgYGBgMLCQMEAwQGCgMDAwUFBQQDAwQEBgYDAwQEBAsMCwAAAAQDBAgGCQgCBAQFBwMFAwUHBQYGBgYHBgYHAwMHBwcFCggHBwgHBggJBAQIBwoJCQcJCAYHCQcLCAcHBAUDBgYCBgYFBwYEBgcEAwYECgcGBgYFBQQHBgkGBgUFAwUJBAMGBgYHAwYECQUFBwUJBgQHBAQDBwYDBAQGBQoLCgUICAgICAgLBwcHBwcEBAUECQkJCQkJCQcJCQkJCQcHBgYGBgYGBgkFBgYGBgQEBAQGBwYGBgYGBwYHBwcHBgYGBAwKBAQDBQcLAwMDBQUFBQQEBAQHBwMDBAQEDA0MAAAABAMECAcKCQIEBAUHAwUDBQgFBwYHBgcGBwcDBAcHBwYLCQgICQgHCQoFBAgHCwkJBwkIBwcJCAwJBwcEBQQHBgMGBwYHBgUHCAQDBwQLCAcHBwUGBQgGCQYHBQUDBQoEAwYHBwcDBwUKBgYIBQoGBAcEBQMIBwMEBAYGCwwLBgkJCQkJCQwICAgICAUFBQUJCQkJCQkJBwkJCQkJBwgHBgYGBgYGCgYGBgYGBAQEBAcIBwcHBwcHBwgICAgHBwcEDQsEBAMFBwsDAwMGBgYFBAQEBQgHAwMEBQUNDg0AAAAEBAQJBwsKAwQEBQgEBQMFCAYHBwcHCAcHCAQECAgIBgwJCQkKCAgJCwUECQgMCgoICgkHCAoJDQkICAQFBAcHAwcHBggHBQcIBAMIBAwIBwcHBgYFCAcKBwcGBQMFCwQEBwcHCAMHBQsGBggGCwcFCAUFAwgHAwQFBwYMDAwGCQkJCQkJDQkICAgIBQUGBQoKCgoKCgoICgoKCgoICAgHBwcHBwcKBgcHBwcFBAUFCAgHBwcHBwgHCAgICAcHBwQODAQFAwUIDAQEBAYGBgYEBAUFCAgDAwUFBQ4PDgAAAAUEBQoIDAsDBQUGCQQGAwYJBggHCAcIBwgIBAQICQgHDQoJCQsJCAoMBQUKCQ0LCwkLCggJCwkOCgkJBQYECAcDBwgHCAcGCAkFBAgFDQkICAgGBwYJBwsHCAYGAwYLBQQHCAgIAwgGDAcHCQYMBwUJBQUDCQgDBQUHBw0NDQcKCgoKCgoOCQkJCQkFBQYGCwsLCwsLCwkLCwsLCwkJCAcHBwcHBwsHBwcHBwUFBQUICQgICAgICQgJCQkJCAgIBQ8MBQUEBgkNBAQEBwcHBgUFBQYJCQQEBQUFDxAOAAAABQQFCggNCwMFBQYJBAYEBgkGCAgICAkICAkEBQkJCQcOCwoKCwkJCw0GBQoJDgwMCQwLCAkMCg8LCQkFBgUICAMICQcJCAYJCgUECQUOCgkJCQYHBgkICwgIBwYEBgwFBAgICAkECAYMBwcKBwwIBgkFBgQJCAQFBQgHDg4NBwsLCwsLCw8KCQkJCQYGBwYMDAwMDAwMCQwMDAwMCQoJCAgICAgIDAcICAgIBQUFBQkKCQkJCQkJCQkJCQkICQgFEA0FBQQGCQ4EBAQHBwcHBQUFBgkJBAQFBgYQEQ8AAAAFBAULCQ4MAwUFBwoEBwQHCgcJCAkICQgJCQQFCgoKCA8LCwsMCgkLDgYFCwoPDA0KDQsJCgwLEAwKCgUHBQkIBAgJCAoIBgkKBQQJBQ8KCQkJBwcHCggMCAkHBwQHDQUECAkJCgQJBg0ICAoHDQgGCgYGBAoJBAUGCAgPDw4ICwsLCwsLEAsKCgoKBgYHBwwMDQ0NDQ0KDQwMDAwKCgkICAgICAgNCAgICAgGBQYGCQoJCQkJCQoJCgoKCgkJCQURDgUGBAcKDwQEBAgICAcFBQYGCgoEBAYGBhESEAAAAAYFBgwKDw0DBQUHCwUHBAcLBwoJCgkKCQoKBQUKCwoIEAwLCw0LCgwOBwYMChANDQsODAkKDQwRDAsLBgcFCQkECQoICgkHCgsGBQoGEAsKCgoHCAcLCQ0JCQgHBAcOBgUJCQkKBAkHDggICwgOCQYLBgcECwoEBgYJCBAQDwgMDAwMDAwRCwsLCwsHBwcHDQ0NDQ0NDQsODQ0NDQsLCgkJCQkJCQ4ICQkJCQYGBgYKCwoKCgoKCwoLCwsLCQoJBhIPBgYEBwoQBQUFCAgIBwYGBgcLCwQEBgYHEhMRAAAABgUGDAoQDgQGBgcLBQcECAsICgoKCQsKCgsFBgsLCwkRDQwMDgsLDQ8HBg0LEQ4OCw4NCgsODBINCwsGCAYKCQQKCgkLCQcKDAYFCgYRDAoKCggIBwsKDgkKCAcEBw8GBQkKCgsECgcPCQkMCA8JBwsGBwQLCgQGBgkJEREQCQ0NDQ0NDRIMCwsLCwcHCAcODg4ODg4OCw4ODg4OCwwKCgoKCgoKDgkJCQkJBwYGBwoMCgoKCgoLCgsLCwsKCgoGExAGBgUHCxEFBQUICAgIBgYGBwsLBQUGBwcTFBIAAAAGBQYNCxAOBAYGCAwFCAQIDAgLCgsKCwoLCwUGCwwLCRINDQ0ODAsOEAcGDQwSDw8MDw0KDA8NEw4MDAYIBgoKBAoLCQsKBwsMBgULBhIMCwsLCAkIDAoOCgoJCAUIDwYFCgsLCwUKCBAJCQwJEAoHDAcHBAwLBQcHCgkSEhEJDQ0NDQ0NEg0MDAwMBwcICA8PDw8PDw8MDw8PDw8MDAsKCgoKCgoPCQoKCgoHBgcHCwwLCwsLCwwLDAwMDAoLCgYUEQYHBQgMEgUFBQkJCQgHBwcIDAwFBQcHBxQWEwAAAAcFBw4LEQ8EBgYIDAUIBQgNCQsLCwoMCwsMBQYMDAwKEw4NDQ8NDA4RCAcODBIQEAwQDgsMDw4UDgwMBggGCwoECwwKDAoIDA0HBQwHEw0LDAwJCQgMCw8KCwkIBQgQBwUKCwsMBQsIEQoJDQkRCgcMBwgFDQsFBwcKCRMTEgoODg4ODg4TDQ0NDQ0ICAkIEBAQEBAQEAwQDw8PDwwNDAsLCwsLCxAKCgoKCgcHBwcMDQsLCwsLDAsMDAwMCwwLBxYSBwcFCAwTBQUFCQkJCQcHBwgNDAUFBwgIFRcUAAAABwYHDgwSEAQHBwkNBgkFCQ0JDAsMCwwLDAwGBw0NDQoTDw4OEA0MDxIIBw8NExAQDREPCw0QDhUPDQ0HCQcLCwULDAoNCwgMDQcFDAcUDgwMDAkKCA0LEAsMCgkFCREHBgsMDA0FDAgRCgoOCRELCA0HCAUNDAUHBwsKFBQTCg8PDw8PDxQODQ0NDQgICQkQEBAQEBAQDREQEBAQDQ4MCwsLCwsLEQoLCwsLCAcICAwODAwMDAwNDA0NDQ0MDAwHFxMHBwUJDRQGBgYKCgoJBwcICA0NBQUHCAgWGBUAAAAHBgcPDBMRBAcHCQ4GCQUJDgkMDAwLDQwMDQYHDQ4NCxQQDw4RDg0QEwgHDw0UEREOERAMDREPFhAODgcJBwwLBQwNCw0LCQ0OBwYNBxUODQ0NCQoJDgwRCwwKCQUJEgcGCwwMDQUMCRILCg4KEgsIDggJBQ4MBQgICwoVFRQLEBAQEBAQFQ4ODg4OCAgKCREREREREREOEREREREODg0MDAwMDAwSCwsLCwsIBwgIDQ4NDQ0NDQ4NDg4ODgwNDAcYFAcIBgkOFQYGBgoKCgoICAgJDg4GBggICBcZFgAAAAgGCBANFBEFBwcKDgYJBQoOCg0MDQwODA0OBgcODg4LFRAPDxEODRATCQgQDhUSEg4SEA0OEhAXEQ4OBwoHDAwFDA0LDgwJDQ8IBg0IFg8NDQ0KCwkODBEMDQoJBgkTCAYMDQ0OBg0JEwsLDwoTDAkOCAkFDw0FCAgMCxYWFQsQEBAQEBAWDw4ODg4JCQoJEhISEhISEg4SEhISEg4PDQwMDAwMDBILDAwMDAgICAgNDw0NDQ0NDg0ODg4ODQ0NCBkUCAgGCg4WBgYGCwsLCggICAkODgYGCAkJGBoXAAAACAcIEA4VEgUICAoPBwoGCg8KDg0NDA4NDQ4HBw4PDgwWERAQEg8OERQJCBEPFhMTDxMRDQ8TEBgRDw8ICggNDAUMDgwODAkODwgGDggXDw4ODgoLCg8NEgwNCwoGChMIBwwNDQ4GDQoUDAsPCxQMCQ8JCQYPDQYICAwLFhcWDBERERERERcQDw8PDwkJCgoTExMTExMTDxMTExMTDxAODAwMDAwMEwwMDAwMCQgJCQ4PDg4ODg4PDg8PDw8NDg0IGhUICAYKDxcHBwcLCwsLCAgJCg8PBgYJCQkZGxgAAAAJBwgRDhYTBQgIChAHCgYKEAsODQ4NDw0ODwcIDxAPDBcSERATEA8SFQoIEQ8XExQPFBIODxMRGRIPEAgKCA4NBg0ODA8NCg4QCAYPCBgQDg4OCwwKEA0TDQ4LCgYKFAkHDQ4ODwYOChUMDBALFQ0JEAkKBhAOBgkJDQwXGBYMEhISEhISGBAQEBAQCgoLChQTFBQUFBQQFBMTExMPEA8NDQ0NDQ0UDA0NDQ0JCAkJDxAODg4ODhAOEBAQEA4ODggbFggJBgoPGAcHBwwMDAsJCQkKEBAGBgkJChocGQAAAAkHCRIPFhQFCAgLEAcLBgsQCw8ODw0PDg8PBwgQEBANGBIRERQQDxMWCgkSEBgUFBAUEg4QFBIaExAQCAsIDg0GDg8NEA0KDxEJBw8JGREPDw8LDAsQDhQNDgwLBgsVCQcNDg4QBg4KFQ0MEQwVDQoQCQoGEA8GCQkNDBgZFw0SEhISEhIZERAQEBAKCgsLFBQUFBQUFBAUFBQUFBARDw4ODg4ODhUNDQ0NDQkJCQkPEQ8PDw8PEA8QEBAQDg8OCRwXCQkHCxAZBwcHDAwMCwkJCQoQEAcHCQoKGx0aAAAACQcJEw8XFAUJCQsRBwsGCxEMDw4PDhAODxAHCBAREA0ZExISFBEQExcKCRMQGRUVERUTDxEVEhsTEREJCwgPDgYOEA0QDgsQEQkHEAkaEQ8QEAsNCxEOFQ4PDAsHCxYJBw4PDxAHDwsWDQ0RDBYOChEKCgYRDwYJCQ4NGRoYDRMTExMTExoSEREREQoKDAsVFRUVFRUVERUVFRUVEREQDg4ODg4OFg0ODg4OCgkKChARDw8PDw8RDxEREREPEA8JHRgJCgcLERoHBwcNDQ0MCQkKCxERBwcKCgocHhsAAAAKCAkTEBgVBQkJDBEICwcMEgwQDxAOEQ8QEQgJERERDhoUEhIVEhAUGAsJFBEaFhYRFhQPERYTHBQREQkMCQ8OBg8QDhEOCxASCgcQChsSEBAQDA0LEg8VDg8NCwcLFwoIDhAQEQcPCxcODRINFw4KEQoLBxIQBwoKDg0aGxkOFBQUFBQUGxISEhISCwsMCxYWFhYWFhYRFhYWFhYREhAPDw8PDw8WDg4ODg4KCgoKEBIQEBAQEBEQEhISEg8QDwoeGQkKBwwRGwgICA0NDQwKCgoLEhEHBwoLCx0fHAAAAAoIChQQGRYGCQkMEggMBwwSDBAPEA8RDxARCAkREhEOGxUTExYSERUYCwoUEhsWFxIXFBASFhQdFRISCQwJEA8GDxEOEQ8LERMKBxEKHBMQEREMDgwSDxYPEA0MBwwYCggPEBARBxAMGA4OEw0YDwsSCgsHEhAHCgoPDhscGg4VFRUVFRUcExISEhILCw0MFxYXFxcXFxIXFhYWFhITEQ8PDw8PDxcODw8PDwsKCgsRExAQEBAQEhASEhISEBEQCh8aCgoHDBIcCAgIDg4ODQoKCgwSEgcHCgsLHiAdAAAACggKFREaFwYKCgwTCAwHDRMNERARDxIQERIICRITEg8cFRQUFxMSFRkMChUSHBcYExgVEBIXFB4WExMKDQkQDwcQEQ8SDwwREwoIEQocExEREQ0ODBMQFw8RDgwHDBgKCA8RERIHEQwZDw4TDhkPCxMLDAcTEQcKCg8OHB0bDxUVFRUVFR0UExMTEwwMDQwXFxgYGBgYExgXFxcXExMREBAQEBAQGA8PDw8PCwoLCxETERERERETERMTExMREREKIBsKCwgMEh0ICAgODg4NCgoLDBMTCAgLCwsfIR4AAAALCAoVERsXBgoKDRMIDQcNFA0SEBEQEhAREggKExMTDx0WFBQXExIWGgwKFhMdGBgTGRYRExgVHxYTEwoNChEQBxASDxMQDBIUCwgSCx0UEhISDQ4NFBAYEBEODQgNGQsIEBEREwgRDBoPDxQOGhAMEwsMBxQRBwsLEA8dHhwPFhYWFhYWHhQTExMTDAwNDRgYGBgYGBgTGRgYGBgTFBIQEBAQEBAZDxAQEBALCwsLEhQSEhISEhMSFBQUFBESEQshHAoLCA0THggICA8PDw4LCwsMExMICAsMDCAjHwAAAAsJCxYSHBgGCgoNFAkNBw0UDhIREhATERITCQoTFBMQHhcVFRgUExcbDAsWFB4ZGRQZFxIUGRYgFxQUCg0KERAHERIQExENEhQLCBMLHhUSEhIODw0UERgQEg8NCA0aCwkREhITCBINGhAPFQ4aEAwUCwwHFBIICwsQDx4fHRAXFxcXFxcfFRQUFBQMDA4NGRkZGRkZGRQZGRkZGRQVExERERERERoQEBAQEAwLCwwTFRISEhISFBIUFBQUEhISCyMcCwsIDRQfCQkJDw8PDgsLCw0UFAgICwwMISQgAAAACwkLFxMcGQYLCw4UCQ4IDhUOExETERQRExQJChQVFBAfFxYWGRUTGBwNCxcUHhoaFBoXEhQaFiEYFBULDgoSEQcRExAUEQ0TFQsJEwsfFRMTEw4PDRURGRESDw4IDhsLCRESEhQIEg0bEBAVDxsRDBQMDQgVEwgLCxEQHyAeEBcXFxcXFyAWFRUVFQ0NDg0aGhoaGhoaFBoaGhoaFBUTERERERERGhARERERDAsMDBMVExMTExMVExUVFRUSExILJB0LDAkOFCAJCQkQEBAOCwsMDRUUCAgMDA0iJSEAAAAMCQsXEx0aBwsLDhUJDggOFQ8TEhMRFBITFAkLFBUUER8YFhYaFRQYHQ0LGBUfGhsVGxgTFRoXIhkVFQsOCxIRCBIUERQSDRQWDAkUDCAWExQUDhAOFRIaERMPDggOHAwJEhMTFAgTDhwREBYPHBENFQwNCBUTCAwMERAgIR4RGBgYGBgYIRYVFRUVDQ0PDhsaGxsbGxsVGxoaGhoVFhQSEhISEhIbEREREREMDAwMFBYTExMTExUTFRUVFRMUEwwlHgsMCQ4VIQkJCRAQEA8MDAwOFRUJCQwNDSMmIgAAAAwKDBgUHhoHCwsPFgkOCA8WDxQSFBIVExQVCgsVFhURIBkXFxoWFBkeDQwYFSAbGxYcGRMVGxgjGRYWCw8LExIIExQRFRIOFBYMCRQMIRcUFBQPEA4WExsSExAOCQ4cDAoSExMVCRMOHRERFxAdEg0WDA4IFhQIDAwSESEiHxEZGRkZGRkiFxYWFhYNDQ8OGxsbGxsbGxYcGxsbGxYXFBMTExMTExwREhISEg0MDQ0UFxQUFBQUFhQWFhYWExQTDCYfDAwJDxYhCQkJEBAQDwwMDQ4WFgkJDA0NJCcjAAAADAoMGRQfGwcMDA8WCg8IDxcPFBMUExUTFBUKCxYWFhIhGhgYGxcVGh4ODBkWIRwcFh0ZFBYcGCQaFhYMDwsTEggTFRIWEw4VFwwJFQwiFxQVFQ8RDxcTGxMUEA8JDx0MChMUFBYJFA4eEhEXEB4SDRYNDggXFAkMDRMRIiMgEhoaGhoaGiMYFxcXFw4OEA8cHBwcHBwcFh0cHBwcFhcVExMTExMTHRESEhISDQwNDRUXFBQUFBQWFBcXFxcUFRQMJyAMDQkPFiIKCgoREREQDAwNDhcWCQkNDg4lKCQAAAANCgwZFSAcBwwMDxcKDwkPFxAVFBUTFhQVFgoMFhcWEiIaGBgcFxYaHw4MGhciHR0XHRoUFx0ZJRsXFwwPDBQTCBQVEhYTDxUYDQoWDSMYFRUVEBEPFxQcExQRDwkPHg0KExUVFgkUDx8SEhgRHxMOFw0OCRcVCQ0NExIjJCESGhoaGhoaJBgXFxcXDg4QDx0dHR0dHR0XHR0dHR0XGBUUFBQUFBQeEhMTExMNDQ0NFhgVFRUVFRcVFxcXFxQVFA0oIQwNCg8XIwoKChERERANDQ0PFxcJCQ0ODiYpJQAAAA0KDRoVIR0HDAwQGAoQCRAYEBUUFRQWFBUWCgwXGBcTIxsZGR0YFhsgDw0bFyMdHhgeGxUXHRomGxgYDBAMFRMIFBYTFxQPFhgNChYNJBgWFhYQEg8YFB0UFREQCRAfDQoUFRUXCRUPHxMSGREfEw4YDQ8JGBUJDQ0UEiQkIhMbGxsbGxslGRgYGBgPDxEPHh0eHh4eHhgeHR0dHRgZFhQUFBQUFB4SExMTEw4NDg4WGBYWFhYWGBYYGBgYFRYVDSkiDQ0KEBckCgoKEhISEQ0NDg8YGAoKDQ4PJyomAAAADQsNGxYiHQgNDRAYCxAJEBkRFhUWFBcVFhcLDBcYFxMkHBoaHhgXHCEPDRsYJB4fGB8cFRgeGiYcGBgNEAwVFAkVFhMXFA8WGQ0KFw0lGRYWFhESEBkVHhQVEhAKECANCxQWFhcKFRAgExIZEiAUDxgODwkZFgkNDhQSJSUjExwcHBwcHCYaGBgYGA8PERAeHh8fHx8fGB8eHh4eGBkXFRUVFRUVHxMUFBQUDg0ODhcZFhYWFhYYFhkZGRkVFhUNKiMNDgoQGCULCwsSEhIRDQ0OEBgYCgoODw8oKycAAAAOCw0bFyIeCA0NERkLEAkRGREXFRYVGBUWGAsMGBkYFCUcGhoeGRcdIg8NHBglHx8ZIBwWGB8bJx0ZGQ0RDRYUCRUXFBgVEBcaDgoXDiYaFxcXERMQGRUeFRYSEAoQIA4LFRYWGAoWECEUExoSIRQPGQ4QCRkWCg4OFRMlJiQUHBwcHBwcJxoZGRkZDw8REB8fHx8fHx8ZIB8fHx8ZGhcVFRUVFRUgExQUFBQODg4PFxoXFxcXFxkXGRkZGRYXFg4rJA0OChEZJgsLCxMTExIODg4QGRkKCg4PDyksKAAAAA4LDhwXIx8IDQ0RGQsRChEaEhcWFxUYFhcYCw0ZGRkUJh0bGx8aGB0jEA4dGSYgIBkhHRYZIBwoHhkaDRENFhUJFhgUGRUQGBoOCxgOJxoXGBgRExEaFh8VFxMRChEhDgsVFxcZChcQIhQTGhIiFQ8ZDxAKGhcKDg4VEyYnJRQdHR0dHR0oGxoaGhoQEBIRICAgICAgIBkhICAgIBkbGBYWFhYWFiEUFRUVFQ8ODw8YGhcXFxcXGRcaGhoaFxgXDiwkDg4LERknCwsLExMTEg4ODxAaGQoKDw8QKi0pAAAADgsOHRgkIAgODhEaCxEKEhoSGBYYFhkWGBkLDRkaGRUnHhwcIBoZHiMQDh0aJyEhGiEeFxohHCkeGhoOEg0XFQkWGBUZFhEYGw4LGA4oGxgYGBIUERoWIBYXExEKESIOCxYXFxkKFxEjFRQbEyMVEBoPEAobGAoODxYUJygmFR4eHh4eHikcGhoaGhAQEhEhISEhISEhGiEhISEhGhsYFhYWFhYWIhQVFRUVDw4PDxgbGBgYGBgaGBoaGhoXGBcOLSUODwsRGigLCwsUFBQSDg4PERoaCwsPEBArLikAAAAPDA4eGCUgCA4OEhsMEgoSGxIYFxgWGRcYGQwNGhsaFSgeHBwhGxkfJBEOHhooISIbIh4YGiEdKh8bGw4SDhcWChcZFRoWERkbDwsZDykcGBkZEhQSGxchFhgUEgsSIw8MFhgYGgsYESQVFBwTJBYQGw8RChsYCg8PFhQoKScVHh4eHh4eKhwbGxsbERETEiIhIiIiIiIbIiEhISEbHBkXFxcXFxciFRYWFhYQDw8QGRwYGBgYGBsYGxsbGxgZGA8uJg4PCxIbKQwMDBQUFBMPDw8RGxsLCw8QECwvKgAAAA8MDx4ZJiEJDg4SGwwSChIcExkXGRcaFxkaDA4aGxoWKR8dHSEcGh8lEQ8fGykiIxsjHxgbIh4rIBsbDhIOGBYKFxkWGhcRGRwPCxoPKhwZGRkTFRIcFyEXGBQSCxIkDwwXGBgaCxgSJBYVHBQkFhAbEBEKHBkKDw8XFSkqJxYfHx8fHx8rHRwcHBwRERMSIiIjIyMjIxsjIiIiIhscGhcXFxcXFyMVFxcXFxAPEBAaHBkZGRkZGxkcHBwcGBkYDy8nDw8LEhsqDAwMFRUVEw8PEBIcGwsLEBERLTErAAAADwwPHxknIgkODhMcDBIKExwTGRgZFxsYGRsMDhscGxYqIB4eIhwaICYRDx8bKSMjHCQgGRsjHiwgHBwPEw4YFwoYGhYbFxIaHQ8MGg8rHRoaGhMVEhwYIhcZFBILEiUPDBcZGRsLGRIlFhUdFCUXERwQEQscGQsPEBcVKisoFiAgICAgICweHBwcHBERFBIjIyMjIyMjHCQjIyMjHB0aGBgYGBgYJBYXFxcXEA8QEBodGhoaGhocGhwcHBwZGhkPMSgPEAwTHCsMDAwVFRUUDw8QEhwcCwsQEREuMiwAAAAQDQ8gGigjCQ8PEx0MEwsTHRQaGBoYGxgaGw0OHB0cFyshHh4jHRshJxIPIBwqJCQcJSAZHCQfLSEcHQ8TDhkXChgbFxwYEhodEAwbECweGhsbFBUTHRgjGBkVEwsTJRANGBoaHAsZEiYXFh4VJhcRHRASCx0aCxAQGBYrLCkXISEhISEhLR4dHR0dEhIUEyQkJCQkJCQdJSQkJCQcHhsYGBgYGBglFhgYGBgREBARGx4aGhoaGh0aHR0dHRkbGRAyKQ8QDBMcLAwMDBYWFhQQEBASHR0MDBAREi8zLQAAABANECAbKSMJDw8UHQ0TCxQeFBsZGhgcGRocDQ8cHRwXKyEfHyQdHCIoEhAhHSskJR0lIRodJCAuIh0dDxQPGRgKGRsXHBgSGx4QDBsQLR4bGxsUFhMeGSQYGhUTCxMmEA0YGhocCxoTJxcWHhUnGBIdERILHhoLEBAYFiwtKhchISEhISEuHx0dHR0SEhQTJSQlJSUlJR0lJCQkJB0eGxkZGRkZGSYXGBgYGBEQEREbHhsbGxsbHRseHh4eGhsaEDMqEBEMFB0tDQ0NFhYWFRAQERMdHQwMERISMDQuAAAAEA0QIRspJAkPDxQeDRQLFB4VGxkbGRwZGxwNDx0eHRgsIiAgJB4cIikSECIdLCUmHiYiGh0lIC8jHh4QFA8aGAsZHBgdGRMcHxAMHBAuHxscHBQWFB4ZJBkaFhQMFCcQDRkbGx0MGhMoGBcfFigYEh4REwseGwsQERkXLS4rGCIiIiIiIi8gHh4eHhISFRQlJSYmJiYmHiYlJSUlHh8cGRkZGRkZJhcZGRkZERARERwfGxsbGxseGx4eHh4aHBoQNCsQEQwUHi4NDQ0XFxcVEBAREx4eDAwREhIAAAACAAAAAwAAABQAAwABAAAAFAAEAJgAAAAiACAABAACAH4A/wExAVMCxwLaAtwgFCAaIB4gIiA6IEQgdCCsIhL//wAAACAAoAExAVICxgLaAtwgEyAYIBwgIiA5IEQgdCCsIhL////k/8P/kv9y/gD97v3t4LfgtOCz4LDgmuCR4GLgK97GAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAsIGSwIGBmI7AAUFhlWS2wASwgZCCwwFCwBCZasAtDW1ghIyEbilggsFBQWCGwQFkbILA4UFghsDhZWSCwBUVhZLAoUFghsAVFILAwUFghsDBZGyCwwFBYIGYgiophILAKUFhgGyCwIFBYIbAKYBsgsDZQWCGwNmAbYFlZWRuwACtZWSOwAFBYZVlZLbACLLAHI0KwBiNCsAAjQrAAQ7AGQ1FYsAdDK7IAAQBDYEKwFmUcWS2wAyywAEMgRbAJQ2OwCkNiRC2wBCywAEMgRSCwACsjsQYEJWAgRYojYSBkILAgUFghsAAbsDBQWLAgG7BAWVkjsABQWGVZsAMlI2FERC2wBSywAWAgILANQ0qwAFBYILANI0JZsA5DSrAAUlggsA4jQlktsAYssABDsAIlQrIAAQBDYEKxDQIlQrEOAiVCsAEWIyCwAyVQWLAAQ7AEJUKKiiCKI2GwBSohI7ABYSCKI2GwBSohG7AAQ7ACJUKwAiVhsAUqIVmwDUNHsA5DR2CwgGKwCUNjsApDYiCxAQAVQyBGiiNhOLACQyBGiiNhOLUCAQIBAQFDYEJDYEItsAcsALAII0K2Dw8IAgABCENCQkMgYGCwAWGxBgIrLbAILCBgsA9gIEMjsAFgQ7ACJbACJVFYIyA8sAFgI7ASZRwbISFZLbAJLLAIK7AIKi2wCiwgIEcgsAlDY7AKQ2IjYTgjIIpVWCBHILAJQ2OwCkNiI2E4GyFZLbALLACwARawCiqwARUwLbAMLCA1sAFgLbANLACwAEVjsApDYrAAK7AJQ7AKQ2FjsApDYrAAK7AAFrEAAC4jsABHsABGYWA4sQwBFSotsA4sIDwgR7AJQ2OwCkNisABDYTgtsA8sLhc8LbAQLCA8IEewCUNjsApDYrAAQ2GwAUNjOC2wESyxAgAWJSAusAhDYCBGsAAjQrACJbAIQ2BJiopJI2KwASNCshABARUUKi2wEiywABUgsAhDYEawACNCsgABARUUEy6wDiotsBMssAAVILAIQ2BGsAAjQrIAAQEVFBMusA4qLbAULLEAARQTsA8qLbAVLLARKi2wGiywABawBCWwCENgsAQlsAhDYEmwAStlii4jICA8ijgjIC5GsAIlRlJYIDxZLrEJARQrLbAdLLAAFrAEJbAIQ2CwBCUgLrAIQ2BJILAFI0KwASsgsGBQWCCwQFFYswMgBCAbswMmBBpZQkIjILAIQ2CwDEMgsAhDYIojSSNGYLAFQ7CAYmAgsAArIIqKYSCwA0NgZCOwBENhZFBYsANDYRuwBENgWbADJbCAYmEjICCwBCYjRmE4GyOwDENGsAIlsAhDYLAMQ7AIQ2BJYCCwBUOwgGJgIyCwACsjsAVDYLAAK7AFJWGwBSWwgGKwBCZhILAEJWBkI7ADJWBkUFghGyMhWSMgILAEJiNGYThZiiAgPLAFI0KKOCMgLkawAiVGUlggPFkusQkBFCuwBUMusAkrLbAbLLAAFrAEJbAIQ2CwBCYgLrAIQ2BJsAErIyA8IC4jOLEJARQrLbAYLLEMBCVCsAAWsAQlsAhDYLAEJSAusAhDYEkgsAUjQrABKyCwYFBYILBAUVizAyAEIBuzAyYEGllCQiMgsAhDYEawBUOwgGJgILAAKyCKimEgsANDYGQjsARDYWRQWLADQ2EbsARDYFmwAyWwgGJhsAIlRmE4IyA8IzgbISCwCENgLiA8LyFZsQkBFCstsBcssAwjQrAAEz6xCQEUKy2wGSywABawBCWwCENgsAQlsAhDYEmwAStlii4jICA8ijgusQkBFCstsBwssAAWsAQlsAhDYLAEJSAusAhDYEkgsAUjQrABKyCwYFBYILBAUVizAyAEIBuzAyYEGllCQiMgsAhDYLAMQyCwCENgiiNJI0ZgsAVDsIBiYCCwACsgiophILADQ2BkI7AEQ2FkUFiwA0NhG7AEQ2BZsAMlsIBiYSMgsAMmI0ZhOBsjsAxDRrACJbAIQ2CwDEOwCENgSWAgsAVDsIBiYCMgsAArI7AFQ2CwACuwBSVhsAUlsIBisAQmYSCwBCVgZCOwAyVgZFBYIRsjIVkjILADJiNGYThZIyAgPLAFI0IjOLEJARQrsAVDLrAJKy2wFiywABM+sQkBFCstsB4ssAAWICCwBCawAiWwCENgIyAusAhDYEkjPDgusQkBFCstsB8ssAAWICCwBCawAiWwCENgIyAusAhDYEkjPDgjIC5GsAIlRlJYIDxZLrEJARQrLbAgLLAAFiAgsAQmsAIlsAhDYCMgLrAIQ2BJIzw4IyAuRrACJUZQWCA8WS6xCQEUKy2wISywABYgILAEJrACJbAIQ2AjIC6wCENgSSM8OCMgLkawAiVGUlggPFkjIC5GsAIlRlBYIDxZLrEJARQrLbAiLLAAFiCwDCNCILAIQ2AuICA8Ly6xCQEUKy2wIyywABYgsAwjQiCwCENgLiAgPC8jIC5GsAIlRlJYIDxZLrEJARQrLbAkLLAAFiCwDCNCILAIQ2AuICA8LyMgLkawAiVGUFggPFkusQkBFCstsCUssAAWILAMI0IgsAhDYC4gIDwvIyAuRrACJUZSWCA8WSMgLkawAiVGUFggPFkusQkBFCstsCYssAAWsAMlsAhDYLACJbAIQ2BJsABUWC4gPCMhG7ACJbAIQ2CwAiWwCENgSbgQAGOwBCWwAyVJY7AEJbAIQ2CwAyWwCENgSbgQAGNiIy4jICA8ijgjIVkusQkBFCstsCcssAAWsAMlsAhDYLACJbAIQ2BJsABUWC4gPCMhG7ACJbAIQ2CwAiWwCENgSbgQAGOwBCWwAyVJY7AEJbAIQ2CwAyWwCENgSbgQAGNiIy4jICA8ijgjIVkjIC5GsAIlRlJYIDxZLrEJARQrLbAoLLAAFrADJbAIQ2CwAiWwCENgSbAAVFguIDwjIRuwAiWwCENgsAIlsAhDYEm4EABjsAQlsAMlSWOwBCWwCENgsAMlsAhDYEm4EABjYiMuIyAgPIo4IyFZIyAuRrACJUZQWCA8WS6xCQEUKy2wKSywABawAyWwCENgsAIlsAhDYEmwAFRYLiA8IyEbsAIlsAhDYLACJbAIQ2BJuBAAY7AEJbADJUljsAQlsAhDYLADJbAIQ2BJuBAAY2IjLiMgIDyKOCMhWSMgLkawAiVGUlggPFkjIC5GsAIlRlBYIDxZLrEJARQrLbAqLLAAFiCwCENgsAxDIC6wCENgSSBgsCBgZrCAYiMgIDyKOC6xCQEUKy2wKyywABYgsAhDYLAMQyAusAhDYEkgYLAgYGawgGIjICA8ijgjIC5GsAIlRlJYIDxZLrEJARQrLbAsLLAAFiCwCENgsAxDIC6wCENgSSBgsCBgZrCAYiMgIDyKOCMgLkawAiVGUFggPFkusQkBFCstsC0ssAAWILAIQ2CwDEMgLrAIQ2BJIGCwIGBmsIBiIyAgPIo4IyAuRrACJUZSWCA8WSMgLkawAiVGUFggPFkusQkBFCstsC4sKy2wLyywLiqwARUwLbgAMCxLuAAIUFixAQGOWbgB/4W4AEQduQAIAANfXi24ADEsICBFaUSwAWAtuAAyLLgAMSohLbgAMywgRrADJUZSWCNZIIogiklkiiBGIGhhZLAEJUYgaGFkUlgjZYpZLyCwAFNYaSCwAFRYIbBAWRtpILAAVFghsEBlWVk6LbgANCwgRrAEJUZSWCOKWSBGIGphZLAEJUYgamFkUlgjilkv/S24ADUsSyCwAyZQWFFYsIBEG7BARFkbISEgRbDAUFiwwEQbIVlZLbgANiwgIEVpRLABYCAgRX1pGESwAWAtuAA3LLgANiotuAA4LEsgsAMmU1iwQBuwAFmKiiCwAyZTWCMhsICKihuKI1kgsAMmU1gjIbgAwIqKG4ojWSCwAyZTWCMhuAEAioobiiNZILADJlNYIyG4AUCKihuKI1kguAADJlNYsAMlRbgBgFBYIyG4AYAjIRuwAyVFIyEjIVkbIVlELbgAOSxLU1hFRBshIVktuAAwKwG6AAUAFAAyKwG/ABgAOQAvACQAGgAQAAAAOCsAvwAUAHUAYABKADYAHAAAADgrvwAVAFcARwA4ACwAHAAAADgrvwAWAGIAUAA+AC4AHAAAADgrvwAXAD0AMgAnABwAEQAAADgrALoAGQAFADcruAATIEV9aRhEugCQABsAAXO6AFAAGwABdLoAHwAdAAFzugBfAB0AAXO6AG8AHQABc7oAnwAdAAFzugDPAB0AAXO6AN8AHQABc7oAPwAdAAF0ugBPAB0AAXS6AF8AHQABdLoATwAfAAF0ugB/ACEAAXO6AL8AIQABc7oATwAhAAF0ugA/ACEAAXO5CAAIAGMgsAojQiCwACNwsBBFICCwKGBmIIpVWLAKQ2MjYrAJI0KzBQYDAiuzBwwDAiuzDRIDAisbsQkKQ0JZsgsoAkVSQrMHDAQCKwAAALwAWAC8AMMAWABYBd0AAAXuBAAAAP4ABeP/5wXuBBn/5/3nAEIAYQCCAHQAuQDIAAAAKf4AABoFwwAtBAAAGQYZAAIAAAAAAA4ArgADAAEECQAAAcQAAAADAAEECQABABIBxAADAAEECQACAA4B1gADAAEECQADADgB5AADAAEECQAEACICHAADAAEECQAFABoCPgADAAEECQAGACICWAADAAEECQAHAGgCegADAAEECQAIAC4C4gADAAEECQAJAE4DEAADAAEECQALACIDXgADAAEECQAMACIDXgADAAEECQANAcQDgAADAAEECQAOADQFRABDAG8AcAB5AHIAaQBnAGgAdAAgACgAYwApACAAMgAwADEAMQAsACAAQwB5AHIAZQBhAGwAIAAoAHcAdwB3AC4AYwB5AHIAZQBhAGwALgBvAHIAZwApAA0AdwBpAHQAaAAgAFIAZQBzAGUAcgB2AGUAZAAgAEYAbwBuAHQAIABOAGEAbQBlACAAIgBBAG0AZQB0AGgAeQBzAHQAYQAiAC4ADQANAFQAaABpAHMAIABGAG8AbgB0ACAAUwBvAGYAdAB3AGEAcgBlACAAaQBzACAAbABpAGMAZQBuAHMAZQBkACAAdQBuAGQAZQByACAAdABoAGUAIABTAEkATAAgAE8AcABlAG4AIABGAG8AbgB0ACAATABpAGMAZQBuAHMAZQAsAA0AVgBlAHIAcwBpAG8AbgAgADEALgAxAC4AIABUAGgAaQBzACAAbABpAGMAZQBuAHMAZQAgAGkAcwAgAGEAdgBhAGkAbABhAGIAbABlACAAdwBpAHQAaAAgAGEAIABGAEEAUQAgAGEAdAA6AA0AaAB0AHQAcAA6AC8ALwBzAGMAcgBpAHAAdABzAC4AcwBpAGwALgBvAHIAZwAvAE8ARgBMAEEAbQBlAHQAaAB5AHMAdABhAFIAZQBnAHUAbABhAHIAMQAuADAAMAAzADsAVQBLAFcATgA7AEEAbQBlAHQAaAB5AHMAdABhAC0AUgBlAGcAdQBsAGEAcgBBAG0AZQB0AGgAeQBzAHQAYQAgAFIAZQBnAHUAbABhAHIAVgBlAHIAcwBpAG8AbgAgADEALgAwADAAMwBBAG0AZQB0AGgAeQBzAHQAYQAtAFIAZQBnAHUAbABhAHIAQQBtAGUAdABoAHkAcwB0AGEAIABpAHMAIABhACAAdAByAGEAZABlAG0AYQByAGsAIABvAGYAIABDAHkAcgBlAGEAbAAgACgAdwB3AHcALgBjAHkAcgBlAGEAbAAuAG8AcgBnACkALgBDAHkAcgBlAGEAbAAgACgAdwB3AHcALgBjAHkAcgBlAGEAbAAuAG8AcgBnACkASwBvAG4AcwB0AGEAbgB0AGkAbgAgAFYAaQBuAG8AZwByAGEAZABvAHYALAAgAEEAbABlAHgAZQBpACAAVgBhAG4AeQBhAHMAaABpAG4AaAB0AHQAcAA6AC8ALwBjAHkAcgBlAGEAbAAuAG8AcgBnAEMAbwBwAHkAcgBpAGcAaAB0ACAAKABjACkAIAAyADAAMQAxACwAIABDAHkAcgBlAGEAbAAgACgAdwB3AHcALgBjAHkAcgBlAGEAbAAuAG8AcgBnACkADQB3AGkAdABoACAAUgBlAHMAZQByAHYAZQBkACAARgBvAG4AdAAgAE4AYQBtAGUAIAAiAEEAbQBlAHQAaAB5AHMAdABhACIALgANAA0AVABoAGkAcwAgAEYAbwBuAHQAIABTAG8AZgB0AHcAYQByAGUAIABpAHMAIABsAGkAYwBlAG4AcwBlAGQAIAB1AG4AZABlAHIAIAB0AGgAZQAgAFMASQBMACAATwBwAGUAbgAgAEYAbwBuAHQAIABMAGkAYwBlAG4AcwBlACwAIABWAGUAcgBzAGkAbwBuACAAMQAuADEALgAgAFQAaABpAHMAIABsAGkAYwBlAG4AcwBlACAAaQBzACAAYQB2AGEAaQBsAGEAYgBsAGUAIAB3AGkAdABoACAAYQAgAEYAQQBRACAAYQB0ADoADQBoAHQAdABwADoALwAvAHMAYwByAGkAcAB0AHMALgBzAGkAbAAuAG8AcgBnAC8ATwBGAEwAaAB0AHQAcAA6AC8ALwBzAGMAcgBpAHAAdABzAC4AcwBpAGwALgBvAHIAZwAvAE8ARgBMAAAAAgAAAAAAAP9mAGYAAAAAAAAAAAAAAAAAAAAAAAAAAADeAAAAAQACAQIAAwAEAAUABgAHAAgACQAKAAsADAANAA4ADwAQABEAEgATABQAFQAWABcAGAAZABoAGwAcAB0AHgAfACAAIQAiACMAJAAlACYAJwAoACkAKgArACwALQAuAC8AMAAxADIAMwA0ADUANgA3ADgAOQA6ADsAPAA9AD4APwBAAEEAQgBDAEQARQBGAEcASABJAEoASwBMAE0ATgBPAFAAUQBSAFMAVABVAFYAVwBYAFkAWgBbAFwAXQBeAF8AYABhAQMAowCEAIUAvQCWAOgAhgCOAIsAnQCpAKQBBACKANoAgwCTAPIA8wCNAJcAiADDAN4A8QCeAKoA9QD0APYAogCtAMkAxwCuAGIAYwCQAGQAywBlAMgAygDPAMwAzQDOAOkAZgDTANAA0QCvAGcA8ACRANYA1ADVAGgA6wDtAIkAagBpAGsAbQBsAG4AoABvAHEAcAByAHMAdQB0AHYAdwDqAHgAegB5AHsAfQB8ALgAoQB/AH4AgACBAOwA7gC6ANcAsACxANgA4QDdANkAsgCzALYAtwDEALQAtQDFAIcAvgC/ALwBBQEGAO8BBwEIAQkBCgELB25vQnJlYWsHdW5pMDBBMAd1bmkwMEFEB3VuaTIwNzQERXVybwpncmF2ZS5jYXNlCmFjdXRlLmNhc2UPY2lyY3VtZmxleC5jYXNlDWRpZXJlc2lzLmNhc2UKdGlsZGUuY2FzZQAAAAAAAgAIAA7//wAP");
+          }
+        </style></g></svg>
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/image/logo_icon.svg b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/image/logo_icon.svg
new file mode 100644
index 0000000000000000000000000000000000000000..c46dd3b5407c1f82dce4f6096acf8c8a30a6cfba
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/image/logo_icon.svg
@@ -0,0 +1,31 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg id="_图层_2" data-name="图层 2" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 34.59 36">
+  <defs>
+    <style>
+      .cls-1 {
+        fill: #36569b;
+      }
+
+      .cls-2 {
+        fill: #1b3882;
+      }
+
+      .cls-3 {
+        fill: #5878b4;
+      }
+    </style>
+  </defs>
+  <g id="_图层_1-2" data-name="图层 1">
+    <g>
+      <g id="_3" data-name="3">
+        <path class="cls-3" d="m16.53,22.65l-6.37,3.07,5.27-.16,1.1-2.91Zm-4.19,10.95l1.12-2.91-5.27.17,4.15,2.74Zm9.3-.29l6.37-3.07-5.27.16-1.1,2.91Zm4.19-10.95l-1.12,2.91,5.27-.17-4.15-2.74Zm5.72,3.81l-7.08.23-1.73-1.14,1.5-3.95-2.06-1.36-3.16,1.53-1.48,3.89-2.67,1.29-7.14.23-3.16,1.53,2.07,1.36,7.13-.23h0s1.69,1.11,1.69,1.11l-1.51,3.98,2.06,1.36,3.16-1.53,1.5-3.95h0s2.56-1.24,2.56-1.24h0s7.23-.24,7.23-.24l3.16-1.53-2.06-1.36Zm-11.29,2.56c-.99.48-2.31.52-2.96.1-.65-.42-.37-1.15.62-1.63.99-.48,2.31-.52,2.96-.1.65.42.37,1.15-.62,1.63Z"/>
+      </g>
+      <g id="_2" data-name="2">
+        <path class="cls-1" d="m33.5,19.84l-1.26-6.51-1.46,1.88,2.72,4.63Zm-6.05-14.69l-4.16-2.74,2.71,4.64,1.45-1.89Zm-6.73.58l1.26,6.51,1.46-1.88-2.72-4.63Zm6.05,14.69l4.16,2.74-2.71-4.64-1.45,1.89Zm7.19,1.91l-3.63-6.2h0s-.53-2.74-.53-2.74l1.96-2.56-.63-3.23-2.07-1.36-1.96,2.56-1.69-1.11-3.71-6.33-2.07-1.36.63,3.23,3.68,6.28h0s.51,2.62.51,2.62h0s-1.99,2.6-1.99,2.6l.63,3.23,2.06,1.36,1.95-2.54,1.73,1.14,3.69,6.29,2.07,1.36-.63-3.23Zm-6.47-7.7c-.65-.42-1.33-1.59-1.52-2.6-.2-1.01.17-1.49.81-1.06.65.42,1.33,1.59,1.52,2.6.2,1.01-.17,1.49-.81,1.06Z"/>
+      </g>
+      <g id="_1" data-name="1">
+        <path class="cls-2" d="m11.96,2.82l-6.37,3.07,3.81,1.74,2.55-4.81ZM1.07,14.37l1.26,6.53,2.56-4.8-3.82-1.73Zm7.99,9.59l6.37-3.07-3.81-1.74-2.55,4.81Zm10.89-11.55l-1.26-6.53-2.56,4.8,3.82,1.73Zm.45,2.53l-5.13-2.32h0s-.53-2.71-.53-2.71l3.47-6.53-.63-3.24-3.16,1.53-3.42,6.43-2.67,1.29h0s-5.17-2.34-5.17-2.34l-3.16,1.53.63,3.24,5.17,2.33.51,2.65h0s-3.49,6.57-3.49,6.57l.63,3.24,3.16-1.53,3.46-6.52,2.56-1.24h0s5.24,2.37,5.24,2.37l3.16-1.53-.63-3.24Zm-9.52.24c-.99.48-1.95.04-2.14-.97-.2-1.01.44-2.22,1.43-2.69.99-.48,1.95-.04,2.14.97.2,1.01-.44,2.22-1.43,2.7Z"/>
+      </g>
+    </g>
+  </g>
+</svg>
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/js/custom.js b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/js/custom.js
new file mode 100644
index 0000000000000000000000000000000000000000..84da69d47fae8e8994685aca3b99151d01a77978
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_static/js/custom.js
@@ -0,0 +1,10 @@
+var collapsedSections = [];
+
+$(document).ready(function () {
+  $('.model-summary').DataTable({
+    "stateSave": false,
+    "lengthChange": false,
+    "pageLength": 20,
+    "order": []
+  });
+});
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/404.html b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/404.html
new file mode 100644
index 0000000000000000000000000000000000000000..64910175d5d69946845b04d5e6a378de205e8388
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/404.html
@@ -0,0 +1,18 @@
+{% extends "layout.html" %}
+
+{% block body %}
+
+<h1>Page Not Found</h1>
+<p>
+  The page you are looking for cannot be found.
+</p>
+<p>
+  If you just switched documentation versions, it is likely that the page you were on is moved. You can look for it in
+  the content table left, or go to <a href="{{ pathto(root_doc) }}">the homepage</a>.
+</p>
+<!-- <p>
+  If you cannot find documentation you want, please <a
+    href="">open an issue</a> to tell us!
+</p> -->
+
+{% endblock %}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/autosummary/class.rst b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/autosummary/class.rst
new file mode 100644
index 0000000000000000000000000000000000000000..4c3a7a9abf5c5b14ac3ef3b00a2f070480295358
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/autosummary/class.rst
@@ -0,0 +1,13 @@
+.. role:: hidden
+    :class: hidden-section
+.. currentmodule:: {{ module }}
+
+
+{{ name | underline}}
+
+.. autoclass:: {{ name }}
+    :members:
+
+..
+  autogenerated from _templates/autosummary/class.rst
+  note it does not have :inherited-members:
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/callable.rst b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/callable.rst
new file mode 100644
index 0000000000000000000000000000000000000000..3a7b9d2b96c76dfa3eb1d8bef56f58f219fe7760
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/_templates/callable.rst
@@ -0,0 +1,14 @@
+.. role:: hidden
+    :class: hidden-section
+.. currentmodule:: {{ module }}
+
+
+{{ name | underline}}
+
+.. autoclass:: {{ name }}
+    :members:
+    :special-members: __call__
+
+..
+  autogenerated from _templates/callable.rst
+  note it does not have :inherited-members:
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/conf.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/conf.py
new file mode 100644
index 0000000000000000000000000000000000000000..9228347854208e213271aba5e3a2254d05a8a65e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/conf.py
@@ -0,0 +1,242 @@
+# flake8: noqa
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+
+import ast
+import os
+import subprocess
+import sys
+
+import pytorch_sphinx_theme
+from sphinx.builders.html import StandaloneHTMLBuilder
+
+sys.path.insert(0, os.path.abspath('../../'))
+
+# -- Project information -----------------------------------------------------
+
+project = 'VLMEvalKit'
+copyright = '2023, VLMEvalKit'
+author = 'VLMEvalKit Authors'
+
+# The full version, including alpha/beta/rc tags
+version_file = '../../vlmeval/__init__.py'
+
+
+def get_version():
+    with open(version_file, 'r') as f:
+        file_content = f.read()
+    # Parse the file content into an abstract syntax tree (AST)
+    tree = ast.parse(file_content, filename=version_file)
+
+    # Iterate through the body of the AST, looking for an assignment to __version__
+    for node in tree.body:
+        if isinstance(node, ast.Assign):
+            for target in node.targets:
+                if isinstance(target, ast.Name) and target.id == '__version__':
+                    return node.value.s
+    raise ValueError('__version__ not found')
+
+
+release = get_version()
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.napoleon',
+    'sphinx.ext.viewcode',
+    'myst_parser',
+    'sphinx_copybutton',
+    'sphinx_tabs.tabs',
+    'notfound.extension',
+    'sphinxcontrib.jquery',
+    'sphinx_design',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+source_suffix = {
+    '.rst': 'restructuredtext',
+    '.md': 'markdown',
+}
+
+language = 'cn'
+
+# The master toctree document.
+root_doc = 'index'
+html_context = {
+    'github_version': 'latest',
+}
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'pytorch_sphinx_theme'
+html_theme_path = [pytorch_sphinx_theme.get_html_theme_path()]
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+# yapf: disable
+html_theme_options = {
+    'menu': [
+        {
+            'name': 'GitHub',
+            'url': 'https://github.com/open-compass/VLMEvalKit'
+        },
+    ],
+    # Specify the language of shared menu
+    'menu_lang': 'cn',
+    # Disable the default edit on GitHub
+    'default_edit_on_github': False,
+}
+# yapf: enable
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+html_css_files = [
+    'https://cdn.datatables.net/v/bs4/dt-1.12.1/datatables.min.css',
+    'css/readthedocs.css'
+]
+html_js_files = [
+    'https://cdn.datatables.net/v/bs4/dt-1.12.1/datatables.min.js',
+    'js/custom.js'
+]
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'vlmevalkitdoc'
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (root_doc, 'vlmevalkit.tex', 'VLMEvalKit Documentation', author,
+     'manual'),
+]
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [(root_doc, 'vlmevalkit', 'VLMEvalKit Documentation', [author],
+              1)]
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (root_doc, 'vlmevalkit', 'VLMEvalKit Documentation', author,
+     'VLMEvalKit Authors', 'AGI evaluation toolbox and benchmark.',
+     'Miscellaneous'),
+]
+
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+# set priority when building html
+StandaloneHTMLBuilder.supported_image_types = [
+    'image/svg+xml', 'image/gif', 'image/png', 'image/jpeg'
+]
+
+# -- Extension configuration -------------------------------------------------
+# Ignore >>> when copying code
+copybutton_prompt_text = r'>>> |\.\.\. '
+copybutton_prompt_is_regexp = True
+
+# Auto-generated header anchors
+myst_heading_anchors = 3
+# Enable "colon_fence" extension of myst.
+myst_enable_extensions = ['colon_fence', 'dollarmath']
+
+# Configuration for intersphinx
+intersphinx_mapping = {
+    'python': ('https://docs.python.org/3', None),
+    'numpy': ('https://numpy.org/doc/stable', None),
+    'torch': ('https://pytorch.org/docs/stable/', None),
+    'mmengine': ('https://mmengine.readthedocs.io/en/latest/', None),
+    'transformers':
+    ('https://huggingface.co/docs/transformers/main/en/', None),
+}
+napoleon_custom_sections = [
+    # Custom sections for data elements.
+    ('Meta fields', 'params_style'),
+    ('Data fields', 'params_style'),
+]
+
+# Disable docstring inheritance
+autodoc_inherit_docstrings = False
+# Mock some imports during generate API docs.
+autodoc_mock_imports = ['rich', 'attr', 'einops']
+# Disable displaying type annotations, these can be very verbose
+autodoc_typehints = 'none'
+
+# The not found page
+notfound_template = '404.html'
+
+
+def builder_inited_handler(app):
+    subprocess.run(['./cp_origin_docs.sh'])
+
+
+def setup(app):
+    app.connect('builder-inited', builder_inited_handler)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/cp_origin_docs.sh b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/cp_origin_docs.sh
new file mode 100644
index 0000000000000000000000000000000000000000..1e728323684a0aad1571eb392871d6c5de6644fc
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/cp_origin_docs.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+# Copy *.md files from docs/ if it doesn't have a Chinese translation
+
+for filename in $(find ../en/ -name '*.md' -printf "%P\n");
+do
+    mkdir -p $(dirname $filename)
+    cp -n ../en/$filename ./$filename
+done
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/docutils.conf b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/docutils.conf
new file mode 100644
index 0000000000000000000000000000000000000000..0c00c84688701117f231fd0c8ec295fb747b7d8f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/docutils.conf
@@ -0,0 +1,2 @@
+[html writers]
+table_style: colwidths-auto
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/index.rst b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/index.rst
new file mode 100644
index 0000000000000000000000000000000000000000..5147a23b2052044664a987910d4bf04ccf286d14
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/docs/zh-CN/index.rst
@@ -0,0 +1,49 @@
+欢迎来到 VLMEvalKit 中文教程！
+==========================================
+
+VLMEvalKit 上手路线
+-------------------------------
+
+为了用户能够快速上手，我们推荐以下流程：
+
+- 对于想要使用 VLMEvalKit 的用户，我们推荐先阅读 开始你的第一步_ 部分来设置环境，并启动一个迷你实验熟悉流程。
+
+- 若您想进行更多模块的自定义，例如增加数据集和模型，我们提供了 进阶教程_ 。
+
+我们始终非常欢迎用户的 PRs 和 Issues 来完善 VLMEvalKit！
+
+.. _快速开始:
+.. toctree::
+   :maxdepth: 1
+   :caption: 快速开始
+
+   Quickstart.md
+
+
+.. .. _教程:
+.. .. toctree::
+..    :maxdepth: 1
+..    :caption: 教程
+
+..    user_guides/framework_overview.md
+
+.. _进阶教程:
+.. toctree::
+   :maxdepth: 1
+   :caption: 进阶教程
+
+   Development.md
+   ConfigSystem.md
+
+.. .. _其他说明:
+.. .. toctree::
+..    :maxdepth: 1
+..    :caption: 其他说明
+
+..    notes/contribution_guide.md
+
+索引与表格
+==================
+
+* :ref:`genindex`
+* :ref:`search`
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/cg_av_counting.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/cg_av_counting.py
new file mode 100644
index 0000000000000000000000000000000000000000..6ebcdae61371e4520d586f5c0af5657fd370b0ce
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/cg_av_counting.py
@@ -0,0 +1,415 @@
+import json
+import os
+import os.path as osp
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+import portalocker
+from huggingface_hub import snapshot_download
+from PIL import Image
+
+from vlmeval.smp import (dump, get_cache_path, get_file_extension, get_intermediate_file_path,
+                         load, md5, modelscope_flag_set)
+from ..utils.cgbench import post_process, unzip_hf_zip
+from ..video_base import VideoBaseDataset
+from .utils import get_timestampes, rating_func
+
+
+class CGAVCounting(VideoBaseDataset):
+
+    dataset = "CG-AV-Counting"
+
+    TYPE = "Video-Counting"
+
+    MD5 = "d1cd8486353ab85178098d443264a7d0"
+
+    SYS = ""
+
+    def __init__(
+        self,
+        dataset="CG-AV-Counting",
+        use_frame_time=False,
+        nframe=0,
+        fps=-1,
+    ):
+        super().__init__(dataset=dataset, nframe=nframe, fps=fps)
+        self.use_frame_time = use_frame_time
+        self.dataset_name = dataset
+        self.frame_tmpl_clue = 'frame-{}.jpg'
+
+    @classmethod
+    def supported_datasets(cls):
+        return ["CGAVCounting"]
+
+    def frame_paths_clue(self, video, timestamp_list):
+        frame_root = osp.join(self.frame_root, video)
+        os.makedirs(frame_root, exist_ok=True)
+        return [osp.join(frame_root, self.frame_tmpl_clue.format(i)) for i in timestamp_list]
+
+    def save_video_frames_clue(self, video, uid, timestamp_list):
+        if type(uid) is not str:
+            uid = str(uid)
+        import decord
+        frame_paths = self.frame_paths_clue(uid, timestamp_list)
+        flag = np.all([osp.exists(p) for p in frame_paths])
+        if flag:
+            frame = Image.open(frame_paths[0])
+            return frame_paths, frame.width, frame.height
+        vid_path = osp.join(self.data_root, video)
+        vid = decord.VideoReader(vid_path)
+        frames = []
+        # 获取视频的帧率
+        fps = vid.get_avg_fps()
+        lock_path = osp.splitext(vid_path)[0] + '.lock'
+        with portalocker.Lock(lock_path, 'w', timeout=30):
+            for timestamp_sec in timestamp_list:
+                # 计算视频帧对应的索引
+                frame_idx = int(timestamp_sec * fps)
+
+                # 获取对应帧
+                frame = vid[frame_idx]
+
+                # 将帧转换为PIL图像
+                img = Image.fromarray(frame.asnumpy())
+                frames.append(img)
+            for im, pth in zip(frames, frame_paths):
+                if not osp.exists(pth):
+                    im.save(pth)
+        return frame_paths, frames[0].width, frames[0].height
+
+    def format_time(self, t):
+        return f"{t:.2f}"
+
+    def get_output_filename(self, item):
+        video_id = Path(item["video"]).stem
+        start_str = self.format_time(item["query_interval"][0])
+        end_str = self.format_time(item["query_interval"][1])
+        return f"{video_id}_{start_str}_{end_str}.mp4"
+
+    def prepare_dataset(self, dataset_name="CG-AV-Counting", repo_id="CG-Bench/CG-AV-Counting"):
+
+        def check_integrity(pth):
+            data_file = osp.join(pth, f"{dataset_name}.tsv")
+
+            if not os.path.exists(data_file):
+                return False
+
+            if md5(data_file) != self.MD5:
+                return False
+            data = load(data_file)
+            for video_pth in data["video"]:
+                if not osp.exists(osp.join(pth, video_pth)):
+                    return False
+            return True
+
+        cache_path = get_cache_path(repo_id)
+
+        if cache_path is not None and check_integrity(cache_path):
+            dataset_path = cache_path
+        else:
+
+            def generate_tsv(pth):
+
+                tsv_file = osp.join(pth, f"{dataset_name}.tsv")
+
+                task_modes = ["long_acc", "ref_acc", "clue_acc"]
+                all_data = []
+                for task_mode in task_modes:
+                    with open(osp.join(pth, "cg-av-counting.json"), "r") as f:
+                        data_file = pd.DataFrame(json.load(f))
+
+                    data_file = data_file.assign(index=range(len(data_file)))
+                    data_file["video_uid"] = data_file["video"].replace(".mp4", "")
+                    data_file["video"] = data_file["video"].apply(lambda x: f"cg_videos_720p/{x}")
+
+                    data_file["ref_video_path"] = ""
+                    data_file["ref_video_uid"] = ""
+
+                    if task_mode in ["ref_acc"]:
+                        data_file["ref_video_path"] = data_file.apply(
+                            lambda row: f"ref_videos/{self.get_output_filename(row)}", axis=1
+                        )
+                        data_file["ref_video_uid"] = data_file["ref_video_path"].apply(
+                            lambda x: x.split("/")[-1].replace(".mp4", ""))
+
+                    data_file["task_mode"] = task_mode
+
+                    if task_mode == "clue_acc":
+                        data_file["answer"] = data_file["clue"].apply(json.dumps)
+
+                    data_file = data_file[
+                        [
+                            "index",
+                            "video_uid",
+                            "video",
+                            "ref_video_path",
+                            "ref_video_uid",
+                            "question",
+                            "answer",
+                            "type",
+                            "category",
+                            "task_mode"
+                        ]
+                    ]
+
+                    all_data.append(data_file)
+
+                final_data = pd.concat(all_data, ignore_index=True)
+                final_data["index"] = range(len(final_data))
+                final_data.to_csv(tsv_file, sep="\t", index=False)
+            dataset_path = cache_path
+
+            if modelscope_flag_set():
+                from modelscope import dataset_snapshot_download
+
+                dataset_path = dataset_snapshot_download(dataset_id=repo_id)
+            else:
+                dataset_path = snapshot_download(repo_id=repo_id, repo_type="dataset")
+
+            unzip_hf_zip(dataset_path)
+
+            generate_tsv(dataset_path)
+
+        tsv_file = osp.join(dataset_path, f"{dataset_name}.tsv")
+
+        return dict(data_file=tsv_file, root=dataset_path)
+
+    def build_prompt(self, line, video_llm):
+        if isinstance(line, int):
+            assert line < len(self)
+            line = self.data.iloc[line]
+        task_mode = line["task_mode"]
+        assert task_mode in ["long_acc", "clue_acc", "ref_acc"]
+        if task_mode == "long_acc":
+            user_prompt = ""
+            message = []
+            video_path = line["video"]
+            if video_llm:
+                message.append(dict(type="video", value=osp.join(self.data_root, video_path)))
+            else:
+                image_paths, frame_indices, vid_fps = self.save_video_frames(
+                    video_path, uid=line["video_uid"], num_frames=self.nframe, fps=self.fps
+                )
+                message.extend(dict(type="image", value=im) for im in image_paths)
+
+                if self.use_frame_time:
+                    user_prompt += get_timestampes(frame_indices, vid_fps)
+
+            user_prompt += (
+                f"Please answer the question '{line['question']}' with a number. Just output the number itself, "
+                "don't output anything else."
+            )
+            message.append(dict(type="text", value=user_prompt))
+        elif task_mode == "ref_acc":
+            user_prompt = ""
+            message = []
+            video_path = line["ref_video_path"]
+            if video_llm:
+                message.append(dict(type="video", value=osp.join(self.data_root, video_path)))
+            else:
+                image_paths, frame_indices, vid_fps = self.save_video_frames(
+                    video_path, uid=line["ref_video_uid"], num_frames=self.nframe, fps=self.fps
+                )
+                message.extend(dict(type="image", value=im) for im in image_paths)
+
+                if self.use_frame_time:
+                    user_prompt += get_timestampes(frame_indices, vid_fps)
+            user_prompt += (
+                f"Please answer the question '{line['question']}' with a number. Just output the number itself, "
+                "don't output anything else."
+            )
+            message.append(dict(type="text", value=user_prompt))
+        elif task_mode == "clue_acc":
+            if line["category"] == "event":
+                user_prompt = ""
+                message = []
+                video_path = line["video"]
+                if video_llm:
+                    message.append(dict(type="video", value=osp.join(self.data_root, video_path)))
+                else:
+                    image_paths, frame_indices, vid_fps = self.save_video_frames(
+                        video_path, uid=line["video_uid"], num_frames=self.nframe, fps=self.fps
+                    )
+                    message.extend(dict(type="image", value=im) for im in image_paths)
+                    user_prompt += get_timestampes(frame_indices, vid_fps)
+
+                user_prompt += (
+                    f"Watch the video and provide your answer to the question '{line['question']}', "
+                    "including the start and end timestamps for each event."
+                    "Format your answer in JSON, enclosed in <answer> and </answer> tags. "
+                    "The output should look like this: <answer>[[\"start_time\", \"end_time\"], ...]</answer>. "
+                    "Ensure each timestamp is in seconds (e.g., 'xx.xx')."
+                )
+                message.append(dict(type="text", value=user_prompt))
+            elif line["category"] == "object":
+                user_prompt = ""
+                message = []
+                video_path = line["video"]
+                clue_timestamp_list = []
+                for clue in json.loads(line["answer"]):
+                    if clue["timestamp"] not in clue_timestamp_list:
+                        clue_timestamp_list.append(clue["timestamp"])
+                image_paths, width, height = self.save_video_frames_clue(
+                    video_path, uid=line["video_uid"], timestamp_list=clue_timestamp_list
+                )
+                message.append(
+                    dict(type="text", value=f"There are {len(image_paths)} frames in the size of {width}x{height}"))
+                for idx, im in enumerate(image_paths):
+                    message.append(dict(type="text", value=f"Frame{idx + 1}:"))
+                    message.append(dict(type="image", value=im))
+                user_prompt += (
+                    f"Answer the question '{line['question']}', "
+                    "including the bounding box for the query object in the first frame "
+                    "where it appears. For subsequent frames where the object appears, "
+                    "do not provide the bounding box again. "
+                    "Format your answer in JSON, enclosed within <answer> and </answer> tags. "
+                    "The output should look like this: "
+                    "<answer>{\"Frame1\": [[x_min, y_min, x_max, y_max]], \"Frame2\": [...],...}</answer>. "
+                    "In the output, each frame should either contain the bounding box of the object "
+                    "(if it appears for the first time in that frame) or an empty list `[]` "
+                    "(if the object does not appear or it has already been labeled in a previous frame). "
+                    "Ensure that bounding boxes are listed as [x_min, y_min, x_max, y_max]."
+                )
+                message.append(dict(type="text", value=user_prompt))
+            elif line["category"] == "attribute":
+                user_prompt = ""
+                message = []
+                video_path = line["video"]
+                clue_timestamp_list = []
+                for clue_ in json.loads(line["answer"]):
+                    for clue in clue_:
+                        if clue["timestamp"] not in clue_timestamp_list:
+                            clue_timestamp_list.append(clue["timestamp"])
+                image_paths, width, height = self.save_video_frames_clue(
+                    video_path, uid=line["video_uid"], timestamp_list=clue_timestamp_list
+                )
+                message.append(dict(
+                    type="text",
+                    value=f"There are {len(image_paths)} frames in the size of {width}x{height}"))
+                for idx, im in enumerate(image_paths):
+                    message.append(dict(type="text", value=f"Frame{idx + 1}:"))
+                    message.append(dict(type="image", value=im))
+                user_prompt += (
+                    f"Answer the question '{line['question']}', clustering the objects according to the question. "
+                    "For each unique cluster, assign a unique label and return the bounding box for each object in "
+                    "the first frame where it appears. For subsequent frames where the object appears, "
+                    "do not output anything. "
+                    "Format your answer in JSON, enclosed within <answer> and </answer> tags. "
+                    "The output should look like this: "
+                    "<answer>{\"Frame 1\": [{\"bbox\": [x_min, y_min, x_max, y_max], 'label': \"Label 1\"}], "
+                    "\"Frame 2\": [...], ...}</answer>. "
+                    "In the output, each frame should either contain the bounding box and label for the object "
+                    "(if it appears for the first time in that frame) or an empty list `[]` "
+                    "(if the object has already been labeled or does not appear in that frame). "
+                    "The label should correspond to a unique object cluster according to the question."
+                )
+                message.append(dict(type="text", value=user_prompt))
+        print(message)
+        return message
+
+    def save_video_frames(self, video, uid, num_frames=8, fps=-1):
+
+        if type(uid) is not str:
+            uid = str(uid)
+        import decord
+        vid_path = osp.join(self.data_root, video)
+        vid = decord.VideoReader(vid_path)
+        vid_fps = vid.get_avg_fps()
+        n_frames = len(vid)
+
+        if num_frames > 0 and fps < 0:
+            step_size = len(vid) / (num_frames + 1)
+            indices = [int(i * step_size) for i in range(1, num_frames + 1)]
+
+            frame_paths = self.frame_paths(uid)
+        elif fps > 0:
+            total_duration = n_frames / vid_fps
+            required_frames = int(total_duration * fps)
+            step_size = vid_fps / fps
+            indices = [int(i * step_size) for i in range(required_frames)]
+            frame_paths = self.frame_paths_fps(uid, len(indices))
+
+        # Save and validate frames
+        valid_paths = []
+        valid_indices = []
+        lock_path = osp.splitext(vid_path)[0] + '.lock'
+        with portalocker.Lock(lock_path, 'w', timeout=30):
+            if not np.all([osp.exists(p) for p in frame_paths]):
+                images = [vid[i].asnumpy() for i in indices]
+                for i, (img_array, path) in enumerate(zip(images, frame_paths)):
+                    if osp.exists(path):
+                        try:
+                            with Image.open(path) as img:
+                                img.verify()
+                            valid_paths.append(path)
+                            valid_indices.append(indices[i])
+                        except Exception:
+                            continue
+                    else:
+                        try:
+                            img = Image.fromarray(img_array)
+                            img.save(path)
+                            img.verify()
+                            valid_paths.append(path)
+                            valid_indices.append(indices[i])
+                        except Exception:
+                            continue
+            else:
+                for i, path in enumerate(frame_paths):
+                    try:
+                        with Image.open(path) as img:
+                            img.verify()
+                        valid_paths.append(path)
+                        valid_indices.append(indices[i])
+                    except Exception:
+                        continue
+
+        return valid_paths, valid_indices, vid_fps
+
+    def evaluate(self, eval_file, **judge_kwargs):
+
+        assert get_file_extension(eval_file) in ['xlsx', 'json', 'tsv'], \
+            'data file should be an supported format (xlsx/json/tsv) file'
+
+        tgt_file = get_intermediate_file_path(eval_file, '_rating', 'json')
+        score_file = get_intermediate_file_path(eval_file, '_score', 'csv')
+
+        data = load(eval_file)
+
+        data_un = data[~pd.isna(data["prediction"])]
+        data_pred_na = data[pd.isna(data["prediction"])]
+
+        data_pred_na["score"] = -1
+
+        scores_df = data_un.apply(
+            lambda row: post_process(
+                response=row["prediction"],
+                right_answer=row["answer"],
+                task_mode=row["task_mode"],
+                category=row["category"]
+            ),
+            axis=1,
+            result_type='expand'
+        )
+
+        data_un = pd.concat([data_un, scores_df], axis=1)
+
+        data = pd.concat([data_pred_na, data_un])
+
+        rejected_count = (data["score"] == -1).sum()
+
+        print(
+            f"Among {len(data)} questions, "
+            f"failed to obtain prediction for {len(data_pred_na)} questions, "
+            f"failed to obtain the score for {rejected_count - len(data_pred_na)} questions. "
+            f"Those questions will be counted as -1 score in ALL rating, and will not be counted in VALID rating."
+        )
+
+        dump(data, score_file)
+
+        rating = rating_func(score_file)
+
+        dump(rating, tgt_file)
+
+        return rating
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/requirements.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/requirements.txt
new file mode 100644
index 0000000000000000000000000000000000000000..3c5cbc61f3a46f87a82fe359c84df7822555974d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/requirements.txt
@@ -0,0 +1,2 @@
+scipy
+word2number
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..353ff838ca5a1fae04bb9a081ea0f8e88ef83f61
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/CGAVCounting/utils.py
@@ -0,0 +1,423 @@
+import json
+import math
+import re
+import signal
+import zipfile
+from pathlib import Path
+
+import numpy as np
+from tqdm import tqdm
+
+from vlmeval.smp import load
+
+
+def rating_func(data_path):
+    df = load(data_path)
+
+    task_mode_fields = {
+        "long_acc": ["acc", "oboa", "mae", "rmse"],
+        "ref_acc": ["acc", "oboa", "mae", "rmse"],
+        "clue_acc": ["wcs", "ifa"],
+    }
+
+    rating = {}
+
+    for task_mode, fields in task_mode_fields.items():
+        sub_df = df[df["task_mode"] == task_mode]
+        for field in fields:
+            values = sub_df[field]
+            if field == "rmse":
+                # RMSE: sqrt(mean(x^2))
+                rmse_val = np.sqrt(values.mean())
+                rating[f"{task_mode}/rmse"] = round(rmse_val, 4)
+            else:
+                rating[f"{task_mode}/{field}"] = round(values.mean(), 4)
+
+    return rating
+
+
+def get_timestampes(frame_indices, fps):
+    seconds = list(map(lambda x: str(round(x / fps, 4)), frame_indices))
+    timestamps = ", ".join(seconds)
+    return "A total of {frame_num} frames are sampled. Their corresponding timestamps are:\n\n{timestamps}\n\n".format(
+        frame_num=len(frame_indices), timestamps=timestamps
+    )
+
+
+def time_str_to_seconds(time_str: str) -> float:
+    time_str = time_str.strip()
+    if '.' in time_str:
+        time_main, milliseconds = time_str.split('.')
+        milliseconds = float(f"0.{milliseconds}")
+    else:
+        time_main = time_str
+        milliseconds = 0.0
+
+    parts = list(map(int, time_main.split(":")))
+
+    if len(parts) == 2:
+        minutes, seconds = parts
+        total_seconds = minutes * 60 + seconds
+    elif len(parts) == 3:
+        hours, minutes, seconds = parts
+        total_seconds = hours * 3600 + minutes * 60 + seconds
+    else:
+        raise ValueError(f"Invalid time format: {time_str}")
+
+    return total_seconds + milliseconds
+
+
+def extract_outer_json(text):
+    stack = []
+    start_idx = None
+    opening = {'{': '}', '[': ']'}
+    closing = {'}': '{', ']': '['}
+
+    for i, char in enumerate(text):
+        if char in opening:
+            if not stack:
+                start_idx = i  # 最外层起点
+            stack.append(char)
+        elif char in closing:
+            if stack and stack[-1] == closing[char]:
+                stack.pop()
+                if not stack and start_idx is not None:
+                    candidate = text[start_idx:i + 1]
+                    try:
+                        return json.dumps(json.loads(candidate))
+                    except json.JSONDecodeError:
+                        continue  # 尝试下一个 JSON 块
+    return None
+
+
+def compute_tiou(t1, t2):
+    """Temporal IoU"""
+    inter_start = max(t1[0], t2[0])
+    inter_end = min(t1[1], t2[1])
+    inter = max(0.0, inter_end - inter_start)
+    union = max(t1[1], t2[1]) - min(t1[0], t2[0])
+    return inter / union if union > 0 else 0.0
+
+
+def compute_sIoU(box1, box2):
+    """
+    Complete IoU (sIoU) between two bounding boxes.
+    Args:
+        box1 (list or np.array): [x1, y1, x2, y2] of ground truth box
+        box2 (list or np.array): [x1, y1, x2, y2] of predicted box
+
+    Returns:
+        IoU (float): The IoU score between the two boxes.
+    """
+
+    # Ensure the coordinates are ordered: [min_x, min_y, max_x, max_y]
+    box1 = np.array([min(box1[0], box1[2]), min(box1[1], box1[3]),
+                     max(box1[0], box1[2]), max(box1[1], box1[3])])
+    box2 = np.array([min(box2[0], box2[2]), min(box2[1], box2[3]),
+                     max(box2[0], box2[2]), max(box2[1], box2[3])])
+
+    # Compute the intersection area
+    inter_x1 = max(box1[0], box2[0])
+    inter_y1 = max(box1[1], box2[1])
+    inter_x2 = min(box1[2], box2[2])
+    inter_y2 = min(box1[3], box2[3])
+
+    inter_area = max(0, inter_x2 - inter_x1) * max(0, inter_y2 - inter_y1)
+
+    # Compute areas of the individual boxes
+    area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])
+
+    # Compute union area
+    union = area1 + area2 - inter_area
+    iou = inter_area / union if union > 0 else 0.0
+
+    return iou
+
+
+def greedy_matching(gt_instances, pred_instances, iou_func):
+    """Greedy matching based on maximum IoU"""
+    unmatched_gt = set(range(len(gt_instances)))
+    unmatched_pred = set(range(len(pred_instances)))
+    matches = []
+
+    while unmatched_gt and unmatched_pred:
+        max_iou = -1
+        best_match = None
+        for gt_idx in unmatched_gt:
+            for pred_idx in unmatched_pred:
+                iou = iou_func(gt_instances[gt_idx], pred_instances[pred_idx])
+                if iou > max_iou:
+                    max_iou = iou
+                    best_match = (gt_idx, pred_idx)
+
+        if best_match:
+            gt_idx, pred_idx = best_match
+            matches.append((gt_idx, pred_idx))
+            unmatched_gt.remove(gt_idx)
+            unmatched_pred.remove(pred_idx)
+
+    return matches
+
+
+def compute_cluster_pair_wcs(gt, pred, iou_type):
+    if iou_type == 'tIoU':
+        loc_sum = 0.0
+        for g in gt:
+            loc_sum += max([compute_tiou(g, p) for p in pred] or [0.0])
+        loc_acc = loc_sum / len(gt) if gt else 0.0
+        count_penalty = 1.0 - abs(len(pred) - len(gt)) / max(len(gt), 1)
+        # count_penalty = 1.0
+        return math.sqrt(loc_acc * max(0, count_penalty))
+
+    elif iou_type == 'sIoU':
+        # group by frame index
+        from collections import defaultdict
+        gt_by_f = defaultdict(list)
+        pred_by_f = defaultdict(list)
+        for f, box in gt:
+            gt_by_f[f].append(box)
+        for f, box in pred:
+            pred_by_f[f].append(box)
+
+        all_f = set(gt_by_f) | set(pred_by_f)
+        wcs = 0.0
+        for f in all_f:
+            gt_f = gt_by_f.get(f, [])
+            pred_f = pred_by_f.get(f, [])
+            matches = greedy_matching(gt_f, pred_f, compute_sIoU)
+            loc_sum = sum([compute_sIoU(gt_f[i], pred_f[j]) for i, j in matches])
+            loc_acc = loc_sum / len(gt_f) if gt_f else 0.0
+            count_penalty = 1.0 - abs(len(pred_f) - len(gt_f)) / max(len(gt_f), 1)
+            # count_penalty = 1.0
+            wcs += math.sqrt(loc_acc * max(0, count_penalty))
+        return wcs / max(len(all_f), 1)
+
+    else:
+        raise ValueError("Unsupported iou_type")
+
+
+class TimeoutException(Exception):
+    pass
+
+
+def timeout_handler(signum, frame):
+    raise TimeoutException("Function execution exceeded the time limit.")
+
+
+def compute_wcs_unlabeled(gt_clusters, pred_clusters, iou_type='tIoU',
+                          timeout=10):  # 主要是给attribute用的，但是object和event视作一个cluster也能用
+    from scipy.optimize import linear_sum_assignment
+
+    # Set the timeout signal handler
+    signal.signal(signal.SIGALRM, timeout_handler)
+    signal.alarm(timeout)  # Set the alarm to go off in 'timeout' seconds
+
+    try:
+        # Original function logic
+        K = len(gt_clusters)
+        M = len(pred_clusters)
+
+        # Build cost matrix (we want max score → min cost)
+        score_matrix = np.zeros((K, M))
+        for i in range(K):
+            for j in range(M):
+                score_matrix[i, j] = compute_cluster_pair_wcs(gt_clusters[i], pred_clusters[j], iou_type)
+
+        cost_matrix = -score_matrix  # maximize score → minimize cost
+
+        row_ind, col_ind = linear_sum_assignment(cost_matrix)
+
+        matched_scores = [score_matrix[i, j] for i, j in zip(row_ind, col_ind)]
+
+        # WCS = average over gt clusters (including unmatched = 0)
+        total_wcs = sum(matched_scores)
+        return total_wcs / K
+
+    except TimeoutException:
+        print(gt_clusters, pred_clusters)
+        print("Function execution exceeded the time limit.")
+        return None  # or you can return some default value to indicate timeout
+
+    finally:
+        signal.alarm(0)  # Cancel the alarm after the function completes or times out
+
+
+def post_process(response, right_answer, task_mode, category):
+    from word2number import w2n
+    if task_mode in ["long_acc", "ref_acc"]:
+        result = {"acc": 0, "oboa": 0, "mae": 0, "rmse": 0}
+        if response:
+            try:
+                pred = w2n.word_to_num(response)
+            except Exception:
+                pred = 0
+            if abs(float(right_answer) - float(pred)) <= 1e-5:
+                result["acc"] = 1
+
+            if abs(float(right_answer) - float(pred)) <= 1:
+                result["oboa"] = 1
+
+            if abs(float(right_answer) - float(pred)) <= max(2 * float(right_answer), 100):
+                result["mae"] = abs(float(right_answer) - float(pred))
+                result["rmse"] = abs(float(right_answer) - float(pred)) ** 2
+            else:
+                result["mae"] = abs(float(right_answer) * 2)
+                result["rmse"] = abs(float(right_answer) * 2) ** 2
+    elif task_mode == "clue_acc":
+        result = {"wcs": 0, "ifa": 0}
+        if response:
+            clues = json.loads(right_answer)
+            content_match = re.search(r"<answer>(.*?)</answer>", response, re.DOTALL)
+            student_answer = content_match.group(1).strip() if content_match else response.strip()
+            j = None
+            try:
+                try:
+                    j = json.loads(student_answer)
+                except Exception:
+                    j = json.loads(extract_outer_json(student_answer))
+            except Exception:
+                pass
+            if j is not None:
+                try:
+                    if category == "event":
+                        pred = []
+                        for e in j:
+
+                            if isinstance(e[0], str) and isinstance(e[1], str) and ":" in e[0] and ":" in e[1]:
+                                pred.append([time_str_to_seconds(e[0]), time_str_to_seconds(e[1])])
+                            else:
+                                pred.append([float(e[0].split(" ")[0]) if isinstance(e[0], str) else e[0],
+                                             float(e[1].split(" ")[0]) if isinstance(e[1], str) else e[1]])
+                        gt = []
+                        for e in clues:
+                            gt.append([float(e['start']), float(e['end'])])
+
+                        result["wcs"] = compute_wcs_unlabeled([gt], [pred], "tIoU")
+                        result["ifa"] = 1
+                    elif category == "object":
+                        gt = []
+                        clue_timestamp_list = []
+                        for clue in clues:
+                            if clue["timestamp"] not in clue_timestamp_list:
+                                clue_timestamp_list.append(clue["timestamp"])
+                        for clue in clues:
+                            gt.append((clue_timestamp_list.index(clue["timestamp"]), clue['bbox']))
+                        pred = []
+                        for key in j.keys():
+                            if "Frame" not in key:
+                                continue
+                            idx = int(key.replace("Frame", "")) - 1
+                            if len(j[key]) == 0:
+                                continue
+                            if isinstance(j[key][0], list) and len(j[key][0]) == 4:
+                                for e in j[key]:
+                                    if isinstance(e, list) and len(e) == 4:
+                                        pred.append((idx, e))
+                            elif isinstance(j[key][0], list) and len(j[key][0]) == 2:
+                                for ii in range(int(len(j[key]) // 2)):
+                                    if isinstance(j[key][ii * 2], list) and len(j[key][ii * 2]) == 2 and isinstance(
+                                            j[key][ii * 2 + 1], list) and len(j[key][ii * 2 + 1]) == 2:
+                                        pred.append((idx, [j[key][ii * 2][0], j[key][ii * 2][1], j[key][ii * 2 + 1][0],
+                                                           j[key][ii * 2 + 1][1]]))
+                        result["wcs"] = compute_wcs_unlabeled([gt], [pred], "sIoU")
+                        result["ifa"] = 1
+                    elif category == "attribute":
+                        gt = []
+                        clue_timestamp_list = []
+                        for clue_ in clues:
+                            for clue in clue_:
+                                if clue["timestamp"] not in clue_timestamp_list:
+                                    clue_timestamp_list.append(clue["timestamp"])
+                        for clue_ in clues:
+                            gt_ = []
+                            for clue in clue_:
+                                gt_.append((clue_timestamp_list.index(clue["timestamp"]), clue['bbox']))
+                            gt.append(gt_)
+                        pred = {}
+                        for key in j.keys():
+                            if "Frame" not in key:
+                                continue
+                            idx = int(key.replace("Frame", "")) - 1
+                            for e in j[key]:
+                                if e['label'] not in pred.keys():
+                                    pred[e['label']] = []
+                                if 'bbox' in e:
+                                    if isinstance(e['bbox'], list) and len(e['bbox']) == 4:
+                                        pred[e['label']].append((idx, e['bbox']))
+                                if 'bbox_2d' in e:
+                                    if isinstance(e['bbox_2d'], list) and len(e['bbox_2d']) == 4:
+                                        pred[e['label']].append((idx, e['bbox_2d']))
+                        pred_list = [pred[key] for key in pred]
+                        result["wcs"] = compute_wcs_unlabeled(gt, pred_list, "sIoU")
+                        result["ifa"] = 1
+                except Exception:
+                    pass
+
+    return result
+
+
+def get_chunk_number(filename):
+    try:
+        num = filename.split("chunk_")[1].split(".zip")[0]
+        return int(num)
+    except Exception:
+        return float('inf')
+
+
+def auto_merge_and_unzip_parts(target_dir, extract_dir, zip_prefix=None):
+    target_dir = Path(target_dir)
+    extract_dir = Path(extract_dir)
+    extract_dir.mkdir(parents=True, exist_ok=True)
+
+    # 匹配 zip 分卷：例如 video_chunk_001.zip.part000
+    part_files = sorted(target_dir.glob("*.zip.part*"))
+    groups = {}
+
+    # 分组：根据前缀提取 group 名（即 zip 文件名）
+    for part_file in part_files:
+        match = re.match(r"(.*\.zip)\.part\d+$", part_file.name)
+        if match:
+            zip_name = match.group(1)
+            if zip_prefix is None or Path(zip_name).stem.startswith(zip_prefix):
+                groups.setdefault(zip_name, []).append(part_file)
+
+    if not groups:
+        print(f"No matching zip parts found with prefix: {zip_prefix}")
+        return
+
+    # 合并每一组分卷 -> 解压
+    for zip_name, parts in tqdm(groups.items(), desc="Merging and unzipping"):
+        parts = sorted(parts, key=lambda p: int(p.name.split("part")[-1]))
+        zip_path = target_dir / zip_name
+
+        # 合并分卷
+        with open(zip_path, 'wb') as outfile:
+            for part in parts:
+                with open(part, 'rb') as infile:
+                    outfile.write(infile.read())
+
+        # 解压合并后的 zip 文件
+        with zipfile.ZipFile(zip_path, 'r') as zip_ref:
+            zip_ref.extractall(extract_dir)
+
+        # 删除合并后的 zip 文件（可注释）
+        zip_path.unlink()
+
+
+def unzip_hf_zip(target_dir):
+    target_dir = Path(target_dir)
+
+    videos_dir = target_dir / "cg_videos_720p"
+    ref_videos_dir = target_dir / "ref_videos"
+
+    if videos_dir.exists() and ref_videos_dir.exists():
+        print("all target dirs exist, skip.")
+        return
+
+    videos_dir.mkdir(parents=True, exist_ok=True)
+
+    auto_merge_and_unzip_parts(target_dir, ref_videos_dir, zip_prefix="ref_videos")
+    auto_merge_and_unzip_parts(target_dir, videos_dir, zip_prefix="videos")
+
+    print("sucessfully unzip all files.")
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/EgoExoBench/tf2023_preprocess.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/EgoExoBench/tf2023_preprocess.py
new file mode 100644
index 0000000000000000000000000000000000000000..1011de0fac0df6b012193531b750269a18580c39
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/EgoExoBench/tf2023_preprocess.py
@@ -0,0 +1,72 @@
+import json
+import os
+
+import cv2
+import numpy as np
+
+# replace the path with your actual path
+ann_file = 'EgoExoBench/MCQ/Ego-Exo-View-Transition/ego_wearer_identification.json'
+
+
+def add_bbox(bbox_img_path):
+
+    bbox_dir = os.path.dirname(bbox_img_path)
+    os.makedirs(bbox_dir, exist_ok=True)
+    vid, frame_idx, person_id = bbox_img_path.split('/')[-4], bbox_img_path.split('/')[-2], bbox_img_path.split('/')[-1].split('.')[0]  # noqa: E501
+    import os.path as osp
+    json_file = os.path.join(osp.dirname(osp.dirname(osp.dirname(osp.dirname(bbox_img_path)))), vid, 'Segmentation/T', frame_idx + '.json')  # noqa: E501
+    ori_img_path = json_file.replace('.json', '.jpg')
+
+    with open(json_file, mode='r', encoding="utf-8") as f:
+        configs = json.load(f)
+    shapes = configs["shapes"]
+
+    mask = np.zeros((configs["imageHeight"], configs["imageWidth"], 1), np.uint8)
+
+    if not os.path.exists(ori_img_path):
+        ori_img_path = ori_img_path.replace('T/', '')
+
+    if not os.path.exists(ori_img_path):
+        ori_img_path = ori_img_path.replace('Segmentation/', 'frame/T/')
+
+    original_image = cv2.imread(ori_img_path)
+
+    for shape in shapes:
+        if shape['label'] != person_id:
+            continue
+
+        cv2.fillPoly(mask, [np.array(shape["points"], np.int32)], 1)
+
+        retval, labels, stats, centroids = cv2.connectedComponentsWithStats(mask, connectivity=8)
+        stats = stats[stats[:, 4].argsort()]
+        bboxs = stats[:-1]
+
+        for b in bboxs:
+            x0, y0 = b[0], b[1]
+            x1 = b[0] + b[2]
+            y1 = b[1] + b[3]
+
+            start_point, end_point = (x0, y0), (x1, y1)
+            color = (0, 0, 255)
+            thickness = 2
+            mask_bboxs = cv2.rectangle(original_image, start_point, end_point, color, thickness)
+            mask_bboxs = cv2.resize(mask_bboxs, (540, 360))
+            cv2.imwrite(bbox_img_path, mask_bboxs)
+        return
+
+
+def rescale_img(img_path, width, height):
+    img = cv2.imread(img_path)
+    resized_img = cv2.resize(img, (width, height))
+    cv2.imwrite(img_path, resized_img)
+
+
+with open(ann_file, 'r') as f:
+    ann_data = json.load(f)
+    for aitem in ann_data.values():
+        image_paths = []
+        for oitem in aitem['options']:
+            add_bbox(oitem['image_paths'][0])
+
+        for img_path in aitem['query']['image_paths']:
+            rescale_img(img_path, 960, 540)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/EgoExoBench/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/EgoExoBench/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..084db79f1f5b1704a841ca641ce397ff5fa82a50
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/EgoExoBench/utils.py
@@ -0,0 +1,758 @@
+import json
+import math
+import numbers
+import random
+import re
+
+import numpy as np
+import pandas as pd
+import torch
+import torchvision
+from PIL import Image, ImageOps
+
+from vlmeval.smp import load
+from ..utils.multiple_choice import extract_answer_from_item
+
+
+def get_dimension_rating(data_path, category_type='subtask_type'):
+    data = load(data_path)
+    result_board = {}
+    for idx, item in data.iterrows():
+        if item[category_type] not in result_board:
+            result_board[item[category_type]] = [0, 0]
+        result_board[item[category_type]][1] += 1
+        if item['score']:
+            result_board[item[category_type]][0] += 1
+
+    correct = 0
+    total = 0
+    for key, value in result_board.items():
+        correct += value[0]
+        total += value[1]
+        result_board[key].append(f'{value[0] / value[1] * 100:.2f}%')
+
+    result_board['overall'] = [correct, total, f'{correct / total * 100:.2f}%']
+
+    return result_board
+
+
+def extract_characters_regex(s):
+    s = s.strip()
+    answer_prefixes = [
+        'The best answer is',
+        'The correct answer is',
+        'The answer is',
+        'The answer',
+        'The best option is'
+        'The correct option is',
+        'Best answer:'
+        'Best option:',
+        'Answer:',
+        'Option:',
+    ]
+    for answer_prefix in answer_prefixes:
+        s = s.replace(answer_prefix, '')
+
+    if len(s.split()) > 10 and not re.search('[ABCD]', s):
+        return ''
+    matches = re.search(r'[ABCD]', s)
+    if matches is None:
+        return ''
+    return matches[0]
+
+
+def extract_option(model, input_item, dataset_name):
+    options = input_item['question'].split('\n')[1:]
+    for id, option in enumerate(options):
+        option_id = chr(ord('A') + id) + '.'
+        if option.find(option_id) >= 0:
+            input_item[chr(ord('A') + id)] = option[option.find(option_id) + len(option_id):].strip('. \n')
+    return extract_answer_from_item(model, input_item, dataset_name)['opt']
+
+
+def process_results(score_file, model_name):
+    from sklearn.metrics import (accuracy_score, confusion_matrix, f1_score, precision_score,
+                                 recall_score)
+    data = pd.read_excel(score_file)
+
+    # Create the prediction column based on the Score and Answer columns
+    data['prediction'] = data.apply(
+        lambda row: row['answer'] if row['score'] == 1 else ('Yes' if row['answer'] == 'No' else 'No'), axis=1
+    )
+
+    # Recompute metrics for tamper types including 'original' in the calculations but exclude 'original' from the output
+    grouped_metrics_with_original_excluding_original = {}
+
+    original_group = data[data['tamper_type'] == 'original']
+
+    for tamper_type, group in data[data['tamper_type'] != 'original'].groupby('tamper_type'):
+        # Combine the current group with the 'original' group
+        combined_group = pd.concat([group, original_group])
+
+        # Extract ground truth and predictions for the combined group
+        y_true_group = combined_group['answer'].map({'Yes': 1, 'No': 0})
+        y_pred_group = combined_group['prediction'].map({'Yes': 1, 'No': 0})
+
+        # Calculate metrics for the combined group
+        accuracy = accuracy_score(y_true_group, y_pred_group)
+        precision = precision_score(y_true_group, y_pred_group, zero_division=0)
+        recall = recall_score(y_true_group, y_pred_group, zero_division=0)
+        f1 = f1_score(y_true_group, y_pred_group, zero_division=0)
+        conf_matrix = confusion_matrix(y_true_group, y_pred_group)
+
+        # Store metrics for the tamper_type
+        grouped_metrics_with_original_excluding_original[tamper_type] = {
+            "Accuracy": accuracy,
+            "Precision": precision,
+            "Recall": recall,
+            "F1 Score": f1,
+            "Confusion Matrix": conf_matrix.tolist()  # Convert to list for JSON compatibility
+        }
+
+        # Add the Macro Average row to the Dictionary
+        # grouped_metrics_with_original_excluding_original["overall"] = macro_averages
+
+    # Display the metrics in a dataframe for clarity
+    df_grouped_metrics_with_original_excluding_original = pd.DataFrame.from_dict(
+        grouped_metrics_with_original_excluding_original, orient='index'
+    )
+
+    # Compute Macro Averages for Accuracy, Precision, Recall, and F1 Score
+    macro_averages = {
+        "Accuracy": df_grouped_metrics_with_original_excluding_original["Accuracy"].mean(),
+        "Precision": df_grouped_metrics_with_original_excluding_original["Precision"].mean(),
+        "Recall": df_grouped_metrics_with_original_excluding_original["Recall"].mean(),
+        "F1 Score": df_grouped_metrics_with_original_excluding_original["F1 Score"].mean(),
+        "Confusion Matrix": "N/A"  # Macro average doesn't have a meaningful confusion matrix
+    }
+
+    # # Add the Macro Average row to the DataFrame
+    df_grouped_metrics_with_original_excluding_original.loc["overall"] = macro_averages
+
+    # df_grouped_metrics_with_original_excluding_original
+    metrics_dict = json.loads(df_grouped_metrics_with_original_excluding_original.T.to_json())
+    # Process Model Level Metrics
+    formatted_data = []
+    for task, task_metrics in metrics_dict.items():
+        task_metrics['Model'] = model_name
+        task_metrics['Task'] = task
+        formatted_data.append(task_metrics)
+
+    df_metrics = pd.DataFrame(formatted_data)
+
+    # Reorder columns to make 'Model' and 'Task' appear first
+    columns_order = ['Model', 'Task'] + [col for col in df_metrics.columns if col not in ['Model', 'Task']]
+    df_metrics = df_metrics[columns_order]
+
+    return df_metrics
+
+
+def aggregate_metrics_with_macro_average(score_file):
+    from sklearn.metrics import (accuracy_score, confusion_matrix, f1_score, precision_score,
+                                 recall_score)
+
+    # Load data
+    data = pd.read_excel(score_file)
+
+    # Create the prediction column based on the Score and Answer columns
+    data['prediction'] = data.apply(
+        lambda row: row['answer'] if row['score'] == 1 else ('Yes' if row['answer'] == 'No' else 'No'), axis=1
+    )
+
+    # Initialize a dictionary to store metrics
+    task_type_metrics = {}
+
+    # Process each task_type separately
+    for task_type, task_group in data.groupby('task_type'):
+        # Separate the 'original' group for the current task_type
+        original_group = task_group[task_group['tamper_type'] == 'original']
+
+        # Skip if there is no 'original' data for this task_type
+        if original_group.empty:
+            continue
+
+        # Process each tamper type for the current task_type (excluding 'original')
+        tamper_metrics = {}
+        for tamper_type, tamper_group in task_group[task_group['tamper_type'] != 'original'].groupby('tamper_type'):
+
+            # Combine the tamper group with the original group of the current task_type
+            combined_group = pd.concat([tamper_group, original_group])
+
+            # Map answers and predictions to binary values
+            y_true = combined_group['answer'].map({'Yes': 1, 'No': 0})
+            y_pred = combined_group['prediction'].map({'Yes': 1, 'No': 0})
+
+            # Compute metrics
+            accuracy = accuracy_score(y_true, y_pred)
+            precision = precision_score(y_true, y_pred, zero_division=0)
+            recall = recall_score(y_true, y_pred, zero_division=0)
+            f1 = f1_score(y_true, y_pred, zero_division=0)
+            conf_matrix = confusion_matrix(y_true, y_pred)
+
+            # Store metrics for the tamper_type
+            tamper_metrics[tamper_type] = {
+                "Accuracy": accuracy,
+                "Precision": precision,
+                "Recall": recall,
+                "F1 Score": f1,
+                "Confusion Matrix": conf_matrix.tolist()  # Convert to list for JSON compatibility
+            }
+
+        # Compute Macro Averages for the current task_type
+        metrics_df = pd.DataFrame(tamper_metrics).T
+        macro_average = {
+            "Accuracy": metrics_df["Accuracy"].mean(),
+            "Precision": metrics_df["Precision"].mean(),
+            "Recall": metrics_df["Recall"].mean(),
+            "F1 Score": metrics_df["F1 Score"].mean(),
+            "Confusion Matrix": "N/A"  # Macro average doesn't have a meaningful confusion matrix
+        }
+
+        # Add the macro average as "overall" for the task_type
+        tamper_metrics["overall"] = macro_average
+
+        # Add tamper metrics for the current task_type to the main dictionary
+        task_type_metrics[task_type] = tamper_metrics
+
+    # Transform the nested dictionary into a DataFrame
+    dataframes = []
+    for task_type, metrics in task_type_metrics.items():
+        task_df = pd.DataFrame.from_dict(metrics, orient='index')
+        task_df['task_type'] = task_type  # Add the task_type as a column
+        dataframes.append(task_df)
+
+    # Combine all task-specific DataFrames into a single DataFrame
+    result_df = pd.concat(dataframes).reset_index().rename(columns={'index': 'tamper_type'})
+    # Reorder the columns to place task_type first, then tamper_type
+    result_df = result_df[['task_type', 'tamper_type', 'Accuracy', 'Precision', 'Recall',
+                           'F1 Score', 'Confusion Matrix']]
+
+    # Select only numeric columns for aggregation
+    numeric_columns = ['Accuracy', 'Precision', 'Recall', 'F1 Score']
+
+    # Group by task_type and tamper_type, and calculate the mean for numeric columns
+    average_metrics = result_df.groupby(['task_type', 'tamper_type'])[numeric_columns].mean().reset_index()
+
+    return average_metrics
+
+
+def check_ans(pred, gt):
+    """
+    Checks if the predicted answer matches the ground truth.
+
+    Args:
+        pred (str): The predicted answer.
+        gt (str): The ground truth answer.
+
+    Returns:
+        bool: True if the predicted answer matches the ground truth, False otherwise.
+    """
+    # Convert both predictions and ground truths to lowercase and split them into options and contents
+    flag = False
+
+    # Split prediction into option and content
+    pred_list = pred.lower().strip().split(' ')
+    pred_option, _ = pred_list[0], ' '.join(pred_list[1:])
+
+    # Split ground truth into option and content
+    gt_list = gt.lower().strip().split(' ')
+    gt_option, gt_content = gt_list[0], ' '.join(gt_list[1:])
+
+    # Remove trailing period from ground truth content if present
+    if gt_content[-1] == '.':
+        gt_content = gt_content[:-1]
+
+    # Check for matching conditions
+    # Condition 1: If the predicted option is a substring of the ground truth option
+    if pred_option.replace('.', '') in gt_option:
+        flag = True
+    # Condition 2: If the ground truth option is a substring of the predicted option
+    elif gt_option in pred_option:
+        flag = True
+    # Condition 3: If the ground truth is a substring of the predicted answer
+    elif gt in pred:
+        flag = True
+
+    return flag
+
+
+def check_ans_with_model(pred, gt, model, item, dataset_name='MVBench'):
+    """
+    Checks if the predicted answer matches the ground truth using a given model.
+
+    Args:
+        pred (str): The predicted answer.
+        gt (str): The ground truth answer.
+        model: A machine learning model used for additional verification.
+        item (dict): An item containing information about the question or task.
+        dataset_name (str, optional): Name of the dataset being used. Defaults to 'MVBench'.
+
+    Returns:
+        bool: True if the predicted answer matches the ground truth, False otherwise.
+    """
+    # Initialize flag to track match status
+    flag = False
+
+    # Preprocess prediction and ground truth by converting to lowercase and splitting into options and contents
+    pred_list = pred.lower().strip().split(' ')
+    pred_option, _ = pred_list[0], ' '.join(pred_list[1:])
+    gt_list = gt.lower().strip().split(' ')
+    gt_option, gt_content = gt_list[0], ' '.join(gt_list[1:])
+
+    # Remove trailing period from ground truth content if presen
+    if gt_content[-1] == '.':
+        gt_content = gt_content[:-1]
+
+    # Check for matching conditions
+    # Condition 1: If the predicted option is a substring of the ground truth option
+    if pred_option.replace('.', '') in gt_option:
+        flag = True
+    # Condition 2: If the ground truth option is a substring of the predicted option
+    elif gt_option in pred_option:
+        flag = True
+    # Condition 3: Use the provided model to verify the answer
+    elif extract_answer_from_item(model, item, dataset_name)['opt'] == item['answer']:
+        flag = True
+
+    return flag
+
+
+def check_ans_advanced(pred, gt):
+    number_table = {
+        0: 'zero',
+        1: 'one',
+        2: 'two',
+        3: 'three',
+        4: 'four',
+        5: 'five',
+        6: 'six',
+        7: 'seven',
+        8: 'eight',
+        9: 'nine',
+    }
+    flag = False
+
+    pred_list = pred.lower().split(' ')
+    pred_option, _ = pred_list[0], ' '.join(pred_list[1:])
+    gt_list = gt.lower().split(' ')
+    gt_option, gt_content = gt_list[0], ' '.join(gt_list[1:])
+    if gt_content[-1] == '.':
+        gt_content = gt_content[:-1]
+
+    try:
+        gt_content = number_table[int(gt_content.strip('. \n'))]
+        print(gt_content)
+    except Exception:
+        pass
+
+    if pred_option.replace('.', '') in gt_option:
+        flag = True
+    elif gt_option in pred_option:
+        flag = True
+    elif gt_content.lower().strip('. \n') in pred.lower().strip('. \n'):
+        flag = True
+
+    return flag
+
+
+class GroupRandomCrop(object):
+    def __init__(self, size):
+        if isinstance(size, numbers.Number):
+            self.size = (int(size), int(size))
+        else:
+            self.size = size
+
+    def __call__(self, img_group):
+
+        w, h = img_group[0].size
+        th, tw = self.size
+
+        out_images = list()
+
+        x1 = random.randint(0, w - tw)
+        y1 = random.randint(0, h - th)
+
+        for img in img_group:
+            assert (img.size[0] == w and img.size[1] == h)
+            if w == tw and h == th:
+                out_images.append(img)
+            else:
+                out_images.append(img.crop((x1, y1, x1 + tw, y1 + th)))
+
+        return out_images
+
+
+class MultiGroupRandomCrop(object):
+    def __init__(self, size, groups=1):
+        if isinstance(size, numbers.Number):
+            self.size = (int(size), int(size))
+        else:
+            self.size = size
+        self.groups = groups
+
+    def __call__(self, img_group):
+
+        w, h = img_group[0].size
+        th, tw = self.size
+
+        out_images = list()
+
+        for i in range(self.groups):
+            x1 = random.randint(0, w - tw)
+            y1 = random.randint(0, h - th)
+
+            for img in img_group:
+                assert (img.size[0] == w and img.size[1] == h)
+                if w == tw and h == th:
+                    out_images.append(img)
+                else:
+                    out_images.append(img.crop((x1, y1, x1 + tw, y1 + th)))
+
+        return out_images
+
+
+class GroupCenterCrop(object):
+    def __init__(self, size):
+        self.worker = torchvision.transforms.CenterCrop(size)
+
+    def __call__(self, img_group):
+        return [self.worker(img) for img in img_group]
+
+
+class GroupRandomHorizontalFlip(object):
+    """Randomly horizontally flips the given PIL.Image with a probability of 0.5
+    """
+
+    def __init__(self, is_flow=False):
+        self.is_flow = is_flow
+
+    def __call__(self, img_group, is_flow=False):
+        v = random.random()
+        if v < 0.5:
+            ret = [img.transpose(Image.FLIP_LEFT_RIGHT) for img in img_group]
+            if self.is_flow:
+                for i in range(0, len(ret), 2):
+                    # invert flow pixel values when flipping
+                    ret[i] = ImageOps.invert(ret[i])
+            return ret
+        else:
+            return img_group
+
+
+class GroupNormalize(object):
+    def __init__(self, mean, std):
+        self.mean = mean
+        self.std = std
+
+    def __call__(self, tensor):
+        rep_mean = self.mean * (tensor.size()[0] // len(self.mean))
+        rep_std = self.std * (tensor.size()[0] // len(self.std))
+
+        # TODO: make efficient
+        for t, m, s in zip(tensor, rep_mean, rep_std):
+            t.sub_(m).div_(s)
+
+        return tensor
+
+
+class GroupScale(object):
+    """ Rescales the input PIL.Image to the given 'size'.
+    'size' will be the size of the smaller edge.
+    For example, if height > width, then image will be
+    rescaled to (size * height / width, size)
+    size: size of the smaller edge
+    interpolation: Default: PIL.Image.BILINEAR
+    """
+
+    def __init__(self, size, interpolation=Image.BILINEAR):
+        self.worker = torchvision.transforms.Resize(size, interpolation)
+
+    def __call__(self, img_group):
+        return [self.worker(img) for img in img_group]
+
+
+class GroupOverSample(object):
+    def __init__(self, crop_size, scale_size=None, flip=True):
+        self.crop_size = crop_size if not isinstance(
+            crop_size, int) else (crop_size, crop_size)
+
+        if scale_size is not None:
+            self.scale_worker = GroupScale(scale_size)
+        else:
+            self.scale_worker = None
+        self.flip = flip
+
+    def __call__(self, img_group):
+
+        if self.scale_worker is not None:
+            img_group = self.scale_worker(img_group)
+
+        image_w, image_h = img_group[0].size
+        crop_w, crop_h = self.crop_size
+
+        offsets = GroupMultiScaleCrop.fill_fix_offset(
+            False, image_w, image_h, crop_w, crop_h)
+        oversample_group = list()
+        for o_w, o_h in offsets:
+            normal_group = list()
+            flip_group = list()
+            for i, img in enumerate(img_group):
+                crop = img.crop((o_w, o_h, o_w + crop_w, o_h + crop_h))
+                normal_group.append(crop)
+                flip_crop = crop.copy().transpose(Image.FLIP_LEFT_RIGHT)
+
+                if img.mode == 'L' and i % 2 == 0:
+                    flip_group.append(ImageOps.invert(flip_crop))
+                else:
+                    flip_group.append(flip_crop)
+
+            oversample_group.extend(normal_group)
+            if self.flip:
+                oversample_group.extend(flip_group)
+        return oversample_group
+
+
+class GroupFullResSample(object):
+    def __init__(self, crop_size, scale_size=None, flip=True):
+        self.crop_size = crop_size if not isinstance(
+            crop_size, int) else (crop_size, crop_size)
+
+        if scale_size is not None:
+            self.scale_worker = GroupScale(scale_size)
+        else:
+            self.scale_worker = None
+        self.flip = flip
+
+    def __call__(self, img_group):
+
+        if self.scale_worker is not None:
+            img_group = self.scale_worker(img_group)
+
+        image_w, image_h = img_group[0].size
+        crop_w, crop_h = self.crop_size
+
+        w_step = (image_w - crop_w) // 4
+        h_step = (image_h - crop_h) // 4
+
+        offsets = list()
+        offsets.append((0 * w_step, 2 * h_step))  # left
+        offsets.append((4 * w_step, 2 * h_step))  # right
+        offsets.append((2 * w_step, 2 * h_step))  # center
+
+        oversample_group = list()
+        for o_w, o_h in offsets:
+            normal_group = list()
+            flip_group = list()
+            for i, img in enumerate(img_group):
+                crop = img.crop((o_w, o_h, o_w + crop_w, o_h + crop_h))
+                normal_group.append(crop)
+                if self.flip:
+                    flip_crop = crop.copy().transpose(Image.FLIP_LEFT_RIGHT)
+
+                    if img.mode == 'L' and i % 2 == 0:
+                        flip_group.append(ImageOps.invert(flip_crop))
+                    else:
+                        flip_group.append(flip_crop)
+
+            oversample_group.extend(normal_group)
+            oversample_group.extend(flip_group)
+        return oversample_group
+
+
+class GroupMultiScaleCrop(object):
+
+    def __init__(self, input_size, scales=None, max_distort=1,
+                 fix_crop=True, more_fix_crop=True):
+        self.scales = scales if scales is not None else [1, .875, .75, .66]
+        self.max_distort = max_distort
+        self.fix_crop = fix_crop
+        self.more_fix_crop = more_fix_crop
+        self.input_size = input_size if not isinstance(input_size, int) else [
+            input_size, input_size]
+        self.interpolation = Image.BILINEAR
+
+    def __call__(self, img_group):
+
+        im_size = img_group[0].size
+
+        crop_w, crop_h, offset_w, offset_h = self._sample_crop_size(im_size)
+        crop_img_group = [
+            img.crop(
+                (offset_w,
+                 offset_h,
+                 offset_w + crop_w,
+                 offset_h + crop_h)) for img in img_group]
+        ret_img_group = [img.resize((self.input_size[0], self.input_size[1]), self.interpolation)
+                         for img in crop_img_group]
+        return ret_img_group
+
+    def _sample_crop_size(self, im_size):
+        image_w, image_h = im_size[0], im_size[1]
+
+        # find a crop size
+        base_size = min(image_w, image_h)
+        crop_sizes = [int(base_size * x) for x in self.scales]
+        crop_h = [
+            self.input_size[1] if abs(
+                x - self.input_size[1]) < 3 else x for x in crop_sizes]
+        crop_w = [
+            self.input_size[0] if abs(
+                x - self.input_size[0]) < 3 else x for x in crop_sizes]
+
+        pairs = []
+        for i, h in enumerate(crop_h):
+            for j, w in enumerate(crop_w):
+                if abs(i - j) <= self.max_distort:
+                    pairs.append((w, h))
+
+        crop_pair = random.choice(pairs)
+        if not self.fix_crop:
+            w_offset = random.randint(0, image_w - crop_pair[0])
+            h_offset = random.randint(0, image_h - crop_pair[1])
+        else:
+            w_offset, h_offset = self._sample_fix_offset(
+                image_w, image_h, crop_pair[0], crop_pair[1])
+
+        return crop_pair[0], crop_pair[1], w_offset, h_offset
+
+    def _sample_fix_offset(self, image_w, image_h, crop_w, crop_h):
+        offsets = self.fill_fix_offset(
+            self.more_fix_crop, image_w, image_h, crop_w, crop_h)
+        return random.choice(offsets)
+
+    @staticmethod
+    def fill_fix_offset(more_fix_crop, image_w, image_h, crop_w, crop_h):
+        w_step = (image_w - crop_w) // 4
+        h_step = (image_h - crop_h) // 4
+
+        ret = list()
+        ret.append((0, 0))  # upper left
+        ret.append((4 * w_step, 0))  # upper right
+        ret.append((0, 4 * h_step))  # lower left
+        ret.append((4 * w_step, 4 * h_step))  # lower right
+        ret.append((2 * w_step, 2 * h_step))  # center
+
+        if more_fix_crop:
+            ret.append((0, 2 * h_step))  # center left
+            ret.append((4 * w_step, 2 * h_step))  # center right
+            ret.append((2 * w_step, 4 * h_step))  # lower center
+            ret.append((2 * w_step, 0 * h_step))  # upper center
+
+            ret.append((1 * w_step, 1 * h_step))  # upper left quarter
+            ret.append((3 * w_step, 1 * h_step))  # upper right quarter
+            ret.append((1 * w_step, 3 * h_step))  # lower left quarter
+            ret.append((3 * w_step, 3 * h_step))  # lower righ quarter
+
+        return ret
+
+
+class GroupRandomSizedCrop(object):
+    """Random crop the given PIL.Image to a random size of (0.08 to 1.0) of the original size
+    and and a random aspect ratio of 3/4 to 4/3 of the original aspect ratio
+    This is popularly used to train the Inception networks
+    size: size of the smaller edge
+    interpolation: Default: PIL.Image.BILINEAR
+    """
+
+    def __init__(self, size, interpolation=Image.BILINEAR):
+        self.size = size
+        self.interpolation = interpolation
+
+    def __call__(self, img_group):
+        for attempt in range(10):
+            area = img_group[0].size[0] * img_group[0].size[1]
+            target_area = random.uniform(0.08, 1.0) * area
+            aspect_ratio = random.uniform(3. / 4, 4. / 3)
+
+            w = int(round(math.sqrt(target_area * aspect_ratio)))
+            h = int(round(math.sqrt(target_area / aspect_ratio)))
+
+            if random.random() < 0.5:
+                w, h = h, w
+
+            if w <= img_group[0].size[0] and h <= img_group[0].size[1]:
+                x1 = random.randint(0, img_group[0].size[0] - w)
+                y1 = random.randint(0, img_group[0].size[1] - h)
+                found = True
+                break
+        else:
+            found = False
+            x1 = 0
+            y1 = 0
+
+        if found:
+            out_group = list()
+            for img in img_group:
+                img = img.crop((x1, y1, x1 + w, y1 + h))
+                assert (img.size == (w, h))
+                out_group.append(
+                    img.resize(
+                        (self.size, self.size), self.interpolation))
+            return out_group
+        else:
+            # Fallback
+            scale = GroupScale(self.size, interpolation=self.interpolation)
+            crop = GroupRandomCrop(self.size)
+            return crop(scale(img_group))
+
+
+class ConvertDataFormat(object):
+    def __init__(self, model_type):
+        self.model_type = model_type
+
+    def __call__(self, images):
+        if self.model_type == '2D':
+            return images
+        tc, h, w = images.size()
+        t = tc // 3
+        images = images.view(t, 3, h, w)
+        images = images.permute(1, 0, 2, 3)
+        return images
+
+
+class Stack(object):
+
+    def __init__(self, roll=False):
+        self.roll = roll
+
+    def __call__(self, img_group):
+        if img_group[0].mode == 'L':
+            return np.concatenate([np.expand_dims(x, 2)
+                                   for x in img_group], axis=2)
+        elif img_group[0].mode == 'RGB':
+            if self.roll:
+                return np.concatenate([np.array(x)[:, :, ::-1]
+                                       for x in img_group], axis=2)
+            else:
+                # print(np.concatenate(img_group, axis=2).shape)
+                # print(img_group[0].shape)
+                return np.concatenate(img_group, axis=2)
+
+
+class ToTorchFormatTensor(object):
+    """ Converts a PIL.Image (RGB) or numpy.ndarray (H x W x C) in the range [0, 255]
+    to a torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0] """
+
+    def __init__(self, div=True):
+        self.div = div
+
+    def __call__(self, pic):
+        if isinstance(pic, np.ndarray):
+            # handle numpy array
+            img = torch.from_numpy(pic).permute(2, 0, 1).contiguous()
+        else:
+            # handle PIL Image
+            img = torch.ByteTensor(
+                torch.ByteStorage.from_buffer(
+                    pic.tobytes()))
+            img = img.view(pic.size[1], pic.size[0], len(pic.mode))
+            # put it from HWC to CHW format
+            # yikes, this transpose takes 80% of the loading time/CPU
+            img = img.transpose(0, 1).transpose(0, 2).contiguous()
+        return img.float().div(255) if self.div else img.float()
+
+
+class IdentityTransform(object):
+
+    def __call__(self, data):
+        return data
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/osworld_g.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/osworld_g.py
new file mode 100644
index 0000000000000000000000000000000000000000..d0eab3fb0e19a8317d4818d4f5a390dccd9a45dd
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/osworld_g.py
@@ -0,0 +1,441 @@
+import ast
+import itertools
+import json
+import os
+import os.path as osp
+import re
+from collections import defaultdict
+
+import numpy as np
+import pandas as pd
+from PIL import Image
+from tqdm import tqdm
+
+from vlmeval.dataset.image_base import ImageBaseDataset
+from vlmeval.smp import LMUDataRoot, dump, get_intermediate_file_path, get_logger, load, toliststr
+
+logger = get_logger(__name__)
+
+SYSTEM_PROMPT = """You are a GUI agent. You are given a task and a screenshot of the screen. You need to perform pyautogui click/moveTo action to complete the task. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+
+USER_INSTRUCTION = """Please complete the following tasks by clicking using `pyautogui.click`:\n{instruction}"""  # noqa: E501
+
+SYSTEM_PROMPT_V2 = """You are a GUI agent. You are given a screenshot of the screen and the description of a target element. You need to click the target element using `pyautogui.click`. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+
+USER_INSTRUCTION_V2 = """Please click the following target element using `pyautogui.click`:\n{description}"""
+
+
+def parse_bbox_aguvis(response):
+    match = re.search(r"x=([\d.]+), y=([\d.]+)", response)
+    if match:
+        click_point = [float(match.group(1)), float(match.group(2))]
+    else:
+        click_point = [0.0, 0.0]
+    return click_point
+
+
+def compute_iou(box1, box2):
+    """
+    Compute the Intersection over Union (IoU) of two bounding boxes.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - float: IoU of box1 and box2.
+    """
+    # Determine the coordinates of the intersection rectangle
+    x_left = max(box1[0], box2[0])
+    y_top = max(box1[1], box2[1])
+    x_right = min(box1[2], box2[2])
+    y_bottom = min(box1[3], box2[3])
+
+    # Compute the area of intersection
+    intersection_area = max(0, x_right - x_left) * max(0, y_bottom - y_top)
+
+    # Compute the area of both bounding boxes
+    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
+
+    # Compute the area of the union
+    union_area = box1_area + box2_area - intersection_area
+
+    # Compute the Intersection over Union
+    iou = intersection_area / union_area
+
+    return iou
+
+
+def compute_accuracy(box1, box2, threshold=0.5):
+    """
+    Compute the accuracy of two bounding boxes based on a specified threshold.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - threshold (float): Threshold for the IoU to consider the prediction correct.
+
+    Returns:
+    - float: Accuracy of the prediction based on the IoU threshold.
+    """
+    iou = compute_iou(box1, box2)
+    return iou >= threshold
+
+
+def compute_center_accuracy(box1, box2):
+    """
+    Compute if the center point of box 2 is within box 1.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - bool: True if the center point of box 2 is within box 1, False otherwise.
+    """
+    # Compute the center point of box 2
+    center_x = (box2[0] + box2[2]) / 2
+    center_y = (box2[1] + box2[3]) / 2
+
+    # Check if the center point is within box 1
+    return box1[0] <= center_x <= box1[2] and box1[1] <= center_y <= box1[3]
+
+
+def convert_bbox(bbox, image_path, convert_xywh_to_x1y1x2y2=True):
+    new_bbox = bbox if isinstance(bbox, list) else ast.literal_eval(bbox)
+    if convert_xywh_to_x1y1x2y2:
+        new_bbox = [
+            new_bbox[0],
+            new_bbox[1],
+            new_bbox[0] + new_bbox[2],
+            new_bbox[1] + new_bbox[3],
+        ]
+    image = Image.open(image_path)
+    img_size = image.size
+    new_bbox = [
+        new_bbox[0] / img_size[0],
+        new_bbox[1] / img_size[1],
+        new_bbox[2] / img_size[0],
+        new_bbox[3] / img_size[1],
+    ]
+    return new_bbox
+
+
+class OSWorld_G(ImageBaseDataset):
+    MODALITY = "IMAGE"
+    TYPE = "GUI"
+    DATASET_URL = {
+        "OSWorld_G": "https://opencompass.openxlab.space/utils/VLMEval/OSWorld_G.tsv",  # Optional, dummy URL
+    }  # path
+    DATASET_MD5 = {
+        'OSWorld_G': 'eee81b61210f580cbc98b11c6bced928'
+    }
+    EVAL_TYPE = "point"  # point or rectangle
+    RE_TYPE = "functional"  # type of referring expressions: functional or composite
+
+    def __init__(
+        self,
+        dataset="OSWorld_G",
+        skip_noimg=True,
+        skeleton=False,
+        re_type="functional",
+    ):
+        # st()
+        ROOT = LMUDataRoot()
+        # You can override this variable to save image files to a different directory
+        self.dataset_name = dataset
+        self.img_root = osp.join(ROOT, "images", self.dataset_name)
+        self.RE_TYPE = re_type
+        if skeleton:
+            return
+
+        data = self.load_data(dataset)
+        self.skip_noimg = skip_noimg
+        if skip_noimg and "image" in data:
+            data = data[~pd.isna(data["image"])]
+
+        data["index"] = [str(idx + 1) for idx, x in enumerate(data["bbox"])]
+
+        self.meta_only = True
+        self.parse_response_func = parse_bbox_aguvis  # TODO: parse function can be specified through kwargs when initializing the dataset # noqa: E501
+
+        # The image field can store the base64 encoded image or another question index (for saving space) # noqa: E501
+        if "image" in data:
+            data["image"] = [str(x) for x in data["image"]]
+            image_map = {x: y for x, y in zip(data["index"], data["image"])}
+            for k in image_map:
+                if len(image_map[k]) <= 64:
+                    idx = image_map[k]
+                    assert idx in image_map and len(image_map[idx]) > 64
+                    image_map[k] = image_map[idx]
+
+            images = [toliststr(image_map[k]) for k in data["index"]]
+            data["image"] = [x[0] if len(x) == 1 else x for x in images]
+            self.meta_only = False
+
+        self.data = data
+
+    @classmethod
+    def get_action_space(self):
+        return ""
+
+    @classmethod
+    def get_trajectory(self, line):
+        traj_dict = {}
+        if self.RE_TYPE == "functional":
+            traj_dict["task"] = line["question"]
+        else:
+            traj_dict["task"] = line["description"]
+        return traj_dict
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        tgt_path = self.dump_image(line)
+
+        if self.RE_TYPE == "functional":
+            user_instruction = USER_INSTRUCTION.format(instruction=line["question"])
+        else:
+            user_instruction = USER_INSTRUCTION_V2.format(
+                description=line["description"]
+            )
+
+        msgs = []
+        # add system prompt
+        if self.RE_TYPE == "functional":
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT))
+        else:
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT_V2))
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=user_instruction))
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        # st()
+        if self.EVAL_TYPE == "point":
+            return self.evaluate_point(eval_file, **judge_kwargs)
+
+        elif self.EVAL_TYPE == "rectangle":
+            return self.evaluate_rectangle(eval_file, **judge_kwargs)
+
+    def evaluate_rectangle(self, eval_file, **judge_kwargs):
+        scorers = {
+            "IoU": compute_iou,
+            "ACC@0.1": lambda x, y: compute_accuracy(x, y, 0.1),
+            "ACC@0.3": lambda x, y: compute_accuracy(x, y, 0.3),
+            "ACC@0.5": lambda x, y: compute_accuracy(x, y, 0.5),
+            "ACC@0.7": lambda x, y: compute_accuracy(x, y, 0.7),
+            "ACC@0.9": lambda x, y: compute_accuracy(x, y, 0.9),
+            "Center_ACC": compute_center_accuracy,
+        }
+        results_dict = {}
+        for key in scorers.keys():
+            results_dict.update(
+                {
+                    key: [],
+                    key + "_text": [],
+                    key + "_icon": [],
+                }
+            )
+
+        result = []
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = convert_bbox(
+                line["bbox"], os.path.join(self.img_root, line["image_path"]), convert_xywh_to_x1y1x2y2=False
+            )
+            prediction = str(line["prediction"])
+            try:
+                click_point = parse_bbox_aguvis(prediction)
+
+                match = {}
+                for score_key, score_value in scorers.items():
+                    score = score_value(bbox, click_point)
+                    if score_key != "IoU":
+                        match[score_key.replace("ACC", "match")] = score
+                    results_dict[score_key].append(score)
+                    if line["ui_type"] == "text":
+                        results_dict[score_key + "_text"].append(score)
+                    else:
+                        results_dict[score_key + "_icon"].append(score)
+            except Exception:
+                click_point = None
+                match = {score_key: False for score_key in scorers.keys() if score_key != "IoU"}
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "source": line["application"],
+                    "pred": click_point,
+                    "num_matched": sum(match.values()),
+                    **match,
+                }
+            )
+        for key in results_dict:
+            if len(results_dict[key]) == 0:
+                results_dict[key] = str(0)
+            else:
+                results_dict[key] = str(sum(results_dict[key]) / len(results_dict[key]))
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(results_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            failure_cases = [res for res in result if not res["match"] and res["is_wrong_format"]]
+            failure_cases.sort(key=lambda r: r["num_matched"], reverse=True)
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+        return results_dict
+
+    def evaluate_point(self, eval_file, **judge_kwargs):
+        # -1: format_err, 0: wrong, 1: correct
+        stats = defaultdict(list)
+        # Will include instance-level results
+        result = []
+
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = (
+                line["bbox"]
+                if isinstance(line["bbox"], list)
+                else ast.literal_eval(line["bbox"])
+            )
+            # The format of bbox is (x1, y1, x2, y2)
+
+            image = Image.open(os.path.join(self.img_root, line["image_path"]))
+            img_size = image.size
+
+            def make_safe(value):
+                if value == -1:
+                    # we can tolerate -1 as a special value and nomalize it to 0
+                    return 0
+                else:
+                    return value
+
+            bbox = [
+                make_safe(bbox[0]) / img_size[0],
+                make_safe(bbox[1]) / img_size[1],
+                make_safe(bbox[0] + bbox[2]) / img_size[0],
+                make_safe(bbox[1] + bbox[3]) / img_size[1],
+            ]
+
+            key = line["category"] + ":" + line['ui_type']
+            prediction = str(line["prediction"])
+            try:
+                click_point = self.parse_response_func(prediction)
+                # Do Normalization By Default
+                # if click_point[0] > 1 or click_point[1] > 1:
+                click_point = (click_point[0] / 1000, click_point[1] / 1000)
+
+                match = (bbox[0] <= click_point[0] <= bbox[2]) and \
+                    (bbox[1] <= click_point[1] <= bbox[3])
+                # draw click point and box on image
+                # from PIL import ImageDraw
+                # draw = ImageDraw.Draw(image)
+                # draw.rectangle([bbox[0] * img_size[0], bbox[1] * img_size[1],
+                #                 bbox[2] * img_size[0], bbox[3] * img_size[1]], outline="red", width=2)
+                # draw.ellipse([click_point[0] * img_size[0] - 5, click_point[1] * img_size[1] - 5,
+                #               click_point[0] * img_size[0] + 5, click_point[1] * img_size[1] + 5],
+                #              outline="red", width=2)
+                # image.save(f"debug_{i}.png")
+
+                if match:
+                    stats[key].append(1)
+                else:
+                    stats[key].append(0)
+                is_wrong_format = False
+
+            except Exception as e:
+                logger.warning(f"exception in screenspot eval:{e}")
+                stats[key].append(-1)
+                match, is_wrong_format, click_point = False, True, None
+
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "source": line["application"],
+                    "match": match,
+                    "is_wrong_format": is_wrong_format,
+                    "pred": click_point,
+                }
+            )
+
+        final_score_dict = {}
+        # Record the number of each category
+        final_score_dict.update({k + ':cnt': len(stats[k]) for k in stats})
+        # Calculate the Overall stats
+        full_stats = []
+        for v in stats.values():
+            full_stats.extend(v)
+        final_score_dict['Overall_Accuracy'] = np.mean([x > 0 for x in full_stats]) * 100
+        final_score_dict['Format_Err_Rate'] = np.mean([x < 0 for x in full_stats]) * 100
+        # Calculate the Accuracy of Text / Icon
+        text_stats = [v for k, v in stats.items() if k.split(":")[1] == "text" for x in v]
+        text_stats = itertools.chain(*text_stats)
+        final_score_dict['Text_Accuracy'] = np.mean([x > 0 for x in text_stats]) * 100
+        icon_stats = [v for k, v in stats.items() if k.split(":")[1] == "icon" for x in v]
+        icon_stats = itertools.chain(*icon_stats)
+        final_score_dict['Icon_Accuracy'] = np.mean([x > 0 for x in icon_stats]) * 100
+        # Calculate the Accuracy of Each Category
+        cates = list(set(data['category']))
+        for c in cates:
+            sub_stats = [v for k, v in stats.items() if k.split(":")[0] == c for x in v]
+            sub_stats = itertools.chain(*sub_stats)
+            final_score_dict[c + '_Accuracy'] = np.mean([x > 0 for x in sub_stats]) * 100
+
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(final_score_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            def click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                w, h = x2 - x1, y2 - y1
+                abs_shift_to_center = [abs(x - xc), abs(y - yc)]  # noqa: E501
+                width_outside, height_outside = [max(0, abs_shift_to_center[0] - w / 2), max(0, abs_shift_to_center[1] - h / 2)]  # noqa: E501
+                return (width_outside ** 2 + height_outside ** 2) ** 0.5  # noqa: E501
+
+            wrong_format_result = [res for res in result if res["is_wrong_format"]]
+            missed_result = [res for res in result if not res["match"] and not res["is_wrong_format"]]
+            missed_result.sort(key=lambda r: click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            failure_cases = wrong_format_result + missed_result
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+
+        successful_cases_path = os.environ.get("SUCCESSFUL_CASES_PATH", None)
+        if successful_cases_path is not None:
+            def _click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                x_shift, y_shift = x - xc, y - yc
+                return (x_shift ** 2 + y_shift ** 2) ** 0.5
+
+            successful_cases = [res for res in result if res["match"]]
+            successful_cases.sort(key=lambda r: _click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            with open(successful_cases_path, "w") as f:
+                json.dump(successful_cases, f, indent=4, ensure_ascii=False)
+        return final_score_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot.py
new file mode 100644
index 0000000000000000000000000000000000000000..5d418f71165e8cf7573f058744a641a824891378
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot.py
@@ -0,0 +1,462 @@
+import ast
+import itertools
+import json
+import os
+import os.path as osp
+import re
+from collections import defaultdict
+
+import numpy as np
+import pandas as pd
+from PIL import Image
+from tqdm import tqdm
+
+from vlmeval.dataset.image_base import ImageBaseDataset
+from vlmeval.smp import LMUDataRoot, dump, get_intermediate_file_path, get_logger, load, toliststr
+
+logger = get_logger(__name__)
+
+"""
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [42, 1102, 197, 70],
+    "question": "view the details of the item",
+    "data_type": "text",
+    "data_source": "shop"
+},
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [93, 74, 86, 132],
+    "question": "view the previous photo",
+    "data_type": "icon",
+    "data_source": "shop"
+}
+"""
+
+SYSTEM_PROMPT = """You are a GUI agent. You are given a task and a screenshot of the screen. You need to perform pyautogui click/moveTo action to complete the task. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+
+USER_INSTRUCTION = """Please complete the following tasks by clicking using `pyautogui.click`:\n{instruction}"""
+
+SYSTEM_PROMPT_V2 = """You are a GUI agent. You are given a screenshot of the screen and the description of a target element. You need to click the target element using `pyautogui.click`. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+USER_INSTRUCTION_V2 = """Please click the following target element using `pyautogui.click`:\n{description}"""
+
+
+def parse_bbox_aguvis(response):
+    match = re.search(r"x=([\d.]+), y=([\d.]+)", response)
+    if match:
+        click_point = [float(match.group(1)), float(match.group(2))]
+    else:
+        click_point = [0.0, 0.0]
+    return click_point
+
+
+def compute_iou(box1, box2):
+    """
+    Compute the Intersection over Union (IoU) of two bounding boxes.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - float: IoU of box1 and box2.
+    """
+    # Determine the coordinates of the intersection rectangle
+    x_left = max(box1[0], box2[0])
+    y_top = max(box1[1], box2[1])
+    x_right = min(box1[2], box2[2])
+    y_bottom = min(box1[3], box2[3])
+
+    # Compute the area of intersection
+    intersection_area = max(0, x_right - x_left) * max(0, y_bottom - y_top)
+
+    # Compute the area of both bounding boxes
+    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
+
+    # Compute the area of the union
+    union_area = box1_area + box2_area - intersection_area
+
+    # Compute the Intersection over Union
+    iou = intersection_area / union_area
+
+    return iou
+
+
+def compute_accuracy(box1, box2, threshold=0.5):
+    """
+    Compute the accuracy of two bounding boxes based on a specified threshold.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - threshold (float): Threshold for the IoU to consider the prediction correct.
+
+    Returns:
+    - float: Accuracy of the prediction based on the IoU threshold.
+    """
+    iou = compute_iou(box1, box2)
+    return iou >= threshold
+
+
+def compute_center_accuracy(box1, box2):
+    """
+    Compute if the center point of box 2 is within box 1.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - bool: True if the center point of box 2 is within box 1, False otherwise.
+    """
+    # Compute the center point of box 2
+    center_x = (box2[0] + box2[2]) / 2
+    center_y = (box2[1] + box2[3]) / 2
+
+    # Check if the center point is within box 1
+    return box1[0] <= center_x <= box1[2] and box1[1] <= center_y <= box1[3]
+
+
+def convert_bbox(bbox, image_path):
+    new_bbox = bbox if isinstance(bbox, list) else ast.literal_eval(bbox)
+    new_bbox = [
+        new_bbox[0],
+        new_bbox[1],
+        new_bbox[0] + new_bbox[2],
+        new_bbox[1] + new_bbox[3],
+    ]
+    image = Image.open(image_path)
+    img_size = image.size
+    new_bbox = [
+        new_bbox[0] / img_size[0],
+        new_bbox[1] / img_size[1],
+        new_bbox[2] / img_size[0],
+        new_bbox[3] / img_size[1],
+    ]
+    return new_bbox
+
+
+class ScreenSpot(ImageBaseDataset):
+    MODALITY = "IMAGE"
+    TYPE = "GUI"
+    DATASET_URL = {
+        "ScreenSpot_Mobile": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot/ScreenSpot_Mobile.tsv",  # noqa
+        "ScreenSpot_Desktop": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot/ScreenSpot_Desktop.tsv",  # noqa
+        "ScreenSpot_Web": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot/ScreenSpot_Web.tsv",  # noqa
+        "ScreenSpot_v2_Mobile": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_v2/ScreenSpot_v2_Mobile.tsv",  # noqa
+        "ScreenSpot_v2_Desktop": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_v2/ScreenSpot_v2_Desktop.tsv",  # noqa
+        "ScreenSpot_v2_Web": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_v2/ScreenSpot_v2_Web.tsv",  # noqa
+    }  # path
+    DATASET_URL_V2 = {
+        "ScreenSpot_Mobile": "$WORK_DIR/screenspot_mobile_ug.json",
+        "ScreenSpot_Desktop": "$WORK_DIR/screenspot_desktop_ug.json",
+        "ScreenSpot_Web": "$WORK_DIR/screenspot_web_ug.json",
+    }  # path
+    DATASET_MD5 = {
+        "ScreenSpot_Mobile": "a5b5299843a75c9b9574c47bc13b2c53",
+        "ScreenSpot_Desktop": "e6e7bac21b6b2475276404fce2458132",
+        "ScreenSpot_Web": "e51d168c14b8582427cf3107d236cfc5",
+        "ScreenSpot_v2_Mobile": "234c858ab4f0e787e8388a73df65a4b7",
+        "ScreenSpot_v2_Desktop": "5f2aa2a497327bd33b2512a0c75cf994",
+        "ScreenSpot_v2_Web": "01cd0877ee1b735a6d5190b053ba9482",
+    }
+    EVAL_TYPE = "point"  # point or rectangle
+    RE_TYPE = "functional"  # type of referring expressions: functional or composite
+
+    def __init__(
+        self,
+        dataset="ScreenSpot_Mobile",
+        skip_noimg=True,
+        skeleton=False,
+        re_type="functional",
+    ):
+        # st()
+        ROOT = LMUDataRoot()
+        # You can override this variable to save image files to a different directory
+        self.dataset_name = dataset
+        self.img_root = osp.join(ROOT, "images", self.dataset_name)
+        self.RE_TYPE = re_type
+        if skeleton:
+            return
+
+        data = self.load_data(dataset)
+        self.skip_noimg = skip_noimg
+        if skip_noimg and "image" in data:
+            data = data[~pd.isna(data["image"])]
+
+        self.meta_only = True
+        self.parse_response_func = parse_bbox_aguvis  # TODO: parse function can be specified through kwargs when initializing the dataset # noqa: E501
+
+        # The image field can store the base64 encoded image or another question index (for saving space)
+        if "image" in data:
+            data["image"] = [str(x) for x in data["image"]]
+            image_map = {x: y for x, y in zip(data["index"], data["image"])}
+            for k in image_map:
+                if len(image_map[k]) <= 64:
+                    idx = image_map[k]
+                    assert idx in image_map and len(image_map[idx]) > 64
+                    image_map[k] = image_map[idx]
+
+            images = [toliststr(image_map[k]) for k in data["index"]]
+            data["image"] = [x[0] if len(x) == 1 else x for x in images]
+            self.meta_only = False
+
+        self.data = data
+
+    def prepare_tsv(self, url, file_md5=None):
+        # st()
+        if self.RE_TYPE == "functional":
+            return super().prepare_tsv(url=url, file_md5=file_md5)
+        else:
+            data_path = self.DATASET_URL_V2[self.dataset_name]
+        return pd.DataFrame(load(data_path))
+
+    @classmethod
+    def get_action_space(self):
+        return ""
+
+    @classmethod
+    def get_trajectory(self, line):
+        traj_dict = {}
+        if self.RE_TYPE == "functional":
+            traj_dict["task"] = line["question"]
+        else:
+            traj_dict["task"] = line["description"]
+        return traj_dict
+
+    def build_prompt(self, line):
+        # st()
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        tgt_path = self.dump_image(line)
+
+        if self.RE_TYPE == "functional":
+            user_instruction = USER_INSTRUCTION.format(instruction=line["question"])
+        else:
+            user_instruction = USER_INSTRUCTION_V2.format(
+                description=line["description"]
+            )
+
+        msgs = []
+        # add system prompt
+        if self.RE_TYPE == "functional":
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT))
+        else:
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT_V2))
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=user_instruction))
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        # st()
+        if self.EVAL_TYPE == "point":
+            return self.evaluate_point(eval_file, **judge_kwargs)
+
+        elif self.EVAL_TYPE == "rectangle":
+            return self.evaluate_rectangle(eval_file, **judge_kwargs)
+
+    def evaluate_rectangle(self, eval_file, **judge_kwargs):
+        scorers = {
+            "IoU": compute_iou,
+            "ACC@0.1": lambda x, y: compute_accuracy(x, y, 0.1),
+            "ACC@0.3": lambda x, y: compute_accuracy(x, y, 0.3),
+            "ACC@0.5": lambda x, y: compute_accuracy(x, y, 0.5),
+            "ACC@0.7": lambda x, y: compute_accuracy(x, y, 0.7),
+            "ACC@0.9": lambda x, y: compute_accuracy(x, y, 0.9),
+            "Center_ACC": compute_center_accuracy,
+        }
+        results_dict = {}
+        for key in scorers.keys():
+            results_dict.update(
+                {
+                    key: [],
+                    key + "_text": [],
+                    key + "_icon": [],
+                }
+            )
+
+        result = []
+        data = load(eval_file)
+
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = convert_bbox(
+                line["bbox"], os.path.join(self.img_root, line["image_path"])
+            )
+            prediction = str(line["prediction"])
+            try:
+                click_point = parse_bbox_aguvis(prediction)
+
+                match = {}
+                for score_key, score_value in scorers.items():
+                    score = score_value(bbox, click_point)
+                    if score_key != "IoU":
+                        match[score_key.replace("ACC", "match")] = score
+                    results_dict[score_key].append(score)
+                    if line["data_type"] == "text":
+                        results_dict[score_key + "_text"].append(score)
+                    else:
+                        results_dict[score_key + "_icon"].append(score)
+            except Exception:
+                click_point = None
+                match = {score_key: False for score_key in scorers.keys() if score_key != "IoU"}
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["data_type"],
+                    "source": line["data_source"],
+                    "pred": click_point,
+                    "num_matched": sum(match.values()),
+                    **match,
+                }
+            )
+        for key in results_dict:
+            if len(results_dict[key]) == 0:
+                results_dict[key] = str(0)
+            else:
+                results_dict[key] = str(sum(results_dict[key]) / len(results_dict[key]))
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(results_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            failure_cases = [res for res in result if not res["match"] and res["is_wrong_format"]]
+            failure_cases.sort(key=lambda r: r["num_matched"], reverse=True)
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+        return results_dict
+
+    def evaluate_point(self, eval_file, **judge_kwargs):
+        # -1: format_err, 0: wrong, 1: correct
+        stats = defaultdict(list)
+        # Will include instance-level results
+        result = []
+
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = (
+                line["bbox"]
+                if isinstance(line["bbox"], list)
+                else ast.literal_eval(line["bbox"])
+            )
+            # The format of bbox is (x1, y1, w, h)
+            x1, y1, w, h = bbox
+            bbox = (x1, y1, x1 + w - 1, y1 + h - 1)
+
+            image = Image.open(os.path.join(self.img_root, line["image_path"]))
+            img_size = image.size
+
+            def make_safe(value):
+                if value == -1:
+                    # we can tolerate -1 as a special value and nomalize it to 0
+                    return 0
+                else:
+                    return value
+
+            bbox = [
+                make_safe(bbox[0]) / img_size[0],
+                make_safe(bbox[1]) / img_size[1],
+                make_safe(bbox[2]) / img_size[0],
+                make_safe(bbox[3]) / img_size[1],
+            ]
+
+            if any([x < 0 or x > 1 for x in bbox]):
+                raise ValueError(f"bbox out of range: {bbox} | {line['bbox']} | {img_size}")
+
+            key = line['data_type'] if 'category' not in line else line['category'] + ":" + line['data_type']
+            prediction = str(line["prediction"])
+            try:
+                click_point = parse_bbox_aguvis(prediction)
+                # Do Normalization By Default
+                if click_point[0] > 1 or click_point[1] > 1:
+                    click_point = (click_point[0] / img_size[0], click_point[1] / img_size[1])
+
+                match = (bbox[0] <= click_point[0] <= bbox[2]) and \
+                    (bbox[1] <= click_point[1] <= bbox[3])
+
+                if match:
+                    stats[key].append(1)
+                else:
+                    stats[key].append(0)
+                is_wrong_format = False
+
+            except Exception as e:
+                logger.warning(f"exception in screenspot eval:{e}")
+                stats[key].append(-1)
+                match, is_wrong_format, click_point = False, True, None
+
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["data_type"],
+                    "source": line["data_source"],
+                    "match": match,
+                    "is_wrong_format": is_wrong_format,
+                    "pred": click_point,
+                }
+            )
+
+        final_score_dict = {}
+        # Record the number of each category
+        final_score_dict.update({k + ':cnt': len(stats[k]) for k in stats})
+        # Calculate the Overall stats
+        full_stats = []
+        for v in stats.values():
+            full_stats.extend(v)
+        final_score_dict['Overall_Accuracy'] = np.mean([x > 0 for x in full_stats]) * 100
+        final_score_dict['Format_Err_Rate'] = np.mean([x < 0 for x in full_stats]) * 100
+        # Calculate the Accuracy of Text / Icon
+        text_stats = [v for k, v in stats.items() if k.endswith('text') for x in v]
+        text_stats = itertools.chain(*text_stats)
+        final_score_dict['Text_Accuracy'] = np.mean([x > 0 for x in text_stats]) * 100
+        icon_stats = [v for k, v in stats.items() if k.endswith('icon') for x in v]
+        icon_stats = itertools.chain(*icon_stats)
+        final_score_dict['Icon_Accuracy'] = np.mean([x > 0 for x in icon_stats]) * 100
+        # Calculate the Accuracy of Each Category
+        if 'category' in data:
+            cates = list(set(data['category']))
+            for c in cates:
+                sub_stats = [v for k, v in stats.items() if k.split(":")[0] == c for x in v]
+                sub_stats = itertools.chain(*sub_stats)
+                final_score_dict[c + '_Accuracy'] = np.mean([x > 0 for x in sub_stats]) * 100
+
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(final_score_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            def click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                w, h = x2 - x1, y2 - y1
+                abs_shift_to_center = [abs(x - xc), abs(y - yc)]  # noqa: E501
+                width_outside, height_outside = [max(0, abs_shift_to_center[0] - w / 2), max(0, abs_shift_to_center[1] - h / 2)]  # noqa: E501
+                return (width_outside ** 2 + height_outside ** 2) ** 0.5  # noqa: E501
+
+            wrong_format_result = [res for res in result if res["is_wrong_format"]]
+            missed_result = [res for res in result if not res["match"] and not res["is_wrong_format"]]
+            missed_result.sort(key=lambda r: click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            failure_cases = wrong_format_result + missed_result
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+        return final_score_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot_pro.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot_pro.py
new file mode 100644
index 0000000000000000000000000000000000000000..79c9b215204ca1f3daa56b4a1e3506844b09c030
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot_pro.py
@@ -0,0 +1,461 @@
+import ast
+import itertools
+import json
+import os
+import os.path as osp
+import re
+from collections import defaultdict
+
+import numpy as np
+import pandas as pd
+from PIL import Image
+from tqdm import tqdm
+
+from vlmeval.dataset.image_base import ImageBaseDataset
+from vlmeval.smp import LMUDataRoot, dump, get_intermediate_file_path, get_logger, load, toliststr
+
+logger = get_logger(__name__)
+
+"""
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [42, 1102, 197, 70],
+    "question": "view the details of the item",
+    "data_type": "text",
+    "data_source": "shop"
+},
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [93, 74, 86, 132],
+    "question": "view the previous photo",
+    "data_type": "icon",
+    "data_source": "shop"
+}
+"""
+
+SYSTEM_PROMPT = """You are a GUI agent. You are given a task and a screenshot of the screen. You need to perform pyautogui click/moveTo action to complete the task. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+
+USER_INSTRUCTION = """Please complete the following tasks by clicking using `pyautogui.click`:\n{instruction}"""  # noqa: E501
+
+SYSTEM_PROMPT_V2 = """You are a GUI agent. You are given a screenshot of the screen and the description of a target element. You need to click the target element using `pyautogui.click`. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+USER_INSTRUCTION_V2 = """Please click the following target element using `pyautogui.click`:\n{description}"""
+
+
+def parse_bbox_aguvis(response):
+    match = re.search(r"x=([\d.]+), y=([\d.]+)", response)
+    if match:
+        click_point = [float(match.group(1)), float(match.group(2))]
+    else:
+        click_point = [0.0, 0.0]
+    return click_point
+
+
+def compute_iou(box1, box2):
+    """
+    Compute the Intersection over Union (IoU) of two bounding boxes.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - float: IoU of box1 and box2.
+    """
+    # Determine the coordinates of the intersection rectangle
+    x_left = max(box1[0], box2[0])
+    y_top = max(box1[1], box2[1])
+    x_right = min(box1[2], box2[2])
+    y_bottom = min(box1[3], box2[3])
+
+    # Compute the area of intersection
+    intersection_area = max(0, x_right - x_left) * max(0, y_bottom - y_top)
+
+    # Compute the area of both bounding boxes
+    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
+
+    # Compute the area of the union
+    union_area = box1_area + box2_area - intersection_area
+
+    # Compute the Intersection over Union
+    iou = intersection_area / union_area
+
+    return iou
+
+
+def compute_accuracy(box1, box2, threshold=0.5):
+    """
+    Compute the accuracy of two bounding boxes based on a specified threshold.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - threshold (float): Threshold for the IoU to consider the prediction correct.
+
+    Returns:
+    - float: Accuracy of the prediction based on the IoU threshold.
+    """
+    iou = compute_iou(box1, box2)
+    return iou >= threshold
+
+
+def compute_center_accuracy(box1, box2):
+    """
+    Compute if the center point of box 2 is within box 1.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - bool: True if the center point of box 2 is within box 1, False otherwise.
+    """
+    # Compute the center point of box 2
+    center_x = (box2[0] + box2[2]) / 2
+    center_y = (box2[1] + box2[3]) / 2
+
+    # Check if the center point is within box 1
+    return box1[0] <= center_x <= box1[2] and box1[1] <= center_y <= box1[3]
+
+
+def convert_bbox(bbox, image_path, convert_xywh_to_x1y1x2y2=True):
+    new_bbox = bbox if isinstance(bbox, list) else ast.literal_eval(bbox)
+    if convert_xywh_to_x1y1x2y2:
+        new_bbox = [
+            new_bbox[0],
+            new_bbox[1],
+            new_bbox[0] + new_bbox[2],
+            new_bbox[1] + new_bbox[3],
+        ]
+    image = Image.open(image_path)
+    img_size = image.size
+    new_bbox = [
+        new_bbox[0] / img_size[0],
+        new_bbox[1] / img_size[1],
+        new_bbox[2] / img_size[0],
+        new_bbox[3] / img_size[1],
+    ]
+    return new_bbox
+
+
+class ScreenSpot_Pro(ImageBaseDataset):
+    MODALITY = "IMAGE"
+    TYPE = "GUI"
+    DATASET_URL = {
+        "ScreenSpot_Pro_Development": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_Pro/ScreenSpot_Pro_Development.tsv",  # noqa
+        "ScreenSpot_Pro_Creative": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_Pro/ScreenSpot_Pro_Creative.tsv",  # noqa
+        "ScreenSpot_Pro_CAD": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_Pro/ScreenSpot_Pro_CAD.tsv",  # noqa
+        "ScreenSpot_Pro_Scientific": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_Pro/ScreenSpot_Pro_Scientific.tsv",  # noqa
+        "ScreenSpot_Pro_Office": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_Pro/ScreenSpot_Pro_Office.tsv",  # noqa
+        "ScreenSpot_Pro_OS": "https://opencompass.openxlab.space/utils/benchmarks/GUI/ScreenSpot_Pro/ScreenSpot_Pro_OS.tsv",  # noqa
+    }  # path
+    DATASET_MD5 = {
+        'ScreenSpot_Pro_Development': '45b93df1d5814885011d682fe1b0f959',
+        'ScreenSpot_Pro_Creative': 'a15867fee82ba8cd95581895c55f03cd',
+        'ScreenSpot_Pro_CAD': '0faa3bc29eba359766c3a7ca2c4d8917',
+        'ScreenSpot_Pro_Scientific': 'edc2e1f2b53af5fff6480b77c4986b81',
+        'ScreenSpot_Pro_Office': '8756c128cf567274c2647423ccc4eaf0',
+        'ScreenSpot_Pro_OS': '49c3eaaa7df6d22475c39120fe8f1c06'
+    }
+    EVAL_TYPE = "point"  # point or rectangle
+    RE_TYPE = "functional"  # type of referring expressions: functional or composite
+
+    def __init__(
+        self,
+        dataset="ScreenSpot_Pro_Development",
+        skip_noimg=True,
+        skeleton=False,
+        re_type="functional",
+    ):
+        # st()
+        ROOT = LMUDataRoot()
+        # You can override this variable to save image files to a different directory
+        self.dataset_name = dataset
+        self.img_root = osp.join(ROOT, "images", self.dataset_name)
+        self.RE_TYPE = re_type
+        if skeleton:
+            return
+
+        data = self.load_data(dataset)
+        self.skip_noimg = skip_noimg
+        if skip_noimg and "image" in data:
+            data = data[~pd.isna(data["image"])]
+
+        data["index"] = [str(idx + 1) for idx, x in enumerate(data["bbox"])]
+
+        self.meta_only = True
+        self.parse_response_func = parse_bbox_aguvis  # TODO: parse function can be specified through kwargs when initializing the dataset # noqa: E501
+
+        # The image field can store the base64 encoded image or another question index (for saving space) # noqa: E501
+        if "image" in data:
+            data["image"] = [str(x) for x in data["image"]]
+            image_map = {x: y for x, y in zip(data["index"], data["image"])}
+            for k in image_map:
+                if len(image_map[k]) <= 64:
+                    idx = image_map[k]
+                    assert idx in image_map and len(image_map[idx]) > 64
+                    image_map[k] = image_map[idx]
+
+            images = [toliststr(image_map[k]) for k in data["index"]]
+            data["image"] = [x[0] if len(x) == 1 else x for x in images]
+            self.meta_only = False
+
+        self.data = data
+
+    @classmethod
+    def get_action_space(self):
+        return ""
+
+    @classmethod
+    def get_trajectory(self, line):
+        traj_dict = {}
+        if self.RE_TYPE == "functional":
+            traj_dict["task"] = line["question"]
+        else:
+            traj_dict["task"] = line["description"]
+        return traj_dict
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        tgt_path = self.dump_image(line)
+
+        if self.RE_TYPE == "functional":
+            user_instruction = USER_INSTRUCTION.format(instruction=line["question"])
+        else:
+            user_instruction = USER_INSTRUCTION_V2.format(
+                description=line["description"]
+            )
+
+        msgs = []
+        # add system prompt
+        if self.RE_TYPE == "functional":
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT))
+        else:
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT_V2))
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=user_instruction))
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        # st()
+        if self.EVAL_TYPE == "point":
+            return self.evaluate_point(eval_file, **judge_kwargs)
+
+        elif self.EVAL_TYPE == "rectangle":
+            return self.evaluate_rectangle(eval_file, **judge_kwargs)
+
+    def evaluate_rectangle(self, eval_file, **judge_kwargs):
+        scorers = {
+            "IoU": compute_iou,
+            "ACC@0.1": lambda x, y: compute_accuracy(x, y, 0.1),
+            "ACC@0.3": lambda x, y: compute_accuracy(x, y, 0.3),
+            "ACC@0.5": lambda x, y: compute_accuracy(x, y, 0.5),
+            "ACC@0.7": lambda x, y: compute_accuracy(x, y, 0.7),
+            "ACC@0.9": lambda x, y: compute_accuracy(x, y, 0.9),
+            "Center_ACC": compute_center_accuracy,
+        }
+        results_dict = {}
+        for key in scorers.keys():
+            results_dict.update(
+                {
+                    key: [],
+                    key + "_text": [],
+                    key + "_icon": [],
+                }
+            )
+
+        result = []
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = convert_bbox(
+                line["bbox"], os.path.join(self.img_root, line["image_path"]), convert_xywh_to_x1y1x2y2=False
+            )
+            prediction = str(line["prediction"])
+            try:
+                click_point = parse_bbox_aguvis(prediction)
+
+                match = {}
+                for score_key, score_value in scorers.items():
+                    score = score_value(bbox, click_point)
+                    if score_key != "IoU":
+                        match[score_key.replace("ACC", "match")] = score
+                    results_dict[score_key].append(score)
+                    if line["ui_type"] == "text":
+                        results_dict[score_key + "_text"].append(score)
+                    else:
+                        results_dict[score_key + "_icon"].append(score)
+            except Exception:
+                click_point = None
+                match = {score_key: False for score_key in scorers.keys() if score_key != "IoU"}
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "source": line["application"],
+                    "pred": click_point,
+                    "num_matched": sum(match.values()),
+                    **match,
+                }
+            )
+        for key in results_dict:
+            if len(results_dict[key]) == 0:
+                results_dict[key] = str(0)
+            else:
+                results_dict[key] = str(sum(results_dict[key]) / len(results_dict[key]))
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(results_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            failure_cases = [res for res in result if not res["match"] and res["is_wrong_format"]]
+            failure_cases.sort(key=lambda r: r["num_matched"], reverse=True)
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+        return results_dict
+
+    def evaluate_point(self, eval_file, **judge_kwargs):
+        # -1: format_err, 0: wrong, 1: correct
+        stats = defaultdict(list)
+        # Will include instance-level results
+        result = []
+
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = (
+                line["bbox"]
+                if isinstance(line["bbox"], list)
+                else ast.literal_eval(line["bbox"])
+            )
+            # The format of bbox is (x1, y1, x2, y2)
+
+            image = Image.open(os.path.join(self.img_root, line["image_path"]))
+            img_size = image.size
+
+            def make_safe(value):
+                if value == -1:
+                    # we can tolerate -1 as a special value and nomalize it to 0
+                    return 0
+                else:
+                    return value
+
+            bbox = [
+                make_safe(bbox[0]) / img_size[0],
+                make_safe(bbox[1]) / img_size[1],
+                make_safe(bbox[2]) / img_size[0],
+                make_safe(bbox[3]) / img_size[1],
+            ]
+
+            if any([x < 0 or x > 1 for x in bbox]):
+                raise ValueError(f"bbox out of range: {bbox} | {line['bbox']} | {img_size}")
+
+            key = line["category"] + ":" + line['ui_type']
+            prediction = str(line["prediction"])
+            try:
+                click_point = self.parse_response_func(prediction)
+                # Do Normalization By Default
+                if click_point[0] > 1 or click_point[1] > 1:
+                    click_point = (click_point[0] / img_size[0], click_point[1] / img_size[1])
+
+                match = (bbox[0] <= click_point[0] <= bbox[2]) and \
+                    (bbox[1] <= click_point[1] <= bbox[3])
+
+                if match:
+                    stats[key].append(1)
+                else:
+                    stats[key].append(0)
+                is_wrong_format = False
+
+            except Exception as e:
+                logger.warning(f"exception in screenspot eval:{e}")
+                stats[key].append(-1)
+                match, is_wrong_format, click_point = False, True, None
+
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "source": line["application"],
+                    "match": match,
+                    "is_wrong_format": is_wrong_format,
+                    "pred": click_point,
+                }
+            )
+
+        final_score_dict = {}
+        # Record the number of each category
+        final_score_dict.update({k + ':cnt': len(stats[k]) for k in stats})
+        # Calculate the Overall stats
+        full_stats = []
+        for v in stats.values():
+            full_stats.extend(v)
+        final_score_dict['Overall_Accuracy'] = np.mean([x > 0 for x in full_stats]) * 100
+        final_score_dict['Format_Err_Rate'] = np.mean([x < 0 for x in full_stats]) * 100
+        # Calculate the Accuracy of Text / Icon
+        text_stats = [v for k, v in stats.items() if k.split(":")[1] == "text" for x in v]
+        text_stats = itertools.chain(*text_stats)
+        final_score_dict['Text_Accuracy'] = np.mean([x > 0 for x in text_stats]) * 100
+        icon_stats = [v for k, v in stats.items() if k.split(":")[1] == "icon" for x in v]
+        icon_stats = itertools.chain(*icon_stats)
+        final_score_dict['Icon_Accuracy'] = np.mean([x > 0 for x in icon_stats]) * 100
+        # Calculate the Accuracy of Each Category
+        cates = list(set(data['category']))
+        for c in cates:
+            sub_stats = [v for k, v in stats.items() if k.split(":")[0] == c for x in v]
+            sub_stats = itertools.chain(*sub_stats)
+            final_score_dict[c + '_Accuracy'] = np.mean([x > 0 for x in sub_stats]) * 100
+
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(final_score_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            def click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                w, h = x2 - x1, y2 - y1
+                abs_shift_to_center = [abs(x - xc), abs(y - yc)]  # noqa: E501
+                width_outside, height_outside = [max(0, abs_shift_to_center[0] - w / 2), max(0, abs_shift_to_center[1] - h / 2)]  # noqa: E501
+                return (width_outside ** 2 + height_outside ** 2) ** 0.5  # noqa: E501
+
+            wrong_format_result = [res for res in result if res["is_wrong_format"]]
+            missed_result = [res for res in result if not res["match"] and not res["is_wrong_format"]]
+            missed_result.sort(key=lambda r: click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            failure_cases = wrong_format_result + missed_result
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+
+        successful_cases_path = os.environ.get("SUCCESSFUL_CASES_PATH", None)
+        if successful_cases_path is not None:
+            def _click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                x_shift, y_shift = x - xc, y - yc
+                return (x_shift ** 2 + y_shift ** 2) ** 0.5
+
+            successful_cases = [res for res in result if res["match"]]
+            successful_cases.sort(key=lambda r: _click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            with open(successful_cases_path, "w") as f:
+                json.dump(successful_cases, f, indent=4, ensure_ascii=False)
+        return final_score_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot_v2.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot_v2.py
new file mode 100644
index 0000000000000000000000000000000000000000..0120cabb895e9ce2fd85b240f5e8f3c5ad1b9b13
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/screenspot_v2.py
@@ -0,0 +1,203 @@
+import ast
+import os.path as osp
+import re
+
+import pandas as pd
+from PIL import Image
+
+from vlmeval.smp import LMUDataRoot, get_logger, load, toliststr
+from .screenspot import ScreenSpot
+
+logger = get_logger(__name__)
+
+"""
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [42, 1102, 197, 70],
+    "instruction": "view the details of the item",
+    "data_type": "text",
+    "data_source": "shop"
+},
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [93, 74, 86, 132],
+    "instruction": "view the previous photo",
+    "data_type": "icon",
+    "data_source": "shop"
+}
+"""
+
+SYSTEM_PROMPT = """You are a GUI agent. You are given a task and a screenshot of the screen. You need to perform pyautogui click/moveTo action to complete the task. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+
+USER_INSTRUCTION = """Please complete the following tasks by clicking using `pyautogui.click`:\n{instruction}"""  # noqa: E501
+
+SYSTEM_PROMPT_V2 = """You are a GUI agent. You are given a screenshot of the screen and the description of a target element. You need to click the target element using `pyautogui.click`. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+USER_INSTRUCTION_V2 = """Please click the following target element using `pyautogui.click`:\n{description}"""
+
+
+def parse_bbox_aguvis(response):
+    match = re.search(r"x=([\d.]+), y=([\d.]+)", response)
+    if match:
+        click_point = [float(match.group(1)), float(match.group(2))]
+    else:
+        click_point = [0.0, 0.0]
+    return click_point
+
+
+def compute_iou(box1, box2):
+    """
+    Compute the Intersection over Union (IoU) of two bounding boxes.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - float: IoU of box1 and box2.
+    """
+    # Determine the coordinates of the intersection rectangle
+    x_left = max(box1[0], box2[0])
+    y_top = max(box1[1], box2[1])
+    x_right = min(box1[2], box2[2])
+    y_bottom = min(box1[3], box2[3])
+
+    # Compute the area of intersection
+    intersection_area = max(0, x_right - x_left) * max(0, y_bottom - y_top)
+
+    # Compute the area of both bounding boxes
+    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
+
+    # Compute the area of the union
+    union_area = box1_area + box2_area - intersection_area
+
+    # Compute the Intersection over Union
+    iou = intersection_area / union_area
+
+    return iou
+
+
+def compute_accuracy(box1, box2, threshold=0.5):
+    """
+    Compute the accuracy of two bounding boxes based on a specified threshold.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - threshold (float): Threshold for the IoU to consider the prediction correct.
+
+    Returns:
+    - float: Accuracy of the prediction based on the IoU threshold.
+    """
+    iou = compute_iou(box1, box2)
+    return iou >= threshold
+
+
+def compute_center_accuracy(box1, box2):
+    """
+    Compute if the center point of box 2 is within box 1.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - bool: True if the center point of box 2 is within box 1, False otherwise.
+    """
+    # Compute the center point of box 2
+    center_x = (box2[0] + box2[2]) / 2
+    center_y = (box2[1] + box2[3]) / 2
+
+    # Check if the center point is within box 1
+    return box1[0] <= center_x <= box1[2] and box1[1] <= center_y <= box1[3]
+
+
+def convert_bbox(bbox, image_path):
+    new_bbox = bbox if isinstance(bbox, list) else ast.literal_eval(bbox)
+    new_bbox = [
+        new_bbox[0],
+        new_bbox[1],
+        new_bbox[0] + new_bbox[2],
+        new_bbox[1] + new_bbox[3],
+    ]
+    image = Image.open(image_path)
+    img_size = image.size
+    new_bbox = [
+        new_bbox[0] / img_size[0],
+        new_bbox[1] / img_size[1],
+        new_bbox[2] / img_size[0],
+        new_bbox[3] / img_size[1],
+    ]
+    return new_bbox
+
+
+class ScreenSpotV2(ScreenSpot):
+    MODALITY = "IMAGE"
+    TYPE = "GUI"
+    DATASET_URL = {
+        "ScreenSpot_v2_Mobile": "ScreenSpot_v2_Mobile.tsv",
+        "ScreenSpot_v2_Desktop": "ScreenSpot_v2_Desktop.tsv",
+        "ScreenSpot_v2_Web": "ScreenSpot_v2_Web.tsv",
+    }  # path
+    DATASET_MD5 = {}
+    EVAL_TYPE = "point"  # point or rectangle
+    RE_TYPE = "functional"  # type of referring expressions: functional or composite
+
+    def __init__(
+        self,
+        dataset="ScreenSpot_Mobile",
+        skip_noimg=True,
+        skeleton=False,
+        re_type="functional",
+    ):
+        # st()
+        ROOT = LMUDataRoot()
+        # You can override this variable to save image files to a different directory
+        self.dataset_name = dataset
+        self.img_root = osp.join(ROOT, "ScreenSpot_v2", "screenspotv2_image")
+        self.RE_TYPE = re_type
+        if skeleton:
+            return
+
+        data = self.load_data(dataset)
+        self.skip_noimg = skip_noimg
+        if skip_noimg and "image" in data:
+            data = data[~pd.isna(data["image"])]
+
+        data["index"] = [str(idx + 1) for idx, x in enumerate(data["bbox"])]
+
+        self.meta_only = True
+        self.parse_response_func = parse_bbox_aguvis  # TODO: parse function can be specified through kwargs when initializing the dataset # noqa: E501
+
+        # The image field can store the base64 encoded image or another question index (for saving space)
+        if "image" in data:
+            data["image"] = [str(x) for x in data["image"]]
+            image_map = {x: y for x, y in zip(data["index"], data["image"])}
+            for k in image_map:
+                if len(image_map[k]) <= 64:
+                    idx = image_map[k]
+                    assert idx in image_map and len(image_map[idx]) > 64
+                    image_map[k] = image_map[idx]
+
+            images = [toliststr(image_map[k]) for k in data["index"]]
+            data["image"] = [x[0] if len(x) == 1 else x for x in images]
+            self.meta_only = False
+
+        if "img_filename" in data:
+            paths = [toliststr(x) for x in data["img_filename"]]
+            data["image_path"] = [x[0] if len(x) == 1 else x for x in paths]
+
+        # if np.all([istype(x, int) for x in data["index"]]):
+        #     data["index"] = [int(x) for x in data["index"]]
+
+        self.data = data
+        self.post_build(dataset)
+
+    def prepare_tsv(self, url, file_md5=None):
+        # st()
+        if self.RE_TYPE == "functional":
+            data_root = LMUDataRoot()
+            data_path = osp.join(data_root, "ScreenSpot_v2", url)
+        else:
+            data_path = self.DATASET_URL_V2[self.dataset_name]
+        return pd.DataFrame(load(data_path))
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/vbgd.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/vbgd.py
new file mode 100644
index 0000000000000000000000000000000000000000..d953605ab939d44ab6cb52a92000a85cc127d718
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/vbgd.py
@@ -0,0 +1,447 @@
+import ast
+import itertools
+import json
+import os
+import os.path as osp
+import re
+from collections import defaultdict
+
+import numpy as np
+import pandas as pd
+from PIL import Image
+from tqdm import tqdm
+
+from vlmeval.dataset.image_base import ImageBaseDataset
+from vlmeval.smp import LMUDataRoot, dump, get_intermediate_file_path, get_logger, load, toliststr
+
+logger = get_logger(__name__)
+
+"""
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [42, 1102, 197, 70],
+    "question": "view the details of the item",
+    "data_type": "text",
+    "data_source": "shop"
+},
+{
+    "img_filename": "web_3b0ad239-da6b-4f6f-8f12-f674dc90ff33.png",
+    "bbox": [93, 74, 86, 132],
+    "question": "view the previous photo",
+    "data_type": "icon",
+    "data_source": "shop"
+}
+"""
+
+SYSTEM_PROMPT = """You are a GUI agent. You are given a task and a screenshot of the screen. You need to perform pyautogui click/moveTo action to complete the task. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+
+USER_INSTRUCTION = """Please complete the following tasks by clicking using `pyautogui.click`:\n{instruction}"""  # noqa: E501
+
+SYSTEM_PROMPT_V2 = """You are a GUI agent. You are given a screenshot of the screen and the description of a target element. You need to click the target element using `pyautogui.click`. The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"""  # noqa: E501
+USER_INSTRUCTION_V2 = """Please click the following target element using `pyautogui.click`:\n{description}"""
+
+
+def parse_bbox_aguvis(response):
+    match = re.search(r"x=([\d.]+), y=([\d.]+)", response)
+    if match:
+        click_point = [float(match.group(1)), float(match.group(2))]
+    else:
+        click_point = [0.0, 0.0]
+    return click_point
+
+
+def compute_iou(box1, box2):
+    """
+    Compute the Intersection over Union (IoU) of two bounding boxes.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - float: IoU of box1 and box2.
+    """
+    # Determine the coordinates of the intersection rectangle
+    x_left = max(box1[0], box2[0])
+    y_top = max(box1[1], box2[1])
+    x_right = min(box1[2], box2[2])
+    y_bottom = min(box1[3], box2[3])
+
+    # Compute the area of intersection
+    intersection_area = max(0, x_right - x_left) * max(0, y_bottom - y_top)
+
+    # Compute the area of both bounding boxes
+    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
+
+    # Compute the area of the union
+    union_area = box1_area + box2_area - intersection_area
+
+    # Compute the Intersection over Union
+    iou = intersection_area / union_area
+
+    return iou
+
+
+def compute_accuracy(box1, box2, threshold=0.5):
+    """
+    Compute the accuracy of two bounding boxes based on a specified threshold.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - threshold (float): Threshold for the IoU to consider the prediction correct.
+
+    Returns:
+    - float: Accuracy of the prediction based on the IoU threshold.
+    """
+    iou = compute_iou(box1, box2)
+    return iou >= threshold
+
+
+def compute_center_accuracy(box1, box2):
+    """
+    Compute if the center point of box 2 is within box 1.
+
+    Parameters:
+    - box1 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+    - box2 (list of float): Bounding box [x_min, y_min, x_max, y_max].
+
+    Returns:
+    - bool: True if the center point of box 2 is within box 1, False otherwise.
+    """
+    # Compute the center point of box 2
+    center_x = (box2[0] + box2[2]) / 2
+    center_y = (box2[1] + box2[3]) / 2
+
+    # Check if the center point is within box 1
+    return box1[0] <= center_x <= box1[2] and box1[1] <= center_y <= box1[3]
+
+
+def convert_bbox(bbox, image_path, convert_xywh_to_x1y1x2y2=True):
+    new_bbox = bbox if isinstance(bbox, list) else ast.literal_eval(bbox)
+    if convert_xywh_to_x1y1x2y2:
+        new_bbox = [
+            new_bbox[0],
+            new_bbox[1],
+            new_bbox[0] + new_bbox[2],
+            new_bbox[1] + new_bbox[3],
+        ]
+    image = Image.open(image_path)
+    img_size = image.size
+    new_bbox = [
+        new_bbox[0] / img_size[0],
+        new_bbox[1] / img_size[1],
+        new_bbox[2] / img_size[0],
+        new_bbox[3] / img_size[1],
+    ]
+    return new_bbox
+
+
+class VBGD(ImageBaseDataset):
+    MODALITY = "IMAGE"
+    TYPE = "GUI"
+    DATASET_URL = {"VBGD": "https://huggingface.co/datasets/Zery/VBGD_Dataset/resolve/main/VBGD.tsv"}  # path
+    DATASET_MD5 = {"VBGD": "54615d8e27a93b3be13c71ddc09a6277"}
+    EVAL_TYPE = "point"  # point or rectangle
+    RE_TYPE = "functional"  # type of referring expressions: functional or composite
+
+    def __init__(
+        self,
+        dataset="VBGD_Development",
+        skip_noimg=True,
+        skeleton=False,
+        re_type="functional",
+    ):
+        # st()
+        ROOT = LMUDataRoot()
+        # You can override this variable to save image files to a different directory
+        self.dataset_name = dataset
+        self.img_root = osp.join(ROOT, "images", self.dataset_name)
+        self.RE_TYPE = re_type
+        if skeleton:
+            return
+
+        data = self.load_data(dataset)
+        self.skip_noimg = skip_noimg
+        if skip_noimg and "image" in data:
+            data = data[~pd.isna(data["image"])]
+
+        data["index"] = [str(idx + 1) for idx, x in enumerate(data["bbox"])]
+
+        self.meta_only = True
+        self.parse_response_func = parse_bbox_aguvis  # TODO: parse function can be specified through kwargs when initializing the dataset # noqa: E501
+
+        # The image field can store the base64 encoded image or another question index (for saving space) # noqa: E501
+        if "image" in data:
+            data["image"] = [str(x) for x in data["image"]]
+            image_map = {x: y for x, y in zip(data["index"], data["image"])}
+            for k in image_map:
+                if len(image_map[k]) <= 64:
+                    idx = image_map[k]
+                    assert idx in image_map and len(image_map[idx]) > 64
+                    image_map[k] = image_map[idx]
+
+            images = [toliststr(image_map[k]) for k in data["index"]]
+            data["image"] = [x[0] if len(x) == 1 else x for x in images]
+            self.meta_only = False
+
+        self.data = data
+
+    @classmethod
+    def get_action_space(self):
+        return ""
+
+    @classmethod
+    def get_trajectory(self, line):
+        traj_dict = {}
+        if self.RE_TYPE == "functional":
+            traj_dict["task"] = line["question"]
+        else:
+            traj_dict["task"] = line["description"]
+        return traj_dict
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        tgt_path = self.dump_image(line)
+
+        if self.RE_TYPE == "functional":
+            user_instruction = USER_INSTRUCTION.format(instruction=line["question"])
+        else:
+            user_instruction = USER_INSTRUCTION_V2.format(
+                description=line["description"]
+            )
+
+        msgs = []
+        # add system prompt
+        if self.RE_TYPE == "functional":
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT))
+        else:
+            msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT_V2))
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=user_instruction))
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        # st()
+        if self.EVAL_TYPE == "point":
+            return self.evaluate_point(eval_file, **judge_kwargs)
+
+        elif self.EVAL_TYPE == "rectangle":
+            return self.evaluate_rectangle(eval_file, **judge_kwargs)
+
+    def evaluate_rectangle(self, eval_file, **judge_kwargs):
+        scorers = {
+            "IoU": compute_iou,
+            "ACC@0.1": lambda x, y: compute_accuracy(x, y, 0.1),
+            "ACC@0.3": lambda x, y: compute_accuracy(x, y, 0.3),
+            "ACC@0.5": lambda x, y: compute_accuracy(x, y, 0.5),
+            "ACC@0.7": lambda x, y: compute_accuracy(x, y, 0.7),
+            "ACC@0.9": lambda x, y: compute_accuracy(x, y, 0.9),
+            "Center_ACC": compute_center_accuracy,
+        }
+        results_dict = {}
+        for key in scorers.keys():
+            results_dict.update(
+                {
+                    key: [],
+                    key + "_text": [],
+                    key + "_icon": [],
+                }
+            )
+
+        result = []
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = convert_bbox(
+                line["bbox"], os.path.join(self.img_root, line["image_path"]), convert_xywh_to_x1y1x2y2=False
+            )
+            prediction = str(line["prediction"])
+            try:
+                click_point = parse_bbox_aguvis(prediction)
+
+                match = {}
+                for score_key, score_value in scorers.items():
+                    score = score_value(bbox, click_point)
+                    if score_key != "IoU":
+                        match[score_key.replace("ACC", "match")] = score
+                    results_dict[score_key].append(score)
+                    if line["ui_type"] == "text":
+                        results_dict[score_key + "_text"].append(score)
+                    else:
+                        results_dict[score_key + "_icon"].append(score)
+            except Exception:
+                click_point = None
+                match = {score_key: False for score_key in scorers.keys() if score_key != "IoU"}
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "source": line["application"],
+                    "pred": click_point,
+                    "num_matched": sum(match.values()),
+                    **match,
+                }
+            )
+        for key in results_dict:
+            if len(results_dict[key]) == 0:
+                results_dict[key] = str(0)
+            else:
+                results_dict[key] = str(sum(results_dict[key]) / len(results_dict[key]))
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(results_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            failure_cases = [res for res in result if not res["match"] and res["is_wrong_format"]]
+            failure_cases.sort(key=lambda r: r["num_matched"], reverse=True)
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+        return results_dict
+
+    def evaluate_point(self, eval_file, **judge_kwargs):
+        # -1: format_err, 0: wrong, 1: correct
+        stats = defaultdict(list)
+        # Will include instance-level results
+        result = []
+
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = (
+                line["bbox"]
+                if isinstance(line["bbox"], list)
+                else ast.literal_eval(line["bbox"])
+            )
+            # The format of bbox is (x1, y1, x2, y2)
+
+            image = Image.open(os.path.join(self.img_root, line["image_path"]))
+            img_size = image.size
+
+            def make_safe(value):
+                if value == -1:
+                    # we can tolerate -1 as a special value and nomalize it to 0
+                    return 0
+                else:
+                    return value
+
+            bbox = [
+                make_safe(bbox[0]) / img_size[0],
+                make_safe(bbox[1]) / img_size[1],
+                make_safe(bbox[2]) / img_size[0],
+                make_safe(bbox[3]) / img_size[1],
+            ]
+
+            if any([x < 0 or x > 1 for x in bbox]):
+                raise ValueError(f"bbox out of range: {bbox} | {line['bbox']} | {img_size}")
+
+            key = line["category"] + ":" + line['ui_type']
+            prediction = str(line["prediction"])
+            try:
+                click_point = self.parse_response_func(prediction)
+                # Do Normalization By Default
+                if click_point[0] > 1 or click_point[1] > 1:
+                    click_point = (click_point[0] / img_size[0], click_point[1] / img_size[1])
+
+                match = (bbox[0] <= click_point[0] <= bbox[2]) and \
+                    (bbox[1] <= click_point[1] <= bbox[3])
+
+                if match:
+                    stats[key].append(1)
+                else:
+                    stats[key].append(0)
+                is_wrong_format = False
+
+            except Exception as e:
+                logger.warning(f"exception in screenspot eval:{e}")
+                stats[key].append(-1)
+                match, is_wrong_format, click_point = False, True, None
+
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "source": line["application"],
+                    "match": match,
+                    "is_wrong_format": is_wrong_format,
+                    "pred": click_point,
+                }
+            )
+
+        final_score_dict = {}
+        # Record the number of each category
+        final_score_dict.update({k + ':cnt': len(stats[k]) for k in stats})
+        # Calculate the Overall stats
+        full_stats = []
+        for v in stats.values():
+            full_stats.extend(v)
+        final_score_dict['Overall_Accuracy'] = np.mean([x > 0 for x in full_stats]) * 100
+        final_score_dict['Format_Err_Rate'] = np.mean([x < 0 for x in full_stats]) * 100
+        # Calculate the Accuracy of Text / Icon
+        text_stats = [v for k, v in stats.items() if k.split(":")[1] == "text" for x in v]
+        text_stats = itertools.chain(*text_stats)
+        final_score_dict['Text_Accuracy'] = np.mean([x > 0 for x in text_stats]) * 100
+        icon_stats = [v for k, v in stats.items() if k.split(":")[1] == "icon" for x in v]
+        icon_stats = itertools.chain(*icon_stats)
+        final_score_dict['Icon_Accuracy'] = np.mean([x > 0 for x in icon_stats]) * 100
+        # Calculate the Accuracy of Each Category
+        cates = list(set(data['category']))
+        for c in cates:
+            sub_stats = [v for k, v in stats.items() if k.split(":")[0] == c for x in v]
+            sub_stats = itertools.chain(*sub_stats)
+            final_score_dict[c + '_Accuracy'] = np.mean([x > 0 for x in sub_stats]) * 100
+
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(final_score_dict, score_pth)
+
+        failure_cases_path = os.environ.get("FAILURE_CASES_PATH", None)
+        if failure_cases_path is not None:
+            def click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                w, h = x2 - x1, y2 - y1
+                abs_shift_to_center = [abs(x - xc), abs(y - yc)]  # noqa: E501
+                width_outside, height_outside = [max(0, abs_shift_to_center[0] - w / 2), max(0, abs_shift_to_center[1] - h / 2)]  # noqa: E501
+                return (width_outside ** 2 + height_outside ** 2) ** 0.5  # noqa: E501
+
+            wrong_format_result = [res for res in result if res["is_wrong_format"]]
+            missed_result = [res for res in result if not res["match"] and not res["is_wrong_format"]]
+            missed_result.sort(key=lambda r: click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            failure_cases = wrong_format_result + missed_result
+
+            with open(failure_cases_path, "w") as f:
+                json.dump(failure_cases, f, indent=4, ensure_ascii=False)
+
+        successful_cases_path = os.environ.get("SUCCESSFUL_CASES_PATH", None)
+        if successful_cases_path is not None:
+            def _click_distance(bbox, click_point):
+                x, y = click_point
+                x1, y1, x2, y2 = bbox
+                xc, yc = (x1 + x2) / 2, (y1 + y2) / 2
+                x_shift, y_shift = x - xc, y - yc
+                return (x_shift ** 2 + y_shift ** 2) ** 0.5
+
+            successful_cases = [res for res in result if res["match"]]
+            successful_cases.sort(key=lambda r: _click_distance(r["parsed_bbox"], r["pred"]), reverse=True)
+            with open(successful_cases_path, "w") as f:
+                json.dump(successful_cases, f, indent=4, ensure_ascii=False)
+        return final_score_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/venusbench.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/venusbench.py
new file mode 100644
index 0000000000000000000000000000000000000000..b5eb574a30de7f1d8e2af5479217b22ce8311f04
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/GUI/venusbench.py
@@ -0,0 +1,186 @@
+import ast
+import os
+import os.path as osp
+import re
+from collections import defaultdict
+
+import numpy as np
+import pandas as pd
+from PIL import Image
+from tqdm import tqdm
+
+from vlmeval.dataset.image_base import ImageBaseDataset
+from vlmeval.smp import LMUDataRoot, dump, get_intermediate_file_path, get_logger, load, toliststr
+
+logger = get_logger(__name__)
+
+SYSTEM_PROMPT = "You are a GUI agent. You are given a task and a screenshot of the screen. " \
+    "You need to perform pyautogui click/moveTo action to complete the task. " \
+    "The answer format is `pyautogui.click(x=?, y=?), x and y is necessary`"
+
+USER_INSTRUCTION = "Please complete the following tasks by clicking using `pyautogui.click`:\n{instruction}"
+
+
+def parse_bbox_aguvis(response):
+    match = re.search(r"x=([\d.]+), y=([\d.]+)", response)
+    if match:
+        click_point = [float(match.group(1)), float(match.group(2))]
+    else:
+        click_point = [0.0, 0.0]
+    return click_point
+
+
+class VenusBench_GD(ImageBaseDataset):
+    MODALITY = "IMAGE"
+    TYPE = "GUI"
+    DATASET_URL = {
+        "VenusBench-GD": "https://huggingface.co/datasets/Zery/VBGD_Dataset/resolve/main/VenusBench.tsv",
+    }
+    DATASET_MD5 = {
+        'VenusBench-GD': '6a2fe92d3ecf5a3b6503a1fe4891c5ea'
+    }
+
+    def __init__(
+        self,
+        dataset="VenusBench-GD",
+        skip_noimg=True,
+        skeleton=False,
+    ):
+        ROOT = LMUDataRoot()
+        self.dataset_name = dataset
+        self.img_root = osp.join(ROOT, "images", self.dataset_name)
+
+        if skeleton:
+            return
+
+        data = self.load_data(dataset)
+        self.skip_noimg = skip_noimg
+        if skip_noimg and "image" in data:
+            data = data[~pd.isna(data["image"])]
+
+        # Verify we have index properly
+        if "index" not in data:
+            data["index"] = [str(idx + 1) for idx in range(len(data))]
+
+        self.meta_only = True
+        self.parse_response_func = parse_bbox_aguvis
+
+        if "image" in data:
+            data["image"] = [str(x) for x in data["image"]]
+            image_map = {x: y for x, y in zip(data["index"], data["image"])}
+            for k in image_map:
+                if len(image_map[k]) <= 64:
+                    idx = image_map[k]
+                    assert idx in image_map and len(image_map[idx]) > 64
+                    image_map[k] = image_map[idx]
+
+            images = [toliststr(image_map[k]) for k in data["index"]]
+            data["image"] = [x[0] if len(x) == 1 else x for x in images]
+            self.meta_only = False
+
+        self.data = data
+
+    @classmethod
+    def get_action_space(self):
+        return ""
+
+    @classmethod
+    def get_trajectory(self, line):
+        traj_dict = {}
+        traj_dict["task"] = line["question"]
+        return traj_dict
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        tgt_path = self.dump_image(line)
+        user_instruction = USER_INSTRUCTION.format(instruction=line["question"])
+        msgs = []
+        msgs.append(dict(role="system", type="text", value=SYSTEM_PROMPT))
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=user_instruction))
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        stats = defaultdict(list)
+        result = []
+
+        data = load(eval_file)
+        assert "bbox" in data and "prediction" in data
+        lt = len(data)
+        lines = [data.iloc[i] for i in range(lt)]
+
+        for i in tqdm(range(len(lines))):
+            line = lines[i]
+            bbox = (
+                line["bbox"]
+                if isinstance(line["bbox"], list)
+                else ast.literal_eval(line["bbox"])
+            )
+            # The format of bbox in VenusBench-GD is (x_min, y_min, x_max, y_max)
+            image = Image.open(os.path.join(self.img_root, line["image_path"]))
+            img_size = image.size
+
+            # Absolute to relative
+            bbox = [
+                bbox[0] / img_size[0],
+                bbox[1] / img_size[1],
+                bbox[2] / img_size[0],
+                bbox[3] / img_size[1],
+            ]
+
+            key = line["category"] + ":" + line['ui_type']
+            prediction = str(line["prediction"])
+            try:
+                click_point = self.parse_response_func(prediction)
+                if click_point[0] > 1 or click_point[1] > 1:
+                    click_point = (click_point[0] / img_size[0], click_point[1] / img_size[1])
+
+                match = (bbox[0] <= click_point[0] <= bbox[2]) and \
+                    (bbox[1] <= click_point[1] <= bbox[3])
+
+                if match:
+                    stats[key].append(1)
+                else:
+                    stats[key].append(0)
+                is_wrong_format = False
+            except Exception as e:
+                logger.warning(f"exception in venusbench eval:{e}")
+                stats[key].append(-1)
+                match, is_wrong_format, click_point = False, True, None
+
+            result.append(
+                {
+                    "img_path": os.path.join(self.img_root, line["image_path"]),
+                    "text": line["question"],
+                    "bbox": line["bbox"],
+                    "parsed_bbox": bbox,
+                    "type": line["ui_type"],
+                    "category": line["category"],
+                    "match": match,
+                    "is_wrong_format": is_wrong_format,
+                    "pred": click_point,
+                }
+            )
+
+        final_score_dict = {}
+        final_score_dict.update({k + ':cnt': len(stats[k]) for k in stats})
+
+        full_stats = []
+        for v in stats.values():
+            full_stats.extend(v)
+        final_score_dict['Overall_Accuracy'] = np.mean([x > 0 for x in full_stats]) * 100
+        final_score_dict['Format_Err_Rate'] = np.mean([x < 0 for x in full_stats]) * 100
+
+        cates = list(set([line["category"] for line in lines]))
+        for c in cates:
+            sub_stats = [v for k, v in stats.items() if k.split(":")[0] == c for x in v]
+            if len(sub_stats) > 0:
+                final_score_dict[c + '_Accuracy'] = np.mean([x[0] > 0 for x in [sub_stats]]) * 100
+
+        score_pth = get_intermediate_file_path(eval_file, '_score', 'json')
+        dump(final_score_dict, score_pth)
+        return final_score_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/data_preprocess.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/data_preprocess.py
new file mode 100644
index 0000000000000000000000000000000000000000..094453606e808dc3b330dd078bfd7b83890ec918
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/data_preprocess.py
@@ -0,0 +1,449 @@
+import html
+import os
+import re
+import shutil
+import subprocess
+import unicodedata
+import uuid
+
+from bs4 import BeautifulSoup
+from pylatexenc.latex2text import LatexNodes2Text
+
+
+def remove_markdown_fences(content):
+    content = re.sub(r'^```markdown\n?', '', content, flags=re.MULTILINE)
+    content = re.sub(r'```\n?$', '', content, flags=re.MULTILINE)
+    return content
+
+# Standardize all consecutive characters
+def replace_repeated_chars(input_str):
+    input_str = re.sub(r'_{4,}', '____', input_str) # Replace more than 4 consecutive underscores with 4 underscores
+    input_str = re.sub(r' {4,}', '    ', input_str)   # Replace more than 4 consecutive spaces with 4 spaces
+    return re.sub(r'([^a-zA-Z0-9])\1{10,}', r'\1\1\1\1', input_str) # For other consecutive symbols (except numbers and letters), replace more than 10 occurrences with 4
+
+# Special Unicode handling
+def fullwidth_to_halfwidth(s):
+    result = []
+    for char in s:
+        code = ord(char)
+        # Convert full-width space to half-width space
+        if code == 0x3000:
+            code = 0x0020
+        # Convert other full-width characters to half-width
+        elif 0xFF01 <= code <= 0xFF5E:
+            code -= 0xFEE0
+        result.append(chr(code))
+    return ''.join(result)
+
+def find_special_unicode(s):
+    special_chars = {}
+    for char in s:
+        if ord(char) > 127:  # Non-ASCII characters
+            # unicode_name = unicodedata.name(char, None)
+            unicode_name = unicodedata.category(char)
+            special_chars[char] = f'U+{ord(char):04X} ({unicode_name})'
+    return special_chars
+
+# # Define dictionary for Unicode character replacements
+# unicode_replacements = {
+#     "\u00A9": r"$\copyright$",  # Copyright symbol © to latex
+#     "\u00AE": r"$^\circledR$",  # Registered trademark ® to latex
+#     "\u2122": r"$^\text{TM}$",   # Trademark ™ to latex
+#     "\u2018": "'",             # Left single quote to straight quote
+#     "\u2019": "'",             # Right single quote to straight quote
+#     "\u201C": "\"",            # Left double quote to straight quote
+#     "\u201D": "\"",            # Right double quote to straight quote
+#     "\u2013": "-",             # En dash to hyphen
+#     "\u2014": "-",             # Em dash to hyphen
+#     "\u2026": "...",           # Unicode ellipsis to three dots
+#     "\u2103": r"$\textdegree C$",  # ℃
+#     "\u03B1": r"$\alpha$",         # α
+#     "\u03B2": r"$\beta$",          # β
+#     "\u03A3": r"$\Sigma$",         # Σ
+# }
+
+# # Use regex to replace Unicode characters
+# def replace_unicode(match):
+#     char = match.group(0)
+#     return unicode_replacements.get(char, char)
+
+inline_reg = re.compile(
+    r'\$(.*?)\$|'
+    r'\\\((.*?)\\\)',
+)
+
+def textblock2unicode(text):
+    inline_matches = inline_reg.finditer(text)
+    removal_positions = []
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        # print('-------- content-------', content)
+        # Remove escape characters \
+        clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+        try:
+            if any(char in clean_content for char in r'\^_'):
+                if clean_content.endswith('\\'):
+                    clean_content += ' '
+                # inline_array.append(match.group(0))
+                unicode_content = LatexNodes2Text().latex_to_text(clean_content)
+                removal_positions.append((position[0], position[1], unicode_content))
+        except:
+            continue
+
+    # Remove inline formulas from original text
+    for start, end, unicode_content in sorted(removal_positions, reverse=True):
+        text = text[:start] + unicode_content.strip() + text[end:]
+
+    return text
+
+def normalized_formula(text):
+    # Normalize math formulas before matching
+    filter_list = ['\\mathbf', '\\mathrm', '\\mathnormal', '\\mathit', '\\mathbb', '\\mathcal', '\\mathscr', '\\mathfrak', '\\mathsf', '\\mathtt',
+                   '\\textbf', '\\text', '\\boldmath', '\\boldsymbol', '\\operatorname', '\\bm',
+                   '\\symbfit', '\\mathbfcal', '\\symbf', '\\scriptscriptstyle', '\\notag',
+                   '\\setlength', '\\coloneqq', '\\space', '\\thickspace', '\\thinspace', '\\medspace', '\\nobreakspace', '\\negmedspace',
+                   '\\quad', '\\qquad', '\\enspace', '\\substackw', ' ']
+                #    '\\left', '\\right', '{', '}', ' ']
+
+    # delimiter_filter
+    pattern = re.compile(r"\\\[(.+?)(?<!\\)\\\]")
+    match = pattern.search(text)
+
+    if match:
+        text = match.group(1).strip()
+
+    tag_pattern = re.compile(r"\\tag\{.*?\}")
+    text = tag_pattern.sub('', text)
+    hspace_pattern = re.compile(r"\\hspace\{.*?\}")
+    text = hspace_pattern.sub('', text)
+    begin_pattern = re.compile(r"\\begin\{.*?\}")
+    text = begin_pattern.sub('', text)
+    end_pattern = re.compile(r"\\end\{.*?\}")
+    text = end_pattern.sub('', text)
+    col_sep = re.compile(r"\\arraycolsep.*?\}")
+    text = col_sep.sub('', text)
+    text = text.strip('.')
+
+    for filter_text in filter_list:
+        text = text.replace(filter_text, '')
+
+    # text = normalize_text(delimiter_filter(text))
+    # text = delimiter_filter(text)
+    text = text.lower()
+    return text
+
+def normalized_html_table(text):
+    def process_table_html(md_i):
+        """
+        pred_md format edit
+        """
+        def process_table_html(html_content):
+            soup = BeautifulSoup(html_content, 'html.parser')
+            th_tags = soup.find_all('th')
+            for th in th_tags:
+                th.name = 'td'
+            thead_tags = soup.find_all('thead')
+            for thead in thead_tags:
+                thead.unwrap()  # unwrap()会移除标签但保留其内容
+            math_tags = soup.find_all('math')
+            for math_tag in math_tags:
+                alttext = math_tag.get('alttext', '')
+                alttext = f'${alttext}$'
+                if alttext:
+                    math_tag.replace_with(alttext)
+            span_tags = soup.find_all('span')
+            for span in span_tags:
+                span.unwrap()
+            return str(soup)
+
+        table_res=''
+        table_res_no_space=''
+        if '<table' in md_i.replace(" ","").replace("'",'"'):
+            md_i = process_table_html(md_i)
+            table_res = html.unescape(md_i).replace('\n', '')
+            table_res = unicodedata.normalize('NFKC', table_res).strip()
+            pattern = r'<table\b[^>]*>(.*)</table>'
+            tables = re.findall(pattern, table_res, re.DOTALL | re.IGNORECASE)
+            table_res = ''.join(tables)
+            # table_res = re.sub('<table.*?>','',table_res)
+            table_res = re.sub('( style=".*?")', "", table_res)
+            table_res = re.sub('( height=".*?")', "", table_res)
+            table_res = re.sub('( width=".*?")', "", table_res)
+            table_res = re.sub('( align=".*?")', "", table_res)
+            table_res = re.sub('( class=".*?")', "", table_res)
+            table_res = re.sub('</?tbody>',"",table_res)
+
+            table_res = re.sub(r'\s+', " ", table_res)
+            table_res_no_space = '<html><body><table border="1" >' + table_res.replace(' ','') + '</table></body></html>'
+            # table_res_no_space = re.sub(' (style=".*?")',"",table_res_no_space)
+            # table_res_no_space = re.sub(r'[ ]', " ", table_res_no_space)
+            table_res_no_space = re.sub('colspan="', ' colspan="', table_res_no_space)
+            table_res_no_space = re.sub('rowspan="', ' rowspan="', table_res_no_space)
+            table_res_no_space = re.sub('border="', ' border="', table_res_no_space)
+
+            table_res = '<html><body><table border="1" >' + table_res + '</table></body></html>'
+            # table_flow.append(table_res)
+            # table_flow_no_space.append(table_res_no_space)
+
+        return table_res, table_res_no_space
+
+    def clean_table(input_str,flag=True):
+        if flag:
+            input_str = input_str.replace('<sup>', '').replace('</sup>', '')
+            input_str = input_str.replace('<sub>', '').replace('</sub>', '')
+            input_str = input_str.replace('<span>', '').replace('</span>', '')
+            input_str = input_str.replace('<div>', '').replace('</div>', '')
+            input_str = input_str.replace('<p>', '').replace('</p>', '')
+            input_str = input_str.replace('<spandata-span-identity="">', '')
+            input_str = re.sub('<colgroup>.*?</colgroup>','',input_str)
+        return input_str
+
+    norm_text, _ = process_table_html(text)
+    norm_text = clean_table(norm_text)
+    return norm_text
+
+def normalized_latex_table(text):
+    def latex_template(latex_code):
+        template = r'''
+        \documentclass[border=20pt]{article}
+        \usepackage{subcaption}
+        \usepackage{url}
+        \usepackage{graphicx}
+        \usepackage{caption}
+        \usepackage{multirow}
+        \usepackage{booktabs}
+        \usepackage{color}
+        \usepackage{colortbl}
+        \usepackage{xcolor,soul,framed}
+        \usepackage{fontspec}
+        \usepackage{amsmath,amssymb,mathtools,bm,mathrsfs,textcomp}
+        \setlength{\parindent}{0pt}''' + \
+        r'''
+        \begin{document}
+        ''' + \
+        latex_code + \
+        r'''
+        \end{document}'''
+
+        return template
+
+    def process_table_latex(latex_code):
+        SPECIAL_STRINGS= [
+            ['\\\\vspace\\{.*?\\}', ''],
+            ['\\\\hspace\\{.*?\\}', ''],
+            ['\\\\rule\{.*?\\}\\{.*?\\}', ''],
+            ['\\\\addlinespace\\[.*?\\]', ''],
+            ['\\\\addlinespace', ''],
+            ['\\\\renewcommand\\{\\\\arraystretch\\}\\{.*?\\}', ''],
+            ['\\\\arraystretch\\{.*?\\}', ''],
+            ['\\\\(row|column)?colors?\\{[^}]*\\}(\\{[^}]*\\}){0,2}', ''],
+            ['\\\\color\\{.*?\\}', ''],
+            ['\\\\textcolor\\{.*?\\}', ''],
+            ['\\\\rowcolor(\\[.*?\\])?\\{.*?\\}', ''],
+            ['\\\\columncolor(\\[.*?\\])?\\{.*?\\}', ''],
+            ['\\\\cellcolor(\\[.*?\\])?\\{.*?\\}', ''],
+            ['\\\\colorbox\\{.*?\\}', ''],
+            ['\\\\(tiny|scriptsize|footnotesize|small|normalsize|large|Large|LARGE|huge|Huge)', ''],
+            [r'\s+', ' '],
+            ['\\\\centering', ''],
+            ['\\\\begin\\{table\\}\\[.*?\\]', '\\\\begin{table}'],
+            ['\t', ''],
+            ['@{}', ''],
+            ['\\\\toprule(\\[.*?\\])?', '\\\\hline'],
+            ['\\\\bottomrule(\\[.*?\\])?', '\\\\hline'],
+            ['\\\\midrule(\\[.*?\\])?', '\\\\hline'],
+            ['p\\{[^}]*\\}', 'l'],
+            ['m\\{[^}]*\\}', 'c'],
+            ['\\\\scalebox\\{[^}]*\\}\\{([^}]*)\\}', '\\1'],
+            ['\\\\textbf\\{([^}]*)\\}', '\\1'],
+            ['\\\\textit\\{([^}]*)\\}', '\\1'],
+            ['\\\\cmidrule(\\[.*?\\])?\\(.*?\\)\\{([0-9]-[0-9])\\}', '\\\\cline{\\2}'],
+            ['\\\\hline', ''],
+            [r'\\multicolumn\{1\}\{[^}]*\}\{((?:[^{}]|(?:\{[^{}]*\}))*)\}', r'\1']
+        ]
+        pattern = r'\\begin\{tabular\}.*\\end\{tabular\}'  # 注意这里不用 .*?
+        matches = re.findall(pattern, latex_code, re.DOTALL)
+        latex_code = ' '.join(matches)
+
+        for special_str in SPECIAL_STRINGS:
+            latex_code = re.sub(fr'{special_str[0]}', fr'{special_str[1]}', latex_code)
+
+        return latex_code
+
+    def convert_latex_to_html(latex_content, cache_dir='./temp'):
+        if not os.path.exists(cache_dir):
+            os.makedirs(cache_dir)
+
+        uuid_str = str(uuid.uuid1())
+        with open(f'{cache_dir}/{uuid_str}.tex', 'w') as f:
+            f.write(latex_template(latex_content))
+
+        cmd = ['latexmlc', '--quiet', '--nocomments', f'--log={cache_dir}/{uuid_str}.log',
+               f'{cache_dir}/{uuid_str}.tex', f'--dest={cache_dir}/{uuid_str}.html']
+        try:
+            subprocess.run(cmd, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+            with open(f'{cache_dir}/{uuid_str}.html', 'r') as f:
+                html_content = f.read()
+
+            pattern = r'<table\b[^>]*>(.*)</table>'
+            tables = re.findall(pattern, html_content, re.DOTALL | re.IGNORECASE)
+            tables = [f'<table>{table}</table>' for table in tables]
+            html_content = '\n'.join(tables)
+
+        except Exception as e:
+            html_content = ''
+
+        shutil.rmtree(cache_dir)
+        return html_content
+
+    html_text = convert_latex_to_html(text)
+    normlized_tables = normalized_html_table(html_text)
+    return normlized_tables
+
+
+def normalized_table(text, format='html'):
+    if format not in ['html', 'latex']:
+        raise ValueError('Invalid format: {}'.format(format))
+    else:
+        return globals()['normalized_{}_table'.format(format)](text)
+
+
+def textblock_with_norm_formula(text):
+    inline_matches = inline_reg.finditer(text)
+    removal_positions = []
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        # print('-------- content-------', content)
+
+        norm_content = normalized_formula(content)
+        removal_positions.append((position[0], position[1], norm_content))
+
+    # Remove inline formulas from original text
+    for start, end, norm_content in sorted(removal_positions, reverse=True):
+        text = text[:start] + norm_content.strip() + text[end:]
+
+    return text
+
+# def inline_filter_unicode(text):
+#     # Ensure text is string type
+#     if not isinstance(text, str):
+#         text = str(text)
+
+#     # Convert LaTeX content to Unicode representation
+#     text = LatexNodes2Text().latex_to_text(text)
+
+#     inline_array = []
+#     inline_matches = inline_reg.finditer(text)
+
+#     for match in inline_matches:
+#         position = [match.start(), match.end()]
+#         content = match.group(1) if match.group(1) is not None else match.group(2)
+
+#         # Remove escape characters \
+#         clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+#         if any(char in clean_content for char in r'\^_'):
+#             # inline_array.append(match.group(0))
+#             inline_array.append({
+#                 'category_type': 'equation_inline',
+#                 'position': position,
+#                 'content': match.group(0),
+#             })
+#             text = text.replace(match.group(0), '')
+#             # print('-----Found inline formula: ', match.group(0))
+#         else:
+#             text = text.replace(match.group(0), content)
+#         # # Add to inline_array
+#         # inline_array.append({
+#         #     'category_type': 'equation_inline',
+#         #     'position': position,
+#         #     'content': content,
+#         # })
+
+#         # # Remove matched formula from original text, can choose to replace with spaces or remove directly
+#         # text = text[:position[0]] + ' '*(position[1]-position[0]) + text[position[1]:]
+
+#     return text, inline_array
+
+def inline_filter_unicode(text):
+    # Ensure text is string type
+    if not isinstance(text, str):
+        text = str(text)
+
+    # Replace inline formula boundary markers
+    #print('--------text-------',text)
+    placeholder = '__INLINE_FORMULA_BOUNDARY__'
+    text_copy = text.replace('$', placeholder).replace('\\(', placeholder).replace('\\)', placeholder)
+    #print('--------text_copy-------',text_copy)
+    # Convert LaTeX content to Unicode representation
+    text_copy = LatexNodes2Text().latex_to_text(text_copy)
+    #print('--------text_copy---unicode----',text_copy)
+    # Restore boundary markers
+    text_copy = text_copy.replace(placeholder, '$')
+
+    inline_array = []
+    inline_matches = inline_reg.finditer(text_copy)
+    # Record positions of inline formulas to be removed
+    removal_positions = []
+
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        print('-------- content-------', content)
+        # Remove escape characters \
+        clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+        if any(char in clean_content for char in r'\^_'):
+            # inline_array.append(match.group(0))
+            inline_array.append({
+                'category_type': 'equation_inline',
+                'position': position,
+                'content': content,
+            })
+            removal_positions.append((position[0], position[1]))
+
+    # Remove inline formulas from original text
+    for start, end in sorted(removal_positions, reverse=True):
+        text = text[:start] + text[end:]
+
+    return text, inline_array
+
+def inline_filter(text):
+    # Ensure text is string type
+    if not isinstance(text, str):
+        text = str(text)
+
+    inline_array = []
+    inline_matches = inline_reg.finditer(text)
+
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        # print('inline_content: ', content)
+
+        # Remove escape characters \
+        clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+        if any(char in clean_content for char in r'\^_'):
+            # inline_array.append(match.group(0))
+            inline_array.append({
+                'category_type': 'equation_inline',
+                'position': position,
+                'content': match.group(0),
+            })
+            text = text.replace(match.group(0), '')
+            # print('-----Found inline formula: ', match.group(0))
+        else:
+            text = text.replace(match.group(0), content)
+
+    return text, inline_array
+
+# Text OCR quality check processing:
+def clean_string(input_string):
+    # Use regex to keep Chinese characters, English letters and numbers
+    input_string = input_string.replace('\\t', '').replace('\\n', '').replace('\t', '').replace('\n', '').replace('/t', '').replace('/n', '')
+    cleaned_string = re.sub(r'[^\w\u4e00-\u9fff]', '', input_string)
+    return cleaned_string
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/metrics.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/metrics.py
new file mode 100644
index 0000000000000000000000000000000000000000..39b534881603eb73eda4394d7a99a8f5e9673019
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/metrics.py
@@ -0,0 +1,486 @@
+import copy
+import json
+import pdb
+import random
+import time
+from collections import defaultdict, deque
+
+import evaluate
+import Levenshtein
+import pandas as pd
+from apted import APTED, Config
+from apted.helpers import Tree
+from lxml import etree, html
+from tabulate import tabulate
+from tqdm import tqdm
+
+from .utils import normalized_table, save_paired_result
+
+
+def show_result(results):
+    for metric_name in results.keys():
+        print(f'{metric_name}:')
+        score_table = [[k,v] for k,v in results[metric_name].items()]
+        print(tabulate(score_table))
+        print('='*100)
+
+def sort_nested_dict(d):
+    # If it's a dictionary, recursively sort it
+    if isinstance(d, dict):
+        # Sort the current dictionary
+        sorted_dict = {k: sort_nested_dict(v) for k, v in sorted(d.items())}
+        return sorted_dict
+    # If not a dictionary, return directly
+    return d
+
+def get_full_labels_results(samples:dict):
+    if not samples:
+        return {}
+    label_group_dict = defaultdict(lambda: defaultdict(list))
+    for sample in samples:
+        label_list = []
+        if not sample.get("gt_attribute"):
+            continue
+        for anno in sample["gt_attribute"]:
+            for k,v in anno.items():
+                label_list.append(k+": "+str(v))
+        for label_name in list(set(label_list)):  # Currently if there are merged cases, calculate based on the set of all labels involved after merging
+            for metric, score in sample['metric'].items():
+                label_group_dict[label_name][metric].append(score)
+
+    print('----Anno Attribute---------------')
+    result = {}
+    result['sample_count'] = {}
+    for attribute in label_group_dict.keys():
+        for metric, scores in label_group_dict[attribute].items():
+            mean_score = sum(scores) / len(scores)
+            if not result.get(metric):
+                result[metric] = {}
+            result[metric][attribute] = mean_score
+            result['sample_count'][attribute] = len(scores)
+    result = sort_nested_dict(result)
+    show_result(result)
+    return result
+
+
+def get_page_split(samples, page_info):   # Page level metric
+    if not page_info:
+        return {}
+    result_list = defaultdict(list)
+
+
+    for sample in samples:
+        img_name = sample['img_id'] if sample['img_id'].endswith('.jpg') else '_'.join(sample['img_id'].split('_')[:-1])
+        page_info_s = page_info[img_name]
+        if not sample.get('metric'):
+            continue
+        for metric, score in sample['metric'].items():
+            gt = sample['norm_gt'] if sample.get('norm_gt') else sample['gt']
+            pred = sample['norm_pred'] if sample.get('norm_pred') else sample['pred']
+            result_list[metric].append({
+                'image_name': img_name,
+                'metric': metric,
+                'attribute': 'ALL',
+                'score': score,
+                'upper_len': max(len(gt), len(pred))
+            })
+            for k,v in page_info_s.items():
+                if isinstance(v, list): # special issue
+                    for special_issue in v:
+                        if 'table' not in special_issue:  # Table-related special fields have duplicates
+                            result_list[metric].append({
+                                'image_name': img_name,
+                                'metric': metric,
+                                'attribute': special_issue,
+                                'score': score,
+                                'upper_len': max(len(gt), len(pred))
+                            })
+                else:
+                    result_list[metric].append({
+                        'image_name': img_name,
+                        'metric': metric,
+                        'attribute': k+": "+str(v),
+                        'score': score,
+                        'upper_len': max(len(gt), len(pred))
+                    })
+
+    # Page level logic, accumulation is only done within pages, and mean operation is performed between pages
+    result = {}
+    if result_list.get('Edit_dist'):
+        df = pd.DataFrame(result_list['Edit_dist'])
+        up_total_avg = df.groupby(["image_name", "attribute"]).apply(lambda x: (x["score"]*x['upper_len']).sum() / x['upper_len'].sum()).groupby('attribute').mean()  # At page level, accumulate edits, denominator is sum of max(gt, pred) from each sample
+        result['Edit_dist'] = up_total_avg.to_dict()
+    for metric in result_list.keys():
+        if metric == 'Edit_dist':
+            continue
+        df = pd.DataFrame(result_list[metric])
+        page_avg = df.groupby(["image_name", "attribute"]).apply(lambda x: x["score"].mean()).groupby('attribute').mean()
+        result[metric] = page_avg.to_dict()
+
+    result = sort_nested_dict(result)
+    # print('----Page Attribute---------------')
+    show_result(result)
+    return result
+
+
+def get_groups(samples, group_info):
+    group_samples = defaultdict(list)
+    for sample in samples:
+        group_samples['all'].append(sample)
+        for group in group_info:
+            select_flag = True
+            for k, v in group.items():
+                for gt_attribute in sample['gt_attribute']:   # gt_attribute is a list containing all merged gt attributes
+                    if not gt_attribute:   # if no GT attributes, don't include in calculation
+                        select_flag = False
+                    elif gt_attribute[k] != v:  # if any gt attribute doesn't meet criteria, don't select
+                        select_flag = False
+            if select_flag:
+                group_samples[str(group)].append(sample)
+    return group_samples
+
+
+class Registry:
+    def __init__(self):
+        self._registry = {}
+    def register(self, name):
+        def decorator(item):
+            if name in self._registry:
+                raise ValueError(f"Item {name} already registered.")
+            self._registry[name] = item
+            return item
+        return decorator
+    def get(self, name):
+        if name not in self._registry:
+            raise ValueError(f"Item {name} not found in registry.")
+        return self._registry[name]
+    def list_items(self):
+        return list(self._registry.keys())
+
+METRIC_REGISTRY = Registry()
+
+
+@METRIC_REGISTRY.register("TEDS")
+class call_TEDS():
+    def __init__(self, samples):
+        self.samples = samples
+    def evaluate(self, group_info=[], save_name='default'):
+        teds = TEDS(structure_only=False)
+        teds_structure_only = TEDS(structure_only=True)
+
+        group_scores = defaultdict(list)
+        group_scores_structure_only = defaultdict(list)
+
+        samples = self.samples
+        for sample in samples:
+            gt = sample['norm_gt'] if sample.get('norm_gt') else sample['gt']
+            pred = sample['norm_pred'] if sample.get('norm_pred') else sample['pred']
+
+            score = teds.evaluate(pred, gt)
+            score_structure_only = teds_structure_only.evaluate(pred, gt)
+            # print('TEDS score:', score)
+            group_scores['all'].append(score)
+            group_scores_structure_only['all'].append(score_structure_only)
+
+            if not sample.get('metric'):
+                sample['metric'] = {}
+            sample['metric']['TEDS'] = score
+            sample['metric']['TEDS_structure_only'] = score_structure_only
+
+            for group in group_info:
+                select_flag = True
+                for k, v in group.items():
+                    for gt_attribute in sample['gt_attribute']:   # gt_attribute is a list containing all merged gt attributes
+                        if not gt_attribute:   # if no GT attributes, don't include in calculation
+                            select_flag = False
+                        elif gt_attribute[k] != v:  # if any gt attribute doesn't meet criteria, don't select
+                            select_flag = False
+                if select_flag:
+                    group_scores[str(group)].append(score)
+
+        result = {}
+        for group_name, scores in group_scores.items():
+            if len(scores) > 0:
+                result[group_name] = sum(scores) / len(scores)    # average of normalized scores at sample level
+            else:
+                result[group_name] = 'NaN'
+                print(f'Warning: Empyty matched samples for {group_name}.')
+
+        structure_only_result = {}
+        for group_name, scores in group_scores_structure_only.items():
+            if len(scores) > 0:
+                structure_only_result[group_name] = sum(scores) / len(scores)    # average of normalized scores at sample level
+            else:
+                structure_only_result[group_name] = 'NaN'
+                print(f'Warning: Empyty matched samples for {group_name}.')
+
+        return samples,{'TEDS': result, 'TEDS_structure_only': structure_only_result}
+
+
+@METRIC_REGISTRY.register("BLEU")
+class call_BLEU():
+    def __init__(self, samples):
+        self.samples = samples
+    def evaluate(self, group_info=[], save_name='default'):
+        group_samples = get_groups(self.samples, group_info)
+        result = {}
+        bleu = evaluate.load("bleu", keep_in_memory=True, experiment_id=random.randint(1,1e8))
+
+        for group_name, samples in group_samples.items():
+            predictions, references = [], []
+            for sample in samples:
+                gt = sample['norm_gt'] if sample.get('norm_gt') else sample['gt']
+                pred = sample['norm_pred'] if sample.get('norm_pred') else sample['pred']
+                predictions.append(pred)
+                references.append(gt)
+
+        if not predictions or not any(predictions) or not references or not any(references):
+            bleu_score = 0
+        else:
+            try:
+                bleu_results = bleu.compute(predictions=predictions, references=references)
+                bleu_score = bleu_results["bleu"]
+            except ZeroDivisionError:
+                bleu_score = 0
+
+        result[group_name] = bleu_score
+
+        return self.samples,{'BLEU': result}
+
+@METRIC_REGISTRY.register("METEOR")
+class call_METEOR():
+    def __init__(self, samples):
+        self.samples = samples
+    def evaluate(self, group_info=[], save_name='default'):
+        group_samples = get_groups(self.samples, group_info)
+        result = {}
+        for group_name, samples in group_samples.items():
+            predictions, references = [], []
+            for sample in samples:
+                gt = sample['norm_gt'] if sample.get('norm_gt') else sample['gt']
+                pred = sample['norm_pred'] if sample.get('norm_pred') else sample['pred']
+                predictions.append(gt)
+                references.append(pred)
+            meteor = evaluate.load('meteor', keep_in_memory=True, experiment_id=random.randint(1,1e8))
+            meteor_results = meteor.compute(predictions=predictions, references=references)
+            result[group_name] = meteor_results['meteor']
+
+        return self.samples,{'METEOR': result}
+
+
+@METRIC_REGISTRY.register("Edit_dist")
+class call_Edit_dist():
+    def __init__(self, samples):
+        self.samples = samples
+    def evaluate(self, group_info=[], save_name='default'):
+        samples = self.samples
+        for sample in samples:
+            img_name = sample['img_id'] if sample['img_id'].endswith('.jpg') else '_'.join(sample['img_id'].split('_')[:-1])
+            sample['image_name'] = img_name
+            gt = sample['norm_gt'] if sample.get('norm_gt') else sample['gt']
+            pred = sample['norm_pred'] if sample.get('norm_pred') else sample['pred']
+            upper_len = max(len(pred), len(gt))
+            sample['upper_len'] = upper_len
+            if len(pred) > 0 or len(gt) > 0:
+                edit_dist = Levenshtein.distance(pred, gt)
+                if not sample.get('metric'):
+                    sample['metric'] = {}
+                sample['metric']['Edit_dist'] = edit_dist / upper_len
+                sample['Edit_num'] = edit_dist
+
+        if isinstance(samples, list):
+            saved_samples = samples
+        else:
+            saved_samples = samples.samples
+
+        if not saved_samples:
+            return {'Edit_dist': {'ALL_page_avg': 'NaN'}}
+
+        df = pd.DataFrame(saved_samples)
+        up_total_avg = df.groupby("image_name").apply(lambda x: x['Edit_num'].sum() / x['upper_len'].sum()) # page level, sum of edits divided by sum of max(gt,pred) lengths for each sample
+        per_img_score = up_total_avg.to_dict()
+
+        return samples,{'Edit_dist': {'ALL_page_avg': up_total_avg.mean()}}
+
+
+@METRIC_REGISTRY.register("CDM")
+class call_CDM():
+    def __init__(self, samples):
+        self.samples = samples
+    def evaluate(self, group_info=[], save_name='default'):
+        if isinstance(self.samples, list):
+            cdm_samples = copy.deepcopy(self.samples)
+        else:
+            cdm_samples = copy.deepcopy(self.samples.samples)
+        for idx, sample in enumerate(cdm_samples):
+            sample['img_name'] = sample['img_id']
+            sample['img_id'] = str(idx)
+            sample['gt'] = sample['gt'].lstrip("$$").rstrip("$$").strip()
+            sample['pred'] = sample['pred'].split("```latex")[-1].split("```")[0]
+            sample['pred'] = sample['pred'].lstrip("$$").rstrip("$$").strip()
+
+        return  self.samples,False
+
+
+class TEDS(object):
+    ''' Tree Edit Distance basead Similarity
+    '''
+    def __init__(self, structure_only=False, n_jobs=1, ignore_nodes=None):
+        assert isinstance(n_jobs, int) and (n_jobs >= 1), 'n_jobs must be an integer greather than 1'
+        self.structure_only = structure_only
+        self.n_jobs = n_jobs
+        self.ignore_nodes = ignore_nodes
+        self.__tokens__ = []
+
+    def tokenize(self, node):
+        ''' Tokenizes table cells
+        '''
+        self.__tokens__.append('<%s>' % node.tag)
+        if node.text is not None:
+            self.__tokens__ += list(node.text)
+        for n in node.getchildren():
+            self.tokenize(n)
+        if node.tag != 'unk':
+            self.__tokens__.append('</%s>' % node.tag)
+        if node.tag != 'td' and node.tail is not None:
+            self.__tokens__ += list(node.tail)
+
+    def load_html_tree(self, node, parent=None):
+        ''' Converts HTML tree to the format required by apted
+        '''
+        global __tokens__
+        if node.tag == 'td':
+            if self.structure_only:
+                cell = []
+            else:
+                self.__tokens__ = []
+                self.tokenize(node)
+                cell = self.__tokens__[1:-1].copy()
+            new_node = TableTree(node.tag,
+                                 int(node.attrib.get('colspan', '1')),
+                                 int(node.attrib.get('rowspan', '1')),
+                                 cell, *deque())
+        else:
+            new_node = TableTree(node.tag, None, None, None, *deque())
+        if parent is not None:
+            parent.children.append(new_node)
+        if node.tag != 'td':
+            for n in node.getchildren():
+                self.load_html_tree(n, new_node)
+        if parent is None:
+            return new_node
+
+    def evaluate(self, pred, true):
+        ''' Computes TEDS score between the prediction and the ground truth of a
+            given sample
+        '''
+        if (not pred) or (not true):
+            return 0.0
+        parser = html.HTMLParser(remove_comments=True, encoding='utf-8')
+        pred = html.fromstring(pred, parser=parser)
+        true = html.fromstring(true, parser=parser)
+        if pred.xpath('body/table') and true.xpath('body/table'):
+            pred = pred.xpath('body/table')[0]
+            true = true.xpath('body/table')[0]
+            if self.ignore_nodes:
+                etree.strip_tags(pred, *self.ignore_nodes)
+                etree.strip_tags(true, *self.ignore_nodes)
+            n_nodes_pred = len(pred.xpath(".//*"))
+            n_nodes_true = len(true.xpath(".//*"))
+            n_nodes = max(n_nodes_pred, n_nodes_true)
+            tree_pred = self.load_html_tree(pred)
+            tree_true = self.load_html_tree(true)
+            distance = APTED(tree_pred, tree_true, CustomConfig()).compute_edit_distance()
+            return 1.0 - (float(distance) / n_nodes)
+        else:
+            return 0.0
+
+    def batch_evaluate(self, pred_json, true_json):
+        ''' Computes TEDS score between the prediction and the ground truth of
+            a batch of samples
+            @params pred_json: {'FILENAME': 'HTML CODE', ...}
+            @params true_json: {'FILENAME': {'html': 'HTML CODE'}, ...}
+            @output: {'FILENAME': 'TEDS SCORE', ...}
+        '''
+        samples = true_json.keys()
+        # if self.n_jobs == 1:
+        scores = [self.evaluate(pred_json.get(filename, ''), true_json[filename]['html']) for filename in tqdm(samples)]
+        # else:
+        #     inputs = [{'pred': pred_json.get(filename, ''), 'true': true_json[filename]['html']} for filename in samples]
+        #     scores = parallel_process(inputs, self.evaluate, use_kwargs=True, n_jobs=self.n_jobs, front_num=1)
+        scores = dict(zip(samples, scores))
+        return scores
+
+
+class CustomConfig(Config):
+    @staticmethod
+    def maximum(*sequences):
+        """Get maximum possible value
+        """
+        return max(map(len, sequences))
+
+    def normalized_distance(self, *sequences):
+        """Get distance from 0 to 1
+        """
+        return float(Levenshtein.distance(*sequences)) / self.maximum(*sequences)
+
+    def rename(self, node1, node2):
+        """Compares attributes of trees"""
+        if (node1.tag != node2.tag) or (node1.colspan != node2.colspan) or (node1.rowspan != node2.rowspan):
+            return 1.
+        if node1.tag == 'td':
+            if node1.content or node2.content:
+                return self.normalized_distance(node1.content, node2.content)
+        return 0.
+
+
+class TableTree(Tree):
+    def __init__(self, tag, colspan=None, rowspan=None, content=None, *children):
+        self.tag = tag
+        self.colspan = colspan
+        self.rowspan = rowspan
+        self.content = content
+        self.children = list(children)
+
+    def bracket(self):
+        """Show tree using brackets notation"""
+        if self.tag == 'td':
+            result = '"tag": %s, "colspan": %d, "rowspan": %d, "text": %s' % \
+                     (self.tag, self.colspan, self.rowspan, self.content)
+        else:
+            result = '"tag": %s' % self.tag
+        for child in self.children:
+            result += child.bracket()
+        return "{{{}}}".format(result)
+
+
+class recogition_end2end_base_dataset():
+    def __init__(self, samples):
+        img_id = 0
+        for sample in samples:
+            if not sample.get('img_id'):
+                sample['img_id'] = img_id
+            img_id += 1
+        self.samples = samples
+    def __getitem__(self, idx):
+        return self.samples[idx]
+
+
+class recogition_end2end_table_dataset(recogition_end2end_base_dataset):
+    def __init__(self, samples, table_format):
+        self.pred_table_format = table_format
+        self.samples = self.normalize_data(samples)
+
+    def normalize_data(self, samples):
+        img_id = 0
+        for sample in samples:
+            p = sample['pred']
+            r = sample['gt']
+            p = normalized_table(p, self.pred_table_format)
+            r = normalized_table(r)
+            sample['norm_gt'] = r
+            sample['norm_pred'] = p
+            sample['img_id'] = sample['img_id'] if sample.get('img_id') else img_id
+            img_id += 1
+
+        return samples
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/omnidocbench.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/omnidocbench.py
new file mode 100644
index 0000000000000000000000000000000000000000..a41f9313e3f68dacc94e3b1b1460c3ec0ce992ac
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/omnidocbench.py
@@ -0,0 +1,557 @@
+import base64
+import copy
+import json
+import os
+import tempfile
+
+import numpy as np
+import pandas as pd
+import torch.distributed as dist
+from tqdm import tqdm
+
+from vlmeval.smp import dump, get_intermediate_file_path, load
+from ..image_base import ImageBaseDataset
+
+# from ..utils import get_intermediate_file_path, load, dump
+
+
+class OmniDocBench(ImageBaseDataset):
+
+    MODALITY = 'IMAGE'
+    TYPE = 'QA'
+
+    DATASET_URL = {'OmniDocBench':'https://huggingface.co/datasets/ouyanglinke/OmniDocBench_tsv/resolve/main/OmniDocBench.tsv'}
+    DATASET_MD5 = {'OmniDocBench': '0fa5ccf31e682e219cb9ca83da741a59'}
+
+
+    system_prompt = r'''You are an AI assistant specialized in converting PDF images to Markdown format. Please follow these instructions for the conversion:
+
+    1. Text Processing:
+    - Accurately recognize all text content in the PDF image without guessing or inferring.
+    - Convert the recognized text into Markdown format.
+    - Maintain the original document structure, including headings, paragraphs, lists, etc.
+
+    2. Mathematical Formula Processing:
+    - Convert all mathematical formulas to LaTeX format.
+    # - Enclose inline formulas with \( \). For example: This is an inline formula \( E = mc^2 \)
+    - Enclose block formulas with \\[ \\]. For example: \[ \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} \]
+
+    3. Table Processing:
+    - Convert tables to HTML format.
+    - Wrap the entire table with <table> and </table>.
+
+    4. Figure Handling:
+    - Ignore figures content in the PDF image. Do not attempt to describe or convert images.
+
+    5. Output Format:
+    - Ensure the output Markdown document has a clear structure with appropriate line breaks between elements.
+    - For complex layouts, try to maintain the original document's structure and format as closely as possible.
+
+    Please strictly follow these guidelines to ensure accuracy and consistency in the conversion. Your task is to accurately convert the content of the PDF image into Markdown format without adding any extra explanations or comments.
+    '''
+
+    def __init__(self,dataset='OmniDocBench',**kwargs):
+        super().__init__(dataset,**kwargs)
+        print(f'self.img_root:{self.img_root}')
+
+    def build_prompt(self, line):
+
+        image_path = self.dump_image(line)[0]
+        msg = [
+            dict(type='image', value=image_path),
+            dict(type='text', value=self.system_prompt)
+        ]
+        return msg
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        tsv_path=self.data_path
+        End2end_evaluator=end2end_evaluator(eval_file,tsv_path)
+        Table_evalutor=table_evalutor(eval_file,tsv_path)
+
+        metrics_all=End2end_evaluator.score()
+        metircs_table=Table_evalutor.score()
+
+        return metrics_all
+
+
+class end2end_evaluator():
+    def __init__(self,
+                 eval_file,
+                 tsv_path,
+                 match_method:str='quick_match',
+                 filter_types:dict=None):
+        self.eval_file=eval_file
+        self.match_method=match_method
+        self.references=[]
+        self.predictions = load(eval_file)['prediction'].tolist()
+        self.dafault_metircs_dict={
+            'text_block':
+                {'metric': ['Edit_dist', 'BLEU', 'METEOR']},
+            'display_formula':
+                {'metric': ['Edit_dist', 'CDM']},
+            'table':
+                {'metric': ['TEDS', 'Edit_dist']},
+            'reading_order':
+                {'metric': ['Edit_dist']}
+            }
+
+        references = load(tsv_path)['answer'].tolist()
+
+        load_success,load_fail=0,0
+        for i,ans in tqdm(enumerate(references),desc='Loading data'):
+            try:
+                ans = json.loads(ans)
+                load_success+=1
+                self.references.append(ans) #[{},{}]
+            except json.JSONDecodeError as e:
+                load_fail+=1
+                continue
+        print(f'load_success:{load_success},load_fail:{load_fail}')
+
+        filtered_gt_samples = []
+        if filter_types:
+            for gt_sample in self.references:
+                select_flag = True
+                for k, v in filter_types.items():
+                    if gt_sample["page_info"]["page_attribute"][k] != v:
+                        select_flag = False
+                if select_flag:
+                    filtered_gt_samples.append(gt_sample)
+        else:
+            filtered_gt_samples = self.references #[{},{},{}]
+        self.references=filtered_gt_samples
+
+
+    def score(self)->dict:
+        samples=self.get_matched_elements(self.references,self.predictions)
+        metrics=self.process_generated_metric_results(samples)
+        return metrics
+
+    def get_page_elements(self, selected_annos):
+        saved_element_dict = defaultdict(list)
+        related_truncated = []
+        truncated_all = {}
+        for relation in selected_annos["extra"]["relation"]:   # Handle truncated text issues
+            if relation["relation_type"] == 'truncated':
+                truncated_all[relation["source_anno_id"]] = ""
+                truncated_all[relation["target_anno_id"]] = ""
+                exist_flag = False
+                for merge_list in related_truncated:
+                    if relation["source_anno_id"] in merge_list or relation["target_anno_id"] in merge_list:  # Consider cases where three text blocks may need to be merged
+                        merge_list.append(relation["source_anno_id"])
+                        merge_list.append(relation["target_anno_id"])
+                        exist_flag = True
+                if not exist_flag:
+                    related_truncated.append([relation["source_anno_id"], relation["target_anno_id"]])
+
+        for item in selected_annos['layout_dets']:
+            if item['anno_id'] not in truncated_all.keys():
+                saved_element_dict[item["category_type"]].append(item)
+            else:
+                truncated_all[item['anno_id']] = item
+
+        for merge_list in related_truncated:
+            text_block_list = [truncated_all[key] for key in merge_list]
+            sorted_block = sorted(text_block_list, key=lambda x: x['order'])
+            text = ""
+            for block in sorted_block:
+                text += block['text']
+            merged_block = {
+                "category_type": sorted_block[0]["category_type"], # Directly use information from the first block
+                "order": sorted_block[0]["order"],
+                "anno_id": sorted_block[0]["anno_id"],
+                "text": text,
+                "merge_list": sorted_block
+            }
+            saved_element_dict[sorted_block[0]["category_type"]].append(merged_block)
+
+        return saved_element_dict
+
+    def get_page_elements_list(self, gt_page_elements, category_list):
+        element_list = []
+        for category_type in category_list:
+            if gt_page_elements.get(category_type):
+                element_list.extend(gt_page_elements[category_type])
+        return element_list
+
+    def get_sorted_text_list(self, selected_annos):
+        # txt_type: text, latex, html
+        text_list = []
+        for item in selected_annos:
+            if item.get('order'):
+                order = item['order']
+            else:
+                order = 0
+            # 【txt_type,selecte_annos]
+            text_list.append((order, item))
+        sorted_text_list = sorted(text_list, key=lambda x: x[0])
+        return [_[1] for _ in sorted_text_list]
+
+    def filtered_out_ignore(self, items, ignore_category_list):
+        filted_items = []
+        for item in items:
+            if item['gt_category_type'] not in ignore_category_list:
+                filted_items.append(item)
+        return filted_items
+
+    def get_order_paired(self, order_match_s, img_name):
+        matched = [(item['gt_position'], item['pred_position']) for item in order_match_s if (item['gt_position'] != [""] and item['pred_position'] != "")]
+        gt_idx_all = [item['gt_position'] for item in order_match_s if (item['gt_position'] != [""])]
+        read_order_pred = [i[0] for i in sorted(matched, key=lambda x: x[1])]
+        read_order_gt = sum(gt_idx_all, []) # Convert to one-dimensional list
+        read_order_gt = [x for x in read_order_gt if x]
+        gt = sorted(read_order_gt)
+        pred = sum(read_order_pred, [])
+        pred = [x for x in pred if x]
+        if len(pred) > 0 or len(gt) > 0:
+            import Levenshtein
+            edit = Levenshtein.distance(gt, pred)/ max(len(pred), len(gt))
+            return {
+                'gt': gt,
+                'pred': pred,
+                'img_id': img_name,
+                'edit': edit
+            }
+        else:
+            return {}  # If both GT and pred are empty for the page, return empty
+
+    def formula_format(self, formula_matches, img_name):
+        # formated_list = []
+        for i, item in enumerate(formula_matches):
+            item["img_id"] = img_name + '_' + str(i)
+        return formula_matches
+
+    def get_matched_elements(self,references:list,predictions:list)->dict:
+        from .metrics import recogition_end2end_base_dataset, recogition_end2end_table_dataset
+
+        plain_text_match = []
+        display_formula_match = []
+        html_table_match = []
+        latex_table_match = []
+        order_match = []
+
+
+        for i,sample in enumerate(references):
+            img_name = os.path.basename(sample["page_info"]["image_path"])
+            pred_content = predictions[i]
+            result = self.process_get_matched_elements(sample, pred_content, img_name)
+            [plain_text_match_clean, formated_display_formula, latex_table_match_s, html_table_match_s, order_match_single] = result
+
+            if order_match_single:
+                order_match.append(order_match_single)
+            if plain_text_match_clean:
+                plain_text_match.extend(plain_text_match_clean)
+            if formated_display_formula:
+                display_formula_match.extend(formated_display_formula)
+            if latex_table_match_s:
+                latex_table_match.extend(latex_table_match_s)
+            if html_table_match_s:
+                html_table_match.extend(html_table_match_s)
+
+        if len(latex_table_match) > len(html_table_match):
+            table_match = latex_table_match
+            table_format = 'latex'
+        else:
+            table_match = html_table_match
+            table_format = 'html'
+
+        matched_samples_all = {
+            "text_block": recogition_end2end_base_dataset(plain_text_match),
+            "display_formula": recogition_end2end_base_dataset(display_formula_match),
+            "table": recogition_end2end_table_dataset(table_match, table_format),
+            "reading_order": recogition_end2end_base_dataset(order_match)
+        }
+
+        return matched_samples_all
+
+    def process_get_matched_elements(self, sample, pred_content, img_name):
+        from func_timeout import FunctionTimedOut, func_timeout
+
+        from .utils import (match_gt2pred_no_split, match_gt2pred_quick, match_gt2pred_simple,
+                            md_tex_filter)
+
+        if self.match_method == 'simple_match':   # add match choice
+            match_gt2pred = match_gt2pred_simple
+        elif self.match_method == 'quick_match':
+            match_gt2pred = match_gt2pred_quick
+        elif self.match_method == 'no_split':
+            match_gt2pred = match_gt2pred_no_split
+        else:
+            # print('Invalid match method name. The quick_match will be used.')
+            match_gt2pred = match_gt2pred_quick
+
+        pred_dataset = md_tex_filter(pred_content)
+        gt_page_elements = self.get_page_elements(sample)
+
+        text_all = self.get_page_elements_list(gt_page_elements, ['text_block', 'title', 'code_txt', 'code_txt_caption', 'reference', 'equation_caption',
+                                                'figure_caption', 'figure_footnote', 'table_caption', 'table_footnote', 'code_algorithm', 'code_algorithm_caption',
+                                                'header', 'footer', 'page_footnote', 'page_number'])
+
+
+        display_formula_match_s = []
+        plain_text_match_clean = []
+        latex_table_match_s = []
+        html_table_match_s = []
+        order_match_single = []
+        if text_all:
+            gt_text_list = self.get_sorted_text_list(text_all)
+            try:
+                plain_text_match_s = func_timeout(
+                    30, match_gt2pred, args=(gt_text_list, pred_dataset['text_all'], 'text', img_name)
+                )
+            except FunctionTimedOut as e1:
+                print(f'Time out for plain text match of {img_name}, match_gt2pred_simple will be used.')
+                plain_text_match_s = match_gt2pred_simple(gt_text_list, pred_dataset['text_all'], 'text', img_name)
+            except Exception as e:
+                print(str(e))
+                sys.exit()
+
+            if not plain_text_match_s:
+                print(f'No text match of {img_name}. The plain text match will be empty.')
+            else:
+                plain_text_match_clean = self.filtered_out_ignore(plain_text_match_s, ['figure_caption', 'figure_footnote', 'table_caption', 'table_footnote', 'code_algorithm', 'code_algorithm_caption', 'header', 'footer', 'page_footnote', 'page_number', 'equation_caption'])
+
+
+        if gt_page_elements.get('equation_isolated'):
+            gt_display_list = self.get_sorted_text_list(gt_page_elements['equation_isolated'])
+            display_formula_match_s = match_gt2pred(gt_display_list, pred_dataset['equation_isolated'], 'formula', img_name)
+            display_formula_match_s = [x for x in display_formula_match_s if x['gt_idx'] != [""]]
+            if not display_formula_match_s:
+                print(f'No display_formula_match of {img_name}. The display_formula_match will be empty.')
+
+        if gt_page_elements.get('table'):
+            gt_table_list = self.get_sorted_text_list(gt_page_elements['table'])
+            if pred_dataset['latex_table']:
+                latex_table_match_s = match_gt2pred_simple(gt_table_list, pred_dataset['latex_table'], 'latex_table', img_name)
+                latex_table_match_s = [x for x in latex_table_match_s if x['gt_idx'] != [""]]
+            if pred_dataset['html_table']:
+                html_table_match_s = match_gt2pred_simple(gt_table_list, pred_dataset['html_table'], 'html_table', img_name)
+                html_table_match_s = [x for x in html_table_match_s if x['gt_idx'] != [""]]
+            else:
+                html_table_match_s = match_gt2pred_simple(gt_table_list, [], 'html_table', img_name)
+                html_table_match_s = [x for x in html_table_match_s if x['gt_idx'] != [""]]
+
+
+        order_match_s = plain_text_match_clean
+        if order_match_s:
+            order_match_single = self.get_order_paired(order_match_s, img_name)
+
+        return [plain_text_match_clean, display_formula_match_s, latex_table_match_s, html_table_match_s, order_match_single]
+
+    def process_generated_metric_results(self,samples,save_name:str='end2end_quick_match'):
+        from .metrics import METRIC_REGISTRY, get_full_labels_results, get_page_split, show_result
+
+        result_all={}
+        page_info={}
+        metircs_dict=self.dafault_metircs_dict
+        pages=self.references #gt_samples list
+
+        for page in pages:
+                img_path=os.path.basename(page['page_info']['image_path'])
+                page_info[img_path]=page['page_info']['page_attribute']
+
+        for element in metircs_dict.keys():
+
+            result={}
+            group_info=metircs_dict[element].get('group',[])
+            # samples = samples.get(element) ##
+            cur_samples = samples[element]
+
+            for metric in metircs_dict[element]['metric']:
+                metric_val = METRIC_REGISTRY.get(metric)
+
+                cur_samples,result_s = metric_val(cur_samples).evaluate(group_info, f"{save_name}_{element}")
+                if result_s:
+                    result.update(result_s)
+
+            if result:
+                print(f"{element}")
+                show_result(result)
+            result_all[element]={}
+
+
+            group_result=get_full_labels_results(cur_samples)
+            page_result=get_page_split(cur_samples,page_info)
+
+            result_all[element]={
+                'all':result,
+                'group':group_result,
+                'page':page_result
+            }
+            if isinstance(cur_samples,list):
+                saved_samples=cur_samples
+            else:
+                saved_samples=cur_samples.samples
+            # NOTE: The original code has a bug here, it will overwrite the result file in each iteration.
+            # I will fix it by adding element to the filename.
+            # NOTE: Fixed typo .josn -> .json
+            result_file = get_intermediate_file_path(self.eval_file, f'_{save_name}_{element}_result', 'json')
+            dump(saved_samples, result_file)
+
+        metric_result_file = get_intermediate_file_path(self.eval_file, f'_{save_name}_metric_result', 'json')
+        dump(result_all, metric_result_file)
+
+        dict_list = []
+        save_dict={}
+        en_overall=[]
+        ch_overall=[]
+        for category_type, metric in [("text_block", "Edit_dist"), ("display_formula", "Edit_dist"), ("display_formula", "CDM"), ("table", "TEDS"), ("table", "Edit_dist"), ("reading_order", "Edit_dist")]:
+            if metric == 'CDM':
+                save_dict[category_type+'_'+metric+'_EN'] = '-'
+                save_dict[category_type+'_'+metric+'_CH'] = '-'
+            elif metric == "TEDS":
+                save_dict[category_type+'_'+metric+'_EN'] = result_all[category_type]["page"][metric]["language: english"] * 100
+                save_dict[category_type+'_'+metric+'_CH'] = result_all[category_type]["page"][metric]["language: simplified_chinese"] * 100
+            else:
+                save_dict[category_type+'_'+metric+'_EN'] = result_all[category_type]["page"][metric].get("language: english", np.nan)
+                save_dict[category_type+'_'+metric+'_CH'] = result_all[category_type]["page"][metric].get("language: simplified_chinese",np.nan)
+            if metric == "Edit_dist":
+                en_overall.append(result_all[category_type]["page"][metric].get("language: english", np.nan))
+                ch_overall.append(result_all[category_type]["page"][metric].get("language: simplified_chinese",np.nan))
+
+        save_dict['overall_EN'] = sum(en_overall) / len(en_overall)
+        save_dict['overall_CH'] = sum(ch_overall) / len(ch_overall)
+        dict_list.append(save_dict)
+        df = pd.DataFrame(dict_list,index=['end2end',]).round(3)
+
+        e2e_eval_file = get_intermediate_file_path(self.eval_file, '_End2End_Evaluation', 'json')
+        dump(result_all, e2e_eval_file)
+        
+        overall_file = get_intermediate_file_path(self.eval_file, '_overall')
+        dump(df, overall_file)
+
+        print(f"The save path of End2End_Evaluation is: {e2e_eval_file}")
+        print(f"The save path of overall metrics is: {overall_file}")
+        return df
+
+
+class table_evalutor():
+    def __init__(self,eval_file,tsv_path):
+        self.eval_file = eval_file
+        gt_key='html'
+        pred_key='pred'
+        self.category_filter='table'
+        self.category_type='table'
+        self.metircs_list=['TEDS','Edit_dist']
+        self.gt_samples,self.table_samples=self.load_data(eval_file,tsv_path,pred_key,gt_key)
+
+    def load_data(self,eval_file,gt_file,pred_key,gt_key):
+        from .data_preprocess import (clean_string, normalized_formula, normalized_table,
+                                      textblock2unicode)
+        samples=[]
+        preds=[]
+        predictions=load(eval_file)['prediction'].tolist()
+        gt_samples=load(gt_file)['answer'].tolist()
+        load_success,load_fail=0,0
+        for i,gt_sample in tqdm(enumerate(gt_samples),desc='Loading data'):
+            try:
+                ans=json.loads(gt_sample)
+                for item in ans['layout_dets']:
+                    if item['category_type']=="table":
+                        item['pred']=predictions[i]
+                        load_success+=1
+                        preds.append(ans)
+
+            except json.JSONDecodeError as e:
+                load_fail+=1
+                continue
+        print(f'load_table_success:{load_success},load_table_fail:{load_fail}')
+
+        count=0
+        for pred in preds:
+            img_name = os.path.basename(pred['page_info']['image_path'])
+            for i, ann in enumerate(pred['layout_dets']):
+                if not ann.get(gt_key):
+                    continue
+                if self.category_filter:
+                    if ann['category_type'] not in self.category_filter:
+                        continue
+                if not ann.get(pred_key):
+                    # print(f'Cannot find pred for {img_name}. ann is {ann}')
+                    # pdb.set_trace()
+                    count += 1
+                    continue
+                else:
+                    gt_text = ann[gt_key]
+                    norm_gt = gt_text
+                    pred_text = ann[pred_key]
+                    norm_pred = pred_text
+                    if self.category_type:
+                        if self.category_type == 'text':
+                            norm_gt = clean_string(textblock2unicode(ann[gt_key]))
+                            norm_pred = clean_string(textblock2unicode(ann[pred_key]))
+                        elif self.category_type == 'formula':
+                            norm_gt = normalized_formula(ann[gt_key])
+                            norm_pred = normalized_formula(ann[pred_key])
+                        elif self.category_type == 'table':
+                            norm_gt = normalized_table(ann[gt_key], gt_key)
+                            norm_pred = normalized_table(ann[pred_key], gt_key)
+                        else:
+                            raise ValueError(f'Invalid category type: {self.category_type}')
+
+                samples.append({
+                    "gt": gt_text,
+                    "norm_gt": norm_gt,
+                    "gt_attribute": [ann['attribute']],
+                    'pred': pred_text,
+                    "norm_pred": norm_pred,
+                    'img_id': img_name
+                })
+
+        print(f'Cannot find pred for {count} samples.')
+        return preds,samples
+
+    def score(self)->dict:
+        metrics=self.process_generated_metric_results()
+        return metrics
+
+    def process_generated_metric_results(self,save_name:str='OmniDocBench_table'):
+        from .metrics import METRIC_REGISTRY, get_full_labels_results, get_page_split, show_result
+
+        p_scores={}
+        page_info={}
+        no_page_flag=False
+        samples=self.table_samples
+        pages=self.gt_samples
+
+        for page in pages:
+            if 'page_info' not in page:
+                no_page_flag=True
+                break
+            img_path=os.path.basename(page['page_info']['image_path'])
+            page_info[img_path]=page['page_info']['page_attribute']
+
+        for metric in self.metircs_list:
+            metric_val=METRIC_REGISTRY.get(metric)
+            samples, result = metric_val(samples).evaluate({}, save_name)
+            if result:
+                p_scores.update(result)
+        show_result(p_scores)
+        group_result=get_full_labels_results(samples)
+        if no_page_flag:
+            page_result={}
+        else:
+            page_result=get_page_split(samples,page_info)
+
+        result_all={
+            'all':p_scores,
+            'group':group_result,
+            'page':page_result
+        }
+
+        metric_result_file = get_intermediate_file_path(self.eval_file, f'_{save_name}_metric_result', 'json')
+        dump(result_all, metric_result_file)
+
+        dict_list=[]
+        dict_list.append(result_all["group"]["TEDS"])
+
+        df4 = pd.DataFrame(dict_list, index=['OmniDocBench_table'])
+        df4 = df4 * 100
+        df4 = df4.round(1)
+        selected_columns = df4[["language: table_en", "language: table_simplified_chinese", "language: table_en_ch_mixed", "line: full_line", "line: less_line", "line: fewer_line", "line: wireless_line",
+                        "with_span: True", "with_span: False", "include_equation: True", "include_equation: False", "include_background: True", "include_background: False", "table_layout: vertical", "table_layout: horizontal"]]
+
+        table_attr_file = get_intermediate_file_path(self.eval_file, '_table_attribute')
+        dump(selected_columns, table_attr_file)
+        print(f'The save path of table_attribute is :{table_attr_file}')
+        return selected_columns
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/requirements.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/requirements.txt
new file mode 100644
index 0000000000000000000000000000000000000000..fe248debd3144e7513d507ac4c9ca88fdd2d3e94
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/requirements.txt
@@ -0,0 +1,13 @@
+accelerate>=0.26.0
+apted
+BeautifulSoup4
+evaluate
+func_timeout
+jmespath
+Levenshtein
+lxml
+nltk
+pylatexenc
+qwen_vl_utils
+scipy
+torchvision
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..4071e8f0c1257fc066f3b03c1c2100df51b79f07
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/OmniDocBench/utils.py
@@ -0,0 +1,1918 @@
+import copy
+import html
+import json
+import os
+import pdb
+import re
+import shutil
+import subprocess
+import sys
+import unicodedata
+import uuid
+from collections import defaultdict
+
+import Levenshtein
+import numpy as np
+from bs4 import BeautifulSoup
+from pylatexenc.latex2text import LatexNodes2Text
+from pylatexenc.latexencode import unicode_to_latex
+from pylatexenc.latexwalker import (LatexCharsNode, LatexEnvironmentNode, LatexGroupNode,
+                                    LatexMacroNode, LatexSpecialsNode, LatexWalker)
+from scipy.optimize import linear_sum_assignment
+
+
+def read_md_file(filepath):
+    with open(filepath, 'r', encoding='utf-8') as file:
+        content = file.read()
+
+    return content
+
+def save_paired_result(preds, gts, save_path):
+    save_result = []
+    formula_id = 0
+    for gt, pred in zip(gts, preds):
+        save_result.append({
+            "gt": gt,
+            "pred": pred,
+            "img_id": formula_id
+        })
+        formula_id += 1
+    with open(save_path, 'w', encoding='utf-8') as f:
+        json.dump(save_result, f, indent=4, ensure_ascii=False)
+
+def remove_markdown_fences(content):
+    content = re.sub(r'^```markdown\n?', '', content, flags=re.MULTILINE)
+    content = re.sub(r'```\n?$', '', content, flags=re.MULTILINE)
+    return content
+
+# Standardize all consecutive characters
+def replace_repeated_chars(input_str):
+    input_str = re.sub(r'_{4,}', '____', input_str) # Replace more than 4 consecutive underscores with 4 underscores
+    input_str = re.sub(r' {4,}', '    ', input_str)   # Replace more than 4 consecutive spaces with 4 spaces
+    return re.sub(r'([^a-zA-Z0-9])\1{10,}', r'\1\1\1\1', input_str) # For other consecutive symbols (except numbers and letters), replace more than 10 occurrences with 4
+
+# Special Unicode handling
+def fullwidth_to_halfwidth(s):
+    result = []
+    for char in s:
+        code = ord(char)
+        # Convert full-width space to half-width space
+        if code == 0x3000:
+            code = 0x0020
+        # Convert other full-width characters to half-width
+        elif 0xFF01 <= code <= 0xFF5E:
+            code -= 0xFEE0
+        result.append(chr(code))
+    return ''.join(result)
+
+def find_special_unicode(s):
+    special_chars = {}
+    for char in s:
+        if ord(char) > 127:  # Non-ASCII characters
+            # unicode_name = unicodedata.name(char, None)
+            unicode_name = unicodedata.category(char)
+            special_chars[char] = f'U+{ord(char):04X} ({unicode_name})'
+    return special_chars
+
+
+inline_reg = re.compile(
+    r'\$(.*?)\$|'
+    r'\\\((.*?)\\\)',
+)
+
+def textblock2unicode(text):
+    inline_matches = inline_reg.finditer(text)
+    removal_positions = []
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        # print('-------- content-------', content)
+        # Remove escape characters \
+        clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+        try:
+            if any(char in clean_content for char in r'\^_'):
+                if clean_content.endswith('\\'):
+                    clean_content += ' '
+                # inline_array.append(match.group(0))
+                unicode_content = LatexNodes2Text().latex_to_text(clean_content)
+                removal_positions.append((position[0], position[1], unicode_content))
+        except:
+            continue
+
+    # Remove inline formulas from original text
+    for start, end, unicode_content in sorted(removal_positions, reverse=True):
+        text = text[:start] + unicode_content.strip() + text[end:]
+
+    return text
+
+def normalized_formula(text):
+    # Normalize math formulas before matching
+    filter_list = ['\\mathbf', '\\mathrm', '\\mathnormal', '\\mathit', '\\mathbb', '\\mathcal', '\\mathscr', '\\mathfrak', '\\mathsf', '\\mathtt',
+                   '\\textbf', '\\text', '\\boldmath', '\\boldsymbol', '\\operatorname', '\\bm',
+                   '\\symbfit', '\\mathbfcal', '\\symbf', '\\scriptscriptstyle', '\\notag',
+                   '\\setlength', '\\coloneqq', '\\space', '\\thickspace', '\\thinspace', '\\medspace', '\\nobreakspace', '\\negmedspace',
+                   '\\quad', '\\qquad', '\\enspace', '\\substackw', ' ']
+                #    '\\left', '\\right', '{', '}', ' ']
+
+    # delimiter_filter
+    pattern = re.compile(r"\\\[(.+?)(?<!\\)\\\]")
+    match = pattern.search(text)
+
+    if match:
+        text = match.group(1).strip()
+
+    tag_pattern = re.compile(r"\\tag\{.*?\}")
+    text = tag_pattern.sub('', text)
+    hspace_pattern = re.compile(r"\\hspace\{.*?\}")
+    text = hspace_pattern.sub('', text)
+    begin_pattern = re.compile(r"\\begin\{.*?\}")
+    text = begin_pattern.sub('', text)
+    end_pattern = re.compile(r"\\end\{.*?\}")
+    text = end_pattern.sub('', text)
+    col_sep = re.compile(r"\\arraycolsep.*?\}")
+    text = col_sep.sub('', text)
+    text = text.strip('.')
+
+    for filter_text in filter_list:
+        text = text.replace(filter_text, '')
+
+    # text = normalize_text(delimiter_filter(text))
+    # text = delimiter_filter(text)
+    text = text.lower()
+    return text
+
+def normalized_html_table(text):
+    def process_table_html(md_i):
+        """
+        pred_md format edit
+        """
+        def process_table_html(html_content):
+            soup = BeautifulSoup(html_content, 'html.parser')
+            th_tags = soup.find_all('th')
+            for th in th_tags:
+                th.name = 'td'
+            thead_tags = soup.find_all('thead')
+            for thead in thead_tags:
+                thead.unwrap()  # unwrap()会移除标签但保留其内容
+            math_tags = soup.find_all('math')
+            for math_tag in math_tags:
+                alttext = math_tag.get('alttext', '')
+                alttext = f'${alttext}$'
+                if alttext:
+                    math_tag.replace_with(alttext)
+            span_tags = soup.find_all('span')
+            for span in span_tags:
+                span.unwrap()
+            return str(soup)
+
+        table_res=''
+        table_res_no_space=''
+        if '<table' in md_i.replace(" ","").replace("'",'"'):
+            md_i = process_table_html(md_i)
+            table_res = html.unescape(md_i).replace('\n', '')
+            table_res = unicodedata.normalize('NFKC', table_res).strip()
+            pattern = r'<table\b[^>]*>(.*)</table>'
+            tables = re.findall(pattern, table_res, re.DOTALL | re.IGNORECASE)
+            table_res = ''.join(tables)
+            # table_res = re.sub('<table.*?>','',table_res)
+            table_res = re.sub('( style=".*?")', "", table_res)
+            table_res = re.sub('( height=".*?")', "", table_res)
+            table_res = re.sub('( width=".*?")', "", table_res)
+            table_res = re.sub('( align=".*?")', "", table_res)
+            table_res = re.sub('( class=".*?")', "", table_res)
+            table_res = re.sub('</?tbody>',"",table_res)
+
+            table_res = re.sub(r'\s+', " ", table_res)
+            table_res_no_space = '<html><body><table border="1" >' + table_res.replace(' ','') + '</table></body></html>'
+            # table_res_no_space = re.sub(' (style=".*?")',"",table_res_no_space)
+            # table_res_no_space = re.sub(r'[ ]', " ", table_res_no_space)
+            table_res_no_space = re.sub('colspan="', ' colspan="', table_res_no_space)
+            table_res_no_space = re.sub('rowspan="', ' rowspan="', table_res_no_space)
+            table_res_no_space = re.sub('border="', ' border="', table_res_no_space)
+
+            table_res = '<html><body><table border="1" >' + table_res + '</table></body></html>'
+            # table_flow.append(table_res)
+            # table_flow_no_space.append(table_res_no_space)
+
+        return table_res, table_res_no_space
+
+    def clean_table(input_str,flag=True):
+        if flag:
+            input_str = input_str.replace('<sup>', '').replace('</sup>', '')
+            input_str = input_str.replace('<sub>', '').replace('</sub>', '')
+            input_str = input_str.replace('<span>', '').replace('</span>', '')
+            input_str = input_str.replace('<div>', '').replace('</div>', '')
+            input_str = input_str.replace('<p>', '').replace('</p>', '')
+            input_str = input_str.replace('<spandata-span-identity="">', '')
+            input_str = re.sub('<colgroup>.*?</colgroup>','',input_str)
+        return input_str
+
+    norm_text, _ = process_table_html(text)
+    norm_text = clean_table(norm_text)
+    return norm_text
+
+def normalized_latex_table(text):
+    def latex_template(latex_code):
+        template = r'''
+        \documentclass[border=20pt]{article}
+        \usepackage{subcaption}
+        \usepackage{url}
+        \usepackage{graphicx}
+        \usepackage{caption}
+        \usepackage{multirow}
+        \usepackage{booktabs}
+        \usepackage{color}
+        \usepackage{colortbl}
+        \usepackage{xcolor,soul,framed}
+        \usepackage{fontspec}
+        \usepackage{amsmath,amssymb,mathtools,bm,mathrsfs,textcomp}
+        \setlength{\parindent}{0pt}''' + \
+        r'''
+        \begin{document}
+        ''' + \
+        latex_code + \
+        r'''
+        \end{document}'''
+
+        return template
+
+    def process_table_latex(latex_code):
+        SPECIAL_STRINGS= [
+            ['\\\\vspace\\{.*?\\}', ''],
+            ['\\\\hspace\\{.*?\\}', ''],
+            ['\\\\rule\{.*?\\}\\{.*?\\}', ''],
+            ['\\\\addlinespace\\[.*?\\]', ''],
+            ['\\\\addlinespace', ''],
+            ['\\\\renewcommand\\{\\\\arraystretch\\}\\{.*?\\}', ''],
+            ['\\\\arraystretch\\{.*?\\}', ''],
+            ['\\\\(row|column)?colors?\\{[^}]*\\}(\\{[^}]*\\}){0,2}', ''],
+            ['\\\\color\\{.*?\\}', ''],
+            ['\\\\textcolor\\{.*?\\}', ''],
+            ['\\\\rowcolor(\\[.*?\\])?\\{.*?\\}', ''],
+            ['\\\\columncolor(\\[.*?\\])?\\{.*?\\}', ''],
+            ['\\\\cellcolor(\\[.*?\\])?\\{.*?\\}', ''],
+            ['\\\\colorbox\\{.*?\\}', ''],
+            ['\\\\(tiny|scriptsize|footnotesize|small|normalsize|large|Large|LARGE|huge|Huge)', ''],
+            [r'\s+', ' '],
+            ['\\\\centering', ''],
+            ['\\\\begin\\{table\\}\\[.*?\\]', '\\\\begin{table}'],
+            ['\t', ''],
+            ['@{}', ''],
+            ['\\\\toprule(\\[.*?\\])?', '\\\\hline'],
+            ['\\\\bottomrule(\\[.*?\\])?', '\\\\hline'],
+            ['\\\\midrule(\\[.*?\\])?', '\\\\hline'],
+            ['p\\{[^}]*\\}', 'l'],
+            ['m\\{[^}]*\\}', 'c'],
+            ['\\\\scalebox\\{[^}]*\\}\\{([^}]*)\\}', '\\1'],
+            ['\\\\textbf\\{([^}]*)\\}', '\\1'],
+            ['\\\\textit\\{([^}]*)\\}', '\\1'],
+            ['\\\\cmidrule(\\[.*?\\])?\\(.*?\\)\\{([0-9]-[0-9])\\}', '\\\\cline{\\2}'],
+            ['\\\\hline', ''],
+            [r'\\multicolumn\{1\}\{[^}]*\}\{((?:[^{}]|(?:\{[^{}]*\}))*)\}', r'\1']
+        ]
+        pattern = r'\\begin\{tabular\}.*\\end\{tabular\}'  # 注意这里不用 .*?
+        matches = re.findall(pattern, latex_code, re.DOTALL)
+        latex_code = ' '.join(matches)
+
+        for special_str in SPECIAL_STRINGS:
+            latex_code = re.sub(fr'{special_str[0]}', fr'{special_str[1]}', latex_code)
+
+        return latex_code
+
+    def convert_latex_to_html(latex_content, cache_dir='./temp'):
+        if not os.path.exists(cache_dir):
+            os.makedirs(cache_dir)
+
+        uuid_str = str(uuid.uuid1())
+        with open(f'{cache_dir}/{uuid_str}.tex', 'w') as f:
+            f.write(latex_template(latex_content))
+
+        cmd = ['latexmlc', '--quiet', '--nocomments', f'--log={cache_dir}/{uuid_str}.log',
+               f'{cache_dir}/{uuid_str}.tex', f'--dest={cache_dir}/{uuid_str}.html']
+        try:
+            subprocess.run(cmd, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+            with open(f'{cache_dir}/{uuid_str}.html', 'r') as f:
+                html_content = f.read()
+
+            pattern = r'<table\b[^>]*>(.*)</table>'
+            tables = re.findall(pattern, html_content, re.DOTALL | re.IGNORECASE)
+            tables = [f'<table>{table}</table>' for table in tables]
+            html_content = '\n'.join(tables)
+
+        except Exception as e:
+            html_content = ''
+
+        shutil.rmtree(cache_dir)
+        return html_content
+
+    html_text = convert_latex_to_html(text)
+    normlized_tables = normalized_html_table(html_text)
+    return normlized_tables
+
+
+def normalized_table(text, format='html'):
+    if format not in ['html', 'latex']:
+        raise ValueError('Invalid format: {}'.format(format))
+    else:
+        return globals()['normalized_{}_table'.format(format)](text)
+
+
+def textblock_with_norm_formula(text):
+    inline_matches = inline_reg.finditer(text)
+    removal_positions = []
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        # print('-------- content-------', content)
+
+        norm_content = normalized_formula(content)
+        removal_positions.append((position[0], position[1], norm_content))
+
+    # Remove inline formulas from original text
+    for start, end, norm_content in sorted(removal_positions, reverse=True):
+        text = text[:start] + norm_content.strip() + text[end:]
+
+    return text
+
+
+def inline_filter_unicode(text):
+    # Ensure text is string type
+    if not isinstance(text, str):
+        text = str(text)
+
+    # Replace inline formula boundary markers
+    #print('--------text-------',text)
+    placeholder = '__INLINE_FORMULA_BOUNDARY__'
+    text_copy = text.replace('$', placeholder).replace('\\(', placeholder).replace('\\)', placeholder)
+    #print('--------text_copy-------',text_copy)
+    # Convert LaTeX content to Unicode representation
+    text_copy = LatexNodes2Text().latex_to_text(text_copy)
+    #print('--------text_copy---unicode----',text_copy)
+    # Restore boundary markers
+    text_copy = text_copy.replace(placeholder, '$')
+
+    inline_array = []
+    inline_matches = inline_reg.finditer(text_copy)
+    # Record positions of inline formulas to be removed
+    removal_positions = []
+
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        print('-------- content-------', content)
+        # Remove escape characters \
+        clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+        if any(char in clean_content for char in r'\^_'):
+            # inline_array.append(match.group(0))
+            inline_array.append({
+                'category_type': 'equation_inline',
+                'position': position,
+                'content': content,
+            })
+            removal_positions.append((position[0], position[1]))
+
+    # Remove inline formulas from original text
+    for start, end in sorted(removal_positions, reverse=True):
+        text = text[:start] + text[end:]
+
+    return text, inline_array
+
+def inline_filter(text):
+    # Ensure text is string type
+    if not isinstance(text, str):
+        text = str(text)
+
+    inline_array = []
+    inline_matches = inline_reg.finditer(text)
+
+    for match in inline_matches:
+        position = [match.start(), match.end()]
+        content = match.group(1) if match.group(1) is not None else match.group(2)
+        # print('inline_content: ', content)
+
+        # Remove escape characters \
+        clean_content = re.sub(r'\\([\\_&%^])', '', content)
+
+        if any(char in clean_content for char in r'\^_'):
+            # inline_array.append(match.group(0))
+            inline_array.append({
+                'category_type': 'equation_inline',
+                'position': position,
+                'content': match.group(0),
+            })
+            text = text.replace(match.group(0), '')
+            # print('-----Found inline formula: ', match.group(0))
+        else:
+            text = text.replace(match.group(0), content)
+
+    return text, inline_array
+
+# Text OCR quality check processing:
+def clean_string(input_string):
+    # Use regex to keep Chinese characters, English letters and numbers
+    input_string = input_string.replace('\\t', '').replace('\\n', '').replace('\t', '').replace('\n', '').replace('/t', '').replace('/n', '')
+    cleaned_string = re.sub(r'[^\w\u4e00-\u9fff]', '', input_string)
+    return cleaned_string
+
+def extract_tabular(text):
+    begin_pattern = r'\\begin{tabular}'
+    end_pattern = r'\\end{tabular}'
+
+    tabulars = []
+    positions = []
+    current_pos = 0
+    stack = []
+
+    while current_pos < len(text):
+        begin_match = re.search(begin_pattern, text[current_pos:])
+        end_match = re.search(end_pattern, text[current_pos:])
+
+        if not begin_match and not end_match:
+            break
+
+        if begin_match and (not end_match or begin_match.start() < end_match.start()):
+            stack.append(current_pos + begin_match.start())
+            current_pos += begin_match.start() + len(end_pattern)
+        elif end_match:
+            if stack:
+                start_pos = stack.pop()
+                if not stack:
+                    end_pos = current_pos + end_match.start() + len(end_pattern)
+                    tabular_code = text[start_pos:end_pos]
+                    tabulars.append(tabular_code)
+                    positions.append((start_pos, end_pos))
+            current_pos += end_match.start() + len(end_pattern)
+        else:
+            current_pos += 1
+
+    if stack:
+        new_start = stack[0] + len(begin_pattern)
+        new_tabulars, new_positions = extract_tabular(text[new_start:])
+        new_positions = [(start + new_start, end + new_start) for start, end in new_positions]
+        tabulars.extend(new_tabulars)
+        positions.extend(new_positions)
+
+    return tabulars, positions
+
+# math reg
+    # r'\\begin{equation\*?}(.*?)\\end{equation\*?}|'
+    # r'\\begin{align\*?}(.*?)\\end{align\*?}|'
+    # r'\\begin{gather\*?}(.*?)\\end{gather\*?}|'
+display_reg = re.compile(
+    r'\$\$(.*?)\$\$|'
+    r'\\\[(.*?)\\\]|'
+    r'\$(.*?)\$|'
+    r'\\\((.*?)\\\)',
+    re.DOTALL
+)
+
+# inline_reg = re.compile(
+#     r'(?<!\$)\$(?!\$)(.*?)(?<!\$)\$(?!\$)|'
+#     r'\\\((.*?)\\\)',
+# )
+inline_reg = re.compile(
+    r'\$(.*?)\$|'
+    r'\\\((.*?)\\\)',
+)
+
+# table
+table_reg = re.compile(
+    r'\\begin{table\*?}(.*?)\\end{table\*?}|'
+    r'\\begin{tabular\*?}(.*?)\\end{tabular\*?}',
+    re.DOTALL
+)
+md_table_reg = re.compile(
+    r'\|\s*.*?\s*\|\n',
+    re.DOTALL)
+html_table_reg = re.compile(
+    r'(<table.*?</table>)',
+    re.DOTALL
+)
+
+# title
+title_reg = re.compile(
+    r'^\s*#.*$',
+    re.MULTILINE)
+
+# img
+img_pattern = r'!\[.*?\]\(.*?\)'
+
+# code block
+code_block_reg = re.compile(
+    r'```(\w+)\n(.*?)```',
+    re.DOTALL
+)
+
+
+def md_tex_filter(content):
+    '''
+    Input: 1 page md or tex content - String
+    Output: text, display, inline, table, title, code - list
+    '''
+    content = re.sub(img_pattern, '', content)  # remove image
+    content = remove_markdown_fences(content)   # remove markdown fences
+    content = replace_repeated_chars(content) # replace all consecutive characters
+
+
+
+    pred_all = []
+    latex_table_array, table_positions = extract_tex_table(content)
+    for latex_table, position in zip(latex_table_array, table_positions):
+        position = [position[0], position[0]+len(latex_table)]   # !!!
+        pred_all.append({
+            'category_type': 'latex_table',
+            'position': position,
+            'content': latex_table
+        })
+        content = content[:position[0]] + ' '*(position[1]-position[0]) + content[position[1]:]  # replace latex table with space
+
+
+    # extract html table
+    html_table_array, table_positions = extract_html_table(content)
+    for html_table, position in zip(html_table_array, table_positions):
+        position = [position[0], position[0]+len(html_table)]
+        pred_all.append({
+            'category_type': 'html_table',
+            'position': position,
+            'content': html_table
+        })
+        content = content[:position[0]] + ' '*(position[1]-position[0]) + content[position[1]:]  # replace html table with space
+
+    # extract interline formula
+    display_matches = display_reg.finditer(content)
+    for match in display_matches:
+        matched = match.group(0)
+        if matched:
+            single_line = ''.join(matched.split())
+            position = [match.start(), match.end()]
+            # replace $$ with \[\]
+            dollar_pattern = re.compile(r'\$\$(.*?)\$\$|\$(.*?)\$|\\\((.*?)\\\)', re.DOTALL)
+            sub_match = dollar_pattern.search(single_line)
+            if sub_match is None:
+                # pass
+                content = content[:position[0]] + ' '*(position[1]-position[0]) + content[position[1]:]
+                pred_all.append({
+                    'category_type': 'equation_isolated',
+                    'position': position,
+                    'content': single_line
+                })
+            elif sub_match.group(1):
+                single_line = re.sub(dollar_pattern, r'\\[\1\\]', single_line)
+                content = content[:position[0]] + ' '*(position[1]-position[0]) + content[position[1]:]  # replace equation with space
+                pred_all.append({
+                    'category_type': 'equation_isolated',
+                    'position': position,
+                    'content': single_line
+                })
+            else:
+                single_line = re.sub(dollar_pattern, r'\\[\2\3\\]', single_line)
+                pred_all.append({
+                    'category_type': 'equation_isolated',
+                    'position': position,
+                    'content': single_line,
+                    'fine_category_type': 'equation_inline'
+                })
+
+
+    # extract md table with ||
+    md_table_mathces = md_table_reg.findall(content+'\n')
+    if len(md_table_mathces) >= 2:
+        # print("md table found!")
+        # print("content:", content)
+        content = convert_markdown_to_html(content)
+        # print('----------content after converting md table to html:', content)
+        html_table_matches = html_table_reg.finditer(content)
+        if html_table_matches:
+            for match in html_table_matches:
+                matched = match.group(0)
+                position = [match.start(), match.end()]
+                # content = content.replace(match, '')
+                # print('content after removing the md table:', content)
+                content = content[:position[0]] + ' '*(position[1]-position[0]) + content[position[1]:]  # replace md table with space
+                pred_all.append({
+                    'category_type': 'html_table',
+                    'position': position,
+                    'content': matched.strip(),
+                    'fine_category_type': 'md2html_table'
+                })
+    # print('---------After md table: \n', content)
+
+    # extract code blocks
+    code_matches = code_block_reg.finditer(content)
+    if code_matches:
+        for match in code_matches:
+            position = [match.start(), match.end()]
+            language = match.group(1)
+            code = match.group(2).strip()
+            # content = content.replace(match.group(0), '')
+            content = content[:position[0]] + ' '*(position[1]-position[0]) + content[position[1]:]  # replace code block with space
+            pred_all.append({
+                'category_type': 'text_all',
+                'position': position,
+                'content': code,
+                'language': language,
+                'fine_category_type': 'code'
+            })
+
+
+    # Remove latex style
+    content = re.sub(r'\\title\{(.*?)\}', r'\1', content)
+    content = re.sub(r'\\title\s*\{\s*(.*?)\s*\}', r'\1', content, flags=re.DOTALL)
+    content = re.sub(r'\\text\s*\{\s*(.*?)\s*\}', r'\1', content, flags=re.DOTALL)
+    content = re.sub(r'\\section\*?\{(.*?)\}', r'\1', content)
+    content = re.sub(r'\\section\*?\{\s*(.*?)\s*\}', r'\1', content, flags=re.DOTALL)
+
+    # extract texts
+    res = content.split('\n\n')
+    if len(res) == 1:
+        res = content.split('\n')  # some models do not use double newlines, so use single newlines to split
+
+    content_position = 0
+    for text in res:
+        position = [content_position, content_position+len(text)]
+        content_position += len(text)
+        text = text.strip()
+        text = text.strip('\n')
+        # print('ori_text: ', text)
+        text = '\n'.join([_.strip() for _ in text.split('\n') if _.strip()])   # avoid some single newline content with many spaces
+        # print('after strip text: ', text)
+
+        if text:  # Check if the stripped text is not empty
+            if text.startswith('<table') and text.endswith('</table>'):
+                pred_all.append({
+                    'category_type': 'html_table',
+                    'position': position,
+                    'content': text,
+                })
+
+            elif text.startswith('$') and text.endswith('$'):
+                if text.replace('$', '').strip():
+                    pred_all.append({
+                        'category_type': 'equation_isolated',
+                        'position': position,
+                        'content': text.strip(),
+                    })
+            else:
+                text = text.strip()
+                if text:
+                    pred_all.append({
+                        'category_type': 'text_all',
+                        'position': position,
+                        'content': text,
+                        'fine_category_type': 'text_block'
+                    })
+
+    pred_dataset = defaultdict(list)
+    pred_all = sorted(pred_all, key=lambda x: x['position'][0])
+    for item in pred_all:
+        pred_dataset[item['category_type']].append(item)
+    # pdb.set_trace()
+    return pred_dataset
+
+
+def extract_tex_table(content):
+    tables = []
+    tables_positions = []
+
+    pattern = r'\\begin{table}(.*?)\\end{table}'
+    for match in re.finditer(pattern, content, re.DOTALL):
+        start_pos = match.start()
+        end_pos = match.end()
+        table_content = match.group(0)
+        tables.append(table_content)
+        tables_positions.append((start_pos, end_pos))
+        content = content[:start_pos] + ' '*(end_pos-start_pos) + content[end_pos:]
+
+    tabulars, tabular_positions = extract_tabular(content)
+    all_tables = tables + tabulars
+    all_positions = tables_positions + tabular_positions
+
+    all_result = sorted([[pos, table]for pos, table in zip(all_positions, all_tables)], key=lambda x: x[0][0])
+    all_tables = [x[1] for x in all_result]
+    all_positions = [x[0] for x in all_result]
+
+    return all_tables, all_positions
+
+
+def extract_html_table(text):
+    begin_pattern = r'<table(?:[^>]*)>'
+    end_pattern = r'</table>'
+
+    tabulars = []
+    positions = []
+    current_pos = 0
+    stack = []
+
+    while current_pos < len(text):
+        begin_match = re.search(begin_pattern, text[current_pos:])
+        end_match = re.search(end_pattern, text[current_pos:])
+
+        if not begin_match and not end_match:
+            break
+
+        if begin_match and (not end_match or begin_match.start() < end_match.start()):
+            stack.append(current_pos + begin_match.start())
+            current_pos += begin_match.start() + len(end_pattern)
+        elif end_match:
+            if stack:
+                start_pos = stack.pop()
+                if not stack:
+                    end_pos = current_pos + end_match.start() + len(end_pattern)
+                    tabular_code = text[start_pos:end_pos]
+                    tabulars.append(tabular_code)
+                    positions.append((start_pos, end_pos))
+            current_pos += end_match.start() + len(end_pattern)
+        else:
+            current_pos += 1
+
+    if stack:
+        new_start = stack[0] + len(begin_pattern)
+        new_tabulars, new_positions = extract_html_table(text[new_start:])
+        new_positions = [(start + new_start, end + new_start) for start, end in new_positions]
+        tabulars.extend(new_tabulars)
+        positions.extend(new_positions)
+
+    return tabulars, positions
+
+
+def extract_node_content(node):
+    """ Recursively extract content from LatexEnvironmentNode and rebuild LaTeX table representation """
+    if isinstance(node, LatexCharsNode):
+        return node.chars  # Use chars attribute
+    elif isinstance(node, LatexGroupNode):
+        return "{" + "".join(extract_node_content(n) for n in node.nodelist) + "}"
+    elif isinstance(node, LatexMacroNode):
+        # Extract macro command and its arguments
+        macro_content = "\\" + node.macroname
+        if node.nodeargs:
+            macro_content += "".join([extract_node_content(arg) for arg in node.nodeargs])
+        return macro_content
+    elif isinstance(node, LatexEnvironmentNode):
+        # Extract environment, preserve environment name and arguments
+        content = "\\begin{" + node.environmentname + "}"
+        if node.nodeargd and node.nodeargd.argnlist:
+            # content += "".join("{" + extract_node_content(arg) + "}" for arg in node.nodeargd)
+            # content += "".join("{" + extract_node_content(node.nodeargd) + "}")
+            content += "{" + extract_node_content(node.nodeargd.argnlist[0]) + "}"
+        if node.nodelist:
+            content += "".join(extract_node_content(n) for n in node.nodelist)
+        content += "\\end{" + node.environmentname + "}"
+        return content
+    elif isinstance(node, LatexSpecialsNode):  # Changed to LatexSpecialsNode
+        return node.specials_chars
+    else:
+        return ""
+
+def get_node_end_pos(node):
+    """Recursively determine the end position of a node"""
+    if hasattr(node, 'nodelist') and node.nodelist:
+        # If the node has child nodes, recursively find the end position of the last child node
+        return get_node_end_pos(node.nodelist[-1])
+    elif hasattr(node, 'pos_end'):
+        # If the node has pos_end attribute, return it directly
+        return node.pos_end
+    else:
+        # If there are no child nodes, assume the node ends at the last character of its content
+        return node.pos + len(str(node))
+
+def remove_tex_table(content):
+    tables, positions = extract_tex_table(content)
+
+    # Delete in reverse order by position to avoid affecting unprocessed start positions
+    for start, end in sorted(positions, reverse=True):
+        content = content[:start] + content[end:]  # Remove table content
+
+    return content
+
+
+
+def get_pred_category_type(pred_idx, pred_items):
+    # if pred_idx:
+    if pred_items[pred_idx].get('fine_category_type'):
+        pred_pred_category_type = pred_items[pred_idx]['fine_category_type']
+    else:
+        pred_pred_category_type = pred_items[pred_idx]['category_type']
+    # else:
+    #     pred_pred_category_type = ""
+    return pred_pred_category_type
+
+
+def compute_edit_distance_matrix_new(gt_lines, matched_lines):
+    try:
+        distance_matrix = np.zeros((len(gt_lines), len(matched_lines)))
+        for i, gt_line in enumerate(gt_lines):
+            for j, matched_line in enumerate(matched_lines):
+                if len(gt_line) == 0 and len(matched_line) == 0:
+                    distance_matrix[i][j] = 0
+                else:
+                    distance_matrix[i][j] = Levenshtein.distance(gt_line, matched_line) / max(len(matched_line), len(gt_line))
+        return distance_matrix
+    except ZeroDivisionError:
+        #print("ZeroDivisionError occurred. Outputting norm_gt_lines and norm_pred_lines:")
+        # print("norm_gt_lines:", gt_lines)
+        # print("norm_pred_lines:", matched_lines)
+        raise
+
+def get_gt_pred_lines(gt_items, pred_items, line_type):
+    norm_html_lines = []
+    gt_lines = []
+    gt_cat_list = []
+    for item in gt_items:
+        if item.get('fine_category_type'):
+            gt_cat_list.append(item['fine_category_type'])
+        else:
+            gt_cat_list.append(item['category_type'])
+        if item.get('content'):
+            gt_lines.append(str(item['content']))
+            norm_html_lines.append(str(item['content']))
+        elif line_type == 'text':
+            gt_lines.append(str(item['text']))
+        elif line_type == 'html_table':
+            gt_lines.append(str(item['html']))
+        elif line_type == 'formula':
+            gt_lines.append(str(item['latex']))
+        elif line_type == 'latex_table':
+            gt_lines.append(str(item['latex']))
+            norm_html_lines.append(str(item['html']))
+
+    pred_lines = [str(item['content']) for item in pred_items]
+
+
+    if line_type == 'formula':
+        norm_gt_lines = [normalized_formula(_) for _ in gt_lines]
+        norm_pred_lines = [normalized_formula(_) for _ in pred_lines]
+    elif line_type == 'text':
+        # norm_gt_lines = [textblock_with_norm_formula(_) for _ in gt_lines]
+        # norm_pred_lines = [textblock_with_norm_formula(_) for _ in pred_lines]
+        norm_gt_lines = [clean_string(textblock2unicode(_)) for _ in gt_lines]
+        norm_pred_lines = [clean_string(textblock2unicode(_)) for _ in pred_lines]
+        # norm_gt_lines = get_norm_text_lines(gt_lines)
+        # norm_pred_lines = get_norm_text_lines(pred_lines)
+    else:
+        norm_gt_lines = gt_lines
+        norm_pred_lines = pred_lines
+
+    if line_type == 'latex_table':
+        gt_lines = norm_html_lines
+
+
+    filtered_lists = [(a, b, c) for a, b, c in zip(gt_lines, norm_gt_lines, gt_cat_list) if a and b]
+
+    # decompress to three lists
+    if filtered_lists:
+        gt_lines_c, norm_gt_lines_c, gt_cat_list_c = zip(*filtered_lists)
+
+        # convert to lists
+        gt_lines_c = list(gt_lines_c)
+        norm_gt_lines_c = list(norm_gt_lines_c)
+        gt_cat_list_c = list(gt_cat_list_c)
+    else:
+        gt_lines_c = []
+        norm_gt_lines_c = []
+        gt_cat_list_c = []
+
+    # pred's empty values
+    filtered_lists = [(a, b) for a, b in zip(pred_lines, norm_pred_lines) if a and b]
+
+    # decompress to two lists
+    if filtered_lists:
+        pred_lines_c, norm_pred_lines_c = zip(*filtered_lists)
+
+        # convert to lists
+        pred_lines_c = list(pred_lines_c)
+        norm_pred_lines_c = list(norm_pred_lines_c)
+    else:
+        pred_lines_c = []
+        norm_pred_lines_c = []
+
+    return gt_lines_c, norm_gt_lines_c, gt_cat_list_c, pred_lines_c, norm_pred_lines_c
+    # return gt_lines, norm_gt_lines, gt_cat_list, pred_lines, norm_pred_lines
+
+
+def match_gt2pred_simple(gt_items, pred_items, line_type, img_name):
+
+    gt_lines, norm_gt_lines, gt_cat_list, pred_lines, norm_pred_lines = get_gt_pred_lines(gt_items, pred_items, line_type)
+
+    match_list = []
+    if not norm_gt_lines: # not matched pred should be concatenated
+        # print("One of the lists is empty. Returning an empty gt result.")
+        # for pred_idx in range(len(norm_pred_lines)):
+        pred_idx_list = range(len(norm_pred_lines))
+        match_list.append({
+            'gt_idx': [""],
+            'gt': "",
+            'pred_idx': pred_idx_list,
+            'pred': ''.join(pred_lines[_] for _ in pred_idx_list),
+            'gt_position': [""],
+            'pred_position': pred_items[pred_idx_list[0]]['position'][0],  # get the first pred's position
+            'norm_gt': "",
+            'norm_pred': ''.join(norm_pred_lines[_] for _ in pred_idx_list),
+            'gt_category_type': "",
+            'pred_category_type': get_pred_category_type(pred_idx_list[0], pred_items), # get the first pred's category
+            'gt_attribute': [{}],
+            'edit': 1,
+            'img_id': img_name
+        })
+        return match_list
+    elif not norm_pred_lines: # not matched gt should be separated
+        # print("One of the lists is empty. Returning an empty pred result.")
+        for gt_idx in range(len(norm_gt_lines)):
+            match_list.append({
+                'gt_idx': [gt_idx],
+                'gt': gt_lines[gt_idx],
+                'pred_idx': [""],
+                'pred': "",
+                'gt_position': [gt_items[gt_idx].get('order') if gt_items[gt_idx].get('order') else gt_items[gt_idx].get('position', [""])[0]],
+                'pred_position': "",
+                'norm_gt': norm_gt_lines[gt_idx],
+                'norm_pred': "",
+                'gt_category_type': gt_cat_list[gt_idx],
+                'pred_category_type': "",
+                'gt_attribute': [gt_items[gt_idx].get("attribute", {})],
+                'edit': 1,
+                'img_id': img_name
+            })
+        return match_list
+
+    cost_matrix = compute_edit_distance_matrix_new(norm_gt_lines, norm_pred_lines)
+
+    row_ind, col_ind = linear_sum_assignment(cost_matrix)
+
+
+    for gt_idx in range(len(norm_gt_lines)):
+        if gt_idx in row_ind:
+            row_i = list(row_ind).index(gt_idx)
+            pred_idx = int(col_ind[row_i])
+            pred_line = pred_lines[pred_idx]
+            norm_pred_line = norm_pred_lines[pred_idx]
+            edit = cost_matrix[gt_idx][pred_idx]
+            # print('edit_dist', edit)
+            # if edit > 0.7:
+            #     print('! Not match')
+        else:
+            # print('No match pred')
+            pred_idx = ""
+            pred_line = ""
+            norm_pred_line = ""
+            edit = 1
+
+        match_list.append({
+            'gt_idx': [gt_idx],
+            'gt': gt_lines[gt_idx],
+            'norm_gt': norm_gt_lines[gt_idx],
+            'gt_category_type': gt_cat_list[gt_idx],
+            'gt_position': [gt_items[gt_idx].get('order') if gt_items[gt_idx].get('order') else gt_items[gt_idx].get('position', [""])[0]],
+            'gt_attribute': [gt_items[gt_idx].get("attribute", {})],
+            'pred_idx': [pred_idx],
+            'pred': pred_line,
+            'norm_pred': norm_pred_line,
+            'pred_category_type': get_pred_category_type(pred_idx, pred_items) if pred_idx else "",
+            'pred_position': pred_items[pred_idx]['position'][0] if pred_idx else "",
+            'edit': edit,
+            'img_id': img_name
+        })
+        # print('-'*10)
+        # [([0,1], 0),(2, 1), (1,2)] --> [0,2,1]/[0,1,2]
+
+    pred_idx_list = [pred_idx for pred_idx in range(len(norm_pred_lines)) if pred_idx not in col_ind] # get not matched preds
+    if pred_idx_list: # if there are still remaining pred_idx, concatenate all preds
+        match_list.append({
+            'gt_idx': [""],
+            'gt': "",
+            'pred_idx': pred_idx_list,
+            'pred': ''.join(pred_lines[_] for _ in pred_idx_list),
+            'gt_position': [""],
+            'pred_position': pred_items[pred_idx_list[0]]['position'][0],  # get the first pred's position
+            'norm_gt': "",
+            'norm_pred': ''.join(norm_pred_lines[_] for _ in pred_idx_list),
+            'gt_category_type': "",
+            'pred_category_type': get_pred_category_type(pred_idx_list[0], pred_items), # get the first pred's category
+            'gt_attribute': [{}],
+            'edit': 1,
+            'img_id': img_name
+        })
+    return match_list
+
+
+def match_gt2pred_no_split(gt_items, pred_items, line_type, img_name):
+    # directly concatenate gt and pred by position
+    gt_lines, norm_gt_lines, gt_cat_list, pred_lines, norm_pred_lines = get_gt_pred_lines(gt_items, pred_items, line_type)
+    gt_line_with_position = []
+    for gt_line, norm_gt_line, gt_item in zip(gt_lines, norm_gt_lines, gt_items):
+        gt_position = gt_item['order'] if gt_item.get('order') else gt_item.get('position', [""])[0]
+        if gt_position:
+            gt_line_with_position.append((gt_position, gt_line, norm_gt_line))
+    sorted_gt_lines = sorted(gt_line_with_position, key=lambda x: x[0])
+    gt = '\n\n'.join([_[1] for _ in sorted_gt_lines])
+    norm_gt = '\n\n'.join([_[2] for _ in sorted_gt_lines])
+    pred_line_with_position = [(pred_item['position'], pred_line, pred_norm_line) for pred_line, pred_norm_line, pred_item in zip(pred_lines, norm_pred_lines, pred_items)]
+    sorted_pred_lines = sorted(pred_line_with_position, key=lambda x: x[0])
+    pred = '\n\n'.join([_[1] for _ in sorted_pred_lines])
+    norm_pred = '\n\n'.join([_[2] for _ in sorted_pred_lines])
+    # edit = Levenshtein.distance(norm_gt, norm_pred)/max(len(norm_gt), len(norm_pred))
+    if norm_gt or norm_pred:
+        return [{
+                'gt_idx': [0],
+                'gt': gt,
+                'norm_gt': norm_gt,
+                'gt_category_type': "text_merge",
+                'gt_position': [""],
+                'gt_attribute': [{}],
+                'pred_idx': [0],
+                'pred': pred,
+                'norm_pred': norm_pred,
+                'pred_category_type': "text_merge",
+                'pred_position': "",
+                # 'edit': edit,
+                'img_id': img_name
+            }]
+    else:
+        return []
+
+
+import copy
+import pdb
+from collections import Counter, defaultdict
+
+import evaluate
+# from rapidfuzz.distance import Levenshtein
+import Levenshtein
+import numpy as np
+from Levenshtein import distance as Levenshtein_distance
+from scipy.optimize import linear_sum_assignment
+
+
+def match_gt2pred_quick(gt_items, pred_items, line_type, img_name):
+
+    gt_lines, norm_gt_lines, gt_cat_list, pred_lines, norm_pred_lines= get_gt_pred_lines(gt_items, pred_items, line_type)
+    all_gt_indices = set(range(len(norm_gt_lines)))
+    all_pred_indices = set(range(len(norm_pred_lines)))
+
+    if not norm_gt_lines:
+        match_list = []
+        for pred_idx in range(len(norm_pred_lines)):
+            match_list.append({
+                'gt_idx': [""],
+                'gt': "",
+                'pred_idx': [pred_idx],
+                'pred': pred_lines[pred_idx],
+                'gt_position': "",
+                'pred_position': pred_items[pred_idx]['position'][0],
+                'norm_gt': "",
+                'norm_pred': norm_pred_lines[pred_idx],
+                'gt_category_type': "",
+                'pred_category_type': get_pred_category_type(pred_idx, pred_items),
+                'gt_attribute': [{}],
+                'edit': 1,
+                'img_id': img_name
+            })
+        return match_list
+    elif not norm_pred_lines:
+        match_list = []
+        for gt_idx in range(len(norm_gt_lines)):
+            match_list.append({
+                'gt_idx': [gt_idx],
+                'gt': gt_lines[gt_idx],
+                'pred_idx': [""],
+                'pred': "",
+                'gt_position': [gt_items[gt_idx].get('order') if gt_items[gt_idx].get('order') else gt_items[gt_idx].get('position', [""])[0]],
+                'pred_position': "",
+                'norm_gt': norm_gt_lines[gt_idx],
+                'norm_pred': "",
+                'gt_category_type': gt_cat_list[gt_idx],
+                'pred_category_type': "",
+                'gt_attribute': [gt_items[gt_idx].get("attribute", {})],
+                'edit': 1,
+                'img_id': img_name
+            })
+        return match_list
+    elif len(norm_gt_lines) == 1 and len(norm_pred_lines) == 1:
+        edit_distance = Levenshtein_distance(norm_gt_lines[0], norm_pred_lines[0])
+        normalized_edit_distance = edit_distance / max(len(norm_gt_lines[0]), len(norm_pred_lines[0]))
+        return [{
+            'gt_idx': [0],
+            'gt': gt_lines[0],
+            'pred_idx': [0],
+            'pred': pred_lines[0],
+            'gt_position': [gt_items[0].get('order') if gt_items[0].get('order') else gt_items[0].get('position', [""])[0]],
+            'pred_position': pred_items[0]['position'][0],
+            'norm_gt': norm_gt_lines[0],
+            'norm_pred': norm_pred_lines[0],
+            'gt_category_type': gt_cat_list[0],
+            'pred_category_type': get_pred_category_type(0, pred_items),
+            'gt_attribute': [gt_items[0].get("attribute", {})],
+            'edit': normalized_edit_distance,
+            'img_id': img_name
+        }]
+
+    cost_matrix = compute_edit_distance_matrix_new(norm_gt_lines, norm_pred_lines)
+
+    matched_col_idx, row_ind, cost_list = cal_final_match(cost_matrix, norm_gt_lines, norm_pred_lines)
+
+    gt_lens_dict, pred_lens_dict = initialize_indices(norm_gt_lines, norm_pred_lines)
+
+    matches, unmatched_gt_indices, unmatched_pred_indices = process_matches(matched_col_idx, row_ind, cost_list, norm_gt_lines, norm_pred_lines, pred_lines)
+
+    matching_dict = fuzzy_match_unmatched_items(unmatched_gt_indices, norm_gt_lines, norm_pred_lines)
+
+    final_matches = merge_matches(matches, matching_dict)
+
+    recalculate_edit_distances(final_matches, gt_lens_dict, norm_gt_lines, norm_pred_lines)
+
+    converted_results = convert_final_matches(final_matches, norm_gt_lines, norm_pred_lines)
+
+    merged_results = merge_duplicates_add_unmatched(converted_results, norm_gt_lines, norm_pred_lines, gt_lines, pred_lines, all_gt_indices, all_pred_indices)
+
+    for entry in merged_results:
+            entry['gt_idx'] = [entry['gt_idx']] if not isinstance(entry['gt_idx'], list) else entry['gt_idx']
+            entry['pred_idx'] = [entry['pred_idx']] if not isinstance(entry['pred_idx'], list) else entry['pred_idx']
+            entry['gt_position'] = [gt_items[_].get('order') if gt_items[_].get('order') else gt_items[_].get('position', [""])[0] for _ in entry['gt_idx']] if entry['gt_idx'] != [""] else [""]
+            entry['pred_position'] = pred_items[entry['pred_idx'][0]]['position'][0] if entry['pred_idx'] != [""] else ""
+            entry['gt'] = ''.join([gt_lines[_] for _ in entry['gt_idx']]) if entry['gt_idx'] != [""] else ""
+            entry['pred'] = ''.join([pred_lines[_] for _ in entry['pred_idx']]) if entry['pred_idx'] != [""] else ""
+            entry['norm_gt'] = ''.join([norm_gt_lines[_] for _ in entry['gt_idx']]) if entry['gt_idx'] != [""] else ""
+            entry['norm_pred'] = ''.join([norm_pred_lines[_] for _ in entry['pred_idx']]) if entry['pred_idx'] != [""] else ""
+
+            if entry['gt_idx'] != [""]:
+                ignore_type = ['figure_caption', 'figure_footnote', 'table_caption', 'table_footnote', 'code_algorithm', 'code_algorithm_caption', 'header', 'footer', 'page_footnote', 'page_number', 'equation_caption']
+                gt_cagegory_clean = [gt_cat_list[_] for _ in entry['gt_idx'] if gt_cat_list[_] not in ignore_type]
+                if gt_cagegory_clean:
+                    entry['gt_category_type'] = Counter(gt_cagegory_clean).most_common(1)[0][0]
+                else:
+                    entry['gt_category_type'] = Counter([gt_cat_list[_] for _ in entry['gt_idx']]).most_common(1)[0][0]
+            else:
+                entry['gt_category_type'] = ""
+            entry['pred_category_type'] = get_pred_category_type(entry['pred_idx'][0], pred_items) if entry['pred_idx'] != [""] else ""
+            entry['gt_attribute'] = [gt_items[_].get("attribute", {}) for _ in entry['gt_idx']] if entry['gt_idx'] != [""] else [{}]
+            entry['img_id'] = img_name
+
+    return merged_results
+
+
+def merge_duplicates_add_unmatched(converted_results, norm_gt_lines, norm_pred_lines, gt_lines, pred_lines, all_gt_indices, all_pred_indices):
+    merged_results = []
+    processed_pred = set()
+    processed_gt = set()
+
+    for entry in converted_results:
+        pred_idx = tuple(entry['pred_idx']) if isinstance(entry['pred_idx'], list) else (entry['pred_idx'],)
+        if pred_idx not in processed_pred and pred_idx != ("",):
+            merged_entry = {
+                'gt_idx': [entry['gt_idx']],
+                'gt': entry['gt'],
+                'pred_idx': entry['pred_idx'],
+                'pred': entry['pred'],
+                'edit': entry['edit']
+            }
+            for other_entry in converted_results:
+                other_pred_idx = tuple(other_entry['pred_idx']) if isinstance(other_entry['pred_idx'], list) else (other_entry['pred_idx'],)
+                if other_pred_idx == pred_idx and other_entry is not entry:
+                    merged_entry['gt_idx'].append(other_entry['gt_idx'])
+                    merged_entry['gt'] += other_entry['gt']
+                    processed_gt.add(other_entry['gt_idx'])
+            merged_results.append(merged_entry)
+            processed_pred.add(pred_idx)
+            processed_gt.add(entry['gt_idx'])
+
+    for entry in converted_results:
+        if entry['gt_idx'] not in processed_gt:
+            merged_results.append(entry)
+
+    for gt_idx in range(len(norm_gt_lines)):
+        if gt_idx not in processed_gt:
+            merged_results.append({
+                'gt_idx': [gt_idx],
+                'gt': gt_lines[gt_idx],
+                'pred_idx': [""],
+                'pred': "",
+                'edit': 1
+            })
+    return merged_results
+
+
+
+
+def formula_format(formula_matches, img_name):
+    return [
+        {
+            "gt": item["gt"],
+            "pred": item["pred"],
+            "img_id": f"{img_name}_{i}"
+        }
+        for i, item in enumerate(formula_matches)
+    ]
+
+
+def merge_lists_with_sublists(main_list, sub_lists):
+    main_list_final = list(copy.deepcopy(main_list))
+    for sub_list in sub_lists:
+        pop_idx = main_list_final.index(sub_list[0])
+        for _ in sub_list:
+            main_list_final.pop(pop_idx)
+        main_list_final.insert(pop_idx, sub_list)
+    return main_list_final
+
+
+def sub_pred_fuzzy_matching(gt, pred):
+
+    min_d = float('inf')
+    # pos = -1
+
+    gt_len = len(gt)
+    pred_len = len(pred)
+
+    if gt_len >= pred_len and pred_len > 0:
+        for i in range(gt_len - pred_len + 1):
+            sub = gt[i:i + pred_len]
+            dist = Levenshtein_distance(sub, pred)/pred_len
+            if dist < min_d:
+                min_d = dist
+                pos = i
+
+        return min_d
+    else:
+        return False
+
+def sub_gt_fuzzy_matching(pred, gt):
+
+    min_d = float('inf')
+    pos = ""
+    matched_sub = ""
+    gt_len = len(gt)
+    pred_len = len(pred)
+
+    if pred_len >= gt_len and gt_len > 0:
+        for i in range(pred_len - gt_len + 1):
+            sub = pred[i:i + gt_len]
+            dist = Levenshtein.distance(sub, gt)  /gt_len
+            if dist < min_d:
+                min_d = dist
+                pos = i
+                matched_sub = sub
+        return min_d, pos, gt_len, matched_sub
+    else:
+        return 1, "", gt_len, ""
+
+
+def get_final_subset(subset_certain, subset_certain_cost):
+    if not subset_certain or not subset_certain_cost:
+        return []
+
+    subset_turple = sorted([(a, b) for a, b in zip(subset_certain, subset_certain_cost)], key=lambda x: x[0][0])
+
+    group_list = defaultdict(list)
+    group_idx = 0
+    group_list[group_idx].append(subset_turple[0])
+
+    for item in subset_turple[1:]:
+        overlap_flag = False
+        for subset in group_list[group_idx]:
+            for idx in item[0]:
+                if idx in subset[0]:
+                    overlap_flag = True
+                    break
+            if overlap_flag:
+                break
+        if overlap_flag:
+            group_list[group_idx].append(item)
+        else:
+            group_idx += 1
+            group_list[group_idx].append(item)
+
+    final_subset = []
+    for _, group in group_list.items():
+        if len(group) == 1:
+            final_subset.append(group[0][0])
+        else:
+            path_dict = defaultdict(list)
+            path_idx = 0
+            path_dict[path_idx].append(group[0])
+
+            for subset in group[1:]:
+                new_path = True
+                for path_idx_s, path_items in path_dict.items():
+                    is_dup = False
+                    is_same = False
+                    for path_item in path_items:
+                        if path_item[0] == subset[0]:
+                            is_dup = True
+                            is_same = True
+                            if path_item[1] > subset[1]:
+                                path_dict[path_idx_s].pop(path_dict[path_idx_s].index(path_item))
+                                path_dict[path_idx_s].append(subset)
+                        else:
+                            for num_1 in path_item[0]:
+                                for num_2 in subset[0]:
+                                    if num_1 == num_2:
+                                        is_dup = True
+                    if not is_dup:
+                        path_dict[path_idx_s].append(subset)
+                        new_path = False
+                    if is_same:
+                        new_path = False
+                if new_path:
+                    path_idx = len(path_dict.keys())
+                    path_dict[path_idx].append(subset)
+
+            saved_cost = float('inf')
+            saved_subset = []
+            for path_idx, path in path_dict.items():
+                avg_cost = sum([i[1] for i in path]) / len(path)
+                if avg_cost < saved_cost:
+                    saved_subset = [i[0] for i in path]
+                    saved_cost = avg_cost
+
+            final_subset.extend(saved_subset)
+
+    return final_subset
+
+def judge_pred_merge(gt_list, pred_list, threshold=0.6):
+    if len(pred_list) == 1:
+        return False, False
+
+    cur_pred = ' '.join(pred_list[:-1])
+    merged_pred = ' '.join(pred_list)
+
+    cur_dist = Levenshtein.distance(gt_list[0], cur_pred) / max(len(gt_list[0]), len(cur_pred))
+    merged_dist = Levenshtein.distance(gt_list[0], merged_pred) / max(len(gt_list[0]), len(merged_pred))
+
+    if merged_dist > cur_dist:
+        return False, False
+
+    cur_fuzzy_dists = [sub_pred_fuzzy_matching(gt_list[0], cur_pred) for cur_pred in pred_list[:-1]]
+    if any(dist is False or dist > threshold for dist in cur_fuzzy_dists):
+        return False, False
+
+    add_fuzzy_dist = sub_pred_fuzzy_matching(gt_list[0], pred_list[-1])
+    if add_fuzzy_dist is False:
+        return False, False
+
+    merged_pred_flag = add_fuzzy_dist < threshold
+    continue_flag = len(merged_pred) <= len(gt_list[0])
+
+    return merged_pred_flag, continue_flag
+
+def deal_with_truncated(cost_matrix, norm_gt_lines, norm_pred_lines):
+    matched_first = np.argwhere(cost_matrix < 0.25)
+    masked_gt_idx = [i[0] for i in matched_first]
+    unmasked_gt_idx = [i for i in range(cost_matrix.shape[0]) if i not in masked_gt_idx]
+    masked_pred_idx = [i[1] for i in matched_first]
+    unmasked_pred_idx = [i for i in range(cost_matrix.shape[1]) if i not in masked_pred_idx]
+
+    merges_gt_dict = {}
+    merges_pred_dict = {}
+    merged_gt_subsets = []
+
+    for gt_idx in unmasked_gt_idx:
+        check_merge_subset = []
+        merged_dist = []
+
+        for pred_idx in unmasked_pred_idx:
+            step = 1
+            merged_pred = [norm_pred_lines[pred_idx]]
+
+            while True:
+                if pred_idx + step in masked_pred_idx or pred_idx + step >= len(norm_pred_lines):
+                    break
+                else:
+                    merged_pred.append(norm_pred_lines[pred_idx + step])
+                    merged_pred_flag, continue_flag = judge_pred_merge([norm_gt_lines[gt_idx]], merged_pred)
+                    if not merged_pred_flag:
+                        break
+                    else:
+                        step += 1
+                    if not continue_flag:
+                        break
+
+            check_merge_subset.append(list(range(pred_idx, pred_idx + step)))
+            matched_line = ' '.join([norm_pred_lines[i] for i in range(pred_idx, pred_idx + step)])
+            dist = Levenshtein_distance(norm_gt_lines[gt_idx], matched_line) / max(len(matched_line), len(norm_gt_lines[gt_idx]))
+            merged_dist.append(dist)
+
+        if not merged_dist:
+            subset_certain = []
+            min_cost_idx = ""
+            min_cost = float('inf')
+        else:
+            min_cost = min(merged_dist)
+            min_cost_idx = merged_dist.index(min_cost)
+            subset_certain = check_merge_subset[min_cost_idx]
+
+        merges_gt_dict[gt_idx] = {
+            'merge_subset': check_merge_subset,
+            'merged_cost': merged_dist,
+            'min_cost_idx': min_cost_idx,
+            'subset_certain': subset_certain,
+            'min_cost': min_cost
+        }
+
+    subset_certain = [merges_gt_dict[gt_idx]['subset_certain'] for gt_idx in unmasked_gt_idx if merges_gt_dict[gt_idx]['subset_certain']]
+    subset_certain_cost = [merges_gt_dict[gt_idx]['min_cost'] for gt_idx in unmasked_gt_idx if merges_gt_dict[gt_idx]['subset_certain']]
+
+    subset_certain_final = get_final_subset(subset_certain, subset_certain_cost)
+
+    if not subset_certain_final:
+        return cost_matrix, norm_pred_lines, range(len(norm_pred_lines))
+
+    final_pred_idx_list = merge_lists_with_sublists(range(len(norm_pred_lines)), subset_certain_final)
+    final_norm_pred_lines = [' '.join(norm_pred_lines[idx_list[0]:idx_list[-1]+1]) if isinstance(idx_list, list) else norm_pred_lines[idx_list] for idx_list in final_pred_idx_list]
+
+    new_cost_matrix = compute_edit_distance_matrix_new(norm_gt_lines, final_norm_pred_lines)
+
+    return new_cost_matrix, final_norm_pred_lines, final_pred_idx_list
+
+def cal_move_dist(gt, pred):
+    assert len(gt) == len(pred), 'Not right length'
+    step = 0
+    for i, gt_c in enumerate(gt):
+        if gt_c != pred[i]:
+            step += abs(i - pred.index(gt_c))
+            pred[i], pred[pred.index(gt_c)] = pred[pred.index(gt_c)], pred[i]
+    return step / len(gt)
+
+def cal_final_match(cost_matrix, norm_gt_lines, norm_pred_lines):
+    min_indice = cost_matrix.argmax(axis=1)
+
+    new_cost_matrix, final_norm_pred_lines, final_pred_idx_list = deal_with_truncated(cost_matrix, norm_gt_lines, norm_pred_lines)
+
+    row_ind, col_ind = linear_sum_assignment(new_cost_matrix)
+
+    cost_list = [new_cost_matrix[r][c] for r, c in zip(row_ind, col_ind)]
+    matched_col_idx = [final_pred_idx_list[i] for i in col_ind]
+
+    return matched_col_idx, row_ind, cost_list
+
+def initialize_indices(norm_gt_lines, norm_pred_lines):
+    gt_lens_dict = {idx: len(gt_line) for idx, gt_line in enumerate(norm_gt_lines)}
+    pred_lens_dict = {idx: len(pred_line) for idx, pred_line in enumerate(norm_pred_lines)}
+    return gt_lens_dict, pred_lens_dict
+
+def process_matches(matched_col_idx, row_ind, cost_list, norm_gt_lines, norm_pred_lines, pred_lines):
+    matches = {}
+    unmatched_gt_indices = []
+    unmatched_pred_indices = []
+
+    for i in range(len(norm_gt_lines)):
+        if i in row_ind:
+            idx = list(row_ind).index(i)
+            pred_idx = matched_col_idx[idx]
+
+            if pred_idx is None or (isinstance(pred_idx, list) and None in pred_idx):
+                unmatched_pred_indices.append(pred_idx)
+                continue
+
+            if isinstance(pred_idx, list):
+                pred_line = ' | '.join(norm_pred_lines[pred_idx[0]:pred_idx[-1]+1])
+                ori_pred_line = ' | '.join(pred_lines[pred_idx[0]:pred_idx[-1]+1])
+                matched_pred_indices_range = list(range(pred_idx[0], pred_idx[-1]+1))
+            else:
+                pred_line = norm_pred_lines[pred_idx]
+                ori_pred_line = pred_lines[pred_idx]
+                matched_pred_indices_range = [pred_idx]
+
+            edit = cost_list[idx]
+
+            if edit > 0.7:
+                unmatched_pred_indices.extend(matched_pred_indices_range)
+                unmatched_gt_indices.append(i)
+            else:
+                matches[i] = {
+                    'pred_indices': matched_pred_indices_range,
+                    'edit_distance': edit,
+                }
+                for matched_pred_idx in matched_pred_indices_range:
+                    if matched_pred_idx in unmatched_pred_indices:
+                        unmatched_pred_indices.remove(matched_pred_idx)
+        else:
+            unmatched_gt_indices.append(i)
+
+    return matches, unmatched_gt_indices, unmatched_pred_indices
+
+def fuzzy_match_unmatched_items(unmatched_gt_indices, norm_gt_lines, norm_pred_lines):
+    matching_dict = {}
+
+    for pred_idx, pred_content in enumerate(norm_pred_lines):
+        if isinstance(pred_idx, list):
+            continue
+
+        matching_indices = []
+
+        for unmatched_gt_idx in unmatched_gt_indices:
+            gt_content = norm_gt_lines[unmatched_gt_idx]
+            cur_fuzzy_dist_unmatch, cur_pos, gt_lens, matched_field = sub_gt_fuzzy_matching(pred_content, gt_content)
+            if cur_fuzzy_dist_unmatch < 0.4:
+                matching_indices.append(unmatched_gt_idx)
+
+        if matching_indices:
+            matching_dict[pred_idx] = matching_indices
+
+    return matching_dict
+
+def merge_matches(matches, matching_dict):
+    final_matches = {}
+    processed_gt_indices = set()
+
+    for gt_idx, match_info in matches.items():
+        pred_indices = match_info['pred_indices']
+        edit_distance = match_info['edit_distance']
+
+        pred_key = tuple(sorted(pred_indices))
+
+        if pred_key in final_matches:
+            if gt_idx not in processed_gt_indices:
+                final_matches[pred_key]['gt_indices'].append(gt_idx)
+                processed_gt_indices.add(gt_idx)
+        else:
+            final_matches[pred_key] = {
+                'gt_indices': [gt_idx],
+                'edit_distance': edit_distance
+            }
+            processed_gt_indices.add(gt_idx)
+
+    for pred_idx, gt_indices in matching_dict.items():
+        pred_key = (pred_idx,) if not isinstance(pred_idx, (list, tuple)) else tuple(sorted(pred_idx))
+
+        if pred_key in final_matches:
+            for gt_idx in gt_indices:
+                if gt_idx not in processed_gt_indices:
+                    final_matches[pred_key]['gt_indices'].append(gt_idx)
+                    processed_gt_indices.add(gt_idx)
+        else:
+            final_matches[pred_key] = {
+                'gt_indices': [gt_idx for gt_idx in gt_indices if gt_idx not in processed_gt_indices],
+                'edit_distance': None
+            }
+            processed_gt_indices.update(final_matches[pred_key]['gt_indices'])
+
+    return final_matches
+
+
+
+def recalculate_edit_distances(final_matches, gt_lens_dict, norm_gt_lines, norm_pred_lines):
+    for pred_key, info in final_matches.items():
+        gt_indices = sorted(set(info['gt_indices']))
+
+        if not gt_indices:
+            info['edit_distance'] = 1
+            continue
+
+        if len(gt_indices) > 1:
+            merged_gt_content = ''.join(norm_gt_lines[gt_idx] for gt_idx in gt_indices)
+            pred_content = norm_pred_lines[pred_key[0]] if isinstance(pred_key[0], int) else ''
+
+            try:
+                edit_distance = Levenshtein_distance(merged_gt_content, pred_content)
+                normalized_edit_distance = edit_distance / max(len(merged_gt_content), len(pred_content))
+            except ZeroDivisionError:
+                normalized_edit_distance = 1
+
+            info['edit_distance'] = normalized_edit_distance
+        else:
+            gt_idx = gt_indices[0]
+            pred_content = ' '.join(norm_pred_lines[pred_idx] for pred_idx in pred_key if isinstance(pred_idx, int))
+
+            try:
+                edit_distance = Levenshtein_distance(norm_gt_lines[gt_idx], pred_content)
+                normalized_edit_distance = edit_distance / max(len(norm_gt_lines[gt_idx]), len(pred_content))
+            except ZeroDivisionError:
+                normalized_edit_distance = 1
+
+            info['edit_distance'] = normalized_edit_distance
+            info['pred_content'] = pred_content
+
+
+def convert_final_matches(final_matches, norm_gt_lines, norm_pred_lines):
+    converted_results = []
+
+    all_gt_indices = set(range(len(norm_gt_lines)))
+    all_pred_indices = set(range(len(norm_pred_lines)))
+
+    for pred_key, info in final_matches.items():
+        pred_content = ' '.join(norm_pred_lines[pred_idx] for pred_idx in pred_key if isinstance(pred_idx, int))
+
+        for gt_idx in sorted(set(info['gt_indices'])):
+            result_entry = {
+                'gt_idx': int(gt_idx),
+                'gt': norm_gt_lines[gt_idx],
+                'pred_idx': list(pred_key),
+                'pred': pred_content,
+                'edit': info['edit_distance']
+            }
+            converted_results.append(result_entry)
+
+    matched_gt_indices = set().union(*[set(info['gt_indices']) for info in final_matches.values()])
+    unmatched_gt_indices = all_gt_indices - matched_gt_indices
+    matched_pred_indices = set(idx for pred_key in final_matches.keys() for idx in pred_key if isinstance(idx, int))
+    unmatched_pred_indices = all_pred_indices - matched_pred_indices
+
+    if unmatched_pred_indices:
+        if unmatched_gt_indices:
+            distance_matrix = [
+                [Levenshtein_distance(norm_gt_lines[gt_idx], norm_pred_lines[pred_idx]) for pred_idx in unmatched_pred_indices]
+                for gt_idx in unmatched_gt_indices
+            ]
+
+            row_ind, col_ind = linear_sum_assignment(distance_matrix)
+
+            for i, j in zip(row_ind, col_ind):
+                gt_idx = list(unmatched_gt_indices)[i]
+                pred_idx = list(unmatched_pred_indices)[j]
+                result_entry = {
+                    'gt_idx': int(gt_idx),
+                    'gt': norm_gt_lines[gt_idx],
+                    'pred_idx': [pred_idx],
+                    'pred': norm_pred_lines[pred_idx],
+                    'edit': 1
+                }
+                converted_results.append(result_entry)
+
+            matched_gt_indices.update(list(unmatched_gt_indices)[i] for i in row_ind)
+        else:
+            result_entry = {
+                'gt_idx': "",
+                'gt': '',
+                'pred_idx': list(unmatched_pred_indices),
+                'pred': ' '.join(norm_pred_lines[pred_idx] for pred_idx in unmatched_pred_indices),
+                'edit': 1
+            }
+            converted_results.append(result_entry)
+    else:
+        for gt_idx in unmatched_gt_indices:
+            result_entry = {
+                'gt_idx': int(gt_idx),
+                'gt': norm_gt_lines[gt_idx],
+                'pred_idx': "",
+                'pred': '',
+                'edit': 1
+            }
+            converted_results.append(result_entry)
+
+    return converted_results
+
+import json
+
+
+def read_md_file(filepath):
+    with open(filepath, 'r', encoding='utf-8') as file:
+        content = file.read()
+
+    return content
+
+def save_paired_result(preds, gts, save_path):
+    save_result = []
+    formula_id = 0
+    for gt, pred in zip(gts, preds):
+        save_result.append({
+            "gt": gt,
+            "pred": pred,
+            "img_id": formula_id
+        })
+        formula_id += 1
+    with open(save_path, 'w', encoding='utf-8') as f:
+        json.dump(save_result, f, indent=4, ensure_ascii=False)
+
+
+import os
+import re
+
+import matplotlib.font_manager as fm
+import matplotlib.pyplot as plt
+import numpy as np
+
+font = fm.FontProperties(fname=r'font/SimHei.ttf')
+
+
+def print_aligned_dict(data):
+    # Find the maximum length of all keys
+    max_key_length = max(len(key) for key in data['testcase1'])
+
+    # Print header
+    print(f"{' ' * (max_key_length + 4)}", end="")
+    for key in data:
+        print(f"{key:>{max_key_length}}", end="")
+    print()
+
+    # Print dictionary content
+    for subkey in data['testcase1']:
+        print(f"{subkey:<{max_key_length + 4}}", end="")
+        for key in data:
+            print(f"{data[key][subkey]:>{max_key_length}}", end="")
+        print()
+def create_dict_from_folders(directory):
+    body = {}
+    for folder_name in os.listdir(directory):
+        folder_path = os.path.join(directory, folder_name)
+        if os.path.isdir(folder_path):
+            body[folder_name] = {}
+    return body
+
+
+def create_radar_chart(df, title, filename):
+    labels = df.columns
+
+    # Calculate angles
+    angles = np.linspace(0, 2 * np.pi, len(labels), endpoint=False).tolist()
+    angles += angles[:1]
+
+    # Initialize radar chart
+    fig, ax = plt.subplots(figsize=(10, 6), subplot_kw=dict(polar=True), dpi=200)
+    # ax.spines['polar'].set_visible(False)
+
+    # Draw radar chart for each dataset
+    for index, row in df.iterrows():
+        values = row.tolist()
+        values += values[:1]
+        ax.fill(angles, values, alpha=0.1)
+        ax.plot(angles, values, label=index)
+
+        # Add percentage labels next to each data point
+        for angle, value in zip(angles, values):
+            ax.text(angle, value, '{:.1%}'.format(value), ha='center', va='center', fontsize=7, alpha=0.7)
+
+    # Set labels
+    ax.set_yticklabels([])
+    ax.set_xticks(angles[:-1])
+    ax.set_xticklabels(labels, fontproperties=font)
+    ax.spines['polar'].set_visible(False)  # Hide the outermost circle
+    ax.grid(False)
+    for j in np.arange(0, 1.2, 0.2):
+        ax.plot(angles, len(values) * [j], '-.', lw=0.5, color='black', alpha=0.5)
+    for j in range(len(values)):
+        ax.plot([angles[j], angles[j]], [0, 1], '-.', lw=0.5, color='black', alpha=0.5)
+
+    # Add title and legend
+    plt.legend(loc='upper right', bbox_to_anchor=(0.1, 0.1))
+
+    ax.tick_params(pad=30)
+    ax.set_theta_zero_location('N')
+    # Save chart to file
+    plt.savefig(filename)
+
+# The function is from https://github.com/intsig-textin/markdown_tester
+def markdown_to_html(markdown_table):
+    rows = [row.strip() for row in markdown_table.strip().split('\n')]
+    num_columns = len(rows[0].split('|')) - 2
+
+    html_table = '<table>\n  <thead>\n    <tr>\n'
+
+    header_cells = [cell.strip() for cell in rows[0].split('|')[1:-1]]
+    for cell in header_cells:
+        html_table += f'      <th>{cell}</th>\n'
+    html_table += '    </tr>\n  </thead>\n  <tbody>\n'
+
+    for row in rows[2:]:
+        cells = [cell.strip() for cell in row.split('|')[1:-1]]
+        html_table += '    <tr>\n'
+        for cell in cells:
+            html_table += f'      <td>{cell}</td>\n'
+        html_table += '    </tr>\n'
+
+    html_table += '  </tbody>\n</table>\n'
+    return html_table
+def convert_markdown_to_html(self, markdown_content, md_type):
+    # Define a regex pattern to find Markdown tables with newlines
+    markdown_content = markdown_content.replace('\r', '')
+    pattern = re.compile(r'\|\s*.*?\s*\|\n', re.DOTALL)
+
+    # Find all matches in the Markdown content
+    matches = pattern.findall(markdown_content)
+    for match in matches:
+        html_table = markdown_to_html(match)
+        markdown_content = markdown_content.replace(match, html_table, 1)  # Only replace the first occurrence
+    res_html = convert_table(replace_table_with_placeholder(markdown_content))
+
+    return res_html
+def convert_table_str(s):
+    s = re.sub(r'<table.*?>','<table>',s)
+    s = re.sub(r'<th','<td',s)
+    s = re.sub(r'</th>','</td>',s)
+    # s = re.sub(r'<td rowspan="(.)">',lambda x:f'<td colspan="1" rowspan="{x.group(1)}">',s)
+    # s = re.sub(r'<td colspan="(.)">',lambda x:f'<td colspan="{x.group(1)}" rowspan="1">',s)
+    res = ''
+    res += '\n\n'
+    temp_item = ''
+    for c in s:
+        temp_item += c
+        if c == '>' and not re.search(r'<td.*?>\$',temp_item):
+            res += temp_item+'\n'
+            temp_item = ''
+    return res+'\n'
+def merge_table(md):
+    table_temp = ''
+    for line in md:
+        table_temp += line
+    return convert_table_str(table_temp)
+def find_md_table_mode(line):
+    if re.search(r'-*?:',line) or re.search(r'---',line) or re.search(r':-*?',line):
+        return True
+    return False
+def delete_table_and_body(input_list):
+    res = []
+    for line in input_list:
+        if not re.search(r'</?t(able|head|body)>',line):
+            res.append(line)
+    return res
+def merge_tables(input_str):
+    # Delete HTML comments
+    input_str = re.sub(r'<!--[\s\S]*?-->', '', input_str)
+
+    # Use regex to find each <table> block
+    table_blocks = re.findall(r'<table>[\s\S]*?</table>', input_str)
+
+    # Process each <table> block, replace <th> with <td>
+    output_lines = []
+    for block in table_blocks:
+        block_lines = block.split('\n')
+        for i, line in enumerate(block_lines):
+            if '<th>' in line:
+                block_lines[i] = line.replace('<th>', '<td>').replace('</th>', '</td>')
+        final_tr = delete_table_and_body(block_lines)
+        if len(final_tr) > 2:
+            output_lines.extend(final_tr)  # Ignore <table> and </table> tags, keep only table content
+
+    # Rejoin the processed strings
+    merged_output = '<table>\n{}\n</table>'.format('\n'.join(output_lines))
+
+    return "\n\n" + merged_output + "\n\n"
+
+def replace_table_with_placeholder(input_string):
+    lines = input_string.split('\n')
+    output_lines = []
+
+    in_table_block = False
+    temp_block = ""
+    last_line = ""
+
+    org_table_list = []
+    in_org_table = False
+
+    for idx, line in enumerate(lines):
+        # if not in_org_table:
+        # if "<table>" not in last_line and in_table_block == False and temp_block != "":
+        #     output_lines.append(merge_tables(temp_block))
+        #     temp_block = ""
+        if "<table>" in line:
+            # if "<table><tr" in line:
+            #     org_table_list.append(line)
+            #     in_org_table = True
+            #     output_lines.append(last_line)
+            #     continue
+            # else:
+            in_table_block = True
+            temp_block += last_line
+        elif in_table_block:
+            if not find_md_table_mode(last_line) and "</thead>" not in last_line:
+                temp_block += "\n" + last_line
+            if "</table>" in last_line:
+                if "<table>" not in line:
+                    in_table_block = False
+                    output_lines.append(merge_tables(temp_block))
+                    temp_block = ""
+        else:
+            output_lines.append(last_line)
+
+        last_line = line
+        # else:
+        #     org_table_list.append(line)
+        #     if "</table" in line:
+        #         in_org_table = False
+        #         last_line = merge_table(org_table_list)
+        #         org_table_list = []
+
+    if last_line:
+        if in_table_block or "</table>" in last_line:
+            temp_block += "\n" + last_line
+            output_lines.append(merge_tables(temp_block))
+        else:
+            output_lines.append(last_line)
+    # if "</table>" in last_line:
+    #     output_lines.append(merge_tables(temp_block))
+
+    return '\n'.join(output_lines)
+
+def convert_table(input_str):
+    # Replace <table>
+    output_str = input_str.replace("<table>", "<table border=\"1\" >")
+
+    # Replace <td>
+    output_str = output_str.replace("<td>", "<td colspan=\"1\" rowspan=\"1\">")
+
+    return output_str
+
+def convert_markdown_to_html(markdown_content):
+    # Define a regex pattern to find Markdown tables with newlines
+    markdown_content = markdown_content.replace('\r', '')+'\n'
+    pattern = re.compile(r'\|\s*.*?\s*\|\n', re.DOTALL)
+
+    # Find all matches in the Markdown content
+    matches = pattern.findall(markdown_content)
+
+    for match in matches:
+        html_table = markdown_to_html(match)
+        markdown_content = markdown_content.replace(match, html_table, 1)  # Only replace the first occurrence
+
+    res_html = convert_table(replace_table_with_placeholder(markdown_content))
+
+    return res_html
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/deep_research.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/deep_research.py
new file mode 100644
index 0000000000000000000000000000000000000000..76e0ca695dec7fcfbf1ed8b25fb7581aaaa32f06
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/deep_research.py
@@ -0,0 +1,183 @@
+from typing import Any, Dict, List
+
+import pandas as pd
+from datasets import load_dataset
+from json_repair import repair_json
+
+from vlmeval.smp import dump, get_intermediate_file_path, load
+from vlmeval.utils.mp_util import track_progress_rich
+from ..text_base import TextBaseDataset
+from ..utils.judge_util import build_judge
+
+
+def extract_final_answer(answer_with_thinking: str, start_tag='<answer>', end_tag='</answer>'):
+    answer_with_thinking = str(answer_with_thinking)
+    start_index = answer_with_thinking.rfind(start_tag)
+    if start_index != -1:
+        end_index = answer_with_thinking.find(end_tag, start_index)
+        if end_index != -1:
+            return answer_with_thinking[start_index + len(start_tag):end_index].strip()
+    return None
+
+
+def eval_model_output(ques_dict, judge):
+    newline = '\n'
+    prompt = f"""
+You are an expert in systematically validating and evaluating \
+LLM-generated solutions. Your task is to rigorously analyze the \
+correctness of a provided solution by comparing it step-by-step \
+against the reference solution, and output **only** a structured \
+verification list—with no additional text.
+
+## Instructions
+1. Break down the given LLM solution into individual steps and \
+evaluate each one against the corresponding reference solution steps.
+2. For each step, include the following three components:
+   - **solution_step**: The specific part of the LLM solution \
+being evaluated.
+   - **reason**: A clear, critical explanation of whether the \
+step contains errors, omissions, or deviations from the reference \
+approach. Be stringent in your assessment.
+   - **judge**: Your verdict: either `"correct"` or `"incorrect"`.
+3. If the final LLM answer is incorrect, you must identify \
+at least one step in your analysis as incorrect.
+4. Justify your judgments rigorously, pointing out even minor \
+inaccuracies or logical flaws.
+5. Do not attempt to answer the original question—your role \
+is strictly to evaluate.
+6. Output **only** a list of dictionaries in the exact format \
+provided below. Do not include any other text or comments.
+
+## Question
+{ques_dict['question']}
+
+## Reference Solution Steps
+{newline.join(ques_dict['steps'])}
+
+## Reference Answer
+{ques_dict['answer']}
+
+## LLM Solution Steps
+{ques_dict['prediction']}
+
+## LLM Answer
+{extract_final_answer(ques_dict['prediction'])}
+
+## Output Example
+[
+    {{"solution_step": "step content", \
+"reason": "reason of the judgement", \
+"judge": "correct or incorrect"}},
+    {{"solution_step": "step content", \
+"reason": "reason of the judgement", \
+"judge": "correct or incorrect"}},
+]
+"""
+
+    try:
+        messages = [
+            {"role": "system", "value": "You are a helpful assistant.", "type": "text"},
+            {"role": "user", "value": prompt, "type": "text"},
+        ]
+        llm_judge = judge.generate(messages)
+        start_index = llm_judge.find('[')
+        end_index = llm_judge.rfind(']') + 1
+        llm_judge = eval(repair_json(llm_judge[start_index:end_index]))
+        correct_step_count = 0
+        for step in llm_judge:
+            if step["judge"] == "correct":
+                correct_step_count += 1
+        step_level_acc = correct_step_count / len(llm_judge)
+    except Exception as e:
+        print(e)
+        llm_judge = None
+        step_level_acc = 0
+
+    ques_dict['exact_match'] = 1 if (
+        ques_dict['answer'] == ques_dict['prediction']
+        or ques_dict['answer'] == extract_final_answer(
+            ques_dict['prediction']
+        )
+    ) else 0
+    ques_dict['llm_judge'] = llm_judge
+    ques_dict['step_level_acc'] = step_level_acc
+    return ques_dict
+
+
+class SGI_Bench_Deep_Research(TextBaseDataset):
+    TYPE = 'QA'
+
+    @classmethod
+    def supported_datasets(cls):
+        return ["SGI-DeepResearch"]
+
+    def load_data(self, dataset):
+        hf = load_dataset("InternScience/SGI-DeepResearch", split="test")
+
+        rows: List[Dict[str, Any]] = []
+        idx = 0
+        for prob in hf:
+            rows.append(
+                {
+                    "index": idx,
+                    "id": prob["idx"],
+                    "question": prob["question"],
+                    "steps": prob["steps"],
+                    "answer": prob["answer"],
+                    "discipline": prob["discipline"],
+                    "direction": prob["direction"],
+                    "type": prob["type"]
+                }
+            )
+            idx += 1
+        return pd.DataFrame(rows)
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        question = line['question'] + """
+You can reason step by step before giving the final answer. \
+The final answer should be enclosed by <answer> and </answer>.
+
+Example:
+Step 1. ...
+Step 2. ...
+...
+<answer>1.00</answer>
+"""
+
+        msgs = [{'type': 'text', 'value': question}]
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        data = load(eval_file)
+        data = pd.DataFrame(data)
+
+        if judge_kwargs.get('model') is None:
+            judge_kwargs['model'] = 'o4-mini'
+        if judge_kwargs.get('max_tokens') is None:
+            judge_kwargs['max_tokens'] = None
+
+        inp_list = []
+        judge = build_judge(**judge_kwargs)
+        for item in data.to_dict(orient="records"):
+            inp_list.append({"ques_dict": item, "judge": judge})
+        out_list = track_progress_rich(
+            func=eval_model_output,
+            tasks=inp_list,
+            nproc=judge_kwargs.get('nproc', 48)
+        )
+
+        exact_match = sum([item['exact_match'] for item in out_list]) / len(out_list)
+        step_level_acc = sum([item['step_level_acc'] for item in out_list]) / len(out_list)
+
+        result = {
+            'Exact Match': exact_match,
+            'Step Level Acc': step_level_acc
+        }
+
+        score_file = get_intermediate_file_path(eval_file, '_score', 'json')
+        result_file = get_intermediate_file_path(eval_file, '_result', 'json')
+        dump(out_list, score_file)
+        dump(result, result_file)
+        return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment.py
new file mode 100644
index 0000000000000000000000000000000000000000..44e00f6f88fa8b375e244920433c8e740c80ac69
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment.py
@@ -0,0 +1,599 @@
+import ast
+import os
+import platform
+import shutil
+import subprocess
+import time
+from pathlib import Path
+from typing import Any, Dict, List
+
+import pandas as pd
+import requests
+from datasets import load_dataset
+from json_repair import repair_json
+from requests.adapters import HTTPAdapter
+from urllib3.util.retry import Retry
+
+from vlmeval.smp import LMUDataRoot, dump, get_intermediate_file_path, load
+from vlmeval.utils.mp_util import track_progress_rich
+from ..text_base import TextBaseDataset
+from ..utils.judge_util import build_judge
+
+save_dir = "./outputs/sgi_code_logs"
+tmp_data_dir = "./outputs/sgi_tmp_data"
+
+env = os.environ.copy()
+env["PYTHONIOENCODING"] = "utf-8"
+
+
+def run_script_in_folder(folder_path):
+    """
+    Run data.py (if exists) and main.py in the given folder,
+    print immediate status, and return execution results.
+    """
+    script_name = 'data_en.py'
+    script_path_full = folder_path / script_name
+    try:
+        result = subprocess.run(
+            ["conda", "run", "-n", "dryexp", "python", script_name],
+            capture_output=True,
+            text=True,
+            timeout=10 * 60,  # 10-minute timeout
+            encoding="utf-8",
+            cwd=str(folder_path),
+            env=env,
+            shell=platform.system() == "Windows"
+        )
+        if result.returncode == 0:
+            # print(f"✅")
+            result = (str(script_path_full), True, "")
+        else:
+            print("❌")
+            error_message = (
+                result.stderr.strip()
+                if result.stderr else "Unknown error"
+            )
+            result = (str(script_path_full), False, error_message)
+    except subprocess.TimeoutExpired:
+        print("❌")
+        print("      Error: Execution timed out after 10 minutes")
+        result = (
+            str(script_path_full), False,
+            "Execution timed out after 10 minutes"
+        )
+    except Exception as e:
+        print("❌")
+        print(f"      Error: {e}")
+        result = (str(script_path_full), False, str(e))
+    return result
+
+
+def run_script(ques_dict):
+    ques_dict['unit_test'] = []
+    for unit_test_idx in range(5):
+        folder_path = os.path.join(
+            save_dir, ques_dict['idx'],
+            f"unit_test_{unit_test_idx}"
+        )
+        unit_test_dict = {}
+
+        try:
+            # Run the script and capture output
+            start_time = time.time()
+            result = subprocess.run(
+                ["conda", "run", "-n", "dryexp", "python", 'main_model.py'],
+                capture_output=True,
+                text=True,
+                timeout=300,  # 5 minutes timeout
+                encoding="utf-8",
+                cwd=str(folder_path),
+                env=env,
+                shell=platform.system() == "Windows"
+            )
+            end_time = time.time()
+            elapsed = end_time - start_time
+            model_code_output = f"{result.stderr}\n{result.stdout}".strip()
+
+            if result.returncode == 0:
+                # print(f"✅")
+                unit_test_dict["model_error"] = "[No Error]"
+                unit_test_dict["model_runtime"] = elapsed
+                unit_test_dict["model_returncode"] = result.returncode
+                unit_test_dict["model_code_output"] = model_code_output
+            else:
+                # print(f"❌")
+                # print(f"      Error: {error_message}")
+                unit_test_dict["model_error"] = (
+                    "[WRONG]" + result.stderr.strip()
+                    if result.stderr else "Unknown error"
+                )
+                unit_test_dict["model_runtime"] = elapsed
+                unit_test_dict["model_returncode"] = result.returncode
+                unit_test_dict["model_code_output"] = model_code_output
+        except subprocess.TimeoutExpired:
+            # print(f"❌")
+            # print(f"      Error: Execution timed out after 5 minutes")
+            unit_test_dict["model_error"] = (
+                "[WRONG]Execution timed out after 5 minutes"
+            )
+            unit_test_dict["model_runtime"] = 300.0
+            unit_test_dict["model_returncode"] = -1  # Terminated
+            unit_test_dict["model_code_output"] = unit_test_dict["model_error"]
+        except Exception as e:
+            # print(f"❌")
+            # print(f"      Error: {e}")
+            unit_test_dict["model_error"] = "[WRONG]" + str(e)
+            unit_test_dict["model_runtime"] = -1
+            unit_test_dict["model_returncode"] = 1  # Error
+            unit_test_dict["model_code_output"] = unit_test_dict["model_error"]
+        ques_dict['unit_test'].append(unit_test_dict)
+    return ques_dict
+
+
+def eval_model_output(ques_dict, judge):
+    for unit_test_idx in range(5):
+        unit_test_dict = ques_dict['unit_test'][unit_test_idx]
+        correct_output = ques_dict[f"unit_test_{unit_test_idx}_output"]
+        unit_test_dict['exact_match'] = (
+            1 if (unit_test_dict['model_code_output']
+                  == correct_output) else 0
+        )
+
+        if unit_test_dict["exact_match"]:
+            unit_test_dict["llm_judge"] = {
+                "judgment": "correct",
+                "reason": "Exact match."
+            }
+            unit_test_dict['pass'] = 1
+            ques_dict['unit_test'][unit_test_idx] = unit_test_dict
+            continue
+
+        if (unit_test_dict["model_error"].startswith("[WRONG]")
+                or unit_test_dict["model_returncode"] != 0):
+            unit_test_dict["llm_judge"] = {
+                "judgment": "incorrect",
+                "reason": "There are problems running "
+                          "the completed code."
+            }
+            unit_test_dict['pass'] = 0
+            ques_dict['unit_test'][unit_test_idx] = unit_test_dict
+            continue
+
+        prompt = f"""
+You are an expert in evaluating model output accuracy. Your task \
+is to precisely determine whether the model output matches the \
+reference output and provide a brief explanation.
+
+## Instructions
+1. Check all numerical values and ensure strict accuracy—every \
+digit must match exactly. Any inconsistency should be considered \
+incorrect.
+2. For training-related loss values or metrics, if the difference \
+between model output and reference output loss or metric values \
+is greater than 2%, consider it incorrect.
+3. The output should be a dictionary without any other text in \
+the following format:
+example = {{
+    "judgment": "Placeholder, use 'correct' if outputs match, \
+'incorrect' otherwise",
+    "reason": "Brief explanation placeholder"
+}}
+
+## Reference Output
+{correct_output}
+
+## Model Output
+{unit_test_dict["model_code_output"]}
+"""
+
+        try:
+            messages = [
+                {"role": "system",
+                 "value": "You are a helpful assistant.",
+                 "type": "text"},
+                {"role": "user", "value": prompt, "type": "text"},
+            ]
+            llm_judge = judge.generate(messages)
+            start_index = llm_judge.find('{')
+            end_index = llm_judge.rfind('}') + 1
+            llm_judge = eval(repair_json(llm_judge[start_index:end_index]))
+        except Exception as e:
+            print(e)
+            llm_judge = None
+
+        unit_test_dict['llm_judge'] = llm_judge
+        if llm_judge and isinstance(llm_judge, dict):
+            unit_test_dict['pass'] = (
+                1 if llm_judge.get('judgment') == 'correct'
+                else 0
+            )
+        else:
+            unit_test_dict['pass'] = 0
+        ques_dict['unit_test'][unit_test_idx] = unit_test_dict
+
+    ques_dict['pass_nums'] = sum(
+        [ut['pass'] for ut in ques_dict['unit_test']]
+    )
+    ques_dict['model_average_runtime'] = [
+        ut['model_runtime']
+        for ut in ques_dict['unit_test']
+        if ut['model_runtime'] > 0
+    ]
+    avg_rt = ques_dict['model_average_runtime']
+    ques_dict['model_average_runtime'] = (
+        sum(avg_rt) / len(avg_rt) if len(avg_rt) > 0
+        else -1
+    )
+    ques_dict['se'] = sum(
+        [1 if ut['model_returncode'] == 0 else 0
+         for ut in ques_dict['unit_test']]
+    ) / 5
+    return ques_dict
+
+
+def download_file(url: str, dir_path: str):
+    os.makedirs(dir_path, exist_ok=True)
+    filename = url.split("/")[-1]
+    save_path = os.path.join(dir_path, filename)
+
+    if os.path.exists(save_path):
+        return save_path
+    session = requests.Session()
+    retries = Retry(
+        total=3, backoff_factor=1,
+        status_forcelist=[500, 502, 503, 504]
+    )
+    session.mount('http://', HTTPAdapter(max_retries=retries))
+    session.mount('https://', HTTPAdapter(max_retries=retries))
+
+    try:
+        with session.get(url, stream=True, timeout=30) as response:
+            response.raise_for_status()
+            with open(save_path, "wb") as f:
+                for chunk in response.iter_content(chunk_size=8192):
+                    if chunk:
+                        f.write(chunk)
+
+        return save_path
+
+    except Exception as e:
+        if os.path.exists(save_path):
+            os.remove(save_path)
+        print(f"Error downloading {url}: {e}")
+        raise e
+
+
+def extract_final_answer(
+    answer_with_thinking: str,
+    start_tag='<answer>',
+    end_tag='</answer>'
+):
+    answer_with_thinking = str(answer_with_thinking)
+    start_index = answer_with_thinking.rfind(start_tag)
+    if start_index != -1:
+        end_index = answer_with_thinking.find(end_tag, start_index)
+        if end_index != -1:
+            return answer_with_thinking[
+                start_index + len(start_tag):end_index
+            ].strip()
+    return None
+
+
+def check_syntax(code_string):
+    try:
+        # Try to compile the code string
+        compile(code_string, '<string>', 'exec')
+        return True
+    except SyntaxError:
+        return False
+
+
+def get_function_lines(file_content):
+    node = ast.parse(file_content)
+
+    function_lines = {}
+
+    for item in node.body:
+        if isinstance(item, ast.FunctionDef):
+            func_name = item.name
+            start_line = item.lineno
+            end_line = item.end_lineno
+            function_lines[func_name] = (start_line, end_line)
+
+    return function_lines
+
+
+def replace_code(
+    content_1, start_line_1, end_line_1,
+    content_2, start_line_2, end_line_2
+):
+    lines_1 = content_1.splitlines(keepends=True)
+    lines_2 = content_2.splitlines(keepends=True)
+
+    lines_1[start_line_1 - 1:end_line_1] = lines_2[start_line_2 - 1:end_line_2]
+
+    return ''.join(lines_1)
+
+
+def replace_function(main_code, new_code, function_name):
+    assert check_syntax(main_code), "wrong main_code"
+    assert check_syntax(new_code), "wrong new_code"
+    functions_dict_1 = get_function_lines(main_code)
+    functions_dict_2 = get_function_lines(new_code)
+
+    start_line_1, end_line_1 = functions_dict_1[function_name]
+    start_line_2, end_line_2 = functions_dict_2[function_name]
+
+    main_code_after_replacing = replace_code(
+        main_code, start_line_1, end_line_1,
+        new_code, start_line_2, end_line_2
+    )
+    assert check_syntax(main_code_after_replacing), \
+        "wrong main_code after replacing"
+    return main_code_after_replacing
+
+
+class SGI_Bench_Dry_Experiment(TextBaseDataset):
+    TYPE = 'QA'
+
+    @classmethod
+    def supported_datasets(cls):
+        return ["SGI-DryExperiment"]
+
+    def load_data(self, dataset):
+        hf = load_dataset("InternScience/SGI-DryExperiment", split="test")
+
+        rows: List[Dict[str, Any]] = []
+        idx = 0
+        for prob in hf:
+            rows.append(
+                {
+                    "index": idx,
+                    "idx": prob["idx"],
+                    "question": prob["question"],
+                    "data_code": prob["data_code"],
+                    "main_code": prob["main_code"],
+                    "incomplete_main_code": prob["incomplete_main_code"],
+                    "incomplete_functions": prob["incomplete_functions"],
+                    "unit_test_0_data": prob["unit_test_0_data"],
+                    "unit_test_0_output": prob["unit_test_0_output"],
+                    "unit_test_1_data": prob["unit_test_1_data"],
+                    "unit_test_1_output": prob["unit_test_1_output"],
+                    "unit_test_2_data": prob["unit_test_2_data"],
+                    "unit_test_2_output": prob["unit_test_2_output"],
+                    "unit_test_3_data": prob["unit_test_3_data"],
+                    "unit_test_3_output": prob["unit_test_3_output"],
+                    "unit_test_4_data": prob["unit_test_4_data"],
+                    "unit_test_4_output": prob["unit_test_4_output"],
+                    "function_type": prob["function_type"],
+                    "runtime": prob["runtime"],
+                    "discipline": prob["discipline"],
+                    "direction": prob["direction"],
+                }
+            )
+            idx += 1
+        return pd.DataFrame(rows)
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        question = line['question'] + """
+Output the completed function enclosed within <answer> and </answer> tags.
+
+Example 1:
+<answer>
+def hello():
+    print("Hello")
+</answer>
+
+Example 2:
+<answer>
+def add(a, b):
+    return a+b
+
+def minus(a, b):
+    return a-b
+</answer>
+
+"""
+
+        msgs = [{'type': 'text', 'value': question}]
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        save_dir_last = 'sgi_code_logs'
+        global save_dir
+        work_dir = str(Path(eval_file).parents[0])
+        save_dir = os.path.join(work_dir, save_dir_last)
+        tmp_data_dir_last = 'sgi_tmp_data'
+        global tmp_data_dir
+        tmp_data_dir = os.path.join(LMUDataRoot(), tmp_data_dir_last)
+        data = load(eval_file)
+        data = pd.DataFrame(data)
+
+        # 输入数据准备
+        data_flag = os.path.join(
+            save_dir, 'data_construction.json'
+        )
+        if not os.path.exists(data_flag):
+            os.makedirs(os.path.join(save_dir), exist_ok=True)
+            os.makedirs(os.path.join(tmp_data_dir), exist_ok=True)
+            os.makedirs(os.path.join(tmp_data_dir, "0206"), exist_ok=True)
+            os.makedirs(os.path.join(tmp_data_dir, "0200"), exist_ok=True)
+            os.makedirs(os.path.join(tmp_data_dir, "0236"), exist_ok=True)
+
+            _base = "https://raw.githubusercontent.com/InternScience/SGI-Bench/main/evaluation/task_3_dry_experiment/data"  # noqa: E501
+            download_file(
+                f"{_base}/SGI_DryExperiment_0206/t10k-images-idx3-ubyte.gz",
+                tmp_data_dir + "/0206")
+            download_file(
+                f"{_base}/SGI_DryExperiment_0206/t10k-labels-idx1-ubyte.gz",
+                tmp_data_dir + "/0206")
+            download_file(
+                f"{_base}/SGI_DryExperiment_0206/train-images-idx3-ubyte.gz",
+                tmp_data_dir + "/0206")
+            download_file(
+                f"{_base}/SGI_DryExperiment_0206/train-labels-idx1-ubyte.gz",
+                tmp_data_dir + "/0206")
+
+            download_file(
+                f"{_base}/SGI_DryExperiment_0200/adult.data",
+                tmp_data_dir + "/0200")
+            download_file(
+                f"{_base}/SGI_DryExperiment_0200/adult.test",
+                tmp_data_dir + "/0200")
+
+            download_file(
+                f"{_base}/SGI_DryExperiment_0236/3d-user-study-data.zip",
+                tmp_data_dir + "/0236")
+
+            code_dir_list = []
+            for index, item in data.iterrows():
+                for unit_test_idx in range(5):
+                    code_dir = os.path.join(
+                        save_dir, item['idx'],
+                        f"unit_test_{unit_test_idx}"
+                    )
+                    code_dir_list.append(
+                        {'folder_path': Path(code_dir)}
+                    )
+                    os.makedirs(code_dir, exist_ok=True)
+                    data_dir = os.path.join(
+                        save_dir, item['idx'],
+                        f"unit_test_{unit_test_idx}", 'data'
+                    )
+                    os.makedirs(data_dir, exist_ok=True)
+
+                    data_py = os.path.join(
+                        code_dir, "data_en.py"
+                    )
+                    with open(data_py, "w", encoding="utf-8") as f:
+                        f.write(
+                            item[f"unit_test_{unit_test_idx}_data"]
+                        )
+                    main_py = os.path.join(
+                        code_dir, "main_en.py"
+                    )
+                    with open(main_py, "w", encoding="utf-8") as f:
+                        f.write(item["main_code"])
+
+            for i in range(5):
+                dst = os.path.join(
+                    save_dir,
+                    f"SGI_DryExperiment_0206/unit_test_{i}/data/mnist_raw"
+                )
+                shutil.copytree(
+                    tmp_data_dir + "/0206", dst,
+                    dirs_exist_ok=True
+                )
+
+            for i in range(5):
+                dst = os.path.join(
+                    save_dir,
+                    f"SGI_DryExperiment_0200/unit_test_{i}/data"
+                )
+                shutil.copytree(
+                    tmp_data_dir + "/0200", dst,
+                    dirs_exist_ok=True
+                )
+
+            for i in range(5):
+                dst = os.path.join(
+                    save_dir,
+                    f"SGI_DryExperiment_0236/unit_test_{i}"
+                    f"/data/em_3d_user_study"
+                )
+                shutil.copytree(
+                    tmp_data_dir + "/0236", dst,
+                    dirs_exist_ok=True
+                )
+
+            all_results = track_progress_rich(
+                tasks=code_dir_list,
+                func=run_script_in_folder,
+                nproc=judge_kwargs.get("nproc", 4)
+            )
+            dump(all_results, os.path.join(save_dir, 'data_construction.json'))
+        # 输入数据准备
+
+        # 代码保存
+        for index, item in data.iterrows():
+            main_code = item['incomplete_main_code']
+            incomplete_functions = item['incomplete_functions']
+            answer = extract_final_answer(item['prediction'])
+            for incomplete_function in eval(incomplete_functions):
+                try:
+                    main_code = replace_function(
+                        main_code, answer,
+                        incomplete_function
+                    )
+                except Exception:
+                    pass
+            for unit_test_idx in range(5):
+                save_path = os.path.join(
+                    save_dir, item['idx'],
+                    f"unit_test_{unit_test_idx}",
+                    "main_model.py"
+                )
+                with open(save_path, 'w', encoding='utf-8') as f:
+                    f.write(main_code)
+        # 代码保存
+
+        # 代码运行
+        inp_list = [
+            {"ques_dict": item}
+            for item in data.to_dict(orient="records")
+        ]
+        out_list = track_progress_rich(
+            tasks=inp_list, func=run_script, nproc=100
+        )
+        # 代码运行
+        if judge_kwargs.get('model') is None:
+            judge_kwargs['model'] = 'o4-mini'
+        if judge_kwargs.get('max_tokens') is None:
+            judge_kwargs['max_tokens'] = None
+        # 代码评测
+        judge = build_judge(**judge_kwargs)
+        in_list = [
+            {"ques_dict": item, "judge": judge}
+            for item in out_list
+        ]
+        out_list = track_progress_rich(
+            tasks=in_list,
+            func=eval_model_output, nproc=100
+        )
+        # 代码评测
+
+        PassAll_5 = sum(
+            [1 if (item['pass_nums'] == 5) else 0
+             for item in out_list]
+        ) / len(out_list)
+        PassAll_3 = sum(
+            [1 if (item['pass_nums'] >= 3) else 0
+             for item in out_list]
+        ) / len(out_list)
+        PassAll_1 = sum(
+            [1 if (item['pass_nums'] >= 1) else 0
+             for item in out_list]
+        ) / len(out_list)
+        runtimes = [
+            item['model_average_runtime']
+            for item in out_list
+            if item['model_average_runtime'] > 0
+        ]
+        AET = sum(runtimes) / len(runtimes)
+        SER = sum([item['se'] for item in out_list]) / len(out_list)
+
+        result = {
+            'PassAll@5': PassAll_5,
+            'PassAll@3': PassAll_3,
+            'PassAll@1': PassAll_1,
+            'AET': AET,
+            'SER': SER
+        }
+
+        score_file = get_intermediate_file_path(eval_file, '_score', 'json')
+        result_file = get_intermediate_file_path(eval_file, '_result', 'json')
+        dump(out_list, score_file)
+        dump(result, result_file)
+        return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment_requirements.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment_requirements.txt
new file mode 100644
index 0000000000000000000000000000000000000000..823dcaba239dbcfbd48340c3ed647c898ff795e6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment_requirements.txt
@@ -0,0 +1,67 @@
+absl-py==2.3.1
+aiohappyeyeballs==2.6.1
+aiohttp==3.12.15
+aiosignal==1.4.0
+async-timeout==5.0.1
+attrs==25.3.0
+certifi==2025.8.3
+cftime==1.6.4.post1
+charset-normalizer==3.4.3
+colorama==0.4.6
+contourpy==1.3.2
+cycler==0.12.1
+datasets==2.15.0
+dill==0.3.7
+filelock==3.19.1
+fire==0.7.1
+fonttools==4.59.2
+frozenlist==1.7.0
+fsspec==2023.10.0
+h5py==3.10.0
+huggingface-hub==0.34.4
+idna==3.10
+imageio==2.37.0
+joblib==1.5.2
+kiwisolver==1.4.9
+lazy_loader==0.4
+matplotlib==3.7.2
+ml_dtypes==0.5.3
+multidict==6.6.4
+multiprocess==0.70.15
+netCDF4==1.6.4
+networkx==3.4.2
+numpy==1.24.3
+opt_einsum==3.4.0
+packaging==25.0
+pandas==2.0.3
+pathlib==1.0.1
+patsy==1.0.1
+Pillow==10.1.0
+propcache==0.3.2
+pyarrow==21.0.0
+pyarrow-hotfix==0.7
+pyparsing==3.0.9
+python-dateutil==2.9.0.post0
+pytz==2025.2
+PyWavelets==1.4.1
+PyYAML==6.0.2
+rdkit==2023.9.5
+requests==2.31.0
+scikit-image==0.22.0
+scikit-learn==1.3.2
+scipy==1.11.4
+seaborn==0.12.2
+six==1.17.0
+statsmodels==0.14.0
+termcolor==3.1.0
+threadpoolctl==3.6.0
+tifffile==2025.5.10
+toolz==1.0.0
+tqdm==4.66.2
+typing_extensions==4.15.0
+tzdata==2025.2
+urllib3==2.5.0
+xarray==2023.6.0
+xgboost==1.7.6
+xxhash==3.5.0
+yarl==1.20.1
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/experimental_reasoning.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/experimental_reasoning.py
new file mode 100644
index 0000000000000000000000000000000000000000..14f74fbc5e4fb937e61377624352b194cf7f6e2d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/experimental_reasoning.py
@@ -0,0 +1,306 @@
+import ast
+import base64
+import io
+import os
+import os.path as osp
+import re
+from typing import Any, Dict, List
+
+import pandas as pd
+from datasets import load_dataset
+
+from vlmeval.smp import (decode_base64_to_image_file, dump, encode_image_to_base64,
+                         get_intermediate_file_path, load, read_ok, toliststr)
+from vlmeval.utils.mp_util import track_progress_rich
+from ..image_base import ImageBaseDataset
+from ..utils.judge_util import build_judge
+
+
+def extract_answer_from_response(response):
+    match = re.search(r"\\boxed\{([A-Za-z])\}", response)
+    if match:
+        return match.group(1)
+    else:
+        return None
+
+
+def mm_reasoning_is_correct(pred, gold):
+    try:
+        ans = extract_answer_from_response(pred).strip()
+    except Exception:
+        return False
+    return ans.lower() == gold.lower()
+
+
+def b64_encode_image(img) -> str:
+    buffered = io.BytesIO()
+    img.save(buffered, format="PNG")
+    return base64.b64encode(buffered.getvalue()).decode("utf-8")
+
+
+def judge_aux(judge, row):
+    reference_steps = row["steps"]
+    reference_steps = "\n".join([f"{i + 1}. {step}" for i, step in enumerate(reference_steps)])
+
+    judge_prompt = (
+        f"You are a strict evaluator assessing the "
+        f"**validity of the model prediction's reasoning "
+        f"process**. You must score this reasoning validity "
+        f"on a scale from 0 to 10, where 0 means the "
+        f"reasoning is completely invalid and 10 means the "
+        f"reasoning is fully rigorous.\n"
+        f"# Input\n"
+        f"Question:\n"
+        f"```\n"
+        f"{row['question']}\n"
+        f"```\n"
+        f"Reference Reasoning:\n"
+        f"```\n"
+        f"{reference_steps}\n"
+        f"```\n"
+        f"Model Prediction:\n"
+        f"```\n"
+        f"{row['prediction']}\n"
+        f"```\n"
+        f"# Evaluation Rules\n"
+        f"1. First, identify the **complete reasoning "
+        f"process** from the model prediction (ignore only "
+        f"the final answer if it is not accompanied by "
+        f"reasoning).\n"
+        f"2. Evaluate reasoning validity against two core "
+        f"criteria:\n"
+        f"   - **Logical Coherence**: Check if the reasoning "
+        f"steps are sequential, self-consistent, and free of "
+        f"contradictions (e.g., no conflicting premises or "
+        f"illogical deductions).\n"
+        f"   - **Alignment with Reference Reasoning**: Check "
+        f"if the reasoning direction, key premises, and "
+        f"deduction logic match the reference reasoning "
+        f"(partial alignment counts for partial credit).\n"
+        f"3. Deduct points for:\n"
+        f"   - Irrelevant content (reasoning that does not "
+        f"address the question or key conditions).\n"
+        f"   - Missing key reasoning steps (even if the "
+        f"final answer is correct).\n"
+        f"   - Flawed logic (e.g., circular reasoning, "
+        f"false premises leading to conclusions).\n"
+        f"4. Do not prioritize the correctness of the "
+        f"**final answer**\u2014a correct answer with invalid "
+        f"reasoning still scores low, while an incorrect "
+        f"answer with partially valid reasoning may score "
+        f"higher.\n"
+        f"# Scoring Guide\n"
+        f"- **10**: Reasoning is fully rigorous, logically "
+        f"coherent (no contradictions), and perfectly "
+        f"aligned with the reference reasoning (all key "
+        f"steps and logic match).\n"
+        f"- **7-9**: Reasoning is mostly coherent, with "
+        f"minor logical gaps or partial misalignment with "
+        f"the reference reasoning (no major "
+        f"contradictions).\n"
+        f"- **4-6**: Reasoning has obvious logical flaws "
+        f"(e.g., one missing key step, minor "
+        f"contradictions) or limited alignment with the "
+        f"reference reasoning (only some core logic "
+        f"matches).\n"
+        f"- **1-3**: Reasoning is barely valid, with severe "
+        f"logical flaws (e.g., multiple contradictions) or "
+        f"almost no alignment with the reference reasoning "
+        f"(only tangentially related to the question).\n"
+        f"- **0**: Reasoning is completely invalid, "
+        f"contradictory (self-conflicting logic), or "
+        f"irrelevant (no connection to the question or key "
+        f"conditions).\n"
+        f"# Strict Output format example\n"
+        f"6"
+    )
+    try:
+        msgs = []
+        msgs.append({'role': 'system', 'value': 'You are a helpful assistant.'})
+        msgs.append({'role': 'user', 'type': 'text', 'value': judge_prompt})
+
+        images = ast.literal_eval(row['step_images'])
+        for image in images:
+            msgs.append({'role': 'user', 'value': image, 'type': 'image'})
+        llm_judge = judge.generate(msgs).strip()
+        pattern = r"(\d+)"
+        match = re.search(pattern, llm_judge)
+        rv_score = float(match.group(1)) if match else 0.0
+    except Exception:
+        rv_score = 0.0
+
+    mcc_score = mm_reasoning_is_correct(row['prediction'], chr(ord('A') + int(row['answer'])))
+    return dict(mcc_score=mcc_score, rv_score=rv_score)
+
+
+class SGI_Bench_Experimental_Reasoning(ImageBaseDataset):
+    TYPE = 'MCQ '
+
+    @classmethod
+    def supported_datasets(cls):
+        return ["SGI-Experimental-Reasoning"]
+
+    def dump_images(self, line):
+        step_dir = osp.join(self.img_root, 'step_images')
+        os.makedirs(self.img_root, exist_ok=True)
+        os.makedirs(step_dir, exist_ok=True)
+
+        results = {}
+
+        def _process_field(key_name, path_key_name, save_root):
+            tgt_paths = []
+            if key_name in line:
+                content = line[key_name]
+                if path_key_name in line and isinstance(line[path_key_name], list):
+                    fnames = line[path_key_name]
+                else:
+                    count = len(content) if isinstance(content, list) else 1
+                    fnames = [f"{line['index']}_{i}.png" for i in range(count)]
+                imgs = content if isinstance(content, list) else [content]
+                for img, fname in zip(imgs, fnames):
+                    full_path = osp.join(save_root, fname)
+                    if not read_ok(full_path):
+                        decode_base64_to_image_file(img, full_path)
+                    tgt_paths.append(full_path)
+
+            elif path_key_name in line:
+                paths = toliststr(line[path_key_name])
+                read_ok_flag = [read_ok(x) for x in paths]
+
+                if not all(read_ok_flag):
+                    paths_abs = [osp.join(save_root, x) for x in paths]
+                    read_ok_flag = [read_ok(x) for x in paths_abs]
+                    assert read_ok_flag, f"Field `{key_name}` missing and files not found: {paths}"
+                    tgt_paths = paths_abs
+                else:
+                    tgt_paths = paths
+
+            return tgt_paths
+
+        if 'image' in line or 'image_path' in line:
+            results['image'] = _process_field('image', 'image_path', self.img_root)
+        if 'step_images' in line or 'step_image_path' in line:
+            results['step_images'] = _process_field('step_images', 'step_image_path', step_dir)
+
+        return results
+
+    def load_data(self, dataset):
+        hf = load_dataset("InternScience/SGI-Reasoning", split="test")
+
+        rows: List[Dict[str, Any]] = []
+        idx = 0
+
+        for prob in hf:
+            current_row = {
+                "index": idx,  #
+                "id": prob["idx"],
+                "question": prob["question"],
+                "image": [encode_image_to_base64(img) for img in prob["images"]],
+                "options": prob["options"],
+                "steps": prob["steps"],
+                "step_images": [encode_image_to_base64(img) for img in prob["step_images"]],
+                "answer": prob["answer"],
+                "image_type": prob["image_type"],
+                "discipline": prob["discipline"],
+                "direction": prob["direction"],
+                "type": prob["type"]
+            }
+            saved_paths = self.dump_images(current_row)
+            if 'image' in saved_paths:
+                current_row['image'] = saved_paths['image']
+
+            if 'step_images' in saved_paths:
+                current_row['step_images'] = saved_paths['step_images']
+            rows.append(current_row)
+            idx += 1
+
+        return pd.DataFrame(rows)
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        question = (
+            "Please solve the following multiple-choice "
+            "question step-by-step. Each question is "
+            "provided with several options labeled A, B, "
+            "C, D, E, etc. Carefully analyze the question "
+            "and each option, reason step-by-step, then "
+            "select the single most correct option.\n\n"
+            "Your final output **must** include both "
+            "**the reasoning** and **the final answer**. "
+            "The final answer must meet two core "
+            "requirements:\n"
+            "1. It consists solely of the corresponding "
+            "letter of the correct option (e.g., A, B, C, "
+            "D, E, etc.);\n"
+            "2. This letter is enclosed in the \\boxed{} "
+            "format. Example: \\boxed{A}"
+            "\n\nQuestion:\n" + line['question']
+            + "\n\nOptions:\n"
+        )
+        for i, option in enumerate(line['options']):
+            option_label = chr(ord('A') + i)
+            question += f"{option_label}. {option}\n"
+
+        msgs = []
+        if isinstance(line['image'], list):
+            for p in line['image']:
+                msgs.append({'type': 'image', 'value': p})
+        elif isinstance(line['image'], str):
+            msgs.append({'type': 'image', 'value': line['image']})
+        msgs.append({'type': 'text', 'value': question})
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        data = load(eval_file)
+        data = pd.DataFrame(data)
+
+        data['mcc'] = 0
+        data['rv'] = 0
+
+        all_mcc, all_rv = [], []
+        if judge_kwargs.get('model') is None:
+            judge_kwargs['model'] = 'o4-mini'
+        if judge_kwargs.get('max_tokens') is None:
+            judge_kwargs['max_tokens'] = None
+        judge = build_judge(**judge_kwargs)
+
+        tups = []
+        indices = []
+        tmp_file = get_intermediate_file_path(eval_file, '_judge_tmp', 'pkl')
+        if osp.exists(tmp_file):
+            ans = load(tmp_file)
+        else:
+            ans = {}
+
+        for index, row in data.iterrows():
+            if index in ans:
+                continue
+            tups.append(dict(judge=judge, row=row))
+            indices.append(index)
+
+        if len(indices) > 0:
+            track_progress_rich(
+                judge_aux,
+                tasks=tups,
+                nproc=judge_kwargs.get('nproc', 32),
+                save=tmp_file,
+                keys=indices
+            )
+            ans = load(tmp_file)
+
+        for index, res in ans.items():
+            rv_score = res['rv_score']
+            mcc_score = res['mcc_score']
+            all_mcc.append(mcc_score)
+            data.at[index, 'mcc'] = 1 if mcc_score else 0
+            all_rv.append(rv_score)
+            data['rv'] = data['rv'].astype(float)
+            data.at[index, 'rv'] = rv_score / 10.0
+
+        score_file = get_intermediate_file_path(eval_file, '_score', 'csv')
+        result = {"MCC": sum(all_mcc) / len(all_mcc), "RV": sum(all_rv) / (10.0 * len(all_rv))}
+        result_file = get_intermediate_file_path(eval_file, '_result', 'json')
+        dump(data, score_file)
+        dump(result, result_file)
+        return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py
new file mode 100644
index 0000000000000000000000000000000000000000..6ea6b54a4f4a5cdf59c84bf34898d330cd8d54c7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py
@@ -0,0 +1,896 @@
+import ast
+import json
+import os.path as osp
+import re
+import time
+from datetime import datetime
+from typing import Any, Dict, List
+
+import networkx as nx
+import numpy as np
+import pandas as pd
+from datasets import load_dataset
+
+from vlmeval.smp import dump, get_intermediate_file_path, load
+from vlmeval.smp.log import get_logger
+from vlmeval.utils.mp_util import track_progress_rich
+from ..text_base import TextBaseDataset
+from ..utils.judge_util import build_judge
+from .utils import (flip_evaluation_result, format_idea_data, get_context_from_data,
+                    get_evaluation_prompt_modified, parse_evaluation_result)
+
+embedding_model = None
+logger = get_logger(__name__)
+
+
+def parse_generated_idea(text: str) -> Dict[str, Any]:
+    """Parse the generated research proposal text into a structured dictionary"""
+    json_block_pattern = r"```(?:json)?\s*([\s\S]*?)```"
+    json_block_match = re.search(json_block_pattern, text)
+    if json_block_match:
+        json_str = json_block_match.group(1).strip()
+        try:
+            parsed_data = json.loads(json_str)
+            return parsed_data
+        except json.JSONDecodeError:
+            pass
+    try:
+        parsed_data = json.loads(text)
+        return parsed_data
+    except json.JSONDecodeError:
+        pass
+    result = {}
+    idea_patterns = [
+        r"[\"']?Idea[\"']?\s*:\s*[\"'](.*?)[\"']",
+        r"1\.\s*Idea[:\s-]+(.*?)(?=\n\s*(?:2\.|Implementation))",
+    ]
+    for pattern in idea_patterns:
+        match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
+        if match:
+            result["Idea"] = match.group(1).strip()
+            break
+    steps_patterns = [
+        r"[\"']?ImplementationSteps[\"']?\s*:\s*\{(.*?)\}",
+        r"2\.\s*Implementation Steps[:\s-]+(.*?)(?=\n\s*(?:3\.|Implementation Order))",
+    ]
+    for pattern in steps_patterns:
+        match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
+        if match:
+            steps_text = match.group(1).strip()
+            steps_dict = {}
+            step_matches = re.findall(r"[\"'](\d+)[\"']\s*:\s*[\"'](.*?)[\"']", steps_text)
+            for step_num, step_desc in step_matches:
+                steps_dict[step_num] = step_desc.strip()
+            if steps_dict:
+                result["ImplementationSteps"] = steps_dict
+                break
+    order_patterns = [
+        r"[\"']?ImplementationOrder[\"']?\s*:\s*\[(.*?)\]",
+        r"3\.\s*Implementation Order[:\s-]+(.*?)(?=\n\s*(?:4\.|Dataset))",
+    ]
+    for pattern in order_patterns:
+        match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
+        if match:
+            order_text = match.group(1).strip()
+            order_list = re.findall(r'["\']([^"\']+)["\']', order_text)
+            if order_list:
+                result["ImplementationOrder"] = order_list
+                break
+    dataset_patterns = [
+        r"[\"']?Dataset[\"']?\s*:\s*[\"'](.*?)[\"'](?=\s*,\s*[\"'])",
+        r"4\.\s*Dataset[:\s-]+(.*?)(?=\n\s*(?:5\.|Evaluation))",
+    ]
+    for pattern in dataset_patterns:
+        match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
+        if match:
+            result["Dataset"] = match.group(1).strip()
+            break
+    metrics_patterns = [
+        r"[\"']?EvaluationMetrics[\"']?\s*:\s*\{(.*?)\}(?=\s*,\s*[\"'])",
+        r"5\.\s*Evaluation Metrics[:\s-]+(.*?)(?=\n\s*(?:6\.|Expected))",
+    ]
+    for pattern in metrics_patterns:
+        match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
+        if match:
+            metrics_text = match.group(1).strip()
+            metrics_dict = {}
+            metric_matches = re.findall(r"[\"']([^\"']+)[\"']\s*:\s*[\"'](.*?)[\"']", metrics_text)
+            for metric_name, metric_desc in metric_matches:
+                metrics_dict[metric_name.strip()] = metric_desc.strip()
+            if metrics_dict:
+                result["EvaluationMetrics"] = metrics_dict
+                break
+    outcome_patterns = [
+        r"[\"']?ExpectedOutcome[\"']?\s*:\s*[\"'](.*?)[\"']",
+        r"6\.\s*Expected Outcome[:\s-]+(.*?)$",
+    ]
+    for pattern in outcome_patterns:
+        match = re.search(pattern, text, re.DOTALL | re.IGNORECASE)
+        if match:
+            result["ExpectedOutcome"] = match.group(1).strip()
+            break
+    if not result:
+        result["full_text"] = text
+    return result
+
+
+# 所有原始工具函数
+def cosine_similarity(a: np.ndarray, b: np.ndarray) -> np.ndarray:
+    a_norm = np.linalg.norm(a, axis=1, keepdims=True)
+    b_norm = np.linalg.norm(b, axis=1, keepdims=True)
+    a_norm = np.where(a_norm == 0, 1, a_norm)
+    b_norm = np.where(b_norm == 0, 1, b_norm)
+    a_normalized = a / a_norm
+    b_normalized = b / b_norm
+    return np.dot(a_normalized, b_normalized.T)
+
+
+def edge_jaccard(G1, G2):
+    edges1 = set(G1.edges())
+    edges2 = set(G2.edges())
+    if not edges1 and not edges2:
+        return 1.0
+    return len(edges1 & edges2) / len(edges1 | edges2)
+
+
+def node_text_similarity(G1, G2):
+    texts1 = [G1.nodes[n]['text'] for n in G1.nodes()]
+    texts2 = [G2.nodes[n]['text'] for n in G2.nodes()]
+    if not texts1 or not texts2:
+        logger.warning("node_text_similarity: One of the graphs has no node texts.")
+        return 0.0
+    try:
+        combined_text1 = ' '.join(texts1)
+        combined_text2 = ' '.join(texts2)
+        if len(combined_text1.strip()) < 3 or len(combined_text2.strip()) < 3:
+            logger.warning("node_text_similarity: One of the texts is too short to compare.")
+            return 0.0
+        words1 = set(combined_text1.lower().split())
+        words2 = set(combined_text2.lower().split())
+        if not words1 or not words2:
+            return 0.0
+        intersection = words1.intersection(words2)
+        union = words1.union(words2)
+        jaccard_sim = len(intersection) / len(union) if union else 0.0
+        return jaccard_sim
+    except Exception:
+        return 0.0
+
+
+def graph_similarity(dict1, dict2, alpha=0.5):
+    if not all(k in dict1 for k in ["ImplementationSteps", "ImplementationOrder"]) or \
+            not all(k in dict2 for k in ["ImplementationSteps", "ImplementationOrder"]):
+        logger.warning("graph_similarity: One of the graphs is missing necessary keys.")
+        return 0.0
+    if not dict1["ImplementationSteps"] or not dict1["ImplementationOrder"] or \
+            not dict2["ImplementationSteps"] or not dict2["ImplementationOrder"]:
+        logger.warning("graph_similarity: One of the graphs is missing necessary keys.")
+        return 0.0
+    try:
+        G1 = nx.DiGraph()
+        G2 = nx.DiGraph()
+        for k, v in dict1["ImplementationSteps"].items():
+            G1.add_node(str(k), text=v)
+        for k, v in dict2["ImplementationSteps"].items():
+            G2.add_node(str(k), text=v)
+        if len(G1.nodes()) == 0 or len(G2.nodes()) == 0:
+            return 0.0
+
+        def process_order_items(order_list, graph, step_keys):
+            edges_added = False
+            if all(o.isdigit() for o in order_list):
+                nodes = sorted([o for o in order_list if o in step_keys])
+                for i in range(len(nodes) - 1):
+                    graph.add_edge(nodes[i], nodes[i + 1])
+                    edges_added = True
+            else:
+                for o in order_list:
+                    if "-" in o:
+                        try:
+                            src, dst = o.split("-")
+                            if src in step_keys and dst in step_keys:
+                                graph.add_edge(src, dst)
+                                edges_added = True
+                        except Exception:
+                            pass
+            return edges_added
+
+        step_keys1 = [str(k) for k in dict1["ImplementationSteps"].keys()]
+        step_keys2 = [str(k) for k in dict2["ImplementationSteps"].keys()]
+        edges_added_G1 = process_order_items(dict1["ImplementationOrder"], G1, step_keys1)
+        edges_added_G2 = process_order_items(dict2["ImplementationOrder"], G2, step_keys2)
+        if not edges_added_G1:
+            nodes1 = sorted([n for n in G1.nodes()])
+            for i in range(len(nodes1) - 1):
+                G1.add_edge(nodes1[i], nodes1[i + 1])
+                edges_added_G1 = True
+        if not edges_added_G2:
+            nodes2 = sorted([n for n in G2.nodes()])
+            for i in range(len(nodes2) - 1):
+                G2.add_edge(nodes2[i], nodes2[i + 1])
+                edges_added_G2 = True
+        if not edges_added_G1 or not edges_added_G2:
+            logger.warning(
+                "graph_similarity: One of the graphs has no edges, only node text similarity will be computed.")
+            return node_text_similarity(G1, G2)
+        edge_sim = edge_jaccard(G1, G2)
+        text_sim = node_text_similarity(G1, G2)
+        return alpha * edge_sim + (1 - alpha) * text_sim
+    except Exception:
+        return 0.0
+
+
+def calculate_semantic_repetition(text: str) -> float:
+    sentences = [s.strip() for s in re.split(r'[.!?。！？]', text) if len(s.strip()) > 10]
+    if len(sentences) < 2:
+        return 0.0
+    try:
+        if embedding_model is None:
+            logger.warning("embedding_model is not available, cannot compute semantic repetition")
+            return 0.0
+        sentence_embeddings = embedding_model.encode(sentences)
+        similarity_matrix = cosine_similarity(sentence_embeddings, sentence_embeddings)
+        upper_triangle = []
+        for i in range(len(sentences)):
+            for j in range(i + 1, len(sentences)):
+                upper_triangle.append(similarity_matrix[i][j])
+        if not upper_triangle:
+            return 0.0
+        avg_similarity = np.mean(upper_triangle)
+        penalty = max(0, (avg_similarity - 0.2) * 10)
+        return min(penalty, 10.0)
+    except Exception as e:
+        logger.error(f"calculate_semantic_repetition error: {e}")
+        return 0.0
+
+
+def get_vote_from_model(model, original_idea_data, generated_idea_data, context=None, swap_positions=False):
+    original_idea_text = format_idea_data(original_idea_data)
+    generated_idea_text = format_idea_data(generated_idea_data)
+
+    # determine positions for evaluation
+    if swap_positions:
+        # swap positions: generated idea as A, original idea as B
+        prompt = get_evaluation_prompt_modified(generated_idea_text, original_idea_text, context)
+        positions_swapped = True
+    else:
+        # default positions: original idea as A, generated idea as B
+        prompt = get_evaluation_prompt_modified(original_idea_text, generated_idea_text, context)
+        positions_swapped = False
+
+    MAX_RETRIES = 5
+    retry_count = 0
+    while retry_count < MAX_RETRIES:
+        try:
+            response = model.generate(message=dict(type='text', value=prompt), temperature=0.1)
+            if response is None:
+                retry_count += 1
+                logger.warning(f"model {model.model} API call failed, retry {retry_count}")
+                time.sleep(1)
+                continue
+            evaluation_result = parse_evaluation_result(response)
+            if evaluation_result is None:
+                retry_count += 1
+                logger.warning(f"model {model.model} evaluation result parse error, retry {retry_count}")
+                time.sleep(1)
+                continue
+            if positions_swapped:
+                evaluation_result = flip_evaluation_result(evaluation_result)
+            return evaluation_result
+        except Exception as e:
+            retry_count += 1
+            logger.error(f"model {model.model} evaluation error: {e}, try {retry_count}")
+            time.sleep(1)
+
+    logger.warning(f"model {model.model} evaluation failed after {MAX_RETRIES} retries")
+    return None
+
+
+def compare_ideas_with_voting(original_idea_data, generated_idea_data, context=None, judge_models=None):
+    dimensions = ["effectiveness", "novelty", "detailedness", "feasibility", "overall"]
+    vote_counts = {
+        dim: {"original": 0, "generated": 0} for dim in dimensions
+    }
+    all_evaluations = []
+
+    for model in judge_models:
+        for swap in [False, True]:  # each model votes twice, once with normal positions, once with swapped positions
+            evaluation = get_vote_from_model(
+                model=model,
+                original_idea_data=original_idea_data,
+                generated_idea_data=generated_idea_data,
+                context=context,
+                swap_positions=swap
+            )
+            if evaluation:
+                vote_detail = {
+                    "model": model,
+                    "positions_swapped": swap,
+                    "results": {}
+                }
+                for dim in dimensions:
+                    dim_result = evaluation.get(dim, {})
+                    judgment = dim_result.get("judgment", "")
+                    reason = dim_result.get("reason", "No reason provided")
+                    if judgment == "win_A":
+                        vote_counts[dim]["original"] += 1
+                        result = "original_wins"
+                    elif judgment == "win_B":
+                        vote_counts[dim]["generated"] += 1
+                        result = "generated_wins"
+                    else:
+                        logger.warning(f"error: {judgment}")
+                        continue
+                    vote_detail["results"][dim] = {
+                        "result": result,
+                        "reason": reason
+                    }
+                all_evaluations.append(vote_detail)
+            else:
+                logger.error(f"model {model} evaluation failed, could not get votes")
+
+    final_results = {}
+    for dim in dimensions:
+        original_votes = vote_counts[dim]["original"]
+        generated_votes = vote_counts[dim]["generated"]
+        lose_gate = 2
+        if dim == "novelty":
+            win_gate = 4
+        else:
+            win_gate = 3
+
+        if generated_votes > win_gate:
+            result = "win"
+            reason = f"Generated idea received {generated_votes} votes, Original idea received {original_votes} votes."
+        elif generated_votes <= lose_gate:
+            result = "lose"
+            reason = f"Original idea received {original_votes} votes, Generated idea received {generated_votes} votes."
+        else:
+            result = "tie"
+            reason = f"Generated idea received {generated_votes} votes, Original idea received {original_votes} votes."
+
+        final_results[dim] = {
+            "res": result,
+            "reason": reason,
+            "vote_detail": {
+                "original_votes": original_votes,
+                "generated_votes": generated_votes
+            }
+        }
+
+    return {
+        "final_results": final_results,
+        "all_evaluations": all_evaluations
+    }
+
+
+# ImprovedIdeaEvaluator class
+class ImprovedIdeaEvaluator:
+    def __init__(self, idea_dict: dict):
+        self.idea_dict = idea_dict
+        self.original_data = {k: v for k, v in idea_dict.items() if k not in ["generated_idea_text", "generated_data"]}
+        self.original_data["Idea"] = self.original_data.get("core_idea", "")
+        self.original_data["RelatedWork"] = ast.literal_eval(self.original_data.get("related_work", "{}"))
+        self.original_data["ExistingSolutions"] = ast.literal_eval(self.original_data.get("existing_solutions", "{}"))
+        self.original_data["ImplementationSteps"] = ast.literal_eval(
+            self.original_data.get("implementation_steps", "{}"))
+        self.original_data["ImplementationOrder"] = ast.literal_eval(
+            self.original_data.get("implementation_order", "[]"))
+        self.original_data["EvaluationMetrics"] = ast.literal_eval(self.original_data.get("evaluation_metrics", "{}"))
+        self.original_data["Dataset"] = self.original_data.get("data", "")
+        self.original_data["ExpectedOutcome"] = self.original_data.get("expected_outcome", "")
+        self.generated_data = idea_dict["generated_data"]
+        self.idea = self.generated_data.get("Idea", "")
+        self.generated_data["Idea"] = self.idea
+        self.implementation_steps = self.generated_data.get("ImplementationSteps", {})
+        self.implementation_order = self.generated_data.get("ImplementationOrder", {})
+        self.dataset = self.generated_data.get("Dataset", "")
+        self.generated_data["Dataset"] = self.dataset
+        self.evaluation_metrics = self.generated_data.get("EvaluationMetrics", "")
+        self.expected_outcome = self.generated_data.get("ExpectedOutcome", "")
+        self.raw_scores = {
+            "novelty_similarity": 0.0,
+            "cutting_edge": 0.0,
+            "effectiveness_objective": 0.0,
+            "feasibility_objective": 0.0,
+            "completeness": 0.0,
+            "length_penalty": 0.0,
+            "repetition_penalty": 0.0
+        }
+        self.scores = {
+            "novelty_objective": 0.0,
+            "feasibility_objective": 0.0,
+            "detailedness_objective": 0.0,
+            "effectiveness_objective": 0.0,
+            "novelty": "",
+            "effectiveness": "",
+            "detailedness": "",
+            "feasibility": "",
+        }
+        self.details = {}
+
+    def evaluate_novelty_objective(self) -> None:
+        try:
+            text_to_compare = self.idea
+            related_work = self.original_data.get("RelatedWork", {})
+            existing_methods = self.original_data.get("ExistingSolutions", {})
+            all_existing_text = []
+            all_existing_text.extend(related_work.values())
+            all_existing_text.extend(existing_methods.values())
+            if all_existing_text and embedding_model is not None:
+                idea_embedding = embedding_model.encode([text_to_compare])
+                similarities = []
+                for existing_text in all_existing_text:
+                    existing_embedding = embedding_model.encode([existing_text])
+                    similarity = cosine_similarity(
+                        idea_embedding.reshape(1, -1),
+                        existing_embedding.reshape(1, -1)
+                    )[0][0]
+                    similarities.append(similarity)
+                avg_similarity = np.mean(similarities)
+                novelty_similarity_score = (1 - avg_similarity) * 10
+                novelty_similarity_score = max(0, min(10, novelty_similarity_score))
+            else:
+                novelty_similarity_score = 0.0
+            self.raw_scores["novelty_similarity"] = novelty_similarity_score
+            ref_related_work = self.original_data.get("related_work_test", "")
+            idea_embedding = embedding_model.encode([self.idea])
+            similarities = []
+            ref_related_work = ast.literal_eval(ref_related_work)
+            for key, value in ref_related_work.items():
+                snippet_data = f"{key}: {value}"
+                snippet_embedding = embedding_model.encode([snippet_data])
+                similarity = cosine_similarity(
+                    idea_embedding.reshape(1, -1),
+                    snippet_embedding.reshape(1, -1)
+                )[0][0]
+                similarities.append(similarity)
+            avg_similarity = np.mean(similarities)
+            cutting_edge_score = (1 - avg_similarity) * 10
+            cutting_edge_score = max(0, min(10, cutting_edge_score))
+            self.raw_scores["cutting_edge"] = cutting_edge_score
+        except Exception as e:
+            logger.error(f"Error in novelty evaluation: {e}")
+            self.raw_scores["novelty_similarity"] = 0.0
+            self.raw_scores["cutting_edge"] = 0.0
+            self.details["novelty_similarity"] = f"error: {str(e)}"
+            self.details["cutting_edge"] = f"error: {str(e)}"
+
+    def evaluate_effectiveness_objective(self) -> None:
+        try:
+            original_terms = self.original_data.get("keywords", [])
+            if embedding_model is None:
+                self.scores["effectiveness_objective"] = 0.0
+                self.details["effectiveness_objective"] = "embedding_model is not available"
+                return
+            terms_text = ", ".join([str(term) for term in original_terms])
+            idea_text = self.idea
+            try:
+                embeddings = embedding_model.encode([terms_text, idea_text], normalize_embeddings=True)
+                similarity = np.dot(embeddings[0], embeddings[1])
+                prof_score = similarity * 10
+                self.scores["effectiveness_objective"] = max(0, min(10, prof_score))
+            except Exception as e:
+                logger.error(f"Error computing embedding similarity: {e}")
+                matched_terms = []
+                generated_text_lower = idea_text.lower() if isinstance(idea_text, str) else ""
+                for term in original_terms:
+                    term_str = str(term).lower()
+                    if term_str in generated_text_lower:
+                        matched_terms.append(term)
+                hit_rate = len(matched_terms) / len(original_terms) if original_terms else 0
+                self.scores["effectiveness_objective"] = hit_rate * 10
+                similarity = hit_rate
+        except Exception as e:
+            logger.error(f"Error in effectiveness_objective evaluation: {e}")
+            self.scores["effectiveness_objective"] = 0.0
+
+    def evaluate_completeness(self) -> None:
+        required_sections = [
+            "Idea",
+            "ImplementationSteps",
+            "ImplementationOrder",
+            "EvaluationMetrics",
+            "Dataset",
+            "ExpectedOutcome"
+        ]
+        section_found = {
+            "Idea": self.idea is not None,
+            "ImplementationSteps": self.implementation_steps is not None,
+            "ImplementationOrder": self.implementation_order is not None,
+            "EvaluationMetrics": self.evaluation_metrics is not None,
+            "Data": self.dataset is not None,
+            "ExpectedOutcome": self.expected_outcome is not None
+        }
+        total_sections = len(required_sections)
+        completed_sections = sum(section_found.values())
+        self.raw_scores["completeness"] = (completed_sections / total_sections) * 10
+        self.details["completeness"] = {
+            "total_sections": total_sections,
+            "completed_sections": completed_sections,
+            "completion_rate": completed_sections / total_sections,
+        }
+        missing_sections = [section for section, found in section_found.items() if not found]
+        if missing_sections:
+            logger.warning(f"Missing required sections: {', '.join(missing_sections)}")
+
+    def evaluate_feasibility_objective(self) -> None:
+        try:
+            generated_implementation = {
+                "ImplementationSteps": self.implementation_steps,
+                "ImplementationOrder": self.implementation_order
+            }
+            original_implementation = {
+                "ImplementationSteps": self.original_data["ImplementationSteps"],
+                "ImplementationOrder": self.original_data["ImplementationOrder"]
+            }
+            similarity = graph_similarity(
+                generated_implementation,
+                original_implementation,
+                alpha=0.6
+            )
+            self.scores["feasibility_objective"] = similarity * 10
+            self.details["feasibility_objective"] = {
+                "score": similarity,
+            }
+        except Exception as e:
+            logger.error(f"Error evaluating feasibility objective: {e}")
+            self.scores["feasibility_objective"] = 0.0
+            self.details["feasibility_objective"] = {"error": str(e)}
+
+    def evaluate_penalties(self) -> None:
+        if self.idea:
+            char_count = len(self.idea)
+            penalty = 0.0
+            if char_count > 700:
+                excess_chars = char_count - 700
+                penalty += excess_chars / 100.0
+            elif char_count < 300:
+                deficit_chars = 300 - char_count
+                penalty += deficit_chars / 100.0
+            self.raw_scores["length_penalty"] = min(penalty, 10.0)
+        else:
+            self.raw_scores["length_penalty"] = 0.0
+        if isinstance(self.idea, str):
+            self.raw_scores["repetition_penalty"] = calculate_semantic_repetition(self.idea)
+        else:
+            self.raw_scores["repetition_penalty"] = 0.0
+        self.details["penalties"] = {
+            "text_length": len(self.idea),
+            "length_penalty": self.raw_scores["length_penalty"],
+            "repetition_penalty": self.raw_scores["repetition_penalty"]
+        }
+
+    def LLM_multi_rounds(self, llm_judges):
+        try:
+            context = get_context_from_data(self.original_data)
+            evaluation_results = compare_ideas_with_voting(
+                original_idea_data=self.original_data,
+                generated_idea_data=self.generated_data,
+                context=context,
+                judge_models=llm_judges
+            )
+            summary = {
+                "evaluation_details": evaluation_results,
+                "timestamp": datetime.now().isoformat()
+            }
+            self.scores["novelty_subjective"] = evaluation_results["final_results"]["novelty"]["res"]
+            self.scores["effectiveness_subjective"] = evaluation_results["final_results"]["effectiveness"]["res"]
+            self.scores["detailedness_subjective"] = evaluation_results["final_results"]["detailedness"]["res"]
+            self.scores["feasibility_subjective"] = evaluation_results["final_results"]["feasibility"]["res"]
+            return {
+                "success": True,
+                "result": summary
+            }
+        except Exception as e:
+            logger.error(f"Error in LLM_multi_rounds: {e}")
+            return {
+                "success": False,
+                "error": str(e)
+            }
+
+    def merge_scores(self) -> None:
+        self.scores["novelty_objective"] = (
+            0.5 * self.raw_scores["novelty_similarity"]
+            + 0.5 * self.raw_scores["cutting_edge"]
+        )
+        self.scores["detailedness_objective"] = (
+            0.2 * self.raw_scores["completeness"]
+            + 0.4 * (10 - self.raw_scores["repetition_penalty"])
+            + 0.4 * (10 - self.raw_scores["length_penalty"])
+        )
+
+    def calculate_final_score(self, llm_judges) -> Dict[str, Any]:
+        self.LLM_multi_rounds(llm_judges)
+        self.evaluate_novelty_objective()
+        self.evaluate_effectiveness_objective()
+        self.evaluate_completeness()
+        self.evaluate_feasibility_objective()
+        self.evaluate_penalties()
+        self.merge_scores()
+
+        self.idea_dict.update({
+            "effectiveness_objective": float(self.scores["effectiveness_objective"]) * 10,
+            "novelty_objective": float(self.scores["novelty_objective"]) * 10,
+            "detailedness_objective": float(self.scores["detailedness_objective"]) * 10,
+            "feasibility_objective": float(self.scores["feasibility_objective"]) * 10,
+            "effectiveness_subjective": 100 if self.scores["effectiveness_subjective"] == 'win' else 0,
+            "novelty_subjective": 100 if self.scores["novelty_subjective"] == 'win' else 0,
+            "detailedness_subjective": 100 if self.scores["detailedness_subjective"] == 'win' else 0,
+            "feasibility_subjective": 100 if self.scores["feasibility_subjective"] == 'win' else 0,
+        })
+
+        self.idea_dict.update({
+            "effectiveness": (self.idea_dict["effectiveness_objective"] + self.idea_dict[
+                "effectiveness_subjective"]) / 2,
+            "novelty": (self.idea_dict["novelty_objective"] + self.idea_dict["novelty_subjective"]) / 2,
+            "detailedness": (self.idea_dict["detailedness_objective"] + self.idea_dict["detailedness_subjective"]) / 2,
+            "feasibility": (self.idea_dict["feasibility_objective"] + self.idea_dict["feasibility_subjective"]) / 2,
+        })
+
+        self.idea_dict["final_score"] = (
+            self.idea_dict["effectiveness"]
+            + self.idea_dict["novelty"]
+            + self.idea_dict["detailedness"]
+            + self.idea_dict["feasibility"]
+        ) / 4
+
+        return self.idea_dict
+
+
+def evaluate_single_idea(ques_dict, llm_judges):
+    try:
+        evaluator = ImprovedIdeaEvaluator(ques_dict)
+        evaluation_result = evaluator.calculate_final_score(llm_judges=llm_judges)
+        output = evaluation_result
+        return output
+    except Exception as e:
+        logger.error(f"evaluation error: {e}")
+        output = {
+            "error": str(e),
+            "final_score": 0.0
+        }
+        return output
+
+
+class SGI_Bench_Idea_Generation(TextBaseDataset):
+    TYPE = 'QA'
+    example = {
+        "Idea": (
+            "We propose an adaptive optimization framework based on a dynamic feature interaction "
+            "network. This framework captures feature correlations through a hierarchical attention "
+            "mechanism and combines it with a data distribution-aware dynamic weight adjustment "
+            "strategy to improve the model's adaptability to heterogeneous data while ensuring "
+            "computational efficiency."
+        ),
+        "ImplementationSteps": {
+            "1": (
+                "Data preprocessing: missing value filling, outlier handling, feature "
+                "normalization and type conversion, and building a basic feature set"
+            ),
+            "2": (
+                "Feature engineering: generating statistically derived features, time series "
+                "features, and cross-features, and building a feature candidate pool"
+            ),
+            "3": (
+                "Model architecture design: building a basic network module, integrating a "
+                "hierarchical attention mechanism with a dynamic interaction layer"
+            ),
+            "4": (
+                "Dynamic weight mechanism implementation: designing a data distribution-aware "
+                "weight adjustment function and embedding it into the network's intermediate layers"
+            ),
+            "5": (
+                "Model training and tuning: adopting a phased training strategy, using grid "
+                "search and early stopping to optimize hyperparameters"
+            ),
+            "6": (
+                "Performance Verification: Conduct comparative experiments on multiple datasets "
+                "to analyze model performance differences in different scenarios."
+            )
+        },
+        "ImplementationOrder": ["1-2", "2-3", "3-4", "4-5", "1-5", "5-6"],
+        "Dataset": (
+            "Contains three types of public datasets and one actual business data: "
+            "1) Public structured dataset (approximately 500,000 samples, 30+ features); "
+            "2) Text-numeric mixed dataset (approximately 200,000 samples, including text "
+            "embedding features); 3) Time series sparse dataset (approximately 100,000 samples, "
+            "spanning 1 year); 4) Real transaction data from an e-commerce platform "
+            "(approximately 1 million samples, including user behavior and product attribute "
+            "features)"
+        ),
+        "EvaluationMetrics": {
+            "Prediction Accuracy": (
+                "AUC and F1-score are used for classification tasks; MAE and RMSE are used for "
+                "regression tasks to evaluate the basic predictive ability of the model."
+            ),
+            "Robustness": (
+                "Performance decay rate is calculated through data perturbation testing (adding "
+                "noise and simulating feature loss) to measure model stability."
+            ),
+            "Efficiency": (
+                "Record model training time, inference latency, and memory usage to evaluate "
+                "computing resource consumption."
+            ),
+            "Interpretability": (
+                "Use SHAP values and feature importance ranking to quantify the feature "
+                "contribution to model decisions."
+            ),
+            "Generalization": (
+                "Performance retention across datasets to evaluate the model's adaptability "
+                "to unseen data."
+            )
+        },
+        "ExpectedOutcome": (
+            "The proposed framework outperforms existing mainstream methods in comprehensive "
+            "performance (accuracy, robustness, and efficiency) across multiple datasets, "
+            "particularly in scenarios with uneven data distribution and cross-scenario migration. "
+            "It also enhances model interpretability through a dynamic feature interaction "
+            "mechanism, providing effective support for practical business decision-making."
+        )
+    }
+
+    @classmethod
+    def supported_datasets(cls):
+        return ["SGI-IdeaGeneration"]
+
+    def load_data(self, dataset):
+        hf = load_dataset("InternScience/SGI-IdeaGeneration", split="test")
+        rows: List[Dict[str, Any]] = []
+        idx = 0
+        for prob in hf:
+            rows.append({
+                "index": idx,
+                "id": prob.get("idx", idx),
+                "question": prob["question"],
+                "discipline": prob["discipline"],
+                "core_idea": prob["core_idea"],
+                "related_work": prob["related_work"],
+                "related_work_test": prob.get("related_work_test", "{}"),
+                "existing_solutions": prob["existing_solutions"],
+                "implementation_steps": prob["implementation_steps"],
+                "implementation_order": prob["implementation_order"],
+                "data": prob["data"],
+                "evaluation_metrics": prob["evaluation_metrics"],
+                "expected_outcome": prob["expected_outcome"],
+                "keywords": prob.get("keywords", [])
+            })
+            idx += 1
+        return pd.DataFrame(rows)
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+
+        prompt = line['question'] + f"""\n\n### Example:
+```json
+{json.dumps(self.example, indent=4)}
+```"""
+        msgs = [{'type': 'text', 'value': prompt}]
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        data = load(eval_file)
+        data = pd.DataFrame(data)
+        global embedding_model
+        # 尝试加载嵌入模型进行评估
+        if embedding_model is None:
+            try:
+                from sentence_transformers import SentenceTransformer
+                logger.info("Loading SentenceTransformer embedding model...")
+                embedding_model = SentenceTransformer('all-MiniLM-L6-v2')
+                logger.info("SentenceTransformer embedding model loaded successfully.")
+            except Exception as e:
+                logger.error(f"Failed to load SentenceTransformer model: {e}")
+                embedding_model = None
+
+        data['generated_data'] = None
+        data['generated_data'] = data['generated_data'].astype(object)
+
+        data['generated_idea_text'] = None
+        data['generated_idea_text'] = data['generated_idea_text'].astype(object)
+
+        # 处理每个生成的想法
+        # Default judge models
+        JUDGE_MODELS = ["gpt-5.1", "gemini-3-pro-preview", "claude-sonnet-4-5-20250929"]
+        # JUDGE_MODELS = ["gpt-5.1-2025-11-13", "gemini-3-pro-preview", "claude-sonnet-4-5-20250929"]
+        llm_judges = [build_judge(**{**judge_kwargs, 'model': i}) for i in JUDGE_MODELS]
+        tups = []
+        indices = []
+        for idx, row in data.iterrows():
+            prediction = row['prediction']
+
+            # 解析生成的想法
+            if isinstance(prediction, str):
+                parsed_data = parse_generated_idea(prediction)
+                data.at[idx, 'generated_data'] = parsed_data
+                data.at[idx, 'generated_idea_text'] = prediction
+
+                # 构建评估所需的原始数据字典
+                ques_dict = row.to_dict()
+                ques_dict['generated_data'] = parsed_data
+                ques_dict['generated_idea_text'] = prediction
+
+                # 评估单个想法
+                tups.append((ques_dict, llm_judges))
+                indices.append(idx)
+
+        tmp_file = get_intermediate_file_path(eval_file, '_judge_tmp', 'pkl')
+        ans = {}
+        if osp.exists(tmp_file):
+            ans = load(tmp_file)
+        tups = [x for x, i in zip(tups, indices) if i not in ans]
+        indices = [i for i in indices if i not in ans]
+
+        if len(indices):
+            track_progress_rich(
+                func=evaluate_single_idea,
+                tasks=tups,
+                nproc=judge_kwargs.get('nproc', 32),
+                save=tmp_file,
+                keys=indices,
+            )
+            ans = load(tmp_file)
+
+        for idx, evaluation_result in ans.items():
+            # 将评估结果添加到数据中
+            for key, value in evaluation_result.items():
+                if key not in ['generated_data', 'generated_idea_text']:
+                    data.loc[idx, key] = value
+
+        # 计算平均分数
+        successful_evaluations = data[~data['final_score'].isna()]
+        if len(successful_evaluations) > 0:
+            avg_effectiveness_objective = successful_evaluations['effectiveness_objective'].mean()
+            avg_novelty_objective = successful_evaluations['novelty_objective'].mean()
+            avg_detailedness_objective = successful_evaluations['detailedness_objective'].mean()
+            avg_feasibility_objective = successful_evaluations['feasibility_objective'].mean()
+
+            avg_effectiveness_subjective = successful_evaluations['effectiveness_subjective'].mean()
+            avg_novelty_subjective = successful_evaluations['novelty_subjective'].mean()
+            avg_detailedness_subjective = successful_evaluations['detailedness_subjective'].mean()
+            avg_feasibility_subjective = successful_evaluations['feasibility_subjective'].mean()
+
+            effectiveness_score = successful_evaluations['effectiveness'].mean()
+            novelty_score = successful_evaluations['novelty'].mean()
+            detailedness_score = successful_evaluations['detailedness'].mean()
+            feasibility_score = successful_evaluations['feasibility'].mean()
+
+            avg_final_score = successful_evaluations['final_score'].mean()
+
+            result = {
+                "final_score": float(avg_final_score),
+                "effectiveness": float(effectiveness_score),
+                "novelty": float(novelty_score),
+                "detailedness": float(detailedness_score),
+                "feasibility": float(feasibility_score),
+                "details": {
+                    "effectiveness_objective": float(avg_effectiveness_objective),
+                    "effectiveness_subjective": float(avg_effectiveness_subjective),
+                    "novelty_objective": float(avg_novelty_objective),
+                    "novelty_subjective": float(avg_novelty_subjective),
+                    "detailedness_objective": float(avg_detailedness_objective),
+                    "detailedness_subjective": float(avg_detailedness_subjective),
+                    "feasibility_objective": float(avg_feasibility_objective),
+                    "feasibility_subjective": float(avg_feasibility_subjective),
+                    "successful_evaluations": len(successful_evaluations),
+                    "total_evaluations": len(data)
+                }
+            }
+        else:
+            result = {
+                "final_score": 0.0,
+                "effectiveness": 0.0,
+                "novelty": 0.0,
+                "detailedness": 0.0,
+                "feasibility": 0.0,
+                "error": "No successful evaluations"
+            }
+
+        # 保存结果
+        score_file = get_intermediate_file_path(eval_file, '_score', 'csv')
+        result_file = get_intermediate_file_path(eval_file, '_result', 'json')
+        dump(data, score_file)
+        dump(result, result_file)
+
+        return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/readme.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/readme.md
new file mode 100644
index 0000000000000000000000000000000000000000..a81f1dd77bd407bdc149a24ab1bd7a52a501e0df
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/readme.md
@@ -0,0 +1,12 @@
+## 整体说明
+SGI-Bench-1.0 包含5个子数据集(deep research , dry experiment , wet experiment , experimental reasoning, idea generation)
+
+## 注意事项
+1. dry experiment , experimental reasoning和deep research以及idea generation使用了模型进行评估，需要设置`OPENAI_API_KEY`,以及`OPENAI_API_BASE`环境变量
+2. dry experiment评测过程中需要下载文件，默认路径是`./outputs`，可以通过`--judge-args`命令行参数传入`work_dir`参数进行控制。<br> 评测之前还需要运行以下命令
+3. idea generation 的评测需要额外安装`sentence_transformers`包
+```
+conda create -n dryexp python=3.10.18
+conda activate dryexp
+pip install -r vlmeval/dataset/SGI_Bench_1_0/dry_experiment_requirements.txt
+```
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..307ca3a865b9abbe17b59540b28d8f27f1eb816b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/utils.py
@@ -0,0 +1,232 @@
+import re
+
+# ############################ Idea Generation ##############################
+
+
+def format_idea_data(idea_data):
+    fields = [
+        "Idea",
+        "ImplementationSteps",
+        "ImplementationOrder",
+        "Dataset",
+        "EvaluationMetrics",
+        "ExpectedOutcome"
+    ]
+
+    formatted_text = ""
+    for field in fields:
+        if field in idea_data and idea_data[field]:
+            formatted_text += f"{field}: {idea_data[field]}\n\n"
+
+    return formatted_text.strip()
+
+
+def get_context_from_data(data):
+    context_fields = [
+        "related_work",
+        "challenge",
+        "limitation",
+        "motivation",
+        "task_objective",
+        "existing_solutions"
+    ]
+    context = ""
+    for field in context_fields:
+        if field in data and data[field]:
+            context += f"{field}: {data[field]}\n\n"
+
+    return context.strip()
+
+
+def flip_evaluation_result(result):
+    flipped = {}
+    mapping = {
+        "win_A": "win_B",
+        "win_B": "win_A"
+    }
+
+    for key, value in result.items():
+        if isinstance(value, dict) and "judgment" in value:
+            flipped[key] = {
+                "judgment": mapping.get(value["judgment"], value["judgment"]),
+                "reason": value.get("reason", "")
+            }
+        else:
+            flipped[key] = mapping.get(value, value)
+
+    return flipped
+
+
+def get_evaluation_prompt_modified(hypothesis_A, hypothesis_B, context=None):
+    context_text = f"Context:\n{context}\n\n" if context else ""
+
+    prompt = f"""
+You are assisting researchers tasked with comparing TWO research hypotheses (Hypothesis A and Hypothesis B).
+Your job is to evaluate both hypotheses across five separate dimensions defined below, and to choose a winner
+(either Hypothesis A or Hypothesis B) for each dimension. Ties are NOT allowed — you MUST pick one winner per
+dimension. Base your judgments on scientific principles and the provided context only.
+
+##Background context:
+{context_text}
+
+##Hypothesis A:
+{hypothesis_A}
+
+##Hypothesis B:
+{hypothesis_B}
+
+##Definition of each dimension:
+###1) Effectiveness
+Which hypothesis is more likely to produce a successful experimental or empirical outcome in service of the stated
+research objective? Evaluate the likelihood that, if implemented using standard practices in the relevant discipline,
+the hypothesis will achieve the intended measurable result. Focus on mechanistic plausibility, causal logic, and
+whether the hypothesis addresses the core problem directly.
+
+###2)Novelty
+Novelty: Which hypothesis presents more innovative or original approaches? Compare the similarity between the idea
+and the related work and existing solutions in the background to assess its novelty. A lower similarity to the core
+idea indicates greater novelty.
+
+###3) Detailedness (Level of Specification)
+Which hypothesis provides clearer, more actionable, and more complete specification of mechanisms, assumptions,
+experimental steps, required variables, and dependencies? Detailedness rewards clarity that would enable a competent
+researcher to design an experiment or implementation with minimal ambiguity.
+
+###4) Feasibility
+Which hypothesis presents a more realistic and implementable solution given current technological constraints?
+
+###5) Overall
+Considering the overall aspects together but emphasizing conceptual coherence and scientific grounding, which
+hypothesis is superior overall? This is a synthesis judgment: prefer the hypothesis that is logically consistent,
+grounded in accepted principles, avoids critical unstated assumptions or contradictions, and is most defensible as
+a scientific proposition.
+
+Unified constraints:
+- Use only the provided context and widely accepted scientific principles in the relevant discipline. Do NOT invent
+facts external to the context unless they are broadly standard domain knowledge.
+- When a dimension explicitly says to ignore other factors (e.g., Novelty should ignore feasibility), strictly follow
+that guidance for that dimension. When evaluating a certain dimension, it should focus on this dimension itself and
+ignore the influence of other dimensions.
+- Be concise but specific: for each dimension provide a short judgment line (exact format below) plus 1–3 sentences
+of succinct reasoning grounded in the definitions above.
+- Format must match exactly (case-insensitive for "Win A/Win B") and include a reason after "because".
+
+
+##Output format (MUST FOLLOW EXACTLY)
+
+Format your response exactly as follows:
+Effectiveness: [Win A/Win B] because ...
+Novelty: [Win A/Win B] because ...
+Detailedness: [Win A/Win B] because ...
+Feasibility: [Win A/Win B] because ...
+Overall: [Win A/Win B] because ...
+"""
+    return prompt
+
+
+def parse_evaluation_result(result):
+    dimensions = ["effectiveness", "novelty", "detailedness", "feasibility", "overall"]
+    parsed_results = {}
+    all_valid = True
+
+    for dim in dimensions:
+        judgment = extract_win_lose(result, dim.capitalize())
+        reason = extract_reason(result, dim.capitalize())
+
+        if judgment is None:
+            all_valid = False
+            break
+
+        parsed_results[dim] = {
+            "judgment": judgment,
+            "reason": reason
+        }
+
+    if not all_valid:
+        return None
+
+    return parsed_results
+
+
+def extract_win_lose(result_text, dimension):
+    pattern = rf"{dimension}\s*:\s*\[\s*(Win\s*A|Win\s*B)\s*\]"
+    match = re.search(pattern, result_text, re.IGNORECASE)
+    if match:
+        judgment = match.group(1).strip().upper()
+        if "WIN A" in judgment:
+            return "win_A"
+        else:
+            return "win_B"
+
+    backup_pattern = rf"{dimension}\s*:\s*(Win\s*A|Win\s*B)\s+"
+    match = re.search(backup_pattern, result_text, re.IGNORECASE)
+    if match:
+        judgment = match.group(1).strip().upper()
+        if "WIN A" in judgment:
+            return "win_A"
+        else:
+            return "win_B"
+
+    line_pattern = rf"{dimension}[^\n]*?(Win\s*A|Win\s*B)"
+    match = re.search(line_pattern, result_text, re.IGNORECASE)
+    if match:
+        judgment = match.group(1).strip().upper()
+        if "WIN A" in judgment:
+            return "win_A"
+        else:
+            return "win_B"
+
+    return None
+
+
+def extract_reason(result_text, dimension):
+    pattern = rf"{dimension}\s*:\s*\[[^\]]+\]\s*because\s*(.*?)(?=\n\w|$)"
+    match = re.search(pattern, result_text, re.IGNORECASE | re.DOTALL)
+    if match:
+        reason = match.group(1).strip()
+        return reason
+
+    backup_pattern = rf"{dimension}\s*:[^\n]*?(because|due to|as|since)([^\n]+)"
+    match = re.search(backup_pattern, result_text, re.IGNORECASE)
+    if match:
+        reason = match.group(2).strip()
+        return reason
+
+    fallback_pattern = rf"{dimension}\s*:[^\n]*(.*?)(?=\n\w+:|$)"
+    match = re.search(fallback_pattern, result_text, re.IGNORECASE | re.DOTALL)
+    if match:
+        text = match.group(1).strip()
+        reason = re.sub(r"\[(Win\s*A|Win\s*B)\]", "", text).strip()
+        return reason
+
+    return "No specific reason provided"
+
+
+# ############################ Idea Generation ##############################
+
+
+def mean(lst: list):
+    assert len(lst) > 0, "list length must > 0"
+    return sum(lst) / len(lst)
+
+
+def show_results(results: list[dict], metric_name: str, category_name: str = None, precision: int = 2, scale=1):
+    category_dict = {}
+    for item in results:
+        if category_name is None:
+            item_category = 'default'
+        else:
+            item_category = item[category_name]
+        if item_category not in category_dict:
+            category_dict[item_category] = []
+
+        item_metric = float(item[metric_name])
+        category_dict[item_category].append(item_metric)
+
+    for k, v in category_dict.items():
+        category_dict[k] = round(mean(v) * scale, precision)
+
+    if category_name is None:
+        return category_dict['default']
+    else:
+        return category_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/wet_experiment.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/wet_experiment.py
new file mode 100644
index 0000000000000000000000000000000000000000..0c6cffed9ea8781026b46c79e0de596b39a09900
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/SGI_Bench_1_0/wet_experiment.py
@@ -0,0 +1,360 @@
+import re
+from itertools import combinations
+from typing import Any, Dict, List
+
+import pandas as pd
+from datasets import load_dataset
+
+from vlmeval.smp import dump, get_intermediate_file_path, load
+from ..text_base import TextBaseDataset
+
+
+def parse_experiment_steps(text):
+    # Regular expression to match experiment steps until
+    # encountering a single line containing only a right
+    # parenthesis ")"
+    # Match format: variable_name = <action_name>(parameter_list)
+    # Capture groups:
+    #   1: output variable name (e.g., "multimer_cells")
+    #   2: action name (e.g., "Incubate cells with MHC
+    #      multimers")
+    #   3: parameter list (e.g.,
+    #      "cells=washed_cells,\nmultimer_pool=...")
+    #   Condition: parameter list continues until a single
+    #   line of ")" (whitespace allowed around it)
+    step_pattern = (
+        r'(\w+)\s*=\s*<([^>]+)>\(\s*([\s\S]*?)'
+        r'(?=\n\s*\)\s*$)'
+    )
+    # Regular expression to match each parameter line
+    # Match format: key=value or key=value,
+    # Capture groups:
+    #   1: parameter key (e.g., "cells")
+    #   2: parameter value (e.g., "washed_cells" or
+    #      "\"tetramer pool (23 nM each)\"")
+    #   (?:,)? : optionally match a trailing comma at the
+    #   end of the line, ignore the comma
+    param_pattern = (
+        r'^\s*(\w+)\s*=\s*(.*?)\s*(?:,)?\s*$'
+    )
+    steps = []
+
+    for match in re.finditer(step_pattern, text, re.MULTILINE):
+        output_var = match.group(1).strip()  # Extract output variable name
+        action_name = match.group(2).strip()  # Extract action name
+        params = match.group(3).strip()      # Extract parameter list
+
+        param_dict = {}
+        # Split the parameter list by lines,
+        # ignoring empty lines and single-line ")"
+        param_lines = [
+            line.strip()
+            for line in params.split('\n')
+            if line.strip() and line.strip() != ')'
+        ]
+        for line in param_lines:
+            param_match = re.match(param_pattern, line)
+            if param_match:
+                key = param_match.group(1)       # Extract parameter key
+                value = param_match.group(2).strip()
+                # If the value starts and ends with
+                # double quotes, remove the quotes
+                if value.startswith('"') and value.endswith('"'):
+                    value = value[1:-1]
+                param_dict[key] = value
+
+        # Build the step dictionary
+        steps.append({
+            "action": action_name,
+            "input": param_dict,
+            "output": output_var
+        })
+
+    return steps
+
+
+def identify_variable_types(steps):
+    """
+    Identify raw variables and generated variables in the experimental steps.
+    Raw variables: variables that never appear as outputs in any step.
+    Generated variables: variables that appear as outputs of some step.
+
+    Returns:
+        original_vars (set): set of raw variables
+        generated_vars (set): set of generated variables (function outputs)
+        output_to_step_map (dict): mapping from output variable name to the index of
+            its generating step (for reverse lookup)
+    """
+    generated_vars = set()
+    all_input_vars = set()
+    output_to_step_map = {}
+
+    for idx, step in enumerate(steps):
+        output_var = step["output"]
+        generated_vars.add(output_var)
+        output_to_step_map[output_var] = idx  # Store the step index
+
+        for input_val in step["input"].values():
+            # Simple check whether it is a variable (non-string literal, non-numeric)
+            # If input_val is a string and does not start and end with quotes,
+            # and is not purely numeric, consider it a variable
+            if (
+                isinstance(input_val, str)
+                and not (input_val.startswith('"') and input_val.endswith('"'))
+                and not (
+                    input_val.replace('.', '', 1).isdigit()
+                    or (
+                        input_val.startswith('-')
+                        and input_val[1:].replace('.', '', 1).isdigit()
+                    )
+                )
+            ):
+                all_input_vars.add(input_val)
+
+    # Raw variables are those input variables that are not in the set of output variables of any step
+    original_vars = all_input_vars - generated_vars
+
+    return original_vars, generated_vars, output_to_step_map
+
+
+def compare_exp_steps(gt_steps, pred_steps):
+    def kendall_tau_distance(seq1, seq2):
+        if len(seq1) != len(seq2):
+            return 0.0
+        n = len(seq1)
+        if n <= 1:
+            return 1.0
+        inversions = 0
+        for i, j in combinations(range(n), 2):
+            if (seq1[i] < seq1[j] and seq2[i] > seq2[j]) or (seq1[i] > seq1[j] and seq2[i] < seq2[j]):
+                inversions += 1
+        max_inversions = n * (n - 1) / 2
+        return 1.0 - (inversions / max_inversions if max_inversions > 0 else 0.0)
+
+    results = {
+        "order_similarity": 0.0,
+        "error_rate": 0.0,
+        "details": []
+    }
+
+    actions_gt = [step["action"] for step in gt_steps]
+    actions_pred = [step["action"] for step in pred_steps]
+
+    results["order_similarity"] = kendall_tau_distance(actions_gt, actions_pred)
+
+    # Identify variable types and build output mappings
+    original_vars_gt, generated_vars_gt, output_to_step_map_gt = identify_variable_types(gt_steps)
+    original_vars_pred, generated_vars_pred, output_to_step_map_pred = identify_variable_types(pred_steps)
+    # output_to_step_map_pred is only used to judge whether an input is a generated variable
+
+    # Dictionary mapping variable names in pred_steps to corresponding variables in gt_steps
+    var_map_pred2gt = {}
+
+    error_count = 0
+    min_len = min(len(gt_steps), len(pred_steps))
+
+    for i in range(min_len):
+        step_gt = gt_steps[i]
+        step_pred = pred_steps[i]
+        detail = {
+            "step": i + 1,
+            "action_gt": step_gt["action"],
+            "action_pred": step_pred["action"],
+            "status": "✅ success",
+            "message": ""
+        }
+
+        # 1. Check whether the action names match
+        if step_gt["action"] != step_pred["action"]:
+            detail["status"] = "❌ error"
+            detail["message"] += f"Action mismatch: expected '{step_gt['action']}', got '{step_pred['action']}'. "
+            error_count += 1
+            results["details"].append(detail)
+            continue
+
+        # 2. Check the set of parameter keys
+        keys_gt = set(step_gt["input"].keys())
+        keys_pred = set(step_pred["input"].keys())
+        if keys_gt != keys_pred:
+            detail["status"] = "❌ error"
+            detail["message"] += f"Parameter keys mismatch: expected {keys_gt}, got {keys_pred}. "
+            error_count += 1
+            results["details"].append(detail)
+            continue
+
+        # 3. Check argument passing
+        is_step_error = False  # Flag whether the current step has parameter errors
+        for key in keys_gt:
+            value_gt = step_gt["input"][key]
+            value_pred = step_pred["input"][key]
+
+            # Determine whether the parameter is a raw variable or a generated variable
+            is_input_var_gt_generated = value_gt in generated_vars_gt
+            is_input_var_pred_generated = value_pred in generated_vars_pred
+
+            # Case 1: Both gt_steps and pred_steps inputs are generated variables (outputs from previous steps)
+            if is_input_var_gt_generated and is_input_var_pred_generated:
+                # Try mapping variables from pred_steps to the corresponding variables in gt_steps
+                mapped_value_pred = var_map_pred2gt.get(value_pred)
+
+                # If the variable from pred_steps successfully maps to the corresponding variable in gt_steps,
+                # and the mapped value matches the expected value in gt_steps
+                if mapped_value_pred == value_gt:
+                    pass  # Match succeeds; continue
+                else:
+                    detail["status"] = "❌ error"
+                    detail["message"] += (
+                        f"Parameter '{key}' generated variable reference mismatch: "
+                        f"expected from '{value_gt}', got from '{value_pred}' "
+                        f"(mapped as '{mapped_value_pred}'). "
+                    )
+                    is_step_error = True
+            # Case 2: Both inputs are raw variables (literals or inputs not defined as function outputs)
+            elif not is_input_var_gt_generated and not is_input_var_pred_generated:
+                # For raw variables, do not strictly require identical values;
+                # even if values differ, consider it correct
+                pass
+            # Case 3: Type mismatch (one is a raw variable, the other is a generated variable)
+            else:
+                detail["status"] = "❌ error"
+                detail["message"] += (
+                    f"Parameter '{key}' type mismatch: "
+                    f"expected {'generated variable' if is_input_var_gt_generated else 'raw variable'}, "
+                    f"got {'generated variable' if is_input_var_pred_generated else 'raw variable'}. "
+                )
+                is_step_error = True
+
+        # If the current step has no parameter errors, update the variable mapping
+        if not is_step_error:
+            # Only when the action and parameters both match,
+            # map the output variable in pred_steps to the output variable in gt_steps
+            var_map_pred2gt[step_pred["output"]] = step_gt["output"]
+        else:
+            # If the step has errors, increment the error count
+            error_count += 1
+
+        results["details"].append(detail)
+
+    # Handle the case where lengths are inconsistent
+    if len(gt_steps) != len(pred_steps):
+        error_count += abs(len(gt_steps) - len(pred_steps))
+        if len(pred_steps) > len(gt_steps):
+            for i in range(min_len, len(pred_steps)):
+                results["details"].append({
+                    "step": i + 1,
+                    "action_gt": None,
+                    "action_pred": pred_steps[i]["action"],
+                    "status": "❌ error",
+                    "message": "Extra step."
+                })
+        elif len(gt_steps) > len(pred_steps):
+            for i in range(min_len, len(gt_steps)):
+                results["details"].append({
+                    "step": i + 1,
+                    "action_gt": gt_steps[i]["action"],
+                    "action_pred": None,
+                    "status": "❌ error",
+                    "message": "Missing step."
+                })
+
+    results["parameter_acc"] = 1 - (error_count / max(len(gt_steps), len(pred_steps)))
+
+    return results
+
+
+def extract_final_answer(answer_with_thinking: str, start_tag='<answer>', end_tag='</answer>'):
+    answer_with_thinking = str(answer_with_thinking)
+    start_index = answer_with_thinking.rfind(start_tag)
+    if start_index != -1:
+        end_index = answer_with_thinking.find(end_tag, start_index)
+        if end_index != -1:
+            return answer_with_thinking[start_index + len(start_tag):end_index].strip()
+    return None
+
+
+class SGI_Bench_Wet_Experiment(TextBaseDataset):
+    TYPE = 'QA'
+
+    @classmethod
+    def supported_datasets(cls):
+        return ["SGI-WetExperiment"]
+
+    def load_data(self, dataset):
+        hf = load_dataset("InternScience/SGI-WetExperiment", split="test")
+
+        rows: List[Dict[str, Any]] = []
+        idx = 0
+        for prob in hf:
+            rows.append(
+                {
+                    "index": idx,
+                    "id": prob["idx"],
+                    "question": prob["question"],
+                    "answer": prob["answer"],
+                    "discipline": prob["discipline"],
+                    "direction": prob["direction"]
+                }
+            )
+            idx += 1
+        return pd.DataFrame(rows)
+
+    def build_prompt(self, line):
+        if isinstance(line, int):
+            line = self.data.iloc[line]
+        question = line['question'] + """
+The final answer should be enclosed by <answer> and </answer>.
+
+Example:
+<answer>
+dataset = <Load dataset>(
+    source="imagenet"
+)
+
+model_init = <Initialize model>(
+    model_type="CNN"
+)
+
+model_trained = <Train model>(
+    model=model_init,
+    data=dataset
+)
+
+metrics = <Calculate metrics>(
+    model=model_trained,
+    data=dataset
+)
+</answer>
+"""
+
+        msgs = [{'type': 'text', 'value': question}]
+        return msgs
+
+    def evaluate(self, eval_file, **judge_kwargs):
+        data = load(eval_file)
+        data = pd.DataFrame(data)
+
+        data['action_sequence_similarity'] = 0
+        data['parameter_accuracy'] = 0
+        for index, row in data.iterrows():
+            target_steps = row['answer']
+            target_steps = parse_experiment_steps(target_steps)
+            extracted_text = extract_final_answer(row['prediction'])
+            if extracted_text:
+                prediction_steps = parse_experiment_steps(extracted_text)
+            else:
+                prediction_steps = []
+
+            steps_result = compare_exp_steps(target_steps, prediction_steps)
+
+            data.loc[index, 'action_sequence_similarity'] = steps_result['order_similarity']
+            data.loc[index, 'parameter_accuracy'] = steps_result['parameter_acc']
+
+        score_file = get_intermediate_file_path(eval_file, '_score', 'csv')
+        result = {
+            "Action_Sequence_Similarity": data['action_sequence_similarity'].mean(),
+            "Parameter_Accuracy": data['parameter_accuracy'].mean(),
+        }
+        result_file = get_intermediate_file_path(eval_file, '_result', 'json')
+        dump(data, score_file)
+        dump(result, result_file)
+        return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..2fe699198099deb764580cf4143f9b448f902b15
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/__init__.py
@@ -0,0 +1 @@
+from .render import compare_rendered_equations, render_equation  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/auto-render.min.js b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/auto-render.min.js
new file mode 100644
index 0000000000000000000000000000000000000000..418ba30dc5c6fb630daba974ca595fe9cd976d92
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/auto-render.min.js
@@ -0,0 +1 @@
+!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t(require("katex")):"function"==typeof define&&define.amd?define(["katex"],t):"object"==typeof exports?exports.renderMathInElement=t(require("katex")):e.renderMathInElement=t(e.katex)}("undefined"!=typeof self?self:this,(function(e){return function(){"use strict";var t={757:function(t){t.exports=e}},n={};function r(e){var o=n[e];if(void 0!==o)return o.exports;var i=n[e]={exports:{}};return t[e](i,i.exports,r),i.exports}r.n=function(e){var t=e&&e.__esModule?function(){return e.default}:function(){return e};return r.d(t,{a:t}),t},r.d=function(e,t){for(var n in t)r.o(t,n)&&!r.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},r.o=function(e,t){return Object.prototype.hasOwnProperty.call(e,t)};var o={};r.d(o,{default:function(){return p}});var i=r(757),a=r.n(i);const l=function(e,t,n){let r=n,o=0;const i=e.length;for(;r<t.length;){const n=t[r];if(o<=0&&t.slice(r,r+i)===e)return r;"\\"===n?r++:"{"===n?o++:"}"===n&&o--,r++}return-1},s=/^\\begin{/;var d=function(e,t){let n;const r=[],o=new RegExp("("+t.map((e=>e.left.replace(/[-/\\^$*+?.()|[\]{}]/g,"\\$&"))).join("|")+")");for(;n=e.search(o),-1!==n;){n>0&&(r.push({type:"text",data:e.slice(0,n)}),e=e.slice(n));const o=t.findIndex((t=>e.startsWith(t.left)));if(n=l(t[o].right,e,t[o].left.length),-1===n)break;const i=e.slice(0,n+t[o].right.length),a=s.test(i)?i:e.slice(t[o].left.length,n);r.push({type:"math",data:a,rawData:i,display:t[o].display}),e=e.slice(n+t[o].right.length)}return""!==e&&r.push({type:"text",data:e}),r};const c=function(e,t){const n=d(e,t.delimiters);if(1===n.length&&"text"===n[0].type)return null;const r=document.createDocumentFragment();for(let e=0;e<n.length;e++)if("text"===n[e].type)r.appendChild(document.createTextNode(n[e].data));else{const o=document.createElement("span");let i=n[e].data;t.displayMode=n[e].display;try{t.preProcess&&(i=t.preProcess(i)),a().render(i,o,t)}catch(o){if(!(o instanceof a().ParseError))throw o;t.errorCallback("KaTeX auto-render: Failed to parse `"+n[e].data+"` with ",o),r.appendChild(document.createTextNode(n[e].rawData));continue}r.appendChild(o)}return r},f=function(e,t){for(let n=0;n<e.childNodes.length;n++){const r=e.childNodes[n];if(3===r.nodeType){let o=r.textContent,i=r.nextSibling,a=0;for(;i&&i.nodeType===Node.TEXT_NODE;)o+=i.textContent,i=i.nextSibling,a++;const l=c(o,t);if(l){for(let e=0;e<a;e++)r.nextSibling.remove();n+=l.childNodes.length-1,e.replaceChild(l,r)}else n+=a}else if(1===r.nodeType){const e=" "+r.className+" ";-1===t.ignoredTags.indexOf(r.nodeName.toLowerCase())&&t.ignoredClasses.every((t=>-1===e.indexOf(" "+t+" ")))&&f(r,t)}}};var p=function(e,t){if(!e)throw new Error("No element provided to render");const n={};for(const e in t)t.hasOwnProperty(e)&&(n[e]=t[e]);n.delimiters=n.delimiters||[{left:"$$",right:"$$",display:!0},{left:"\\(",right:"\\)",display:!1},{left:"\\begin{equation}",right:"\\end{equation}",display:!0},{left:"\\begin{align}",right:"\\end{align}",display:!0},{left:"\\begin{alignat}",right:"\\end{alignat}",display:!0},{left:"\\begin{gather}",right:"\\end{gather}",display:!0},{left:"\\begin{CD}",right:"\\end{CD}",display:!0},{left:"\\[",right:"\\]",display:!0}],n.ignoredTags=n.ignoredTags||["script","noscript","style","textarea","pre","code","option"],n.ignoredClasses=n.ignoredClasses||[],n.errorCallback=n.errorCallback||console.error,n.macros=n.macros||{},f(e,n)};return o=o.default}()}));
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/katex.min.css b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/katex.min.css
new file mode 100644
index 0000000000000000000000000000000000000000..30156f0857ddafd0a7bceeb12546e17161ba10c6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/katex.min.css
@@ -0,0 +1 @@
+@font-face{font-family:KaTeX_AMS;font-style:normal;font-weight:400;src:url(fonts/KaTeX_AMS-Regular.woff2) format("woff2"),url(fonts/KaTeX_AMS-Regular.woff) format("woff"),url(fonts/KaTeX_AMS-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Caligraphic;font-style:normal;font-weight:700;src:url(fonts/KaTeX_Caligraphic-Bold.woff2) format("woff2"),url(fonts/KaTeX_Caligraphic-Bold.woff) format("woff"),url(fonts/KaTeX_Caligraphic-Bold.ttf) format("truetype")}@font-face{font-family:KaTeX_Caligraphic;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Caligraphic-Regular.woff2) format("woff2"),url(fonts/KaTeX_Caligraphic-Regular.woff) format("woff"),url(fonts/KaTeX_Caligraphic-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Fraktur;font-style:normal;font-weight:700;src:url(fonts/KaTeX_Fraktur-Bold.woff2) format("woff2"),url(fonts/KaTeX_Fraktur-Bold.woff) format("woff"),url(fonts/KaTeX_Fraktur-Bold.ttf) format("truetype")}@font-face{font-family:KaTeX_Fraktur;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Fraktur-Regular.woff2) format("woff2"),url(fonts/KaTeX_Fraktur-Regular.woff) format("woff"),url(fonts/KaTeX_Fraktur-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Main;font-style:normal;font-weight:700;src:url(fonts/KaTeX_Main-Bold.woff2) format("woff2"),url(fonts/KaTeX_Main-Bold.woff) format("woff"),url(fonts/KaTeX_Main-Bold.ttf) format("truetype")}@font-face{font-family:KaTeX_Main;font-style:italic;font-weight:700;src:url(fonts/KaTeX_Main-BoldItalic.woff2) format("woff2"),url(fonts/KaTeX_Main-BoldItalic.woff) format("woff"),url(fonts/KaTeX_Main-BoldItalic.ttf) format("truetype")}@font-face{font-family:KaTeX_Main;font-style:italic;font-weight:400;src:url(fonts/KaTeX_Main-Italic.woff2) format("woff2"),url(fonts/KaTeX_Main-Italic.woff) format("woff"),url(fonts/KaTeX_Main-Italic.ttf) format("truetype")}@font-face{font-family:KaTeX_Main;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Main-Regular.woff2) format("woff2"),url(fonts/KaTeX_Main-Regular.woff) format("woff"),url(fonts/KaTeX_Main-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Math;font-style:italic;font-weight:700;src:url(fonts/KaTeX_Math-BoldItalic.woff2) format("woff2"),url(fonts/KaTeX_Math-BoldItalic.woff) format("woff"),url(fonts/KaTeX_Math-BoldItalic.ttf) format("truetype")}@font-face{font-family:KaTeX_Math;font-style:italic;font-weight:400;src:url(fonts/KaTeX_Math-Italic.woff2) format("woff2"),url(fonts/KaTeX_Math-Italic.woff) format("woff"),url(fonts/KaTeX_Math-Italic.ttf) format("truetype")}@font-face{font-family:"KaTeX_SansSerif";font-style:normal;font-weight:700;src:url(fonts/KaTeX_SansSerif-Bold.woff2) format("woff2"),url(fonts/KaTeX_SansSerif-Bold.woff) format("woff"),url(fonts/KaTeX_SansSerif-Bold.ttf) format("truetype")}@font-face{font-family:"KaTeX_SansSerif";font-style:italic;font-weight:400;src:url(fonts/KaTeX_SansSerif-Italic.woff2) format("woff2"),url(fonts/KaTeX_SansSerif-Italic.woff) format("woff"),url(fonts/KaTeX_SansSerif-Italic.ttf) format("truetype")}@font-face{font-family:"KaTeX_SansSerif";font-style:normal;font-weight:400;src:url(fonts/KaTeX_SansSerif-Regular.woff2) format("woff2"),url(fonts/KaTeX_SansSerif-Regular.woff) format("woff"),url(fonts/KaTeX_SansSerif-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Script;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Script-Regular.woff2) format("woff2"),url(fonts/KaTeX_Script-Regular.woff) format("woff"),url(fonts/KaTeX_Script-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Size1;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Size1-Regular.woff2) format("woff2"),url(fonts/KaTeX_Size1-Regular.woff) format("woff"),url(fonts/KaTeX_Size1-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Size2;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Size2-Regular.woff2) format("woff2"),url(fonts/KaTeX_Size2-Regular.woff) format("woff"),url(fonts/KaTeX_Size2-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Size3;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Size3-Regular.woff2) format("woff2"),url(fonts/KaTeX_Size3-Regular.woff) format("woff"),url(fonts/KaTeX_Size3-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Size4;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Size4-Regular.woff2) format("woff2"),url(fonts/KaTeX_Size4-Regular.woff) format("woff"),url(fonts/KaTeX_Size4-Regular.ttf) format("truetype")}@font-face{font-family:KaTeX_Typewriter;font-style:normal;font-weight:400;src:url(fonts/KaTeX_Typewriter-Regular.woff2) format("woff2"),url(fonts/KaTeX_Typewriter-Regular.woff) format("woff"),url(fonts/KaTeX_Typewriter-Regular.ttf) format("truetype")}.katex{font:normal 1.21em KaTeX_Main,Times New Roman,serif;line-height:1.2;text-indent:0;text-rendering:auto}.katex *{-ms-high-contrast-adjust:none!important;border-color:currentColor}.katex .katex-version:after{content:"0.16.21"}.katex .katex-mathml{clip:rect(1px,1px,1px,1px);border:0;height:1px;overflow:hidden;padding:0;position:absolute;width:1px}.katex .katex-html>.newline{display:block}.katex .base{position:relative;white-space:nowrap;width:-webkit-min-content;width:-moz-min-content;width:min-content}.katex .base,.katex .strut{display:inline-block}.katex .textbf{font-weight:700}.katex .textit{font-style:italic}.katex .textrm{font-family:KaTeX_Main}.katex .textsf{font-family:KaTeX_SansSerif}.katex .texttt{font-family:KaTeX_Typewriter}.katex .mathnormal{font-family:KaTeX_Math;font-style:italic}.katex .mathit{font-family:KaTeX_Main;font-style:italic}.katex .mathrm{font-style:normal}.katex .mathbf{font-family:KaTeX_Main;font-weight:700}.katex .boldsymbol{font-family:KaTeX_Math;font-style:italic;font-weight:700}.katex .amsrm,.katex .mathbb,.katex .textbb{font-family:KaTeX_AMS}.katex .mathcal{font-family:KaTeX_Caligraphic}.katex .mathfrak,.katex .textfrak{font-family:KaTeX_Fraktur}.katex .mathboldfrak,.katex .textboldfrak{font-family:KaTeX_Fraktur;font-weight:700}.katex .mathtt{font-family:KaTeX_Typewriter}.katex .mathscr,.katex .textscr{font-family:KaTeX_Script}.katex .mathsf,.katex .textsf{font-family:KaTeX_SansSerif}.katex .mathboldsf,.katex .textboldsf{font-family:KaTeX_SansSerif;font-weight:700}.katex .mathitsf,.katex .mathsfit,.katex .textitsf{font-family:KaTeX_SansSerif;font-style:italic}.katex .mainrm{font-family:KaTeX_Main;font-style:normal}.katex .vlist-t{border-collapse:collapse;display:inline-table;table-layout:fixed}.katex .vlist-r{display:table-row}.katex .vlist{display:table-cell;position:relative;vertical-align:bottom}.katex .vlist>span{display:block;height:0;position:relative}.katex .vlist>span>span{display:inline-block}.katex .vlist>span>.pstrut{overflow:hidden;width:0}.katex .vlist-t2{margin-right:-2px}.katex .vlist-s{display:table-cell;font-size:1px;min-width:2px;vertical-align:bottom;width:2px}.katex .vbox{align-items:baseline;display:inline-flex;flex-direction:column}.katex .hbox{width:100%}.katex .hbox,.katex .thinbox{display:inline-flex;flex-direction:row}.katex .thinbox{max-width:0;width:0}.katex .msupsub{text-align:left}.katex .mfrac>span>span{text-align:center}.katex .mfrac .frac-line{border-bottom-style:solid;display:inline-block;width:100%}.katex .hdashline,.katex .hline,.katex .mfrac .frac-line,.katex .overline .overline-line,.katex .rule,.katex .underline .underline-line{min-height:1px}.katex .mspace{display:inline-block}.katex .clap,.katex .llap,.katex .rlap{position:relative;width:0}.katex .clap>.inner,.katex .llap>.inner,.katex .rlap>.inner{position:absolute}.katex .clap>.fix,.katex .llap>.fix,.katex .rlap>.fix{display:inline-block}.katex .llap>.inner{right:0}.katex .clap>.inner,.katex .rlap>.inner{left:0}.katex .clap>.inner>span{margin-left:-50%;margin-right:50%}.katex .rule{border:0 solid;display:inline-block;position:relative}.katex .hline,.katex .overline .overline-line,.katex .underline .underline-line{border-bottom-style:solid;display:inline-block;width:100%}.katex .hdashline{border-bottom-style:dashed;display:inline-block;width:100%}.katex .sqrt>.root{margin-left:.2777777778em;margin-right:-.5555555556em}.katex .fontsize-ensurer.reset-size1.size1,.katex .sizing.reset-size1.size1{font-size:1em}.katex .fontsize-ensurer.reset-size1.size2,.katex .sizing.reset-size1.size2{font-size:1.2em}.katex .fontsize-ensurer.reset-size1.size3,.katex .sizing.reset-size1.size3{font-size:1.4em}.katex .fontsize-ensurer.reset-size1.size4,.katex .sizing.reset-size1.size4{font-size:1.6em}.katex .fontsize-ensurer.reset-size1.size5,.katex .sizing.reset-size1.size5{font-size:1.8em}.katex .fontsize-ensurer.reset-size1.size6,.katex .sizing.reset-size1.size6{font-size:2em}.katex .fontsize-ensurer.reset-size1.size7,.katex .sizing.reset-size1.size7{font-size:2.4em}.katex .fontsize-ensurer.reset-size1.size8,.katex .sizing.reset-size1.size8{font-size:2.88em}.katex .fontsize-ensurer.reset-size1.size9,.katex .sizing.reset-size1.size9{font-size:3.456em}.katex .fontsize-ensurer.reset-size1.size10,.katex .sizing.reset-size1.size10{font-size:4.148em}.katex .fontsize-ensurer.reset-size1.size11,.katex .sizing.reset-size1.size11{font-size:4.976em}.katex .fontsize-ensurer.reset-size2.size1,.katex .sizing.reset-size2.size1{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size2.size2,.katex .sizing.reset-size2.size2{font-size:1em}.katex .fontsize-ensurer.reset-size2.size3,.katex .sizing.reset-size2.size3{font-size:1.1666666667em}.katex .fontsize-ensurer.reset-size2.size4,.katex .sizing.reset-size2.size4{font-size:1.3333333333em}.katex .fontsize-ensurer.reset-size2.size5,.katex .sizing.reset-size2.size5{font-size:1.5em}.katex .fontsize-ensurer.reset-size2.size6,.katex .sizing.reset-size2.size6{font-size:1.6666666667em}.katex .fontsize-ensurer.reset-size2.size7,.katex .sizing.reset-size2.size7{font-size:2em}.katex .fontsize-ensurer.reset-size2.size8,.katex .sizing.reset-size2.size8{font-size:2.4em}.katex .fontsize-ensurer.reset-size2.size9,.katex .sizing.reset-size2.size9{font-size:2.88em}.katex .fontsize-ensurer.reset-size2.size10,.katex .sizing.reset-size2.size10{font-size:3.4566666667em}.katex .fontsize-ensurer.reset-size2.size11,.katex .sizing.reset-size2.size11{font-size:4.1466666667em}.katex .fontsize-ensurer.reset-size3.size1,.katex .sizing.reset-size3.size1{font-size:.7142857143em}.katex .fontsize-ensurer.reset-size3.size2,.katex .sizing.reset-size3.size2{font-size:.8571428571em}.katex .fontsize-ensurer.reset-size3.size3,.katex .sizing.reset-size3.size3{font-size:1em}.katex .fontsize-ensurer.reset-size3.size4,.katex .sizing.reset-size3.size4{font-size:1.1428571429em}.katex .fontsize-ensurer.reset-size3.size5,.katex .sizing.reset-size3.size5{font-size:1.2857142857em}.katex .fontsize-ensurer.reset-size3.size6,.katex .sizing.reset-size3.size6{font-size:1.4285714286em}.katex .fontsize-ensurer.reset-size3.size7,.katex .sizing.reset-size3.size7{font-size:1.7142857143em}.katex .fontsize-ensurer.reset-size3.size8,.katex .sizing.reset-size3.size8{font-size:2.0571428571em}.katex .fontsize-ensurer.reset-size3.size9,.katex .sizing.reset-size3.size9{font-size:2.4685714286em}.katex .fontsize-ensurer.reset-size3.size10,.katex .sizing.reset-size3.size10{font-size:2.9628571429em}.katex .fontsize-ensurer.reset-size3.size11,.katex .sizing.reset-size3.size11{font-size:3.5542857143em}.katex .fontsize-ensurer.reset-size4.size1,.katex .sizing.reset-size4.size1{font-size:.625em}.katex .fontsize-ensurer.reset-size4.size2,.katex .sizing.reset-size4.size2{font-size:.75em}.katex .fontsize-ensurer.reset-size4.size3,.katex .sizing.reset-size4.size3{font-size:.875em}.katex .fontsize-ensurer.reset-size4.size4,.katex .sizing.reset-size4.size4{font-size:1em}.katex .fontsize-ensurer.reset-size4.size5,.katex .sizing.reset-size4.size5{font-size:1.125em}.katex .fontsize-ensurer.reset-size4.size6,.katex .sizing.reset-size4.size6{font-size:1.25em}.katex .fontsize-ensurer.reset-size4.size7,.katex .sizing.reset-size4.size7{font-size:1.5em}.katex .fontsize-ensurer.reset-size4.size8,.katex .sizing.reset-size4.size8{font-size:1.8em}.katex .fontsize-ensurer.reset-size4.size9,.katex .sizing.reset-size4.size9{font-size:2.16em}.katex .fontsize-ensurer.reset-size4.size10,.katex .sizing.reset-size4.size10{font-size:2.5925em}.katex .fontsize-ensurer.reset-size4.size11,.katex .sizing.reset-size4.size11{font-size:3.11em}.katex .fontsize-ensurer.reset-size5.size1,.katex .sizing.reset-size5.size1{font-size:.5555555556em}.katex .fontsize-ensurer.reset-size5.size2,.katex .sizing.reset-size5.size2{font-size:.6666666667em}.katex .fontsize-ensurer.reset-size5.size3,.katex .sizing.reset-size5.size3{font-size:.7777777778em}.katex .fontsize-ensurer.reset-size5.size4,.katex .sizing.reset-size5.size4{font-size:.8888888889em}.katex .fontsize-ensurer.reset-size5.size5,.katex .sizing.reset-size5.size5{font-size:1em}.katex .fontsize-ensurer.reset-size5.size6,.katex .sizing.reset-size5.size6{font-size:1.1111111111em}.katex .fontsize-ensurer.reset-size5.size7,.katex .sizing.reset-size5.size7{font-size:1.3333333333em}.katex .fontsize-ensurer.reset-size5.size8,.katex .sizing.reset-size5.size8{font-size:1.6em}.katex .fontsize-ensurer.reset-size5.size9,.katex .sizing.reset-size5.size9{font-size:1.92em}.katex .fontsize-ensurer.reset-size5.size10,.katex .sizing.reset-size5.size10{font-size:2.3044444444em}.katex .fontsize-ensurer.reset-size5.size11,.katex .sizing.reset-size5.size11{font-size:2.7644444444em}.katex .fontsize-ensurer.reset-size6.size1,.katex .sizing.reset-size6.size1{font-size:.5em}.katex .fontsize-ensurer.reset-size6.size2,.katex .sizing.reset-size6.size2{font-size:.6em}.katex .fontsize-ensurer.reset-size6.size3,.katex .sizing.reset-size6.size3{font-size:.7em}.katex .fontsize-ensurer.reset-size6.size4,.katex .sizing.reset-size6.size4{font-size:.8em}.katex .fontsize-ensurer.reset-size6.size5,.katex .sizing.reset-size6.size5{font-size:.9em}.katex .fontsize-ensurer.reset-size6.size6,.katex .sizing.reset-size6.size6{font-size:1em}.katex .fontsize-ensurer.reset-size6.size7,.katex .sizing.reset-size6.size7{font-size:1.2em}.katex .fontsize-ensurer.reset-size6.size8,.katex .sizing.reset-size6.size8{font-size:1.44em}.katex .fontsize-ensurer.reset-size6.size9,.katex .sizing.reset-size6.size9{font-size:1.728em}.katex .fontsize-ensurer.reset-size6.size10,.katex .sizing.reset-size6.size10{font-size:2.074em}.katex .fontsize-ensurer.reset-size6.size11,.katex .sizing.reset-size6.size11{font-size:2.488em}.katex .fontsize-ensurer.reset-size7.size1,.katex .sizing.reset-size7.size1{font-size:.4166666667em}.katex .fontsize-ensurer.reset-size7.size2,.katex .sizing.reset-size7.size2{font-size:.5em}.katex .fontsize-ensurer.reset-size7.size3,.katex .sizing.reset-size7.size3{font-size:.5833333333em}.katex .fontsize-ensurer.reset-size7.size4,.katex .sizing.reset-size7.size4{font-size:.6666666667em}.katex .fontsize-ensurer.reset-size7.size5,.katex .sizing.reset-size7.size5{font-size:.75em}.katex .fontsize-ensurer.reset-size7.size6,.katex .sizing.reset-size7.size6{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size7.size7,.katex .sizing.reset-size7.size7{font-size:1em}.katex .fontsize-ensurer.reset-size7.size8,.katex .sizing.reset-size7.size8{font-size:1.2em}.katex .fontsize-ensurer.reset-size7.size9,.katex .sizing.reset-size7.size9{font-size:1.44em}.katex .fontsize-ensurer.reset-size7.size10,.katex .sizing.reset-size7.size10{font-size:1.7283333333em}.katex .fontsize-ensurer.reset-size7.size11,.katex .sizing.reset-size7.size11{font-size:2.0733333333em}.katex .fontsize-ensurer.reset-size8.size1,.katex .sizing.reset-size8.size1{font-size:.3472222222em}.katex .fontsize-ensurer.reset-size8.size2,.katex .sizing.reset-size8.size2{font-size:.4166666667em}.katex .fontsize-ensurer.reset-size8.size3,.katex .sizing.reset-size8.size3{font-size:.4861111111em}.katex .fontsize-ensurer.reset-size8.size4,.katex .sizing.reset-size8.size4{font-size:.5555555556em}.katex .fontsize-ensurer.reset-size8.size5,.katex .sizing.reset-size8.size5{font-size:.625em}.katex .fontsize-ensurer.reset-size8.size6,.katex .sizing.reset-size8.size6{font-size:.6944444444em}.katex .fontsize-ensurer.reset-size8.size7,.katex .sizing.reset-size8.size7{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size8.size8,.katex .sizing.reset-size8.size8{font-size:1em}.katex .fontsize-ensurer.reset-size8.size9,.katex .sizing.reset-size8.size9{font-size:1.2em}.katex .fontsize-ensurer.reset-size8.size10,.katex .sizing.reset-size8.size10{font-size:1.4402777778em}.katex .fontsize-ensurer.reset-size8.size11,.katex .sizing.reset-size8.size11{font-size:1.7277777778em}.katex .fontsize-ensurer.reset-size9.size1,.katex .sizing.reset-size9.size1{font-size:.2893518519em}.katex .fontsize-ensurer.reset-size9.size2,.katex .sizing.reset-size9.size2{font-size:.3472222222em}.katex .fontsize-ensurer.reset-size9.size3,.katex .sizing.reset-size9.size3{font-size:.4050925926em}.katex .fontsize-ensurer.reset-size9.size4,.katex .sizing.reset-size9.size4{font-size:.462962963em}.katex .fontsize-ensurer.reset-size9.size5,.katex .sizing.reset-size9.size5{font-size:.5208333333em}.katex .fontsize-ensurer.reset-size9.size6,.katex .sizing.reset-size9.size6{font-size:.5787037037em}.katex .fontsize-ensurer.reset-size9.size7,.katex .sizing.reset-size9.size7{font-size:.6944444444em}.katex .fontsize-ensurer.reset-size9.size8,.katex .sizing.reset-size9.size8{font-size:.8333333333em}.katex .fontsize-ensurer.reset-size9.size9,.katex .sizing.reset-size9.size9{font-size:1em}.katex .fontsize-ensurer.reset-size9.size10,.katex .sizing.reset-size9.size10{font-size:1.2002314815em}.katex .fontsize-ensurer.reset-size9.size11,.katex .sizing.reset-size9.size11{font-size:1.4398148148em}.katex .fontsize-ensurer.reset-size10.size1,.katex .sizing.reset-size10.size1{font-size:.2410800386em}.katex .fontsize-ensurer.reset-size10.size2,.katex .sizing.reset-size10.size2{font-size:.2892960463em}.katex .fontsize-ensurer.reset-size10.size3,.katex .sizing.reset-size10.size3{font-size:.337512054em}.katex .fontsize-ensurer.reset-size10.size4,.katex .sizing.reset-size10.size4{font-size:.3857280617em}.katex .fontsize-ensurer.reset-size10.size5,.katex .sizing.reset-size10.size5{font-size:.4339440694em}.katex .fontsize-ensurer.reset-size10.size6,.katex .sizing.reset-size10.size6{font-size:.4821600771em}.katex .fontsize-ensurer.reset-size10.size7,.katex .sizing.reset-size10.size7{font-size:.5785920926em}.katex .fontsize-ensurer.reset-size10.size8,.katex .sizing.reset-size10.size8{font-size:.6943105111em}.katex .fontsize-ensurer.reset-size10.size9,.katex .sizing.reset-size10.size9{font-size:.8331726133em}.katex .fontsize-ensurer.reset-size10.size10,.katex .sizing.reset-size10.size10{font-size:1em}.katex .fontsize-ensurer.reset-size10.size11,.katex .sizing.reset-size10.size11{font-size:1.1996142719em}.katex .fontsize-ensurer.reset-size11.size1,.katex .sizing.reset-size11.size1{font-size:.2009646302em}.katex .fontsize-ensurer.reset-size11.size2,.katex .sizing.reset-size11.size2{font-size:.2411575563em}.katex .fontsize-ensurer.reset-size11.size3,.katex .sizing.reset-size11.size3{font-size:.2813504823em}.katex .fontsize-ensurer.reset-size11.size4,.katex .sizing.reset-size11.size4{font-size:.3215434084em}.katex .fontsize-ensurer.reset-size11.size5,.katex .sizing.reset-size11.size5{font-size:.3617363344em}.katex .fontsize-ensurer.reset-size11.size6,.katex .sizing.reset-size11.size6{font-size:.4019292605em}.katex .fontsize-ensurer.reset-size11.size7,.katex .sizing.reset-size11.size7{font-size:.4823151125em}.katex .fontsize-ensurer.reset-size11.size8,.katex .sizing.reset-size11.size8{font-size:.578778135em}.katex .fontsize-ensurer.reset-size11.size9,.katex .sizing.reset-size11.size9{font-size:.6945337621em}.katex .fontsize-ensurer.reset-size11.size10,.katex .sizing.reset-size11.size10{font-size:.8336012862em}.katex .fontsize-ensurer.reset-size11.size11,.katex .sizing.reset-size11.size11{font-size:1em}.katex .delimsizing.size1{font-family:KaTeX_Size1}.katex .delimsizing.size2{font-family:KaTeX_Size2}.katex .delimsizing.size3{font-family:KaTeX_Size3}.katex .delimsizing.size4{font-family:KaTeX_Size4}.katex .delimsizing.mult .delim-size1>span{font-family:KaTeX_Size1}.katex .delimsizing.mult .delim-size4>span{font-family:KaTeX_Size4}.katex .nulldelimiter{display:inline-block;width:.12em}.katex .delimcenter,.katex .op-symbol{position:relative}.katex .op-symbol.small-op{font-family:KaTeX_Size1}.katex .op-symbol.large-op{font-family:KaTeX_Size2}.katex .accent>.vlist-t,.katex .op-limits>.vlist-t{text-align:center}.katex .accent .accent-body{position:relative}.katex .accent .accent-body:not(.accent-full){width:0}.katex .overlay{display:block}.katex .mtable .vertical-separator{display:inline-block;min-width:1px}.katex .mtable .arraycolsep{display:inline-block}.katex .mtable .col-align-c>.vlist-t{text-align:center}.katex .mtable .col-align-l>.vlist-t{text-align:left}.katex .mtable .col-align-r>.vlist-t{text-align:right}.katex .svg-align{text-align:left}.katex svg{fill:currentColor;stroke:currentColor;fill-rule:nonzero;fill-opacity:1;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;display:block;height:inherit;position:absolute;width:100%}.katex svg path{stroke:none}.katex img{border-style:none;max-height:none;max-width:none;min-height:0;min-width:0}.katex .stretchy{display:block;overflow:hidden;position:relative;width:100%}.katex .stretchy:after,.katex .stretchy:before{content:""}.katex .hide-tail{overflow:hidden;position:relative;width:100%}.katex .halfarrow-left{left:0;overflow:hidden;position:absolute;width:50.2%}.katex .halfarrow-right{overflow:hidden;position:absolute;right:0;width:50.2%}.katex .brace-left{left:0;overflow:hidden;position:absolute;width:25.1%}.katex .brace-center{left:25%;overflow:hidden;position:absolute;width:50%}.katex .brace-right{overflow:hidden;position:absolute;right:0;width:25.1%}.katex .x-arrow-pad{padding:0 .5em}.katex .cd-arrow-pad{padding:0 .55556em 0 .27778em}.katex .mover,.katex .munder,.katex .x-arrow{text-align:center}.katex .boxpad{padding:0 .3em}.katex .fbox,.katex .fcolorbox{border:.04em solid;box-sizing:border-box}.katex .cancel-pad{padding:0 .2em}.katex .cancel-lap{margin-left:-.2em;margin-right:-.2em}.katex .sout{border-bottom-style:solid;border-bottom-width:.08em}.katex .angl{border-right:.049em solid;border-top:.049em solid;box-sizing:border-box;margin-right:.03889em}.katex .anglpad{padding:0 .03889em}.katex .eqn-num:before{content:"(" counter(katexEqnNo) ")";counter-increment:katexEqnNo}.katex .mml-eqn-num:before{content:"(" counter(mmlEqnNo) ")";counter-increment:mmlEqnNo}.katex .mtr-glue{width:50%}.katex .cd-vert-arrow{display:inline-block;position:relative}.katex .cd-label-left{display:inline-block;position:absolute;right:calc(50% + .3em);text-align:left}.katex .cd-label-right{display:inline-block;left:calc(50% + .3em);position:absolute;text-align:right}.katex-display{display:block;margin:1em 0;text-align:center}.katex-display>.katex{display:block;text-align:center;white-space:nowrap}.katex-display>.katex>.katex-html{display:block;position:relative}.katex-display>.katex>.katex-html>.tag{position:absolute;right:0}.katex-display.leqno>.katex>.katex-html>.tag{left:0;right:auto}.katex-display.fleqn>.katex{padding-left:2em;text-align:left}body{counter-reset:katexEqnNo mmlEqnNo}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/katex.min.js b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/katex.min.js
new file mode 100644
index 0000000000000000000000000000000000000000..2edae98a59a75bbf1c205acabbd45aacb973bb91
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/katex.min.js
@@ -0,0 +1 @@
+!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t():"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?exports.katex=t():e.katex=t()}("undefined"!=typeof self?self:this,(function(){return function(){"use strict";var e={d:function(t,r){for(var n in r)e.o(r,n)&&!e.o(t,n)&&Object.defineProperty(t,n,{enumerable:!0,get:r[n]})},o:function(e,t){return Object.prototype.hasOwnProperty.call(e,t)}},t={};e.d(t,{default:function(){return Wn}});class r{constructor(e,t){this.name=void 0,this.position=void 0,this.length=void 0,this.rawMessage=void 0;let n,o,s="KaTeX parse error: "+e;const i=t&&t.loc;if(i&&i.start<=i.end){const e=i.lexer.input;n=i.start,o=i.end,n===e.length?s+=" at end of input: ":s+=" at position "+(n+1)+": ";const t=e.slice(n,o).replace(/[^]/g,"$&\u0332");let r,a;r=n>15?"\u2026"+e.slice(n-15,n):e.slice(0,n),a=o+15<e.length?e.slice(o,o+15)+"\u2026":e.slice(o),s+=r+t+a}const a=new Error(s);return a.name="ParseError",a.__proto__=r.prototype,a.position=n,null!=n&&null!=o&&(a.length=o-n),a.rawMessage=e,a}}r.prototype.__proto__=Error.prototype;var n=r;const o=/([A-Z])/g,s={"&":"&amp;",">":"&gt;","<":"&lt;",'"':"&quot;","'":"&#x27;"},i=/[&><"']/g;const a=function(e){return"ordgroup"===e.type||"color"===e.type?1===e.body.length?a(e.body[0]):e:"font"===e.type?a(e.body):e};var l={contains:function(e,t){return-1!==e.indexOf(t)},deflt:function(e,t){return void 0===e?t:e},escape:function(e){return String(e).replace(i,(e=>s[e]))},hyphenate:function(e){return e.replace(o,"-$1").toLowerCase()},getBaseElem:a,isCharacterBox:function(e){const t=a(e);return"mathord"===t.type||"textord"===t.type||"atom"===t.type},protocolFromUrl:function(e){const t=/^[\x00-\x20]*([^\\/#?]*?)(:|&#0*58|&#x0*3a|&colon)/i.exec(e);return t?":"!==t[2]?null:/^[a-zA-Z][a-zA-Z0-9+\-.]*$/.test(t[1])?t[1].toLowerCase():null:"_relative"}};const h={displayMode:{type:"boolean",description:"Render math in display mode, which puts the math in display style (so \\int and \\sum are large, for example), and centers the math on the page on its own line.",cli:"-d, --display-mode"},output:{type:{enum:["htmlAndMathml","html","mathml"]},description:"Determines the markup language of the output.",cli:"-F, --format <type>"},leqno:{type:"boolean",description:"Render display math in leqno style (left-justified tags)."},fleqn:{type:"boolean",description:"Render display math flush left."},throwOnError:{type:"boolean",default:!0,cli:"-t, --no-throw-on-error",cliDescription:"Render errors (in the color given by --error-color) instead of throwing a ParseError exception when encountering an error."},errorColor:{type:"string",default:"#cc0000",cli:"-c, --error-color <color>",cliDescription:"A color string given in the format 'rgb' or 'rrggbb' (no #). This option determines the color of errors rendered by the -t option.",cliProcessor:e=>"#"+e},macros:{type:"object",cli:"-m, --macro <def>",cliDescription:"Define custom macro of the form '\\foo:expansion' (use multiple -m arguments for multiple macros).",cliDefault:[],cliProcessor:(e,t)=>(t.push(e),t)},minRuleThickness:{type:"number",description:"Specifies a minimum thickness, in ems, for fraction lines, `\\sqrt` top lines, `{array}` vertical lines, `\\hline`, `\\hdashline`, `\\underline`, `\\overline`, and the borders of `\\fbox`, `\\boxed`, and `\\fcolorbox`.",processor:e=>Math.max(0,e),cli:"--min-rule-thickness <size>",cliProcessor:parseFloat},colorIsTextColor:{type:"boolean",description:"Makes \\color behave like LaTeX's 2-argument \\textcolor, instead of LaTeX's one-argument \\color mode change.",cli:"-b, --color-is-text-color"},strict:{type:[{enum:["warn","ignore","error"]},"boolean","function"],description:"Turn on strict / LaTeX faithfulness mode, which throws an error if the input uses features that are not supported by LaTeX.",cli:"-S, --strict",cliDefault:!1},trust:{type:["boolean","function"],description:"Trust the input, enabling all HTML features such as \\url.",cli:"-T, --trust"},maxSize:{type:"number",default:1/0,description:"If non-zero, all user-specified sizes, e.g. in \\rule{500em}{500em}, will be capped to maxSize ems. Otherwise, elements and spaces can be arbitrarily large",processor:e=>Math.max(0,e),cli:"-s, --max-size <n>",cliProcessor:parseInt},maxExpand:{type:"number",default:1e3,description:"Limit the number of macro expansions to the specified number, to prevent e.g. infinite macro loops. If set to Infinity, the macro expander will try to fully expand as in LaTeX.",processor:e=>Math.max(0,e),cli:"-e, --max-expand <n>",cliProcessor:e=>"Infinity"===e?1/0:parseInt(e)},globalGroup:{type:"boolean",cli:!1}};function c(e){if(e.default)return e.default;const t=e.type,r=Array.isArray(t)?t[0]:t;if("string"!=typeof r)return r.enum[0];switch(r){case"boolean":return!1;case"string":return"";case"number":return 0;case"object":return{}}}class m{constructor(e){this.displayMode=void 0,this.output=void 0,this.leqno=void 0,this.fleqn=void 0,this.throwOnError=void 0,this.errorColor=void 0,this.macros=void 0,this.minRuleThickness=void 0,this.colorIsTextColor=void 0,this.strict=void 0,this.trust=void 0,this.maxSize=void 0,this.maxExpand=void 0,this.globalGroup=void 0,e=e||{};for(const t in h)if(h.hasOwnProperty(t)){const r=h[t];this[t]=void 0!==e[t]?r.processor?r.processor(e[t]):e[t]:c(r)}}reportNonstrict(e,t,r){let o=this.strict;if("function"==typeof o&&(o=o(e,t,r)),o&&"ignore"!==o){if(!0===o||"error"===o)throw new n("LaTeX-incompatible input and strict mode is set to 'error': "+t+" ["+e+"]",r);"warn"===o?"undefined"!=typeof console&&console.warn("LaTeX-incompatible input and strict mode is set to 'warn': "+t+" ["+e+"]"):"undefined"!=typeof console&&console.warn("LaTeX-incompatible input and strict mode is set to unrecognized '"+o+"': "+t+" ["+e+"]")}}useStrictBehavior(e,t,r){let n=this.strict;if("function"==typeof n)try{n=n(e,t,r)}catch(e){n="error"}return!(!n||"ignore"===n)&&(!0===n||"error"===n||("warn"===n?("undefined"!=typeof console&&console.warn("LaTeX-incompatible input and strict mode is set to 'warn': "+t+" ["+e+"]"),!1):("undefined"!=typeof console&&console.warn("LaTeX-incompatible input and strict mode is set to unrecognized '"+n+"': "+t+" ["+e+"]"),!1)))}isTrusted(e){if(e.url&&!e.protocol){const t=l.protocolFromUrl(e.url);if(null==t)return!1;e.protocol=t}const t="function"==typeof this.trust?this.trust(e):this.trust;return Boolean(t)}}class p{constructor(e,t,r){this.id=void 0,this.size=void 0,this.cramped=void 0,this.id=e,this.size=t,this.cramped=r}sup(){return u[d[this.id]]}sub(){return u[g[this.id]]}fracNum(){return u[f[this.id]]}fracDen(){return u[b[this.id]]}cramp(){return u[y[this.id]]}text(){return u[x[this.id]]}isTight(){return this.size>=2}}const u=[new p(0,0,!1),new p(1,0,!0),new p(2,1,!1),new p(3,1,!0),new p(4,2,!1),new p(5,2,!0),new p(6,3,!1),new p(7,3,!0)],d=[4,5,4,5,6,7,6,7],g=[5,5,5,5,7,7,7,7],f=[2,3,4,5,6,7,6,7],b=[3,3,5,5,7,7,7,7],y=[1,1,3,3,5,5,7,7],x=[0,1,2,3,2,3,2,3];var w={DISPLAY:u[0],TEXT:u[2],SCRIPT:u[4],SCRIPTSCRIPT:u[6]};const v=[{name:"latin",blocks:[[256,591],[768,879]]},{name:"cyrillic",blocks:[[1024,1279]]},{name:"armenian",blocks:[[1328,1423]]},{name:"brahmic",blocks:[[2304,4255]]},{name:"georgian",blocks:[[4256,4351]]},{name:"cjk",blocks:[[12288,12543],[19968,40879],[65280,65376]]},{name:"hangul",blocks:[[44032,55215]]}];const k=[];function S(e){for(let t=0;t<k.length;t+=2)if(e>=k[t]&&e<=k[t+1])return!0;return!1}v.forEach((e=>e.blocks.forEach((e=>k.push(...e)))));const M=80,z={doubleleftarrow:"M262 157\nl10-10c34-36 62.7-77 86-123 3.3-8 5-13.3 5-16 0-5.3-6.7-8-20-8-7.3\n 0-12.2.5-14.5 1.5-2.3 1-4.8 4.5-7.5 10.5-49.3 97.3-121.7 169.3-217 216-28\n 14-57.3 25-88 33-6.7 2-11 3.8-13 5.5-2 1.7-3 4.2-3 7.5s1 5.8 3 7.5\nc2 1.7 6.3 3.5 13 5.5 68 17.3 128.2 47.8 180.5 91.5 52.3 43.7 93.8 96.2 124.5\n 157.5 9.3 8 15.3 12.3 18 13h6c12-.7 18-4 18-10 0-2-1.7-7-5-15-23.3-46-52-87\n-86-123l-10-10h399738v-40H218c328 0 0 0 0 0l-10-8c-26.7-20-65.7-43-117-69 2.7\n-2 6-3.7 10-5 36.7-16 72.3-37.3 107-64l10-8h399782v-40z\nm8 0v40h399730v-40zm0 194v40h399730v-40z",doublerightarrow:"M399738 392l\n-10 10c-34 36-62.7 77-86 123-3.3 8-5 13.3-5 16 0 5.3 6.7 8 20 8 7.3 0 12.2-.5\n 14.5-1.5 2.3-1 4.8-4.5 7.5-10.5 49.3-97.3 121.7-169.3 217-216 28-14 57.3-25 88\n-33 6.7-2 11-3.8 13-5.5 2-1.7 3-4.2 3-7.5s-1-5.8-3-7.5c-2-1.7-6.3-3.5-13-5.5-68\n-17.3-128.2-47.8-180.5-91.5-52.3-43.7-93.8-96.2-124.5-157.5-9.3-8-15.3-12.3-18\n-13h-6c-12 .7-18 4-18 10 0 2 1.7 7 5 15 23.3 46 52 87 86 123l10 10H0v40h399782\nc-328 0 0 0 0 0l10 8c26.7 20 65.7 43 117 69-2.7 2-6 3.7-10 5-36.7 16-72.3 37.3\n-107 64l-10 8H0v40zM0 157v40h399730v-40zm0 194v40h399730v-40z",leftarrow:"M400000 241H110l3-3c68.7-52.7 113.7-120\n 135-202 4-14.7 6-23 6-25 0-7.3-7-11-21-11-8 0-13.2.8-15.5 2.5-2.3 1.7-4.2 5.8\n-5.5 12.5-1.3 4.7-2.7 10.3-4 17-12 48.7-34.8 92-68.5 130S65.3 228.3 18 247\nc-10 4-16 7.7-18 11 0 8.7 6 14.3 18 17 47.3 18.7 87.8 47 121.5 85S196 441.3 208\n 490c.7 2 1.3 5 2 9s1.2 6.7 1.5 8c.3 1.3 1 3.3 2 6s2.2 4.5 3.5 5.5c1.3 1 3.3\n 1.8 6 2.5s6 1 10 1c14 0 21-3.7 21-11 0-2-2-10.3-6-25-20-79.3-65-146.7-135-202\n l-3-3h399890zM100 241v40h399900v-40z",leftbrace:"M6 548l-6-6v-35l6-11c56-104 135.3-181.3 238-232 57.3-28.7 117\n-45 179-50h399577v120H403c-43.3 7-81 15-113 26-100.7 33-179.7 91-237 174-2.7\n 5-6 9-10 13-.7 1-7.3 1-20 1H6z",leftbraceunder:"M0 6l6-6h17c12.688 0 19.313.3 20 1 4 4 7.313 8.3 10 13\n 35.313 51.3 80.813 93.8 136.5 127.5 55.688 33.7 117.188 55.8 184.5 66.5.688\n 0 2 .3 4 1 18.688 2.7 76 4.3 172 5h399450v120H429l-6-1c-124.688-8-235-61.7\n-331-161C60.687 138.7 32.312 99.3 7 54L0 41V6z",leftgroup:"M400000 80\nH435C64 80 168.3 229.4 21 260c-5.9 1.2-18 0-18 0-2 0-3-1-3-3v-38C76 61 257 0\n 435 0h399565z",leftgroupunder:"M400000 262\nH435C64 262 168.3 112.6 21 82c-5.9-1.2-18 0-18 0-2 0-3 1-3 3v38c76 158 257 219\n 435 219h399565z",leftharpoon:"M0 267c.7 5.3 3 10 7 14h399993v-40H93c3.3\n-3.3 10.2-9.5 20.5-18.5s17.8-15.8 22.5-20.5c50.7-52 88-110.3 112-175 4-11.3 5\n-18.3 3-21-1.3-4-7.3-6-18-6-8 0-13 .7-15 2s-4.7 6.7-8 16c-42 98.7-107.3 174.7\n-196 228-6.7 4.7-10.7 8-12 10-1.3 2-2 5.7-2 11zm100-26v40h399900v-40z",leftharpoonplus:"M0 267c.7 5.3 3 10 7 14h399993v-40H93c3.3-3.3 10.2-9.5\n 20.5-18.5s17.8-15.8 22.5-20.5c50.7-52 88-110.3 112-175 4-11.3 5-18.3 3-21-1.3\n-4-7.3-6-18-6-8 0-13 .7-15 2s-4.7 6.7-8 16c-42 98.7-107.3 174.7-196 228-6.7 4.7\n-10.7 8-12 10-1.3 2-2 5.7-2 11zm100-26v40h399900v-40zM0 435v40h400000v-40z\nm0 0v40h400000v-40z",leftharpoondown:"M7 241c-4 4-6.333 8.667-7 14 0 5.333.667 9 2 11s5.333\n 5.333 12 10c90.667 54 156 130 196 228 3.333 10.667 6.333 16.333 9 17 2 .667 5\n 1 9 1h5c10.667 0 16.667-2 18-6 2-2.667 1-9.667-3-21-32-87.333-82.667-157.667\n-152-211l-3-3h399907v-40zM93 281 H400000 v-40L7 241z",leftharpoondownplus:"M7 435c-4 4-6.3 8.7-7 14 0 5.3.7 9 2 11s5.3 5.3 12\n 10c90.7 54 156 130 196 228 3.3 10.7 6.3 16.3 9 17 2 .7 5 1 9 1h5c10.7 0 16.7\n-2 18-6 2-2.7 1-9.7-3-21-32-87.3-82.7-157.7-152-211l-3-3h399907v-40H7zm93 0\nv40h399900v-40zM0 241v40h399900v-40zm0 0v40h399900v-40z",lefthook:"M400000 281 H103s-33-11.2-61-33.5S0 197.3 0 164s14.2-61.2 42.5\n-83.5C70.8 58.2 104 47 142 47 c16.7 0 25 6.7 25 20 0 12-8.7 18.7-26 20-40 3.3\n-68.7 15.7-86 37-10 12-15 25.3-15 40 0 22.7 9.8 40.7 29.5 54 19.7 13.3 43.5 21\n 71.5 23h399859zM103 281v-40h399897v40z",leftlinesegment:"M40 281 V428 H0 V94 H40 V241 H400000 v40z\nM40 281 V428 H0 V94 H40 V241 H400000 v40z",leftmapsto:"M40 281 V448H0V74H40V241H400000v40z\nM40 281 V448H0V74H40V241H400000v40z",leftToFrom:"M0 147h400000v40H0zm0 214c68 40 115.7 95.7 143 167h22c15.3 0 23\n-.3 23-1 0-1.3-5.3-13.7-16-37-18-35.3-41.3-69-70-101l-7-8h399905v-40H95l7-8\nc28.7-32 52-65.7 70-101 10.7-23.3 16-35.7 16-37 0-.7-7.7-1-23-1h-22C115.7 265.3\n 68 321 0 361zm0-174v-40h399900v40zm100 154v40h399900v-40z",longequal:"M0 50 h400000 v40H0z m0 194h40000v40H0z\nM0 50 h400000 v40H0z m0 194h40000v40H0z",midbrace:"M200428 334\nc-100.7-8.3-195.3-44-280-108-55.3-42-101.7-93-139-153l-9-14c-2.7 4-5.7 8.7-9 14\n-53.3 86.7-123.7 153-211 199-66.7 36-137.3 56.3-212 62H0V214h199568c178.3-11.7\n 311.7-78.3 403-201 6-8 9.7-12 11-12 .7-.7 6.7-1 18-1s17.3.3 18 1c1.3 0 5 4 11\n 12 44.7 59.3 101.3 106.3 170 141s145.3 54.3 229 60h199572v120z",midbraceunder:"M199572 214\nc100.7 8.3 195.3 44 280 108 55.3 42 101.7 93 139 153l9 14c2.7-4 5.7-8.7 9-14\n 53.3-86.7 123.7-153 211-199 66.7-36 137.3-56.3 212-62h199568v120H200432c-178.3\n 11.7-311.7 78.3-403 201-6 8-9.7 12-11 12-.7.7-6.7 1-18 1s-17.3-.3-18-1c-1.3 0\n-5-4-11-12-44.7-59.3-101.3-106.3-170-141s-145.3-54.3-229-60H0V214z",oiintSize1:"M512.6 71.6c272.6 0 320.3 106.8 320.3 178.2 0 70.8-47.7 177.6\n-320.3 177.6S193.1 320.6 193.1 249.8c0-71.4 46.9-178.2 319.5-178.2z\nm368.1 178.2c0-86.4-60.9-215.4-368.1-215.4-306.4 0-367.3 129-367.3 215.4 0 85.8\n60.9 214.8 367.3 214.8 307.2 0 368.1-129 368.1-214.8z",oiintSize2:"M757.8 100.1c384.7 0 451.1 137.6 451.1 230 0 91.3-66.4 228.8\n-451.1 228.8-386.3 0-452.7-137.5-452.7-228.8 0-92.4 66.4-230 452.7-230z\nm502.4 230c0-111.2-82.4-277.2-502.4-277.2s-504 166-504 277.2\nc0 110 84 276 504 276s502.4-166 502.4-276z",oiiintSize1:"M681.4 71.6c408.9 0 480.5 106.8 480.5 178.2 0 70.8-71.6 177.6\n-480.5 177.6S202.1 320.6 202.1 249.8c0-71.4 70.5-178.2 479.3-178.2z\nm525.8 178.2c0-86.4-86.8-215.4-525.7-215.4-437.9 0-524.7 129-524.7 215.4 0\n85.8 86.8 214.8 524.7 214.8 438.9 0 525.7-129 525.7-214.8z",oiiintSize2:"M1021.2 53c603.6 0 707.8 165.8 707.8 277.2 0 110-104.2 275.8\n-707.8 275.8-606 0-710.2-165.8-710.2-275.8C311 218.8 415.2 53 1021.2 53z\nm770.4 277.1c0-131.2-126.4-327.6-770.5-327.6S248.4 198.9 248.4 330.1\nc0 130 128.8 326.4 772.7 326.4s770.5-196.4 770.5-326.4z",rightarrow:"M0 241v40h399891c-47.3 35.3-84 78-110 128\n-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20\n 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7\n 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85\n-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5\n-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67\n 151.7 139 205zm0 0v40h399900v-40z",rightbrace:"M400000 542l\n-6 6h-17c-12.7 0-19.3-.3-20-1-4-4-7.3-8.3-10-13-35.3-51.3-80.8-93.8-136.5-127.5\ns-117.2-55.8-184.5-66.5c-.7 0-2-.3-4-1-18.7-2.7-76-4.3-172-5H0V214h399571l6 1\nc124.7 8 235 61.7 331 161 31.3 33.3 59.7 72.7 85 118l7 13v35z",rightbraceunder:"M399994 0l6 6v35l-6 11c-56 104-135.3 181.3-238 232-57.3\n 28.7-117 45-179 50H-300V214h399897c43.3-7 81-15 113-26 100.7-33 179.7-91 237\n-174 2.7-5 6-9 10-13 .7-1 7.3-1 20-1h17z",rightgroup:"M0 80h399565c371 0 266.7 149.4 414 180 5.9 1.2 18 0 18 0 2 0\n 3-1 3-3v-38c-76-158-257-219-435-219H0z",rightgroupunder:"M0 262h399565c371 0 266.7-149.4 414-180 5.9-1.2 18 0 18\n 0 2 0 3 1 3 3v38c-76 158-257 219-435 219H0z",rightharpoon:"M0 241v40h399993c4.7-4.7 7-9.3 7-14 0-9.3\n-3.7-15.3-11-18-92.7-56.7-159-133.7-199-231-3.3-9.3-6-14.7-8-16-2-1.3-7-2-15-2\n-10.7 0-16.7 2-18 6-2 2.7-1 9.7 3 21 15.3 42 36.7 81.8 64 119.5 27.3 37.7 58\n 69.2 92 94.5zm0 0v40h399900v-40z",rightharpoonplus:"M0 241v40h399993c4.7-4.7 7-9.3 7-14 0-9.3-3.7-15.3-11\n-18-92.7-56.7-159-133.7-199-231-3.3-9.3-6-14.7-8-16-2-1.3-7-2-15-2-10.7 0-16.7\n 2-18 6-2 2.7-1 9.7 3 21 15.3 42 36.7 81.8 64 119.5 27.3 37.7 58 69.2 92 94.5z\nm0 0v40h399900v-40z m100 194v40h399900v-40zm0 0v40h399900v-40z",rightharpoondown:"M399747 511c0 7.3 6.7 11 20 11 8 0 13-.8 15-2.5s4.7-6.8\n 8-15.5c40-94 99.3-166.3 178-217 13.3-8 20.3-12.3 21-13 5.3-3.3 8.5-5.8 9.5\n-7.5 1-1.7 1.5-5.2 1.5-10.5s-2.3-10.3-7-15H0v40h399908c-34 25.3-64.7 57-92 95\n-27.3 38-48.7 77.7-64 119-3.3 8.7-5 14-5 16zM0 241v40h399900v-40z",rightharpoondownplus:"M399747 705c0 7.3 6.7 11 20 11 8 0 13-.8\n 15-2.5s4.7-6.8 8-15.5c40-94 99.3-166.3 178-217 13.3-8 20.3-12.3 21-13 5.3-3.3\n 8.5-5.8 9.5-7.5 1-1.7 1.5-5.2 1.5-10.5s-2.3-10.3-7-15H0v40h399908c-34 25.3\n-64.7 57-92 95-27.3 38-48.7 77.7-64 119-3.3 8.7-5 14-5 16zM0 435v40h399900v-40z\nm0-194v40h400000v-40zm0 0v40h400000v-40z",righthook:"M399859 241c-764 0 0 0 0 0 40-3.3 68.7-15.7 86-37 10-12 15-25.3\n 15-40 0-22.7-9.8-40.7-29.5-54-19.7-13.3-43.5-21-71.5-23-17.3-1.3-26-8-26-20 0\n-13.3 8.7-20 26-20 38 0 71 11.2 99 33.5 0 0 7 5.6 21 16.7 14 11.2 21 33.5 21\n 66.8s-14 61.2-42 83.5c-28 22.3-61 33.5-99 33.5L0 241z M0 281v-40h399859v40z",rightlinesegment:"M399960 241 V94 h40 V428 h-40 V281 H0 v-40z\nM399960 241 V94 h40 V428 h-40 V281 H0 v-40z",rightToFrom:"M400000 167c-70.7-42-118-97.7-142-167h-23c-15.3 0-23 .3-23\n 1 0 1.3 5.3 13.7 16 37 18 35.3 41.3 69 70 101l7 8H0v40h399905l-7 8c-28.7 32\n-52 65.7-70 101-10.7 23.3-16 35.7-16 37 0 .7 7.7 1 23 1h23c24-69.3 71.3-125 142\n-167z M100 147v40h399900v-40zM0 341v40h399900v-40z",twoheadleftarrow:"M0 167c68 40\n 115.7 95.7 143 167h22c15.3 0 23-.3 23-1 0-1.3-5.3-13.7-16-37-18-35.3-41.3-69\n-70-101l-7-8h125l9 7c50.7 39.3 85 86 103 140h46c0-4.7-6.3-18.7-19-42-18-35.3\n-40-67.3-66-96l-9-9h399716v-40H284l9-9c26-28.7 48-60.7 66-96 12.7-23.333 19\n-37.333 19-42h-46c-18 54-52.3 100.7-103 140l-9 7H95l7-8c28.7-32 52-65.7 70-101\n 10.7-23.333 16-35.7 16-37 0-.7-7.7-1-23-1h-22C115.7 71.3 68 127 0 167z",twoheadrightarrow:"M400000 167\nc-68-40-115.7-95.7-143-167h-22c-15.3 0-23 .3-23 1 0 1.3 5.3 13.7 16 37 18 35.3\n 41.3 69 70 101l7 8h-125l-9-7c-50.7-39.3-85-86-103-140h-46c0 4.7 6.3 18.7 19 42\n 18 35.3 40 67.3 66 96l9 9H0v40h399716l-9 9c-26 28.7-48 60.7-66 96-12.7 23.333\n-19 37.333-19 42h46c18-54 52.3-100.7 103-140l9-7h125l-7 8c-28.7 32-52 65.7-70\n 101-10.7 23.333-16 35.7-16 37 0 .7 7.7 1 23 1h22c27.3-71.3 75-127 143-167z",tilde1:"M200 55.538c-77 0-168 73.953-177 73.953-3 0-7\n-2.175-9-5.437L2 97c-1-2-2-4-2-6 0-4 2-7 5-9l20-12C116 12 171 0 207 0c86 0\n 114 68 191 68 78 0 168-68 177-68 4 0 7 2 9 5l12 19c1 2.175 2 4.35 2 6.525 0\n 4.35-2 7.613-5 9.788l-19 13.05c-92 63.077-116.937 75.308-183 76.128\n-68.267.847-113-73.952-191-73.952z",tilde2:"M344 55.266c-142 0-300.638 81.316-311.5 86.418\n-8.01 3.762-22.5 10.91-23.5 5.562L1 120c-1-2-1-3-1-4 0-5 3-9 8-10l18.4-9C160.9\n 31.9 283 0 358 0c148 0 188 122 331 122s314-97 326-97c4 0 8 2 10 7l7 21.114\nc1 2.14 1 3.21 1 4.28 0 5.347-3 9.626-7 10.696l-22.3 12.622C852.6 158.372 751\n 181.476 676 181.476c-149 0-189-126.21-332-126.21z",tilde3:"M786 59C457 59 32 175.242 13 175.242c-6 0-10-3.457\n-11-10.37L.15 138c-1-7 3-12 10-13l19.2-6.4C378.4 40.7 634.3 0 804.3 0c337 0\n 411.8 157 746.8 157 328 0 754-112 773-112 5 0 10 3 11 9l1 14.075c1 8.066-.697\n 16.595-6.697 17.492l-21.052 7.31c-367.9 98.146-609.15 122.696-778.15 122.696\n -338 0-409-156.573-744-156.573z",tilde4:"M786 58C457 58 32 177.487 13 177.487c-6 0-10-3.345\n-11-10.035L.15 143c-1-7 3-12 10-13l22-6.7C381.2 35 637.15 0 807.15 0c337 0 409\n 177 744 177 328 0 754-127 773-127 5 0 10 3 11 9l1 14.794c1 7.805-3 13.38-9\n 14.495l-20.7 5.574c-366.85 99.79-607.3 139.372-776.3 139.372-338 0-409\n -175.236-744-175.236z",vec:"M377 20c0-5.333 1.833-10 5.5-14S391 0 397 0c4.667 0 8.667 1.667 12 5\n3.333 2.667 6.667 9 10 19 6.667 24.667 20.333 43.667 41 57 7.333 4.667 11\n10.667 11 18 0 6-1 10-3 12s-6.667 5-14 9c-28.667 14.667-53.667 35.667-75 63\n-1.333 1.333-3.167 3.5-5.5 6.5s-4 4.833-5 5.5c-1 .667-2.5 1.333-4.5 2s-4.333 1\n-7 1c-4.667 0-9.167-1.833-13.5-5.5S337 184 337 178c0-12.667 15.667-32.333 47-59\nH213l-171-1c-8.667-6-13-12.333-13-19 0-4.667 4.333-11.333 13-20h359\nc-16-25.333-24-45-24-59z",widehat1:"M529 0h5l519 115c5 1 9 5 9 10 0 1-1 2-1 3l-4 22\nc-1 5-5 9-11 9h-2L532 67 19 159h-2c-5 0-9-4-11-9l-5-22c-1-6 2-12 8-13z",widehat2:"M1181 0h2l1171 176c6 0 10 5 10 11l-2 23c-1 6-5 10\n-11 10h-1L1182 67 15 220h-1c-6 0-10-4-11-10l-2-23c-1-6 4-11 10-11z",widehat3:"M1181 0h2l1171 236c6 0 10 5 10 11l-2 23c-1 6-5 10\n-11 10h-1L1182 67 15 280h-1c-6 0-10-4-11-10l-2-23c-1-6 4-11 10-11z",widehat4:"M1181 0h2l1171 296c6 0 10 5 10 11l-2 23c-1 6-5 10\n-11 10h-1L1182 67 15 340h-1c-6 0-10-4-11-10l-2-23c-1-6 4-11 10-11z",widecheck1:"M529,159h5l519,-115c5,-1,9,-5,9,-10c0,-1,-1,-2,-1,-3l-4,-22c-1,\n-5,-5,-9,-11,-9h-2l-512,92l-513,-92h-2c-5,0,-9,4,-11,9l-5,22c-1,6,2,12,8,13z",widecheck2:"M1181,220h2l1171,-176c6,0,10,-5,10,-11l-2,-23c-1,-6,-5,-10,\n-11,-10h-1l-1168,153l-1167,-153h-1c-6,0,-10,4,-11,10l-2,23c-1,6,4,11,10,11z",widecheck3:"M1181,280h2l1171,-236c6,0,10,-5,10,-11l-2,-23c-1,-6,-5,-10,\n-11,-10h-1l-1168,213l-1167,-213h-1c-6,0,-10,4,-11,10l-2,23c-1,6,4,11,10,11z",widecheck4:"M1181,340h2l1171,-296c6,0,10,-5,10,-11l-2,-23c-1,-6,-5,-10,\n-11,-10h-1l-1168,273l-1167,-273h-1c-6,0,-10,4,-11,10l-2,23c-1,6,4,11,10,11z",baraboveleftarrow:"M400000 620h-399890l3 -3c68.7 -52.7 113.7 -120 135 -202\nc4 -14.7 6 -23 6 -25c0 -7.3 -7 -11 -21 -11c-8 0 -13.2 0.8 -15.5 2.5\nc-2.3 1.7 -4.2 5.8 -5.5 12.5c-1.3 4.7 -2.7 10.3 -4 17c-12 48.7 -34.8 92 -68.5 130\ns-74.2 66.3 -121.5 85c-10 4 -16 7.7 -18 11c0 8.7 6 14.3 18 17c47.3 18.7 87.8 47\n121.5 85s56.5 81.3 68.5 130c0.7 2 1.3 5 2 9s1.2 6.7 1.5 8c0.3 1.3 1 3.3 2 6\ns2.2 4.5 3.5 5.5c1.3 1 3.3 1.8 6 2.5s6 1 10 1c14 0 21 -3.7 21 -11\nc0 -2 -2 -10.3 -6 -25c-20 -79.3 -65 -146.7 -135 -202l-3 -3h399890z\nM100 620v40h399900v-40z M0 241v40h399900v-40zM0 241v40h399900v-40z",rightarrowabovebar:"M0 241v40h399891c-47.3 35.3-84 78-110 128-16.7 32\n-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20 11 8 0\n13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7 39\n-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85-40.5\n-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5\n-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67\n151.7 139 205zm96 379h399894v40H0zm0 0h399904v40H0z",baraboveshortleftharpoon:"M507,435c-4,4,-6.3,8.7,-7,14c0,5.3,0.7,9,2,11\nc1.3,2,5.3,5.3,12,10c90.7,54,156,130,196,228c3.3,10.7,6.3,16.3,9,17\nc2,0.7,5,1,9,1c0,0,5,0,5,0c10.7,0,16.7,-2,18,-6c2,-2.7,1,-9.7,-3,-21\nc-32,-87.3,-82.7,-157.7,-152,-211c0,0,-3,-3,-3,-3l399351,0l0,-40\nc-398570,0,-399437,0,-399437,0z M593 435 v40 H399500 v-40z\nM0 281 v-40 H399908 v40z M0 281 v-40 H399908 v40z",rightharpoonaboveshortbar:"M0,241 l0,40c399126,0,399993,0,399993,0\nc4.7,-4.7,7,-9.3,7,-14c0,-9.3,-3.7,-15.3,-11,-18c-92.7,-56.7,-159,-133.7,-199,\n-231c-3.3,-9.3,-6,-14.7,-8,-16c-2,-1.3,-7,-2,-15,-2c-10.7,0,-16.7,2,-18,6\nc-2,2.7,-1,9.7,3,21c15.3,42,36.7,81.8,64,119.5c27.3,37.7,58,69.2,92,94.5z\nM0 241 v40 H399908 v-40z M0 475 v-40 H399500 v40z M0 475 v-40 H399500 v40z",shortbaraboveleftharpoon:"M7,435c-4,4,-6.3,8.7,-7,14c0,5.3,0.7,9,2,11\nc1.3,2,5.3,5.3,12,10c90.7,54,156,130,196,228c3.3,10.7,6.3,16.3,9,17c2,0.7,5,1,9,\n1c0,0,5,0,5,0c10.7,0,16.7,-2,18,-6c2,-2.7,1,-9.7,-3,-21c-32,-87.3,-82.7,-157.7,\n-152,-211c0,0,-3,-3,-3,-3l399907,0l0,-40c-399126,0,-399993,0,-399993,0z\nM93 435 v40 H400000 v-40z M500 241 v40 H400000 v-40z M500 241 v40 H400000 v-40z",shortrightharpoonabovebar:"M53,241l0,40c398570,0,399437,0,399437,0\nc4.7,-4.7,7,-9.3,7,-14c0,-9.3,-3.7,-15.3,-11,-18c-92.7,-56.7,-159,-133.7,-199,\n-231c-3.3,-9.3,-6,-14.7,-8,-16c-2,-1.3,-7,-2,-15,-2c-10.7,0,-16.7,2,-18,6\nc-2,2.7,-1,9.7,3,21c15.3,42,36.7,81.8,64,119.5c27.3,37.7,58,69.2,92,94.5z\nM500 241 v40 H399408 v-40z M500 435 v40 H400000 v-40z"};class A{constructor(e){this.children=void 0,this.classes=void 0,this.height=void 0,this.depth=void 0,this.maxFontSize=void 0,this.style=void 0,this.children=e,this.classes=[],this.height=0,this.depth=0,this.maxFontSize=0,this.style={}}hasClass(e){return l.contains(this.classes,e)}toNode(){const e=document.createDocumentFragment();for(let t=0;t<this.children.length;t++)e.appendChild(this.children[t].toNode());return e}toMarkup(){let e="";for(let t=0;t<this.children.length;t++)e+=this.children[t].toMarkup();return e}toText(){return this.children.map((e=>e.toText())).join("")}}var T={"AMS-Regular":{32:[0,0,0,0,.25],65:[0,.68889,0,0,.72222],66:[0,.68889,0,0,.66667],67:[0,.68889,0,0,.72222],68:[0,.68889,0,0,.72222],69:[0,.68889,0,0,.66667],70:[0,.68889,0,0,.61111],71:[0,.68889,0,0,.77778],72:[0,.68889,0,0,.77778],73:[0,.68889,0,0,.38889],74:[.16667,.68889,0,0,.5],75:[0,.68889,0,0,.77778],76:[0,.68889,0,0,.66667],77:[0,.68889,0,0,.94445],78:[0,.68889,0,0,.72222],79:[.16667,.68889,0,0,.77778],80:[0,.68889,0,0,.61111],81:[.16667,.68889,0,0,.77778],82:[0,.68889,0,0,.72222],83:[0,.68889,0,0,.55556],84:[0,.68889,0,0,.66667],85:[0,.68889,0,0,.72222],86:[0,.68889,0,0,.72222],87:[0,.68889,0,0,1],88:[0,.68889,0,0,.72222],89:[0,.68889,0,0,.72222],90:[0,.68889,0,0,.66667],107:[0,.68889,0,0,.55556],160:[0,0,0,0,.25],165:[0,.675,.025,0,.75],174:[.15559,.69224,0,0,.94666],240:[0,.68889,0,0,.55556],295:[0,.68889,0,0,.54028],710:[0,.825,0,0,2.33334],732:[0,.9,0,0,2.33334],770:[0,.825,0,0,2.33334],771:[0,.9,0,0,2.33334],989:[.08167,.58167,0,0,.77778],1008:[0,.43056,.04028,0,.66667],8245:[0,.54986,0,0,.275],8463:[0,.68889,0,0,.54028],8487:[0,.68889,0,0,.72222],8498:[0,.68889,0,0,.55556],8502:[0,.68889,0,0,.66667],8503:[0,.68889,0,0,.44445],8504:[0,.68889,0,0,.66667],8513:[0,.68889,0,0,.63889],8592:[-.03598,.46402,0,0,.5],8594:[-.03598,.46402,0,0,.5],8602:[-.13313,.36687,0,0,1],8603:[-.13313,.36687,0,0,1],8606:[.01354,.52239,0,0,1],8608:[.01354,.52239,0,0,1],8610:[.01354,.52239,0,0,1.11111],8611:[.01354,.52239,0,0,1.11111],8619:[0,.54986,0,0,1],8620:[0,.54986,0,0,1],8621:[-.13313,.37788,0,0,1.38889],8622:[-.13313,.36687,0,0,1],8624:[0,.69224,0,0,.5],8625:[0,.69224,0,0,.5],8630:[0,.43056,0,0,1],8631:[0,.43056,0,0,1],8634:[.08198,.58198,0,0,.77778],8635:[.08198,.58198,0,0,.77778],8638:[.19444,.69224,0,0,.41667],8639:[.19444,.69224,0,0,.41667],8642:[.19444,.69224,0,0,.41667],8643:[.19444,.69224,0,0,.41667],8644:[.1808,.675,0,0,1],8646:[.1808,.675,0,0,1],8647:[.1808,.675,0,0,1],8648:[.19444,.69224,0,0,.83334],8649:[.1808,.675,0,0,1],8650:[.19444,.69224,0,0,.83334],8651:[.01354,.52239,0,0,1],8652:[.01354,.52239,0,0,1],8653:[-.13313,.36687,0,0,1],8654:[-.13313,.36687,0,0,1],8655:[-.13313,.36687,0,0,1],8666:[.13667,.63667,0,0,1],8667:[.13667,.63667,0,0,1],8669:[-.13313,.37788,0,0,1],8672:[-.064,.437,0,0,1.334],8674:[-.064,.437,0,0,1.334],8705:[0,.825,0,0,.5],8708:[0,.68889,0,0,.55556],8709:[.08167,.58167,0,0,.77778],8717:[0,.43056,0,0,.42917],8722:[-.03598,.46402,0,0,.5],8724:[.08198,.69224,0,0,.77778],8726:[.08167,.58167,0,0,.77778],8733:[0,.69224,0,0,.77778],8736:[0,.69224,0,0,.72222],8737:[0,.69224,0,0,.72222],8738:[.03517,.52239,0,0,.72222],8739:[.08167,.58167,0,0,.22222],8740:[.25142,.74111,0,0,.27778],8741:[.08167,.58167,0,0,.38889],8742:[.25142,.74111,0,0,.5],8756:[0,.69224,0,0,.66667],8757:[0,.69224,0,0,.66667],8764:[-.13313,.36687,0,0,.77778],8765:[-.13313,.37788,0,0,.77778],8769:[-.13313,.36687,0,0,.77778],8770:[-.03625,.46375,0,0,.77778],8774:[.30274,.79383,0,0,.77778],8776:[-.01688,.48312,0,0,.77778],8778:[.08167,.58167,0,0,.77778],8782:[.06062,.54986,0,0,.77778],8783:[.06062,.54986,0,0,.77778],8785:[.08198,.58198,0,0,.77778],8786:[.08198,.58198,0,0,.77778],8787:[.08198,.58198,0,0,.77778],8790:[0,.69224,0,0,.77778],8791:[.22958,.72958,0,0,.77778],8796:[.08198,.91667,0,0,.77778],8806:[.25583,.75583,0,0,.77778],8807:[.25583,.75583,0,0,.77778],8808:[.25142,.75726,0,0,.77778],8809:[.25142,.75726,0,0,.77778],8812:[.25583,.75583,0,0,.5],8814:[.20576,.70576,0,0,.77778],8815:[.20576,.70576,0,0,.77778],8816:[.30274,.79383,0,0,.77778],8817:[.30274,.79383,0,0,.77778],8818:[.22958,.72958,0,0,.77778],8819:[.22958,.72958,0,0,.77778],8822:[.1808,.675,0,0,.77778],8823:[.1808,.675,0,0,.77778],8828:[.13667,.63667,0,0,.77778],8829:[.13667,.63667,0,0,.77778],8830:[.22958,.72958,0,0,.77778],8831:[.22958,.72958,0,0,.77778],8832:[.20576,.70576,0,0,.77778],8833:[.20576,.70576,0,0,.77778],8840:[.30274,.79383,0,0,.77778],8841:[.30274,.79383,0,0,.77778],8842:[.13597,.63597,0,0,.77778],8843:[.13597,.63597,0,0,.77778],8847:[.03517,.54986,0,0,.77778],8848:[.03517,.54986,0,0,.77778],8858:[.08198,.58198,0,0,.77778],8859:[.08198,.58198,0,0,.77778],8861:[.08198,.58198,0,0,.77778],8862:[0,.675,0,0,.77778],8863:[0,.675,0,0,.77778],8864:[0,.675,0,0,.77778],8865:[0,.675,0,0,.77778],8872:[0,.69224,0,0,.61111],8873:[0,.69224,0,0,.72222],8874:[0,.69224,0,0,.88889],8876:[0,.68889,0,0,.61111],8877:[0,.68889,0,0,.61111],8878:[0,.68889,0,0,.72222],8879:[0,.68889,0,0,.72222],8882:[.03517,.54986,0,0,.77778],8883:[.03517,.54986,0,0,.77778],8884:[.13667,.63667,0,0,.77778],8885:[.13667,.63667,0,0,.77778],8888:[0,.54986,0,0,1.11111],8890:[.19444,.43056,0,0,.55556],8891:[.19444,.69224,0,0,.61111],8892:[.19444,.69224,0,0,.61111],8901:[0,.54986,0,0,.27778],8903:[.08167,.58167,0,0,.77778],8905:[.08167,.58167,0,0,.77778],8906:[.08167,.58167,0,0,.77778],8907:[0,.69224,0,0,.77778],8908:[0,.69224,0,0,.77778],8909:[-.03598,.46402,0,0,.77778],8910:[0,.54986,0,0,.76042],8911:[0,.54986,0,0,.76042],8912:[.03517,.54986,0,0,.77778],8913:[.03517,.54986,0,0,.77778],8914:[0,.54986,0,0,.66667],8915:[0,.54986,0,0,.66667],8916:[0,.69224,0,0,.66667],8918:[.0391,.5391,0,0,.77778],8919:[.0391,.5391,0,0,.77778],8920:[.03517,.54986,0,0,1.33334],8921:[.03517,.54986,0,0,1.33334],8922:[.38569,.88569,0,0,.77778],8923:[.38569,.88569,0,0,.77778],8926:[.13667,.63667,0,0,.77778],8927:[.13667,.63667,0,0,.77778],8928:[.30274,.79383,0,0,.77778],8929:[.30274,.79383,0,0,.77778],8934:[.23222,.74111,0,0,.77778],8935:[.23222,.74111,0,0,.77778],8936:[.23222,.74111,0,0,.77778],8937:[.23222,.74111,0,0,.77778],8938:[.20576,.70576,0,0,.77778],8939:[.20576,.70576,0,0,.77778],8940:[.30274,.79383,0,0,.77778],8941:[.30274,.79383,0,0,.77778],8994:[.19444,.69224,0,0,.77778],8995:[.19444,.69224,0,0,.77778],9416:[.15559,.69224,0,0,.90222],9484:[0,.69224,0,0,.5],9488:[0,.69224,0,0,.5],9492:[0,.37788,0,0,.5],9496:[0,.37788,0,0,.5],9585:[.19444,.68889,0,0,.88889],9586:[.19444,.74111,0,0,.88889],9632:[0,.675,0,0,.77778],9633:[0,.675,0,0,.77778],9650:[0,.54986,0,0,.72222],9651:[0,.54986,0,0,.72222],9654:[.03517,.54986,0,0,.77778],9660:[0,.54986,0,0,.72222],9661:[0,.54986,0,0,.72222],9664:[.03517,.54986,0,0,.77778],9674:[.11111,.69224,0,0,.66667],9733:[.19444,.69224,0,0,.94445],10003:[0,.69224,0,0,.83334],10016:[0,.69224,0,0,.83334],10731:[.11111,.69224,0,0,.66667],10846:[.19444,.75583,0,0,.61111],10877:[.13667,.63667,0,0,.77778],10878:[.13667,.63667,0,0,.77778],10885:[.25583,.75583,0,0,.77778],10886:[.25583,.75583,0,0,.77778],10887:[.13597,.63597,0,0,.77778],10888:[.13597,.63597,0,0,.77778],10889:[.26167,.75726,0,0,.77778],10890:[.26167,.75726,0,0,.77778],10891:[.48256,.98256,0,0,.77778],10892:[.48256,.98256,0,0,.77778],10901:[.13667,.63667,0,0,.77778],10902:[.13667,.63667,0,0,.77778],10933:[.25142,.75726,0,0,.77778],10934:[.25142,.75726,0,0,.77778],10935:[.26167,.75726,0,0,.77778],10936:[.26167,.75726,0,0,.77778],10937:[.26167,.75726,0,0,.77778],10938:[.26167,.75726,0,0,.77778],10949:[.25583,.75583,0,0,.77778],10950:[.25583,.75583,0,0,.77778],10955:[.28481,.79383,0,0,.77778],10956:[.28481,.79383,0,0,.77778],57350:[.08167,.58167,0,0,.22222],57351:[.08167,.58167,0,0,.38889],57352:[.08167,.58167,0,0,.77778],57353:[0,.43056,.04028,0,.66667],57356:[.25142,.75726,0,0,.77778],57357:[.25142,.75726,0,0,.77778],57358:[.41951,.91951,0,0,.77778],57359:[.30274,.79383,0,0,.77778],57360:[.30274,.79383,0,0,.77778],57361:[.41951,.91951,0,0,.77778],57366:[.25142,.75726,0,0,.77778],57367:[.25142,.75726,0,0,.77778],57368:[.25142,.75726,0,0,.77778],57369:[.25142,.75726,0,0,.77778],57370:[.13597,.63597,0,0,.77778],57371:[.13597,.63597,0,0,.77778]},"Caligraphic-Regular":{32:[0,0,0,0,.25],65:[0,.68333,0,.19445,.79847],66:[0,.68333,.03041,.13889,.65681],67:[0,.68333,.05834,.13889,.52653],68:[0,.68333,.02778,.08334,.77139],69:[0,.68333,.08944,.11111,.52778],70:[0,.68333,.09931,.11111,.71875],71:[.09722,.68333,.0593,.11111,.59487],72:[0,.68333,.00965,.11111,.84452],73:[0,.68333,.07382,0,.54452],74:[.09722,.68333,.18472,.16667,.67778],75:[0,.68333,.01445,.05556,.76195],76:[0,.68333,0,.13889,.68972],77:[0,.68333,0,.13889,1.2009],78:[0,.68333,.14736,.08334,.82049],79:[0,.68333,.02778,.11111,.79611],80:[0,.68333,.08222,.08334,.69556],81:[.09722,.68333,0,.11111,.81667],82:[0,.68333,0,.08334,.8475],83:[0,.68333,.075,.13889,.60556],84:[0,.68333,.25417,0,.54464],85:[0,.68333,.09931,.08334,.62583],86:[0,.68333,.08222,0,.61278],87:[0,.68333,.08222,.08334,.98778],88:[0,.68333,.14643,.13889,.7133],89:[.09722,.68333,.08222,.08334,.66834],90:[0,.68333,.07944,.13889,.72473],160:[0,0,0,0,.25]},"Fraktur-Regular":{32:[0,0,0,0,.25],33:[0,.69141,0,0,.29574],34:[0,.69141,0,0,.21471],38:[0,.69141,0,0,.73786],39:[0,.69141,0,0,.21201],40:[.24982,.74947,0,0,.38865],41:[.24982,.74947,0,0,.38865],42:[0,.62119,0,0,.27764],43:[.08319,.58283,0,0,.75623],44:[0,.10803,0,0,.27764],45:[.08319,.58283,0,0,.75623],46:[0,.10803,0,0,.27764],47:[.24982,.74947,0,0,.50181],48:[0,.47534,0,0,.50181],49:[0,.47534,0,0,.50181],50:[0,.47534,0,0,.50181],51:[.18906,.47534,0,0,.50181],52:[.18906,.47534,0,0,.50181],53:[.18906,.47534,0,0,.50181],54:[0,.69141,0,0,.50181],55:[.18906,.47534,0,0,.50181],56:[0,.69141,0,0,.50181],57:[.18906,.47534,0,0,.50181],58:[0,.47534,0,0,.21606],59:[.12604,.47534,0,0,.21606],61:[-.13099,.36866,0,0,.75623],63:[0,.69141,0,0,.36245],65:[0,.69141,0,0,.7176],66:[0,.69141,0,0,.88397],67:[0,.69141,0,0,.61254],68:[0,.69141,0,0,.83158],69:[0,.69141,0,0,.66278],70:[.12604,.69141,0,0,.61119],71:[0,.69141,0,0,.78539],72:[.06302,.69141,0,0,.7203],73:[0,.69141,0,0,.55448],74:[.12604,.69141,0,0,.55231],75:[0,.69141,0,0,.66845],76:[0,.69141,0,0,.66602],77:[0,.69141,0,0,1.04953],78:[0,.69141,0,0,.83212],79:[0,.69141,0,0,.82699],80:[.18906,.69141,0,0,.82753],81:[.03781,.69141,0,0,.82699],82:[0,.69141,0,0,.82807],83:[0,.69141,0,0,.82861],84:[0,.69141,0,0,.66899],85:[0,.69141,0,0,.64576],86:[0,.69141,0,0,.83131],87:[0,.69141,0,0,1.04602],88:[0,.69141,0,0,.71922],89:[.18906,.69141,0,0,.83293],90:[.12604,.69141,0,0,.60201],91:[.24982,.74947,0,0,.27764],93:[.24982,.74947,0,0,.27764],94:[0,.69141,0,0,.49965],97:[0,.47534,0,0,.50046],98:[0,.69141,0,0,.51315],99:[0,.47534,0,0,.38946],100:[0,.62119,0,0,.49857],101:[0,.47534,0,0,.40053],102:[.18906,.69141,0,0,.32626],103:[.18906,.47534,0,0,.5037],104:[.18906,.69141,0,0,.52126],105:[0,.69141,0,0,.27899],106:[0,.69141,0,0,.28088],107:[0,.69141,0,0,.38946],108:[0,.69141,0,0,.27953],109:[0,.47534,0,0,.76676],110:[0,.47534,0,0,.52666],111:[0,.47534,0,0,.48885],112:[.18906,.52396,0,0,.50046],113:[.18906,.47534,0,0,.48912],114:[0,.47534,0,0,.38919],115:[0,.47534,0,0,.44266],116:[0,.62119,0,0,.33301],117:[0,.47534,0,0,.5172],118:[0,.52396,0,0,.5118],119:[0,.52396,0,0,.77351],120:[.18906,.47534,0,0,.38865],121:[.18906,.47534,0,0,.49884],122:[.18906,.47534,0,0,.39054],160:[0,0,0,0,.25],8216:[0,.69141,0,0,.21471],8217:[0,.69141,0,0,.21471],58112:[0,.62119,0,0,.49749],58113:[0,.62119,0,0,.4983],58114:[.18906,.69141,0,0,.33328],58115:[.18906,.69141,0,0,.32923],58116:[.18906,.47534,0,0,.50343],58117:[0,.69141,0,0,.33301],58118:[0,.62119,0,0,.33409],58119:[0,.47534,0,0,.50073]},"Main-Bold":{32:[0,0,0,0,.25],33:[0,.69444,0,0,.35],34:[0,.69444,0,0,.60278],35:[.19444,.69444,0,0,.95833],36:[.05556,.75,0,0,.575],37:[.05556,.75,0,0,.95833],38:[0,.69444,0,0,.89444],39:[0,.69444,0,0,.31944],40:[.25,.75,0,0,.44722],41:[.25,.75,0,0,.44722],42:[0,.75,0,0,.575],43:[.13333,.63333,0,0,.89444],44:[.19444,.15556,0,0,.31944],45:[0,.44444,0,0,.38333],46:[0,.15556,0,0,.31944],47:[.25,.75,0,0,.575],48:[0,.64444,0,0,.575],49:[0,.64444,0,0,.575],50:[0,.64444,0,0,.575],51:[0,.64444,0,0,.575],52:[0,.64444,0,0,.575],53:[0,.64444,0,0,.575],54:[0,.64444,0,0,.575],55:[0,.64444,0,0,.575],56:[0,.64444,0,0,.575],57:[0,.64444,0,0,.575],58:[0,.44444,0,0,.31944],59:[.19444,.44444,0,0,.31944],60:[.08556,.58556,0,0,.89444],61:[-.10889,.39111,0,0,.89444],62:[.08556,.58556,0,0,.89444],63:[0,.69444,0,0,.54305],64:[0,.69444,0,0,.89444],65:[0,.68611,0,0,.86944],66:[0,.68611,0,0,.81805],67:[0,.68611,0,0,.83055],68:[0,.68611,0,0,.88194],69:[0,.68611,0,0,.75555],70:[0,.68611,0,0,.72361],71:[0,.68611,0,0,.90416],72:[0,.68611,0,0,.9],73:[0,.68611,0,0,.43611],74:[0,.68611,0,0,.59444],75:[0,.68611,0,0,.90138],76:[0,.68611,0,0,.69166],77:[0,.68611,0,0,1.09166],78:[0,.68611,0,0,.9],79:[0,.68611,0,0,.86388],80:[0,.68611,0,0,.78611],81:[.19444,.68611,0,0,.86388],82:[0,.68611,0,0,.8625],83:[0,.68611,0,0,.63889],84:[0,.68611,0,0,.8],85:[0,.68611,0,0,.88472],86:[0,.68611,.01597,0,.86944],87:[0,.68611,.01597,0,1.18888],88:[0,.68611,0,0,.86944],89:[0,.68611,.02875,0,.86944],90:[0,.68611,0,0,.70277],91:[.25,.75,0,0,.31944],92:[.25,.75,0,0,.575],93:[.25,.75,0,0,.31944],94:[0,.69444,0,0,.575],95:[.31,.13444,.03194,0,.575],97:[0,.44444,0,0,.55902],98:[0,.69444,0,0,.63889],99:[0,.44444,0,0,.51111],100:[0,.69444,0,0,.63889],101:[0,.44444,0,0,.52708],102:[0,.69444,.10903,0,.35139],103:[.19444,.44444,.01597,0,.575],104:[0,.69444,0,0,.63889],105:[0,.69444,0,0,.31944],106:[.19444,.69444,0,0,.35139],107:[0,.69444,0,0,.60694],108:[0,.69444,0,0,.31944],109:[0,.44444,0,0,.95833],110:[0,.44444,0,0,.63889],111:[0,.44444,0,0,.575],112:[.19444,.44444,0,0,.63889],113:[.19444,.44444,0,0,.60694],114:[0,.44444,0,0,.47361],115:[0,.44444,0,0,.45361],116:[0,.63492,0,0,.44722],117:[0,.44444,0,0,.63889],118:[0,.44444,.01597,0,.60694],119:[0,.44444,.01597,0,.83055],120:[0,.44444,0,0,.60694],121:[.19444,.44444,.01597,0,.60694],122:[0,.44444,0,0,.51111],123:[.25,.75,0,0,.575],124:[.25,.75,0,0,.31944],125:[.25,.75,0,0,.575],126:[.35,.34444,0,0,.575],160:[0,0,0,0,.25],163:[0,.69444,0,0,.86853],168:[0,.69444,0,0,.575],172:[0,.44444,0,0,.76666],176:[0,.69444,0,0,.86944],177:[.13333,.63333,0,0,.89444],184:[.17014,0,0,0,.51111],198:[0,.68611,0,0,1.04166],215:[.13333,.63333,0,0,.89444],216:[.04861,.73472,0,0,.89444],223:[0,.69444,0,0,.59722],230:[0,.44444,0,0,.83055],247:[.13333,.63333,0,0,.89444],248:[.09722,.54167,0,0,.575],305:[0,.44444,0,0,.31944],338:[0,.68611,0,0,1.16944],339:[0,.44444,0,0,.89444],567:[.19444,.44444,0,0,.35139],710:[0,.69444,0,0,.575],711:[0,.63194,0,0,.575],713:[0,.59611,0,0,.575],714:[0,.69444,0,0,.575],715:[0,.69444,0,0,.575],728:[0,.69444,0,0,.575],729:[0,.69444,0,0,.31944],730:[0,.69444,0,0,.86944],732:[0,.69444,0,0,.575],733:[0,.69444,0,0,.575],915:[0,.68611,0,0,.69166],916:[0,.68611,0,0,.95833],920:[0,.68611,0,0,.89444],923:[0,.68611,0,0,.80555],926:[0,.68611,0,0,.76666],928:[0,.68611,0,0,.9],931:[0,.68611,0,0,.83055],933:[0,.68611,0,0,.89444],934:[0,.68611,0,0,.83055],936:[0,.68611,0,0,.89444],937:[0,.68611,0,0,.83055],8211:[0,.44444,.03194,0,.575],8212:[0,.44444,.03194,0,1.14999],8216:[0,.69444,0,0,.31944],8217:[0,.69444,0,0,.31944],8220:[0,.69444,0,0,.60278],8221:[0,.69444,0,0,.60278],8224:[.19444,.69444,0,0,.51111],8225:[.19444,.69444,0,0,.51111],8242:[0,.55556,0,0,.34444],8407:[0,.72444,.15486,0,.575],8463:[0,.69444,0,0,.66759],8465:[0,.69444,0,0,.83055],8467:[0,.69444,0,0,.47361],8472:[.19444,.44444,0,0,.74027],8476:[0,.69444,0,0,.83055],8501:[0,.69444,0,0,.70277],8592:[-.10889,.39111,0,0,1.14999],8593:[.19444,.69444,0,0,.575],8594:[-.10889,.39111,0,0,1.14999],8595:[.19444,.69444,0,0,.575],8596:[-.10889,.39111,0,0,1.14999],8597:[.25,.75,0,0,.575],8598:[.19444,.69444,0,0,1.14999],8599:[.19444,.69444,0,0,1.14999],8600:[.19444,.69444,0,0,1.14999],8601:[.19444,.69444,0,0,1.14999],8636:[-.10889,.39111,0,0,1.14999],8637:[-.10889,.39111,0,0,1.14999],8640:[-.10889,.39111,0,0,1.14999],8641:[-.10889,.39111,0,0,1.14999],8656:[-.10889,.39111,0,0,1.14999],8657:[.19444,.69444,0,0,.70277],8658:[-.10889,.39111,0,0,1.14999],8659:[.19444,.69444,0,0,.70277],8660:[-.10889,.39111,0,0,1.14999],8661:[.25,.75,0,0,.70277],8704:[0,.69444,0,0,.63889],8706:[0,.69444,.06389,0,.62847],8707:[0,.69444,0,0,.63889],8709:[.05556,.75,0,0,.575],8711:[0,.68611,0,0,.95833],8712:[.08556,.58556,0,0,.76666],8715:[.08556,.58556,0,0,.76666],8722:[.13333,.63333,0,0,.89444],8723:[.13333,.63333,0,0,.89444],8725:[.25,.75,0,0,.575],8726:[.25,.75,0,0,.575],8727:[-.02778,.47222,0,0,.575],8728:[-.02639,.47361,0,0,.575],8729:[-.02639,.47361,0,0,.575],8730:[.18,.82,0,0,.95833],8733:[0,.44444,0,0,.89444],8734:[0,.44444,0,0,1.14999],8736:[0,.69224,0,0,.72222],8739:[.25,.75,0,0,.31944],8741:[.25,.75,0,0,.575],8743:[0,.55556,0,0,.76666],8744:[0,.55556,0,0,.76666],8745:[0,.55556,0,0,.76666],8746:[0,.55556,0,0,.76666],8747:[.19444,.69444,.12778,0,.56875],8764:[-.10889,.39111,0,0,.89444],8768:[.19444,.69444,0,0,.31944],8771:[.00222,.50222,0,0,.89444],8773:[.027,.638,0,0,.894],8776:[.02444,.52444,0,0,.89444],8781:[.00222,.50222,0,0,.89444],8801:[.00222,.50222,0,0,.89444],8804:[.19667,.69667,0,0,.89444],8805:[.19667,.69667,0,0,.89444],8810:[.08556,.58556,0,0,1.14999],8811:[.08556,.58556,0,0,1.14999],8826:[.08556,.58556,0,0,.89444],8827:[.08556,.58556,0,0,.89444],8834:[.08556,.58556,0,0,.89444],8835:[.08556,.58556,0,0,.89444],8838:[.19667,.69667,0,0,.89444],8839:[.19667,.69667,0,0,.89444],8846:[0,.55556,0,0,.76666],8849:[.19667,.69667,0,0,.89444],8850:[.19667,.69667,0,0,.89444],8851:[0,.55556,0,0,.76666],8852:[0,.55556,0,0,.76666],8853:[.13333,.63333,0,0,.89444],8854:[.13333,.63333,0,0,.89444],8855:[.13333,.63333,0,0,.89444],8856:[.13333,.63333,0,0,.89444],8857:[.13333,.63333,0,0,.89444],8866:[0,.69444,0,0,.70277],8867:[0,.69444,0,0,.70277],8868:[0,.69444,0,0,.89444],8869:[0,.69444,0,0,.89444],8900:[-.02639,.47361,0,0,.575],8901:[-.02639,.47361,0,0,.31944],8902:[-.02778,.47222,0,0,.575],8968:[.25,.75,0,0,.51111],8969:[.25,.75,0,0,.51111],8970:[.25,.75,0,0,.51111],8971:[.25,.75,0,0,.51111],8994:[-.13889,.36111,0,0,1.14999],8995:[-.13889,.36111,0,0,1.14999],9651:[.19444,.69444,0,0,1.02222],9657:[-.02778,.47222,0,0,.575],9661:[.19444,.69444,0,0,1.02222],9667:[-.02778,.47222,0,0,.575],9711:[.19444,.69444,0,0,1.14999],9824:[.12963,.69444,0,0,.89444],9825:[.12963,.69444,0,0,.89444],9826:[.12963,.69444,0,0,.89444],9827:[.12963,.69444,0,0,.89444],9837:[0,.75,0,0,.44722],9838:[.19444,.69444,0,0,.44722],9839:[.19444,.69444,0,0,.44722],10216:[.25,.75,0,0,.44722],10217:[.25,.75,0,0,.44722],10815:[0,.68611,0,0,.9],10927:[.19667,.69667,0,0,.89444],10928:[.19667,.69667,0,0,.89444],57376:[.19444,.69444,0,0,0]},"Main-BoldItalic":{32:[0,0,0,0,.25],33:[0,.69444,.11417,0,.38611],34:[0,.69444,.07939,0,.62055],35:[.19444,.69444,.06833,0,.94444],37:[.05556,.75,.12861,0,.94444],38:[0,.69444,.08528,0,.88555],39:[0,.69444,.12945,0,.35555],40:[.25,.75,.15806,0,.47333],41:[.25,.75,.03306,0,.47333],42:[0,.75,.14333,0,.59111],43:[.10333,.60333,.03306,0,.88555],44:[.19444,.14722,0,0,.35555],45:[0,.44444,.02611,0,.41444],46:[0,.14722,0,0,.35555],47:[.25,.75,.15806,0,.59111],48:[0,.64444,.13167,0,.59111],49:[0,.64444,.13167,0,.59111],50:[0,.64444,.13167,0,.59111],51:[0,.64444,.13167,0,.59111],52:[.19444,.64444,.13167,0,.59111],53:[0,.64444,.13167,0,.59111],54:[0,.64444,.13167,0,.59111],55:[.19444,.64444,.13167,0,.59111],56:[0,.64444,.13167,0,.59111],57:[0,.64444,.13167,0,.59111],58:[0,.44444,.06695,0,.35555],59:[.19444,.44444,.06695,0,.35555],61:[-.10889,.39111,.06833,0,.88555],63:[0,.69444,.11472,0,.59111],64:[0,.69444,.09208,0,.88555],65:[0,.68611,0,0,.86555],66:[0,.68611,.0992,0,.81666],67:[0,.68611,.14208,0,.82666],68:[0,.68611,.09062,0,.87555],69:[0,.68611,.11431,0,.75666],70:[0,.68611,.12903,0,.72722],71:[0,.68611,.07347,0,.89527],72:[0,.68611,.17208,0,.8961],73:[0,.68611,.15681,0,.47166],74:[0,.68611,.145,0,.61055],75:[0,.68611,.14208,0,.89499],76:[0,.68611,0,0,.69777],77:[0,.68611,.17208,0,1.07277],78:[0,.68611,.17208,0,.8961],79:[0,.68611,.09062,0,.85499],80:[0,.68611,.0992,0,.78721],81:[.19444,.68611,.09062,0,.85499],82:[0,.68611,.02559,0,.85944],83:[0,.68611,.11264,0,.64999],84:[0,.68611,.12903,0,.7961],85:[0,.68611,.17208,0,.88083],86:[0,.68611,.18625,0,.86555],87:[0,.68611,.18625,0,1.15999],88:[0,.68611,.15681,0,.86555],89:[0,.68611,.19803,0,.86555],90:[0,.68611,.14208,0,.70888],91:[.25,.75,.1875,0,.35611],93:[.25,.75,.09972,0,.35611],94:[0,.69444,.06709,0,.59111],95:[.31,.13444,.09811,0,.59111],97:[0,.44444,.09426,0,.59111],98:[0,.69444,.07861,0,.53222],99:[0,.44444,.05222,0,.53222],100:[0,.69444,.10861,0,.59111],101:[0,.44444,.085,0,.53222],102:[.19444,.69444,.21778,0,.4],103:[.19444,.44444,.105,0,.53222],104:[0,.69444,.09426,0,.59111],105:[0,.69326,.11387,0,.35555],106:[.19444,.69326,.1672,0,.35555],107:[0,.69444,.11111,0,.53222],108:[0,.69444,.10861,0,.29666],109:[0,.44444,.09426,0,.94444],110:[0,.44444,.09426,0,.64999],111:[0,.44444,.07861,0,.59111],112:[.19444,.44444,.07861,0,.59111],113:[.19444,.44444,.105,0,.53222],114:[0,.44444,.11111,0,.50167],115:[0,.44444,.08167,0,.48694],116:[0,.63492,.09639,0,.385],117:[0,.44444,.09426,0,.62055],118:[0,.44444,.11111,0,.53222],119:[0,.44444,.11111,0,.76777],120:[0,.44444,.12583,0,.56055],121:[.19444,.44444,.105,0,.56166],122:[0,.44444,.13889,0,.49055],126:[.35,.34444,.11472,0,.59111],160:[0,0,0,0,.25],168:[0,.69444,.11473,0,.59111],176:[0,.69444,0,0,.94888],184:[.17014,0,0,0,.53222],198:[0,.68611,.11431,0,1.02277],216:[.04861,.73472,.09062,0,.88555],223:[.19444,.69444,.09736,0,.665],230:[0,.44444,.085,0,.82666],248:[.09722,.54167,.09458,0,.59111],305:[0,.44444,.09426,0,.35555],338:[0,.68611,.11431,0,1.14054],339:[0,.44444,.085,0,.82666],567:[.19444,.44444,.04611,0,.385],710:[0,.69444,.06709,0,.59111],711:[0,.63194,.08271,0,.59111],713:[0,.59444,.10444,0,.59111],714:[0,.69444,.08528,0,.59111],715:[0,.69444,0,0,.59111],728:[0,.69444,.10333,0,.59111],729:[0,.69444,.12945,0,.35555],730:[0,.69444,0,0,.94888],732:[0,.69444,.11472,0,.59111],733:[0,.69444,.11472,0,.59111],915:[0,.68611,.12903,0,.69777],916:[0,.68611,0,0,.94444],920:[0,.68611,.09062,0,.88555],923:[0,.68611,0,0,.80666],926:[0,.68611,.15092,0,.76777],928:[0,.68611,.17208,0,.8961],931:[0,.68611,.11431,0,.82666],933:[0,.68611,.10778,0,.88555],934:[0,.68611,.05632,0,.82666],936:[0,.68611,.10778,0,.88555],937:[0,.68611,.0992,0,.82666],8211:[0,.44444,.09811,0,.59111],8212:[0,.44444,.09811,0,1.18221],8216:[0,.69444,.12945,0,.35555],8217:[0,.69444,.12945,0,.35555],8220:[0,.69444,.16772,0,.62055],8221:[0,.69444,.07939,0,.62055]},"Main-Italic":{32:[0,0,0,0,.25],33:[0,.69444,.12417,0,.30667],34:[0,.69444,.06961,0,.51444],35:[.19444,.69444,.06616,0,.81777],37:[.05556,.75,.13639,0,.81777],38:[0,.69444,.09694,0,.76666],39:[0,.69444,.12417,0,.30667],40:[.25,.75,.16194,0,.40889],41:[.25,.75,.03694,0,.40889],42:[0,.75,.14917,0,.51111],43:[.05667,.56167,.03694,0,.76666],44:[.19444,.10556,0,0,.30667],45:[0,.43056,.02826,0,.35778],46:[0,.10556,0,0,.30667],47:[.25,.75,.16194,0,.51111],48:[0,.64444,.13556,0,.51111],49:[0,.64444,.13556,0,.51111],50:[0,.64444,.13556,0,.51111],51:[0,.64444,.13556,0,.51111],52:[.19444,.64444,.13556,0,.51111],53:[0,.64444,.13556,0,.51111],54:[0,.64444,.13556,0,.51111],55:[.19444,.64444,.13556,0,.51111],56:[0,.64444,.13556,0,.51111],57:[0,.64444,.13556,0,.51111],58:[0,.43056,.0582,0,.30667],59:[.19444,.43056,.0582,0,.30667],61:[-.13313,.36687,.06616,0,.76666],63:[0,.69444,.1225,0,.51111],64:[0,.69444,.09597,0,.76666],65:[0,.68333,0,0,.74333],66:[0,.68333,.10257,0,.70389],67:[0,.68333,.14528,0,.71555],68:[0,.68333,.09403,0,.755],69:[0,.68333,.12028,0,.67833],70:[0,.68333,.13305,0,.65277],71:[0,.68333,.08722,0,.77361],72:[0,.68333,.16389,0,.74333],73:[0,.68333,.15806,0,.38555],74:[0,.68333,.14028,0,.525],75:[0,.68333,.14528,0,.76888],76:[0,.68333,0,0,.62722],77:[0,.68333,.16389,0,.89666],78:[0,.68333,.16389,0,.74333],79:[0,.68333,.09403,0,.76666],80:[0,.68333,.10257,0,.67833],81:[.19444,.68333,.09403,0,.76666],82:[0,.68333,.03868,0,.72944],83:[0,.68333,.11972,0,.56222],84:[0,.68333,.13305,0,.71555],85:[0,.68333,.16389,0,.74333],86:[0,.68333,.18361,0,.74333],87:[0,.68333,.18361,0,.99888],88:[0,.68333,.15806,0,.74333],89:[0,.68333,.19383,0,.74333],90:[0,.68333,.14528,0,.61333],91:[.25,.75,.1875,0,.30667],93:[.25,.75,.10528,0,.30667],94:[0,.69444,.06646,0,.51111],95:[.31,.12056,.09208,0,.51111],97:[0,.43056,.07671,0,.51111],98:[0,.69444,.06312,0,.46],99:[0,.43056,.05653,0,.46],100:[0,.69444,.10333,0,.51111],101:[0,.43056,.07514,0,.46],102:[.19444,.69444,.21194,0,.30667],103:[.19444,.43056,.08847,0,.46],104:[0,.69444,.07671,0,.51111],105:[0,.65536,.1019,0,.30667],106:[.19444,.65536,.14467,0,.30667],107:[0,.69444,.10764,0,.46],108:[0,.69444,.10333,0,.25555],109:[0,.43056,.07671,0,.81777],110:[0,.43056,.07671,0,.56222],111:[0,.43056,.06312,0,.51111],112:[.19444,.43056,.06312,0,.51111],113:[.19444,.43056,.08847,0,.46],114:[0,.43056,.10764,0,.42166],115:[0,.43056,.08208,0,.40889],116:[0,.61508,.09486,0,.33222],117:[0,.43056,.07671,0,.53666],118:[0,.43056,.10764,0,.46],119:[0,.43056,.10764,0,.66444],120:[0,.43056,.12042,0,.46389],121:[.19444,.43056,.08847,0,.48555],122:[0,.43056,.12292,0,.40889],126:[.35,.31786,.11585,0,.51111],160:[0,0,0,0,.25],168:[0,.66786,.10474,0,.51111],176:[0,.69444,0,0,.83129],184:[.17014,0,0,0,.46],198:[0,.68333,.12028,0,.88277],216:[.04861,.73194,.09403,0,.76666],223:[.19444,.69444,.10514,0,.53666],230:[0,.43056,.07514,0,.71555],248:[.09722,.52778,.09194,0,.51111],338:[0,.68333,.12028,0,.98499],339:[0,.43056,.07514,0,.71555],710:[0,.69444,.06646,0,.51111],711:[0,.62847,.08295,0,.51111],713:[0,.56167,.10333,0,.51111],714:[0,.69444,.09694,0,.51111],715:[0,.69444,0,0,.51111],728:[0,.69444,.10806,0,.51111],729:[0,.66786,.11752,0,.30667],730:[0,.69444,0,0,.83129],732:[0,.66786,.11585,0,.51111],733:[0,.69444,.1225,0,.51111],915:[0,.68333,.13305,0,.62722],916:[0,.68333,0,0,.81777],920:[0,.68333,.09403,0,.76666],923:[0,.68333,0,0,.69222],926:[0,.68333,.15294,0,.66444],928:[0,.68333,.16389,0,.74333],931:[0,.68333,.12028,0,.71555],933:[0,.68333,.11111,0,.76666],934:[0,.68333,.05986,0,.71555],936:[0,.68333,.11111,0,.76666],937:[0,.68333,.10257,0,.71555],8211:[0,.43056,.09208,0,.51111],8212:[0,.43056,.09208,0,1.02222],8216:[0,.69444,.12417,0,.30667],8217:[0,.69444,.12417,0,.30667],8220:[0,.69444,.1685,0,.51444],8221:[0,.69444,.06961,0,.51444],8463:[0,.68889,0,0,.54028]},"Main-Regular":{32:[0,0,0,0,.25],33:[0,.69444,0,0,.27778],34:[0,.69444,0,0,.5],35:[.19444,.69444,0,0,.83334],36:[.05556,.75,0,0,.5],37:[.05556,.75,0,0,.83334],38:[0,.69444,0,0,.77778],39:[0,.69444,0,0,.27778],40:[.25,.75,0,0,.38889],41:[.25,.75,0,0,.38889],42:[0,.75,0,0,.5],43:[.08333,.58333,0,0,.77778],44:[.19444,.10556,0,0,.27778],45:[0,.43056,0,0,.33333],46:[0,.10556,0,0,.27778],47:[.25,.75,0,0,.5],48:[0,.64444,0,0,.5],49:[0,.64444,0,0,.5],50:[0,.64444,0,0,.5],51:[0,.64444,0,0,.5],52:[0,.64444,0,0,.5],53:[0,.64444,0,0,.5],54:[0,.64444,0,0,.5],55:[0,.64444,0,0,.5],56:[0,.64444,0,0,.5],57:[0,.64444,0,0,.5],58:[0,.43056,0,0,.27778],59:[.19444,.43056,0,0,.27778],60:[.0391,.5391,0,0,.77778],61:[-.13313,.36687,0,0,.77778],62:[.0391,.5391,0,0,.77778],63:[0,.69444,0,0,.47222],64:[0,.69444,0,0,.77778],65:[0,.68333,0,0,.75],66:[0,.68333,0,0,.70834],67:[0,.68333,0,0,.72222],68:[0,.68333,0,0,.76389],69:[0,.68333,0,0,.68056],70:[0,.68333,0,0,.65278],71:[0,.68333,0,0,.78472],72:[0,.68333,0,0,.75],73:[0,.68333,0,0,.36111],74:[0,.68333,0,0,.51389],75:[0,.68333,0,0,.77778],76:[0,.68333,0,0,.625],77:[0,.68333,0,0,.91667],78:[0,.68333,0,0,.75],79:[0,.68333,0,0,.77778],80:[0,.68333,0,0,.68056],81:[.19444,.68333,0,0,.77778],82:[0,.68333,0,0,.73611],83:[0,.68333,0,0,.55556],84:[0,.68333,0,0,.72222],85:[0,.68333,0,0,.75],86:[0,.68333,.01389,0,.75],87:[0,.68333,.01389,0,1.02778],88:[0,.68333,0,0,.75],89:[0,.68333,.025,0,.75],90:[0,.68333,0,0,.61111],91:[.25,.75,0,0,.27778],92:[.25,.75,0,0,.5],93:[.25,.75,0,0,.27778],94:[0,.69444,0,0,.5],95:[.31,.12056,.02778,0,.5],97:[0,.43056,0,0,.5],98:[0,.69444,0,0,.55556],99:[0,.43056,0,0,.44445],100:[0,.69444,0,0,.55556],101:[0,.43056,0,0,.44445],102:[0,.69444,.07778,0,.30556],103:[.19444,.43056,.01389,0,.5],104:[0,.69444,0,0,.55556],105:[0,.66786,0,0,.27778],106:[.19444,.66786,0,0,.30556],107:[0,.69444,0,0,.52778],108:[0,.69444,0,0,.27778],109:[0,.43056,0,0,.83334],110:[0,.43056,0,0,.55556],111:[0,.43056,0,0,.5],112:[.19444,.43056,0,0,.55556],113:[.19444,.43056,0,0,.52778],114:[0,.43056,0,0,.39167],115:[0,.43056,0,0,.39445],116:[0,.61508,0,0,.38889],117:[0,.43056,0,0,.55556],118:[0,.43056,.01389,0,.52778],119:[0,.43056,.01389,0,.72222],120:[0,.43056,0,0,.52778],121:[.19444,.43056,.01389,0,.52778],122:[0,.43056,0,0,.44445],123:[.25,.75,0,0,.5],124:[.25,.75,0,0,.27778],125:[.25,.75,0,0,.5],126:[.35,.31786,0,0,.5],160:[0,0,0,0,.25],163:[0,.69444,0,0,.76909],167:[.19444,.69444,0,0,.44445],168:[0,.66786,0,0,.5],172:[0,.43056,0,0,.66667],176:[0,.69444,0,0,.75],177:[.08333,.58333,0,0,.77778],182:[.19444,.69444,0,0,.61111],184:[.17014,0,0,0,.44445],198:[0,.68333,0,0,.90278],215:[.08333,.58333,0,0,.77778],216:[.04861,.73194,0,0,.77778],223:[0,.69444,0,0,.5],230:[0,.43056,0,0,.72222],247:[.08333,.58333,0,0,.77778],248:[.09722,.52778,0,0,.5],305:[0,.43056,0,0,.27778],338:[0,.68333,0,0,1.01389],339:[0,.43056,0,0,.77778],567:[.19444,.43056,0,0,.30556],710:[0,.69444,0,0,.5],711:[0,.62847,0,0,.5],713:[0,.56778,0,0,.5],714:[0,.69444,0,0,.5],715:[0,.69444,0,0,.5],728:[0,.69444,0,0,.5],729:[0,.66786,0,0,.27778],730:[0,.69444,0,0,.75],732:[0,.66786,0,0,.5],733:[0,.69444,0,0,.5],915:[0,.68333,0,0,.625],916:[0,.68333,0,0,.83334],920:[0,.68333,0,0,.77778],923:[0,.68333,0,0,.69445],926:[0,.68333,0,0,.66667],928:[0,.68333,0,0,.75],931:[0,.68333,0,0,.72222],933:[0,.68333,0,0,.77778],934:[0,.68333,0,0,.72222],936:[0,.68333,0,0,.77778],937:[0,.68333,0,0,.72222],8211:[0,.43056,.02778,0,.5],8212:[0,.43056,.02778,0,1],8216:[0,.69444,0,0,.27778],8217:[0,.69444,0,0,.27778],8220:[0,.69444,0,0,.5],8221:[0,.69444,0,0,.5],8224:[.19444,.69444,0,0,.44445],8225:[.19444,.69444,0,0,.44445],8230:[0,.123,0,0,1.172],8242:[0,.55556,0,0,.275],8407:[0,.71444,.15382,0,.5],8463:[0,.68889,0,0,.54028],8465:[0,.69444,0,0,.72222],8467:[0,.69444,0,.11111,.41667],8472:[.19444,.43056,0,.11111,.63646],8476:[0,.69444,0,0,.72222],8501:[0,.69444,0,0,.61111],8592:[-.13313,.36687,0,0,1],8593:[.19444,.69444,0,0,.5],8594:[-.13313,.36687,0,0,1],8595:[.19444,.69444,0,0,.5],8596:[-.13313,.36687,0,0,1],8597:[.25,.75,0,0,.5],8598:[.19444,.69444,0,0,1],8599:[.19444,.69444,0,0,1],8600:[.19444,.69444,0,0,1],8601:[.19444,.69444,0,0,1],8614:[.011,.511,0,0,1],8617:[.011,.511,0,0,1.126],8618:[.011,.511,0,0,1.126],8636:[-.13313,.36687,0,0,1],8637:[-.13313,.36687,0,0,1],8640:[-.13313,.36687,0,0,1],8641:[-.13313,.36687,0,0,1],8652:[.011,.671,0,0,1],8656:[-.13313,.36687,0,0,1],8657:[.19444,.69444,0,0,.61111],8658:[-.13313,.36687,0,0,1],8659:[.19444,.69444,0,0,.61111],8660:[-.13313,.36687,0,0,1],8661:[.25,.75,0,0,.61111],8704:[0,.69444,0,0,.55556],8706:[0,.69444,.05556,.08334,.5309],8707:[0,.69444,0,0,.55556],8709:[.05556,.75,0,0,.5],8711:[0,.68333,0,0,.83334],8712:[.0391,.5391,0,0,.66667],8715:[.0391,.5391,0,0,.66667],8722:[.08333,.58333,0,0,.77778],8723:[.08333,.58333,0,0,.77778],8725:[.25,.75,0,0,.5],8726:[.25,.75,0,0,.5],8727:[-.03472,.46528,0,0,.5],8728:[-.05555,.44445,0,0,.5],8729:[-.05555,.44445,0,0,.5],8730:[.2,.8,0,0,.83334],8733:[0,.43056,0,0,.77778],8734:[0,.43056,0,0,1],8736:[0,.69224,0,0,.72222],8739:[.25,.75,0,0,.27778],8741:[.25,.75,0,0,.5],8743:[0,.55556,0,0,.66667],8744:[0,.55556,0,0,.66667],8745:[0,.55556,0,0,.66667],8746:[0,.55556,0,0,.66667],8747:[.19444,.69444,.11111,0,.41667],8764:[-.13313,.36687,0,0,.77778],8768:[.19444,.69444,0,0,.27778],8771:[-.03625,.46375,0,0,.77778],8773:[-.022,.589,0,0,.778],8776:[-.01688,.48312,0,0,.77778],8781:[-.03625,.46375,0,0,.77778],8784:[-.133,.673,0,0,.778],8801:[-.03625,.46375,0,0,.77778],8804:[.13597,.63597,0,0,.77778],8805:[.13597,.63597,0,0,.77778],8810:[.0391,.5391,0,0,1],8811:[.0391,.5391,0,0,1],8826:[.0391,.5391,0,0,.77778],8827:[.0391,.5391,0,0,.77778],8834:[.0391,.5391,0,0,.77778],8835:[.0391,.5391,0,0,.77778],8838:[.13597,.63597,0,0,.77778],8839:[.13597,.63597,0,0,.77778],8846:[0,.55556,0,0,.66667],8849:[.13597,.63597,0,0,.77778],8850:[.13597,.63597,0,0,.77778],8851:[0,.55556,0,0,.66667],8852:[0,.55556,0,0,.66667],8853:[.08333,.58333,0,0,.77778],8854:[.08333,.58333,0,0,.77778],8855:[.08333,.58333,0,0,.77778],8856:[.08333,.58333,0,0,.77778],8857:[.08333,.58333,0,0,.77778],8866:[0,.69444,0,0,.61111],8867:[0,.69444,0,0,.61111],8868:[0,.69444,0,0,.77778],8869:[0,.69444,0,0,.77778],8872:[.249,.75,0,0,.867],8900:[-.05555,.44445,0,0,.5],8901:[-.05555,.44445,0,0,.27778],8902:[-.03472,.46528,0,0,.5],8904:[.005,.505,0,0,.9],8942:[.03,.903,0,0,.278],8943:[-.19,.313,0,0,1.172],8945:[-.1,.823,0,0,1.282],8968:[.25,.75,0,0,.44445],8969:[.25,.75,0,0,.44445],8970:[.25,.75,0,0,.44445],8971:[.25,.75,0,0,.44445],8994:[-.14236,.35764,0,0,1],8995:[-.14236,.35764,0,0,1],9136:[.244,.744,0,0,.412],9137:[.244,.745,0,0,.412],9651:[.19444,.69444,0,0,.88889],9657:[-.03472,.46528,0,0,.5],9661:[.19444,.69444,0,0,.88889],9667:[-.03472,.46528,0,0,.5],9711:[.19444,.69444,0,0,1],9824:[.12963,.69444,0,0,.77778],9825:[.12963,.69444,0,0,.77778],9826:[.12963,.69444,0,0,.77778],9827:[.12963,.69444,0,0,.77778],9837:[0,.75,0,0,.38889],9838:[.19444,.69444,0,0,.38889],9839:[.19444,.69444,0,0,.38889],10216:[.25,.75,0,0,.38889],10217:[.25,.75,0,0,.38889],10222:[.244,.744,0,0,.412],10223:[.244,.745,0,0,.412],10229:[.011,.511,0,0,1.609],10230:[.011,.511,0,0,1.638],10231:[.011,.511,0,0,1.859],10232:[.024,.525,0,0,1.609],10233:[.024,.525,0,0,1.638],10234:[.024,.525,0,0,1.858],10236:[.011,.511,0,0,1.638],10815:[0,.68333,0,0,.75],10927:[.13597,.63597,0,0,.77778],10928:[.13597,.63597,0,0,.77778],57376:[.19444,.69444,0,0,0]},"Math-BoldItalic":{32:[0,0,0,0,.25],48:[0,.44444,0,0,.575],49:[0,.44444,0,0,.575],50:[0,.44444,0,0,.575],51:[.19444,.44444,0,0,.575],52:[.19444,.44444,0,0,.575],53:[.19444,.44444,0,0,.575],54:[0,.64444,0,0,.575],55:[.19444,.44444,0,0,.575],56:[0,.64444,0,0,.575],57:[.19444,.44444,0,0,.575],65:[0,.68611,0,0,.86944],66:[0,.68611,.04835,0,.8664],67:[0,.68611,.06979,0,.81694],68:[0,.68611,.03194,0,.93812],69:[0,.68611,.05451,0,.81007],70:[0,.68611,.15972,0,.68889],71:[0,.68611,0,0,.88673],72:[0,.68611,.08229,0,.98229],73:[0,.68611,.07778,0,.51111],74:[0,.68611,.10069,0,.63125],75:[0,.68611,.06979,0,.97118],76:[0,.68611,0,0,.75555],77:[0,.68611,.11424,0,1.14201],78:[0,.68611,.11424,0,.95034],79:[0,.68611,.03194,0,.83666],80:[0,.68611,.15972,0,.72309],81:[.19444,.68611,0,0,.86861],82:[0,.68611,.00421,0,.87235],83:[0,.68611,.05382,0,.69271],84:[0,.68611,.15972,0,.63663],85:[0,.68611,.11424,0,.80027],86:[0,.68611,.25555,0,.67778],87:[0,.68611,.15972,0,1.09305],88:[0,.68611,.07778,0,.94722],89:[0,.68611,.25555,0,.67458],90:[0,.68611,.06979,0,.77257],97:[0,.44444,0,0,.63287],98:[0,.69444,0,0,.52083],99:[0,.44444,0,0,.51342],100:[0,.69444,0,0,.60972],101:[0,.44444,0,0,.55361],102:[.19444,.69444,.11042,0,.56806],103:[.19444,.44444,.03704,0,.5449],104:[0,.69444,0,0,.66759],105:[0,.69326,0,0,.4048],106:[.19444,.69326,.0622,0,.47083],107:[0,.69444,.01852,0,.6037],108:[0,.69444,.0088,0,.34815],109:[0,.44444,0,0,1.0324],110:[0,.44444,0,0,.71296],111:[0,.44444,0,0,.58472],112:[.19444,.44444,0,0,.60092],113:[.19444,.44444,.03704,0,.54213],114:[0,.44444,.03194,0,.5287],115:[0,.44444,0,0,.53125],116:[0,.63492,0,0,.41528],117:[0,.44444,0,0,.68102],118:[0,.44444,.03704,0,.56666],119:[0,.44444,.02778,0,.83148],120:[0,.44444,0,0,.65903],121:[.19444,.44444,.03704,0,.59028],122:[0,.44444,.04213,0,.55509],160:[0,0,0,0,.25],915:[0,.68611,.15972,0,.65694],916:[0,.68611,0,0,.95833],920:[0,.68611,.03194,0,.86722],923:[0,.68611,0,0,.80555],926:[0,.68611,.07458,0,.84125],928:[0,.68611,.08229,0,.98229],931:[0,.68611,.05451,0,.88507],933:[0,.68611,.15972,0,.67083],934:[0,.68611,0,0,.76666],936:[0,.68611,.11653,0,.71402],937:[0,.68611,.04835,0,.8789],945:[0,.44444,0,0,.76064],946:[.19444,.69444,.03403,0,.65972],947:[.19444,.44444,.06389,0,.59003],948:[0,.69444,.03819,0,.52222],949:[0,.44444,0,0,.52882],950:[.19444,.69444,.06215,0,.50833],951:[.19444,.44444,.03704,0,.6],952:[0,.69444,.03194,0,.5618],953:[0,.44444,0,0,.41204],954:[0,.44444,0,0,.66759],955:[0,.69444,0,0,.67083],956:[.19444,.44444,0,0,.70787],957:[0,.44444,.06898,0,.57685],958:[.19444,.69444,.03021,0,.50833],959:[0,.44444,0,0,.58472],960:[0,.44444,.03704,0,.68241],961:[.19444,.44444,0,0,.6118],962:[.09722,.44444,.07917,0,.42361],963:[0,.44444,.03704,0,.68588],964:[0,.44444,.13472,0,.52083],965:[0,.44444,.03704,0,.63055],966:[.19444,.44444,0,0,.74722],967:[.19444,.44444,0,0,.71805],968:[.19444,.69444,.03704,0,.75833],969:[0,.44444,.03704,0,.71782],977:[0,.69444,0,0,.69155],981:[.19444,.69444,0,0,.7125],982:[0,.44444,.03194,0,.975],1009:[.19444,.44444,0,0,.6118],1013:[0,.44444,0,0,.48333],57649:[0,.44444,0,0,.39352],57911:[.19444,.44444,0,0,.43889]},"Math-Italic":{32:[0,0,0,0,.25],48:[0,.43056,0,0,.5],49:[0,.43056,0,0,.5],50:[0,.43056,0,0,.5],51:[.19444,.43056,0,0,.5],52:[.19444,.43056,0,0,.5],53:[.19444,.43056,0,0,.5],54:[0,.64444,0,0,.5],55:[.19444,.43056,0,0,.5],56:[0,.64444,0,0,.5],57:[.19444,.43056,0,0,.5],65:[0,.68333,0,.13889,.75],66:[0,.68333,.05017,.08334,.75851],67:[0,.68333,.07153,.08334,.71472],68:[0,.68333,.02778,.05556,.82792],69:[0,.68333,.05764,.08334,.7382],70:[0,.68333,.13889,.08334,.64306],71:[0,.68333,0,.08334,.78625],72:[0,.68333,.08125,.05556,.83125],73:[0,.68333,.07847,.11111,.43958],74:[0,.68333,.09618,.16667,.55451],75:[0,.68333,.07153,.05556,.84931],76:[0,.68333,0,.02778,.68056],77:[0,.68333,.10903,.08334,.97014],78:[0,.68333,.10903,.08334,.80347],79:[0,.68333,.02778,.08334,.76278],80:[0,.68333,.13889,.08334,.64201],81:[.19444,.68333,0,.08334,.79056],82:[0,.68333,.00773,.08334,.75929],83:[0,.68333,.05764,.08334,.6132],84:[0,.68333,.13889,.08334,.58438],85:[0,.68333,.10903,.02778,.68278],86:[0,.68333,.22222,0,.58333],87:[0,.68333,.13889,0,.94445],88:[0,.68333,.07847,.08334,.82847],89:[0,.68333,.22222,0,.58056],90:[0,.68333,.07153,.08334,.68264],97:[0,.43056,0,0,.52859],98:[0,.69444,0,0,.42917],99:[0,.43056,0,.05556,.43276],100:[0,.69444,0,.16667,.52049],101:[0,.43056,0,.05556,.46563],102:[.19444,.69444,.10764,.16667,.48959],103:[.19444,.43056,.03588,.02778,.47697],104:[0,.69444,0,0,.57616],105:[0,.65952,0,0,.34451],106:[.19444,.65952,.05724,0,.41181],107:[0,.69444,.03148,0,.5206],108:[0,.69444,.01968,.08334,.29838],109:[0,.43056,0,0,.87801],110:[0,.43056,0,0,.60023],111:[0,.43056,0,.05556,.48472],112:[.19444,.43056,0,.08334,.50313],113:[.19444,.43056,.03588,.08334,.44641],114:[0,.43056,.02778,.05556,.45116],115:[0,.43056,0,.05556,.46875],116:[0,.61508,0,.08334,.36111],117:[0,.43056,0,.02778,.57246],118:[0,.43056,.03588,.02778,.48472],119:[0,.43056,.02691,.08334,.71592],120:[0,.43056,0,.02778,.57153],121:[.19444,.43056,.03588,.05556,.49028],122:[0,.43056,.04398,.05556,.46505],160:[0,0,0,0,.25],915:[0,.68333,.13889,.08334,.61528],916:[0,.68333,0,.16667,.83334],920:[0,.68333,.02778,.08334,.76278],923:[0,.68333,0,.16667,.69445],926:[0,.68333,.07569,.08334,.74236],928:[0,.68333,.08125,.05556,.83125],931:[0,.68333,.05764,.08334,.77986],933:[0,.68333,.13889,.05556,.58333],934:[0,.68333,0,.08334,.66667],936:[0,.68333,.11,.05556,.61222],937:[0,.68333,.05017,.08334,.7724],945:[0,.43056,.0037,.02778,.6397],946:[.19444,.69444,.05278,.08334,.56563],947:[.19444,.43056,.05556,0,.51773],948:[0,.69444,.03785,.05556,.44444],949:[0,.43056,0,.08334,.46632],950:[.19444,.69444,.07378,.08334,.4375],951:[.19444,.43056,.03588,.05556,.49653],952:[0,.69444,.02778,.08334,.46944],953:[0,.43056,0,.05556,.35394],954:[0,.43056,0,0,.57616],955:[0,.69444,0,0,.58334],956:[.19444,.43056,0,.02778,.60255],957:[0,.43056,.06366,.02778,.49398],958:[.19444,.69444,.04601,.11111,.4375],959:[0,.43056,0,.05556,.48472],960:[0,.43056,.03588,0,.57003],961:[.19444,.43056,0,.08334,.51702],962:[.09722,.43056,.07986,.08334,.36285],963:[0,.43056,.03588,0,.57141],964:[0,.43056,.1132,.02778,.43715],965:[0,.43056,.03588,.02778,.54028],966:[.19444,.43056,0,.08334,.65417],967:[.19444,.43056,0,.05556,.62569],968:[.19444,.69444,.03588,.11111,.65139],969:[0,.43056,.03588,0,.62245],977:[0,.69444,0,.08334,.59144],981:[.19444,.69444,0,.08334,.59583],982:[0,.43056,.02778,0,.82813],1009:[.19444,.43056,0,.08334,.51702],1013:[0,.43056,0,.05556,.4059],57649:[0,.43056,0,.02778,.32246],57911:[.19444,.43056,0,.08334,.38403]},"SansSerif-Bold":{32:[0,0,0,0,.25],33:[0,.69444,0,0,.36667],34:[0,.69444,0,0,.55834],35:[.19444,.69444,0,0,.91667],36:[.05556,.75,0,0,.55],37:[.05556,.75,0,0,1.02912],38:[0,.69444,0,0,.83056],39:[0,.69444,0,0,.30556],40:[.25,.75,0,0,.42778],41:[.25,.75,0,0,.42778],42:[0,.75,0,0,.55],43:[.11667,.61667,0,0,.85556],44:[.10556,.13056,0,0,.30556],45:[0,.45833,0,0,.36667],46:[0,.13056,0,0,.30556],47:[.25,.75,0,0,.55],48:[0,.69444,0,0,.55],49:[0,.69444,0,0,.55],50:[0,.69444,0,0,.55],51:[0,.69444,0,0,.55],52:[0,.69444,0,0,.55],53:[0,.69444,0,0,.55],54:[0,.69444,0,0,.55],55:[0,.69444,0,0,.55],56:[0,.69444,0,0,.55],57:[0,.69444,0,0,.55],58:[0,.45833,0,0,.30556],59:[.10556,.45833,0,0,.30556],61:[-.09375,.40625,0,0,.85556],63:[0,.69444,0,0,.51945],64:[0,.69444,0,0,.73334],65:[0,.69444,0,0,.73334],66:[0,.69444,0,0,.73334],67:[0,.69444,0,0,.70278],68:[0,.69444,0,0,.79445],69:[0,.69444,0,0,.64167],70:[0,.69444,0,0,.61111],71:[0,.69444,0,0,.73334],72:[0,.69444,0,0,.79445],73:[0,.69444,0,0,.33056],74:[0,.69444,0,0,.51945],75:[0,.69444,0,0,.76389],76:[0,.69444,0,0,.58056],77:[0,.69444,0,0,.97778],78:[0,.69444,0,0,.79445],79:[0,.69444,0,0,.79445],80:[0,.69444,0,0,.70278],81:[.10556,.69444,0,0,.79445],82:[0,.69444,0,0,.70278],83:[0,.69444,0,0,.61111],84:[0,.69444,0,0,.73334],85:[0,.69444,0,0,.76389],86:[0,.69444,.01528,0,.73334],87:[0,.69444,.01528,0,1.03889],88:[0,.69444,0,0,.73334],89:[0,.69444,.0275,0,.73334],90:[0,.69444,0,0,.67223],91:[.25,.75,0,0,.34306],93:[.25,.75,0,0,.34306],94:[0,.69444,0,0,.55],95:[.35,.10833,.03056,0,.55],97:[0,.45833,0,0,.525],98:[0,.69444,0,0,.56111],99:[0,.45833,0,0,.48889],100:[0,.69444,0,0,.56111],101:[0,.45833,0,0,.51111],102:[0,.69444,.07639,0,.33611],103:[.19444,.45833,.01528,0,.55],104:[0,.69444,0,0,.56111],105:[0,.69444,0,0,.25556],106:[.19444,.69444,0,0,.28611],107:[0,.69444,0,0,.53056],108:[0,.69444,0,0,.25556],109:[0,.45833,0,0,.86667],110:[0,.45833,0,0,.56111],111:[0,.45833,0,0,.55],112:[.19444,.45833,0,0,.56111],113:[.19444,.45833,0,0,.56111],114:[0,.45833,.01528,0,.37222],115:[0,.45833,0,0,.42167],116:[0,.58929,0,0,.40417],117:[0,.45833,0,0,.56111],118:[0,.45833,.01528,0,.5],119:[0,.45833,.01528,0,.74445],120:[0,.45833,0,0,.5],121:[.19444,.45833,.01528,0,.5],122:[0,.45833,0,0,.47639],126:[.35,.34444,0,0,.55],160:[0,0,0,0,.25],168:[0,.69444,0,0,.55],176:[0,.69444,0,0,.73334],180:[0,.69444,0,0,.55],184:[.17014,0,0,0,.48889],305:[0,.45833,0,0,.25556],567:[.19444,.45833,0,0,.28611],710:[0,.69444,0,0,.55],711:[0,.63542,0,0,.55],713:[0,.63778,0,0,.55],728:[0,.69444,0,0,.55],729:[0,.69444,0,0,.30556],730:[0,.69444,0,0,.73334],732:[0,.69444,0,0,.55],733:[0,.69444,0,0,.55],915:[0,.69444,0,0,.58056],916:[0,.69444,0,0,.91667],920:[0,.69444,0,0,.85556],923:[0,.69444,0,0,.67223],926:[0,.69444,0,0,.73334],928:[0,.69444,0,0,.79445],931:[0,.69444,0,0,.79445],933:[0,.69444,0,0,.85556],934:[0,.69444,0,0,.79445],936:[0,.69444,0,0,.85556],937:[0,.69444,0,0,.79445],8211:[0,.45833,.03056,0,.55],8212:[0,.45833,.03056,0,1.10001],8216:[0,.69444,0,0,.30556],8217:[0,.69444,0,0,.30556],8220:[0,.69444,0,0,.55834],8221:[0,.69444,0,0,.55834]},"SansSerif-Italic":{32:[0,0,0,0,.25],33:[0,.69444,.05733,0,.31945],34:[0,.69444,.00316,0,.5],35:[.19444,.69444,.05087,0,.83334],36:[.05556,.75,.11156,0,.5],37:[.05556,.75,.03126,0,.83334],38:[0,.69444,.03058,0,.75834],39:[0,.69444,.07816,0,.27778],40:[.25,.75,.13164,0,.38889],41:[.25,.75,.02536,0,.38889],42:[0,.75,.11775,0,.5],43:[.08333,.58333,.02536,0,.77778],44:[.125,.08333,0,0,.27778],45:[0,.44444,.01946,0,.33333],46:[0,.08333,0,0,.27778],47:[.25,.75,.13164,0,.5],48:[0,.65556,.11156,0,.5],49:[0,.65556,.11156,0,.5],50:[0,.65556,.11156,0,.5],51:[0,.65556,.11156,0,.5],52:[0,.65556,.11156,0,.5],53:[0,.65556,.11156,0,.5],54:[0,.65556,.11156,0,.5],55:[0,.65556,.11156,0,.5],56:[0,.65556,.11156,0,.5],57:[0,.65556,.11156,0,.5],58:[0,.44444,.02502,0,.27778],59:[.125,.44444,.02502,0,.27778],61:[-.13,.37,.05087,0,.77778],63:[0,.69444,.11809,0,.47222],64:[0,.69444,.07555,0,.66667],65:[0,.69444,0,0,.66667],66:[0,.69444,.08293,0,.66667],67:[0,.69444,.11983,0,.63889],68:[0,.69444,.07555,0,.72223],69:[0,.69444,.11983,0,.59722],70:[0,.69444,.13372,0,.56945],71:[0,.69444,.11983,0,.66667],72:[0,.69444,.08094,0,.70834],73:[0,.69444,.13372,0,.27778],74:[0,.69444,.08094,0,.47222],75:[0,.69444,.11983,0,.69445],76:[0,.69444,0,0,.54167],77:[0,.69444,.08094,0,.875],78:[0,.69444,.08094,0,.70834],79:[0,.69444,.07555,0,.73611],80:[0,.69444,.08293,0,.63889],81:[.125,.69444,.07555,0,.73611],82:[0,.69444,.08293,0,.64584],83:[0,.69444,.09205,0,.55556],84:[0,.69444,.13372,0,.68056],85:[0,.69444,.08094,0,.6875],86:[0,.69444,.1615,0,.66667],87:[0,.69444,.1615,0,.94445],88:[0,.69444,.13372,0,.66667],89:[0,.69444,.17261,0,.66667],90:[0,.69444,.11983,0,.61111],91:[.25,.75,.15942,0,.28889],93:[.25,.75,.08719,0,.28889],94:[0,.69444,.0799,0,.5],95:[.35,.09444,.08616,0,.5],97:[0,.44444,.00981,0,.48056],98:[0,.69444,.03057,0,.51667],99:[0,.44444,.08336,0,.44445],100:[0,.69444,.09483,0,.51667],101:[0,.44444,.06778,0,.44445],102:[0,.69444,.21705,0,.30556],103:[.19444,.44444,.10836,0,.5],104:[0,.69444,.01778,0,.51667],105:[0,.67937,.09718,0,.23889],106:[.19444,.67937,.09162,0,.26667],107:[0,.69444,.08336,0,.48889],108:[0,.69444,.09483,0,.23889],109:[0,.44444,.01778,0,.79445],110:[0,.44444,.01778,0,.51667],111:[0,.44444,.06613,0,.5],112:[.19444,.44444,.0389,0,.51667],113:[.19444,.44444,.04169,0,.51667],114:[0,.44444,.10836,0,.34167],115:[0,.44444,.0778,0,.38333],116:[0,.57143,.07225,0,.36111],117:[0,.44444,.04169,0,.51667],118:[0,.44444,.10836,0,.46111],119:[0,.44444,.10836,0,.68334],120:[0,.44444,.09169,0,.46111],121:[.19444,.44444,.10836,0,.46111],122:[0,.44444,.08752,0,.43472],126:[.35,.32659,.08826,0,.5],160:[0,0,0,0,.25],168:[0,.67937,.06385,0,.5],176:[0,.69444,0,0,.73752],184:[.17014,0,0,0,.44445],305:[0,.44444,.04169,0,.23889],567:[.19444,.44444,.04169,0,.26667],710:[0,.69444,.0799,0,.5],711:[0,.63194,.08432,0,.5],713:[0,.60889,.08776,0,.5],714:[0,.69444,.09205,0,.5],715:[0,.69444,0,0,.5],728:[0,.69444,.09483,0,.5],729:[0,.67937,.07774,0,.27778],730:[0,.69444,0,0,.73752],732:[0,.67659,.08826,0,.5],733:[0,.69444,.09205,0,.5],915:[0,.69444,.13372,0,.54167],916:[0,.69444,0,0,.83334],920:[0,.69444,.07555,0,.77778],923:[0,.69444,0,0,.61111],926:[0,.69444,.12816,0,.66667],928:[0,.69444,.08094,0,.70834],931:[0,.69444,.11983,0,.72222],933:[0,.69444,.09031,0,.77778],934:[0,.69444,.04603,0,.72222],936:[0,.69444,.09031,0,.77778],937:[0,.69444,.08293,0,.72222],8211:[0,.44444,.08616,0,.5],8212:[0,.44444,.08616,0,1],8216:[0,.69444,.07816,0,.27778],8217:[0,.69444,.07816,0,.27778],8220:[0,.69444,.14205,0,.5],8221:[0,.69444,.00316,0,.5]},"SansSerif-Regular":{32:[0,0,0,0,.25],33:[0,.69444,0,0,.31945],34:[0,.69444,0,0,.5],35:[.19444,.69444,0,0,.83334],36:[.05556,.75,0,0,.5],37:[.05556,.75,0,0,.83334],38:[0,.69444,0,0,.75834],39:[0,.69444,0,0,.27778],40:[.25,.75,0,0,.38889],41:[.25,.75,0,0,.38889],42:[0,.75,0,0,.5],43:[.08333,.58333,0,0,.77778],44:[.125,.08333,0,0,.27778],45:[0,.44444,0,0,.33333],46:[0,.08333,0,0,.27778],47:[.25,.75,0,0,.5],48:[0,.65556,0,0,.5],49:[0,.65556,0,0,.5],50:[0,.65556,0,0,.5],51:[0,.65556,0,0,.5],52:[0,.65556,0,0,.5],53:[0,.65556,0,0,.5],54:[0,.65556,0,0,.5],55:[0,.65556,0,0,.5],56:[0,.65556,0,0,.5],57:[0,.65556,0,0,.5],58:[0,.44444,0,0,.27778],59:[.125,.44444,0,0,.27778],61:[-.13,.37,0,0,.77778],63:[0,.69444,0,0,.47222],64:[0,.69444,0,0,.66667],65:[0,.69444,0,0,.66667],66:[0,.69444,0,0,.66667],67:[0,.69444,0,0,.63889],68:[0,.69444,0,0,.72223],69:[0,.69444,0,0,.59722],70:[0,.69444,0,0,.56945],71:[0,.69444,0,0,.66667],72:[0,.69444,0,0,.70834],73:[0,.69444,0,0,.27778],74:[0,.69444,0,0,.47222],75:[0,.69444,0,0,.69445],76:[0,.69444,0,0,.54167],77:[0,.69444,0,0,.875],78:[0,.69444,0,0,.70834],79:[0,.69444,0,0,.73611],80:[0,.69444,0,0,.63889],81:[.125,.69444,0,0,.73611],82:[0,.69444,0,0,.64584],83:[0,.69444,0,0,.55556],84:[0,.69444,0,0,.68056],85:[0,.69444,0,0,.6875],86:[0,.69444,.01389,0,.66667],87:[0,.69444,.01389,0,.94445],88:[0,.69444,0,0,.66667],89:[0,.69444,.025,0,.66667],90:[0,.69444,0,0,.61111],91:[.25,.75,0,0,.28889],93:[.25,.75,0,0,.28889],94:[0,.69444,0,0,.5],95:[.35,.09444,.02778,0,.5],97:[0,.44444,0,0,.48056],98:[0,.69444,0,0,.51667],99:[0,.44444,0,0,.44445],100:[0,.69444,0,0,.51667],101:[0,.44444,0,0,.44445],102:[0,.69444,.06944,0,.30556],103:[.19444,.44444,.01389,0,.5],104:[0,.69444,0,0,.51667],105:[0,.67937,0,0,.23889],106:[.19444,.67937,0,0,.26667],107:[0,.69444,0,0,.48889],108:[0,.69444,0,0,.23889],109:[0,.44444,0,0,.79445],110:[0,.44444,0,0,.51667],111:[0,.44444,0,0,.5],112:[.19444,.44444,0,0,.51667],113:[.19444,.44444,0,0,.51667],114:[0,.44444,.01389,0,.34167],115:[0,.44444,0,0,.38333],116:[0,.57143,0,0,.36111],117:[0,.44444,0,0,.51667],118:[0,.44444,.01389,0,.46111],119:[0,.44444,.01389,0,.68334],120:[0,.44444,0,0,.46111],121:[.19444,.44444,.01389,0,.46111],122:[0,.44444,0,0,.43472],126:[.35,.32659,0,0,.5],160:[0,0,0,0,.25],168:[0,.67937,0,0,.5],176:[0,.69444,0,0,.66667],184:[.17014,0,0,0,.44445],305:[0,.44444,0,0,.23889],567:[.19444,.44444,0,0,.26667],710:[0,.69444,0,0,.5],711:[0,.63194,0,0,.5],713:[0,.60889,0,0,.5],714:[0,.69444,0,0,.5],715:[0,.69444,0,0,.5],728:[0,.69444,0,0,.5],729:[0,.67937,0,0,.27778],730:[0,.69444,0,0,.66667],732:[0,.67659,0,0,.5],733:[0,.69444,0,0,.5],915:[0,.69444,0,0,.54167],916:[0,.69444,0,0,.83334],920:[0,.69444,0,0,.77778],923:[0,.69444,0,0,.61111],926:[0,.69444,0,0,.66667],928:[0,.69444,0,0,.70834],931:[0,.69444,0,0,.72222],933:[0,.69444,0,0,.77778],934:[0,.69444,0,0,.72222],936:[0,.69444,0,0,.77778],937:[0,.69444,0,0,.72222],8211:[0,.44444,.02778,0,.5],8212:[0,.44444,.02778,0,1],8216:[0,.69444,0,0,.27778],8217:[0,.69444,0,0,.27778],8220:[0,.69444,0,0,.5],8221:[0,.69444,0,0,.5]},"Script-Regular":{32:[0,0,0,0,.25],65:[0,.7,.22925,0,.80253],66:[0,.7,.04087,0,.90757],67:[0,.7,.1689,0,.66619],68:[0,.7,.09371,0,.77443],69:[0,.7,.18583,0,.56162],70:[0,.7,.13634,0,.89544],71:[0,.7,.17322,0,.60961],72:[0,.7,.29694,0,.96919],73:[0,.7,.19189,0,.80907],74:[.27778,.7,.19189,0,1.05159],75:[0,.7,.31259,0,.91364],76:[0,.7,.19189,0,.87373],77:[0,.7,.15981,0,1.08031],78:[0,.7,.3525,0,.9015],79:[0,.7,.08078,0,.73787],80:[0,.7,.08078,0,1.01262],81:[0,.7,.03305,0,.88282],82:[0,.7,.06259,0,.85],83:[0,.7,.19189,0,.86767],84:[0,.7,.29087,0,.74697],85:[0,.7,.25815,0,.79996],86:[0,.7,.27523,0,.62204],87:[0,.7,.27523,0,.80532],88:[0,.7,.26006,0,.94445],89:[0,.7,.2939,0,.70961],90:[0,.7,.24037,0,.8212],160:[0,0,0,0,.25]},"Size1-Regular":{32:[0,0,0,0,.25],40:[.35001,.85,0,0,.45834],41:[.35001,.85,0,0,.45834],47:[.35001,.85,0,0,.57778],91:[.35001,.85,0,0,.41667],92:[.35001,.85,0,0,.57778],93:[.35001,.85,0,0,.41667],123:[.35001,.85,0,0,.58334],125:[.35001,.85,0,0,.58334],160:[0,0,0,0,.25],710:[0,.72222,0,0,.55556],732:[0,.72222,0,0,.55556],770:[0,.72222,0,0,.55556],771:[0,.72222,0,0,.55556],8214:[-99e-5,.601,0,0,.77778],8593:[1e-5,.6,0,0,.66667],8595:[1e-5,.6,0,0,.66667],8657:[1e-5,.6,0,0,.77778],8659:[1e-5,.6,0,0,.77778],8719:[.25001,.75,0,0,.94445],8720:[.25001,.75,0,0,.94445],8721:[.25001,.75,0,0,1.05556],8730:[.35001,.85,0,0,1],8739:[-.00599,.606,0,0,.33333],8741:[-.00599,.606,0,0,.55556],8747:[.30612,.805,.19445,0,.47222],8748:[.306,.805,.19445,0,.47222],8749:[.306,.805,.19445,0,.47222],8750:[.30612,.805,.19445,0,.47222],8896:[.25001,.75,0,0,.83334],8897:[.25001,.75,0,0,.83334],8898:[.25001,.75,0,0,.83334],8899:[.25001,.75,0,0,.83334],8968:[.35001,.85,0,0,.47222],8969:[.35001,.85,0,0,.47222],8970:[.35001,.85,0,0,.47222],8971:[.35001,.85,0,0,.47222],9168:[-99e-5,.601,0,0,.66667],10216:[.35001,.85,0,0,.47222],10217:[.35001,.85,0,0,.47222],10752:[.25001,.75,0,0,1.11111],10753:[.25001,.75,0,0,1.11111],10754:[.25001,.75,0,0,1.11111],10756:[.25001,.75,0,0,.83334],10758:[.25001,.75,0,0,.83334]},"Size2-Regular":{32:[0,0,0,0,.25],40:[.65002,1.15,0,0,.59722],41:[.65002,1.15,0,0,.59722],47:[.65002,1.15,0,0,.81111],91:[.65002,1.15,0,0,.47222],92:[.65002,1.15,0,0,.81111],93:[.65002,1.15,0,0,.47222],123:[.65002,1.15,0,0,.66667],125:[.65002,1.15,0,0,.66667],160:[0,0,0,0,.25],710:[0,.75,0,0,1],732:[0,.75,0,0,1],770:[0,.75,0,0,1],771:[0,.75,0,0,1],8719:[.55001,1.05,0,0,1.27778],8720:[.55001,1.05,0,0,1.27778],8721:[.55001,1.05,0,0,1.44445],8730:[.65002,1.15,0,0,1],8747:[.86225,1.36,.44445,0,.55556],8748:[.862,1.36,.44445,0,.55556],8749:[.862,1.36,.44445,0,.55556],8750:[.86225,1.36,.44445,0,.55556],8896:[.55001,1.05,0,0,1.11111],8897:[.55001,1.05,0,0,1.11111],8898:[.55001,1.05,0,0,1.11111],8899:[.55001,1.05,0,0,1.11111],8968:[.65002,1.15,0,0,.52778],8969:[.65002,1.15,0,0,.52778],8970:[.65002,1.15,0,0,.52778],8971:[.65002,1.15,0,0,.52778],10216:[.65002,1.15,0,0,.61111],10217:[.65002,1.15,0,0,.61111],10752:[.55001,1.05,0,0,1.51112],10753:[.55001,1.05,0,0,1.51112],10754:[.55001,1.05,0,0,1.51112],10756:[.55001,1.05,0,0,1.11111],10758:[.55001,1.05,0,0,1.11111]},"Size3-Regular":{32:[0,0,0,0,.25],40:[.95003,1.45,0,0,.73611],41:[.95003,1.45,0,0,.73611],47:[.95003,1.45,0,0,1.04445],91:[.95003,1.45,0,0,.52778],92:[.95003,1.45,0,0,1.04445],93:[.95003,1.45,0,0,.52778],123:[.95003,1.45,0,0,.75],125:[.95003,1.45,0,0,.75],160:[0,0,0,0,.25],710:[0,.75,0,0,1.44445],732:[0,.75,0,0,1.44445],770:[0,.75,0,0,1.44445],771:[0,.75,0,0,1.44445],8730:[.95003,1.45,0,0,1],8968:[.95003,1.45,0,0,.58334],8969:[.95003,1.45,0,0,.58334],8970:[.95003,1.45,0,0,.58334],8971:[.95003,1.45,0,0,.58334],10216:[.95003,1.45,0,0,.75],10217:[.95003,1.45,0,0,.75]},"Size4-Regular":{32:[0,0,0,0,.25],40:[1.25003,1.75,0,0,.79167],41:[1.25003,1.75,0,0,.79167],47:[1.25003,1.75,0,0,1.27778],91:[1.25003,1.75,0,0,.58334],92:[1.25003,1.75,0,0,1.27778],93:[1.25003,1.75,0,0,.58334],123:[1.25003,1.75,0,0,.80556],125:[1.25003,1.75,0,0,.80556],160:[0,0,0,0,.25],710:[0,.825,0,0,1.8889],732:[0,.825,0,0,1.8889],770:[0,.825,0,0,1.8889],771:[0,.825,0,0,1.8889],8730:[1.25003,1.75,0,0,1],8968:[1.25003,1.75,0,0,.63889],8969:[1.25003,1.75,0,0,.63889],8970:[1.25003,1.75,0,0,.63889],8971:[1.25003,1.75,0,0,.63889],9115:[.64502,1.155,0,0,.875],9116:[1e-5,.6,0,0,.875],9117:[.64502,1.155,0,0,.875],9118:[.64502,1.155,0,0,.875],9119:[1e-5,.6,0,0,.875],9120:[.64502,1.155,0,0,.875],9121:[.64502,1.155,0,0,.66667],9122:[-99e-5,.601,0,0,.66667],9123:[.64502,1.155,0,0,.66667],9124:[.64502,1.155,0,0,.66667],9125:[-99e-5,.601,0,0,.66667],9126:[.64502,1.155,0,0,.66667],9127:[1e-5,.9,0,0,.88889],9128:[.65002,1.15,0,0,.88889],9129:[.90001,0,0,0,.88889],9130:[0,.3,0,0,.88889],9131:[1e-5,.9,0,0,.88889],9132:[.65002,1.15,0,0,.88889],9133:[.90001,0,0,0,.88889],9143:[.88502,.915,0,0,1.05556],10216:[1.25003,1.75,0,0,.80556],10217:[1.25003,1.75,0,0,.80556],57344:[-.00499,.605,0,0,1.05556],57345:[-.00499,.605,0,0,1.05556],57680:[0,.12,0,0,.45],57681:[0,.12,0,0,.45],57682:[0,.12,0,0,.45],57683:[0,.12,0,0,.45]},"Typewriter-Regular":{32:[0,0,0,0,.525],33:[0,.61111,0,0,.525],34:[0,.61111,0,0,.525],35:[0,.61111,0,0,.525],36:[.08333,.69444,0,0,.525],37:[.08333,.69444,0,0,.525],38:[0,.61111,0,0,.525],39:[0,.61111,0,0,.525],40:[.08333,.69444,0,0,.525],41:[.08333,.69444,0,0,.525],42:[0,.52083,0,0,.525],43:[-.08056,.53055,0,0,.525],44:[.13889,.125,0,0,.525],45:[-.08056,.53055,0,0,.525],46:[0,.125,0,0,.525],47:[.08333,.69444,0,0,.525],48:[0,.61111,0,0,.525],49:[0,.61111,0,0,.525],50:[0,.61111,0,0,.525],51:[0,.61111,0,0,.525],52:[0,.61111,0,0,.525],53:[0,.61111,0,0,.525],54:[0,.61111,0,0,.525],55:[0,.61111,0,0,.525],56:[0,.61111,0,0,.525],57:[0,.61111,0,0,.525],58:[0,.43056,0,0,.525],59:[.13889,.43056,0,0,.525],60:[-.05556,.55556,0,0,.525],61:[-.19549,.41562,0,0,.525],62:[-.05556,.55556,0,0,.525],63:[0,.61111,0,0,.525],64:[0,.61111,0,0,.525],65:[0,.61111,0,0,.525],66:[0,.61111,0,0,.525],67:[0,.61111,0,0,.525],68:[0,.61111,0,0,.525],69:[0,.61111,0,0,.525],70:[0,.61111,0,0,.525],71:[0,.61111,0,0,.525],72:[0,.61111,0,0,.525],73:[0,.61111,0,0,.525],74:[0,.61111,0,0,.525],75:[0,.61111,0,0,.525],76:[0,.61111,0,0,.525],77:[0,.61111,0,0,.525],78:[0,.61111,0,0,.525],79:[0,.61111,0,0,.525],80:[0,.61111,0,0,.525],81:[.13889,.61111,0,0,.525],82:[0,.61111,0,0,.525],83:[0,.61111,0,0,.525],84:[0,.61111,0,0,.525],85:[0,.61111,0,0,.525],86:[0,.61111,0,0,.525],87:[0,.61111,0,0,.525],88:[0,.61111,0,0,.525],89:[0,.61111,0,0,.525],90:[0,.61111,0,0,.525],91:[.08333,.69444,0,0,.525],92:[.08333,.69444,0,0,.525],93:[.08333,.69444,0,0,.525],94:[0,.61111,0,0,.525],95:[.09514,0,0,0,.525],96:[0,.61111,0,0,.525],97:[0,.43056,0,0,.525],98:[0,.61111,0,0,.525],99:[0,.43056,0,0,.525],100:[0,.61111,0,0,.525],101:[0,.43056,0,0,.525],102:[0,.61111,0,0,.525],103:[.22222,.43056,0,0,.525],104:[0,.61111,0,0,.525],105:[0,.61111,0,0,.525],106:[.22222,.61111,0,0,.525],107:[0,.61111,0,0,.525],108:[0,.61111,0,0,.525],109:[0,.43056,0,0,.525],110:[0,.43056,0,0,.525],111:[0,.43056,0,0,.525],112:[.22222,.43056,0,0,.525],113:[.22222,.43056,0,0,.525],114:[0,.43056,0,0,.525],115:[0,.43056,0,0,.525],116:[0,.55358,0,0,.525],117:[0,.43056,0,0,.525],118:[0,.43056,0,0,.525],119:[0,.43056,0,0,.525],120:[0,.43056,0,0,.525],121:[.22222,.43056,0,0,.525],122:[0,.43056,0,0,.525],123:[.08333,.69444,0,0,.525],124:[.08333,.69444,0,0,.525],125:[.08333,.69444,0,0,.525],126:[0,.61111,0,0,.525],127:[0,.61111,0,0,.525],160:[0,0,0,0,.525],176:[0,.61111,0,0,.525],184:[.19445,0,0,0,.525],305:[0,.43056,0,0,.525],567:[.22222,.43056,0,0,.525],711:[0,.56597,0,0,.525],713:[0,.56555,0,0,.525],714:[0,.61111,0,0,.525],715:[0,.61111,0,0,.525],728:[0,.61111,0,0,.525],730:[0,.61111,0,0,.525],770:[0,.61111,0,0,.525],771:[0,.61111,0,0,.525],776:[0,.61111,0,0,.525],915:[0,.61111,0,0,.525],916:[0,.61111,0,0,.525],920:[0,.61111,0,0,.525],923:[0,.61111,0,0,.525],926:[0,.61111,0,0,.525],928:[0,.61111,0,0,.525],931:[0,.61111,0,0,.525],933:[0,.61111,0,0,.525],934:[0,.61111,0,0,.525],936:[0,.61111,0,0,.525],937:[0,.61111,0,0,.525],8216:[0,.61111,0,0,.525],8217:[0,.61111,0,0,.525],8242:[0,.61111,0,0,.525],9251:[.11111,.21944,0,0,.525]}};const B={slant:[.25,.25,.25],space:[0,0,0],stretch:[0,0,0],shrink:[0,0,0],xHeight:[.431,.431,.431],quad:[1,1.171,1.472],extraSpace:[0,0,0],num1:[.677,.732,.925],num2:[.394,.384,.387],num3:[.444,.471,.504],denom1:[.686,.752,1.025],denom2:[.345,.344,.532],sup1:[.413,.503,.504],sup2:[.363,.431,.404],sup3:[.289,.286,.294],sub1:[.15,.143,.2],sub2:[.247,.286,.4],supDrop:[.386,.353,.494],subDrop:[.05,.071,.1],delim1:[2.39,1.7,1.98],delim2:[1.01,1.157,1.42],axisHeight:[.25,.25,.25],defaultRuleThickness:[.04,.049,.049],bigOpSpacing1:[.111,.111,.111],bigOpSpacing2:[.166,.166,.166],bigOpSpacing3:[.2,.2,.2],bigOpSpacing4:[.6,.611,.611],bigOpSpacing5:[.1,.143,.143],sqrtRuleThickness:[.04,.04,.04],ptPerEm:[10,10,10],doubleRuleSep:[.2,.2,.2],arrayRuleWidth:[.04,.04,.04],fboxsep:[.3,.3,.3],fboxrule:[.04,.04,.04]},C={"\xc5":"A","\xd0":"D","\xde":"o","\xe5":"a","\xf0":"d","\xfe":"o","\u0410":"A","\u0411":"B","\u0412":"B","\u0413":"F","\u0414":"A","\u0415":"E","\u0416":"K","\u0417":"3","\u0418":"N","\u0419":"N","\u041a":"K","\u041b":"N","\u041c":"M","\u041d":"H","\u041e":"O","\u041f":"N","\u0420":"P","\u0421":"C","\u0422":"T","\u0423":"y","\u0424":"O","\u0425":"X","\u0426":"U","\u0427":"h","\u0428":"W","\u0429":"W","\u042a":"B","\u042b":"X","\u042c":"B","\u042d":"3","\u042e":"X","\u042f":"R","\u0430":"a","\u0431":"b","\u0432":"a","\u0433":"r","\u0434":"y","\u0435":"e","\u0436":"m","\u0437":"e","\u0438":"n","\u0439":"n","\u043a":"n","\u043b":"n","\u043c":"m","\u043d":"n","\u043e":"o","\u043f":"n","\u0440":"p","\u0441":"c","\u0442":"o","\u0443":"y","\u0444":"b","\u0445":"x","\u0446":"n","\u0447":"n","\u0448":"w","\u0449":"w","\u044a":"a","\u044b":"m","\u044c":"a","\u044d":"e","\u044e":"m","\u044f":"r"};function N(e,t,r){if(!T[t])throw new Error("Font metrics not found for font: "+t+".");let n=e.charCodeAt(0),o=T[t][n];if(!o&&e[0]in C&&(n=C[e[0]].charCodeAt(0),o=T[t][n]),o||"text"!==r||S(n)&&(o=T[t][77]),o)return{depth:o[0],height:o[1],italic:o[2],skew:o[3],width:o[4]}}const q={};const I=[[1,1,1],[2,1,1],[3,1,1],[4,2,1],[5,2,1],[6,3,1],[7,4,2],[8,6,3],[9,7,6],[10,8,7],[11,10,9]],R=[.5,.6,.7,.8,.9,1,1.2,1.44,1.728,2.074,2.488],H=function(e,t){return t.size<2?e:I[e-1][t.size-1]};class O{constructor(e){this.style=void 0,this.color=void 0,this.size=void 0,this.textSize=void 0,this.phantom=void 0,this.font=void 0,this.fontFamily=void 0,this.fontWeight=void 0,this.fontShape=void 0,this.sizeMultiplier=void 0,this.maxSize=void 0,this.minRuleThickness=void 0,this._fontMetrics=void 0,this.style=e.style,this.color=e.color,this.size=e.size||O.BASESIZE,this.textSize=e.textSize||this.size,this.phantom=!!e.phantom,this.font=e.font||"",this.fontFamily=e.fontFamily||"",this.fontWeight=e.fontWeight||"",this.fontShape=e.fontShape||"",this.sizeMultiplier=R[this.size-1],this.maxSize=e.maxSize,this.minRuleThickness=e.minRuleThickness,this._fontMetrics=void 0}extend(e){const t={style:this.style,size:this.size,textSize:this.textSize,color:this.color,phantom:this.phantom,font:this.font,fontFamily:this.fontFamily,fontWeight:this.fontWeight,fontShape:this.fontShape,maxSize:this.maxSize,minRuleThickness:this.minRuleThickness};for(const r in e)e.hasOwnProperty(r)&&(t[r]=e[r]);return new O(t)}havingStyle(e){return this.style===e?this:this.extend({style:e,size:H(this.textSize,e)})}havingCrampedStyle(){return this.havingStyle(this.style.cramp())}havingSize(e){return this.size===e&&this.textSize===e?this:this.extend({style:this.style.text(),size:e,textSize:e,sizeMultiplier:R[e-1]})}havingBaseStyle(e){e=e||this.style.text();const t=H(O.BASESIZE,e);return this.size===t&&this.textSize===O.BASESIZE&&this.style===e?this:this.extend({style:e,size:t})}havingBaseSizing(){let e;switch(this.style.id){case 4:case 5:e=3;break;case 6:case 7:e=1;break;default:e=6}return this.extend({style:this.style.text(),size:e})}withColor(e){return this.extend({color:e})}withPhantom(){return this.extend({phantom:!0})}withFont(e){return this.extend({font:e})}withTextFontFamily(e){return this.extend({fontFamily:e,font:""})}withTextFontWeight(e){return this.extend({fontWeight:e,font:""})}withTextFontShape(e){return this.extend({fontShape:e,font:""})}sizingClasses(e){return e.size!==this.size?["sizing","reset-size"+e.size,"size"+this.size]:[]}baseSizingClasses(){return this.size!==O.BASESIZE?["sizing","reset-size"+this.size,"size"+O.BASESIZE]:[]}fontMetrics(){return this._fontMetrics||(this._fontMetrics=function(e){let t;if(t=e>=5?0:e>=3?1:2,!q[t]){const e=q[t]={cssEmPerMu:B.quad[t]/18};for(const r in B)B.hasOwnProperty(r)&&(e[r]=B[r][t])}return q[t]}(this.size)),this._fontMetrics}getColor(){return this.phantom?"transparent":this.color}}O.BASESIZE=6;var E=O;const L={pt:1,mm:7227/2540,cm:7227/254,in:72.27,bp:1.00375,pc:12,dd:1238/1157,cc:14856/1157,nd:685/642,nc:1370/107,sp:1/65536,px:1.00375},D={ex:!0,em:!0,mu:!0},V=function(e){return"string"!=typeof e&&(e=e.unit),e in L||e in D||"ex"===e},P=function(e,t){let r;if(e.unit in L)r=L[e.unit]/t.fontMetrics().ptPerEm/t.sizeMultiplier;else if("mu"===e.unit)r=t.fontMetrics().cssEmPerMu;else{let o;if(o=t.style.isTight()?t.havingStyle(t.style.text()):t,"ex"===e.unit)r=o.fontMetrics().xHeight;else{if("em"!==e.unit)throw new n("Invalid unit: '"+e.unit+"'");r=o.fontMetrics().quad}o!==t&&(r*=o.sizeMultiplier/t.sizeMultiplier)}return Math.min(e.number*r,t.maxSize)},F=function(e){return+e.toFixed(4)+"em"},G=function(e){return e.filter((e=>e)).join(" ")},U=function(e,t,r){if(this.classes=e||[],this.attributes={},this.height=0,this.depth=0,this.maxFontSize=0,this.style=r||{},t){t.style.isTight()&&this.classes.push("mtight");const e=t.getColor();e&&(this.style.color=e)}},Y=function(e){const t=document.createElement(e);t.className=G(this.classes);for(const e in this.style)this.style.hasOwnProperty(e)&&(t.style[e]=this.style[e]);for(const e in this.attributes)this.attributes.hasOwnProperty(e)&&t.setAttribute(e,this.attributes[e]);for(let e=0;e<this.children.length;e++)t.appendChild(this.children[e].toNode());return t},X=/[\s"'>/=\x00-\x1f]/,W=function(e){let t="<"+e;this.classes.length&&(t+=' class="'+l.escape(G(this.classes))+'"');let r="";for(const e in this.style)this.style.hasOwnProperty(e)&&(r+=l.hyphenate(e)+":"+this.style[e]+";");r&&(t+=' style="'+l.escape(r)+'"');for(const e in this.attributes)if(this.attributes.hasOwnProperty(e)){if(X.test(e))throw new n("Invalid attribute name '"+e+"'");t+=" "+e+'="'+l.escape(this.attributes[e])+'"'}t+=">";for(let e=0;e<this.children.length;e++)t+=this.children[e].toMarkup();return t+="</"+e+">",t};class _{constructor(e,t,r,n){this.children=void 0,this.attributes=void 0,this.classes=void 0,this.height=void 0,this.depth=void 0,this.width=void 0,this.maxFontSize=void 0,this.style=void 0,U.call(this,e,r,n),this.children=t||[]}setAttribute(e,t){this.attributes[e]=t}hasClass(e){return l.contains(this.classes,e)}toNode(){return Y.call(this,"span")}toMarkup(){return W.call(this,"span")}}class j{constructor(e,t,r,n){this.children=void 0,this.attributes=void 0,this.classes=void 0,this.height=void 0,this.depth=void 0,this.maxFontSize=void 0,this.style=void 0,U.call(this,t,n),this.children=r||[],this.setAttribute("href",e)}setAttribute(e,t){this.attributes[e]=t}hasClass(e){return l.contains(this.classes,e)}toNode(){return Y.call(this,"a")}toMarkup(){return W.call(this,"a")}}class ${constructor(e,t,r){this.src=void 0,this.alt=void 0,this.classes=void 0,this.height=void 0,this.depth=void 0,this.maxFontSize=void 0,this.style=void 0,this.alt=t,this.src=e,this.classes=["mord"],this.style=r}hasClass(e){return l.contains(this.classes,e)}toNode(){const e=document.createElement("img");e.src=this.src,e.alt=this.alt,e.className="mord";for(const t in this.style)this.style.hasOwnProperty(t)&&(e.style[t]=this.style[t]);return e}toMarkup(){let e='<img src="'+l.escape(this.src)+'" alt="'+l.escape(this.alt)+'"',t="";for(const e in this.style)this.style.hasOwnProperty(e)&&(t+=l.hyphenate(e)+":"+this.style[e]+";");return t&&(e+=' style="'+l.escape(t)+'"'),e+="'/>",e}}const Z={"\xee":"\u0131\u0302","\xef":"\u0131\u0308","\xed":"\u0131\u0301","\xec":"\u0131\u0300"};class K{constructor(e,t,r,n,o,s,i,a){this.text=void 0,this.height=void 0,this.depth=void 0,this.italic=void 0,this.skew=void 0,this.width=void 0,this.maxFontSize=void 0,this.classes=void 0,this.style=void 0,this.text=e,this.height=t||0,this.depth=r||0,this.italic=n||0,this.skew=o||0,this.width=s||0,this.classes=i||[],this.style=a||{},this.maxFontSize=0;const l=function(e){for(let t=0;t<v.length;t++){const r=v[t];for(let t=0;t<r.blocks.length;t++){const n=r.blocks[t];if(e>=n[0]&&e<=n[1])return r.name}}return null}(this.text.charCodeAt(0));l&&this.classes.push(l+"_fallback"),/[\xee\xef\xed\xec]/.test(this.text)&&(this.text=Z[this.text])}hasClass(e){return l.contains(this.classes,e)}toNode(){const e=document.createTextNode(this.text);let t=null;this.italic>0&&(t=document.createElement("span"),t.style.marginRight=F(this.italic)),this.classes.length>0&&(t=t||document.createElement("span"),t.className=G(this.classes));for(const e in this.style)this.style.hasOwnProperty(e)&&(t=t||document.createElement("span"),t.style[e]=this.style[e]);return t?(t.appendChild(e),t):e}toMarkup(){let e=!1,t="<span";this.classes.length&&(e=!0,t+=' class="',t+=l.escape(G(this.classes)),t+='"');let r="";this.italic>0&&(r+="margin-right:"+this.italic+"em;");for(const e in this.style)this.style.hasOwnProperty(e)&&(r+=l.hyphenate(e)+":"+this.style[e]+";");r&&(e=!0,t+=' style="'+l.escape(r)+'"');const n=l.escape(this.text);return e?(t+=">",t+=n,t+="</span>",t):n}}class J{constructor(e,t){this.children=void 0,this.attributes=void 0,this.children=e||[],this.attributes=t||{}}toNode(){const e=document.createElementNS("http://www.w3.org/2000/svg","svg");for(const t in this.attributes)Object.prototype.hasOwnProperty.call(this.attributes,t)&&e.setAttribute(t,this.attributes[t]);for(let t=0;t<this.children.length;t++)e.appendChild(this.children[t].toNode());return e}toMarkup(){let e='<svg xmlns="http://www.w3.org/2000/svg"';for(const t in this.attributes)Object.prototype.hasOwnProperty.call(this.attributes,t)&&(e+=" "+t+'="'+l.escape(this.attributes[t])+'"');e+=">";for(let t=0;t<this.children.length;t++)e+=this.children[t].toMarkup();return e+="</svg>",e}}class Q{constructor(e,t){this.pathName=void 0,this.alternate=void 0,this.pathName=e,this.alternate=t}toNode(){const e=document.createElementNS("http://www.w3.org/2000/svg","path");return this.alternate?e.setAttribute("d",this.alternate):e.setAttribute("d",z[this.pathName]),e}toMarkup(){return this.alternate?'<path d="'+l.escape(this.alternate)+'"/>':'<path d="'+l.escape(z[this.pathName])+'"/>'}}class ee{constructor(e){this.attributes=void 0,this.attributes=e||{}}toNode(){const e=document.createElementNS("http://www.w3.org/2000/svg","line");for(const t in this.attributes)Object.prototype.hasOwnProperty.call(this.attributes,t)&&e.setAttribute(t,this.attributes[t]);return e}toMarkup(){let e="<line";for(const t in this.attributes)Object.prototype.hasOwnProperty.call(this.attributes,t)&&(e+=" "+t+'="'+l.escape(this.attributes[t])+'"');return e+="/>",e}}function te(e){if(e instanceof K)return e;throw new Error("Expected symbolNode but got "+String(e)+".")}const re={bin:1,close:1,inner:1,open:1,punct:1,rel:1},ne={"accent-token":1,mathord:1,"op-token":1,spacing:1,textord:1},oe={math:{},text:{}};var se=oe;function ie(e,t,r,n,o,s){oe[e][o]={font:t,group:r,replace:n},s&&n&&(oe[e][n]=oe[e][o])}const ae="math",le="text",he="main",ce="ams",me="accent-token",pe="bin",ue="close",de="inner",ge="mathord",fe="op-token",be="open",ye="punct",xe="rel",we="spacing",ve="textord";ie(ae,he,xe,"\u2261","\\equiv",!0),ie(ae,he,xe,"\u227a","\\prec",!0),ie(ae,he,xe,"\u227b","\\succ",!0),ie(ae,he,xe,"\u223c","\\sim",!0),ie(ae,he,xe,"\u22a5","\\perp"),ie(ae,he,xe,"\u2aaf","\\preceq",!0),ie(ae,he,xe,"\u2ab0","\\succeq",!0),ie(ae,he,xe,"\u2243","\\simeq",!0),ie(ae,he,xe,"\u2223","\\mid",!0),ie(ae,he,xe,"\u226a","\\ll",!0),ie(ae,he,xe,"\u226b","\\gg",!0),ie(ae,he,xe,"\u224d","\\asymp",!0),ie(ae,he,xe,"\u2225","\\parallel"),ie(ae,he,xe,"\u22c8","\\bowtie",!0),ie(ae,he,xe,"\u2323","\\smile",!0),ie(ae,he,xe,"\u2291","\\sqsubseteq",!0),ie(ae,he,xe,"\u2292","\\sqsupseteq",!0),ie(ae,he,xe,"\u2250","\\doteq",!0),ie(ae,he,xe,"\u2322","\\frown",!0),ie(ae,he,xe,"\u220b","\\ni",!0),ie(ae,he,xe,"\u221d","\\propto",!0),ie(ae,he,xe,"\u22a2","\\vdash",!0),ie(ae,he,xe,"\u22a3","\\dashv",!0),ie(ae,he,xe,"\u220b","\\owns"),ie(ae,he,ye,".","\\ldotp"),ie(ae,he,ye,"\u22c5","\\cdotp"),ie(ae,he,ve,"#","\\#"),ie(le,he,ve,"#","\\#"),ie(ae,he,ve,"&","\\&"),ie(le,he,ve,"&","\\&"),ie(ae,he,ve,"\u2135","\\aleph",!0),ie(ae,he,ve,"\u2200","\\forall",!0),ie(ae,he,ve,"\u210f","\\hbar",!0),ie(ae,he,ve,"\u2203","\\exists",!0),ie(ae,he,ve,"\u2207","\\nabla",!0),ie(ae,he,ve,"\u266d","\\flat",!0),ie(ae,he,ve,"\u2113","\\ell",!0),ie(ae,he,ve,"\u266e","\\natural",!0),ie(ae,he,ve,"\u2663","\\clubsuit",!0),ie(ae,he,ve,"\u2118","\\wp",!0),ie(ae,he,ve,"\u266f","\\sharp",!0),ie(ae,he,ve,"\u2662","\\diamondsuit",!0),ie(ae,he,ve,"\u211c","\\Re",!0),ie(ae,he,ve,"\u2661","\\heartsuit",!0),ie(ae,he,ve,"\u2111","\\Im",!0),ie(ae,he,ve,"\u2660","\\spadesuit",!0),ie(ae,he,ve,"\xa7","\\S",!0),ie(le,he,ve,"\xa7","\\S"),ie(ae,he,ve,"\xb6","\\P",!0),ie(le,he,ve,"\xb6","\\P"),ie(ae,he,ve,"\u2020","\\dag"),ie(le,he,ve,"\u2020","\\dag"),ie(le,he,ve,"\u2020","\\textdagger"),ie(ae,he,ve,"\u2021","\\ddag"),ie(le,he,ve,"\u2021","\\ddag"),ie(le,he,ve,"\u2021","\\textdaggerdbl"),ie(ae,he,ue,"\u23b1","\\rmoustache",!0),ie(ae,he,be,"\u23b0","\\lmoustache",!0),ie(ae,he,ue,"\u27ef","\\rgroup",!0),ie(ae,he,be,"\u27ee","\\lgroup",!0),ie(ae,he,pe,"\u2213","\\mp",!0),ie(ae,he,pe,"\u2296","\\ominus",!0),ie(ae,he,pe,"\u228e","\\uplus",!0),ie(ae,he,pe,"\u2293","\\sqcap",!0),ie(ae,he,pe,"\u2217","\\ast"),ie(ae,he,pe,"\u2294","\\sqcup",!0),ie(ae,he,pe,"\u25ef","\\bigcirc",!0),ie(ae,he,pe,"\u2219","\\bullet",!0),ie(ae,he,pe,"\u2021","\\ddagger"),ie(ae,he,pe,"\u2240","\\wr",!0),ie(ae,he,pe,"\u2a3f","\\amalg"),ie(ae,he,pe,"&","\\And"),ie(ae,he,xe,"\u27f5","\\longleftarrow",!0),ie(ae,he,xe,"\u21d0","\\Leftarrow",!0),ie(ae,he,xe,"\u27f8","\\Longleftarrow",!0),ie(ae,he,xe,"\u27f6","\\longrightarrow",!0),ie(ae,he,xe,"\u21d2","\\Rightarrow",!0),ie(ae,he,xe,"\u27f9","\\Longrightarrow",!0),ie(ae,he,xe,"\u2194","\\leftrightarrow",!0),ie(ae,he,xe,"\u27f7","\\longleftrightarrow",!0),ie(ae,he,xe,"\u21d4","\\Leftrightarrow",!0),ie(ae,he,xe,"\u27fa","\\Longleftrightarrow",!0),ie(ae,he,xe,"\u21a6","\\mapsto",!0),ie(ae,he,xe,"\u27fc","\\longmapsto",!0),ie(ae,he,xe,"\u2197","\\nearrow",!0),ie(ae,he,xe,"\u21a9","\\hookleftarrow",!0),ie(ae,he,xe,"\u21aa","\\hookrightarrow",!0),ie(ae,he,xe,"\u2198","\\searrow",!0),ie(ae,he,xe,"\u21bc","\\leftharpoonup",!0),ie(ae,he,xe,"\u21c0","\\rightharpoonup",!0),ie(ae,he,xe,"\u2199","\\swarrow",!0),ie(ae,he,xe,"\u21bd","\\leftharpoondown",!0),ie(ae,he,xe,"\u21c1","\\rightharpoondown",!0),ie(ae,he,xe,"\u2196","\\nwarrow",!0),ie(ae,he,xe,"\u21cc","\\rightleftharpoons",!0),ie(ae,ce,xe,"\u226e","\\nless",!0),ie(ae,ce,xe,"\ue010","\\@nleqslant"),ie(ae,ce,xe,"\ue011","\\@nleqq"),ie(ae,ce,xe,"\u2a87","\\lneq",!0),ie(ae,ce,xe,"\u2268","\\lneqq",!0),ie(ae,ce,xe,"\ue00c","\\@lvertneqq"),ie(ae,ce,xe,"\u22e6","\\lnsim",!0),ie(ae,ce,xe,"\u2a89","\\lnapprox",!0),ie(ae,ce,xe,"\u2280","\\nprec",!0),ie(ae,ce,xe,"\u22e0","\\npreceq",!0),ie(ae,ce,xe,"\u22e8","\\precnsim",!0),ie(ae,ce,xe,"\u2ab9","\\precnapprox",!0),ie(ae,ce,xe,"\u2241","\\nsim",!0),ie(ae,ce,xe,"\ue006","\\@nshortmid"),ie(ae,ce,xe,"\u2224","\\nmid",!0),ie(ae,ce,xe,"\u22ac","\\nvdash",!0),ie(ae,ce,xe,"\u22ad","\\nvDash",!0),ie(ae,ce,xe,"\u22ea","\\ntriangleleft"),ie(ae,ce,xe,"\u22ec","\\ntrianglelefteq",!0),ie(ae,ce,xe,"\u228a","\\subsetneq",!0),ie(ae,ce,xe,"\ue01a","\\@varsubsetneq"),ie(ae,ce,xe,"\u2acb","\\subsetneqq",!0),ie(ae,ce,xe,"\ue017","\\@varsubsetneqq"),ie(ae,ce,xe,"\u226f","\\ngtr",!0),ie(ae,ce,xe,"\ue00f","\\@ngeqslant"),ie(ae,ce,xe,"\ue00e","\\@ngeqq"),ie(ae,ce,xe,"\u2a88","\\gneq",!0),ie(ae,ce,xe,"\u2269","\\gneqq",!0),ie(ae,ce,xe,"\ue00d","\\@gvertneqq"),ie(ae,ce,xe,"\u22e7","\\gnsim",!0),ie(ae,ce,xe,"\u2a8a","\\gnapprox",!0),ie(ae,ce,xe,"\u2281","\\nsucc",!0),ie(ae,ce,xe,"\u22e1","\\nsucceq",!0),ie(ae,ce,xe,"\u22e9","\\succnsim",!0),ie(ae,ce,xe,"\u2aba","\\succnapprox",!0),ie(ae,ce,xe,"\u2246","\\ncong",!0),ie(ae,ce,xe,"\ue007","\\@nshortparallel"),ie(ae,ce,xe,"\u2226","\\nparallel",!0),ie(ae,ce,xe,"\u22af","\\nVDash",!0),ie(ae,ce,xe,"\u22eb","\\ntriangleright"),ie(ae,ce,xe,"\u22ed","\\ntrianglerighteq",!0),ie(ae,ce,xe,"\ue018","\\@nsupseteqq"),ie(ae,ce,xe,"\u228b","\\supsetneq",!0),ie(ae,ce,xe,"\ue01b","\\@varsupsetneq"),ie(ae,ce,xe,"\u2acc","\\supsetneqq",!0),ie(ae,ce,xe,"\ue019","\\@varsupsetneqq"),ie(ae,ce,xe,"\u22ae","\\nVdash",!0),ie(ae,ce,xe,"\u2ab5","\\precneqq",!0),ie(ae,ce,xe,"\u2ab6","\\succneqq",!0),ie(ae,ce,xe,"\ue016","\\@nsubseteqq"),ie(ae,ce,pe,"\u22b4","\\unlhd"),ie(ae,ce,pe,"\u22b5","\\unrhd"),ie(ae,ce,xe,"\u219a","\\nleftarrow",!0),ie(ae,ce,xe,"\u219b","\\nrightarrow",!0),ie(ae,ce,xe,"\u21cd","\\nLeftarrow",!0),ie(ae,ce,xe,"\u21cf","\\nRightarrow",!0),ie(ae,ce,xe,"\u21ae","\\nleftrightarrow",!0),ie(ae,ce,xe,"\u21ce","\\nLeftrightarrow",!0),ie(ae,ce,xe,"\u25b3","\\vartriangle"),ie(ae,ce,ve,"\u210f","\\hslash"),ie(ae,ce,ve,"\u25bd","\\triangledown"),ie(ae,ce,ve,"\u25ca","\\lozenge"),ie(ae,ce,ve,"\u24c8","\\circledS"),ie(ae,ce,ve,"\xae","\\circledR"),ie(le,ce,ve,"\xae","\\circledR"),ie(ae,ce,ve,"\u2221","\\measuredangle",!0),ie(ae,ce,ve,"\u2204","\\nexists"),ie(ae,ce,ve,"\u2127","\\mho"),ie(ae,ce,ve,"\u2132","\\Finv",!0),ie(ae,ce,ve,"\u2141","\\Game",!0),ie(ae,ce,ve,"\u2035","\\backprime"),ie(ae,ce,ve,"\u25b2","\\blacktriangle"),ie(ae,ce,ve,"\u25bc","\\blacktriangledown"),ie(ae,ce,ve,"\u25a0","\\blacksquare"),ie(ae,ce,ve,"\u29eb","\\blacklozenge"),ie(ae,ce,ve,"\u2605","\\bigstar"),ie(ae,ce,ve,"\u2222","\\sphericalangle",!0),ie(ae,ce,ve,"\u2201","\\complement",!0),ie(ae,ce,ve,"\xf0","\\eth",!0),ie(le,he,ve,"\xf0","\xf0"),ie(ae,ce,ve,"\u2571","\\diagup"),ie(ae,ce,ve,"\u2572","\\diagdown"),ie(ae,ce,ve,"\u25a1","\\square"),ie(ae,ce,ve,"\u25a1","\\Box"),ie(ae,ce,ve,"\u25ca","\\Diamond"),ie(ae,ce,ve,"\xa5","\\yen",!0),ie(le,ce,ve,"\xa5","\\yen",!0),ie(ae,ce,ve,"\u2713","\\checkmark",!0),ie(le,ce,ve,"\u2713","\\checkmark"),ie(ae,ce,ve,"\u2136","\\beth",!0),ie(ae,ce,ve,"\u2138","\\daleth",!0),ie(ae,ce,ve,"\u2137","\\gimel",!0),ie(ae,ce,ve,"\u03dd","\\digamma",!0),ie(ae,ce,ve,"\u03f0","\\varkappa"),ie(ae,ce,be,"\u250c","\\@ulcorner",!0),ie(ae,ce,ue,"\u2510","\\@urcorner",!0),ie(ae,ce,be,"\u2514","\\@llcorner",!0),ie(ae,ce,ue,"\u2518","\\@lrcorner",!0),ie(ae,ce,xe,"\u2266","\\leqq",!0),ie(ae,ce,xe,"\u2a7d","\\leqslant",!0),ie(ae,ce,xe,"\u2a95","\\eqslantless",!0),ie(ae,ce,xe,"\u2272","\\lesssim",!0),ie(ae,ce,xe,"\u2a85","\\lessapprox",!0),ie(ae,ce,xe,"\u224a","\\approxeq",!0),ie(ae,ce,pe,"\u22d6","\\lessdot"),ie(ae,ce,xe,"\u22d8","\\lll",!0),ie(ae,ce,xe,"\u2276","\\lessgtr",!0),ie(ae,ce,xe,"\u22da","\\lesseqgtr",!0),ie(ae,ce,xe,"\u2a8b","\\lesseqqgtr",!0),ie(ae,ce,xe,"\u2251","\\doteqdot"),ie(ae,ce,xe,"\u2253","\\risingdotseq",!0),ie(ae,ce,xe,"\u2252","\\fallingdotseq",!0),ie(ae,ce,xe,"\u223d","\\backsim",!0),ie(ae,ce,xe,"\u22cd","\\backsimeq",!0),ie(ae,ce,xe,"\u2ac5","\\subseteqq",!0),ie(ae,ce,xe,"\u22d0","\\Subset",!0),ie(ae,ce,xe,"\u228f","\\sqsubset",!0),ie(ae,ce,xe,"\u227c","\\preccurlyeq",!0),ie(ae,ce,xe,"\u22de","\\curlyeqprec",!0),ie(ae,ce,xe,"\u227e","\\precsim",!0),ie(ae,ce,xe,"\u2ab7","\\precapprox",!0),ie(ae,ce,xe,"\u22b2","\\vartriangleleft"),ie(ae,ce,xe,"\u22b4","\\trianglelefteq"),ie(ae,ce,xe,"\u22a8","\\vDash",!0),ie(ae,ce,xe,"\u22aa","\\Vvdash",!0),ie(ae,ce,xe,"\u2323","\\smallsmile"),ie(ae,ce,xe,"\u2322","\\smallfrown"),ie(ae,ce,xe,"\u224f","\\bumpeq",!0),ie(ae,ce,xe,"\u224e","\\Bumpeq",!0),ie(ae,ce,xe,"\u2267","\\geqq",!0),ie(ae,ce,xe,"\u2a7e","\\geqslant",!0),ie(ae,ce,xe,"\u2a96","\\eqslantgtr",!0),ie(ae,ce,xe,"\u2273","\\gtrsim",!0),ie(ae,ce,xe,"\u2a86","\\gtrapprox",!0),ie(ae,ce,pe,"\u22d7","\\gtrdot"),ie(ae,ce,xe,"\u22d9","\\ggg",!0),ie(ae,ce,xe,"\u2277","\\gtrless",!0),ie(ae,ce,xe,"\u22db","\\gtreqless",!0),ie(ae,ce,xe,"\u2a8c","\\gtreqqless",!0),ie(ae,ce,xe,"\u2256","\\eqcirc",!0),ie(ae,ce,xe,"\u2257","\\circeq",!0),ie(ae,ce,xe,"\u225c","\\triangleq",!0),ie(ae,ce,xe,"\u223c","\\thicksim"),ie(ae,ce,xe,"\u2248","\\thickapprox"),ie(ae,ce,xe,"\u2ac6","\\supseteqq",!0),ie(ae,ce,xe,"\u22d1","\\Supset",!0),ie(ae,ce,xe,"\u2290","\\sqsupset",!0),ie(ae,ce,xe,"\u227d","\\succcurlyeq",!0),ie(ae,ce,xe,"\u22df","\\curlyeqsucc",!0),ie(ae,ce,xe,"\u227f","\\succsim",!0),ie(ae,ce,xe,"\u2ab8","\\succapprox",!0),ie(ae,ce,xe,"\u22b3","\\vartriangleright"),ie(ae,ce,xe,"\u22b5","\\trianglerighteq"),ie(ae,ce,xe,"\u22a9","\\Vdash",!0),ie(ae,ce,xe,"\u2223","\\shortmid"),ie(ae,ce,xe,"\u2225","\\shortparallel"),ie(ae,ce,xe,"\u226c","\\between",!0),ie(ae,ce,xe,"\u22d4","\\pitchfork",!0),ie(ae,ce,xe,"\u221d","\\varpropto"),ie(ae,ce,xe,"\u25c0","\\blacktriangleleft"),ie(ae,ce,xe,"\u2234","\\therefore",!0),ie(ae,ce,xe,"\u220d","\\backepsilon"),ie(ae,ce,xe,"\u25b6","\\blacktriangleright"),ie(ae,ce,xe,"\u2235","\\because",!0),ie(ae,ce,xe,"\u22d8","\\llless"),ie(ae,ce,xe,"\u22d9","\\gggtr"),ie(ae,ce,pe,"\u22b2","\\lhd"),ie(ae,ce,pe,"\u22b3","\\rhd"),ie(ae,ce,xe,"\u2242","\\eqsim",!0),ie(ae,he,xe,"\u22c8","\\Join"),ie(ae,ce,xe,"\u2251","\\Doteq",!0),ie(ae,ce,pe,"\u2214","\\dotplus",!0),ie(ae,ce,pe,"\u2216","\\smallsetminus"),ie(ae,ce,pe,"\u22d2","\\Cap",!0),ie(ae,ce,pe,"\u22d3","\\Cup",!0),ie(ae,ce,pe,"\u2a5e","\\doublebarwedge",!0),ie(ae,ce,pe,"\u229f","\\boxminus",!0),ie(ae,ce,pe,"\u229e","\\boxplus",!0),ie(ae,ce,pe,"\u22c7","\\divideontimes",!0),ie(ae,ce,pe,"\u22c9","\\ltimes",!0),ie(ae,ce,pe,"\u22ca","\\rtimes",!0),ie(ae,ce,pe,"\u22cb","\\leftthreetimes",!0),ie(ae,ce,pe,"\u22cc","\\rightthreetimes",!0),ie(ae,ce,pe,"\u22cf","\\curlywedge",!0),ie(ae,ce,pe,"\u22ce","\\curlyvee",!0),ie(ae,ce,pe,"\u229d","\\circleddash",!0),ie(ae,ce,pe,"\u229b","\\circledast",!0),ie(ae,ce,pe,"\u22c5","\\centerdot"),ie(ae,ce,pe,"\u22ba","\\intercal",!0),ie(ae,ce,pe,"\u22d2","\\doublecap"),ie(ae,ce,pe,"\u22d3","\\doublecup"),ie(ae,ce,pe,"\u22a0","\\boxtimes",!0),ie(ae,ce,xe,"\u21e2","\\dashrightarrow",!0),ie(ae,ce,xe,"\u21e0","\\dashleftarrow",!0),ie(ae,ce,xe,"\u21c7","\\leftleftarrows",!0),ie(ae,ce,xe,"\u21c6","\\leftrightarrows",!0),ie(ae,ce,xe,"\u21da","\\Lleftarrow",!0),ie(ae,ce,xe,"\u219e","\\twoheadleftarrow",!0),ie(ae,ce,xe,"\u21a2","\\leftarrowtail",!0),ie(ae,ce,xe,"\u21ab","\\looparrowleft",!0),ie(ae,ce,xe,"\u21cb","\\leftrightharpoons",!0),ie(ae,ce,xe,"\u21b6","\\curvearrowleft",!0),ie(ae,ce,xe,"\u21ba","\\circlearrowleft",!0),ie(ae,ce,xe,"\u21b0","\\Lsh",!0),ie(ae,ce,xe,"\u21c8","\\upuparrows",!0),ie(ae,ce,xe,"\u21bf","\\upharpoonleft",!0),ie(ae,ce,xe,"\u21c3","\\downharpoonleft",!0),ie(ae,he,xe,"\u22b6","\\origof",!0),ie(ae,he,xe,"\u22b7","\\imageof",!0),ie(ae,ce,xe,"\u22b8","\\multimap",!0),ie(ae,ce,xe,"\u21ad","\\leftrightsquigarrow",!0),ie(ae,ce,xe,"\u21c9","\\rightrightarrows",!0),ie(ae,ce,xe,"\u21c4","\\rightleftarrows",!0),ie(ae,ce,xe,"\u21a0","\\twoheadrightarrow",!0),ie(ae,ce,xe,"\u21a3","\\rightarrowtail",!0),ie(ae,ce,xe,"\u21ac","\\looparrowright",!0),ie(ae,ce,xe,"\u21b7","\\curvearrowright",!0),ie(ae,ce,xe,"\u21bb","\\circlearrowright",!0),ie(ae,ce,xe,"\u21b1","\\Rsh",!0),ie(ae,ce,xe,"\u21ca","\\downdownarrows",!0),ie(ae,ce,xe,"\u21be","\\upharpoonright",!0),ie(ae,ce,xe,"\u21c2","\\downharpoonright",!0),ie(ae,ce,xe,"\u21dd","\\rightsquigarrow",!0),ie(ae,ce,xe,"\u21dd","\\leadsto"),ie(ae,ce,xe,"\u21db","\\Rrightarrow",!0),ie(ae,ce,xe,"\u21be","\\restriction"),ie(ae,he,ve,"\u2018","`"),ie(ae,he,ve,"$","\\$"),ie(le,he,ve,"$","\\$"),ie(le,he,ve,"$","\\textdollar"),ie(ae,he,ve,"%","\\%"),ie(le,he,ve,"%","\\%"),ie(ae,he,ve,"_","\\_"),ie(le,he,ve,"_","\\_"),ie(le,he,ve,"_","\\textunderscore"),ie(ae,he,ve,"\u2220","\\angle",!0),ie(ae,he,ve,"\u221e","\\infty",!0),ie(ae,he,ve,"\u2032","\\prime"),ie(ae,he,ve,"\u25b3","\\triangle"),ie(ae,he,ve,"\u0393","\\Gamma",!0),ie(ae,he,ve,"\u0394","\\Delta",!0),ie(ae,he,ve,"\u0398","\\Theta",!0),ie(ae,he,ve,"\u039b","\\Lambda",!0),ie(ae,he,ve,"\u039e","\\Xi",!0),ie(ae,he,ve,"\u03a0","\\Pi",!0),ie(ae,he,ve,"\u03a3","\\Sigma",!0),ie(ae,he,ve,"\u03a5","\\Upsilon",!0),ie(ae,he,ve,"\u03a6","\\Phi",!0),ie(ae,he,ve,"\u03a8","\\Psi",!0),ie(ae,he,ve,"\u03a9","\\Omega",!0),ie(ae,he,ve,"A","\u0391"),ie(ae,he,ve,"B","\u0392"),ie(ae,he,ve,"E","\u0395"),ie(ae,he,ve,"Z","\u0396"),ie(ae,he,ve,"H","\u0397"),ie(ae,he,ve,"I","\u0399"),ie(ae,he,ve,"K","\u039a"),ie(ae,he,ve,"M","\u039c"),ie(ae,he,ve,"N","\u039d"),ie(ae,he,ve,"O","\u039f"),ie(ae,he,ve,"P","\u03a1"),ie(ae,he,ve,"T","\u03a4"),ie(ae,he,ve,"X","\u03a7"),ie(ae,he,ve,"\xac","\\neg",!0),ie(ae,he,ve,"\xac","\\lnot"),ie(ae,he,ve,"\u22a4","\\top"),ie(ae,he,ve,"\u22a5","\\bot"),ie(ae,he,ve,"\u2205","\\emptyset"),ie(ae,ce,ve,"\u2205","\\varnothing"),ie(ae,he,ge,"\u03b1","\\alpha",!0),ie(ae,he,ge,"\u03b2","\\beta",!0),ie(ae,he,ge,"\u03b3","\\gamma",!0),ie(ae,he,ge,"\u03b4","\\delta",!0),ie(ae,he,ge,"\u03f5","\\epsilon",!0),ie(ae,he,ge,"\u03b6","\\zeta",!0),ie(ae,he,ge,"\u03b7","\\eta",!0),ie(ae,he,ge,"\u03b8","\\theta",!0),ie(ae,he,ge,"\u03b9","\\iota",!0),ie(ae,he,ge,"\u03ba","\\kappa",!0),ie(ae,he,ge,"\u03bb","\\lambda",!0),ie(ae,he,ge,"\u03bc","\\mu",!0),ie(ae,he,ge,"\u03bd","\\nu",!0),ie(ae,he,ge,"\u03be","\\xi",!0),ie(ae,he,ge,"\u03bf","\\omicron",!0),ie(ae,he,ge,"\u03c0","\\pi",!0),ie(ae,he,ge,"\u03c1","\\rho",!0),ie(ae,he,ge,"\u03c3","\\sigma",!0),ie(ae,he,ge,"\u03c4","\\tau",!0),ie(ae,he,ge,"\u03c5","\\upsilon",!0),ie(ae,he,ge,"\u03d5","\\phi",!0),ie(ae,he,ge,"\u03c7","\\chi",!0),ie(ae,he,ge,"\u03c8","\\psi",!0),ie(ae,he,ge,"\u03c9","\\omega",!0),ie(ae,he,ge,"\u03b5","\\varepsilon",!0),ie(ae,he,ge,"\u03d1","\\vartheta",!0),ie(ae,he,ge,"\u03d6","\\varpi",!0),ie(ae,he,ge,"\u03f1","\\varrho",!0),ie(ae,he,ge,"\u03c2","\\varsigma",!0),ie(ae,he,ge,"\u03c6","\\varphi",!0),ie(ae,he,pe,"\u2217","*",!0),ie(ae,he,pe,"+","+"),ie(ae,he,pe,"\u2212","-",!0),ie(ae,he,pe,"\u22c5","\\cdot",!0),ie(ae,he,pe,"\u2218","\\circ",!0),ie(ae,he,pe,"\xf7","\\div",!0),ie(ae,he,pe,"\xb1","\\pm",!0),ie(ae,he,pe,"\xd7","\\times",!0),ie(ae,he,pe,"\u2229","\\cap",!0),ie(ae,he,pe,"\u222a","\\cup",!0),ie(ae,he,pe,"\u2216","\\setminus",!0),ie(ae,he,pe,"\u2227","\\land"),ie(ae,he,pe,"\u2228","\\lor"),ie(ae,he,pe,"\u2227","\\wedge",!0),ie(ae,he,pe,"\u2228","\\vee",!0),ie(ae,he,ve,"\u221a","\\surd"),ie(ae,he,be,"\u27e8","\\langle",!0),ie(ae,he,be,"\u2223","\\lvert"),ie(ae,he,be,"\u2225","\\lVert"),ie(ae,he,ue,"?","?"),ie(ae,he,ue,"!","!"),ie(ae,he,ue,"\u27e9","\\rangle",!0),ie(ae,he,ue,"\u2223","\\rvert"),ie(ae,he,ue,"\u2225","\\rVert"),ie(ae,he,xe,"=","="),ie(ae,he,xe,":",":"),ie(ae,he,xe,"\u2248","\\approx",!0),ie(ae,he,xe,"\u2245","\\cong",!0),ie(ae,he,xe,"\u2265","\\ge"),ie(ae,he,xe,"\u2265","\\geq",!0),ie(ae,he,xe,"\u2190","\\gets"),ie(ae,he,xe,">","\\gt",!0),ie(ae,he,xe,"\u2208","\\in",!0),ie(ae,he,xe,"\ue020","\\@not"),ie(ae,he,xe,"\u2282","\\subset",!0),ie(ae,he,xe,"\u2283","\\supset",!0),ie(ae,he,xe,"\u2286","\\subseteq",!0),ie(ae,he,xe,"\u2287","\\supseteq",!0),ie(ae,ce,xe,"\u2288","\\nsubseteq",!0),ie(ae,ce,xe,"\u2289","\\nsupseteq",!0),ie(ae,he,xe,"\u22a8","\\models"),ie(ae,he,xe,"\u2190","\\leftarrow",!0),ie(ae,he,xe,"\u2264","\\le"),ie(ae,he,xe,"\u2264","\\leq",!0),ie(ae,he,xe,"<","\\lt",!0),ie(ae,he,xe,"\u2192","\\rightarrow",!0),ie(ae,he,xe,"\u2192","\\to"),ie(ae,ce,xe,"\u2271","\\ngeq",!0),ie(ae,ce,xe,"\u2270","\\nleq",!0),ie(ae,he,we,"\xa0","\\ "),ie(ae,he,we,"\xa0","\\space"),ie(ae,he,we,"\xa0","\\nobreakspace"),ie(le,he,we,"\xa0","\\ "),ie(le,he,we,"\xa0"," "),ie(le,he,we,"\xa0","\\space"),ie(le,he,we,"\xa0","\\nobreakspace"),ie(ae,he,we,null,"\\nobreak"),ie(ae,he,we,null,"\\allowbreak"),ie(ae,he,ye,",",","),ie(ae,he,ye,";",";"),ie(ae,ce,pe,"\u22bc","\\barwedge",!0),ie(ae,ce,pe,"\u22bb","\\veebar",!0),ie(ae,he,pe,"\u2299","\\odot",!0),ie(ae,he,pe,"\u2295","\\oplus",!0),ie(ae,he,pe,"\u2297","\\otimes",!0),ie(ae,he,ve,"\u2202","\\partial",!0),ie(ae,he,pe,"\u2298","\\oslash",!0),ie(ae,ce,pe,"\u229a","\\circledcirc",!0),ie(ae,ce,pe,"\u22a1","\\boxdot",!0),ie(ae,he,pe,"\u25b3","\\bigtriangleup"),ie(ae,he,pe,"\u25bd","\\bigtriangledown"),ie(ae,he,pe,"\u2020","\\dagger"),ie(ae,he,pe,"\u22c4","\\diamond"),ie(ae,he,pe,"\u22c6","\\star"),ie(ae,he,pe,"\u25c3","\\triangleleft"),ie(ae,he,pe,"\u25b9","\\triangleright"),ie(ae,he,be,"{","\\{"),ie(le,he,ve,"{","\\{"),ie(le,he,ve,"{","\\textbraceleft"),ie(ae,he,ue,"}","\\}"),ie(le,he,ve,"}","\\}"),ie(le,he,ve,"}","\\textbraceright"),ie(ae,he,be,"{","\\lbrace"),ie(ae,he,ue,"}","\\rbrace"),ie(ae,he,be,"[","\\lbrack",!0),ie(le,he,ve,"[","\\lbrack",!0),ie(ae,he,ue,"]","\\rbrack",!0),ie(le,he,ve,"]","\\rbrack",!0),ie(ae,he,be,"(","\\lparen",!0),ie(ae,he,ue,")","\\rparen",!0),ie(le,he,ve,"<","\\textless",!0),ie(le,he,ve,">","\\textgreater",!0),ie(ae,he,be,"\u230a","\\lfloor",!0),ie(ae,he,ue,"\u230b","\\rfloor",!0),ie(ae,he,be,"\u2308","\\lceil",!0),ie(ae,he,ue,"\u2309","\\rceil",!0),ie(ae,he,ve,"\\","\\backslash"),ie(ae,he,ve,"\u2223","|"),ie(ae,he,ve,"\u2223","\\vert"),ie(le,he,ve,"|","\\textbar",!0),ie(ae,he,ve,"\u2225","\\|"),ie(ae,he,ve,"\u2225","\\Vert"),ie(le,he,ve,"\u2225","\\textbardbl"),ie(le,he,ve,"~","\\textasciitilde"),ie(le,he,ve,"\\","\\textbackslash"),ie(le,he,ve,"^","\\textasciicircum"),ie(ae,he,xe,"\u2191","\\uparrow",!0),ie(ae,he,xe,"\u21d1","\\Uparrow",!0),ie(ae,he,xe,"\u2193","\\downarrow",!0),ie(ae,he,xe,"\u21d3","\\Downarrow",!0),ie(ae,he,xe,"\u2195","\\updownarrow",!0),ie(ae,he,xe,"\u21d5","\\Updownarrow",!0),ie(ae,he,fe,"\u2210","\\coprod"),ie(ae,he,fe,"\u22c1","\\bigvee"),ie(ae,he,fe,"\u22c0","\\bigwedge"),ie(ae,he,fe,"\u2a04","\\biguplus"),ie(ae,he,fe,"\u22c2","\\bigcap"),ie(ae,he,fe,"\u22c3","\\bigcup"),ie(ae,he,fe,"\u222b","\\int"),ie(ae,he,fe,"\u222b","\\intop"),ie(ae,he,fe,"\u222c","\\iint"),ie(ae,he,fe,"\u222d","\\iiint"),ie(ae,he,fe,"\u220f","\\prod"),ie(ae,he,fe,"\u2211","\\sum"),ie(ae,he,fe,"\u2a02","\\bigotimes"),ie(ae,he,fe,"\u2a01","\\bigoplus"),ie(ae,he,fe,"\u2a00","\\bigodot"),ie(ae,he,fe,"\u222e","\\oint"),ie(ae,he,fe,"\u222f","\\oiint"),ie(ae,he,fe,"\u2230","\\oiiint"),ie(ae,he,fe,"\u2a06","\\bigsqcup"),ie(ae,he,fe,"\u222b","\\smallint"),ie(le,he,de,"\u2026","\\textellipsis"),ie(ae,he,de,"\u2026","\\mathellipsis"),ie(le,he,de,"\u2026","\\ldots",!0),ie(ae,he,de,"\u2026","\\ldots",!0),ie(ae,he,de,"\u22ef","\\@cdots",!0),ie(ae,he,de,"\u22f1","\\ddots",!0),ie(ae,he,ve,"\u22ee","\\varvdots"),ie(le,he,ve,"\u22ee","\\varvdots"),ie(ae,he,me,"\u02ca","\\acute"),ie(ae,he,me,"\u02cb","\\grave"),ie(ae,he,me,"\xa8","\\ddot"),ie(ae,he,me,"~","\\tilde"),ie(ae,he,me,"\u02c9","\\bar"),ie(ae,he,me,"\u02d8","\\breve"),ie(ae,he,me,"\u02c7","\\check"),ie(ae,he,me,"^","\\hat"),ie(ae,he,me,"\u20d7","\\vec"),ie(ae,he,me,"\u02d9","\\dot"),ie(ae,he,me,"\u02da","\\mathring"),ie(ae,he,ge,"\ue131","\\@imath"),ie(ae,he,ge,"\ue237","\\@jmath"),ie(ae,he,ve,"\u0131","\u0131"),ie(ae,he,ve,"\u0237","\u0237"),ie(le,he,ve,"\u0131","\\i",!0),ie(le,he,ve,"\u0237","\\j",!0),ie(le,he,ve,"\xdf","\\ss",!0),ie(le,he,ve,"\xe6","\\ae",!0),ie(le,he,ve,"\u0153","\\oe",!0),ie(le,he,ve,"\xf8","\\o",!0),ie(le,he,ve,"\xc6","\\AE",!0),ie(le,he,ve,"\u0152","\\OE",!0),ie(le,he,ve,"\xd8","\\O",!0),ie(le,he,me,"\u02ca","\\'"),ie(le,he,me,"\u02cb","\\`"),ie(le,he,me,"\u02c6","\\^"),ie(le,he,me,"\u02dc","\\~"),ie(le,he,me,"\u02c9","\\="),ie(le,he,me,"\u02d8","\\u"),ie(le,he,me,"\u02d9","\\."),ie(le,he,me,"\xb8","\\c"),ie(le,he,me,"\u02da","\\r"),ie(le,he,me,"\u02c7","\\v"),ie(le,he,me,"\xa8",'\\"'),ie(le,he,me,"\u02dd","\\H"),ie(le,he,me,"\u25ef","\\textcircled");const ke={"--":!0,"---":!0,"``":!0,"''":!0};ie(le,he,ve,"\u2013","--",!0),ie(le,he,ve,"\u2013","\\textendash"),ie(le,he,ve,"\u2014","---",!0),ie(le,he,ve,"\u2014","\\textemdash"),ie(le,he,ve,"\u2018","`",!0),ie(le,he,ve,"\u2018","\\textquoteleft"),ie(le,he,ve,"\u2019","'",!0),ie(le,he,ve,"\u2019","\\textquoteright"),ie(le,he,ve,"\u201c","``",!0),ie(le,he,ve,"\u201c","\\textquotedblleft"),ie(le,he,ve,"\u201d","''",!0),ie(le,he,ve,"\u201d","\\textquotedblright"),ie(ae,he,ve,"\xb0","\\degree",!0),ie(le,he,ve,"\xb0","\\degree"),ie(le,he,ve,"\xb0","\\textdegree",!0),ie(ae,he,ve,"\xa3","\\pounds"),ie(ae,he,ve,"\xa3","\\mathsterling",!0),ie(le,he,ve,"\xa3","\\pounds"),ie(le,he,ve,"\xa3","\\textsterling",!0),ie(ae,ce,ve,"\u2720","\\maltese"),ie(le,ce,ve,"\u2720","\\maltese");const Se='0123456789/@."';for(let e=0;e<14;e++){const t=Se.charAt(e);ie(ae,he,ve,t,t)}const Me='0123456789!@*()-=+";:?/.,';for(let e=0;e<25;e++){const t=Me.charAt(e);ie(le,he,ve,t,t)}const ze="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";for(let e=0;e<52;e++){const t=ze.charAt(e);ie(ae,he,ge,t,t),ie(le,he,ve,t,t)}ie(ae,ce,ve,"C","\u2102"),ie(le,ce,ve,"C","\u2102"),ie(ae,ce,ve,"H","\u210d"),ie(le,ce,ve,"H","\u210d"),ie(ae,ce,ve,"N","\u2115"),ie(le,ce,ve,"N","\u2115"),ie(ae,ce,ve,"P","\u2119"),ie(le,ce,ve,"P","\u2119"),ie(ae,ce,ve,"Q","\u211a"),ie(le,ce,ve,"Q","\u211a"),ie(ae,ce,ve,"R","\u211d"),ie(le,ce,ve,"R","\u211d"),ie(ae,ce,ve,"Z","\u2124"),ie(le,ce,ve,"Z","\u2124"),ie(ae,he,ge,"h","\u210e"),ie(le,he,ge,"h","\u210e");let Ae="";for(let e=0;e<52;e++){const t=ze.charAt(e);Ae=String.fromCharCode(55349,56320+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56372+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56424+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56580+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56684+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56736+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56788+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56840+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56944+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),e<26&&(Ae=String.fromCharCode(55349,56632+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,56476+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae))}Ae=String.fromCharCode(55349,56668),ie(ae,he,ge,"k",Ae),ie(le,he,ve,"k",Ae);for(let e=0;e<10;e++){const t=e.toString();Ae=String.fromCharCode(55349,57294+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,57314+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,57324+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae),Ae=String.fromCharCode(55349,57334+e),ie(ae,he,ge,t,Ae),ie(le,he,ve,t,Ae)}const Te="\xd0\xde\xfe";for(let e=0;e<3;e++){const t=Te.charAt(e);ie(ae,he,ge,t,t),ie(le,he,ve,t,t)}const Be=[["mathbf","textbf","Main-Bold"],["mathbf","textbf","Main-Bold"],["mathnormal","textit","Math-Italic"],["mathnormal","textit","Math-Italic"],["boldsymbol","boldsymbol","Main-BoldItalic"],["boldsymbol","boldsymbol","Main-BoldItalic"],["mathscr","textscr","Script-Regular"],["","",""],["","",""],["","",""],["mathfrak","textfrak","Fraktur-Regular"],["mathfrak","textfrak","Fraktur-Regular"],["mathbb","textbb","AMS-Regular"],["mathbb","textbb","AMS-Regular"],["mathboldfrak","textboldfrak","Fraktur-Regular"],["mathboldfrak","textboldfrak","Fraktur-Regular"],["mathsf","textsf","SansSerif-Regular"],["mathsf","textsf","SansSerif-Regular"],["mathboldsf","textboldsf","SansSerif-Bold"],["mathboldsf","textboldsf","SansSerif-Bold"],["mathitsf","textitsf","SansSerif-Italic"],["mathitsf","textitsf","SansSerif-Italic"],["","",""],["","",""],["mathtt","texttt","Typewriter-Regular"],["mathtt","texttt","Typewriter-Regular"]],Ce=[["mathbf","textbf","Main-Bold"],["","",""],["mathsf","textsf","SansSerif-Regular"],["mathboldsf","textboldsf","SansSerif-Bold"],["mathtt","texttt","Typewriter-Regular"]],Ne=function(e,t,r){return se[r][e]&&se[r][e].replace&&(e=se[r][e].replace),{value:e,metrics:N(e,t,r)}},qe=function(e,t,r,n,o){const s=Ne(e,t,r),i=s.metrics;let a;if(e=s.value,i){let t=i.italic;("text"===r||n&&"mathit"===n.font)&&(t=0),a=new K(e,i.height,i.depth,t,i.skew,i.width,o)}else"undefined"!=typeof console&&console.warn("No character metrics for '"+e+"' in style '"+t+"' and mode '"+r+"'"),a=new K(e,0,0,0,0,0,o);if(n){a.maxFontSize=n.sizeMultiplier,n.style.isTight()&&a.classes.push("mtight");const e=n.getColor();e&&(a.style.color=e)}return a},Ie=(e,t)=>{if(G(e.classes)!==G(t.classes)||e.skew!==t.skew||e.maxFontSize!==t.maxFontSize)return!1;if(1===e.classes.length){const t=e.classes[0];if("mbin"===t||"mord"===t)return!1}for(const r in e.style)if(e.style.hasOwnProperty(r)&&e.style[r]!==t.style[r])return!1;for(const r in t.style)if(t.style.hasOwnProperty(r)&&e.style[r]!==t.style[r])return!1;return!0},Re=function(e){let t=0,r=0,n=0;for(let o=0;o<e.children.length;o++){const s=e.children[o];s.height>t&&(t=s.height),s.depth>r&&(r=s.depth),s.maxFontSize>n&&(n=s.maxFontSize)}e.height=t,e.depth=r,e.maxFontSize=n},He=function(e,t,r,n){const o=new _(e,t,r,n);return Re(o),o},Oe=(e,t,r,n)=>new _(e,t,r,n),Ee=function(e){const t=new A(e);return Re(t),t},Le=function(e,t,r){let n,o="";switch(e){case"amsrm":o="AMS";break;case"textrm":o="Main";break;case"textsf":o="SansSerif";break;case"texttt":o="Typewriter";break;default:o=e}return n="textbf"===t&&"textit"===r?"BoldItalic":"textbf"===t?"Bold":"textit"===t?"Italic":"Regular",o+"-"+n},De={mathbf:{variant:"bold",fontName:"Main-Bold"},mathrm:{variant:"normal",fontName:"Main-Regular"},textit:{variant:"italic",fontName:"Main-Italic"},mathit:{variant:"italic",fontName:"Main-Italic"},mathnormal:{variant:"italic",fontName:"Math-Italic"},mathsfit:{variant:"sans-serif-italic",fontName:"SansSerif-Italic"},mathbb:{variant:"double-struck",fontName:"AMS-Regular"},mathcal:{variant:"script",fontName:"Caligraphic-Regular"},mathfrak:{variant:"fraktur",fontName:"Fraktur-Regular"},mathscr:{variant:"script",fontName:"Script-Regular"},mathsf:{variant:"sans-serif",fontName:"SansSerif-Regular"},mathtt:{variant:"monospace",fontName:"Typewriter-Regular"}},Ve={vec:["vec",.471,.714],oiintSize1:["oiintSize1",.957,.499],oiintSize2:["oiintSize2",1.472,.659],oiiintSize1:["oiiintSize1",1.304,.499],oiiintSize2:["oiiintSize2",1.98,.659]};var Pe={fontMap:De,makeSymbol:qe,mathsym:function(e,t,r,n){return void 0===n&&(n=[]),"boldsymbol"===r.font&&Ne(e,"Main-Bold",t).metrics?qe(e,"Main-Bold",t,r,n.concat(["mathbf"])):"\\"===e||"main"===se[t][e].font?qe(e,"Main-Regular",t,r,n):qe(e,"AMS-Regular",t,r,n.concat(["amsrm"]))},makeSpan:He,makeSvgSpan:Oe,makeLineSpan:function(e,t,r){const n=He([e],[],t);return n.height=Math.max(r||t.fontMetrics().defaultRuleThickness,t.minRuleThickness),n.style.borderBottomWidth=F(n.height),n.maxFontSize=1,n},makeAnchor:function(e,t,r,n){const o=new j(e,t,r,n);return Re(o),o},makeFragment:Ee,wrapFragment:function(e,t){return e instanceof A?He([],[e],t):e},makeVList:function(e,t){const{children:r,depth:n}=function(e){if("individualShift"===e.positionType){const t=e.children,r=[t[0]],n=-t[0].shift-t[0].elem.depth;let o=n;for(let e=1;e<t.length;e++){const n=-t[e].shift-o-t[e].elem.depth,s=n-(t[e-1].elem.height+t[e-1].elem.depth);o+=n,r.push({type:"kern",size:s}),r.push(t[e])}return{children:r,depth:n}}let t;if("top"===e.positionType){let r=e.positionData;for(let t=0;t<e.children.length;t++){const n=e.children[t];r-="kern"===n.type?n.size:n.elem.height+n.elem.depth}t=r}else if("bottom"===e.positionType)t=-e.positionData;else{const r=e.children[0];if("elem"!==r.type)throw new Error('First child must have type "elem".');if("shift"===e.positionType)t=-r.elem.depth-e.positionData;else{if("firstBaseline"!==e.positionType)throw new Error("Invalid positionType "+e.positionType+".");t=-r.elem.depth}}return{children:e.children,depth:t}}(e);let o=0;for(let e=0;e<r.length;e++){const t=r[e];if("elem"===t.type){const e=t.elem;o=Math.max(o,e.maxFontSize,e.height)}}o+=2;const s=He(["pstrut"],[]);s.style.height=F(o);const i=[];let a=n,l=n,h=n;for(let e=0;e<r.length;e++){const t=r[e];if("kern"===t.type)h+=t.size;else{const e=t.elem,r=t.wrapperClasses||[],n=t.wrapperStyle||{},a=He(r,[s,e],void 0,n);a.style.top=F(-o-h-e.depth),t.marginLeft&&(a.style.marginLeft=t.marginLeft),t.marginRight&&(a.style.marginRight=t.marginRight),i.push(a),h+=e.height+e.depth}a=Math.min(a,h),l=Math.max(l,h)}const c=He(["vlist"],i);let m;if(c.style.height=F(l),a<0){const e=He([],[]),t=He(["vlist"],[e]);t.style.height=F(-a);const r=He(["vlist-s"],[new K("\u200b")]);m=[He(["vlist-r"],[c,r]),He(["vlist-r"],[t])]}else m=[He(["vlist-r"],[c])];const p=He(["vlist-t"],m);return 2===m.length&&p.classes.push("vlist-t2"),p.height=l,p.depth=-a,p},makeOrd:function(e,t,r){const o=e.mode,s=e.text,i=["mord"],a="math"===o||"text"===o&&t.font,l=a?t.font:t.fontFamily;let h="",c="";if(55349===s.charCodeAt(0)&&([h,c]=function(e,t){const r=1024*(e.charCodeAt(0)-55296)+(e.charCodeAt(1)-56320)+65536,o="math"===t?0:1;if(119808<=r&&r<120484){const e=Math.floor((r-119808)/26);return[Be[e][2],Be[e][o]]}if(120782<=r&&r<=120831){const e=Math.floor((r-120782)/10);return[Ce[e][2],Ce[e][o]]}if(120485===r||120486===r)return[Be[0][2],Be[0][o]];if(120486<r&&r<120782)return["",""];throw new n("Unsupported character: "+e)}(s,o)),h.length>0)return qe(s,h,o,t,i.concat(c));if(l){let e,n;if("boldsymbol"===l){const t=function(e,t,r,n,o){return"textord"!==o&&Ne(e,"Math-BoldItalic",t).metrics?{fontName:"Math-BoldItalic",fontClass:"boldsymbol"}:{fontName:"Main-Bold",fontClass:"mathbf"}}(s,o,0,0,r);e=t.fontName,n=[t.fontClass]}else a?(e=De[l].fontName,n=[l]):(e=Le(l,t.fontWeight,t.fontShape),n=[l,t.fontWeight,t.fontShape]);if(Ne(s,e,o).metrics)return qe(s,e,o,t,i.concat(n));if(ke.hasOwnProperty(s)&&"Typewriter"===e.slice(0,10)){const r=[];for(let a=0;a<s.length;a++)r.push(qe(s[a],e,o,t,i.concat(n)));return Ee(r)}}if("mathord"===r)return qe(s,"Math-Italic",o,t,i.concat(["mathnormal"]));if("textord"===r){const e=se[o][s]&&se[o][s].font;if("ams"===e){const e=Le("amsrm",t.fontWeight,t.fontShape);return qe(s,e,o,t,i.concat("amsrm",t.fontWeight,t.fontShape))}if("main"!==e&&e){const r=Le(e,t.fontWeight,t.fontShape);return qe(s,r,o,t,i.concat(r,t.fontWeight,t.fontShape))}{const e=Le("textrm",t.fontWeight,t.fontShape);return qe(s,e,o,t,i.concat(t.fontWeight,t.fontShape))}}throw new Error("unexpected type: "+r+" in makeOrd")},makeGlue:(e,t)=>{const r=He(["mspace"],[],t),n=P(e,t);return r.style.marginRight=F(n),r},staticSvg:function(e,t){const[r,n,o]=Ve[e],s=new Q(r),i=new J([s],{width:F(n),height:F(o),style:"width:"+F(n),viewBox:"0 0 "+1e3*n+" "+1e3*o,preserveAspectRatio:"xMinYMin"}),a=Oe(["overlay"],[i],t);return a.height=o,a.style.height=F(o),a.style.width=F(n),a},svgData:Ve,tryCombineChars:e=>{for(let t=0;t<e.length-1;t++){const r=e[t],n=e[t+1];r instanceof K&&n instanceof K&&Ie(r,n)&&(r.text+=n.text,r.height=Math.max(r.height,n.height),r.depth=Math.max(r.depth,n.depth),r.italic=n.italic,e.splice(t+1,1),t--)}return e}};const Fe={number:3,unit:"mu"},Ge={number:4,unit:"mu"},Ue={number:5,unit:"mu"},Ye={mord:{mop:Fe,mbin:Ge,mrel:Ue,minner:Fe},mop:{mord:Fe,mop:Fe,mrel:Ue,minner:Fe},mbin:{mord:Ge,mop:Ge,mopen:Ge,minner:Ge},mrel:{mord:Ue,mop:Ue,mopen:Ue,minner:Ue},mopen:{},mclose:{mop:Fe,mbin:Ge,mrel:Ue,minner:Fe},mpunct:{mord:Fe,mop:Fe,mrel:Ue,mopen:Fe,mclose:Fe,mpunct:Fe,minner:Fe},minner:{mord:Fe,mop:Fe,mbin:Ge,mrel:Ue,mopen:Fe,mpunct:Fe,minner:Fe}},Xe={mord:{mop:Fe},mop:{mord:Fe,mop:Fe},mbin:{},mrel:{},mopen:{},mclose:{mop:Fe},mpunct:{},minner:{mop:Fe}},We={},_e={},je={};function $e(e){let{type:t,names:r,props:n,handler:o,htmlBuilder:s,mathmlBuilder:i}=e;const a={type:t,numArgs:n.numArgs,argTypes:n.argTypes,allowedInArgument:!!n.allowedInArgument,allowedInText:!!n.allowedInText,allowedInMath:void 0===n.allowedInMath||n.allowedInMath,numOptionalArgs:n.numOptionalArgs||0,infix:!!n.infix,primitive:!!n.primitive,handler:o};for(let e=0;e<r.length;++e)We[r[e]]=a;t&&(s&&(_e[t]=s),i&&(je[t]=i))}function Ze(e){let{type:t,htmlBuilder:r,mathmlBuilder:n}=e;$e({type:t,names:[],props:{numArgs:0},handler(){throw new Error("Should never be called.")},htmlBuilder:r,mathmlBuilder:n})}const Ke=function(e){return"ordgroup"===e.type&&1===e.body.length?e.body[0]:e},Je=function(e){return"ordgroup"===e.type?e.body:[e]},Qe=Pe.makeSpan,et=["leftmost","mbin","mopen","mrel","mop","mpunct"],tt=["rightmost","mrel","mclose","mpunct"],rt={display:w.DISPLAY,text:w.TEXT,script:w.SCRIPT,scriptscript:w.SCRIPTSCRIPT},nt={mord:"mord",mop:"mop",mbin:"mbin",mrel:"mrel",mopen:"mopen",mclose:"mclose",mpunct:"mpunct",minner:"minner"},ot=function(e,t,r,n){void 0===n&&(n=[null,null]);const o=[];for(let r=0;r<e.length;r++){const n=ct(e[r],t);if(n instanceof A){const e=n.children;o.push(...e)}else o.push(n)}if(Pe.tryCombineChars(o),!r)return o;let s=t;if(1===e.length){const r=e[0];"sizing"===r.type?s=t.havingSize(r.size):"styling"===r.type&&(s=t.havingStyle(rt[r.style]))}const i=Qe([n[0]||"leftmost"],[],t),a=Qe([n[1]||"rightmost"],[],t),h="root"===r;return st(o,((e,t)=>{const r=t.classes[0],n=e.classes[0];"mbin"===r&&l.contains(tt,n)?t.classes[0]="mord":"mbin"===n&&l.contains(et,r)&&(e.classes[0]="mord")}),{node:i},a,h),st(o,((e,t)=>{const r=lt(t),n=lt(e),o=r&&n?e.hasClass("mtight")?Xe[r][n]:Ye[r][n]:null;if(o)return Pe.makeGlue(o,s)}),{node:i},a,h),o},st=function(e,t,r,n,o){n&&e.push(n);let s=0;for(;s<e.length;s++){const n=e[s],i=it(n);if(i){st(i.children,t,r,null,o);continue}const a=!n.hasClass("mspace");if(a){const o=t(n,r.node);o&&(r.insertAfter?r.insertAfter(o):(e.unshift(o),s++))}a?r.node=n:o&&n.hasClass("newline")&&(r.node=Qe(["leftmost"])),r.insertAfter=(t=>r=>{e.splice(t+1,0,r),s++})(s)}n&&e.pop()},it=function(e){return e instanceof A||e instanceof j||e instanceof _&&e.hasClass("enclosing")?e:null},at=function(e,t){const r=it(e);if(r){const e=r.children;if(e.length){if("right"===t)return at(e[e.length-1],"right");if("left"===t)return at(e[0],"left")}}return e},lt=function(e,t){return e?(t&&(e=at(e,t)),nt[e.classes[0]]||null):null},ht=function(e,t){const r=["nulldelimiter"].concat(e.baseSizingClasses());return Qe(t.concat(r))},ct=function(e,t,r){if(!e)return Qe();if(_e[e.type]){let n=_e[e.type](e,t);if(r&&t.size!==r.size){n=Qe(t.sizingClasses(r),[n],t);const e=t.sizeMultiplier/r.sizeMultiplier;n.height*=e,n.depth*=e}return n}throw new n("Got group of unknown type: '"+e.type+"'")};function mt(e,t){const r=Qe(["base"],e,t),n=Qe(["strut"]);return n.style.height=F(r.height+r.depth),r.depth&&(n.style.verticalAlign=F(-r.depth)),r.children.unshift(n),r}function pt(e,t){let r=null;1===e.length&&"tag"===e[0].type&&(r=e[0].tag,e=e[0].body);const n=ot(e,t,"root");let o;2===n.length&&n[1].hasClass("tag")&&(o=n.pop());const s=[];let i,a=[];for(let e=0;e<n.length;e++)if(a.push(n[e]),n[e].hasClass("mbin")||n[e].hasClass("mrel")||n[e].hasClass("allowbreak")){let r=!1;for(;e<n.length-1&&n[e+1].hasClass("mspace")&&!n[e+1].hasClass("newline");)e++,a.push(n[e]),n[e].hasClass("nobreak")&&(r=!0);r||(s.push(mt(a,t)),a=[])}else n[e].hasClass("newline")&&(a.pop(),a.length>0&&(s.push(mt(a,t)),a=[]),s.push(n[e]));a.length>0&&s.push(mt(a,t)),r?(i=mt(ot(r,t,!0)),i.classes=["tag"],s.push(i)):o&&s.push(o);const l=Qe(["katex-html"],s);if(l.setAttribute("aria-hidden","true"),i){const e=i.children[0];e.style.height=F(l.height+l.depth),l.depth&&(e.style.verticalAlign=F(-l.depth))}return l}function ut(e){return new A(e)}class dt{constructor(e,t,r){this.type=void 0,this.attributes=void 0,this.children=void 0,this.classes=void 0,this.type=e,this.attributes={},this.children=t||[],this.classes=r||[]}setAttribute(e,t){this.attributes[e]=t}getAttribute(e){return this.attributes[e]}toNode(){const e=document.createElementNS("http://www.w3.org/1998/Math/MathML",this.type);for(const t in this.attributes)Object.prototype.hasOwnProperty.call(this.attributes,t)&&e.setAttribute(t,this.attributes[t]);this.classes.length>0&&(e.className=G(this.classes));for(let t=0;t<this.children.length;t++)if(this.children[t]instanceof gt&&this.children[t+1]instanceof gt){let r=this.children[t].toText()+this.children[++t].toText();for(;this.children[t+1]instanceof gt;)r+=this.children[++t].toText();e.appendChild(new gt(r).toNode())}else e.appendChild(this.children[t].toNode());return e}toMarkup(){let e="<"+this.type;for(const t in this.attributes)Object.prototype.hasOwnProperty.call(this.attributes,t)&&(e+=" "+t+'="',e+=l.escape(this.attributes[t]),e+='"');this.classes.length>0&&(e+=' class ="'+l.escape(G(this.classes))+'"'),e+=">";for(let t=0;t<this.children.length;t++)e+=this.children[t].toMarkup();return e+="</"+this.type+">",e}toText(){return this.children.map((e=>e.toText())).join("")}}class gt{constructor(e){this.text=void 0,this.text=e}toNode(){return document.createTextNode(this.text)}toMarkup(){return l.escape(this.toText())}toText(){return this.text}}var ft={MathNode:dt,TextNode:gt,SpaceNode:class{constructor(e){this.width=void 0,this.character=void 0,this.width=e,this.character=e>=.05555&&e<=.05556?"\u200a":e>=.1666&&e<=.1667?"\u2009":e>=.2222&&e<=.2223?"\u2005":e>=.2777&&e<=.2778?"\u2005\u200a":e>=-.05556&&e<=-.05555?"\u200a\u2063":e>=-.1667&&e<=-.1666?"\u2009\u2063":e>=-.2223&&e<=-.2222?"\u205f\u2063":e>=-.2778&&e<=-.2777?"\u2005\u2063":null}toNode(){if(this.character)return document.createTextNode(this.character);{const e=document.createElementNS("http://www.w3.org/1998/Math/MathML","mspace");return e.setAttribute("width",F(this.width)),e}}toMarkup(){return this.character?"<mtext>"+this.character+"</mtext>":'<mspace width="'+F(this.width)+'"/>'}toText(){return this.character?this.character:" "}},newDocumentFragment:ut};const bt=function(e,t,r){return!se[t][e]||!se[t][e].replace||55349===e.charCodeAt(0)||ke.hasOwnProperty(e)&&r&&(r.fontFamily&&"tt"===r.fontFamily.slice(4,6)||r.font&&"tt"===r.font.slice(4,6))||(e=se[t][e].replace),new ft.TextNode(e)},yt=function(e){return 1===e.length?e[0]:new ft.MathNode("mrow",e)},xt=function(e,t){if("texttt"===t.fontFamily)return"monospace";if("textsf"===t.fontFamily)return"textit"===t.fontShape&&"textbf"===t.fontWeight?"sans-serif-bold-italic":"textit"===t.fontShape?"sans-serif-italic":"textbf"===t.fontWeight?"bold-sans-serif":"sans-serif";if("textit"===t.fontShape&&"textbf"===t.fontWeight)return"bold-italic";if("textit"===t.fontShape)return"italic";if("textbf"===t.fontWeight)return"bold";const r=t.font;if(!r||"mathnormal"===r)return null;const n=e.mode;if("mathit"===r)return"italic";if("boldsymbol"===r)return"textord"===e.type?"bold":"bold-italic";if("mathbf"===r)return"bold";if("mathbb"===r)return"double-struck";if("mathsfit"===r)return"sans-serif-italic";if("mathfrak"===r)return"fraktur";if("mathscr"===r||"mathcal"===r)return"script";if("mathsf"===r)return"sans-serif";if("mathtt"===r)return"monospace";let o=e.text;if(l.contains(["\\imath","\\jmath"],o))return null;se[n][o]&&se[n][o].replace&&(o=se[n][o].replace);return N(o,Pe.fontMap[r].fontName,n)?Pe.fontMap[r].variant:null};function wt(e){if(!e)return!1;if("mi"===e.type&&1===e.children.length){const t=e.children[0];return t instanceof gt&&"."===t.text}if("mo"===e.type&&1===e.children.length&&"true"===e.getAttribute("separator")&&"0em"===e.getAttribute("lspace")&&"0em"===e.getAttribute("rspace")){const t=e.children[0];return t instanceof gt&&","===t.text}return!1}const vt=function(e,t,r){if(1===e.length){const n=St(e[0],t);return r&&n instanceof dt&&"mo"===n.type&&(n.setAttribute("lspace","0em"),n.setAttribute("rspace","0em")),[n]}const n=[];let o;for(let r=0;r<e.length;r++){const s=St(e[r],t);if(s instanceof dt&&o instanceof dt){if("mtext"===s.type&&"mtext"===o.type&&s.getAttribute("mathvariant")===o.getAttribute("mathvariant")){o.children.push(...s.children);continue}if("mn"===s.type&&"mn"===o.type){o.children.push(...s.children);continue}if(wt(s)&&"mn"===o.type){o.children.push(...s.children);continue}if("mn"===s.type&&wt(o))s.children=[...o.children,...s.children],n.pop();else if(("msup"===s.type||"msub"===s.type)&&s.children.length>=1&&("mn"===o.type||wt(o))){const e=s.children[0];e instanceof dt&&"mn"===e.type&&(e.children=[...o.children,...e.children],n.pop())}else if("mi"===o.type&&1===o.children.length){const e=o.children[0];if(e instanceof gt&&"\u0338"===e.text&&("mo"===s.type||"mi"===s.type||"mn"===s.type)){const e=s.children[0];e instanceof gt&&e.text.length>0&&(e.text=e.text.slice(0,1)+"\u0338"+e.text.slice(1),n.pop())}}}n.push(s),o=s}return n},kt=function(e,t,r){return yt(vt(e,t,r))},St=function(e,t){if(!e)return new ft.MathNode("mrow");if(je[e.type]){return je[e.type](e,t)}throw new n("Got group of unknown type: '"+e.type+"'")};function Mt(e,t,r,n,o){const s=vt(e,r);let i;i=1===s.length&&s[0]instanceof dt&&l.contains(["mrow","mtable"],s[0].type)?s[0]:new ft.MathNode("mrow",s);const a=new ft.MathNode("annotation",[new ft.TextNode(t)]);a.setAttribute("encoding","application/x-tex");const h=new ft.MathNode("semantics",[i,a]),c=new ft.MathNode("math",[h]);c.setAttribute("xmlns","http://www.w3.org/1998/Math/MathML"),n&&c.setAttribute("display","block");const m=o?"katex":"katex-mathml";return Pe.makeSpan([m],[c])}const zt=function(e){return new E({style:e.displayMode?w.DISPLAY:w.TEXT,maxSize:e.maxSize,minRuleThickness:e.minRuleThickness})},At=function(e,t){if(t.displayMode){const r=["katex-display"];t.leqno&&r.push("leqno"),t.fleqn&&r.push("fleqn"),e=Pe.makeSpan(r,[e])}return e},Tt=function(e,t,r){const n=zt(r);let o;if("mathml"===r.output)return Mt(e,t,n,r.displayMode,!0);if("html"===r.output){const t=pt(e,n);o=Pe.makeSpan(["katex"],[t])}else{const s=Mt(e,t,n,r.displayMode,!1),i=pt(e,n);o=Pe.makeSpan(["katex"],[s,i])}return At(o,r)};const Bt={widehat:"^",widecheck:"\u02c7",widetilde:"~",utilde:"~",overleftarrow:"\u2190",underleftarrow:"\u2190",xleftarrow:"\u2190",overrightarrow:"\u2192",underrightarrow:"\u2192",xrightarrow:"\u2192",underbrace:"\u23df",overbrace:"\u23de",overgroup:"\u23e0",undergroup:"\u23e1",overleftrightarrow:"\u2194",underleftrightarrow:"\u2194",xleftrightarrow:"\u2194",Overrightarrow:"\u21d2",xRightarrow:"\u21d2",overleftharpoon:"\u21bc",xleftharpoonup:"\u21bc",overrightharpoon:"\u21c0",xrightharpoonup:"\u21c0",xLeftarrow:"\u21d0",xLeftrightarrow:"\u21d4",xhookleftarrow:"\u21a9",xhookrightarrow:"\u21aa",xmapsto:"\u21a6",xrightharpoondown:"\u21c1",xleftharpoondown:"\u21bd",xrightleftharpoons:"\u21cc",xleftrightharpoons:"\u21cb",xtwoheadleftarrow:"\u219e",xtwoheadrightarrow:"\u21a0",xlongequal:"=",xtofrom:"\u21c4",xrightleftarrows:"\u21c4",xrightequilibrium:"\u21cc",xleftequilibrium:"\u21cb","\\cdrightarrow":"\u2192","\\cdleftarrow":"\u2190","\\cdlongequal":"="},Ct={overrightarrow:[["rightarrow"],.888,522,"xMaxYMin"],overleftarrow:[["leftarrow"],.888,522,"xMinYMin"],underrightarrow:[["rightarrow"],.888,522,"xMaxYMin"],underleftarrow:[["leftarrow"],.888,522,"xMinYMin"],xrightarrow:[["rightarrow"],1.469,522,"xMaxYMin"],"\\cdrightarrow":[["rightarrow"],3,522,"xMaxYMin"],xleftarrow:[["leftarrow"],1.469,522,"xMinYMin"],"\\cdleftarrow":[["leftarrow"],3,522,"xMinYMin"],Overrightarrow:[["doublerightarrow"],.888,560,"xMaxYMin"],xRightarrow:[["doublerightarrow"],1.526,560,"xMaxYMin"],xLeftarrow:[["doubleleftarrow"],1.526,560,"xMinYMin"],overleftharpoon:[["leftharpoon"],.888,522,"xMinYMin"],xleftharpoonup:[["leftharpoon"],.888,522,"xMinYMin"],xleftharpoondown:[["leftharpoondown"],.888,522,"xMinYMin"],overrightharpoon:[["rightharpoon"],.888,522,"xMaxYMin"],xrightharpoonup:[["rightharpoon"],.888,522,"xMaxYMin"],xrightharpoondown:[["rightharpoondown"],.888,522,"xMaxYMin"],xlongequal:[["longequal"],.888,334,"xMinYMin"],"\\cdlongequal":[["longequal"],3,334,"xMinYMin"],xtwoheadleftarrow:[["twoheadleftarrow"],.888,334,"xMinYMin"],xtwoheadrightarrow:[["twoheadrightarrow"],.888,334,"xMaxYMin"],overleftrightarrow:[["leftarrow","rightarrow"],.888,522],overbrace:[["leftbrace","midbrace","rightbrace"],1.6,548],underbrace:[["leftbraceunder","midbraceunder","rightbraceunder"],1.6,548],underleftrightarrow:[["leftarrow","rightarrow"],.888,522],xleftrightarrow:[["leftarrow","rightarrow"],1.75,522],xLeftrightarrow:[["doubleleftarrow","doublerightarrow"],1.75,560],xrightleftharpoons:[["leftharpoondownplus","rightharpoonplus"],1.75,716],xleftrightharpoons:[["leftharpoonplus","rightharpoondownplus"],1.75,716],xhookleftarrow:[["leftarrow","righthook"],1.08,522],xhookrightarrow:[["lefthook","rightarrow"],1.08,522],overlinesegment:[["leftlinesegment","rightlinesegment"],.888,522],underlinesegment:[["leftlinesegment","rightlinesegment"],.888,522],overgroup:[["leftgroup","rightgroup"],.888,342],undergroup:[["leftgroupunder","rightgroupunder"],.888,342],xmapsto:[["leftmapsto","rightarrow"],1.5,522],xtofrom:[["leftToFrom","rightToFrom"],1.75,528],xrightleftarrows:[["baraboveleftarrow","rightarrowabovebar"],1.75,901],xrightequilibrium:[["baraboveshortleftharpoon","rightharpoonaboveshortbar"],1.75,716],xleftequilibrium:[["shortbaraboveleftharpoon","shortrightharpoonabovebar"],1.75,716]};var Nt=function(e,t,r,n,o){let s;const i=e.height+e.depth+r+n;if(/fbox|color|angl/.test(t)){if(s=Pe.makeSpan(["stretchy",t],[],o),"fbox"===t){const e=o.color&&o.getColor();e&&(s.style.borderColor=e)}}else{const e=[];/^[bx]cancel$/.test(t)&&e.push(new ee({x1:"0",y1:"0",x2:"100%",y2:"100%","stroke-width":"0.046em"})),/^x?cancel$/.test(t)&&e.push(new ee({x1:"0",y1:"100%",x2:"100%",y2:"0","stroke-width":"0.046em"}));const r=new J(e,{width:"100%",height:F(i)});s=Pe.makeSvgSpan([],[r],o)}return s.height=i,s.style.height=F(i),s},qt=function(e){const t=new ft.MathNode("mo",[new ft.TextNode(Bt[e.replace(/^\\/,"")])]);return t.setAttribute("stretchy","true"),t},It=function(e,t){const{span:r,minWidth:n,height:o}=function(){let r=4e5;const n=e.label.slice(1);if(l.contains(["widehat","widecheck","widetilde","utilde"],n)){const s="ordgroup"===(o=e.base).type?o.body.length:1;let i,a,l;if(s>5)"widehat"===n||"widecheck"===n?(i=420,r=2364,l=.42,a=n+"4"):(i=312,r=2340,l=.34,a="tilde4");else{const e=[1,1,2,2,3,3][s];"widehat"===n||"widecheck"===n?(r=[0,1062,2364,2364,2364][e],i=[0,239,300,360,420][e],l=[0,.24,.3,.3,.36,.42][e],a=n+e):(r=[0,600,1033,2339,2340][e],i=[0,260,286,306,312][e],l=[0,.26,.286,.3,.306,.34][e],a="tilde"+e)}const h=new Q(a),c=new J([h],{width:"100%",height:F(l),viewBox:"0 0 "+r+" "+i,preserveAspectRatio:"none"});return{span:Pe.makeSvgSpan([],[c],t),minWidth:0,height:l}}{const e=[],o=Ct[n],[s,i,a]=o,l=a/1e3,h=s.length;let c,m;if(1===h){c=["hide-tail"],m=[o[3]]}else if(2===h)c=["halfarrow-left","halfarrow-right"],m=["xMinYMin","xMaxYMin"];else{if(3!==h)throw new Error("Correct katexImagesData or update code here to support\n                    "+h+" children.");c=["brace-left","brace-center","brace-right"],m=["xMinYMin","xMidYMin","xMaxYMin"]}for(let n=0;n<h;n++){const o=new Q(s[n]),p=new J([o],{width:"400em",height:F(l),viewBox:"0 0 "+r+" "+a,preserveAspectRatio:m[n]+" slice"}),u=Pe.makeSvgSpan([c[n]],[p],t);if(1===h)return{span:u,minWidth:i,height:l};u.style.height=F(l),e.push(u)}return{span:Pe.makeSpan(["stretchy"],e,t),minWidth:i,height:l}}var o}();return r.height=o,r.style.height=F(o),n>0&&(r.style.minWidth=F(n)),r};function Rt(e,t){if(!e||e.type!==t)throw new Error("Expected node of type "+t+", but got "+(e?"node of type "+e.type:String(e)));return e}function Ht(e){const t=Ot(e);if(!t)throw new Error("Expected node of symbol group type, but got "+(e?"node of type "+e.type:String(e)));return t}function Ot(e){return e&&("atom"===e.type||ne.hasOwnProperty(e.type))?e:null}const Et=(e,t)=>{let r,n,o;e&&"supsub"===e.type?(n=Rt(e.base,"accent"),r=n.base,e.base=r,o=function(e){if(e instanceof _)return e;throw new Error("Expected span<HtmlDomNode> but got "+String(e)+".")}(ct(e,t)),e.base=n):(n=Rt(e,"accent"),r=n.base);const s=ct(r,t.havingCrampedStyle());let i=0;if(n.isShifty&&l.isCharacterBox(r)){const e=l.getBaseElem(r);i=te(ct(e,t.havingCrampedStyle())).skew}const a="\\c"===n.label;let h,c=a?s.height+s.depth:Math.min(s.height,t.fontMetrics().xHeight);if(n.isStretchy)h=It(n,t),h=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:s},{type:"elem",elem:h,wrapperClasses:["svg-align"],wrapperStyle:i>0?{width:"calc(100% - "+F(2*i)+")",marginLeft:F(2*i)}:void 0}]},t);else{let e,r;"\\vec"===n.label?(e=Pe.staticSvg("vec",t),r=Pe.svgData.vec[1]):(e=Pe.makeOrd({mode:n.mode,text:n.label},t,"textord"),e=te(e),e.italic=0,r=e.width,a&&(c+=e.depth)),h=Pe.makeSpan(["accent-body"],[e]);const o="\\textcircled"===n.label;o&&(h.classes.push("accent-full"),c=s.height);let l=i;o||(l-=r/2),h.style.left=F(l),"\\textcircled"===n.label&&(h.style.top=".2em"),h=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:s},{type:"kern",size:-c},{type:"elem",elem:h}]},t)}const m=Pe.makeSpan(["mord","accent"],[h],t);return o?(o.children[0]=m,o.height=Math.max(m.height,o.height),o.classes[0]="mord",o):m},Lt=(e,t)=>{const r=e.isStretchy?qt(e.label):new ft.MathNode("mo",[bt(e.label,e.mode)]),n=new ft.MathNode("mover",[St(e.base,t),r]);return n.setAttribute("accent","true"),n},Dt=new RegExp(["\\acute","\\grave","\\ddot","\\tilde","\\bar","\\breve","\\check","\\hat","\\vec","\\dot","\\mathring"].map((e=>"\\"+e)).join("|"));$e({type:"accent",names:["\\acute","\\grave","\\ddot","\\tilde","\\bar","\\breve","\\check","\\hat","\\vec","\\dot","\\mathring","\\widecheck","\\widehat","\\widetilde","\\overrightarrow","\\overleftarrow","\\Overrightarrow","\\overleftrightarrow","\\overgroup","\\overlinesegment","\\overleftharpoon","\\overrightharpoon"],props:{numArgs:1},handler:(e,t)=>{const r=Ke(t[0]),n=!Dt.test(e.funcName),o=!n||"\\widehat"===e.funcName||"\\widetilde"===e.funcName||"\\widecheck"===e.funcName;return{type:"accent",mode:e.parser.mode,label:e.funcName,isStretchy:n,isShifty:o,base:r}},htmlBuilder:Et,mathmlBuilder:Lt}),$e({type:"accent",names:["\\'","\\`","\\^","\\~","\\=","\\u","\\.",'\\"',"\\c","\\r","\\H","\\v","\\textcircled"],props:{numArgs:1,allowedInText:!0,allowedInMath:!0,argTypes:["primitive"]},handler:(e,t)=>{const r=t[0];let n=e.parser.mode;return"math"===n&&(e.parser.settings.reportNonstrict("mathVsTextAccents","LaTeX's accent "+e.funcName+" works only in text mode"),n="text"),{type:"accent",mode:n,label:e.funcName,isStretchy:!1,isShifty:!0,base:r}},htmlBuilder:Et,mathmlBuilder:Lt}),$e({type:"accentUnder",names:["\\underleftarrow","\\underrightarrow","\\underleftrightarrow","\\undergroup","\\underlinesegment","\\utilde"],props:{numArgs:1},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=t[0];return{type:"accentUnder",mode:r.mode,label:n,base:o}},htmlBuilder:(e,t)=>{const r=ct(e.base,t),n=It(e,t),o="\\utilde"===e.label?.12:0,s=Pe.makeVList({positionType:"top",positionData:r.height,children:[{type:"elem",elem:n,wrapperClasses:["svg-align"]},{type:"kern",size:o},{type:"elem",elem:r}]},t);return Pe.makeSpan(["mord","accentunder"],[s],t)},mathmlBuilder:(e,t)=>{const r=qt(e.label),n=new ft.MathNode("munder",[St(e.base,t),r]);return n.setAttribute("accentunder","true"),n}});const Vt=e=>{const t=new ft.MathNode("mpadded",e?[e]:[]);return t.setAttribute("width","+0.6em"),t.setAttribute("lspace","0.3em"),t};$e({type:"xArrow",names:["\\xleftarrow","\\xrightarrow","\\xLeftarrow","\\xRightarrow","\\xleftrightarrow","\\xLeftrightarrow","\\xhookleftarrow","\\xhookrightarrow","\\xmapsto","\\xrightharpoondown","\\xrightharpoonup","\\xleftharpoondown","\\xleftharpoonup","\\xrightleftharpoons","\\xleftrightharpoons","\\xlongequal","\\xtwoheadrightarrow","\\xtwoheadleftarrow","\\xtofrom","\\xrightleftarrows","\\xrightequilibrium","\\xleftequilibrium","\\\\cdrightarrow","\\\\cdleftarrow","\\\\cdlongequal"],props:{numArgs:1,numOptionalArgs:1},handler(e,t,r){let{parser:n,funcName:o}=e;return{type:"xArrow",mode:n.mode,label:o,body:t[0],below:r[0]}},htmlBuilder(e,t){const r=t.style;let n=t.havingStyle(r.sup());const o=Pe.wrapFragment(ct(e.body,n,t),t),s="\\x"===e.label.slice(0,2)?"x":"cd";let i;o.classes.push(s+"-arrow-pad"),e.below&&(n=t.havingStyle(r.sub()),i=Pe.wrapFragment(ct(e.below,n,t),t),i.classes.push(s+"-arrow-pad"));const a=It(e,t),l=-t.fontMetrics().axisHeight+.5*a.height;let h,c=-t.fontMetrics().axisHeight-.5*a.height-.111;if((o.depth>.25||"\\xleftequilibrium"===e.label)&&(c-=o.depth),i){const e=-t.fontMetrics().axisHeight+i.height+.5*a.height+.111;h=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:o,shift:c},{type:"elem",elem:a,shift:l},{type:"elem",elem:i,shift:e}]},t)}else h=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:o,shift:c},{type:"elem",elem:a,shift:l}]},t);return h.children[0].children[0].children[1].classes.push("svg-align"),Pe.makeSpan(["mrel","x-arrow"],[h],t)},mathmlBuilder(e,t){const r=qt(e.label);let n;if(r.setAttribute("minsize","x"===e.label.charAt(0)?"1.75em":"3.0em"),e.body){const o=Vt(St(e.body,t));if(e.below){const s=Vt(St(e.below,t));n=new ft.MathNode("munderover",[r,s,o])}else n=new ft.MathNode("mover",[r,o])}else if(e.below){const o=Vt(St(e.below,t));n=new ft.MathNode("munder",[r,o])}else n=Vt(),n=new ft.MathNode("mover",[r,n]);return n}});const Pt=Pe.makeSpan;function Ft(e,t){const r=ot(e.body,t,!0);return Pt([e.mclass],r,t)}function Gt(e,t){let r;const n=vt(e.body,t);return"minner"===e.mclass?r=new ft.MathNode("mpadded",n):"mord"===e.mclass?e.isCharacterBox?(r=n[0],r.type="mi"):r=new ft.MathNode("mi",n):(e.isCharacterBox?(r=n[0],r.type="mo"):r=new ft.MathNode("mo",n),"mbin"===e.mclass?(r.attributes.lspace="0.22em",r.attributes.rspace="0.22em"):"mpunct"===e.mclass?(r.attributes.lspace="0em",r.attributes.rspace="0.17em"):"mopen"===e.mclass||"mclose"===e.mclass?(r.attributes.lspace="0em",r.attributes.rspace="0em"):"minner"===e.mclass&&(r.attributes.lspace="0.0556em",r.attributes.width="+0.1111em")),r}$e({type:"mclass",names:["\\mathord","\\mathbin","\\mathrel","\\mathopen","\\mathclose","\\mathpunct","\\mathinner"],props:{numArgs:1,primitive:!0},handler(e,t){let{parser:r,funcName:n}=e;const o=t[0];return{type:"mclass",mode:r.mode,mclass:"m"+n.slice(5),body:Je(o),isCharacterBox:l.isCharacterBox(o)}},htmlBuilder:Ft,mathmlBuilder:Gt});const Ut=e=>{const t="ordgroup"===e.type&&e.body.length?e.body[0]:e;return"atom"!==t.type||"bin"!==t.family&&"rel"!==t.family?"mord":"m"+t.family};$e({type:"mclass",names:["\\@binrel"],props:{numArgs:2},handler(e,t){let{parser:r}=e;return{type:"mclass",mode:r.mode,mclass:Ut(t[0]),body:Je(t[1]),isCharacterBox:l.isCharacterBox(t[1])}}}),$e({type:"mclass",names:["\\stackrel","\\overset","\\underset"],props:{numArgs:2},handler(e,t){let{parser:r,funcName:n}=e;const o=t[1],s=t[0];let i;i="\\stackrel"!==n?Ut(o):"mrel";const a={type:"op",mode:o.mode,limits:!0,alwaysHandleSupSub:!0,parentIsSupSub:!1,symbol:!1,suppressBaseShift:"\\stackrel"!==n,body:Je(o)},h={type:"supsub",mode:s.mode,base:a,sup:"\\underset"===n?null:s,sub:"\\underset"===n?s:null};return{type:"mclass",mode:r.mode,mclass:i,body:[h],isCharacterBox:l.isCharacterBox(h)}},htmlBuilder:Ft,mathmlBuilder:Gt}),$e({type:"pmb",names:["\\pmb"],props:{numArgs:1,allowedInText:!0},handler(e,t){let{parser:r}=e;return{type:"pmb",mode:r.mode,mclass:Ut(t[0]),body:Je(t[0])}},htmlBuilder(e,t){const r=ot(e.body,t,!0),n=Pe.makeSpan([e.mclass],r,t);return n.style.textShadow="0.02em 0.01em 0.04px",n},mathmlBuilder(e,t){const r=vt(e.body,t),n=new ft.MathNode("mstyle",r);return n.setAttribute("style","text-shadow: 0.02em 0.01em 0.04px"),n}});const Yt={">":"\\\\cdrightarrow","<":"\\\\cdleftarrow","=":"\\\\cdlongequal",A:"\\uparrow",V:"\\downarrow","|":"\\Vert",".":"no arrow"},Xt=e=>"textord"===e.type&&"@"===e.text;function Wt(e,t,r){const n=Yt[e];switch(n){case"\\\\cdrightarrow":case"\\\\cdleftarrow":return r.callFunction(n,[t[0]],[t[1]]);case"\\uparrow":case"\\downarrow":{const e={type:"atom",text:n,mode:"math",family:"rel"},o={type:"ordgroup",mode:"math",body:[r.callFunction("\\\\cdleft",[t[0]],[]),r.callFunction("\\Big",[e],[]),r.callFunction("\\\\cdright",[t[1]],[])]};return r.callFunction("\\\\cdparent",[o],[])}case"\\\\cdlongequal":return r.callFunction("\\\\cdlongequal",[],[]);case"\\Vert":{const e={type:"textord",text:"\\Vert",mode:"math"};return r.callFunction("\\Big",[e],[])}default:return{type:"textord",text:" ",mode:"math"}}}$e({type:"cdlabel",names:["\\\\cdleft","\\\\cdright"],props:{numArgs:1},handler(e,t){let{parser:r,funcName:n}=e;return{type:"cdlabel",mode:r.mode,side:n.slice(4),label:t[0]}},htmlBuilder(e,t){const r=t.havingStyle(t.style.sup()),n=Pe.wrapFragment(ct(e.label,r,t),t);return n.classes.push("cd-label-"+e.side),n.style.bottom=F(.8-n.depth),n.height=0,n.depth=0,n},mathmlBuilder(e,t){let r=new ft.MathNode("mrow",[St(e.label,t)]);return r=new ft.MathNode("mpadded",[r]),r.setAttribute("width","0"),"left"===e.side&&r.setAttribute("lspace","-1width"),r.setAttribute("voffset","0.7em"),r=new ft.MathNode("mstyle",[r]),r.setAttribute("displaystyle","false"),r.setAttribute("scriptlevel","1"),r}}),$e({type:"cdlabelparent",names:["\\\\cdparent"],props:{numArgs:1},handler(e,t){let{parser:r}=e;return{type:"cdlabelparent",mode:r.mode,fragment:t[0]}},htmlBuilder(e,t){const r=Pe.wrapFragment(ct(e.fragment,t),t);return r.classes.push("cd-vert-arrow"),r},mathmlBuilder(e,t){return new ft.MathNode("mrow",[St(e.fragment,t)])}}),$e({type:"textord",names:["\\@char"],props:{numArgs:1,allowedInText:!0},handler(e,t){let{parser:r}=e;const o=Rt(t[0],"ordgroup").body;let s="";for(let e=0;e<o.length;e++){s+=Rt(o[e],"textord").text}let i,a=parseInt(s);if(isNaN(a))throw new n("\\@char has non-numeric argument "+s);if(a<0||a>=1114111)throw new n("\\@char with invalid code point "+s);return a<=65535?i=String.fromCharCode(a):(a-=65536,i=String.fromCharCode(55296+(a>>10),56320+(1023&a))),{type:"textord",mode:r.mode,text:i}}});const _t=(e,t)=>{const r=ot(e.body,t.withColor(e.color),!1);return Pe.makeFragment(r)},jt=(e,t)=>{const r=vt(e.body,t.withColor(e.color)),n=new ft.MathNode("mstyle",r);return n.setAttribute("mathcolor",e.color),n};$e({type:"color",names:["\\textcolor"],props:{numArgs:2,allowedInText:!0,argTypes:["color","original"]},handler(e,t){let{parser:r}=e;const n=Rt(t[0],"color-token").color,o=t[1];return{type:"color",mode:r.mode,color:n,body:Je(o)}},htmlBuilder:_t,mathmlBuilder:jt}),$e({type:"color",names:["\\color"],props:{numArgs:1,allowedInText:!0,argTypes:["color"]},handler(e,t){let{parser:r,breakOnTokenText:n}=e;const o=Rt(t[0],"color-token").color;r.gullet.macros.set("\\current@color",o);const s=r.parseExpression(!0,n);return{type:"color",mode:r.mode,color:o,body:s}},htmlBuilder:_t,mathmlBuilder:jt}),$e({type:"cr",names:["\\\\"],props:{numArgs:0,numOptionalArgs:0,allowedInText:!0},handler(e,t,r){let{parser:n}=e;const o="["===n.gullet.future().text?n.parseSizeGroup(!0):null,s=!n.settings.displayMode||!n.settings.useStrictBehavior("newLineInDisplayMode","In LaTeX, \\\\ or \\newline does nothing in display mode");return{type:"cr",mode:n.mode,newLine:s,size:o&&Rt(o,"size").value}},htmlBuilder(e,t){const r=Pe.makeSpan(["mspace"],[],t);return e.newLine&&(r.classes.push("newline"),e.size&&(r.style.marginTop=F(P(e.size,t)))),r},mathmlBuilder(e,t){const r=new ft.MathNode("mspace");return e.newLine&&(r.setAttribute("linebreak","newline"),e.size&&r.setAttribute("height",F(P(e.size,t)))),r}});const $t={"\\global":"\\global","\\long":"\\\\globallong","\\\\globallong":"\\\\globallong","\\def":"\\gdef","\\gdef":"\\gdef","\\edef":"\\xdef","\\xdef":"\\xdef","\\let":"\\\\globallet","\\futurelet":"\\\\globalfuture"},Zt=e=>{const t=e.text;if(/^(?:[\\{}$&#^_]|EOF)$/.test(t))throw new n("Expected a control sequence",e);return t},Kt=(e,t,r,n)=>{let o=e.gullet.macros.get(r.text);null==o&&(r.noexpand=!0,o={tokens:[r],numArgs:0,unexpandable:!e.gullet.isExpandable(r.text)}),e.gullet.macros.set(t,o,n)};$e({type:"internal",names:["\\global","\\long","\\\\globallong"],props:{numArgs:0,allowedInText:!0},handler(e){let{parser:t,funcName:r}=e;t.consumeSpaces();const o=t.fetch();if($t[o.text])return"\\global"!==r&&"\\\\globallong"!==r||(o.text=$t[o.text]),Rt(t.parseFunction(),"internal");throw new n("Invalid token after macro prefix",o)}}),$e({type:"internal",names:["\\def","\\gdef","\\edef","\\xdef"],props:{numArgs:0,allowedInText:!0,primitive:!0},handler(e){let{parser:t,funcName:r}=e,o=t.gullet.popToken();const s=o.text;if(/^(?:[\\{}$&#^_]|EOF)$/.test(s))throw new n("Expected a control sequence",o);let i,a=0;const l=[[]];for(;"{"!==t.gullet.future().text;)if(o=t.gullet.popToken(),"#"===o.text){if("{"===t.gullet.future().text){i=t.gullet.future(),l[a].push("{");break}if(o=t.gullet.popToken(),!/^[1-9]$/.test(o.text))throw new n('Invalid argument number "'+o.text+'"');if(parseInt(o.text)!==a+1)throw new n('Argument number "'+o.text+'" out of order');a++,l.push([])}else{if("EOF"===o.text)throw new n("Expected a macro definition");l[a].push(o.text)}let{tokens:h}=t.gullet.consumeArg();return i&&h.unshift(i),"\\edef"!==r&&"\\xdef"!==r||(h=t.gullet.expandTokens(h),h.reverse()),t.gullet.macros.set(s,{tokens:h,numArgs:a,delimiters:l},r===$t[r]),{type:"internal",mode:t.mode}}}),$e({type:"internal",names:["\\let","\\\\globallet"],props:{numArgs:0,allowedInText:!0,primitive:!0},handler(e){let{parser:t,funcName:r}=e;const n=Zt(t.gullet.popToken());t.gullet.consumeSpaces();const o=(e=>{let t=e.gullet.popToken();return"="===t.text&&(t=e.gullet.popToken()," "===t.text&&(t=e.gullet.popToken())),t})(t);return Kt(t,n,o,"\\\\globallet"===r),{type:"internal",mode:t.mode}}}),$e({type:"internal",names:["\\futurelet","\\\\globalfuture"],props:{numArgs:0,allowedInText:!0,primitive:!0},handler(e){let{parser:t,funcName:r}=e;const n=Zt(t.gullet.popToken()),o=t.gullet.popToken(),s=t.gullet.popToken();return Kt(t,n,s,"\\\\globalfuture"===r),t.gullet.pushToken(s),t.gullet.pushToken(o),{type:"internal",mode:t.mode}}});const Jt=function(e,t,r){const n=N(se.math[e]&&se.math[e].replace||e,t,r);if(!n)throw new Error("Unsupported symbol "+e+" and font size "+t+".");return n},Qt=function(e,t,r,n){const o=r.havingBaseStyle(t),s=Pe.makeSpan(n.concat(o.sizingClasses(r)),[e],r),i=o.sizeMultiplier/r.sizeMultiplier;return s.height*=i,s.depth*=i,s.maxFontSize=o.sizeMultiplier,s},er=function(e,t,r){const n=t.havingBaseStyle(r),o=(1-t.sizeMultiplier/n.sizeMultiplier)*t.fontMetrics().axisHeight;e.classes.push("delimcenter"),e.style.top=F(o),e.height-=o,e.depth+=o},tr=function(e,t,r,n,o,s){const i=function(e,t,r,n){return Pe.makeSymbol(e,"Size"+t+"-Regular",r,n)}(e,t,o,n),a=Qt(Pe.makeSpan(["delimsizing","size"+t],[i],n),w.TEXT,n,s);return r&&er(a,n,w.TEXT),a},rr=function(e,t,r){let n;n="Size1-Regular"===t?"delim-size1":"delim-size4";return{type:"elem",elem:Pe.makeSpan(["delimsizinginner",n],[Pe.makeSpan([],[Pe.makeSymbol(e,t,r)])])}},nr=function(e,t,r){const n=T["Size4-Regular"][e.charCodeAt(0)]?T["Size4-Regular"][e.charCodeAt(0)][4]:T["Size1-Regular"][e.charCodeAt(0)][4],o=new Q("inner",function(e,t){switch(e){case"\u239c":return"M291 0 H417 V"+t+" H291z M291 0 H417 V"+t+" H291z";case"\u2223":return"M145 0 H188 V"+t+" H145z M145 0 H188 V"+t+" H145z";case"\u2225":return"M145 0 H188 V"+t+" H145z M145 0 H188 V"+t+" H145zM367 0 H410 V"+t+" H367z M367 0 H410 V"+t+" H367z";case"\u239f":return"M457 0 H583 V"+t+" H457z M457 0 H583 V"+t+" H457z";case"\u23a2":return"M319 0 H403 V"+t+" H319z M319 0 H403 V"+t+" H319z";case"\u23a5":return"M263 0 H347 V"+t+" H263z M263 0 H347 V"+t+" H263z";case"\u23aa":return"M384 0 H504 V"+t+" H384z M384 0 H504 V"+t+" H384z";case"\u23d0":return"M312 0 H355 V"+t+" H312z M312 0 H355 V"+t+" H312z";case"\u2016":return"M257 0 H300 V"+t+" H257z M257 0 H300 V"+t+" H257zM478 0 H521 V"+t+" H478z M478 0 H521 V"+t+" H478z";default:return""}}(e,Math.round(1e3*t))),s=new J([o],{width:F(n),height:F(t),style:"width:"+F(n),viewBox:"0 0 "+1e3*n+" "+Math.round(1e3*t),preserveAspectRatio:"xMinYMin"}),i=Pe.makeSvgSpan([],[s],r);return i.height=t,i.style.height=F(t),i.style.width=F(n),{type:"elem",elem:i}},or={type:"kern",size:-.008},sr=["|","\\lvert","\\rvert","\\vert"],ir=["\\|","\\lVert","\\rVert","\\Vert"],ar=function(e,t,r,n,o,s){let i,a,h,c,m="",p=0;i=h=c=e,a=null;let u="Size1-Regular";"\\uparrow"===e?h=c="\u23d0":"\\Uparrow"===e?h=c="\u2016":"\\downarrow"===e?i=h="\u23d0":"\\Downarrow"===e?i=h="\u2016":"\\updownarrow"===e?(i="\\uparrow",h="\u23d0",c="\\downarrow"):"\\Updownarrow"===e?(i="\\Uparrow",h="\u2016",c="\\Downarrow"):l.contains(sr,e)?(h="\u2223",m="vert",p=333):l.contains(ir,e)?(h="\u2225",m="doublevert",p=556):"["===e||"\\lbrack"===e?(i="\u23a1",h="\u23a2",c="\u23a3",u="Size4-Regular",m="lbrack",p=667):"]"===e||"\\rbrack"===e?(i="\u23a4",h="\u23a5",c="\u23a6",u="Size4-Regular",m="rbrack",p=667):"\\lfloor"===e||"\u230a"===e?(h=i="\u23a2",c="\u23a3",u="Size4-Regular",m="lfloor",p=667):"\\lceil"===e||"\u2308"===e?(i="\u23a1",h=c="\u23a2",u="Size4-Regular",m="lceil",p=667):"\\rfloor"===e||"\u230b"===e?(h=i="\u23a5",c="\u23a6",u="Size4-Regular",m="rfloor",p=667):"\\rceil"===e||"\u2309"===e?(i="\u23a4",h=c="\u23a5",u="Size4-Regular",m="rceil",p=667):"("===e||"\\lparen"===e?(i="\u239b",h="\u239c",c="\u239d",u="Size4-Regular",m="lparen",p=875):")"===e||"\\rparen"===e?(i="\u239e",h="\u239f",c="\u23a0",u="Size4-Regular",m="rparen",p=875):"\\{"===e||"\\lbrace"===e?(i="\u23a7",a="\u23a8",c="\u23a9",h="\u23aa",u="Size4-Regular"):"\\}"===e||"\\rbrace"===e?(i="\u23ab",a="\u23ac",c="\u23ad",h="\u23aa",u="Size4-Regular"):"\\lgroup"===e||"\u27ee"===e?(i="\u23a7",c="\u23a9",h="\u23aa",u="Size4-Regular"):"\\rgroup"===e||"\u27ef"===e?(i="\u23ab",c="\u23ad",h="\u23aa",u="Size4-Regular"):"\\lmoustache"===e||"\u23b0"===e?(i="\u23a7",c="\u23ad",h="\u23aa",u="Size4-Regular"):"\\rmoustache"!==e&&"\u23b1"!==e||(i="\u23ab",c="\u23a9",h="\u23aa",u="Size4-Regular");const d=Jt(i,u,o),g=d.height+d.depth,f=Jt(h,u,o),b=f.height+f.depth,y=Jt(c,u,o),x=y.height+y.depth;let v=0,k=1;if(null!==a){const e=Jt(a,u,o);v=e.height+e.depth,k=2}const S=g+x+v,M=S+Math.max(0,Math.ceil((t-S)/(k*b)))*k*b;let z=n.fontMetrics().axisHeight;r&&(z*=n.sizeMultiplier);const A=M/2-z,T=[];if(m.length>0){const e=M-g-x,t=Math.round(1e3*M),r=function(e,t){switch(e){case"lbrack":return"M403 1759 V84 H666 V0 H319 V1759 v"+t+" v1759 h347 v-84\nH403z M403 1759 V0 H319 V1759 v"+t+" v1759 h84z";case"rbrack":return"M347 1759 V0 H0 V84 H263 V1759 v"+t+" v1759 H0 v84 H347z\nM347 1759 V0 H263 V1759 v"+t+" v1759 h84z";case"vert":return"M145 15 v585 v"+t+" v585 c2.667,10,9.667,15,21,15\nc10,0,16.667,-5,20,-15 v-585 v"+-t+" v-585 c-2.667,-10,-9.667,-15,-21,-15\nc-10,0,-16.667,5,-20,15z M188 15 H145 v585 v"+t+" v585 h43z";case"doublevert":return"M145 15 v585 v"+t+" v585 c2.667,10,9.667,15,21,15\nc10,0,16.667,-5,20,-15 v-585 v"+-t+" v-585 c-2.667,-10,-9.667,-15,-21,-15\nc-10,0,-16.667,5,-20,15z M188 15 H145 v585 v"+t+" v585 h43z\nM367 15 v585 v"+t+" v585 c2.667,10,9.667,15,21,15\nc10,0,16.667,-5,20,-15 v-585 v"+-t+" v-585 c-2.667,-10,-9.667,-15,-21,-15\nc-10,0,-16.667,5,-20,15z M410 15 H367 v585 v"+t+" v585 h43z";case"lfloor":return"M319 602 V0 H403 V602 v"+t+" v1715 h263 v84 H319z\nMM319 602 V0 H403 V602 v"+t+" v1715 H319z";case"rfloor":return"M319 602 V0 H403 V602 v"+t+" v1799 H0 v-84 H319z\nMM319 602 V0 H403 V602 v"+t+" v1715 H319z";case"lceil":return"M403 1759 V84 H666 V0 H319 V1759 v"+t+" v602 h84z\nM403 1759 V0 H319 V1759 v"+t+" v602 h84z";case"rceil":return"M347 1759 V0 H0 V84 H263 V1759 v"+t+" v602 h84z\nM347 1759 V0 h-84 V1759 v"+t+" v602 h84z";case"lparen":return"M863,9c0,-2,-2,-5,-6,-9c0,0,-17,0,-17,0c-12.7,0,-19.3,0.3,-20,1\nc-5.3,5.3,-10.3,11,-15,17c-242.7,294.7,-395.3,682,-458,1162c-21.3,163.3,-33.3,349,\n-36,557 l0,"+(t+84)+"c0.2,6,0,26,0,60c2,159.3,10,310.7,24,454c53.3,528,210,\n949.7,470,1265c4.7,6,9.7,11.7,15,17c0.7,0.7,7,1,19,1c0,0,18,0,18,0c4,-4,6,-7,6,-9\nc0,-2.7,-3.3,-8.7,-10,-18c-135.3,-192.7,-235.5,-414.3,-300.5,-665c-65,-250.7,-102.5,\n-544.7,-112.5,-882c-2,-104,-3,-167,-3,-189\nl0,-"+(t+92)+"c0,-162.7,5.7,-314,17,-454c20.7,-272,63.7,-513,129,-723c65.3,\n-210,155.3,-396.3,270,-559c6.7,-9.3,10,-15.3,10,-18z";case"rparen":return"M76,0c-16.7,0,-25,3,-25,9c0,2,2,6.3,6,13c21.3,28.7,42.3,60.3,\n63,95c96.7,156.7,172.8,332.5,228.5,527.5c55.7,195,92.8,416.5,111.5,664.5\nc11.3,139.3,17,290.7,17,454c0,28,1.7,43,3.3,45l0,"+(t+9)+"\nc-3,4,-3.3,16.7,-3.3,38c0,162,-5.7,313.7,-17,455c-18.7,248,-55.8,469.3,-111.5,664\nc-55.7,194.7,-131.8,370.3,-228.5,527c-20.7,34.7,-41.7,66.3,-63,95c-2,3.3,-4,7,-6,11\nc0,7.3,5.7,11,17,11c0,0,11,0,11,0c9.3,0,14.3,-0.3,15,-1c5.3,-5.3,10.3,-11,15,-17\nc242.7,-294.7,395.3,-681.7,458,-1161c21.3,-164.7,33.3,-350.7,36,-558\nl0,-"+(t+144)+"c-2,-159.3,-10,-310.7,-24,-454c-53.3,-528,-210,-949.7,\n-470,-1265c-4.7,-6,-9.7,-11.7,-15,-17c-0.7,-0.7,-6.7,-1,-18,-1z";default:throw new Error("Unknown stretchy delimiter.")}}(m,Math.round(1e3*e)),o=new Q(m,r),s=(p/1e3).toFixed(3)+"em",i=(t/1e3).toFixed(3)+"em",a=new J([o],{width:s,height:i,viewBox:"0 0 "+p+" "+t}),l=Pe.makeSvgSpan([],[a],n);l.height=t/1e3,l.style.width=s,l.style.height=i,T.push({type:"elem",elem:l})}else{if(T.push(rr(c,u,o)),T.push(or),null===a){const e=M-g-x+.016;T.push(nr(h,e,n))}else{const e=(M-g-x-v)/2+.016;T.push(nr(h,e,n)),T.push(or),T.push(rr(a,u,o)),T.push(or),T.push(nr(h,e,n))}T.push(or),T.push(rr(i,u,o))}const B=n.havingBaseStyle(w.TEXT),C=Pe.makeVList({positionType:"bottom",positionData:A,children:T},B);return Qt(Pe.makeSpan(["delimsizing","mult"],[C],B),w.TEXT,n,s)},lr=.08,hr=function(e,t,r,n,o){const s=function(e,t,r){t*=1e3;let n="";switch(e){case"sqrtMain":n=function(e,t){return"M95,"+(622+e+t)+"\nc-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14\nc0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54\nc44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10\ns173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429\nc69,-144,104.5,-217.7,106.5,-221\nl"+e/2.075+" -"+e+"\nc5.3,-9.3,12,-14,20,-14\nH400000v"+(40+e)+"H845.2724\ns-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7\nc-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z\nM"+(834+e)+" "+t+"h400000v"+(40+e)+"h-400000z"}(t,M);break;case"sqrtSize1":n=function(e,t){return"M263,"+(601+e+t)+"c0.7,0,18,39.7,52,119\nc34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120\nc340,-704.7,510.7,-1060.3,512,-1067\nl"+e/2.084+" -"+e+"\nc4.7,-7.3,11,-11,19,-11\nH40000v"+(40+e)+"H1012.3\ns-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232\nc-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1\ns-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26\nc-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z\nM"+(1001+e)+" "+t+"h400000v"+(40+e)+"h-400000z"}(t,M);break;case"sqrtSize2":n=function(e,t){return"M983 "+(10+e+t)+"\nl"+e/3.13+" -"+e+"\nc4,-6.7,10,-10,18,-10 H400000v"+(40+e)+"\nH1013.1s-83.4,268,-264.1,840c-180.7,572,-277,876.3,-289,913c-4.7,4.7,-12.7,7,-24,7\ns-12,0,-12,0c-1.3,-3.3,-3.7,-11.7,-7,-25c-35.3,-125.3,-106.7,-373.3,-214,-744\nc-10,12,-21,25,-33,39s-32,39,-32,39c-6,-5.3,-15,-14,-27,-26s25,-30,25,-30\nc26.7,-32.7,52,-63,76,-91s52,-60,52,-60s208,722,208,722\nc56,-175.3,126.3,-397.3,211,-666c84.7,-268.7,153.8,-488.2,207.5,-658.5\nc53.7,-170.3,84.5,-266.8,92.5,-289.5z\nM"+(1001+e)+" "+t+"h400000v"+(40+e)+"h-400000z"}(t,M);break;case"sqrtSize3":n=function(e,t){return"M424,"+(2398+e+t)+"\nc-1.3,-0.7,-38.5,-172,-111.5,-514c-73,-342,-109.8,-513.3,-110.5,-514\nc0,-2,-10.7,14.3,-32,49c-4.7,7.3,-9.8,15.7,-15.5,25c-5.7,9.3,-9.8,16,-12.5,20\ns-5,7,-5,7c-4,-3.3,-8.3,-7.7,-13,-13s-13,-13,-13,-13s76,-122,76,-122s77,-121,77,-121\ns209,968,209,968c0,-2,84.7,-361.7,254,-1079c169.3,-717.3,254.7,-1077.7,256,-1081\nl"+e/4.223+" -"+e+"c4,-6.7,10,-10,18,-10 H400000\nv"+(40+e)+"H1014.6\ns-87.3,378.7,-272.6,1166c-185.3,787.3,-279.3,1182.3,-282,1185\nc-2,6,-10,9,-24,9\nc-8,0,-12,-0.7,-12,-2z M"+(1001+e)+" "+t+"\nh400000v"+(40+e)+"h-400000z"}(t,M);break;case"sqrtSize4":n=function(e,t){return"M473,"+(2713+e+t)+"\nc339.3,-1799.3,509.3,-2700,510,-2702 l"+e/5.298+" -"+e+"\nc3.3,-7.3,9.3,-11,18,-11 H400000v"+(40+e)+"H1017.7\ns-90.5,478,-276.2,1466c-185.7,988,-279.5,1483,-281.5,1485c-2,6,-10,9,-24,9\nc-8,0,-12,-0.7,-12,-2c0,-1.3,-5.3,-32,-16,-92c-50.7,-293.3,-119.7,-693.3,-207,-1200\nc0,-1.3,-5.3,8.7,-16,30c-10.7,21.3,-21.3,42.7,-32,64s-16,33,-16,33s-26,-26,-26,-26\ns76,-153,76,-153s77,-151,77,-151c0.7,0.7,35.7,202,105,604c67.3,400.7,102,602.7,104,\n606zM"+(1001+e)+" "+t+"h400000v"+(40+e)+"H1017.7z"}(t,M);break;case"sqrtTall":n=function(e,t,r){return"M702 "+(e+t)+"H400000"+(40+e)+"\nH742v"+(r-54-t-e)+"l-4 4-4 4c-.667.7 -2 1.5-4 2.5s-4.167 1.833-6.5 2.5-5.5 1-9.5 1\nh-12l-28-84c-16.667-52-96.667 -294.333-240-727l-212 -643 -85 170\nc-4-3.333-8.333-7.667-13 -13l-13-13l77-155 77-156c66 199.333 139 419.667\n219 661 l218 661zM702 "+t+"H400000v"+(40+e)+"H742z"}(t,M,r)}return n}(e,n,r),i=new Q(e,s),a=new J([i],{width:"400em",height:F(t),viewBox:"0 0 400000 "+r,preserveAspectRatio:"xMinYMin slice"});return Pe.makeSvgSpan(["hide-tail"],[a],o)},cr=["(","\\lparen",")","\\rparen","[","\\lbrack","]","\\rbrack","\\{","\\lbrace","\\}","\\rbrace","\\lfloor","\\rfloor","\u230a","\u230b","\\lceil","\\rceil","\u2308","\u2309","\\surd"],mr=["\\uparrow","\\downarrow","\\updownarrow","\\Uparrow","\\Downarrow","\\Updownarrow","|","\\|","\\vert","\\Vert","\\lvert","\\rvert","\\lVert","\\rVert","\\lgroup","\\rgroup","\u27ee","\u27ef","\\lmoustache","\\rmoustache","\u23b0","\u23b1"],pr=["<",">","\\langle","\\rangle","/","\\backslash","\\lt","\\gt"],ur=[0,1.2,1.8,2.4,3],dr=[{type:"small",style:w.SCRIPTSCRIPT},{type:"small",style:w.SCRIPT},{type:"small",style:w.TEXT},{type:"large",size:1},{type:"large",size:2},{type:"large",size:3},{type:"large",size:4}],gr=[{type:"small",style:w.SCRIPTSCRIPT},{type:"small",style:w.SCRIPT},{type:"small",style:w.TEXT},{type:"stack"}],fr=[{type:"small",style:w.SCRIPTSCRIPT},{type:"small",style:w.SCRIPT},{type:"small",style:w.TEXT},{type:"large",size:1},{type:"large",size:2},{type:"large",size:3},{type:"large",size:4},{type:"stack"}],br=function(e){if("small"===e.type)return"Main-Regular";if("large"===e.type)return"Size"+e.size+"-Regular";if("stack"===e.type)return"Size4-Regular";throw new Error("Add support for delim type '"+e.type+"' here.")},yr=function(e,t,r,n){for(let o=Math.min(2,3-n.style.size);o<r.length&&"stack"!==r[o].type;o++){const s=Jt(e,br(r[o]),"math");let i=s.height+s.depth;if("small"===r[o].type){i*=n.havingBaseStyle(r[o].style).sizeMultiplier}if(i>t)return r[o]}return r[r.length-1]},xr=function(e,t,r,n,o,s){let i;"<"===e||"\\lt"===e||"\u27e8"===e?e="\\langle":">"!==e&&"\\gt"!==e&&"\u27e9"!==e||(e="\\rangle"),i=l.contains(pr,e)?dr:l.contains(cr,e)?fr:gr;const a=yr(e,t,i,n);return"small"===a.type?function(e,t,r,n,o,s){const i=Pe.makeSymbol(e,"Main-Regular",o,n),a=Qt(i,t,n,s);return r&&er(a,n,t),a}(e,a.style,r,n,o,s):"large"===a.type?tr(e,a.size,r,n,o,s):ar(e,t,r,n,o,s)};var wr={sqrtImage:function(e,t){const r=t.havingBaseSizing(),n=yr("\\surd",e*r.sizeMultiplier,fr,r);let o=r.sizeMultiplier;const s=Math.max(0,t.minRuleThickness-t.fontMetrics().sqrtRuleThickness);let i,a,l=0,h=0,c=0;return"small"===n.type?(c=1e3+1e3*s+80,e<1?o=1:e<1.4&&(o=.7),l=(1+s+lr)/o,h=(1+s)/o,i=hr("sqrtMain",l,c,s,t),i.style.minWidth="0.853em",a=.833/o):"large"===n.type?(c=1080*ur[n.size],h=(ur[n.size]+s)/o,l=(ur[n.size]+s+lr)/o,i=hr("sqrtSize"+n.size,l,c,s,t),i.style.minWidth="1.02em",a=1/o):(l=e+s+lr,h=e+s,c=Math.floor(1e3*e+s)+80,i=hr("sqrtTall",l,c,s,t),i.style.minWidth="0.742em",a=1.056),i.height=h,i.style.height=F(l),{span:i,advanceWidth:a,ruleWidth:(t.fontMetrics().sqrtRuleThickness+s)*o}},sizedDelim:function(e,t,r,o,s){if("<"===e||"\\lt"===e||"\u27e8"===e?e="\\langle":">"!==e&&"\\gt"!==e&&"\u27e9"!==e||(e="\\rangle"),l.contains(cr,e)||l.contains(pr,e))return tr(e,t,!1,r,o,s);if(l.contains(mr,e))return ar(e,ur[t],!1,r,o,s);throw new n("Illegal delimiter: '"+e+"'")},sizeToMaxHeight:ur,customSizedDelim:xr,leftRightDelim:function(e,t,r,n,o,s){const i=n.fontMetrics().axisHeight*n.sizeMultiplier,a=5/n.fontMetrics().ptPerEm,l=Math.max(t-i,r+i),h=Math.max(l/500*901,2*l-a);return xr(e,h,!0,n,o,s)}};const vr={"\\bigl":{mclass:"mopen",size:1},"\\Bigl":{mclass:"mopen",size:2},"\\biggl":{mclass:"mopen",size:3},"\\Biggl":{mclass:"mopen",size:4},"\\bigr":{mclass:"mclose",size:1},"\\Bigr":{mclass:"mclose",size:2},"\\biggr":{mclass:"mclose",size:3},"\\Biggr":{mclass:"mclose",size:4},"\\bigm":{mclass:"mrel",size:1},"\\Bigm":{mclass:"mrel",size:2},"\\biggm":{mclass:"mrel",size:3},"\\Biggm":{mclass:"mrel",size:4},"\\big":{mclass:"mord",size:1},"\\Big":{mclass:"mord",size:2},"\\bigg":{mclass:"mord",size:3},"\\Bigg":{mclass:"mord",size:4}},kr=["(","\\lparen",")","\\rparen","[","\\lbrack","]","\\rbrack","\\{","\\lbrace","\\}","\\rbrace","\\lfloor","\\rfloor","\u230a","\u230b","\\lceil","\\rceil","\u2308","\u2309","<",">","\\langle","\u27e8","\\rangle","\u27e9","\\lt","\\gt","\\lvert","\\rvert","\\lVert","\\rVert","\\lgroup","\\rgroup","\u27ee","\u27ef","\\lmoustache","\\rmoustache","\u23b0","\u23b1","/","\\backslash","|","\\vert","\\|","\\Vert","\\uparrow","\\Uparrow","\\downarrow","\\Downarrow","\\updownarrow","\\Updownarrow","."];function Sr(e,t){const r=Ot(e);if(r&&l.contains(kr,r.text))return r;throw new n(r?"Invalid delimiter '"+r.text+"' after '"+t.funcName+"'":"Invalid delimiter type '"+e.type+"'",e)}function Mr(e){if(!e.body)throw new Error("Bug: The leftright ParseNode wasn't fully parsed.")}$e({type:"delimsizing",names:["\\bigl","\\Bigl","\\biggl","\\Biggl","\\bigr","\\Bigr","\\biggr","\\Biggr","\\bigm","\\Bigm","\\biggm","\\Biggm","\\big","\\Big","\\bigg","\\Bigg"],props:{numArgs:1,argTypes:["primitive"]},handler:(e,t)=>{const r=Sr(t[0],e);return{type:"delimsizing",mode:e.parser.mode,size:vr[e.funcName].size,mclass:vr[e.funcName].mclass,delim:r.text}},htmlBuilder:(e,t)=>"."===e.delim?Pe.makeSpan([e.mclass]):wr.sizedDelim(e.delim,e.size,t,e.mode,[e.mclass]),mathmlBuilder:e=>{const t=[];"."!==e.delim&&t.push(bt(e.delim,e.mode));const r=new ft.MathNode("mo",t);"mopen"===e.mclass||"mclose"===e.mclass?r.setAttribute("fence","true"):r.setAttribute("fence","false"),r.setAttribute("stretchy","true");const n=F(wr.sizeToMaxHeight[e.size]);return r.setAttribute("minsize",n),r.setAttribute("maxsize",n),r}}),$e({type:"leftright-right",names:["\\right"],props:{numArgs:1,primitive:!0},handler:(e,t)=>{const r=e.parser.gullet.macros.get("\\current@color");if(r&&"string"!=typeof r)throw new n("\\current@color set to non-string in \\right");return{type:"leftright-right",mode:e.parser.mode,delim:Sr(t[0],e).text,color:r}}}),$e({type:"leftright",names:["\\left"],props:{numArgs:1,primitive:!0},handler:(e,t)=>{const r=Sr(t[0],e),n=e.parser;++n.leftrightDepth;const o=n.parseExpression(!1);--n.leftrightDepth,n.expect("\\right",!1);const s=Rt(n.parseFunction(),"leftright-right");return{type:"leftright",mode:n.mode,body:o,left:r.text,right:s.delim,rightColor:s.color}},htmlBuilder:(e,t)=>{Mr(e);const r=ot(e.body,t,!0,["mopen","mclose"]);let n,o,s=0,i=0,a=!1;for(let e=0;e<r.length;e++)r[e].isMiddle?a=!0:(s=Math.max(r[e].height,s),i=Math.max(r[e].depth,i));if(s*=t.sizeMultiplier,i*=t.sizeMultiplier,n="."===e.left?ht(t,["mopen"]):wr.leftRightDelim(e.left,s,i,t,e.mode,["mopen"]),r.unshift(n),a)for(let t=1;t<r.length;t++){const n=r[t].isMiddle;n&&(r[t]=wr.leftRightDelim(n.delim,s,i,n.options,e.mode,[]))}if("."===e.right)o=ht(t,["mclose"]);else{const r=e.rightColor?t.withColor(e.rightColor):t;o=wr.leftRightDelim(e.right,s,i,r,e.mode,["mclose"])}return r.push(o),Pe.makeSpan(["minner"],r,t)},mathmlBuilder:(e,t)=>{Mr(e);const r=vt(e.body,t);if("."!==e.left){const t=new ft.MathNode("mo",[bt(e.left,e.mode)]);t.setAttribute("fence","true"),r.unshift(t)}if("."!==e.right){const t=new ft.MathNode("mo",[bt(e.right,e.mode)]);t.setAttribute("fence","true"),e.rightColor&&t.setAttribute("mathcolor",e.rightColor),r.push(t)}return yt(r)}}),$e({type:"middle",names:["\\middle"],props:{numArgs:1,primitive:!0},handler:(e,t)=>{const r=Sr(t[0],e);if(!e.parser.leftrightDepth)throw new n("\\middle without preceding \\left",r);return{type:"middle",mode:e.parser.mode,delim:r.text}},htmlBuilder:(e,t)=>{let r;if("."===e.delim)r=ht(t,[]);else{r=wr.sizedDelim(e.delim,1,t,e.mode,[]);const n={delim:e.delim,options:t};r.isMiddle=n}return r},mathmlBuilder:(e,t)=>{const r="\\vert"===e.delim||"|"===e.delim?bt("|","text"):bt(e.delim,e.mode),n=new ft.MathNode("mo",[r]);return n.setAttribute("fence","true"),n.setAttribute("lspace","0.05em"),n.setAttribute("rspace","0.05em"),n}});const zr=(e,t)=>{const r=Pe.wrapFragment(ct(e.body,t),t),n=e.label.slice(1);let o,s=t.sizeMultiplier,i=0;const a=l.isCharacterBox(e.body);if("sout"===n)o=Pe.makeSpan(["stretchy","sout"]),o.height=t.fontMetrics().defaultRuleThickness/s,i=-.5*t.fontMetrics().xHeight;else if("phase"===n){const e=P({number:.6,unit:"pt"},t),n=P({number:.35,unit:"ex"},t);s/=t.havingBaseSizing().sizeMultiplier;const a=r.height+r.depth+e+n;r.style.paddingLeft=F(a/2+e);const l=Math.floor(1e3*a*s),c="M400000 "+(h=l)+" H0 L"+h/2+" 0 l65 45 L145 "+(h-80)+" H400000z",m=new J([new Q("phase",c)],{width:"400em",height:F(l/1e3),viewBox:"0 0 400000 "+l,preserveAspectRatio:"xMinYMin slice"});o=Pe.makeSvgSpan(["hide-tail"],[m],t),o.style.height=F(a),i=r.depth+e+n}else{/cancel/.test(n)?a||r.classes.push("cancel-pad"):"angl"===n?r.classes.push("anglpad"):r.classes.push("boxpad");let s=0,l=0,h=0;/box/.test(n)?(h=Math.max(t.fontMetrics().fboxrule,t.minRuleThickness),s=t.fontMetrics().fboxsep+("colorbox"===n?0:h),l=s):"angl"===n?(h=Math.max(t.fontMetrics().defaultRuleThickness,t.minRuleThickness),s=4*h,l=Math.max(0,.25-r.depth)):(s=a?.2:0,l=s),o=Nt(r,n,s,l,t),/fbox|boxed|fcolorbox/.test(n)?(o.style.borderStyle="solid",o.style.borderWidth=F(h)):"angl"===n&&.049!==h&&(o.style.borderTopWidth=F(h),o.style.borderRightWidth=F(h)),i=r.depth+l,e.backgroundColor&&(o.style.backgroundColor=e.backgroundColor,e.borderColor&&(o.style.borderColor=e.borderColor))}var h;let c;if(e.backgroundColor)c=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:o,shift:i},{type:"elem",elem:r,shift:0}]},t);else{const e=/cancel|phase/.test(n)?["svg-align"]:[];c=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:r,shift:0},{type:"elem",elem:o,shift:i,wrapperClasses:e}]},t)}return/cancel/.test(n)&&(c.height=r.height,c.depth=r.depth),/cancel/.test(n)&&!a?Pe.makeSpan(["mord","cancel-lap"],[c],t):Pe.makeSpan(["mord"],[c],t)},Ar=(e,t)=>{let r=0;const n=new ft.MathNode(e.label.indexOf("colorbox")>-1?"mpadded":"menclose",[St(e.body,t)]);switch(e.label){case"\\cancel":n.setAttribute("notation","updiagonalstrike");break;case"\\bcancel":n.setAttribute("notation","downdiagonalstrike");break;case"\\phase":n.setAttribute("notation","phasorangle");break;case"\\sout":n.setAttribute("notation","horizontalstrike");break;case"\\fbox":n.setAttribute("notation","box");break;case"\\angl":n.setAttribute("notation","actuarial");break;case"\\fcolorbox":case"\\colorbox":if(r=t.fontMetrics().fboxsep*t.fontMetrics().ptPerEm,n.setAttribute("width","+"+2*r+"pt"),n.setAttribute("height","+"+2*r+"pt"),n.setAttribute("lspace",r+"pt"),n.setAttribute("voffset",r+"pt"),"\\fcolorbox"===e.label){const r=Math.max(t.fontMetrics().fboxrule,t.minRuleThickness);n.setAttribute("style","border: "+r+"em solid "+String(e.borderColor))}break;case"\\xcancel":n.setAttribute("notation","updiagonalstrike downdiagonalstrike")}return e.backgroundColor&&n.setAttribute("mathbackground",e.backgroundColor),n};$e({type:"enclose",names:["\\colorbox"],props:{numArgs:2,allowedInText:!0,argTypes:["color","text"]},handler(e,t,r){let{parser:n,funcName:o}=e;const s=Rt(t[0],"color-token").color,i=t[1];return{type:"enclose",mode:n.mode,label:o,backgroundColor:s,body:i}},htmlBuilder:zr,mathmlBuilder:Ar}),$e({type:"enclose",names:["\\fcolorbox"],props:{numArgs:3,allowedInText:!0,argTypes:["color","color","text"]},handler(e,t,r){let{parser:n,funcName:o}=e;const s=Rt(t[0],"color-token").color,i=Rt(t[1],"color-token").color,a=t[2];return{type:"enclose",mode:n.mode,label:o,backgroundColor:i,borderColor:s,body:a}},htmlBuilder:zr,mathmlBuilder:Ar}),$e({type:"enclose",names:["\\fbox"],props:{numArgs:1,argTypes:["hbox"],allowedInText:!0},handler(e,t){let{parser:r}=e;return{type:"enclose",mode:r.mode,label:"\\fbox",body:t[0]}}}),$e({type:"enclose",names:["\\cancel","\\bcancel","\\xcancel","\\sout","\\phase"],props:{numArgs:1},handler(e,t){let{parser:r,funcName:n}=e;const o=t[0];return{type:"enclose",mode:r.mode,label:n,body:o}},htmlBuilder:zr,mathmlBuilder:Ar}),$e({type:"enclose",names:["\\angl"],props:{numArgs:1,argTypes:["hbox"],allowedInText:!1},handler(e,t){let{parser:r}=e;return{type:"enclose",mode:r.mode,label:"\\angl",body:t[0]}}});const Tr={};function Br(e){let{type:t,names:r,props:n,handler:o,htmlBuilder:s,mathmlBuilder:i}=e;const a={type:t,numArgs:n.numArgs||0,allowedInText:!1,numOptionalArgs:0,handler:o};for(let e=0;e<r.length;++e)Tr[r[e]]=a;s&&(_e[t]=s),i&&(je[t]=i)}const Cr={};function Nr(e,t){Cr[e]=t}class qr{constructor(e,t,r){this.lexer=void 0,this.start=void 0,this.end=void 0,this.lexer=e,this.start=t,this.end=r}static range(e,t){return t?e&&e.loc&&t.loc&&e.loc.lexer===t.loc.lexer?new qr(e.loc.lexer,e.loc.start,t.loc.end):null:e&&e.loc}}class Ir{constructor(e,t){this.text=void 0,this.loc=void 0,this.noexpand=void 0,this.treatAsRelax=void 0,this.text=e,this.loc=t}range(e,t){return new Ir(t,qr.range(this,e))}}function Rr(e){const t=[];e.consumeSpaces();let r=e.fetch().text;for("\\relax"===r&&(e.consume(),e.consumeSpaces(),r=e.fetch().text);"\\hline"===r||"\\hdashline"===r;)e.consume(),t.push("\\hdashline"===r),e.consumeSpaces(),r=e.fetch().text;return t}const Hr=e=>{if(!e.parser.settings.displayMode)throw new n("{"+e.envName+"} can be used only in display mode.")};function Or(e){if(-1===e.indexOf("ed"))return-1===e.indexOf("*")}function Er(e,t,r){let{hskipBeforeAndAfter:o,addJot:s,cols:i,arraystretch:a,colSeparationType:l,autoTag:h,singleRow:c,emptySingleRow:m,maxNumCols:p,leqno:u}=t;if(e.gullet.beginGroup(),c||e.gullet.macros.set("\\cr","\\\\\\relax"),!a){const t=e.gullet.expandMacroAsText("\\arraystretch");if(null==t)a=1;else if(a=parseFloat(t),!a||a<0)throw new n("Invalid \\arraystretch: "+t)}e.gullet.beginGroup();let d=[];const g=[d],f=[],b=[],y=null!=h?[]:void 0;function x(){h&&e.gullet.macros.set("\\@eqnsw","1",!0)}function w(){y&&(e.gullet.macros.get("\\df@tag")?(y.push(e.subparse([new Ir("\\df@tag")])),e.gullet.macros.set("\\df@tag",void 0,!0)):y.push(Boolean(h)&&"1"===e.gullet.macros.get("\\@eqnsw")))}for(x(),b.push(Rr(e));;){let t=e.parseExpression(!1,c?"\\end":"\\\\");e.gullet.endGroup(),e.gullet.beginGroup(),t={type:"ordgroup",mode:e.mode,body:t},r&&(t={type:"styling",mode:e.mode,style:r,body:[t]}),d.push(t);const o=e.fetch().text;if("&"===o){if(p&&d.length===p){if(c||l)throw new n("Too many tab characters: &",e.nextToken);e.settings.reportNonstrict("textEnv","Too few columns specified in the {array} column argument.")}e.consume()}else{if("\\end"===o){w(),1===d.length&&"styling"===t.type&&0===t.body[0].body.length&&(g.length>1||!m)&&g.pop(),b.length<g.length+1&&b.push([]);break}if("\\\\"!==o)throw new n("Expected & or \\\\ or \\cr or \\end",e.nextToken);{let t;e.consume()," "!==e.gullet.future().text&&(t=e.parseSizeGroup(!0)),f.push(t?t.value:null),w(),b.push(Rr(e)),d=[],g.push(d),x()}}}return e.gullet.endGroup(),e.gullet.endGroup(),{type:"array",mode:e.mode,addJot:s,arraystretch:a,body:g,cols:i,rowGaps:f,hskipBeforeAndAfter:o,hLinesBeforeRow:b,colSeparationType:l,tags:y,leqno:u}}function Lr(e){return"d"===e.slice(0,1)?"display":"text"}const Dr=function(e,t){let r,o;const s=e.body.length,i=e.hLinesBeforeRow;let a=0,h=new Array(s);const c=[],m=Math.max(t.fontMetrics().arrayRuleWidth,t.minRuleThickness),p=1/t.fontMetrics().ptPerEm;let u=5*p;if(e.colSeparationType&&"small"===e.colSeparationType){u=t.havingStyle(w.SCRIPT).sizeMultiplier/t.sizeMultiplier*.2778}const d="CD"===e.colSeparationType?P({number:3,unit:"ex"},t):12*p,g=3*p,f=e.arraystretch*d,b=.7*f,y=.3*f;let x=0;function v(e){for(let t=0;t<e.length;++t)t>0&&(x+=.25),c.push({pos:x,isDashed:e[t]})}for(v(i[0]),r=0;r<e.body.length;++r){const n=e.body[r];let s=b,l=y;a<n.length&&(a=n.length);const c=new Array(n.length);for(o=0;o<n.length;++o){const e=ct(n[o],t);l<e.depth&&(l=e.depth),s<e.height&&(s=e.height),c[o]=e}const m=e.rowGaps[r];let p=0;m&&(p=P(m,t),p>0&&(p+=y,l<p&&(l=p),p=0)),e.addJot&&(l+=g),c.height=s,c.depth=l,x+=s,c.pos=x,x+=l+p,h[r]=c,v(i[r+1])}const k=x/2+t.fontMetrics().axisHeight,S=e.cols||[],M=[];let z,A;const T=[];if(e.tags&&e.tags.some((e=>e)))for(r=0;r<s;++r){const n=h[r],o=n.pos-k,s=e.tags[r];let i;i=!0===s?Pe.makeSpan(["eqn-num"],[],t):!1===s?Pe.makeSpan([],[],t):Pe.makeSpan([],ot(s,t,!0),t),i.depth=n.depth,i.height=n.height,T.push({type:"elem",elem:i,shift:o})}for(o=0,A=0;o<a||A<S.length;++o,++A){let i,c=S[A]||{},p=!0;for(;"separator"===c.type;){if(p||(z=Pe.makeSpan(["arraycolsep"],[]),z.style.width=F(t.fontMetrics().doubleRuleSep),M.push(z)),"|"!==c.separator&&":"!==c.separator)throw new n("Invalid separator type: "+c.separator);{const e="|"===c.separator?"solid":"dashed",r=Pe.makeSpan(["vertical-separator"],[],t);r.style.height=F(x),r.style.borderRightWidth=F(m),r.style.borderRightStyle=e,r.style.margin="0 "+F(-m/2);const n=x-k;n&&(r.style.verticalAlign=F(-n)),M.push(r)}A++,c=S[A]||{},p=!1}if(o>=a)continue;(o>0||e.hskipBeforeAndAfter)&&(i=l.deflt(c.pregap,u),0!==i&&(z=Pe.makeSpan(["arraycolsep"],[]),z.style.width=F(i),M.push(z)));let d=[];for(r=0;r<s;++r){const e=h[r],t=e[o];if(!t)continue;const n=e.pos-k;t.depth=e.depth,t.height=e.height,d.push({type:"elem",elem:t,shift:n})}d=Pe.makeVList({positionType:"individualShift",children:d},t),d=Pe.makeSpan(["col-align-"+(c.align||"c")],[d]),M.push(d),(o<a-1||e.hskipBeforeAndAfter)&&(i=l.deflt(c.postgap,u),0!==i&&(z=Pe.makeSpan(["arraycolsep"],[]),z.style.width=F(i),M.push(z)))}if(h=Pe.makeSpan(["mtable"],M),c.length>0){const e=Pe.makeLineSpan("hline",t,m),r=Pe.makeLineSpan("hdashline",t,m),n=[{type:"elem",elem:h,shift:0}];for(;c.length>0;){const t=c.pop(),o=t.pos-k;t.isDashed?n.push({type:"elem",elem:r,shift:o}):n.push({type:"elem",elem:e,shift:o})}h=Pe.makeVList({positionType:"individualShift",children:n},t)}if(0===T.length)return Pe.makeSpan(["mord"],[h],t);{let e=Pe.makeVList({positionType:"individualShift",children:T},t);return e=Pe.makeSpan(["tag"],[e],t),Pe.makeFragment([h,e])}},Vr={c:"center ",l:"left ",r:"right "},Pr=function(e,t){const r=[],n=new ft.MathNode("mtd",[],["mtr-glue"]),o=new ft.MathNode("mtd",[],["mml-eqn-num"]);for(let s=0;s<e.body.length;s++){const i=e.body[s],a=[];for(let e=0;e<i.length;e++)a.push(new ft.MathNode("mtd",[St(i[e],t)]));e.tags&&e.tags[s]&&(a.unshift(n),a.push(n),e.leqno?a.unshift(o):a.push(o)),r.push(new ft.MathNode("mtr",a))}let s=new ft.MathNode("mtable",r);const i=.5===e.arraystretch?.1:.16+e.arraystretch-1+(e.addJot?.09:0);s.setAttribute("rowspacing",F(i));let a="",l="";if(e.cols&&e.cols.length>0){const t=e.cols;let r="",n=!1,o=0,i=t.length;"separator"===t[0].type&&(a+="top ",o=1),"separator"===t[t.length-1].type&&(a+="bottom ",i-=1);for(let e=o;e<i;e++)"align"===t[e].type?(l+=Vr[t[e].align],n&&(r+="none "),n=!0):"separator"===t[e].type&&n&&(r+="|"===t[e].separator?"solid ":"dashed ",n=!1);s.setAttribute("columnalign",l.trim()),/[sd]/.test(r)&&s.setAttribute("columnlines",r.trim())}if("align"===e.colSeparationType){const t=e.cols||[];let r="";for(let e=1;e<t.length;e++)r+=e%2?"0em ":"1em ";s.setAttribute("columnspacing",r.trim())}else"alignat"===e.colSeparationType||"gather"===e.colSeparationType?s.setAttribute("columnspacing","0em"):"small"===e.colSeparationType?s.setAttribute("columnspacing","0.2778em"):"CD"===e.colSeparationType?s.setAttribute("columnspacing","0.5em"):s.setAttribute("columnspacing","1em");let h="";const c=e.hLinesBeforeRow;a+=c[0].length>0?"left ":"",a+=c[c.length-1].length>0?"right ":"";for(let e=1;e<c.length-1;e++)h+=0===c[e].length?"none ":c[e][0]?"dashed ":"solid ";return/[sd]/.test(h)&&s.setAttribute("rowlines",h.trim()),""!==a&&(s=new ft.MathNode("menclose",[s]),s.setAttribute("notation",a.trim())),e.arraystretch&&e.arraystretch<1&&(s=new ft.MathNode("mstyle",[s]),s.setAttribute("scriptlevel","1")),s},Fr=function(e,t){-1===e.envName.indexOf("ed")&&Hr(e);const r=[],o=e.envName.indexOf("at")>-1?"alignat":"align",s="split"===e.envName,i=Er(e.parser,{cols:r,addJot:!0,autoTag:s?void 0:Or(e.envName),emptySingleRow:!0,colSeparationType:o,maxNumCols:s?2:void 0,leqno:e.parser.settings.leqno},"display");let a,l=0;const h={type:"ordgroup",mode:e.mode,body:[]};if(t[0]&&"ordgroup"===t[0].type){let e="";for(let r=0;r<t[0].body.length;r++){e+=Rt(t[0].body[r],"textord").text}a=Number(e),l=2*a}const c=!l;i.body.forEach((function(e){for(let t=1;t<e.length;t+=2){const r=Rt(e[t],"styling");Rt(r.body[0],"ordgroup").body.unshift(h)}if(c)l<e.length&&(l=e.length);else{const t=e.length/2;if(a<t)throw new n("Too many math in a row: expected "+a+", but got "+t,e[0])}}));for(let e=0;e<l;++e){let t="r",n=0;e%2==1?t="l":e>0&&c&&(n=1),r[e]={type:"align",align:t,pregap:n,postgap:0}}return i.colSeparationType=c?"align":"alignat",i};Br({type:"array",names:["array","darray"],props:{numArgs:1},handler(e,t){const r=(Ot(t[0])?[t[0]]:Rt(t[0],"ordgroup").body).map((function(e){const t=Ht(e).text;if(-1!=="lcr".indexOf(t))return{type:"align",align:t};if("|"===t)return{type:"separator",separator:"|"};if(":"===t)return{type:"separator",separator:":"};throw new n("Unknown column alignment: "+t,e)})),o={cols:r,hskipBeforeAndAfter:!0,maxNumCols:r.length};return Er(e.parser,o,Lr(e.envName))},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["matrix","pmatrix","bmatrix","Bmatrix","vmatrix","Vmatrix","matrix*","pmatrix*","bmatrix*","Bmatrix*","vmatrix*","Vmatrix*"],props:{numArgs:0},handler(e){const t={matrix:null,pmatrix:["(",")"],bmatrix:["[","]"],Bmatrix:["\\{","\\}"],vmatrix:["|","|"],Vmatrix:["\\Vert","\\Vert"]}[e.envName.replace("*","")];let r="c";const o={hskipBeforeAndAfter:!1,cols:[{type:"align",align:r}]};if("*"===e.envName.charAt(e.envName.length-1)){const t=e.parser;if(t.consumeSpaces(),"["===t.fetch().text){if(t.consume(),t.consumeSpaces(),r=t.fetch().text,-1==="lcr".indexOf(r))throw new n("Expected l or c or r",t.nextToken);t.consume(),t.consumeSpaces(),t.expect("]"),t.consume(),o.cols=[{type:"align",align:r}]}}const s=Er(e.parser,o,Lr(e.envName)),i=Math.max(0,...s.body.map((e=>e.length)));return s.cols=new Array(i).fill({type:"align",align:r}),t?{type:"leftright",mode:e.mode,body:[s],left:t[0],right:t[1],rightColor:void 0}:s},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["smallmatrix"],props:{numArgs:0},handler(e){const t=Er(e.parser,{arraystretch:.5},"script");return t.colSeparationType="small",t},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["subarray"],props:{numArgs:1},handler(e,t){const r=(Ot(t[0])?[t[0]]:Rt(t[0],"ordgroup").body).map((function(e){const t=Ht(e).text;if(-1!=="lc".indexOf(t))return{type:"align",align:t};throw new n("Unknown column alignment: "+t,e)}));if(r.length>1)throw new n("{subarray} can contain only one column");let o={cols:r,hskipBeforeAndAfter:!1,arraystretch:.5};if(o=Er(e.parser,o,"script"),o.body.length>0&&o.body[0].length>1)throw new n("{subarray} can contain only one column");return o},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["cases","dcases","rcases","drcases"],props:{numArgs:0},handler(e){const t=Er(e.parser,{arraystretch:1.2,cols:[{type:"align",align:"l",pregap:0,postgap:1},{type:"align",align:"l",pregap:0,postgap:0}]},Lr(e.envName));return{type:"leftright",mode:e.mode,body:[t],left:e.envName.indexOf("r")>-1?".":"\\{",right:e.envName.indexOf("r")>-1?"\\}":".",rightColor:void 0}},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["align","align*","aligned","split"],props:{numArgs:0},handler:Fr,htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["gathered","gather","gather*"],props:{numArgs:0},handler(e){l.contains(["gather","gather*"],e.envName)&&Hr(e);const t={cols:[{type:"align",align:"c"}],addJot:!0,colSeparationType:"gather",autoTag:Or(e.envName),emptySingleRow:!0,leqno:e.parser.settings.leqno};return Er(e.parser,t,"display")},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["alignat","alignat*","alignedat"],props:{numArgs:1},handler:Fr,htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["equation","equation*"],props:{numArgs:0},handler(e){Hr(e);const t={autoTag:Or(e.envName),emptySingleRow:!0,singleRow:!0,maxNumCols:1,leqno:e.parser.settings.leqno};return Er(e.parser,t,"display")},htmlBuilder:Dr,mathmlBuilder:Pr}),Br({type:"array",names:["CD"],props:{numArgs:0},handler(e){return Hr(e),function(e){const t=[];for(e.gullet.beginGroup(),e.gullet.macros.set("\\cr","\\\\\\relax"),e.gullet.beginGroup();;){t.push(e.parseExpression(!1,"\\\\")),e.gullet.endGroup(),e.gullet.beginGroup();const r=e.fetch().text;if("&"!==r&&"\\\\"!==r){if("\\end"===r){0===t[t.length-1].length&&t.pop();break}throw new n("Expected \\\\ or \\cr or \\end",e.nextToken)}e.consume()}let r=[];const o=[r];for(let a=0;a<t.length;a++){const l=t[a];let h={type:"styling",body:[],mode:"math",style:"display"};for(let t=0;t<l.length;t++)if(Xt(l[t])){r.push(h),t+=1;const o=Ht(l[t]).text,a=new Array(2);if(a[0]={type:"ordgroup",mode:"math",body:[]},a[1]={type:"ordgroup",mode:"math",body:[]},"=|.".indexOf(o)>-1);else{if(!("<>AV".indexOf(o)>-1))throw new n('Expected one of "<>AV=|." after @',l[t]);for(let e=0;e<2;e++){let r=!0;for(let h=t+1;h<l.length;h++){if(i=o,("mathord"===(s=l[h]).type||"atom"===s.type)&&s.text===i){r=!1,t=h;break}if(Xt(l[h]))throw new n("Missing a "+o+" character to complete a CD arrow.",l[h]);a[e].body.push(l[h])}if(r)throw new n("Missing a "+o+" character to complete a CD arrow.",l[t])}}const c={type:"styling",body:[Wt(o,a,e)],mode:"math",style:"display"};r.push(c),h={type:"styling",body:[],mode:"math",style:"display"}}else h.body.push(l[t]);a%2==0?r.push(h):r.shift(),r=[],o.push(r)}var s,i;return e.gullet.endGroup(),e.gullet.endGroup(),{type:"array",mode:"math",body:o,arraystretch:1,addJot:!0,rowGaps:[null],cols:new Array(o[0].length).fill({type:"align",align:"c",pregap:.25,postgap:.25}),colSeparationType:"CD",hLinesBeforeRow:new Array(o.length+1).fill([])}}(e.parser)},htmlBuilder:Dr,mathmlBuilder:Pr}),Nr("\\nonumber","\\gdef\\@eqnsw{0}"),Nr("\\notag","\\nonumber"),$e({type:"text",names:["\\hline","\\hdashline"],props:{numArgs:0,allowedInText:!0,allowedInMath:!0},handler(e,t){throw new n(e.funcName+" valid only within array environment")}});var Gr=Tr;$e({type:"environment",names:["\\begin","\\end"],props:{numArgs:1,argTypes:["text"]},handler(e,t){let{parser:r,funcName:o}=e;const s=t[0];if("ordgroup"!==s.type)throw new n("Invalid environment name",s);let i="";for(let e=0;e<s.body.length;++e)i+=Rt(s.body[e],"textord").text;if("\\begin"===o){if(!Gr.hasOwnProperty(i))throw new n("No such environment: "+i,s);const e=Gr[i],{args:t,optArgs:o}=r.parseArguments("\\begin{"+i+"}",e),a={mode:r.mode,envName:i,parser:r},l=e.handler(a,t,o);r.expect("\\end",!1);const h=r.nextToken,c=Rt(r.parseFunction(),"environment");if(c.name!==i)throw new n("Mismatch: \\begin{"+i+"} matched by \\end{"+c.name+"}",h);return l}return{type:"environment",mode:r.mode,name:i,nameGroup:s}}});const Ur=(e,t)=>{const r=e.font,n=t.withFont(r);return ct(e.body,n)},Yr=(e,t)=>{const r=e.font,n=t.withFont(r);return St(e.body,n)},Xr={"\\Bbb":"\\mathbb","\\bold":"\\mathbf","\\frak":"\\mathfrak","\\bm":"\\boldsymbol"};$e({type:"font",names:["\\mathrm","\\mathit","\\mathbf","\\mathnormal","\\mathsfit","\\mathbb","\\mathcal","\\mathfrak","\\mathscr","\\mathsf","\\mathtt","\\Bbb","\\bold","\\frak"],props:{numArgs:1,allowedInArgument:!0},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=Ke(t[0]);let s=n;return s in Xr&&(s=Xr[s]),{type:"font",mode:r.mode,font:s.slice(1),body:o}},htmlBuilder:Ur,mathmlBuilder:Yr}),$e({type:"mclass",names:["\\boldsymbol","\\bm"],props:{numArgs:1},handler:(e,t)=>{let{parser:r}=e;const n=t[0],o=l.isCharacterBox(n);return{type:"mclass",mode:r.mode,mclass:Ut(n),body:[{type:"font",mode:r.mode,font:"boldsymbol",body:n}],isCharacterBox:o}}}),$e({type:"font",names:["\\rm","\\sf","\\tt","\\bf","\\it","\\cal"],props:{numArgs:0,allowedInText:!0},handler:(e,t)=>{let{parser:r,funcName:n,breakOnTokenText:o}=e;const{mode:s}=r,i=r.parseExpression(!0,o);return{type:"font",mode:s,font:"math"+n.slice(1),body:{type:"ordgroup",mode:r.mode,body:i}}},htmlBuilder:Ur,mathmlBuilder:Yr});const Wr=(e,t)=>{let r=t;return"display"===e?r=r.id>=w.SCRIPT.id?r.text():w.DISPLAY:"text"===e&&r.size===w.DISPLAY.size?r=w.TEXT:"script"===e?r=w.SCRIPT:"scriptscript"===e&&(r=w.SCRIPTSCRIPT),r},_r=(e,t)=>{const r=Wr(e.size,t.style),n=r.fracNum(),o=r.fracDen();let s;s=t.havingStyle(n);const i=ct(e.numer,s,t);if(e.continued){const e=8.5/t.fontMetrics().ptPerEm,r=3.5/t.fontMetrics().ptPerEm;i.height=i.height<e?e:i.height,i.depth=i.depth<r?r:i.depth}s=t.havingStyle(o);const a=ct(e.denom,s,t);let l,h,c,m,p,u,d,g,f,b;if(e.hasBarLine?(e.barSize?(h=P(e.barSize,t),l=Pe.makeLineSpan("frac-line",t,h)):l=Pe.makeLineSpan("frac-line",t),h=l.height,c=l.height):(l=null,h=0,c=t.fontMetrics().defaultRuleThickness),r.size===w.DISPLAY.size||"display"===e.size?(m=t.fontMetrics().num1,p=h>0?3*c:7*c,u=t.fontMetrics().denom1):(h>0?(m=t.fontMetrics().num2,p=c):(m=t.fontMetrics().num3,p=3*c),u=t.fontMetrics().denom2),l){const e=t.fontMetrics().axisHeight;m-i.depth-(e+.5*h)<p&&(m+=p-(m-i.depth-(e+.5*h))),e-.5*h-(a.height-u)<p&&(u+=p-(e-.5*h-(a.height-u)));const r=-(e-.5*h);d=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:a,shift:u},{type:"elem",elem:l,shift:r},{type:"elem",elem:i,shift:-m}]},t)}else{const e=m-i.depth-(a.height-u);e<p&&(m+=.5*(p-e),u+=.5*(p-e)),d=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:a,shift:u},{type:"elem",elem:i,shift:-m}]},t)}return s=t.havingStyle(r),d.height*=s.sizeMultiplier/t.sizeMultiplier,d.depth*=s.sizeMultiplier/t.sizeMultiplier,g=r.size===w.DISPLAY.size?t.fontMetrics().delim1:r.size===w.SCRIPTSCRIPT.size?t.havingStyle(w.SCRIPT).fontMetrics().delim2:t.fontMetrics().delim2,f=null==e.leftDelim?ht(t,["mopen"]):wr.customSizedDelim(e.leftDelim,g,!0,t.havingStyle(r),e.mode,["mopen"]),b=e.continued?Pe.makeSpan([]):null==e.rightDelim?ht(t,["mclose"]):wr.customSizedDelim(e.rightDelim,g,!0,t.havingStyle(r),e.mode,["mclose"]),Pe.makeSpan(["mord"].concat(s.sizingClasses(t)),[f,Pe.makeSpan(["mfrac"],[d]),b],t)},jr=(e,t)=>{let r=new ft.MathNode("mfrac",[St(e.numer,t),St(e.denom,t)]);if(e.hasBarLine){if(e.barSize){const n=P(e.barSize,t);r.setAttribute("linethickness",F(n))}}else r.setAttribute("linethickness","0px");const n=Wr(e.size,t.style);if(n.size!==t.style.size){r=new ft.MathNode("mstyle",[r]);const e=n.size===w.DISPLAY.size?"true":"false";r.setAttribute("displaystyle",e),r.setAttribute("scriptlevel","0")}if(null!=e.leftDelim||null!=e.rightDelim){const t=[];if(null!=e.leftDelim){const r=new ft.MathNode("mo",[new ft.TextNode(e.leftDelim.replace("\\",""))]);r.setAttribute("fence","true"),t.push(r)}if(t.push(r),null!=e.rightDelim){const r=new ft.MathNode("mo",[new ft.TextNode(e.rightDelim.replace("\\",""))]);r.setAttribute("fence","true"),t.push(r)}return yt(t)}return r};$e({type:"genfrac",names:["\\dfrac","\\frac","\\tfrac","\\dbinom","\\binom","\\tbinom","\\\\atopfrac","\\\\bracefrac","\\\\brackfrac"],props:{numArgs:2,allowedInArgument:!0},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=t[0],s=t[1];let i,a=null,l=null,h="auto";switch(n){case"\\dfrac":case"\\frac":case"\\tfrac":i=!0;break;case"\\\\atopfrac":i=!1;break;case"\\dbinom":case"\\binom":case"\\tbinom":i=!1,a="(",l=")";break;case"\\\\bracefrac":i=!1,a="\\{",l="\\}";break;case"\\\\brackfrac":i=!1,a="[",l="]";break;default:throw new Error("Unrecognized genfrac command")}switch(n){case"\\dfrac":case"\\dbinom":h="display";break;case"\\tfrac":case"\\tbinom":h="text"}return{type:"genfrac",mode:r.mode,continued:!1,numer:o,denom:s,hasBarLine:i,leftDelim:a,rightDelim:l,size:h,barSize:null}},htmlBuilder:_r,mathmlBuilder:jr}),$e({type:"genfrac",names:["\\cfrac"],props:{numArgs:2},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=t[0],s=t[1];return{type:"genfrac",mode:r.mode,continued:!0,numer:o,denom:s,hasBarLine:!0,leftDelim:null,rightDelim:null,size:"display",barSize:null}}}),$e({type:"infix",names:["\\over","\\choose","\\atop","\\brace","\\brack"],props:{numArgs:0,infix:!0},handler(e){let t,{parser:r,funcName:n,token:o}=e;switch(n){case"\\over":t="\\frac";break;case"\\choose":t="\\binom";break;case"\\atop":t="\\\\atopfrac";break;case"\\brace":t="\\\\bracefrac";break;case"\\brack":t="\\\\brackfrac";break;default:throw new Error("Unrecognized infix genfrac command")}return{type:"infix",mode:r.mode,replaceWith:t,token:o}}});const $r=["display","text","script","scriptscript"],Zr=function(e){let t=null;return e.length>0&&(t=e,t="."===t?null:t),t};$e({type:"genfrac",names:["\\genfrac"],props:{numArgs:6,allowedInArgument:!0,argTypes:["math","math","size","text","math","math"]},handler(e,t){let{parser:r}=e;const n=t[4],o=t[5],s=Ke(t[0]),i="atom"===s.type&&"open"===s.family?Zr(s.text):null,a=Ke(t[1]),l="atom"===a.type&&"close"===a.family?Zr(a.text):null,h=Rt(t[2],"size");let c,m=null;h.isBlank?c=!0:(m=h.value,c=m.number>0);let p="auto",u=t[3];if("ordgroup"===u.type){if(u.body.length>0){const e=Rt(u.body[0],"textord");p=$r[Number(e.text)]}}else u=Rt(u,"textord"),p=$r[Number(u.text)];return{type:"genfrac",mode:r.mode,numer:n,denom:o,continued:!1,hasBarLine:c,barSize:m,leftDelim:i,rightDelim:l,size:p}},htmlBuilder:_r,mathmlBuilder:jr}),$e({type:"infix",names:["\\above"],props:{numArgs:1,argTypes:["size"],infix:!0},handler(e,t){let{parser:r,funcName:n,token:o}=e;return{type:"infix",mode:r.mode,replaceWith:"\\\\abovefrac",size:Rt(t[0],"size").value,token:o}}}),$e({type:"genfrac",names:["\\\\abovefrac"],props:{numArgs:3,argTypes:["math","size","math"]},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=t[0],s=function(e){if(!e)throw new Error("Expected non-null, but got "+String(e));return e}(Rt(t[1],"infix").size),i=t[2],a=s.number>0;return{type:"genfrac",mode:r.mode,numer:o,denom:i,continued:!1,hasBarLine:a,barSize:s,leftDelim:null,rightDelim:null,size:"auto"}},htmlBuilder:_r,mathmlBuilder:jr});const Kr=(e,t)=>{const r=t.style;let n,o;"supsub"===e.type?(n=e.sup?ct(e.sup,t.havingStyle(r.sup()),t):ct(e.sub,t.havingStyle(r.sub()),t),o=Rt(e.base,"horizBrace")):o=Rt(e,"horizBrace");const s=ct(o.base,t.havingBaseStyle(w.DISPLAY)),i=It(o,t);let a;if(o.isOver?(a=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:s},{type:"kern",size:.1},{type:"elem",elem:i}]},t),a.children[0].children[0].children[1].classes.push("svg-align")):(a=Pe.makeVList({positionType:"bottom",positionData:s.depth+.1+i.height,children:[{type:"elem",elem:i},{type:"kern",size:.1},{type:"elem",elem:s}]},t),a.children[0].children[0].children[0].classes.push("svg-align")),n){const e=Pe.makeSpan(["mord",o.isOver?"mover":"munder"],[a],t);a=o.isOver?Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:e},{type:"kern",size:.2},{type:"elem",elem:n}]},t):Pe.makeVList({positionType:"bottom",positionData:e.depth+.2+n.height+n.depth,children:[{type:"elem",elem:n},{type:"kern",size:.2},{type:"elem",elem:e}]},t)}return Pe.makeSpan(["mord",o.isOver?"mover":"munder"],[a],t)};$e({type:"horizBrace",names:["\\overbrace","\\underbrace"],props:{numArgs:1},handler(e,t){let{parser:r,funcName:n}=e;return{type:"horizBrace",mode:r.mode,label:n,isOver:/^\\over/.test(n),base:t[0]}},htmlBuilder:Kr,mathmlBuilder:(e,t)=>{const r=qt(e.label);return new ft.MathNode(e.isOver?"mover":"munder",[St(e.base,t),r])}}),$e({type:"href",names:["\\href"],props:{numArgs:2,argTypes:["url","original"],allowedInText:!0},handler:(e,t)=>{let{parser:r}=e;const n=t[1],o=Rt(t[0],"url").url;return r.settings.isTrusted({command:"\\href",url:o})?{type:"href",mode:r.mode,href:o,body:Je(n)}:r.formatUnsupportedCmd("\\href")},htmlBuilder:(e,t)=>{const r=ot(e.body,t,!1);return Pe.makeAnchor(e.href,[],r,t)},mathmlBuilder:(e,t)=>{let r=kt(e.body,t);return r instanceof dt||(r=new dt("mrow",[r])),r.setAttribute("href",e.href),r}}),$e({type:"href",names:["\\url"],props:{numArgs:1,argTypes:["url"],allowedInText:!0},handler:(e,t)=>{let{parser:r}=e;const n=Rt(t[0],"url").url;if(!r.settings.isTrusted({command:"\\url",url:n}))return r.formatUnsupportedCmd("\\url");const o=[];for(let e=0;e<n.length;e++){let t=n[e];"~"===t&&(t="\\textasciitilde"),o.push({type:"textord",mode:"text",text:t})}const s={type:"text",mode:r.mode,font:"\\texttt",body:o};return{type:"href",mode:r.mode,href:n,body:Je(s)}}}),$e({type:"hbox",names:["\\hbox"],props:{numArgs:1,argTypes:["text"],allowedInText:!0,primitive:!0},handler(e,t){let{parser:r}=e;return{type:"hbox",mode:r.mode,body:Je(t[0])}},htmlBuilder(e,t){const r=ot(e.body,t,!1);return Pe.makeFragment(r)},mathmlBuilder(e,t){return new ft.MathNode("mrow",vt(e.body,t))}}),$e({type:"html",names:["\\htmlClass","\\htmlId","\\htmlStyle","\\htmlData"],props:{numArgs:2,argTypes:["raw","original"],allowedInText:!0},handler:(e,t)=>{let{parser:r,funcName:o,token:s}=e;const i=Rt(t[0],"raw").string,a=t[1];let l;r.settings.strict&&r.settings.reportNonstrict("htmlExtension","HTML extension is disabled on strict mode");const h={};switch(o){case"\\htmlClass":h.class=i,l={command:"\\htmlClass",class:i};break;case"\\htmlId":h.id=i,l={command:"\\htmlId",id:i};break;case"\\htmlStyle":h.style=i,l={command:"\\htmlStyle",style:i};break;case"\\htmlData":{const e=i.split(",");for(let t=0;t<e.length;t++){const r=e[t].split("=");if(2!==r.length)throw new n("Error parsing key-value for \\htmlData");h["data-"+r[0].trim()]=r[1].trim()}l={command:"\\htmlData",attributes:h};break}default:throw new Error("Unrecognized html command")}return r.settings.isTrusted(l)?{type:"html",mode:r.mode,attributes:h,body:Je(a)}:r.formatUnsupportedCmd(o)},htmlBuilder:(e,t)=>{const r=ot(e.body,t,!1),n=["enclosing"];e.attributes.class&&n.push(...e.attributes.class.trim().split(/\s+/));const o=Pe.makeSpan(n,r,t);for(const t in e.attributes)"class"!==t&&e.attributes.hasOwnProperty(t)&&o.setAttribute(t,e.attributes[t]);return o},mathmlBuilder:(e,t)=>kt(e.body,t)}),$e({type:"htmlmathml",names:["\\html@mathml"],props:{numArgs:2,allowedInText:!0},handler:(e,t)=>{let{parser:r}=e;return{type:"htmlmathml",mode:r.mode,html:Je(t[0]),mathml:Je(t[1])}},htmlBuilder:(e,t)=>{const r=ot(e.html,t,!1);return Pe.makeFragment(r)},mathmlBuilder:(e,t)=>kt(e.mathml,t)});const Jr=function(e){if(/^[-+]? *(\d+(\.\d*)?|\.\d+)$/.test(e))return{number:+e,unit:"bp"};{const t=/([-+]?) *(\d+(?:\.\d*)?|\.\d+) *([a-z]{2})/.exec(e);if(!t)throw new n("Invalid size: '"+e+"' in \\includegraphics");const r={number:+(t[1]+t[2]),unit:t[3]};if(!V(r))throw new n("Invalid unit: '"+r.unit+"' in \\includegraphics.");return r}};$e({type:"includegraphics",names:["\\includegraphics"],props:{numArgs:1,numOptionalArgs:1,argTypes:["raw","url"],allowedInText:!1},handler:(e,t,r)=>{let{parser:o}=e,s={number:0,unit:"em"},i={number:.9,unit:"em"},a={number:0,unit:"em"},l="";if(r[0]){const e=Rt(r[0],"raw").string.split(",");for(let t=0;t<e.length;t++){const r=e[t].split("=");if(2===r.length){const e=r[1].trim();switch(r[0].trim()){case"alt":l=e;break;case"width":s=Jr(e);break;case"height":i=Jr(e);break;case"totalheight":a=Jr(e);break;default:throw new n("Invalid key: '"+r[0]+"' in \\includegraphics.")}}}}const h=Rt(t[0],"url").url;return""===l&&(l=h,l=l.replace(/^.*[\\/]/,""),l=l.substring(0,l.lastIndexOf("."))),o.settings.isTrusted({command:"\\includegraphics",url:h})?{type:"includegraphics",mode:o.mode,alt:l,width:s,height:i,totalheight:a,src:h}:o.formatUnsupportedCmd("\\includegraphics")},htmlBuilder:(e,t)=>{const r=P(e.height,t);let n=0;e.totalheight.number>0&&(n=P(e.totalheight,t)-r);let o=0;e.width.number>0&&(o=P(e.width,t));const s={height:F(r+n)};o>0&&(s.width=F(o)),n>0&&(s.verticalAlign=F(-n));const i=new $(e.src,e.alt,s);return i.height=r,i.depth=n,i},mathmlBuilder:(e,t)=>{const r=new ft.MathNode("mglyph",[]);r.setAttribute("alt",e.alt);const n=P(e.height,t);let o=0;if(e.totalheight.number>0&&(o=P(e.totalheight,t)-n,r.setAttribute("valign",F(-o))),r.setAttribute("height",F(n+o)),e.width.number>0){const n=P(e.width,t);r.setAttribute("width",F(n))}return r.setAttribute("src",e.src),r}}),$e({type:"kern",names:["\\kern","\\mkern","\\hskip","\\mskip"],props:{numArgs:1,argTypes:["size"],primitive:!0,allowedInText:!0},handler(e,t){let{parser:r,funcName:n}=e;const o=Rt(t[0],"size");if(r.settings.strict){const e="m"===n[1],t="mu"===o.value.unit;e?(t||r.settings.reportNonstrict("mathVsTextUnits","LaTeX's "+n+" supports only mu units, not "+o.value.unit+" units"),"math"!==r.mode&&r.settings.reportNonstrict("mathVsTextUnits","LaTeX's "+n+" works only in math mode")):t&&r.settings.reportNonstrict("mathVsTextUnits","LaTeX's "+n+" doesn't support mu units")}return{type:"kern",mode:r.mode,dimension:o.value}},htmlBuilder(e,t){return Pe.makeGlue(e.dimension,t)},mathmlBuilder(e,t){const r=P(e.dimension,t);return new ft.SpaceNode(r)}}),$e({type:"lap",names:["\\mathllap","\\mathrlap","\\mathclap"],props:{numArgs:1,allowedInText:!0},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=t[0];return{type:"lap",mode:r.mode,alignment:n.slice(5),body:o}},htmlBuilder:(e,t)=>{let r;"clap"===e.alignment?(r=Pe.makeSpan([],[ct(e.body,t)]),r=Pe.makeSpan(["inner"],[r],t)):r=Pe.makeSpan(["inner"],[ct(e.body,t)]);const n=Pe.makeSpan(["fix"],[]);let o=Pe.makeSpan([e.alignment],[r,n],t);const s=Pe.makeSpan(["strut"]);return s.style.height=F(o.height+o.depth),o.depth&&(s.style.verticalAlign=F(-o.depth)),o.children.unshift(s),o=Pe.makeSpan(["thinbox"],[o],t),Pe.makeSpan(["mord","vbox"],[o],t)},mathmlBuilder:(e,t)=>{const r=new ft.MathNode("mpadded",[St(e.body,t)]);if("rlap"!==e.alignment){const t="llap"===e.alignment?"-1":"-0.5";r.setAttribute("lspace",t+"width")}return r.setAttribute("width","0px"),r}}),$e({type:"styling",names:["\\(","$"],props:{numArgs:0,allowedInText:!0,allowedInMath:!1},handler(e,t){let{funcName:r,parser:n}=e;const o=n.mode;n.switchMode("math");const s="\\("===r?"\\)":"$",i=n.parseExpression(!1,s);return n.expect(s),n.switchMode(o),{type:"styling",mode:n.mode,style:"text",body:i}}}),$e({type:"text",names:["\\)","\\]"],props:{numArgs:0,allowedInText:!0,allowedInMath:!1},handler(e,t){throw new n("Mismatched "+e.funcName)}});const Qr=(e,t)=>{switch(t.style.size){case w.DISPLAY.size:return e.display;case w.TEXT.size:return e.text;case w.SCRIPT.size:return e.script;case w.SCRIPTSCRIPT.size:return e.scriptscript;default:return e.text}};$e({type:"mathchoice",names:["\\mathchoice"],props:{numArgs:4,primitive:!0},handler:(e,t)=>{let{parser:r}=e;return{type:"mathchoice",mode:r.mode,display:Je(t[0]),text:Je(t[1]),script:Je(t[2]),scriptscript:Je(t[3])}},htmlBuilder:(e,t)=>{const r=Qr(e,t),n=ot(r,t,!1);return Pe.makeFragment(n)},mathmlBuilder:(e,t)=>{const r=Qr(e,t);return kt(r,t)}});const en=(e,t,r,n,o,s,i)=>{e=Pe.makeSpan([],[e]);const a=r&&l.isCharacterBox(r);let h,c,m;if(t){const e=ct(t,n.havingStyle(o.sup()),n);c={elem:e,kern:Math.max(n.fontMetrics().bigOpSpacing1,n.fontMetrics().bigOpSpacing3-e.depth)}}if(r){const e=ct(r,n.havingStyle(o.sub()),n);h={elem:e,kern:Math.max(n.fontMetrics().bigOpSpacing2,n.fontMetrics().bigOpSpacing4-e.height)}}if(c&&h){const t=n.fontMetrics().bigOpSpacing5+h.elem.height+h.elem.depth+h.kern+e.depth+i;m=Pe.makeVList({positionType:"bottom",positionData:t,children:[{type:"kern",size:n.fontMetrics().bigOpSpacing5},{type:"elem",elem:h.elem,marginLeft:F(-s)},{type:"kern",size:h.kern},{type:"elem",elem:e},{type:"kern",size:c.kern},{type:"elem",elem:c.elem,marginLeft:F(s)},{type:"kern",size:n.fontMetrics().bigOpSpacing5}]},n)}else if(h){const t=e.height-i;m=Pe.makeVList({positionType:"top",positionData:t,children:[{type:"kern",size:n.fontMetrics().bigOpSpacing5},{type:"elem",elem:h.elem,marginLeft:F(-s)},{type:"kern",size:h.kern},{type:"elem",elem:e}]},n)}else{if(!c)return e;{const t=e.depth+i;m=Pe.makeVList({positionType:"bottom",positionData:t,children:[{type:"elem",elem:e},{type:"kern",size:c.kern},{type:"elem",elem:c.elem,marginLeft:F(s)},{type:"kern",size:n.fontMetrics().bigOpSpacing5}]},n)}}const p=[m];if(h&&0!==s&&!a){const e=Pe.makeSpan(["mspace"],[],n);e.style.marginRight=F(s),p.unshift(e)}return Pe.makeSpan(["mop","op-limits"],p,n)},tn=["\\smallint"],rn=(e,t)=>{let r,n,o,s=!1;"supsub"===e.type?(r=e.sup,n=e.sub,o=Rt(e.base,"op"),s=!0):o=Rt(e,"op");const i=t.style;let a,h=!1;if(i.size===w.DISPLAY.size&&o.symbol&&!l.contains(tn,o.name)&&(h=!0),o.symbol){const e=h?"Size2-Regular":"Size1-Regular";let r="";if("\\oiint"!==o.name&&"\\oiiint"!==o.name||(r=o.name.slice(1),o.name="oiint"===r?"\\iint":"\\iiint"),a=Pe.makeSymbol(o.name,e,"math",t,["mop","op-symbol",h?"large-op":"small-op"]),r.length>0){const e=a.italic,n=Pe.staticSvg(r+"Size"+(h?"2":"1"),t);a=Pe.makeVList({positionType:"individualShift",children:[{type:"elem",elem:a,shift:0},{type:"elem",elem:n,shift:h?.08:0}]},t),o.name="\\"+r,a.classes.unshift("mop"),a.italic=e}}else if(o.body){const e=ot(o.body,t,!0);1===e.length&&e[0]instanceof K?(a=e[0],a.classes[0]="mop"):a=Pe.makeSpan(["mop"],e,t)}else{const e=[];for(let r=1;r<o.name.length;r++)e.push(Pe.mathsym(o.name[r],o.mode,t));a=Pe.makeSpan(["mop"],e,t)}let c=0,m=0;return(a instanceof K||"\\oiint"===o.name||"\\oiiint"===o.name)&&!o.suppressBaseShift&&(c=(a.height-a.depth)/2-t.fontMetrics().axisHeight,m=a.italic),s?en(a,r,n,t,i,m,c):(c&&(a.style.position="relative",a.style.top=F(c)),a)},nn=(e,t)=>{let r;if(e.symbol)r=new dt("mo",[bt(e.name,e.mode)]),l.contains(tn,e.name)&&r.setAttribute("largeop","false");else if(e.body)r=new dt("mo",vt(e.body,t));else{r=new dt("mi",[new gt(e.name.slice(1))]);const t=new dt("mo",[bt("\u2061","text")]);r=e.parentIsSupSub?new dt("mrow",[r,t]):ut([r,t])}return r},on={"\u220f":"\\prod","\u2210":"\\coprod","\u2211":"\\sum","\u22c0":"\\bigwedge","\u22c1":"\\bigvee","\u22c2":"\\bigcap","\u22c3":"\\bigcup","\u2a00":"\\bigodot","\u2a01":"\\bigoplus","\u2a02":"\\bigotimes","\u2a04":"\\biguplus","\u2a06":"\\bigsqcup"};$e({type:"op",names:["\\coprod","\\bigvee","\\bigwedge","\\biguplus","\\bigcap","\\bigcup","\\intop","\\prod","\\sum","\\bigotimes","\\bigoplus","\\bigodot","\\bigsqcup","\\smallint","\u220f","\u2210","\u2211","\u22c0","\u22c1","\u22c2","\u22c3","\u2a00","\u2a01","\u2a02","\u2a04","\u2a06"],props:{numArgs:0},handler:(e,t)=>{let{parser:r,funcName:n}=e,o=n;return 1===o.length&&(o=on[o]),{type:"op",mode:r.mode,limits:!0,parentIsSupSub:!1,symbol:!0,name:o}},htmlBuilder:rn,mathmlBuilder:nn}),$e({type:"op",names:["\\mathop"],props:{numArgs:1,primitive:!0},handler:(e,t)=>{let{parser:r}=e;const n=t[0];return{type:"op",mode:r.mode,limits:!1,parentIsSupSub:!1,symbol:!1,body:Je(n)}},htmlBuilder:rn,mathmlBuilder:nn});const sn={"\u222b":"\\int","\u222c":"\\iint","\u222d":"\\iiint","\u222e":"\\oint","\u222f":"\\oiint","\u2230":"\\oiiint"};$e({type:"op",names:["\\arcsin","\\arccos","\\arctan","\\arctg","\\arcctg","\\arg","\\ch","\\cos","\\cosec","\\cosh","\\cot","\\cotg","\\coth","\\csc","\\ctg","\\cth","\\deg","\\dim","\\exp","\\hom","\\ker","\\lg","\\ln","\\log","\\sec","\\sin","\\sinh","\\sh","\\tan","\\tanh","\\tg","\\th"],props:{numArgs:0},handler(e){let{parser:t,funcName:r}=e;return{type:"op",mode:t.mode,limits:!1,parentIsSupSub:!1,symbol:!1,name:r}},htmlBuilder:rn,mathmlBuilder:nn}),$e({type:"op",names:["\\det","\\gcd","\\inf","\\lim","\\max","\\min","\\Pr","\\sup"],props:{numArgs:0},handler(e){let{parser:t,funcName:r}=e;return{type:"op",mode:t.mode,limits:!0,parentIsSupSub:!1,symbol:!1,name:r}},htmlBuilder:rn,mathmlBuilder:nn}),$e({type:"op",names:["\\int","\\iint","\\iiint","\\oint","\\oiint","\\oiiint","\u222b","\u222c","\u222d","\u222e","\u222f","\u2230"],props:{numArgs:0},handler(e){let{parser:t,funcName:r}=e,n=r;return 1===n.length&&(n=sn[n]),{type:"op",mode:t.mode,limits:!1,parentIsSupSub:!1,symbol:!0,name:n}},htmlBuilder:rn,mathmlBuilder:nn});const an=(e,t)=>{let r,n,o,s,i=!1;if("supsub"===e.type?(r=e.sup,n=e.sub,o=Rt(e.base,"operatorname"),i=!0):o=Rt(e,"operatorname"),o.body.length>0){const e=o.body.map((e=>{const t=e.text;return"string"==typeof t?{type:"textord",mode:e.mode,text:t}:e})),r=ot(e,t.withFont("mathrm"),!0);for(let e=0;e<r.length;e++){const t=r[e];t instanceof K&&(t.text=t.text.replace(/\u2212/,"-").replace(/\u2217/,"*"))}s=Pe.makeSpan(["mop"],r,t)}else s=Pe.makeSpan(["mop"],[],t);return i?en(s,r,n,t,t.style,0,0):s};function ln(e,t,r){const n=ot(e,t,!1),o=t.sizeMultiplier/r.sizeMultiplier;for(let e=0;e<n.length;e++){const s=n[e].classes.indexOf("sizing");s<0?Array.prototype.push.apply(n[e].classes,t.sizingClasses(r)):n[e].classes[s+1]==="reset-size"+t.size&&(n[e].classes[s+1]="reset-size"+r.size),n[e].height*=o,n[e].depth*=o}return Pe.makeFragment(n)}$e({type:"operatorname",names:["\\operatorname@","\\operatornamewithlimits"],props:{numArgs:1},handler:(e,t)=>{let{parser:r,funcName:n}=e;const o=t[0];return{type:"operatorname",mode:r.mode,body:Je(o),alwaysHandleSupSub:"\\operatornamewithlimits"===n,limits:!1,parentIsSupSub:!1}},htmlBuilder:an,mathmlBuilder:(e,t)=>{let r=vt(e.body,t.withFont("mathrm")),n=!0;for(let e=0;e<r.length;e++){const t=r[e];if(t instanceof ft.SpaceNode);else if(t instanceof ft.MathNode)switch(t.type){case"mi":case"mn":case"ms":case"mspace":case"mtext":break;case"mo":{const e=t.children[0];1===t.children.length&&e instanceof ft.TextNode?e.text=e.text.replace(/\u2212/,"-").replace(/\u2217/,"*"):n=!1;break}default:n=!1}else n=!1}if(n){const e=r.map((e=>e.toText())).join("");r=[new ft.TextNode(e)]}const o=new ft.MathNode("mi",r);o.setAttribute("mathvariant","normal");const s=new ft.MathNode("mo",[bt("\u2061","text")]);return e.parentIsSupSub?new ft.MathNode("mrow",[o,s]):ft.newDocumentFragment([o,s])}}),Nr("\\operatorname","\\@ifstar\\operatornamewithlimits\\operatorname@"),Ze({type:"ordgroup",htmlBuilder(e,t){return e.semisimple?Pe.makeFragment(ot(e.body,t,!1)):Pe.makeSpan(["mord"],ot(e.body,t,!0),t)},mathmlBuilder(e,t){return kt(e.body,t,!0)}}),$e({type:"overline",names:["\\overline"],props:{numArgs:1},handler(e,t){let{parser:r}=e;const n=t[0];return{type:"overline",mode:r.mode,body:n}},htmlBuilder(e,t){const r=ct(e.body,t.havingCrampedStyle()),n=Pe.makeLineSpan("overline-line",t),o=t.fontMetrics().defaultRuleThickness,s=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:r},{type:"kern",size:3*o},{type:"elem",elem:n},{type:"kern",size:o}]},t);return Pe.makeSpan(["mord","overline"],[s],t)},mathmlBuilder(e,t){const r=new ft.MathNode("mo",[new ft.TextNode("\u203e")]);r.setAttribute("stretchy","true");const n=new ft.MathNode("mover",[St(e.body,t),r]);return n.setAttribute("accent","true"),n}}),$e({type:"phantom",names:["\\phantom"],props:{numArgs:1,allowedInText:!0},handler:(e,t)=>{let{parser:r}=e;const n=t[0];return{type:"phantom",mode:r.mode,body:Je(n)}},htmlBuilder:(e,t)=>{const r=ot(e.body,t.withPhantom(),!1);return Pe.makeFragment(r)},mathmlBuilder:(e,t)=>{const r=vt(e.body,t);return new ft.MathNode("mphantom",r)}}),$e({type:"hphantom",names:["\\hphantom"],props:{numArgs:1,allowedInText:!0},handler:(e,t)=>{let{parser:r}=e;const n=t[0];return{type:"hphantom",mode:r.mode,body:n}},htmlBuilder:(e,t)=>{let r=Pe.makeSpan([],[ct(e.body,t.withPhantom())]);if(r.height=0,r.depth=0,r.children)for(let e=0;e<r.children.length;e++)r.children[e].height=0,r.children[e].depth=0;return r=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:r}]},t),Pe.makeSpan(["mord"],[r],t)},mathmlBuilder:(e,t)=>{const r=vt(Je(e.body),t),n=new ft.MathNode("mphantom",r),o=new ft.MathNode("mpadded",[n]);return o.setAttribute("height","0px"),o.setAttribute("depth","0px"),o}}),$e({type:"vphantom",names:["\\vphantom"],props:{numArgs:1,allowedInText:!0},handler:(e,t)=>{let{parser:r}=e;const n=t[0];return{type:"vphantom",mode:r.mode,body:n}},htmlBuilder:(e,t)=>{const r=Pe.makeSpan(["inner"],[ct(e.body,t.withPhantom())]),n=Pe.makeSpan(["fix"],[]);return Pe.makeSpan(["mord","rlap"],[r,n],t)},mathmlBuilder:(e,t)=>{const r=vt(Je(e.body),t),n=new ft.MathNode("mphantom",r),o=new ft.MathNode("mpadded",[n]);return o.setAttribute("width","0px"),o}}),$e({type:"raisebox",names:["\\raisebox"],props:{numArgs:2,argTypes:["size","hbox"],allowedInText:!0},handler(e,t){let{parser:r}=e;const n=Rt(t[0],"size").value,o=t[1];return{type:"raisebox",mode:r.mode,dy:n,body:o}},htmlBuilder(e,t){const r=ct(e.body,t),n=P(e.dy,t);return Pe.makeVList({positionType:"shift",positionData:-n,children:[{type:"elem",elem:r}]},t)},mathmlBuilder(e,t){const r=new ft.MathNode("mpadded",[St(e.body,t)]),n=e.dy.number+e.dy.unit;return r.setAttribute("voffset",n),r}}),$e({type:"internal",names:["\\relax"],props:{numArgs:0,allowedInText:!0},handler(e){let{parser:t}=e;return{type:"internal",mode:t.mode}}}),$e({type:"rule",names:["\\rule"],props:{numArgs:2,numOptionalArgs:1,allowedInText:!0,allowedInMath:!0,argTypes:["size","size","size"]},handler(e,t,r){let{parser:n}=e;const o=r[0],s=Rt(t[0],"size"),i=Rt(t[1],"size");return{type:"rule",mode:n.mode,shift:o&&Rt(o,"size").value,width:s.value,height:i.value}},htmlBuilder(e,t){const r=Pe.makeSpan(["mord","rule"],[],t),n=P(e.width,t),o=P(e.height,t),s=e.shift?P(e.shift,t):0;return r.style.borderRightWidth=F(n),r.style.borderTopWidth=F(o),r.style.bottom=F(s),r.width=n,r.height=o+s,r.depth=-s,r.maxFontSize=1.125*o*t.sizeMultiplier,r},mathmlBuilder(e,t){const r=P(e.width,t),n=P(e.height,t),o=e.shift?P(e.shift,t):0,s=t.color&&t.getColor()||"black",i=new ft.MathNode("mspace");i.setAttribute("mathbackground",s),i.setAttribute("width",F(r)),i.setAttribute("height",F(n));const a=new ft.MathNode("mpadded",[i]);return o>=0?a.setAttribute("height",F(o)):(a.setAttribute("height",F(o)),a.setAttribute("depth",F(-o))),a.setAttribute("voffset",F(o)),a}});const hn=["\\tiny","\\sixptsize","\\scriptsize","\\footnotesize","\\small","\\normalsize","\\large","\\Large","\\LARGE","\\huge","\\Huge"];$e({type:"sizing",names:hn,props:{numArgs:0,allowedInText:!0},handler:(e,t)=>{let{breakOnTokenText:r,funcName:n,parser:o}=e;const s=o.parseExpression(!1,r);return{type:"sizing",mode:o.mode,size:hn.indexOf(n)+1,body:s}},htmlBuilder:(e,t)=>{const r=t.havingSize(e.size);return ln(e.body,r,t)},mathmlBuilder:(e,t)=>{const r=t.havingSize(e.size),n=vt(e.body,r),o=new ft.MathNode("mstyle",n);return o.setAttribute("mathsize",F(r.sizeMultiplier)),o}}),$e({type:"smash",names:["\\smash"],props:{numArgs:1,numOptionalArgs:1,allowedInText:!0},handler:(e,t,r)=>{let{parser:n}=e,o=!1,s=!1;const i=r[0]&&Rt(r[0],"ordgroup");if(i){let e="";for(let t=0;t<i.body.length;++t){if(e=i.body[t].text,"t"===e)o=!0;else{if("b"!==e){o=!1,s=!1;break}s=!0}}}else o=!0,s=!0;const a=t[0];return{type:"smash",mode:n.mode,body:a,smashHeight:o,smashDepth:s}},htmlBuilder:(e,t)=>{const r=Pe.makeSpan([],[ct(e.body,t)]);if(!e.smashHeight&&!e.smashDepth)return r;if(e.smashHeight&&(r.height=0,r.children))for(let e=0;e<r.children.length;e++)r.children[e].height=0;if(e.smashDepth&&(r.depth=0,r.children))for(let e=0;e<r.children.length;e++)r.children[e].depth=0;const n=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:r}]},t);return Pe.makeSpan(["mord"],[n],t)},mathmlBuilder:(e,t)=>{const r=new ft.MathNode("mpadded",[St(e.body,t)]);return e.smashHeight&&r.setAttribute("height","0px"),e.smashDepth&&r.setAttribute("depth","0px"),r}}),$e({type:"sqrt",names:["\\sqrt"],props:{numArgs:1,numOptionalArgs:1},handler(e,t,r){let{parser:n}=e;const o=r[0],s=t[0];return{type:"sqrt",mode:n.mode,body:s,index:o}},htmlBuilder(e,t){let r=ct(e.body,t.havingCrampedStyle());0===r.height&&(r.height=t.fontMetrics().xHeight),r=Pe.wrapFragment(r,t);const n=t.fontMetrics().defaultRuleThickness;let o=n;t.style.id<w.TEXT.id&&(o=t.fontMetrics().xHeight);let s=n+o/4;const i=r.height+r.depth+s+n,{span:a,ruleWidth:l,advanceWidth:h}=wr.sqrtImage(i,t),c=a.height-l;c>r.height+r.depth+s&&(s=(s+c-r.height-r.depth)/2);const m=a.height-r.height-s-l;r.style.paddingLeft=F(h);const p=Pe.makeVList({positionType:"firstBaseline",children:[{type:"elem",elem:r,wrapperClasses:["svg-align"]},{type:"kern",size:-(r.height+m)},{type:"elem",elem:a},{type:"kern",size:l}]},t);if(e.index){const r=t.havingStyle(w.SCRIPTSCRIPT),n=ct(e.index,r,t),o=.6*(p.height-p.depth),s=Pe.makeVList({positionType:"shift",positionData:-o,children:[{type:"elem",elem:n}]},t),i=Pe.makeSpan(["root"],[s]);return Pe.makeSpan(["mord","sqrt"],[i,p],t)}return Pe.makeSpan(["mord","sqrt"],[p],t)},mathmlBuilder(e,t){const{body:r,index:n}=e;return n?new ft.MathNode("mroot",[St(r,t),St(n,t)]):new ft.MathNode("msqrt",[St(r,t)])}});const cn={display:w.DISPLAY,text:w.TEXT,script:w.SCRIPT,scriptscript:w.SCRIPTSCRIPT};$e({type:"styling",names:["\\displaystyle","\\textstyle","\\scriptstyle","\\scriptscriptstyle"],props:{numArgs:0,allowedInText:!0,primitive:!0},handler(e,t){let{breakOnTokenText:r,funcName:n,parser:o}=e;const s=o.parseExpression(!0,r),i=n.slice(1,n.length-5);return{type:"styling",mode:o.mode,style:i,body:s}},htmlBuilder(e,t){const r=cn[e.style],n=t.havingStyle(r).withFont("");return ln(e.body,n,t)},mathmlBuilder(e,t){const r=cn[e.style],n=t.havingStyle(r),o=vt(e.body,n),s=new ft.MathNode("mstyle",o),i={display:["0","true"],text:["0","false"],script:["1","false"],scriptscript:["2","false"]}[e.style];return s.setAttribute("scriptlevel",i[0]),s.setAttribute("displaystyle",i[1]),s}});Ze({type:"supsub",htmlBuilder(e,t){const r=function(e,t){const r=e.base;if(r)return"op"===r.type?r.limits&&(t.style.size===w.DISPLAY.size||r.alwaysHandleSupSub)?rn:null:"operatorname"===r.type?r.alwaysHandleSupSub&&(t.style.size===w.DISPLAY.size||r.limits)?an:null:"accent"===r.type?l.isCharacterBox(r.base)?Et:null:"horizBrace"===r.type&&!e.sub===r.isOver?Kr:null;return null}(e,t);if(r)return r(e,t);const{base:n,sup:o,sub:s}=e,i=ct(n,t);let a,h;const c=t.fontMetrics();let m=0,p=0;const u=n&&l.isCharacterBox(n);if(o){const e=t.havingStyle(t.style.sup());a=ct(o,e,t),u||(m=i.height-e.fontMetrics().supDrop*e.sizeMultiplier/t.sizeMultiplier)}if(s){const e=t.havingStyle(t.style.sub());h=ct(s,e,t),u||(p=i.depth+e.fontMetrics().subDrop*e.sizeMultiplier/t.sizeMultiplier)}let d;d=t.style===w.DISPLAY?c.sup1:t.style.cramped?c.sup3:c.sup2;const g=t.sizeMultiplier,f=F(.5/c.ptPerEm/g);let b,y=null;if(h){const t=e.base&&"op"===e.base.type&&e.base.name&&("\\oiint"===e.base.name||"\\oiiint"===e.base.name);(i instanceof K||t)&&(y=F(-i.italic))}if(a&&h){m=Math.max(m,d,a.depth+.25*c.xHeight),p=Math.max(p,c.sub2);const e=4*c.defaultRuleThickness;if(m-a.depth-(h.height-p)<e){p=e-(m-a.depth)+h.height;const t=.8*c.xHeight-(m-a.depth);t>0&&(m+=t,p-=t)}const r=[{type:"elem",elem:h,shift:p,marginRight:f,marginLeft:y},{type:"elem",elem:a,shift:-m,marginRight:f}];b=Pe.makeVList({positionType:"individualShift",children:r},t)}else if(h){p=Math.max(p,c.sub1,h.height-.8*c.xHeight);const e=[{type:"elem",elem:h,marginLeft:y,marginRight:f}];b=Pe.makeVList({positionType:"shift",positionData:p,children:e},t)}else{if(!a)throw new Error("supsub must have either sup or sub.");m=Math.max(m,d,a.depth+.25*c.xHeight),b=Pe.makeVList({positionType:"shift",positionData:-m,children:[{type:"elem",elem:a,marginRight:f}]},t)}const x=lt(i,"right")||"mord";return Pe.makeSpan([x],[i,Pe.makeSpan(["msupsub"],[b])],t)},mathmlBuilder(e,t){let r,n,o=!1;e.base&&"horizBrace"===e.base.type&&(n=!!e.sup,n===e.base.isOver&&(o=!0,r=e.base.isOver)),!e.base||"op"!==e.base.type&&"operatorname"!==e.base.type||(e.base.parentIsSupSub=!0);const s=[St(e.base,t)];let i;if(e.sub&&s.push(St(e.sub,t)),e.sup&&s.push(St(e.sup,t)),o)i=r?"mover":"munder";else if(e.sub)if(e.sup){const r=e.base;i=r&&"op"===r.type&&r.limits&&t.style===w.DISPLAY||r&&"operatorname"===r.type&&r.alwaysHandleSupSub&&(t.style===w.DISPLAY||r.limits)?"munderover":"msubsup"}else{const r=e.base;i=r&&"op"===r.type&&r.limits&&(t.style===w.DISPLAY||r.alwaysHandleSupSub)||r&&"operatorname"===r.type&&r.alwaysHandleSupSub&&(r.limits||t.style===w.DISPLAY)?"munder":"msub"}else{const r=e.base;i=r&&"op"===r.type&&r.limits&&(t.style===w.DISPLAY||r.alwaysHandleSupSub)||r&&"operatorname"===r.type&&r.alwaysHandleSupSub&&(r.limits||t.style===w.DISPLAY)?"mover":"msup"}return new ft.MathNode(i,s)}}),Ze({type:"atom",htmlBuilder(e,t){return Pe.mathsym(e.text,e.mode,t,["m"+e.family])},mathmlBuilder(e,t){const r=new ft.MathNode("mo",[bt(e.text,e.mode)]);if("bin"===e.family){const n=xt(e,t);"bold-italic"===n&&r.setAttribute("mathvariant",n)}else"punct"===e.family?r.setAttribute("separator","true"):"open"!==e.family&&"close"!==e.family||r.setAttribute("stretchy","false");return r}});const mn={mi:"italic",mn:"normal",mtext:"normal"};Ze({type:"mathord",htmlBuilder(e,t){return Pe.makeOrd(e,t,"mathord")},mathmlBuilder(e,t){const r=new ft.MathNode("mi",[bt(e.text,e.mode,t)]),n=xt(e,t)||"italic";return n!==mn[r.type]&&r.setAttribute("mathvariant",n),r}}),Ze({type:"textord",htmlBuilder(e,t){return Pe.makeOrd(e,t,"textord")},mathmlBuilder(e,t){const r=bt(e.text,e.mode,t),n=xt(e,t)||"normal";let o;return o="text"===e.mode?new ft.MathNode("mtext",[r]):/[0-9]/.test(e.text)?new ft.MathNode("mn",[r]):"\\prime"===e.text?new ft.MathNode("mo",[r]):new ft.MathNode("mi",[r]),n!==mn[o.type]&&o.setAttribute("mathvariant",n),o}});const pn={"\\nobreak":"nobreak","\\allowbreak":"allowbreak"},un={" ":{},"\\ ":{},"~":{className:"nobreak"},"\\space":{},"\\nobreakspace":{className:"nobreak"}};Ze({type:"spacing",htmlBuilder(e,t){if(un.hasOwnProperty(e.text)){const r=un[e.text].className||"";if("text"===e.mode){const n=Pe.makeOrd(e,t,"textord");return n.classes.push(r),n}return Pe.makeSpan(["mspace",r],[Pe.mathsym(e.text,e.mode,t)],t)}if(pn.hasOwnProperty(e.text))return Pe.makeSpan(["mspace",pn[e.text]],[],t);throw new n('Unknown type of space "'+e.text+'"')},mathmlBuilder(e,t){let r;if(!un.hasOwnProperty(e.text)){if(pn.hasOwnProperty(e.text))return new ft.MathNode("mspace");throw new n('Unknown type of space "'+e.text+'"')}return r=new ft.MathNode("mtext",[new ft.TextNode("\xa0")]),r}});const dn=()=>{const e=new ft.MathNode("mtd",[]);return e.setAttribute("width","50%"),e};Ze({type:"tag",mathmlBuilder(e,t){const r=new ft.MathNode("mtable",[new ft.MathNode("mtr",[dn(),new ft.MathNode("mtd",[kt(e.body,t)]),dn(),new ft.MathNode("mtd",[kt(e.tag,t)])])]);return r.setAttribute("width","100%"),r}});const gn={"\\text":void 0,"\\textrm":"textrm","\\textsf":"textsf","\\texttt":"texttt","\\textnormal":"textrm"},fn={"\\textbf":"textbf","\\textmd":"textmd"},bn={"\\textit":"textit","\\textup":"textup"},yn=(e,t)=>{const r=e.font;return r?gn[r]?t.withTextFontFamily(gn[r]):fn[r]?t.withTextFontWeight(fn[r]):"\\emph"===r?"textit"===t.fontShape?t.withTextFontShape("textup"):t.withTextFontShape("textit"):t.withTextFontShape(bn[r]):t};$e({type:"text",names:["\\text","\\textrm","\\textsf","\\texttt","\\textnormal","\\textbf","\\textmd","\\textit","\\textup","\\emph"],props:{numArgs:1,argTypes:["text"],allowedInArgument:!0,allowedInText:!0},handler(e,t){let{parser:r,funcName:n}=e;const o=t[0];return{type:"text",mode:r.mode,body:Je(o),font:n}},htmlBuilder(e,t){const r=yn(e,t),n=ot(e.body,r,!0);return Pe.makeSpan(["mord","text"],n,r)},mathmlBuilder(e,t){const r=yn(e,t);return kt(e.body,r)}}),$e({type:"underline",names:["\\underline"],props:{numArgs:1,allowedInText:!0},handler(e,t){let{parser:r}=e;return{type:"underline",mode:r.mode,body:t[0]}},htmlBuilder(e,t){const r=ct(e.body,t),n=Pe.makeLineSpan("underline-line",t),o=t.fontMetrics().defaultRuleThickness,s=Pe.makeVList({positionType:"top",positionData:r.height,children:[{type:"kern",size:o},{type:"elem",elem:n},{type:"kern",size:3*o},{type:"elem",elem:r}]},t);return Pe.makeSpan(["mord","underline"],[s],t)},mathmlBuilder(e,t){const r=new ft.MathNode("mo",[new ft.TextNode("\u203e")]);r.setAttribute("stretchy","true");const n=new ft.MathNode("munder",[St(e.body,t),r]);return n.setAttribute("accentunder","true"),n}}),$e({type:"vcenter",names:["\\vcenter"],props:{numArgs:1,argTypes:["original"],allowedInText:!1},handler(e,t){let{parser:r}=e;return{type:"vcenter",mode:r.mode,body:t[0]}},htmlBuilder(e,t){const r=ct(e.body,t),n=t.fontMetrics().axisHeight,o=.5*(r.height-n-(r.depth+n));return Pe.makeVList({positionType:"shift",positionData:o,children:[{type:"elem",elem:r}]},t)},mathmlBuilder(e,t){return new ft.MathNode("mpadded",[St(e.body,t)],["vcenter"])}}),$e({type:"verb",names:["\\verb"],props:{numArgs:0,allowedInText:!0},handler(e,t,r){throw new n("\\verb ended by end of line instead of matching delimiter")},htmlBuilder(e,t){const r=xn(e),n=[],o=t.havingStyle(t.style.text());for(let t=0;t<r.length;t++){let s=r[t];"~"===s&&(s="\\textasciitilde"),n.push(Pe.makeSymbol(s,"Typewriter-Regular",e.mode,o,["mord","texttt"]))}return Pe.makeSpan(["mord","text"].concat(o.sizingClasses(t)),Pe.tryCombineChars(n),o)},mathmlBuilder(e,t){const r=new ft.TextNode(xn(e)),n=new ft.MathNode("mtext",[r]);return n.setAttribute("mathvariant","monospace"),n}});const xn=e=>e.body.replace(/ /g,e.star?"\u2423":"\xa0");var wn=We;const vn="[ \r\n\t]",kn="(\\\\[a-zA-Z@]+)"+vn+"*",Sn="[\u0300-\u036f]",Mn=new RegExp(Sn+"+$"),zn="("+vn+"+)|\\\\(\n|[ \r\t]+\n?)[ \r\t]*|([!-\\[\\]-\u2027\u202a-\ud7ff\uf900-\uffff]"+Sn+"*|[\ud800-\udbff][\udc00-\udfff]"+Sn+"*|\\\\verb\\*([^]).*?\\4|\\\\verb([^*a-zA-Z]).*?\\5|"+kn+"|\\\\[^\ud800-\udfff])";class An{constructor(e,t){this.input=void 0,this.settings=void 0,this.tokenRegex=void 0,this.catcodes=void 0,this.input=e,this.settings=t,this.tokenRegex=new RegExp(zn,"g"),this.catcodes={"%":14,"~":13}}setCatcode(e,t){this.catcodes[e]=t}lex(){const e=this.input,t=this.tokenRegex.lastIndex;if(t===e.length)return new Ir("EOF",new qr(this,t,t));const r=this.tokenRegex.exec(e);if(null===r||r.index!==t)throw new n("Unexpected character: '"+e[t]+"'",new Ir(e[t],new qr(this,t,t+1)));const o=r[6]||r[3]||(r[2]?"\\ ":" ");if(14===this.catcodes[o]){const t=e.indexOf("\n",this.tokenRegex.lastIndex);return-1===t?(this.tokenRegex.lastIndex=e.length,this.settings.reportNonstrict("commentAtEnd","% comment has no terminating newline; LaTeX would fail because of commenting the end of math mode (e.g. $)")):this.tokenRegex.lastIndex=t+1,this.lex()}return new Ir(o,new qr(this,t,this.tokenRegex.lastIndex))}}class Tn{constructor(e,t){void 0===e&&(e={}),void 0===t&&(t={}),this.current=void 0,this.builtins=void 0,this.undefStack=void 0,this.current=t,this.builtins=e,this.undefStack=[]}beginGroup(){this.undefStack.push({})}endGroup(){if(0===this.undefStack.length)throw new n("Unbalanced namespace destruction: attempt to pop global namespace; please report this as a bug");const e=this.undefStack.pop();for(const t in e)e.hasOwnProperty(t)&&(null==e[t]?delete this.current[t]:this.current[t]=e[t])}endGroups(){for(;this.undefStack.length>0;)this.endGroup()}has(e){return this.current.hasOwnProperty(e)||this.builtins.hasOwnProperty(e)}get(e){return this.current.hasOwnProperty(e)?this.current[e]:this.builtins[e]}set(e,t,r){if(void 0===r&&(r=!1),r){for(let t=0;t<this.undefStack.length;t++)delete this.undefStack[t][e];this.undefStack.length>0&&(this.undefStack[this.undefStack.length-1][e]=t)}else{const t=this.undefStack[this.undefStack.length-1];t&&!t.hasOwnProperty(e)&&(t[e]=this.current[e])}null==t?delete this.current[e]:this.current[e]=t}}var Bn=Cr;Nr("\\noexpand",(function(e){const t=e.popToken();return e.isExpandable(t.text)&&(t.noexpand=!0,t.treatAsRelax=!0),{tokens:[t],numArgs:0}})),Nr("\\expandafter",(function(e){const t=e.popToken();return e.expandOnce(!0),{tokens:[t],numArgs:0}})),Nr("\\@firstoftwo",(function(e){return{tokens:e.consumeArgs(2)[0],numArgs:0}})),Nr("\\@secondoftwo",(function(e){return{tokens:e.consumeArgs(2)[1],numArgs:0}})),Nr("\\@ifnextchar",(function(e){const t=e.consumeArgs(3);e.consumeSpaces();const r=e.future();return 1===t[0].length&&t[0][0].text===r.text?{tokens:t[1],numArgs:0}:{tokens:t[2],numArgs:0}})),Nr("\\@ifstar","\\@ifnextchar *{\\@firstoftwo{#1}}"),Nr("\\TextOrMath",(function(e){const t=e.consumeArgs(2);return"text"===e.mode?{tokens:t[0],numArgs:0}:{tokens:t[1],numArgs:0}}));const Cn={0:0,1:1,2:2,3:3,4:4,5:5,6:6,7:7,8:8,9:9,a:10,A:10,b:11,B:11,c:12,C:12,d:13,D:13,e:14,E:14,f:15,F:15};Nr("\\char",(function(e){let t,r=e.popToken(),o="";if("'"===r.text)t=8,r=e.popToken();else if('"'===r.text)t=16,r=e.popToken();else if("`"===r.text)if(r=e.popToken(),"\\"===r.text[0])o=r.text.charCodeAt(1);else{if("EOF"===r.text)throw new n("\\char` missing argument");o=r.text.charCodeAt(0)}else t=10;if(t){if(o=Cn[r.text],null==o||o>=t)throw new n("Invalid base-"+t+" digit "+r.text);let s;for(;null!=(s=Cn[e.future().text])&&s<t;)o*=t,o+=s,e.popToken()}return"\\@char{"+o+"}"}));const Nn=(e,t,r,o)=>{let s=e.consumeArg().tokens;if(1!==s.length)throw new n("\\newcommand's first argument must be a macro name");const i=s[0].text,a=e.isDefined(i);if(a&&!t)throw new n("\\newcommand{"+i+"} attempting to redefine "+i+"; use \\renewcommand");if(!a&&!r)throw new n("\\renewcommand{"+i+"} when command "+i+" does not yet exist; use \\newcommand");let l=0;if(s=e.consumeArg().tokens,1===s.length&&"["===s[0].text){let t="",r=e.expandNextToken();for(;"]"!==r.text&&"EOF"!==r.text;)t+=r.text,r=e.expandNextToken();if(!t.match(/^\s*[0-9]+\s*$/))throw new n("Invalid number of arguments: "+t);l=parseInt(t),s=e.consumeArg().tokens}return a&&o||e.macros.set(i,{tokens:s,numArgs:l}),""};Nr("\\newcommand",(e=>Nn(e,!1,!0,!1))),Nr("\\renewcommand",(e=>Nn(e,!0,!1,!1))),Nr("\\providecommand",(e=>Nn(e,!0,!0,!0))),Nr("\\message",(e=>{const t=e.consumeArgs(1)[0];return console.log(t.reverse().map((e=>e.text)).join("")),""})),Nr("\\errmessage",(e=>{const t=e.consumeArgs(1)[0];return console.error(t.reverse().map((e=>e.text)).join("")),""})),Nr("\\show",(e=>{const t=e.popToken(),r=t.text;return console.log(t,e.macros.get(r),wn[r],se.math[r],se.text[r]),""})),Nr("\\bgroup","{"),Nr("\\egroup","}"),Nr("~","\\nobreakspace"),Nr("\\lq","`"),Nr("\\rq","'"),Nr("\\aa","\\r a"),Nr("\\AA","\\r A"),Nr("\\textcopyright","\\html@mathml{\\textcircled{c}}{\\char`\xa9}"),Nr("\\copyright","\\TextOrMath{\\textcopyright}{\\text{\\textcopyright}}"),Nr("\\textregistered","\\html@mathml{\\textcircled{\\scriptsize R}}{\\char`\xae}"),Nr("\u212c","\\mathscr{B}"),Nr("\u2130","\\mathscr{E}"),Nr("\u2131","\\mathscr{F}"),Nr("\u210b","\\mathscr{H}"),Nr("\u2110","\\mathscr{I}"),Nr("\u2112","\\mathscr{L}"),Nr("\u2133","\\mathscr{M}"),Nr("\u211b","\\mathscr{R}"),Nr("\u212d","\\mathfrak{C}"),Nr("\u210c","\\mathfrak{H}"),Nr("\u2128","\\mathfrak{Z}"),Nr("\\Bbbk","\\Bbb{k}"),Nr("\xb7","\\cdotp"),Nr("\\llap","\\mathllap{\\textrm{#1}}"),Nr("\\rlap","\\mathrlap{\\textrm{#1}}"),Nr("\\clap","\\mathclap{\\textrm{#1}}"),Nr("\\mathstrut","\\vphantom{(}"),Nr("\\underbar","\\underline{\\text{#1}}"),Nr("\\not",'\\html@mathml{\\mathrel{\\mathrlap\\@not}}{\\char"338}'),Nr("\\neq","\\html@mathml{\\mathrel{\\not=}}{\\mathrel{\\char`\u2260}}"),Nr("\\ne","\\neq"),Nr("\u2260","\\neq"),Nr("\\notin","\\html@mathml{\\mathrel{{\\in}\\mathllap{/\\mskip1mu}}}{\\mathrel{\\char`\u2209}}"),Nr("\u2209","\\notin"),Nr("\u2258","\\html@mathml{\\mathrel{=\\kern{-1em}\\raisebox{0.4em}{$\\scriptsize\\frown$}}}{\\mathrel{\\char`\u2258}}"),Nr("\u2259","\\html@mathml{\\stackrel{\\tiny\\wedge}{=}}{\\mathrel{\\char`\u2258}}"),Nr("\u225a","\\html@mathml{\\stackrel{\\tiny\\vee}{=}}{\\mathrel{\\char`\u225a}}"),Nr("\u225b","\\html@mathml{\\stackrel{\\scriptsize\\star}{=}}{\\mathrel{\\char`\u225b}}"),Nr("\u225d","\\html@mathml{\\stackrel{\\tiny\\mathrm{def}}{=}}{\\mathrel{\\char`\u225d}}"),Nr("\u225e","\\html@mathml{\\stackrel{\\tiny\\mathrm{m}}{=}}{\\mathrel{\\char`\u225e}}"),Nr("\u225f","\\html@mathml{\\stackrel{\\tiny?}{=}}{\\mathrel{\\char`\u225f}}"),Nr("\u27c2","\\perp"),Nr("\u203c","\\mathclose{!\\mkern-0.8mu!}"),Nr("\u220c","\\notni"),Nr("\u231c","\\ulcorner"),Nr("\u231d","\\urcorner"),Nr("\u231e","\\llcorner"),Nr("\u231f","\\lrcorner"),Nr("\xa9","\\copyright"),Nr("\xae","\\textregistered"),Nr("\ufe0f","\\textregistered"),Nr("\\ulcorner",'\\html@mathml{\\@ulcorner}{\\mathop{\\char"231c}}'),Nr("\\urcorner",'\\html@mathml{\\@urcorner}{\\mathop{\\char"231d}}'),Nr("\\llcorner",'\\html@mathml{\\@llcorner}{\\mathop{\\char"231e}}'),Nr("\\lrcorner",'\\html@mathml{\\@lrcorner}{\\mathop{\\char"231f}}'),Nr("\\vdots","{\\varvdots\\rule{0pt}{15pt}}"),Nr("\u22ee","\\vdots"),Nr("\\varGamma","\\mathit{\\Gamma}"),Nr("\\varDelta","\\mathit{\\Delta}"),Nr("\\varTheta","\\mathit{\\Theta}"),Nr("\\varLambda","\\mathit{\\Lambda}"),Nr("\\varXi","\\mathit{\\Xi}"),Nr("\\varPi","\\mathit{\\Pi}"),Nr("\\varSigma","\\mathit{\\Sigma}"),Nr("\\varUpsilon","\\mathit{\\Upsilon}"),Nr("\\varPhi","\\mathit{\\Phi}"),Nr("\\varPsi","\\mathit{\\Psi}"),Nr("\\varOmega","\\mathit{\\Omega}"),Nr("\\substack","\\begin{subarray}{c}#1\\end{subarray}"),Nr("\\colon","\\nobreak\\mskip2mu\\mathpunct{}\\mathchoice{\\mkern-3mu}{\\mkern-3mu}{}{}{:}\\mskip6mu\\relax"),Nr("\\boxed","\\fbox{$\\displaystyle{#1}$}"),Nr("\\iff","\\DOTSB\\;\\Longleftrightarrow\\;"),Nr("\\implies","\\DOTSB\\;\\Longrightarrow\\;"),Nr("\\impliedby","\\DOTSB\\;\\Longleftarrow\\;"),Nr("\\dddot","{\\overset{\\raisebox{-0.1ex}{\\normalsize ...}}{#1}}"),Nr("\\ddddot","{\\overset{\\raisebox{-0.1ex}{\\normalsize ....}}{#1}}");const qn={",":"\\dotsc","\\not":"\\dotsb","+":"\\dotsb","=":"\\dotsb","<":"\\dotsb",">":"\\dotsb","-":"\\dotsb","*":"\\dotsb",":":"\\dotsb","\\DOTSB":"\\dotsb","\\coprod":"\\dotsb","\\bigvee":"\\dotsb","\\bigwedge":"\\dotsb","\\biguplus":"\\dotsb","\\bigcap":"\\dotsb","\\bigcup":"\\dotsb","\\prod":"\\dotsb","\\sum":"\\dotsb","\\bigotimes":"\\dotsb","\\bigoplus":"\\dotsb","\\bigodot":"\\dotsb","\\bigsqcup":"\\dotsb","\\And":"\\dotsb","\\longrightarrow":"\\dotsb","\\Longrightarrow":"\\dotsb","\\longleftarrow":"\\dotsb","\\Longleftarrow":"\\dotsb","\\longleftrightarrow":"\\dotsb","\\Longleftrightarrow":"\\dotsb","\\mapsto":"\\dotsb","\\longmapsto":"\\dotsb","\\hookrightarrow":"\\dotsb","\\doteq":"\\dotsb","\\mathbin":"\\dotsb","\\mathrel":"\\dotsb","\\relbar":"\\dotsb","\\Relbar":"\\dotsb","\\xrightarrow":"\\dotsb","\\xleftarrow":"\\dotsb","\\DOTSI":"\\dotsi","\\int":"\\dotsi","\\oint":"\\dotsi","\\iint":"\\dotsi","\\iiint":"\\dotsi","\\iiiint":"\\dotsi","\\idotsint":"\\dotsi","\\DOTSX":"\\dotsx"};Nr("\\dots",(function(e){let t="\\dotso";const r=e.expandAfterFuture().text;return r in qn?t=qn[r]:("\\not"===r.slice(0,4)||r in se.math&&l.contains(["bin","rel"],se.math[r].group))&&(t="\\dotsb"),t}));const In={")":!0,"]":!0,"\\rbrack":!0,"\\}":!0,"\\rbrace":!0,"\\rangle":!0,"\\rceil":!0,"\\rfloor":!0,"\\rgroup":!0,"\\rmoustache":!0,"\\right":!0,"\\bigr":!0,"\\biggr":!0,"\\Bigr":!0,"\\Biggr":!0,$:!0,";":!0,".":!0,",":!0};Nr("\\dotso",(function(e){return e.future().text in In?"\\ldots\\,":"\\ldots"})),Nr("\\dotsc",(function(e){const t=e.future().text;return t in In&&","!==t?"\\ldots\\,":"\\ldots"})),Nr("\\cdots",(function(e){return e.future().text in In?"\\@cdots\\,":"\\@cdots"})),Nr("\\dotsb","\\cdots"),Nr("\\dotsm","\\cdots"),Nr("\\dotsi","\\!\\cdots"),Nr("\\dotsx","\\ldots\\,"),Nr("\\DOTSI","\\relax"),Nr("\\DOTSB","\\relax"),Nr("\\DOTSX","\\relax"),Nr("\\tmspace","\\TextOrMath{\\kern#1#3}{\\mskip#1#2}\\relax"),Nr("\\,","\\tmspace+{3mu}{.1667em}"),Nr("\\thinspace","\\,"),Nr("\\>","\\mskip{4mu}"),Nr("\\:","\\tmspace+{4mu}{.2222em}"),Nr("\\medspace","\\:"),Nr("\\;","\\tmspace+{5mu}{.2777em}"),Nr("\\thickspace","\\;"),Nr("\\!","\\tmspace-{3mu}{.1667em}"),Nr("\\negthinspace","\\!"),Nr("\\negmedspace","\\tmspace-{4mu}{.2222em}"),Nr("\\negthickspace","\\tmspace-{5mu}{.277em}"),Nr("\\enspace","\\kern.5em "),Nr("\\enskip","\\hskip.5em\\relax"),Nr("\\quad","\\hskip1em\\relax"),Nr("\\qquad","\\hskip2em\\relax"),Nr("\\tag","\\@ifstar\\tag@literal\\tag@paren"),Nr("\\tag@paren","\\tag@literal{({#1})}"),Nr("\\tag@literal",(e=>{if(e.macros.get("\\df@tag"))throw new n("Multiple \\tag");return"\\gdef\\df@tag{\\text{#1}}"})),Nr("\\bmod","\\mathchoice{\\mskip1mu}{\\mskip1mu}{\\mskip5mu}{\\mskip5mu}\\mathbin{\\rm mod}\\mathchoice{\\mskip1mu}{\\mskip1mu}{\\mskip5mu}{\\mskip5mu}"),Nr("\\pod","\\allowbreak\\mathchoice{\\mkern18mu}{\\mkern8mu}{\\mkern8mu}{\\mkern8mu}(#1)"),Nr("\\pmod","\\pod{{\\rm mod}\\mkern6mu#1}"),Nr("\\mod","\\allowbreak\\mathchoice{\\mkern18mu}{\\mkern12mu}{\\mkern12mu}{\\mkern12mu}{\\rm mod}\\,\\,#1"),Nr("\\newline","\\\\\\relax"),Nr("\\TeX","\\textrm{\\html@mathml{T\\kern-.1667em\\raisebox{-.5ex}{E}\\kern-.125emX}{TeX}}");const Rn=F(T["Main-Regular"]["T".charCodeAt(0)][1]-.7*T["Main-Regular"]["A".charCodeAt(0)][1]);Nr("\\LaTeX","\\textrm{\\html@mathml{L\\kern-.36em\\raisebox{"+Rn+"}{\\scriptstyle A}\\kern-.15em\\TeX}{LaTeX}}"),Nr("\\KaTeX","\\textrm{\\html@mathml{K\\kern-.17em\\raisebox{"+Rn+"}{\\scriptstyle A}\\kern-.15em\\TeX}{KaTeX}}"),Nr("\\hspace","\\@ifstar\\@hspacer\\@hspace"),Nr("\\@hspace","\\hskip #1\\relax"),Nr("\\@hspacer","\\rule{0pt}{0pt}\\hskip #1\\relax"),Nr("\\ordinarycolon",":"),Nr("\\vcentcolon","\\mathrel{\\mathop\\ordinarycolon}"),Nr("\\dblcolon",'\\html@mathml{\\mathrel{\\vcentcolon\\mathrel{\\mkern-.9mu}\\vcentcolon}}{\\mathop{\\char"2237}}'),Nr("\\coloneqq",'\\html@mathml{\\mathrel{\\vcentcolon\\mathrel{\\mkern-1.2mu}=}}{\\mathop{\\char"2254}}'),Nr("\\Coloneqq",'\\html@mathml{\\mathrel{\\dblcolon\\mathrel{\\mkern-1.2mu}=}}{\\mathop{\\char"2237\\char"3d}}'),Nr("\\coloneq",'\\html@mathml{\\mathrel{\\vcentcolon\\mathrel{\\mkern-1.2mu}\\mathrel{-}}}{\\mathop{\\char"3a\\char"2212}}'),Nr("\\Coloneq",'\\html@mathml{\\mathrel{\\dblcolon\\mathrel{\\mkern-1.2mu}\\mathrel{-}}}{\\mathop{\\char"2237\\char"2212}}'),Nr("\\eqqcolon",'\\html@mathml{\\mathrel{=\\mathrel{\\mkern-1.2mu}\\vcentcolon}}{\\mathop{\\char"2255}}'),Nr("\\Eqqcolon",'\\html@mathml{\\mathrel{=\\mathrel{\\mkern-1.2mu}\\dblcolon}}{\\mathop{\\char"3d\\char"2237}}'),Nr("\\eqcolon",'\\html@mathml{\\mathrel{\\mathrel{-}\\mathrel{\\mkern-1.2mu}\\vcentcolon}}{\\mathop{\\char"2239}}'),Nr("\\Eqcolon",'\\html@mathml{\\mathrel{\\mathrel{-}\\mathrel{\\mkern-1.2mu}\\dblcolon}}{\\mathop{\\char"2212\\char"2237}}'),Nr("\\colonapprox",'\\html@mathml{\\mathrel{\\vcentcolon\\mathrel{\\mkern-1.2mu}\\approx}}{\\mathop{\\char"3a\\char"2248}}'),Nr("\\Colonapprox",'\\html@mathml{\\mathrel{\\dblcolon\\mathrel{\\mkern-1.2mu}\\approx}}{\\mathop{\\char"2237\\char"2248}}'),Nr("\\colonsim",'\\html@mathml{\\mathrel{\\vcentcolon\\mathrel{\\mkern-1.2mu}\\sim}}{\\mathop{\\char"3a\\char"223c}}'),Nr("\\Colonsim",'\\html@mathml{\\mathrel{\\dblcolon\\mathrel{\\mkern-1.2mu}\\sim}}{\\mathop{\\char"2237\\char"223c}}'),Nr("\u2237","\\dblcolon"),Nr("\u2239","\\eqcolon"),Nr("\u2254","\\coloneqq"),Nr("\u2255","\\eqqcolon"),Nr("\u2a74","\\Coloneqq"),Nr("\\ratio","\\vcentcolon"),Nr("\\coloncolon","\\dblcolon"),Nr("\\colonequals","\\coloneqq"),Nr("\\coloncolonequals","\\Coloneqq"),Nr("\\equalscolon","\\eqqcolon"),Nr("\\equalscoloncolon","\\Eqqcolon"),Nr("\\colonminus","\\coloneq"),Nr("\\coloncolonminus","\\Coloneq"),Nr("\\minuscolon","\\eqcolon"),Nr("\\minuscoloncolon","\\Eqcolon"),Nr("\\coloncolonapprox","\\Colonapprox"),Nr("\\coloncolonsim","\\Colonsim"),Nr("\\simcolon","\\mathrel{\\sim\\mathrel{\\mkern-1.2mu}\\vcentcolon}"),Nr("\\simcoloncolon","\\mathrel{\\sim\\mathrel{\\mkern-1.2mu}\\dblcolon}"),Nr("\\approxcolon","\\mathrel{\\approx\\mathrel{\\mkern-1.2mu}\\vcentcolon}"),Nr("\\approxcoloncolon","\\mathrel{\\approx\\mathrel{\\mkern-1.2mu}\\dblcolon}"),Nr("\\notni","\\html@mathml{\\not\\ni}{\\mathrel{\\char`\u220c}}"),Nr("\\limsup","\\DOTSB\\operatorname*{lim\\,sup}"),Nr("\\liminf","\\DOTSB\\operatorname*{lim\\,inf}"),Nr("\\injlim","\\DOTSB\\operatorname*{inj\\,lim}"),Nr("\\projlim","\\DOTSB\\operatorname*{proj\\,lim}"),Nr("\\varlimsup","\\DOTSB\\operatorname*{\\overline{lim}}"),Nr("\\varliminf","\\DOTSB\\operatorname*{\\underline{lim}}"),Nr("\\varinjlim","\\DOTSB\\operatorname*{\\underrightarrow{lim}}"),Nr("\\varprojlim","\\DOTSB\\operatorname*{\\underleftarrow{lim}}"),Nr("\\gvertneqq","\\html@mathml{\\@gvertneqq}{\u2269}"),Nr("\\lvertneqq","\\html@mathml{\\@lvertneqq}{\u2268}"),Nr("\\ngeqq","\\html@mathml{\\@ngeqq}{\u2271}"),Nr("\\ngeqslant","\\html@mathml{\\@ngeqslant}{\u2271}"),Nr("\\nleqq","\\html@mathml{\\@nleqq}{\u2270}"),Nr("\\nleqslant","\\html@mathml{\\@nleqslant}{\u2270}"),Nr("\\nshortmid","\\html@mathml{\\@nshortmid}{\u2224}"),Nr("\\nshortparallel","\\html@mathml{\\@nshortparallel}{\u2226}"),Nr("\\nsubseteqq","\\html@mathml{\\@nsubseteqq}{\u2288}"),Nr("\\nsupseteqq","\\html@mathml{\\@nsupseteqq}{\u2289}"),Nr("\\varsubsetneq","\\html@mathml{\\@varsubsetneq}{\u228a}"),Nr("\\varsubsetneqq","\\html@mathml{\\@varsubsetneqq}{\u2acb}"),Nr("\\varsupsetneq","\\html@mathml{\\@varsupsetneq}{\u228b}"),Nr("\\varsupsetneqq","\\html@mathml{\\@varsupsetneqq}{\u2acc}"),Nr("\\imath","\\html@mathml{\\@imath}{\u0131}"),Nr("\\jmath","\\html@mathml{\\@jmath}{\u0237}"),Nr("\\llbracket","\\html@mathml{\\mathopen{[\\mkern-3.2mu[}}{\\mathopen{\\char`\u27e6}}"),Nr("\\rrbracket","\\html@mathml{\\mathclose{]\\mkern-3.2mu]}}{\\mathclose{\\char`\u27e7}}"),Nr("\u27e6","\\llbracket"),Nr("\u27e7","\\rrbracket"),Nr("\\lBrace","\\html@mathml{\\mathopen{\\{\\mkern-3.2mu[}}{\\mathopen{\\char`\u2983}}"),Nr("\\rBrace","\\html@mathml{\\mathclose{]\\mkern-3.2mu\\}}}{\\mathclose{\\char`\u2984}}"),Nr("\u2983","\\lBrace"),Nr("\u2984","\\rBrace"),Nr("\\minuso","\\mathbin{\\html@mathml{{\\mathrlap{\\mathchoice{\\kern{0.145em}}{\\kern{0.145em}}{\\kern{0.1015em}}{\\kern{0.0725em}}\\circ}{-}}}{\\char`\u29b5}}"),Nr("\u29b5","\\minuso"),Nr("\\darr","\\downarrow"),Nr("\\dArr","\\Downarrow"),Nr("\\Darr","\\Downarrow"),Nr("\\lang","\\langle"),Nr("\\rang","\\rangle"),Nr("\\uarr","\\uparrow"),Nr("\\uArr","\\Uparrow"),Nr("\\Uarr","\\Uparrow"),Nr("\\N","\\mathbb{N}"),Nr("\\R","\\mathbb{R}"),Nr("\\Z","\\mathbb{Z}"),Nr("\\alef","\\aleph"),Nr("\\alefsym","\\aleph"),Nr("\\Alpha","\\mathrm{A}"),Nr("\\Beta","\\mathrm{B}"),Nr("\\bull","\\bullet"),Nr("\\Chi","\\mathrm{X}"),Nr("\\clubs","\\clubsuit"),Nr("\\cnums","\\mathbb{C}"),Nr("\\Complex","\\mathbb{C}"),Nr("\\Dagger","\\ddagger"),Nr("\\diamonds","\\diamondsuit"),Nr("\\empty","\\emptyset"),Nr("\\Epsilon","\\mathrm{E}"),Nr("\\Eta","\\mathrm{H}"),Nr("\\exist","\\exists"),Nr("\\harr","\\leftrightarrow"),Nr("\\hArr","\\Leftrightarrow"),Nr("\\Harr","\\Leftrightarrow"),Nr("\\hearts","\\heartsuit"),Nr("\\image","\\Im"),Nr("\\infin","\\infty"),Nr("\\Iota","\\mathrm{I}"),Nr("\\isin","\\in"),Nr("\\Kappa","\\mathrm{K}"),Nr("\\larr","\\leftarrow"),Nr("\\lArr","\\Leftarrow"),Nr("\\Larr","\\Leftarrow"),Nr("\\lrarr","\\leftrightarrow"),Nr("\\lrArr","\\Leftrightarrow"),Nr("\\Lrarr","\\Leftrightarrow"),Nr("\\Mu","\\mathrm{M}"),Nr("\\natnums","\\mathbb{N}"),Nr("\\Nu","\\mathrm{N}"),Nr("\\Omicron","\\mathrm{O}"),Nr("\\plusmn","\\pm"),Nr("\\rarr","\\rightarrow"),Nr("\\rArr","\\Rightarrow"),Nr("\\Rarr","\\Rightarrow"),Nr("\\real","\\Re"),Nr("\\reals","\\mathbb{R}"),Nr("\\Reals","\\mathbb{R}"),Nr("\\Rho","\\mathrm{P}"),Nr("\\sdot","\\cdot"),Nr("\\sect","\\S"),Nr("\\spades","\\spadesuit"),Nr("\\sub","\\subset"),Nr("\\sube","\\subseteq"),Nr("\\supe","\\supseteq"),Nr("\\Tau","\\mathrm{T}"),Nr("\\thetasym","\\vartheta"),Nr("\\weierp","\\wp"),Nr("\\Zeta","\\mathrm{Z}"),Nr("\\argmin","\\DOTSB\\operatorname*{arg\\,min}"),Nr("\\argmax","\\DOTSB\\operatorname*{arg\\,max}"),Nr("\\plim","\\DOTSB\\mathop{\\operatorname{plim}}\\limits"),Nr("\\bra","\\mathinner{\\langle{#1}|}"),Nr("\\ket","\\mathinner{|{#1}\\rangle}"),Nr("\\braket","\\mathinner{\\langle{#1}\\rangle}"),Nr("\\Bra","\\left\\langle#1\\right|"),Nr("\\Ket","\\left|#1\\right\\rangle");const Hn=e=>t=>{const r=t.consumeArg().tokens,n=t.consumeArg().tokens,o=t.consumeArg().tokens,s=t.consumeArg().tokens,i=t.macros.get("|"),a=t.macros.get("\\|");t.macros.beginGroup();const l=t=>r=>{e&&(r.macros.set("|",i),o.length&&r.macros.set("\\|",a));let s=t;if(!t&&o.length){"|"===r.future().text&&(r.popToken(),s=!0)}return{tokens:s?o:n,numArgs:0}};t.macros.set("|",l(!1)),o.length&&t.macros.set("\\|",l(!0));const h=t.consumeArg().tokens,c=t.expandTokens([...s,...h,...r]);return t.macros.endGroup(),{tokens:c.reverse(),numArgs:0}};Nr("\\bra@ket",Hn(!1)),Nr("\\bra@set",Hn(!0)),Nr("\\Braket","\\bra@ket{\\left\\langle}{\\,\\middle\\vert\\,}{\\,\\middle\\vert\\,}{\\right\\rangle}"),Nr("\\Set","\\bra@set{\\left\\{\\:}{\\;\\middle\\vert\\;}{\\;\\middle\\Vert\\;}{\\:\\right\\}}"),Nr("\\set","\\bra@set{\\{\\,}{\\mid}{}{\\,\\}}"),Nr("\\angln","{\\angl n}"),Nr("\\blue","\\textcolor{##6495ed}{#1}"),Nr("\\orange","\\textcolor{##ffa500}{#1}"),Nr("\\pink","\\textcolor{##ff00af}{#1}"),Nr("\\red","\\textcolor{##df0030}{#1}"),Nr("\\green","\\textcolor{##28ae7b}{#1}"),Nr("\\gray","\\textcolor{gray}{#1}"),Nr("\\purple","\\textcolor{##9d38bd}{#1}"),Nr("\\blueA","\\textcolor{##ccfaff}{#1}"),Nr("\\blueB","\\textcolor{##80f6ff}{#1}"),Nr("\\blueC","\\textcolor{##63d9ea}{#1}"),Nr("\\blueD","\\textcolor{##11accd}{#1}"),Nr("\\blueE","\\textcolor{##0c7f99}{#1}"),Nr("\\tealA","\\textcolor{##94fff5}{#1}"),Nr("\\tealB","\\textcolor{##26edd5}{#1}"),Nr("\\tealC","\\textcolor{##01d1c1}{#1}"),Nr("\\tealD","\\textcolor{##01a995}{#1}"),Nr("\\tealE","\\textcolor{##208170}{#1}"),Nr("\\greenA","\\textcolor{##b6ffb0}{#1}"),Nr("\\greenB","\\textcolor{##8af281}{#1}"),Nr("\\greenC","\\textcolor{##74cf70}{#1}"),Nr("\\greenD","\\textcolor{##1fab54}{#1}"),Nr("\\greenE","\\textcolor{##0d923f}{#1}"),Nr("\\goldA","\\textcolor{##ffd0a9}{#1}"),Nr("\\goldB","\\textcolor{##ffbb71}{#1}"),Nr("\\goldC","\\textcolor{##ff9c39}{#1}"),Nr("\\goldD","\\textcolor{##e07d10}{#1}"),Nr("\\goldE","\\textcolor{##a75a05}{#1}"),Nr("\\redA","\\textcolor{##fca9a9}{#1}"),Nr("\\redB","\\textcolor{##ff8482}{#1}"),Nr("\\redC","\\textcolor{##f9685d}{#1}"),Nr("\\redD","\\textcolor{##e84d39}{#1}"),Nr("\\redE","\\textcolor{##bc2612}{#1}"),Nr("\\maroonA","\\textcolor{##ffbde0}{#1}"),Nr("\\maroonB","\\textcolor{##ff92c6}{#1}"),Nr("\\maroonC","\\textcolor{##ed5fa6}{#1}"),Nr("\\maroonD","\\textcolor{##ca337c}{#1}"),Nr("\\maroonE","\\textcolor{##9e034e}{#1}"),Nr("\\purpleA","\\textcolor{##ddd7ff}{#1}"),Nr("\\purpleB","\\textcolor{##c6b9fc}{#1}"),Nr("\\purpleC","\\textcolor{##aa87ff}{#1}"),Nr("\\purpleD","\\textcolor{##7854ab}{#1}"),Nr("\\purpleE","\\textcolor{##543b78}{#1}"),Nr("\\mintA","\\textcolor{##f5f9e8}{#1}"),Nr("\\mintB","\\textcolor{##edf2df}{#1}"),Nr("\\mintC","\\textcolor{##e0e5cc}{#1}"),Nr("\\grayA","\\textcolor{##f6f7f7}{#1}"),Nr("\\grayB","\\textcolor{##f0f1f2}{#1}"),Nr("\\grayC","\\textcolor{##e3e5e6}{#1}"),Nr("\\grayD","\\textcolor{##d6d8da}{#1}"),Nr("\\grayE","\\textcolor{##babec2}{#1}"),Nr("\\grayF","\\textcolor{##888d93}{#1}"),Nr("\\grayG","\\textcolor{##626569}{#1}"),Nr("\\grayH","\\textcolor{##3b3e40}{#1}"),Nr("\\grayI","\\textcolor{##21242c}{#1}"),Nr("\\kaBlue","\\textcolor{##314453}{#1}"),Nr("\\kaGreen","\\textcolor{##71B307}{#1}");const On={"^":!0,_:!0,"\\limits":!0,"\\nolimits":!0};class En{constructor(e,t,r){this.settings=void 0,this.expansionCount=void 0,this.lexer=void 0,this.macros=void 0,this.stack=void 0,this.mode=void 0,this.settings=t,this.expansionCount=0,this.feed(e),this.macros=new Tn(Bn,t.macros),this.mode=r,this.stack=[]}feed(e){this.lexer=new An(e,this.settings)}switchMode(e){this.mode=e}beginGroup(){this.macros.beginGroup()}endGroup(){this.macros.endGroup()}endGroups(){this.macros.endGroups()}future(){return 0===this.stack.length&&this.pushToken(this.lexer.lex()),this.stack[this.stack.length-1]}popToken(){return this.future(),this.stack.pop()}pushToken(e){this.stack.push(e)}pushTokens(e){this.stack.push(...e)}scanArgument(e){let t,r,n;if(e){if(this.consumeSpaces(),"["!==this.future().text)return null;t=this.popToken(),({tokens:n,end:r}=this.consumeArg(["]"]))}else({tokens:n,start:t,end:r}=this.consumeArg());return this.pushToken(new Ir("EOF",r.loc)),this.pushTokens(n),t.range(r,"")}consumeSpaces(){for(;;){if(" "!==this.future().text)break;this.stack.pop()}}consumeArg(e){const t=[],r=e&&e.length>0;r||this.consumeSpaces();const o=this.future();let s,i=0,a=0;do{if(s=this.popToken(),t.push(s),"{"===s.text)++i;else if("}"===s.text){if(--i,-1===i)throw new n("Extra }",s)}else if("EOF"===s.text)throw new n("Unexpected end of input in a macro argument, expected '"+(e&&r?e[a]:"}")+"'",s);if(e&&r)if((0===i||1===i&&"{"===e[a])&&s.text===e[a]){if(++a,a===e.length){t.splice(-a,a);break}}else a=0}while(0!==i||r);return"{"===o.text&&"}"===t[t.length-1].text&&(t.pop(),t.shift()),t.reverse(),{tokens:t,start:o,end:s}}consumeArgs(e,t){if(t){if(t.length!==e+1)throw new n("The length of delimiters doesn't match the number of args!");const r=t[0];for(let e=0;e<r.length;e++){const t=this.popToken();if(r[e]!==t.text)throw new n("Use of the macro doesn't match its definition",t)}}const r=[];for(let n=0;n<e;n++)r.push(this.consumeArg(t&&t[n+1]).tokens);return r}countExpansion(e){if(this.expansionCount+=e,this.expansionCount>this.settings.maxExpand)throw new n("Too many expansions: infinite loop or need to increase maxExpand setting")}expandOnce(e){const t=this.popToken(),r=t.text,o=t.noexpand?null:this._getExpansion(r);if(null==o||e&&o.unexpandable){if(e&&null==o&&"\\"===r[0]&&!this.isDefined(r))throw new n("Undefined control sequence: "+r);return this.pushToken(t),!1}this.countExpansion(1);let s=o.tokens;const i=this.consumeArgs(o.numArgs,o.delimiters);if(o.numArgs){s=s.slice();for(let e=s.length-1;e>=0;--e){let t=s[e];if("#"===t.text){if(0===e)throw new n("Incomplete placeholder at end of macro body",t);if(t=s[--e],"#"===t.text)s.splice(e+1,1);else{if(!/^[1-9]$/.test(t.text))throw new n("Not a valid argument number",t);s.splice(e,2,...i[+t.text-1])}}}}return this.pushTokens(s),s.length}expandAfterFuture(){return this.expandOnce(),this.future()}expandNextToken(){for(;;)if(!1===this.expandOnce()){const e=this.stack.pop();return e.treatAsRelax&&(e.text="\\relax"),e}throw new Error}expandMacro(e){return this.macros.has(e)?this.expandTokens([new Ir(e)]):void 0}expandTokens(e){const t=[],r=this.stack.length;for(this.pushTokens(e);this.stack.length>r;)if(!1===this.expandOnce(!0)){const e=this.stack.pop();e.treatAsRelax&&(e.noexpand=!1,e.treatAsRelax=!1),t.push(e)}return this.countExpansion(t.length),t}expandMacroAsText(e){const t=this.expandMacro(e);return t?t.map((e=>e.text)).join(""):t}_getExpansion(e){const t=this.macros.get(e);if(null==t)return t;if(1===e.length){const t=this.lexer.catcodes[e];if(null!=t&&13!==t)return}const r="function"==typeof t?t(this):t;if("string"==typeof r){let e=0;if(-1!==r.indexOf("#")){const t=r.replace(/##/g,"");for(;-1!==t.indexOf("#"+(e+1));)++e}const t=new An(r,this.settings),n=[];let o=t.lex();for(;"EOF"!==o.text;)n.push(o),o=t.lex();n.reverse();return{tokens:n,numArgs:e}}return r}isDefined(e){return this.macros.has(e)||wn.hasOwnProperty(e)||se.math.hasOwnProperty(e)||se.text.hasOwnProperty(e)||On.hasOwnProperty(e)}isExpandable(e){const t=this.macros.get(e);return null!=t?"string"==typeof t||"function"==typeof t||!t.unexpandable:wn.hasOwnProperty(e)&&!wn[e].primitive}}const Ln=/^[\u208a\u208b\u208c\u208d\u208e\u2080\u2081\u2082\u2083\u2084\u2085\u2086\u2087\u2088\u2089\u2090\u2091\u2095\u1d62\u2c7c\u2096\u2097\u2098\u2099\u2092\u209a\u1d63\u209b\u209c\u1d64\u1d65\u2093\u1d66\u1d67\u1d68\u1d69\u1d6a]/,Dn=Object.freeze({"\u208a":"+","\u208b":"-","\u208c":"=","\u208d":"(","\u208e":")","\u2080":"0","\u2081":"1","\u2082":"2","\u2083":"3","\u2084":"4","\u2085":"5","\u2086":"6","\u2087":"7","\u2088":"8","\u2089":"9","\u2090":"a","\u2091":"e","\u2095":"h","\u1d62":"i","\u2c7c":"j","\u2096":"k","\u2097":"l","\u2098":"m","\u2099":"n","\u2092":"o","\u209a":"p","\u1d63":"r","\u209b":"s","\u209c":"t","\u1d64":"u","\u1d65":"v","\u2093":"x","\u1d66":"\u03b2","\u1d67":"\u03b3","\u1d68":"\u03c1","\u1d69":"\u03d5","\u1d6a":"\u03c7","\u207a":"+","\u207b":"-","\u207c":"=","\u207d":"(","\u207e":")","\u2070":"0","\xb9":"1","\xb2":"2","\xb3":"3","\u2074":"4","\u2075":"5","\u2076":"6","\u2077":"7","\u2078":"8","\u2079":"9","\u1d2c":"A","\u1d2e":"B","\u1d30":"D","\u1d31":"E","\u1d33":"G","\u1d34":"H","\u1d35":"I","\u1d36":"J","\u1d37":"K","\u1d38":"L","\u1d39":"M","\u1d3a":"N","\u1d3c":"O","\u1d3e":"P","\u1d3f":"R","\u1d40":"T","\u1d41":"U","\u2c7d":"V","\u1d42":"W","\u1d43":"a","\u1d47":"b","\u1d9c":"c","\u1d48":"d","\u1d49":"e","\u1da0":"f","\u1d4d":"g","\u02b0":"h","\u2071":"i","\u02b2":"j","\u1d4f":"k","\u02e1":"l","\u1d50":"m","\u207f":"n","\u1d52":"o","\u1d56":"p","\u02b3":"r","\u02e2":"s","\u1d57":"t","\u1d58":"u","\u1d5b":"v","\u02b7":"w","\u02e3":"x","\u02b8":"y","\u1dbb":"z","\u1d5d":"\u03b2","\u1d5e":"\u03b3","\u1d5f":"\u03b4","\u1d60":"\u03d5","\u1d61":"\u03c7","\u1dbf":"\u03b8"}),Vn={"\u0301":{text:"\\'",math:"\\acute"},"\u0300":{text:"\\`",math:"\\grave"},"\u0308":{text:'\\"',math:"\\ddot"},"\u0303":{text:"\\~",math:"\\tilde"},"\u0304":{text:"\\=",math:"\\bar"},"\u0306":{text:"\\u",math:"\\breve"},"\u030c":{text:"\\v",math:"\\check"},"\u0302":{text:"\\^",math:"\\hat"},"\u0307":{text:"\\.",math:"\\dot"},"\u030a":{text:"\\r",math:"\\mathring"},"\u030b":{text:"\\H"},"\u0327":{text:"\\c"}},Pn={"\xe1":"a\u0301","\xe0":"a\u0300","\xe4":"a\u0308","\u01df":"a\u0308\u0304","\xe3":"a\u0303","\u0101":"a\u0304","\u0103":"a\u0306","\u1eaf":"a\u0306\u0301","\u1eb1":"a\u0306\u0300","\u1eb5":"a\u0306\u0303","\u01ce":"a\u030c","\xe2":"a\u0302","\u1ea5":"a\u0302\u0301","\u1ea7":"a\u0302\u0300","\u1eab":"a\u0302\u0303","\u0227":"a\u0307","\u01e1":"a\u0307\u0304","\xe5":"a\u030a","\u01fb":"a\u030a\u0301","\u1e03":"b\u0307","\u0107":"c\u0301","\u1e09":"c\u0327\u0301","\u010d":"c\u030c","\u0109":"c\u0302","\u010b":"c\u0307","\xe7":"c\u0327","\u010f":"d\u030c","\u1e0b":"d\u0307","\u1e11":"d\u0327","\xe9":"e\u0301","\xe8":"e\u0300","\xeb":"e\u0308","\u1ebd":"e\u0303","\u0113":"e\u0304","\u1e17":"e\u0304\u0301","\u1e15":"e\u0304\u0300","\u0115":"e\u0306","\u1e1d":"e\u0327\u0306","\u011b":"e\u030c","\xea":"e\u0302","\u1ebf":"e\u0302\u0301","\u1ec1":"e\u0302\u0300","\u1ec5":"e\u0302\u0303","\u0117":"e\u0307","\u0229":"e\u0327","\u1e1f":"f\u0307","\u01f5":"g\u0301","\u1e21":"g\u0304","\u011f":"g\u0306","\u01e7":"g\u030c","\u011d":"g\u0302","\u0121":"g\u0307","\u0123":"g\u0327","\u1e27":"h\u0308","\u021f":"h\u030c","\u0125":"h\u0302","\u1e23":"h\u0307","\u1e29":"h\u0327","\xed":"i\u0301","\xec":"i\u0300","\xef":"i\u0308","\u1e2f":"i\u0308\u0301","\u0129":"i\u0303","\u012b":"i\u0304","\u012d":"i\u0306","\u01d0":"i\u030c","\xee":"i\u0302","\u01f0":"j\u030c","\u0135":"j\u0302","\u1e31":"k\u0301","\u01e9":"k\u030c","\u0137":"k\u0327","\u013a":"l\u0301","\u013e":"l\u030c","\u013c":"l\u0327","\u1e3f":"m\u0301","\u1e41":"m\u0307","\u0144":"n\u0301","\u01f9":"n\u0300","\xf1":"n\u0303","\u0148":"n\u030c","\u1e45":"n\u0307","\u0146":"n\u0327","\xf3":"o\u0301","\xf2":"o\u0300","\xf6":"o\u0308","\u022b":"o\u0308\u0304","\xf5":"o\u0303","\u1e4d":"o\u0303\u0301","\u1e4f":"o\u0303\u0308","\u022d":"o\u0303\u0304","\u014d":"o\u0304","\u1e53":"o\u0304\u0301","\u1e51":"o\u0304\u0300","\u014f":"o\u0306","\u01d2":"o\u030c","\xf4":"o\u0302","\u1ed1":"o\u0302\u0301","\u1ed3":"o\u0302\u0300","\u1ed7":"o\u0302\u0303","\u022f":"o\u0307","\u0231":"o\u0307\u0304","\u0151":"o\u030b","\u1e55":"p\u0301","\u1e57":"p\u0307","\u0155":"r\u0301","\u0159":"r\u030c","\u1e59":"r\u0307","\u0157":"r\u0327","\u015b":"s\u0301","\u1e65":"s\u0301\u0307","\u0161":"s\u030c","\u1e67":"s\u030c\u0307","\u015d":"s\u0302","\u1e61":"s\u0307","\u015f":"s\u0327","\u1e97":"t\u0308","\u0165":"t\u030c","\u1e6b":"t\u0307","\u0163":"t\u0327","\xfa":"u\u0301","\xf9":"u\u0300","\xfc":"u\u0308","\u01d8":"u\u0308\u0301","\u01dc":"u\u0308\u0300","\u01d6":"u\u0308\u0304","\u01da":"u\u0308\u030c","\u0169":"u\u0303","\u1e79":"u\u0303\u0301","\u016b":"u\u0304","\u1e7b":"u\u0304\u0308","\u016d":"u\u0306","\u01d4":"u\u030c","\xfb":"u\u0302","\u016f":"u\u030a","\u0171":"u\u030b","\u1e7d":"v\u0303","\u1e83":"w\u0301","\u1e81":"w\u0300","\u1e85":"w\u0308","\u0175":"w\u0302","\u1e87":"w\u0307","\u1e98":"w\u030a","\u1e8d":"x\u0308","\u1e8b":"x\u0307","\xfd":"y\u0301","\u1ef3":"y\u0300","\xff":"y\u0308","\u1ef9":"y\u0303","\u0233":"y\u0304","\u0177":"y\u0302","\u1e8f":"y\u0307","\u1e99":"y\u030a","\u017a":"z\u0301","\u017e":"z\u030c","\u1e91":"z\u0302","\u017c":"z\u0307","\xc1":"A\u0301","\xc0":"A\u0300","\xc4":"A\u0308","\u01de":"A\u0308\u0304","\xc3":"A\u0303","\u0100":"A\u0304","\u0102":"A\u0306","\u1eae":"A\u0306\u0301","\u1eb0":"A\u0306\u0300","\u1eb4":"A\u0306\u0303","\u01cd":"A\u030c","\xc2":"A\u0302","\u1ea4":"A\u0302\u0301","\u1ea6":"A\u0302\u0300","\u1eaa":"A\u0302\u0303","\u0226":"A\u0307","\u01e0":"A\u0307\u0304","\xc5":"A\u030a","\u01fa":"A\u030a\u0301","\u1e02":"B\u0307","\u0106":"C\u0301","\u1e08":"C\u0327\u0301","\u010c":"C\u030c","\u0108":"C\u0302","\u010a":"C\u0307","\xc7":"C\u0327","\u010e":"D\u030c","\u1e0a":"D\u0307","\u1e10":"D\u0327","\xc9":"E\u0301","\xc8":"E\u0300","\xcb":"E\u0308","\u1ebc":"E\u0303","\u0112":"E\u0304","\u1e16":"E\u0304\u0301","\u1e14":"E\u0304\u0300","\u0114":"E\u0306","\u1e1c":"E\u0327\u0306","\u011a":"E\u030c","\xca":"E\u0302","\u1ebe":"E\u0302\u0301","\u1ec0":"E\u0302\u0300","\u1ec4":"E\u0302\u0303","\u0116":"E\u0307","\u0228":"E\u0327","\u1e1e":"F\u0307","\u01f4":"G\u0301","\u1e20":"G\u0304","\u011e":"G\u0306","\u01e6":"G\u030c","\u011c":"G\u0302","\u0120":"G\u0307","\u0122":"G\u0327","\u1e26":"H\u0308","\u021e":"H\u030c","\u0124":"H\u0302","\u1e22":"H\u0307","\u1e28":"H\u0327","\xcd":"I\u0301","\xcc":"I\u0300","\xcf":"I\u0308","\u1e2e":"I\u0308\u0301","\u0128":"I\u0303","\u012a":"I\u0304","\u012c":"I\u0306","\u01cf":"I\u030c","\xce":"I\u0302","\u0130":"I\u0307","\u0134":"J\u0302","\u1e30":"K\u0301","\u01e8":"K\u030c","\u0136":"K\u0327","\u0139":"L\u0301","\u013d":"L\u030c","\u013b":"L\u0327","\u1e3e":"M\u0301","\u1e40":"M\u0307","\u0143":"N\u0301","\u01f8":"N\u0300","\xd1":"N\u0303","\u0147":"N\u030c","\u1e44":"N\u0307","\u0145":"N\u0327","\xd3":"O\u0301","\xd2":"O\u0300","\xd6":"O\u0308","\u022a":"O\u0308\u0304","\xd5":"O\u0303","\u1e4c":"O\u0303\u0301","\u1e4e":"O\u0303\u0308","\u022c":"O\u0303\u0304","\u014c":"O\u0304","\u1e52":"O\u0304\u0301","\u1e50":"O\u0304\u0300","\u014e":"O\u0306","\u01d1":"O\u030c","\xd4":"O\u0302","\u1ed0":"O\u0302\u0301","\u1ed2":"O\u0302\u0300","\u1ed6":"O\u0302\u0303","\u022e":"O\u0307","\u0230":"O\u0307\u0304","\u0150":"O\u030b","\u1e54":"P\u0301","\u1e56":"P\u0307","\u0154":"R\u0301","\u0158":"R\u030c","\u1e58":"R\u0307","\u0156":"R\u0327","\u015a":"S\u0301","\u1e64":"S\u0301\u0307","\u0160":"S\u030c","\u1e66":"S\u030c\u0307","\u015c":"S\u0302","\u1e60":"S\u0307","\u015e":"S\u0327","\u0164":"T\u030c","\u1e6a":"T\u0307","\u0162":"T\u0327","\xda":"U\u0301","\xd9":"U\u0300","\xdc":"U\u0308","\u01d7":"U\u0308\u0301","\u01db":"U\u0308\u0300","\u01d5":"U\u0308\u0304","\u01d9":"U\u0308\u030c","\u0168":"U\u0303","\u1e78":"U\u0303\u0301","\u016a":"U\u0304","\u1e7a":"U\u0304\u0308","\u016c":"U\u0306","\u01d3":"U\u030c","\xdb":"U\u0302","\u016e":"U\u030a","\u0170":"U\u030b","\u1e7c":"V\u0303","\u1e82":"W\u0301","\u1e80":"W\u0300","\u1e84":"W\u0308","\u0174":"W\u0302","\u1e86":"W\u0307","\u1e8c":"X\u0308","\u1e8a":"X\u0307","\xdd":"Y\u0301","\u1ef2":"Y\u0300","\u0178":"Y\u0308","\u1ef8":"Y\u0303","\u0232":"Y\u0304","\u0176":"Y\u0302","\u1e8e":"Y\u0307","\u0179":"Z\u0301","\u017d":"Z\u030c","\u1e90":"Z\u0302","\u017b":"Z\u0307","\u03ac":"\u03b1\u0301","\u1f70":"\u03b1\u0300","\u1fb1":"\u03b1\u0304","\u1fb0":"\u03b1\u0306","\u03ad":"\u03b5\u0301","\u1f72":"\u03b5\u0300","\u03ae":"\u03b7\u0301","\u1f74":"\u03b7\u0300","\u03af":"\u03b9\u0301","\u1f76":"\u03b9\u0300","\u03ca":"\u03b9\u0308","\u0390":"\u03b9\u0308\u0301","\u1fd2":"\u03b9\u0308\u0300","\u1fd1":"\u03b9\u0304","\u1fd0":"\u03b9\u0306","\u03cc":"\u03bf\u0301","\u1f78":"\u03bf\u0300","\u03cd":"\u03c5\u0301","\u1f7a":"\u03c5\u0300","\u03cb":"\u03c5\u0308","\u03b0":"\u03c5\u0308\u0301","\u1fe2":"\u03c5\u0308\u0300","\u1fe1":"\u03c5\u0304","\u1fe0":"\u03c5\u0306","\u03ce":"\u03c9\u0301","\u1f7c":"\u03c9\u0300","\u038e":"\u03a5\u0301","\u1fea":"\u03a5\u0300","\u03ab":"\u03a5\u0308","\u1fe9":"\u03a5\u0304","\u1fe8":"\u03a5\u0306","\u038f":"\u03a9\u0301","\u1ffa":"\u03a9\u0300"};class Fn{constructor(e,t){this.mode=void 0,this.gullet=void 0,this.settings=void 0,this.leftrightDepth=void 0,this.nextToken=void 0,this.mode="math",this.gullet=new En(e,t,this.mode),this.settings=t,this.leftrightDepth=0}expect(e,t){if(void 0===t&&(t=!0),this.fetch().text!==e)throw new n("Expected '"+e+"', got '"+this.fetch().text+"'",this.fetch());t&&this.consume()}consume(){this.nextToken=null}fetch(){return null==this.nextToken&&(this.nextToken=this.gullet.expandNextToken()),this.nextToken}switchMode(e){this.mode=e,this.gullet.switchMode(e)}parse(){this.settings.globalGroup||this.gullet.beginGroup(),this.settings.colorIsTextColor&&this.gullet.macros.set("\\color","\\textcolor");try{const e=this.parseExpression(!1);return this.expect("EOF"),this.settings.globalGroup||this.gullet.endGroup(),e}finally{this.gullet.endGroups()}}subparse(e){const t=this.nextToken;this.consume(),this.gullet.pushToken(new Ir("}")),this.gullet.pushTokens(e);const r=this.parseExpression(!1);return this.expect("}"),this.nextToken=t,r}parseExpression(e,t){const r=[];for(;;){"math"===this.mode&&this.consumeSpaces();const n=this.fetch();if(-1!==Fn.endOfExpression.indexOf(n.text))break;if(t&&n.text===t)break;if(e&&wn[n.text]&&wn[n.text].infix)break;const o=this.parseAtom(t);if(!o)break;"internal"!==o.type&&r.push(o)}return"text"===this.mode&&this.formLigatures(r),this.handleInfixNodes(r)}handleInfixNodes(e){let t,r=-1;for(let o=0;o<e.length;o++)if("infix"===e[o].type){if(-1!==r)throw new n("only one infix operator per group",e[o].token);r=o,t=e[o].replaceWith}if(-1!==r&&t){let n,o;const s=e.slice(0,r),i=e.slice(r+1);let a;return n=1===s.length&&"ordgroup"===s[0].type?s[0]:{type:"ordgroup",mode:this.mode,body:s},o=1===i.length&&"ordgroup"===i[0].type?i[0]:{type:"ordgroup",mode:this.mode,body:i},a="\\\\abovefrac"===t?this.callFunction(t,[n,e[r],o],[]):this.callFunction(t,[n,o],[]),[a]}return e}handleSupSubscript(e){const t=this.fetch(),r=t.text;this.consume(),this.consumeSpaces();const o=this.parseGroup(e);if(!o)throw new n("Expected group after '"+r+"'",t);return o}formatUnsupportedCmd(e){const t=[];for(let r=0;r<e.length;r++)t.push({type:"textord",mode:"text",text:e[r]});const r={type:"text",mode:this.mode,body:t};return{type:"color",mode:this.mode,color:this.settings.errorColor,body:[r]}}parseAtom(e){const t=this.parseGroup("atom",e);if("text"===this.mode)return t;let r,o;for(;;){this.consumeSpaces();const e=this.fetch();if("\\limits"===e.text||"\\nolimits"===e.text){if(t&&"op"===t.type){const r="\\limits"===e.text;t.limits=r,t.alwaysHandleSupSub=!0}else{if(!t||"operatorname"!==t.type)throw new n("Limit controls must follow a math operator",e);t.alwaysHandleSupSub&&(t.limits="\\limits"===e.text)}this.consume()}else if("^"===e.text){if(r)throw new n("Double superscript",e);r=this.handleSupSubscript("superscript")}else if("_"===e.text){if(o)throw new n("Double subscript",e);o=this.handleSupSubscript("subscript")}else if("'"===e.text){if(r)throw new n("Double superscript",e);const t={type:"textord",mode:this.mode,text:"\\prime"},o=[t];for(this.consume();"'"===this.fetch().text;)o.push(t),this.consume();"^"===this.fetch().text&&o.push(this.handleSupSubscript("superscript")),r={type:"ordgroup",mode:this.mode,body:o}}else{if(!Dn[e.text])break;{const t=Ln.test(e.text),n=[];for(n.push(new Ir(Dn[e.text])),this.consume();;){const e=this.fetch().text;if(!Dn[e])break;if(Ln.test(e)!==t)break;n.unshift(new Ir(Dn[e])),this.consume()}const s=this.subparse(n);t?o={type:"ordgroup",mode:"math",body:s}:r={type:"ordgroup",mode:"math",body:s}}}}return r||o?{type:"supsub",mode:this.mode,base:t,sup:r,sub:o}:t}parseFunction(e,t){const r=this.fetch(),o=r.text,s=wn[o];if(!s)return null;if(this.consume(),t&&"atom"!==t&&!s.allowedInArgument)throw new n("Got function '"+o+"' with no arguments"+(t?" as "+t:""),r);if("text"===this.mode&&!s.allowedInText)throw new n("Can't use function '"+o+"' in text mode",r);if("math"===this.mode&&!1===s.allowedInMath)throw new n("Can't use function '"+o+"' in math mode",r);const{args:i,optArgs:a}=this.parseArguments(o,s);return this.callFunction(o,i,a,r,e)}callFunction(e,t,r,o,s){const i={funcName:e,parser:this,token:o,breakOnTokenText:s},a=wn[e];if(a&&a.handler)return a.handler(i,t,r);throw new n("No function handler for "+e)}parseArguments(e,t){const r=t.numArgs+t.numOptionalArgs;if(0===r)return{args:[],optArgs:[]};const o=[],s=[];for(let i=0;i<r;i++){let r=t.argTypes&&t.argTypes[i];const a=i<t.numOptionalArgs;(t.primitive&&null==r||"sqrt"===t.type&&1===i&&null==s[0])&&(r="primitive");const l=this.parseGroupOfType("argument to '"+e+"'",r,a);if(a)s.push(l);else{if(null==l)throw new n("Null argument, please report this as a bug");o.push(l)}}return{args:o,optArgs:s}}parseGroupOfType(e,t,r){switch(t){case"color":return this.parseColorGroup(r);case"size":return this.parseSizeGroup(r);case"url":return this.parseUrlGroup(r);case"math":case"text":return this.parseArgumentGroup(r,t);case"hbox":{const e=this.parseArgumentGroup(r,"text");return null!=e?{type:"styling",mode:e.mode,body:[e],style:"text"}:null}case"raw":{const e=this.parseStringGroup("raw",r);return null!=e?{type:"raw",mode:"text",string:e.text}:null}case"primitive":{if(r)throw new n("A primitive argument cannot be optional");const t=this.parseGroup(e);if(null==t)throw new n("Expected group as "+e,this.fetch());return t}case"original":case null:case void 0:return this.parseArgumentGroup(r);default:throw new n("Unknown group type as "+e,this.fetch())}}consumeSpaces(){for(;" "===this.fetch().text;)this.consume()}parseStringGroup(e,t){const r=this.gullet.scanArgument(t);if(null==r)return null;let n,o="";for(;"EOF"!==(n=this.fetch()).text;)o+=n.text,this.consume();return this.consume(),r.text=o,r}parseRegexGroup(e,t){const r=this.fetch();let o,s=r,i="";for(;"EOF"!==(o=this.fetch()).text&&e.test(i+o.text);)s=o,i+=s.text,this.consume();if(""===i)throw new n("Invalid "+t+": '"+r.text+"'",r);return r.range(s,i)}parseColorGroup(e){const t=this.parseStringGroup("color",e);if(null==t)return null;const r=/^(#[a-f0-9]{3}|#?[a-f0-9]{6}|[a-z]+)$/i.exec(t.text);if(!r)throw new n("Invalid color: '"+t.text+"'",t);let o=r[0];return/^[0-9a-f]{6}$/i.test(o)&&(o="#"+o),{type:"color-token",mode:this.mode,color:o}}parseSizeGroup(e){let t,r=!1;if(this.gullet.consumeSpaces(),t=e||"{"===this.gullet.future().text?this.parseStringGroup("size",e):this.parseRegexGroup(/^[-+]? *(?:$|\d+|\d+\.\d*|\.\d*) *[a-z]{0,2} *$/,"size"),!t)return null;e||0!==t.text.length||(t.text="0pt",r=!0);const o=/([-+]?) *(\d+(?:\.\d*)?|\.\d+) *([a-z]{2})/.exec(t.text);if(!o)throw new n("Invalid size: '"+t.text+"'",t);const s={number:+(o[1]+o[2]),unit:o[3]};if(!V(s))throw new n("Invalid unit: '"+s.unit+"'",t);return{type:"size",mode:this.mode,value:s,isBlank:r}}parseUrlGroup(e){this.gullet.lexer.setCatcode("%",13),this.gullet.lexer.setCatcode("~",12);const t=this.parseStringGroup("url",e);if(this.gullet.lexer.setCatcode("%",14),this.gullet.lexer.setCatcode("~",13),null==t)return null;const r=t.text.replace(/\\([#$%&~_^{}])/g,"$1");return{type:"url",mode:this.mode,url:r}}parseArgumentGroup(e,t){const r=this.gullet.scanArgument(e);if(null==r)return null;const n=this.mode;t&&this.switchMode(t),this.gullet.beginGroup();const o=this.parseExpression(!1,"EOF");this.expect("EOF"),this.gullet.endGroup();const s={type:"ordgroup",mode:this.mode,loc:r.loc,body:o};return t&&this.switchMode(n),s}parseGroup(e,t){const r=this.fetch(),o=r.text;let s;if("{"===o||"\\begingroup"===o){this.consume();const e="{"===o?"}":"\\endgroup";this.gullet.beginGroup();const t=this.parseExpression(!1,e),n=this.fetch();this.expect(e),this.gullet.endGroup(),s={type:"ordgroup",mode:this.mode,loc:qr.range(r,n),body:t,semisimple:"\\begingroup"===o||void 0}}else if(s=this.parseFunction(t,e)||this.parseSymbol(),null==s&&"\\"===o[0]&&!On.hasOwnProperty(o)){if(this.settings.throwOnError)throw new n("Undefined control sequence: "+o,r);s=this.formatUnsupportedCmd(o),this.consume()}return s}formLigatures(e){let t=e.length-1;for(let r=0;r<t;++r){const n=e[r],o=n.text;"-"===o&&"-"===e[r+1].text&&(r+1<t&&"-"===e[r+2].text?(e.splice(r,3,{type:"textord",mode:"text",loc:qr.range(n,e[r+2]),text:"---"}),t-=2):(e.splice(r,2,{type:"textord",mode:"text",loc:qr.range(n,e[r+1]),text:"--"}),t-=1)),"'"!==o&&"`"!==o||e[r+1].text!==o||(e.splice(r,2,{type:"textord",mode:"text",loc:qr.range(n,e[r+1]),text:o+o}),t-=1)}}parseSymbol(){const e=this.fetch();let t=e.text;if(/^\\verb[^a-zA-Z]/.test(t)){this.consume();let e=t.slice(5);const r="*"===e.charAt(0);if(r&&(e=e.slice(1)),e.length<2||e.charAt(0)!==e.slice(-1))throw new n("\\verb assertion failed --\n                    please report what input caused this bug");return e=e.slice(1,-1),{type:"verb",mode:"text",body:e,star:r}}Pn.hasOwnProperty(t[0])&&!se[this.mode][t[0]]&&(this.settings.strict&&"math"===this.mode&&this.settings.reportNonstrict("unicodeTextInMathMode",'Accented Unicode text character "'+t[0]+'" used in math mode',e),t=Pn[t[0]]+t.slice(1));const r=Mn.exec(t);let o;if(r&&(t=t.substring(0,r.index),"i"===t?t="\u0131":"j"===t&&(t="\u0237")),se[this.mode][t]){this.settings.strict&&"math"===this.mode&&Te.indexOf(t)>=0&&this.settings.reportNonstrict("unicodeTextInMathMode",'Latin-1/Unicode text character "'+t[0]+'" used in math mode',e);const r=se[this.mode][t].group,n=qr.range(e);let s;if(re.hasOwnProperty(r)){const e=r;s={type:"atom",mode:this.mode,family:e,loc:n,text:t}}else s={type:r,mode:this.mode,loc:n,text:t};o=s}else{if(!(t.charCodeAt(0)>=128))return null;this.settings.strict&&(S(t.charCodeAt(0))?"math"===this.mode&&this.settings.reportNonstrict("unicodeTextInMathMode",'Unicode text character "'+t[0]+'" used in math mode',e):this.settings.reportNonstrict("unknownSymbol",'Unrecognized Unicode character "'+t[0]+'" ('+t.charCodeAt(0)+")",e)),o={type:"textord",mode:"text",loc:qr.range(e),text:t}}if(this.consume(),r)for(let t=0;t<r[0].length;t++){const s=r[0][t];if(!Vn[s])throw new n("Unknown accent ' "+s+"'",e);const i=Vn[s][this.mode]||Vn[s].text;if(!i)throw new n("Accent "+s+" unsupported in "+this.mode+" mode",e);o={type:"accent",mode:this.mode,loc:qr.range(e),label:i,isStretchy:!1,isShifty:!0,base:o}}return o}}Fn.endOfExpression=["}","\\endgroup","\\end","\\right","&"];var Gn=function(e,t){if(!("string"==typeof e||e instanceof String))throw new TypeError("KaTeX can only parse string typed expression");const r=new Fn(e,t);delete r.gullet.macros.current["\\df@tag"];let o=r.parse();if(delete r.gullet.macros.current["\\current@color"],delete r.gullet.macros.current["\\color"],r.gullet.macros.get("\\df@tag")){if(!t.displayMode)throw new n("\\tag works only in display equations");o=[{type:"tag",mode:"text",body:o,tag:r.subparse([new Ir("\\df@tag")])}]}return o};let Un=function(e,t,r){t.textContent="";const n=Xn(e,r).toNode();t.appendChild(n)};"undefined"!=typeof document&&"CSS1Compat"!==document.compatMode&&("undefined"!=typeof console&&console.warn("Warning: KaTeX doesn't work in quirks mode. Make sure your website has a suitable doctype."),Un=function(){throw new n("KaTeX doesn't work in quirks mode.")});const Yn=function(e,t,r){if(r.throwOnError||!(e instanceof n))throw e;const o=Pe.makeSpan(["katex-error"],[new K(t)]);return o.setAttribute("title",e.toString()),o.setAttribute("style","color:"+r.errorColor),o},Xn=function(e,t){const r=new m(t);try{const t=Gn(e,r);return Tt(t,e,r)}catch(t){return Yn(t,e,r)}};var Wn={version:"0.16.21",render:Un,renderToString:function(e,t){return Xn(e,t).toMarkup()},ParseError:n,SETTINGS_SCHEMA:h,__parse:function(e,t){const r=new m(t);return Gn(e,r)},__renderToDomTree:Xn,__renderToHTMLTree:function(e,t){const r=new m(t);try{return function(e,t,r){const n=pt(e,zt(r)),o=Pe.makeSpan(["katex"],[n]);return At(o,r)}(Gn(e,r),0,r)}catch(t){return Yn(t,e,r)}},__setFontMetrics:function(e,t){T[e]=t},__defineSymbol:ie,__defineFunction:$e,__defineMacro:Nr,__domTree:{Span:_,Anchor:j,SymbolNode:K,SvgNode:J,PathNode:Q,LineNode:ee}};return t=t.default}()}));
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/render.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/render.py
new file mode 100644
index 0000000000000000000000000000000000000000..325e817420db45ac8d05e3437a7ab74213e20b09
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/olmOCRBench/katex/render.py
@@ -0,0 +1,749 @@
+#!/usr/bin/env python3
+"""
+Extract inner-most spans and their bounding boxes, and the MathML output,
+from rendered LaTeX equations using Playwright and KaTeX.
+Caching is maintained via a SHA1-based hash stored in a sqlite database.
+
+Requirements:
+    pip install playwright
+    python -m playwright install chromium
+
+    Place katex.min.css and katex.min.js in the same directory as this script
+"""
+
+import hashlib
+import json
+import os
+import pathlib
+import re
+import sqlite3
+import threading
+import unittest
+from dataclasses import dataclass
+from typing import List, Optional
+
+from playwright.sync_api import Error as PlaywrightError
+from playwright.sync_api import sync_playwright
+
+# --- New SQLite Cache Implementation ---
+
+
+class EquationCache:
+    def __init__(self, db_path: Optional[str] = None):
+        if db_path is None:
+            # Use the same cache directory as before
+            cache_dir = pathlib.Path.home() / ".cache" / "olmocr" / "bench" / "equations"
+            cache_dir.mkdir(parents=True, exist_ok=True)
+            db_path = str(cache_dir / "cache.db")
+        self.db_path = db_path
+        self.lock = threading.Lock()
+        self._init_db()
+
+    def _init_db(self):
+        with self.lock:
+            conn = sqlite3.connect(self.db_path)
+            c = conn.cursor()
+            # Added an 'error' column to store rendering errors
+            c.execute(
+                """
+                CREATE TABLE IF NOT EXISTS equations (
+                    eq_hash TEXT PRIMARY KEY,
+                    mathml TEXT,
+                    spans TEXT,
+                    error TEXT
+                )
+            """
+            )
+            conn.commit()
+            conn.close()
+
+    def load(self, eq_hash: str) -> Optional["RenderedEquation"]:
+        with self.lock:
+            conn = sqlite3.connect(self.db_path)
+            c = conn.cursor()
+            c.execute("SELECT mathml, spans, error FROM equations WHERE eq_hash = ?", (eq_hash,))
+            row = c.fetchone()
+            conn.close()
+        if row:
+            mathml, spans_json, error = row
+            if error:
+                # In error cases, we return an instance with error set and no spans.
+                return RenderedEquation(mathml=mathml, spans=[], error=error)
+            else:
+                spans_data = json.loads(spans_json)
+                spans = [
+                    SpanInfo(
+                        text=s["text"],
+                        bounding_box=BoundingBox(
+                            x=s["boundingBox"]["x"],
+                            y=s["boundingBox"]["y"],
+                            width=s["boundingBox"]["width"],
+                            height=s["boundingBox"]["height"],
+                        ),
+                    )
+                    for s in spans_data
+                ]
+                return RenderedEquation(mathml=mathml, spans=spans)
+        return None
+
+    def save(self, eq_hash: str, rendered_eq: "RenderedEquation"):
+        spans_data = [
+            {
+                "text": span.text,
+                "boundingBox": {
+                    "x": span.bounding_box.x,
+                    "y": span.bounding_box.y,
+                    "width": span.bounding_box.width,
+                    "height": span.bounding_box.height,
+                },
+            }
+            for span in rendered_eq.spans
+        ]
+        spans_json = json.dumps(spans_data)
+        with self.lock:
+            conn = sqlite3.connect(self.db_path)
+            c = conn.cursor()
+            c.execute(
+                "INSERT OR REPLACE INTO equations (eq_hash, mathml, spans, error) VALUES (?, ?, ?, ?)",
+                (eq_hash, rendered_eq.mathml, spans_json, rendered_eq.error),
+            )
+            conn.commit()
+            conn.close()
+
+    def clear(self):
+        with self.lock:
+            conn = sqlite3.connect(self.db_path)
+            c = conn.cursor()
+            c.execute("DELETE FROM equations")
+            conn.commit()
+            conn.close()
+
+
+# Global instance of EquationCache
+equation_cache = EquationCache()
+
+# --- End SQLite Cache Implementation ---
+
+
+# Thread-local storage for Playwright and browser instances
+_thread_local = threading.local()
+
+
+@dataclass
+class BoundingBox:
+    x: float
+    y: float
+    width: float
+    height: float
+
+
+@dataclass
+class SpanInfo:
+    text: str
+    bounding_box: BoundingBox
+
+
+@dataclass
+class RenderedEquation:
+    mathml: str
+    spans: List[SpanInfo]
+    error: Optional[str] = None  # New field to store error messages if rendering fails
+
+
+def get_equation_hash(equation, bg_color="white", text_color="black", font_size=24):
+    """
+    Calculate SHA1 hash of the equation string and rendering parameters.
+    """
+    params_str = f"{equation}|{bg_color}|{text_color}|{font_size}"
+    return hashlib.sha1(params_str.encode("utf-8")).hexdigest()
+
+
+def init_browser():
+    """
+    Initialize the Playwright and browser instance for the current thread if not already done.
+    """
+    if not hasattr(_thread_local, "playwright"):
+        _thread_local.playwright = sync_playwright().start()
+        _thread_local.browser = _thread_local.playwright.chromium.launch()
+
+
+def get_browser():
+    """
+    Return the browser instance for the current thread.
+    """
+    init_browser()
+    return _thread_local.browser
+
+
+def render_equation(
+    equation,
+    bg_color="white",
+    text_color="black",
+    font_size=24,
+    use_cache=True,
+    debug_dom=False,
+):
+    """
+    Render a LaTeX equation using Playwright and KaTeX, extract the inner-most span elements
+    along with their bounding boxes, and extract the MathML output generated by KaTeX.
+    """
+    # Calculate hash for caching.
+    eq_hash = get_equation_hash(equation, bg_color, text_color, font_size)
+
+    # Try to load from SQLite cache.
+    if use_cache:
+        cached = equation_cache.load(eq_hash)
+        if cached is not None:
+            return cached
+
+    # Escape the equation for use in a JavaScript string.
+    escaped_equation = json.dumps(equation)
+
+    # Get local paths for KaTeX files.
+    script_dir = os.path.dirname(os.path.abspath(__file__))
+    katex_css_path = os.path.join(script_dir, "katex.min.css")
+    katex_js_path = os.path.join(script_dir, "katex.min.js")
+
+    if not os.path.exists(katex_css_path) or not os.path.exists(katex_js_path):
+        raise FileNotFoundError(
+            f"KaTeX files not found. Please ensure katex.min.css and katex.min.js "
+            f"are in {script_dir}"
+        )
+
+    # Get the browser instance for the current thread.
+    browser = get_browser()
+
+    # Create a new page.
+    page = browser.new_page(viewport={"width": 800, "height": 400})
+
+    # Basic HTML structure for rendering.
+    page_html = f"""
+    <!DOCTYPE html>
+    <html>
+    <head>
+        <style>
+            body {{
+                display: flex;
+                justify-content: center;
+                align-items: center;
+                height: 100vh;
+                margin: 0;
+                background-color: {bg_color};
+                color: {text_color};
+            }}
+            #equation-container {{
+                padding: 0;
+                font-size: {font_size}px;
+            }}
+        </style>
+    </head>
+    <body>
+        <div id="equation-container"></div>
+    </body>
+    </html>
+    """
+    page.set_content(page_html)
+    page.add_style_tag(path=katex_css_path)
+    page.add_script_tag(path=katex_js_path)
+    page.wait_for_load_state("networkidle")
+
+    katex_loaded = page.evaluate("typeof katex !== 'undefined'")
+    if not katex_loaded:
+        page.close()
+        raise RuntimeError("KaTeX library failed to load. Check your katex.min.js file.")
+
+    try:
+        error_message = page.evaluate(
+            f"""
+        () => {{
+            try {{
+                katex.render({escaped_equation}, document.getElementById("equation-container"), {{
+                    displayMode: true,
+                    throwOnError: true
+                }});
+                return null;
+            }} catch (error) {{
+                console.error("KaTeX error:", error.message);
+                return error.message;
+            }}
+        }}
+        """
+        )
+    except PlaywrightError as ex:
+        print(escaped_equation)
+        error_message = str(ex)
+        page.close()
+        raise
+
+    if error_message:
+        print(f"Error rendering equation: '{equation}'")
+        print(error_message)
+        # Cache the error result so we don't retry it next time.
+        rendered_eq = RenderedEquation(mathml=error_message, spans=[], error=error_message)
+        if use_cache:
+            equation_cache.save(eq_hash, rendered_eq)
+        page.close()
+        return rendered_eq
+
+    page.wait_for_selector(".katex", state="attached")
+
+    if debug_dom:
+        katex_dom_html = page.evaluate(
+            """
+        () => {
+            return document.getElementById("equation-container").innerHTML;
+        }
+        """
+        )
+        print("\n===== KaTeX DOM HTML =====")
+        print(katex_dom_html)
+
+    # Extract inner-most spans with non-whitespace text.
+    spans_info = page.evaluate(
+        """
+    () => {
+        const spans = Array.from(document.querySelectorAll('span'));
+        const list = [];
+        spans.forEach(span => {
+            if (span.children.length === 0 && /\\S/.test(span.textContent)) {
+                const rect = span.getBoundingClientRect();
+                list.push({
+                    text: span.textContent.trim(),
+                    boundingBox: {
+                        x: rect.x,
+                        y: rect.y,
+                        width: rect.width,
+                        height: rect.height
+                    }
+                });
+            }
+        });
+        return list;
+    }
+    """
+    )
+
+    if debug_dom:
+        print("\n===== Extracted Span Information =====")
+        print(spans_info)
+
+    # Extract MathML output (if available) from the KaTeX output.
+    mathml = page.evaluate(
+        """
+    () => {
+        const mathElem = document.querySelector('.katex-mathml math');
+        return mathElem ? mathElem.outerHTML : "";
+    }
+    """
+    )
+
+    page.close()
+
+    rendered_eq = RenderedEquation(
+        mathml=mathml,
+        spans=[
+            SpanInfo(
+                text=s["text"],
+                bounding_box=BoundingBox(
+                    x=s["boundingBox"]["x"],
+                    y=s["boundingBox"]["y"],
+                    width=s["boundingBox"]["width"],
+                    height=s["boundingBox"]["height"],
+                ),
+            )
+            for s in spans_info
+        ],
+    )
+
+    # Save the successfully rendered equation to the SQLite cache.
+    if use_cache:
+        equation_cache.save(eq_hash, rendered_eq)
+    return rendered_eq
+
+
+def compare_rendered_equations(reference: RenderedEquation, hypothesis: RenderedEquation) -> bool:
+    """
+    Compare two RenderedEquation objects.
+    First, check if the normalized MathML of the hypothesis is contained within that of the reference.
+    If not, perform a neighbor-based matching on the spans.
+    """
+    from bs4 import BeautifulSoup
+
+    def extract_inner(mathml: str) -> str:
+        try:
+            soup = BeautifulSoup(mathml, "xml")
+            semantics = soup.find("semantics")
+            if semantics:
+                inner_parts = [
+                    str(child)
+                    for child in semantics.contents
+                    if getattr(child, "name", None) != "annotation"
+                ]
+                return "".join(inner_parts)
+            else:
+                return str(soup)
+        except Exception as e:
+            print("Error parsing MathML with BeautifulSoup:", e)
+            print(mathml)
+            return mathml
+
+    def normalize(s: str) -> str:
+        return re.sub(r"\s+", "", s)
+
+    reference_inner = normalize(extract_inner(reference.mathml))
+    hypothesis_inner = normalize(extract_inner(hypothesis.mathml))
+    if reference_inner in hypothesis_inner:
+        return True
+
+    H, R = reference.spans, hypothesis.spans
+    H = [span for span in H if span.text != "\u200b"]
+    R = [span for span in R if span.text != "\u200b"]
+
+    def expand_span_info(span_info: SpanInfo) -> list[SpanInfo]:
+        total_elems = len(span_info.text)
+        return [
+            SpanInfo(
+                c,
+                BoundingBox(
+                    span_info.bounding_box.x + (span_info.bounding_box.width * index) / total_elems,
+                    span_info.bounding_box.y,
+                    span_info.bounding_box.width / total_elems,
+                    span_info.bounding_box.height,
+                ),
+            )
+            for index, c in enumerate(span_info.text)
+        ]
+
+    H = [span for sublist in H for span in expand_span_info(sublist)]
+    R = [span for sublist in R for span in expand_span_info(sublist)]
+
+    candidate_map = {}
+    for i, hspan in enumerate(H):
+        candidate_map[i] = [j for j, rsp in enumerate(R) if rsp.text == hspan.text]
+        if not candidate_map[i]:
+            return False
+
+    def compute_neighbors(spans, tol=5):
+        neighbors = {}
+        for i, span in enumerate(spans):
+            cx = span.bounding_box.x + span.bounding_box.width / 2
+            cy = span.bounding_box.y + span.bounding_box.height / 2
+            up = down = left = right = None
+            up_dist = down_dist = left_dist = right_dist = None
+            for j, other in enumerate(spans):
+                if i == j:
+                    continue
+                ocx = other.bounding_box.x + other.bounding_box.width / 2
+                ocy = other.bounding_box.y + other.bounding_box.height / 2
+                if ocy < cy and abs(ocx - cx) <= tol:
+                    dist = cy - ocy
+                    if up is None or dist < up_dist:
+                        up = j
+                        up_dist = dist
+                if ocy > cy and abs(ocx - cx) <= tol:
+                    dist = ocy - cy
+                    if down is None or dist < down_dist:
+                        down = j
+                        down_dist = dist
+                if ocx < cx and abs(ocy - cy) <= tol:
+                    dist = cx - ocx
+                    if left is None or dist < left_dist:
+                        left = j
+                        left_dist = dist
+                if ocx > cx and abs(ocy - cy) <= tol:
+                    dist = ocx - cx
+                    if right is None or dist < right_dist:
+                        right = j
+                        right_dist = dist
+            neighbors[i] = {"up": up, "down": down, "left": left, "right": right}
+        return neighbors
+
+    hyp_neighbors = compute_neighbors(H)
+    ref_neighbors = compute_neighbors(R)
+
+    n = len(H)
+    used = [False] * len(R)
+    assignment = {}
+
+    def backtrack(i):
+        if i == n:
+            return True
+        for cand in candidate_map[i]:
+            if used[cand]:
+                continue
+            assignment[i] = cand
+            used[cand] = True
+            valid = True
+            for direction in ["up", "down", "left", "right"]:
+                hyp_nb = hyp_neighbors[i].get(direction)
+                ref_nb = ref_neighbors[cand].get(direction)
+                if hyp_nb is not None:
+                    expected_text = H[hyp_nb].text
+                    if ref_nb is None:
+                        valid = False
+                        break
+                    if hyp_nb in assignment:
+                        if assignment[hyp_nb] != ref_nb:
+                            valid = False
+                            break
+                    else:
+                        if R[ref_nb].text != expected_text:
+                            valid = False
+                            break
+            if valid:
+                if backtrack(i + 1):
+                    return True
+            used[cand] = False
+            del assignment[i]
+        return False
+
+    return backtrack(0)
+
+
+class TestRenderedEquationComparison(unittest.TestCase):
+    def test_exact_match(self):
+        eq1 = render_equation("a+b", use_cache=False)
+        eq2 = render_equation("a+b", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_whitespace_difference(self):
+        eq1 = render_equation("a+b", use_cache=False)
+        eq2 = render_equation("a + b", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_not_found(self):
+        eq1 = render_equation("c-d", use_cache=False)
+        eq2 = render_equation("a+b", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_align_block_contains_needle(self):
+        eq_plain = render_equation("a+b", use_cache=False)
+        eq_align = render_equation("\\begin{align*}a+b\\end{align*}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq_plain, eq_align))
+
+    def test_align_block_needle_not_in(self):
+        eq_align = render_equation("\\begin{align*}a+b\\end{align*}", use_cache=False)
+        eq_diff = render_equation("c-d", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq_diff, eq_align))
+
+    def test_big(self):
+        ref_rendered = render_equation(
+            "\\nabla \\cdot \\mathbf{E} = \\frac{\\rho}{\\varepsilon_0}",
+            use_cache=False, debug_dom=False
+        )
+        align_rendered = render_equation(
+            """\\begin{align*}\\nabla \\cdot \\mathbf{E} = \\frac{\\rho}{\\varepsilon_0}\\end{align*}""",
+            use_cache=False, debug_dom=False
+        )
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_dot_end1(self):
+        ref_rendered = render_equation(
+            "\\lambda_{g}=\\sum_{s \\in S} \\zeta_{n}^{\\psi(g s)}="
+            "\\sum_{i=1}^{k}\\left[\\sum_{s, R s=\\mathcal{I}_{i}} "
+            "\\zeta_{n}^{\\varphi(g s)}\\right]"
+        )
+        align_rendered = render_equation(
+            "\\lambda_{g}=\\sum_{s \\in S} \\zeta_{n}^{\\psi(g s)}="
+            "\\sum_{i=1}^{k}\\left[\\sum_{s, R s=\\mathcal{I}_{i}} "
+            "\\zeta_{n}^{\\varphi(g s)}\\right]."
+        )
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_x_vs_textx(self):
+        ref_rendered = render_equation(
+            "C_{T}\\left(u_{n}^{T} X_{n}^{\\text {Test }}, \\bar{x}^{\\text {Test }}\\right)"
+        )
+        align_rendered = render_equation(
+            "C_T \\left(u^T_n X^{\\text{Test}}_n,\\overline{ \\text{x}}^{\\text{Test}}\\right)"
+        )
+        self.assertFalse(compare_rendered_equations(ref_rendered, align_rendered))
+
+    @unittest.skip("There is a debate whether bar and overline should be the same, currently they are not")
+    def test_overline(self):
+        ref_rendered = render_equation(
+            "C_{T}\\left(u_{n}^{T} X_{n}^{\\text {Test }}, \\bar{x}^{\\text {Test }}\\right)"
+        )
+        align_rendered = render_equation(
+            "C_T \\left(u^T_n X^{\\text{Test}}_n,\\overline{ x}^{\\text{Test}}\\right)"
+        )
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_parens(self):
+        ref_rendered = render_equation("\\left\\{ \\left( 0_{X},0_{Y},-1\\right) \\right\\} ")
+        align_rendered = render_equation("\\{(0_{X}, 0_{Y}, -1)\\}")
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_dot_end2(self):
+        ref_rendered = render_equation(
+            "\\lambda_{g}=\\sum_{s \\in S} \\zeta_{n}^{\\psi(g s)}="
+            "\\sum_{i=1}^{k}\\left[\\sum_{s, R s=\\mathcal{I}_{i}} "
+            "\\zeta_{n}^{\\psi(g s)}\\right]"
+        )
+        align_rendered = render_equation(
+            "\\lambda_g = \\sum_{s \\in S} \\zeta_n^{\\psi(gs)} = "
+            "\\sum_{i=1}^{k} \\left[ \\sum_{s, Rs = \\mathcal{I}_i} "
+            "\\zeta_n^{\\psi(gs)} \\right]"
+        )
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_lambda(self):
+        ref_rendered = render_equation("\\lambda_g = \\lambda_{g'}")
+        align_rendered = render_equation("\\lambda_{g}=\\lambda_{g^{\\prime}}")
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_gemini(self):
+        ref_rendered = render_equation("u \\in (R/\\operatorname{Ann}_R(x_i))^{\\times}")
+        align_rendered = render_equation(
+            "u \\in\\left(R / \\operatorname{Ann}_{R}\\left(x_{i}\\right)\\right)^{\\times}"
+        )
+        self.assertTrue(compare_rendered_equations(ref_rendered, align_rendered))
+
+    def test_fraction_vs_divided_by(self):
+        eq1 = render_equation("\\frac{a}{b}", use_cache=False)
+        eq2 = render_equation("a / b", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_different_bracket_types(self):
+        eq1 = render_equation("\\left[ a + b \\right]", use_cache=False)
+        eq2 = render_equation("\\left\\{ a + b \\right\\}", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_inline_vs_display_style_fraction(self):
+        eq1 = render_equation("\\frac{1}{2}", use_cache=False)
+        eq2 = render_equation("\\displaystyle\\frac{1}{2}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_matrix_equivalent_forms(self):
+        eq1 = render_equation("\\begin{pmatrix} a & b \\\\ c & d \\end{pmatrix}", use_cache=False)
+        eq2 = render_equation("\\begin{pmatrix} a & b \\\\ c & d \\end{pmatrix}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_different_matrix_types(self):
+        eq1 = render_equation("\\begin{pmatrix} a & b \\\\ c & d \\end{pmatrix}", use_cache=False)
+        eq2 = render_equation("\\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix}", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_thinspace_vs_regular_space(self):
+        eq1 = render_equation("a \\, b", use_cache=False)
+        eq2 = render_equation("a \\: b", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    @unittest.skip(
+        "Currently these compare to the same thing, "
+        "because they use the symbol 'x' with a different span class and thus font"
+    )
+    def test_mathbf_vs_boldsymbol(self):
+        eq1 = render_equation("\\mathbf{x}", use_cache=False)
+        eq2 = render_equation("\\boldsymbol{x}", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_assert_subtle_square_root(self):
+        eq1 = render_equation(
+            "A N'P' = \\int \\beta d\\alpha = "
+            "\\frac{2}{3\\sqrt{3} a}\\int (\\alpha - 2a)^{\\frac{3}{2}} d\\alpha",
+            use_cache=False,
+        )
+        eq2 = render_equation(
+            "AN'P' = \\int \\beta \\, d\\alpha = "
+            "\\frac{2}{3 \\sqrt{3a}} \\int (a - 2a)^{\\frac{3}{2}} d\\alpha",
+        )
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_text_added(self):
+        eq1 = render_equation(
+            "A N'P' = \\int \\beta d\\alpha = "
+            "\\frac{2}{3\\sqrt{3} a}\\int (\\alpha - 2a)^{\\frac{3}{2}} d\\alpha",
+            use_cache=False,
+        )
+        eq2 = render_equation(
+            "AN'P' = \\int \\beta  d\\alpha = "
+            "\\frac{2}{3 \\sqrt{3} a} \\int (\\alpha - 2a)^{\\frac{3}{2}} d\\alpha",
+        )
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+        eq1 = render_equation(
+            "A N'P' = \\int \\beta d\\alpha = "
+            "\\frac{2}{3\\sqrt{3} a}\\int (\\alpha - 2a)^{\\frac{3}{2}} d\\alpha",
+            use_cache=False,
+        )
+        eq2 = render_equation(
+            "\\text{area evolute } AN'P' = \\int \\beta  d\\alpha = "
+            "\\frac{2}{3 \\sqrt{3} a} \\int (\\alpha - 2a)^{\\frac{3}{2}} d\\alpha"
+        )
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_tensor_notation_equivalent(self):
+        eq1 = render_equation("T_{ij}^{kl}", use_cache=False)
+        eq2 = render_equation("T^{kl}_{ij}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_partial_derivative_forms(self):
+        eq1 = render_equation("\\frac{\\partial f}{\\partial x}", use_cache=False)
+        eq2 = render_equation("\\frac{\\partial_f}{\\partial_x}", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_equivalent_sin_forms_diff_parens(self):
+        eq1 = render_equation("\\sin(\\theta)", use_cache=False)
+        eq2 = render_equation("\\sin \\theta", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_aligned_multiline_equation(self):
+        eq1 = render_equation("\\begin{align*} a &= b \\\\ c &= d \\end{align*}", use_cache=False)
+        eq2 = render_equation("\\begin{aligned} a &= b \\\\ c &= d \\end{aligned}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_subscript_order_invariance(self):
+        eq1 = render_equation("x_{i,j}", use_cache=False)
+        eq2 = render_equation("x_{j,i}", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_hat_vs_widehat(self):
+        eq1 = render_equation("\\hat{x}", use_cache=False)
+        eq2 = render_equation("\\widehat{x}", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_equivalent_integral_bounds(self):
+        eq1 = render_equation("\\int_{a}^{b} f(x) dx", use_cache=False)
+        eq2 = render_equation("\\int\\limits_{a}^{b} f(x) dx", use_cache=False)
+        # Could go either way honestly?
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_equivalent_summation_notation(self):
+        eq1 = render_equation("\\sum_{i=1}^{n} x_i", use_cache=False)
+        eq2 = render_equation("\\sum\\limits_{i=1}^{n} x_i", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_different_symbol_with_same_appearance(self):
+        eq1 = render_equation("\\phi", use_cache=False)
+        eq2 = render_equation("\\varphi", use_cache=False)
+        self.assertFalse(compare_rendered_equations(eq1, eq2))
+
+    def test_aligned_vs_gathered(self):
+        eq1 = render_equation("\\begin{aligned} a &= b \\\\ c &= d \\end{aligned}", use_cache=False)
+        eq2 = render_equation("\\begin{gathered} a = b \\\\ c = d \\end{gathered}", use_cache=False)
+        # Different whitespacing, should be invariant to that.
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_identical_but_with_color1(self):
+        eq1 = render_equation("a + b", use_cache=False)
+        eq2 = render_equation("\\color{black}{a + b}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_identical_but_with_color2(self):
+        eq1 = render_equation("a + b", use_cache=False)
+        eq2 = render_equation("\\color{black}{a} + \\color{black}{b}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+        eq1 = render_equation("a + b", use_cache=False)
+        eq2 = render_equation("\\color{red}{a} + \\color{black}{b}", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+    def test_newcommand_expansion(self):
+        eq1 = render_equation("\\alpha + \\beta", use_cache=False)
+        eq2 = render_equation("\\newcommand{\\ab}{\\alpha + \\beta}\\ab", use_cache=False)
+        self.assertTrue(compare_rendered_equations(eq1, eq2))
+
+
+if __name__ == "__main__":
+    unittest.main()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/dominating_set.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/dominating_set.py
new file mode 100644
index 0000000000000000000000000000000000000000..7c60aeab30d62d203e6e4f39744d43b2d2f1c96c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/dominating_set.py
@@ -0,0 +1,74 @@
+import re
+
+
+def validation(graph, answer):
+    """
+    验证给定的答案是否是一个有效的支配集方案
+    参数:
+    graph: 图的表示，包含邻接表
+    answer: 字符串答案，包含"Answer："后的顶点列表
+    返回:
+    (bool, int, str): 布尔值表示答案是否有效，True代表无效，False代表有效，
+                       整数表示支配集的大小（越小越好）。如果答案无效，返回一个比任何可能解都大的值。
+                       字符串提供错误或成功信息。
+    """
+    num_vertices = len(graph)
+    penalty_value = num_vertices + 1
+    if "Answer:" in answer:
+        vertices_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, penalty_value, "invalid answer: no 'Answer:' in answer"
+
+    # 清理字符串
+    vertices_str = vertices_str.strip().replace("'", "").replace('"', '')
+    # 提取顶点列表
+    pattern = r'\[([^\]]*)\]'
+    match = re.search(pattern, vertices_str)
+    if not match:
+        return True, penalty_value, "vertices must be in list format"
+    vertices_content = match.group(1)
+    # 解析顶点
+    try:
+        if vertices_content.strip() == '':
+            # 空列表可能对于非常小的图有效
+            vertices = []
+        else:
+            vertices = [int(x.strip()) for x in vertices_content.split(',') if x.strip() != '']
+        # 去除重复并排序
+        vertices = sorted(list(set(vertices)))
+    except Exception as e:
+        return True, penalty_value, f"vertices must be integers: {str(e)}"
+    # 检查所有顶点是否有效
+    for v in vertices:
+        if v < 0 or v >= num_vertices:
+            return True, penalty_value, f"invalid vertex {v}: must be in range [0, {num_vertices - 1}]"
+    # 检查是否是有效的支配集
+    dominating_set = set(vertices)
+    # 检查每个顶点是否在支配集中或与支配集中的顶点相邻
+    for node in range(num_vertices):
+        node_str = str(node)
+        # 检查节点是否在支配集中
+        if node in dominating_set:
+            continue
+        # 检查节点是否与支配集中的任何顶点相邻
+        is_dominated = False
+        neighbors = graph.get(node_str, [])
+        for neighbor in neighbors:
+            if isinstance(neighbor, str):
+                neighbor = int(neighbor)
+            if neighbor in dominating_set:
+                is_dominated = True
+                break
+        if not is_dominated:
+            return True, penalty_value, f"invalid dominating set: vertex {node} is not dominated"
+    # 有效的支配集
+    dominating_size = len(vertices)
+    # 特殊情况：空支配集
+    if dominating_size == 0:
+        # 空图
+        if num_vertices == 0:
+            return False, 0, "valid dominating set: empty set for empty graph"
+        else:
+            # 对于非空图，空集不是有效的支配集
+            return True, penalty_value, "invalid dominating set: empty set cannot dominate non-empty graph"
+    return False, dominating_size, f"valid dominating set with {dominating_size} vertices"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/feedback_vertex.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/feedback_vertex.py
new file mode 100644
index 0000000000000000000000000000000000000000..2dd0ccca3bbc0186fe4eec7371211d3e6a779823
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/feedback_vertex.py
@@ -0,0 +1,102 @@
+import re
+from collections import defaultdict
+
+
+def validation(graph, answer):
+    """
+    验证给定的答案是否是一个有效的反馈顶点集方案
+    参数:
+    graph: 图的表示，包含邻接表
+    answer: 字符串答案，包含"Answer："后的顶点列表
+
+    返回:
+    (bool, int, str): 布尔值表示答案是否有效，True代表无效，False代表有效，
+                       整数表示反馈顶点集的大小（越小越好）。如果答案无效，返回一个比任何可能解都大的值。
+                       字符串提供错误或成功信息。
+    """
+    num_vertices = len(graph)
+    # 定义一个惩罚值，用于无效答案
+    penalty_value = num_vertices + 1
+    # 解析answer字符串，提取"Answer:"后面的内容
+    if "Answer:" in answer:
+        vertices_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, penalty_value, "invalid answer: no 'Answer:' in answer"
+    # 清理字符串
+    vertices_str = vertices_str.strip().replace("'", "").replace('"', '')
+    # 提取顶点列表
+    pattern = r'\[([^\]]*)\]'
+    match = re.search(pattern, vertices_str)
+    if not match:
+        return True, penalty_value, "vertices must be in list format"
+    vertices_content = match.group(1)
+    # 解析顶点
+    try:
+        if vertices_content.strip() == '':
+            # 空列表对于无环图有效
+            vertices = []
+        else:
+            vertices = [int(x.strip()) for x in vertices_content.split(',') if x.strip() != '']
+        # 去除重复并排序
+        vertices = sorted(list(set(vertices)))
+    except Exception as e:
+        return True, penalty_value, f"vertices must be integers: {str(e)}"
+    # 检查所有顶点是否有效
+    for v in vertices:
+        if v < 0 or v >= num_vertices:
+            return True, penalty_value, f"invalid vertex {v}: must be in range [0, {num_vertices - 1}]"
+    # 构建移除反馈顶点集后的邻接表
+    remaining_adj = defaultdict(list)
+    removed_set = set(vertices)
+    for node_str, neighbors in graph.items():
+        node = int(node_str)
+        if node in removed_set:
+            continue
+        for neighbor_str in neighbors:
+            if isinstance(neighbor_str, str):
+                neighbor = int(neighbor_str)
+            else:
+                neighbor = neighbor_str
+
+            if neighbor not in removed_set:
+                remaining_adj[node].append(neighbor)
+
+    # 检查剩余图是否无环
+    def has_cycle_dfs(adj, num_verts):
+        """使用DFS检查图是否有环"""
+        visited = set()
+        rec_stack = set()
+
+        def dfs(node, parent):
+            visited.add(node)
+            rec_stack.add(node)
+
+            for neighbor in adj.get(node, []):
+                if neighbor not in visited:
+                    if dfs(neighbor, node):
+                        return True
+                elif neighbor != parent:
+                    # 发现后向边（检测到环）
+                    return True
+
+            rec_stack.remove(node)
+            return False
+
+        # 检查所有连通分量
+        for node in range(num_verts):
+            if node in adj and node not in visited:
+                if dfs(node, -1):
+                    return True
+
+        return False
+
+    is_acyclic = not has_cycle_dfs(remaining_adj, num_vertices)
+
+    if not is_acyclic:
+        # 解决方案没有移除所有环
+        return True, penalty_value, "invalid feedback vertex set: remaining graph still contains cycles"
+
+    # 有效的反馈顶点集
+    fvs_size = len(vertices)
+
+    return False, fvs_size, f"valid feedback vertex set with {fvs_size} vertices"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/gcp.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/gcp.py
new file mode 100644
index 0000000000000000000000000000000000000000..8f5c574920e5f5367258e6b79017122aa7ae7dba
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/gcp.py
@@ -0,0 +1,95 @@
+def validation(graph, answer):
+    """
+    验证给定的答案是否是一个有效的图着色方案
+
+    参数:
+    graph: 图的表示，包含邻接表
+    answer: 字符串答案，包含"Answer："后的着色方案
+
+    返回:
+    (bool, int, str): 布尔值表示答案是否有效，True代表无效，False代表有效，
+                       整数表示使用的颜色数。如果答案无效，返回一个比任何可能解都大的值。
+                       字符串提供错误或成功信息。
+    """
+    num_vertices = len(graph)
+    # 定义一个惩罚值，用于无效答案。
+    # 这个值比可能的最大颜色数（每个节点一种颜色）还要大1。
+    penalty_value = num_vertices + 1
+
+    # 解析answer字符串，提取"Answer:"后面的内容
+    if "Answer:" in answer:
+        color_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, penalty_value, "invalid answer: no 'Answer:' in answer"
+
+    # 标准化颜色字符串格式
+    color_str = color_str.strip()
+
+    # 如果已经是 [x,x,x,...,x] 格式，直接使用
+    if color_str.startswith('[') and color_str.endswith(']'):
+        color_str = color_str[1:-1]  # 去掉方括号
+    else:
+        # 处理 -> 格式
+        if '->' in color_str:
+            # 分割并清理每个节点
+            nodes = [node.strip() for node in color_str.split('->')]
+            # 重新组合成标准格式
+            color_str = ','.join(nodes)
+        else:
+            # 处理其他括号格式 {x,x,x} 或 (x,x,x) 或 ['x','x','x']
+            # 移除所有引号
+            color_str = color_str.replace("'", "").replace('"', '')
+
+            # 使用正则表达式匹配括号内容
+            import re
+            pattern = r'[{\[\(]([^)}\]]*)[}\])]'
+            match = re.search(pattern, color_str)
+
+            if match:
+                # 提取括号内的内容
+                color_str = match.group(1)
+            else:
+                # 如果没有任何括号，假设是逗号分隔的列表
+                pass
+
+    # 尝试将颜色字符串转换为整数列表
+    try:
+        # 处理空字符串的情况
+        if not color_str.strip():
+            colors = []
+        else:
+            colors = [int(x.strip()) for x in color_str.split(',')]  # 将颜色字符串转换为整数列表
+    except (ValueError, IndexError):
+        return True, penalty_value, "coloring must be a list of integers"
+
+    if not colors:
+        return True, penalty_value, "coloring cannot be empty"
+
+    if -1 in colors:
+        return True, penalty_value, "not valid response with -1 in coloring list, coloring must be a list of integers and bigger than 0"  # noqa: E501
+
+    # 检查着色方案的节点数量是否与图匹配
+    if len(colors) != num_vertices:
+        return True, penalty_value, f"invalid coloring: not all vertices are colored, overall {num_vertices} vertices, got {len(colors)} vertices"  # noqa: E501
+
+    # 检查是否是有效的着色方案
+
+    # 1. 检查是否有相邻的顶点共享相同颜色
+    for node_str, neighbors in graph.items():
+        node = int(node_str)
+        node_color = colors[node]
+        for neighbor_str in neighbors:
+            neighbor = int(neighbor_str)
+            neighbor_color = colors[neighbor]
+            if node_color == neighbor_color:
+                return True, penalty_value, f"invalid coloring: node {node} and node {neighbor} have the same color {node_color}"  # noqa: E501
+
+    # 2. 如果所有检查都通过，返回有效的着色方案
+    num_used_colors = len(set(colors))
+    return False, num_used_colors, f"valid coloring with {num_used_colors} colors, try to use less color to assign"
+
+
+if __name__ == "__main__":
+    graph = {'0': [2, 6, 7, 11, 13, 14, 16], '1': [4, 5, 6, 8, 14, 15], '2': [0, 6, 7, 11, 12, 13, 14, 15], '3': [4, 8, 9, 10, 13, 15, 16], '4': [1, 3, 5, 8, 9, 12, 13, 16], '5': [1, 4, 9, 11, 12, 13, 14], '6': [0, 1, 2, 9, 10, 11, 12, 15, 16], '7': [0, 2, 9, 10, 11, 12, 14, 15, 16], '8': [1, 3, 4, 9, 12, 13, 14], '9': [3, 4, 5, 6, 7, 8, 10, 12, 16], '10': [3, 6, 7, 9, 13, 16], '11': [0, 2, 5, 6, 7, 13, 14, 15], '12': [2, 4, 5, 6, 7, 8, 9, 13, 15, 16], '13': [0, 2, 3, 4, 5, 8, 10, 11, 12, 14, 15], '14': [0, 1, 2, 5, 7, 8, 11, 13, 15], '15': [1, 2, 3, 6, 7, 11, 12, 13, 14], '16': [0, 3, 4, 6, 7, 9, 10, 12]}  # noqa: E501
+    answer = "Answer: [1, 3, 2, 1, 4, 3, 1, 5, 4, 2, 5, 5, 1, 2, 6, 2, 6]"
+    print(validation(graph, answer))
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/hamiltonian_cycle.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/hamiltonian_cycle.py
new file mode 100644
index 0000000000000000000000000000000000000000..78ab61efb86a9f3b238fbded24661618bd5f8986
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/hamiltonian_cycle.py
@@ -0,0 +1,90 @@
+def validation(graph, answer):
+    """
+    验证给定的答案是否是一个有效的哈密顿回路
+
+    参数:
+    graph: 图的表示，包含nodes和adjacency_list
+    answer: 字符串答案，包含"Answer："后的路径
+
+    返回:
+    (bool, str): 布尔值表示答案是否正确，字符串提供错误信息（如有）
+    """
+    # 解析answer字符串，提取"Answer："后面的内容
+    if "Answer:" in answer:
+        path_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, -1, "invalid answer: no 'Answer:' in answer"
+
+    # 标准化路径字符串格式
+    path_str = path_str.strip()
+
+    # 如果已经是 [x,x,x,...,x] 格式，直接使用
+    if path_str.startswith('[') and path_str.endswith(']'):
+        path_str = path_str[1:-1]  # 去掉方括号
+    else:
+        # 处理 -> 格式
+        if '->' in path_str:
+            # 分割并清理每个节点
+            nodes = [node.strip() for node in path_str.split('->')]
+            # 重新组合成标准格式
+            path_str = ','.join(nodes)
+        else:
+            # 处理其他括号格式 {x,x,x} 或 (x,x,x) 或 ['x','x','x']
+            # 移除所有引号
+            path_str = path_str.replace("'", "").replace('"', '')
+
+            # 使用正则表达式匹配括号内容
+            import re
+            pattern = r'[{\[\(]([^)}\]]*)[}\])]'
+            match = re.search(pattern, path_str)
+
+            if match:
+                # 提取括号内的内容
+                path_str = match.group(1)
+            else:
+                # 如果没有任何括号，假设是逗号分隔的列表
+                pass
+
+    # 尝试将路径字符串转换为节点列表
+    try:
+        # 分割并转换为整数列表
+        try:
+            path = [int(x.strip()) for x in path_str.split(',')]
+        except Exception:
+            return True, -1, "path must be a list of integers"
+
+        if not path:
+            return True, -1, "path cannot be empty"
+    except Exception:
+        return True, -1, "invalid answer format"
+
+    # 获取图的节点数和邻接表
+    adjacency_list = graph
+
+    # 检查路径是否是哈密顿回路
+
+    # 2. 检查起点和终点是否相同（闭环）
+    if path[0] != path[-1]:
+        return True, -1, f"path is not a cycle: start{path[0]} and end{path[-1]} are different"
+
+    # 3. 检查是否访问了所有节点且只访问一次（除了起点/终点）
+    visited = set()
+    for i in range(len(path) - 1):
+        if path[i] in visited:
+            return True, -1, f"node {path[i]} is visited more than once"
+        visited.add(path[i])
+
+    # 4. 检查相邻节点之间是否有边连接
+    for i in range(len(path) - 1):
+        current = str(path[i])  # 转为字符串，因为adjacency_list的键可能是字符串
+        next_node = path[i + 1]  # 保持为整数
+
+        if current not in adjacency_list:
+            return True, -1, f"node {current} is not in adjacency list"
+
+        # 邻接表中的值可能是整数列表
+        neighbors = adjacency_list[current]
+        if next_node not in neighbors:
+            return True, -1, f"node {current} and {next_node} are not connected"
+
+    return False, len(path), f"path length: {len(path)}"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/maximum_cut.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/maximum_cut.py
new file mode 100644
index 0000000000000000000000000000000000000000..f9b37625c98b0c38b213efcce40ffa90fd0e5a00
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/maximum_cut.py
@@ -0,0 +1,91 @@
+import re
+
+
+def validation(question, answer):
+    """
+    验证给定的答案是否是一个有效的最大割方案
+
+    参数:
+    question: 问题表示，包含 num_vertices 和 edges（边及其权重）
+    answer: 字符串答案，包含"Answer："后的两个集合
+
+    返回:
+    (bool, int, str): 布尔值表示答案是否有效，True代表无效，False代表有效，
+                       整数表示割的总权重（越大越好）。如果答案无效，返回0。
+                       字符串提供错误或成功信息。
+    """
+    # 解析问题
+    if isinstance(question, dict):
+        edges_with_weights = question.get("edges", {})
+        num_vertices = question.get("num_vertices", 0)
+    else:
+        return True, 0, "invalid question format"
+
+    penalty_value = 0
+
+    # 解析answer字符串，提取"Answer:"后面的内容
+    if "Answer:" in answer:
+        partition_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, penalty_value, "invalid answer: no 'Answer:' in answer"
+
+    # 清理字符串
+    partition_str = partition_str.strip().replace("'", "").replace('"', '')
+
+    # 解析嵌套列表结构 [[set1], [set2]]
+    try:
+        # 匹配外层括号
+        pattern = r'\[\s*\[([^\]]*)\]\s*,\s*\[([^\]]*)\]\s*\]'
+        match = re.search(pattern, partition_str)
+
+        if not match:
+            return True, penalty_value, "partition must be in format [[set1], [set2]]"
+
+        set1_str = match.group(1).strip()
+        set2_str = match.group(2).strip()
+
+        # 解析每个集合
+        if set1_str == '':
+            set1 = []
+        else:
+            set1 = [int(x.strip()) for x in set1_str.split(',') if x.strip() != '']
+
+        if set2_str == '':
+            set2 = []
+        else:
+            set2 = [int(x.strip()) for x in set2_str.split(',') if x.strip() != '']
+
+        # 去除每个集合内的重复元素
+        set1 = sorted(list(set(set1)))
+        set2 = sorted(list(set(set2)))
+
+    except Exception as e:
+        return True, penalty_value, f"failed to parse partition: {str(e)}"
+
+    # 检查划分是否有效（所有顶点恰好出现一次）
+    all_vertices = set(set1) | set(set2)
+    if len(set1) + len(set2) != len(all_vertices):
+        return True, penalty_value, "partition contains duplicate vertices"
+
+    if all_vertices != set(range(num_vertices)):
+        return True, penalty_value, f"partition must include all {num_vertices} vertices"
+
+    # 检查所有顶点是否有效
+    for v in set1 + set2:
+        if v < 0 or v >= num_vertices:
+            return True, penalty_value, f"invalid vertex {v}: must be in range [0, {num_vertices - 1}]"
+
+    # 转换为集合以便高效查找
+    set1_set = set(set1)
+    set2_set = set(set2)
+
+    # 计算割的值
+    cut_value = 0
+    for edge_key, weight in edges_with_weights.items():
+        u, v = map(int, edge_key.split('-'))
+
+        # 检查这条边是否跨越割
+        if (u in set1_set and v in set2_set) or (u in set2_set and v in set1_set):
+            cut_value += weight
+
+    return False, cut_value, f"valid partition with cut value {cut_value}"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/maximum_set.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/maximum_set.py
new file mode 100644
index 0000000000000000000000000000000000000000..55f5ff764cc09ec4a4fc97b70c276afe98811b36
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/maximum_set.py
@@ -0,0 +1,68 @@
+def validation(graph, answer):
+    """
+    Validate if the given answer is a valid maximum independent set
+
+    Parameters:
+    graph: JSON graph data from the question (adjacency list format)
+    answer: String answer containing the proposed independent set after "Answer:"
+
+    Returns:
+    tuple: (is_invalid, size, message)
+            is_invalid - True if invalid, False if valid
+            size - size of the independent set (-1 if invalid)
+            message - validation feedback
+    """
+    # Parse answer string
+    if "Answer:" not in answer:
+        return True, -1, "Invalid answer format: missing 'Answer:' prefix"
+
+    answer_part = answer.split("Answer:")[-1].split('\n')[0].strip()
+
+    try:
+        # Handle different answer formats
+        if answer_part.startswith('[') and answer_part.endswith(']'):
+            independent_set = eval(answer_part)
+        else:
+            # Try to parse other formats (comma separated, etc.)
+            independent_set = [int(x.strip()) for x in answer_part.replace('[', '').replace(']', '').split(',')]
+
+        if not isinstance(independent_set, list):
+            return True, -1, "Answer must be a list of vertices"
+    except Exception:
+        return True, -1, "Could not parse the independent set from answer"
+
+    # Convert all node keys to strings for consistency
+    graph_nodes = set(graph.keys())
+
+    # Check all nodes exist in graph
+    invalid_nodes = [str(node) for node in independent_set if str(node) not in graph_nodes]
+    if invalid_nodes:
+        return True, -1, f"Nodes not in graph: {', '.join(invalid_nodes)}"
+
+    # Check for duplicate nodes
+    if len(independent_set) != len(set(independent_set)):
+        return True, -1, "Duplicate nodes found in the set"
+
+    # Check for independence (no two nodes are adjacent)
+    for i in range(len(independent_set)):
+        for j in range(i + 1, len(independent_set)):
+            node1 = str(independent_set[i])
+            node2 = str(independent_set[j])
+            if node2 in graph.get(node1, {}):
+                return True, -1, f"Nodes {node1} and {node2} are adjacent (violates independence)"
+
+    # Check for maximality (not necessarily maximum, just not obviously improvable)
+    # Note: This doesn't verify if it's the absolute maximum, just if it's valid
+    neighbor_nodes = set()
+    for node in independent_set:
+        neighbors = graph.get(str(node), [])
+        if isinstance(neighbors, dict):
+            neighbor_nodes.update(neighbors.keys())
+        else:
+            neighbor_nodes.update(str(n) for n in neighbors)
+
+    remaining_nodes = graph_nodes - set(map(str, independent_set)) - neighbor_nodes
+    if remaining_nodes:
+        return False, len(independent_set), f"Valid independent set (size {len(independent_set)}), but possibly not maximal - could potentially add nodes: {remaining_nodes}"  # noqa: E501
+
+    return False, len(independent_set), f"Valid maximal independent set (size {len(independent_set)})"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/mcp.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/mcp.py
new file mode 100644
index 0000000000000000000000000000000000000000..97b5736326ca4c78ba9aa87b1ca1165a7d6f0149
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/mcp.py
@@ -0,0 +1,116 @@
+def validation(graph, answer):
+    """
+    验证给定的答案是否是图中的一个有效团（clique），并返回该团的大小。
+
+    参数:
+      graph: dict，图的邻接表示。兼容以下几种常见形式：
+        1) {int: [int, ...]} 或 {str: [str/int, ...]}
+        2) {int/str: {int/str: 0/1/权重,...}}（邻接字典/加权）
+        3) {int/str: set(...)}
+
+      answer: str，格式为 "Answer: [0, 1, 2]"
+
+    返回:
+      (bool, int, str):
+        - bool: True 表示非法；False 表示合法
+        - int: 团大小（非法时为 -1）
+        - str: 说明
+    """
+    import ast
+    from collections import defaultdict
+
+    # ---------- 1) 解析答案 ----------
+    if "Answer:" not in answer:
+        return True, -1, "invalid answer: no 'Answer:' in answer"
+
+    cut_str = answer.split("Answer:", 1)[-1].strip()
+    try:
+        clique = ast.literal_eval(cut_str)
+    except Exception:
+        return True, -1, "invalid format: cannot parse list after 'Answer:'"
+
+    if not isinstance(clique, list):
+        return True, -1, "invalid format: answer must be a list like [0, 1, 2]"
+    if len(clique) == 0:
+        return True, -1, "invalid clique: empty list"
+    # 去重检查
+    if len(set(clique)) != len(clique):
+        return True, -1, "invalid clique: duplicated vertices in the list"
+
+    # 尝试把节点转成 int（如果本来就是 str 的编号也兼容）
+    norm_clique = []
+    for x in clique:
+        try:
+            norm_clique.append(int(x))
+        except Exception:
+            return True, -1, f"invalid vertex id: {x!r} is not an integer"
+
+    # ---------- 2) 规范化图为: {int: set(int,...)} ----------
+    # 允许 graph 的 key/邻居是 str 或 int；允许邻居是 list/set/dict（值为权重）
+    neighbors = defaultdict(set)
+
+    # 收集所有节点（键 + 邻居里出现的点）
+    all_vertices = set()
+    for u_raw, adj in graph.items():
+        try:
+            u = int(u_raw)
+        except Exception:
+            # 忽略无法转为 int 的键
+            continue
+        all_vertices.add(u)
+
+        if isinstance(adj, dict):
+            # 邻接字典：取权重>0或存在即视为有边
+            for v_raw, w in adj.items():
+                try:
+                    v = int(v_raw)
+                except Exception:
+                    continue
+                all_vertices.add(v)
+                # 只要有条边（权重大于0或存在即认为有边）
+                if isinstance(w, (int, float)):
+                    if w != 0:
+                        neighbors[u].add(v)
+                        neighbors[v].add(u)
+                else:
+                    # 非数值，保守认为存在边
+                    neighbors[u].add(v)
+                    neighbors[v].add(u)
+        elif isinstance(adj, (list, set, tuple)):
+            for v_raw in adj:
+                try:
+                    v = int(v_raw)
+                except Exception:
+                    continue
+                all_vertices.add(v)
+                neighbors[u].add(v)
+                neighbors[v].add(u)
+        else:
+            # 未知结构，跳过
+            pass
+
+    # 若图没有任何边但有点，neighbors 里也应包含孤立点
+    for u in list(all_vertices):
+        neighbors[u] = set(neighbors[u])  # ensure key exists
+
+    # ---------- 3) 基本合法性检查 ----------
+    # 节点是否都存在
+    missing = [u for u in norm_clique if u not in all_vertices]
+    if missing:
+        return True, -1, f"invalid vertices (not in graph): {sorted(missing)}"
+
+    # ---------- 4) 团检查：两两必须相连 ----------
+    cset = set(norm_clique)
+    for i in range(len(norm_clique)):
+        u = norm_clique[i]
+        # 为了 O(1) 查询，使用 set
+        Nu = neighbors.get(u, set())
+        # clique 中除 u 自身外的所有顶点都必须在 Nu 中
+        if not (cset - {u}).issubset(Nu):
+            # 找出缺的边，便于提示
+            missing_neighbors = sorted((cset - {u}) - Nu)
+            return True, -1, f"invalid clique: vertex {u} is not connected to {missing_neighbors}"
+
+    # ---------- 5) 返回 ----------
+    size = len(norm_clique)
+    return False, size, f"valid clique of size {size} (note: not necessarily maximum)"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/minimum_cut.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/minimum_cut.py
new file mode 100644
index 0000000000000000000000000000000000000000..363c735054c0631aef135a4e6e4b0fbee236fa9b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/minimum_cut.py
@@ -0,0 +1,86 @@
+def validation(graph, answer):
+    """
+    Validate the given answer for the Balanced Minimum Bisection Problem.
+
+    Parameters:
+    graph: dict - adjacency list of the graph (keys and neighbor keys are strings)
+    answer: str - answer string containing the partition after 'Answer:'
+
+    Returns:
+    (bool, float, str):
+        - bool: True if invalid, False if valid
+        - float: cut weight if valid, else a large penalty value
+        - str: message explaining result
+    """
+    # Compute a large penalty value (sum of all edge weights)
+    total_weight_sum = 0
+    for u_str, neighbors in graph.items():
+        u = int(u_str)
+        for v_str, weight in neighbors.items():
+            v = int(v_str)
+            if u < v:
+                total_weight_sum += weight
+    if total_weight_sum == 0:
+        total_weight_sum = 1e9
+
+    # Extract answer content
+    if "Answer:" in answer:
+        cut_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, total_weight_sum, "invalid answer: no 'Answer:' in answer"
+
+    cut_str = cut_str.strip()
+    if not (cut_str.startswith('[') and cut_str.endswith(']')):
+        return True, total_weight_sum, "answer should be in format [[subset1], [subset2]]"
+
+    try:
+        import ast
+        subsets = ast.literal_eval(cut_str)
+        if not isinstance(subsets, list) or len(subsets) != 2:
+            return True, total_weight_sum, "answer should contain exactly two subsets"
+        subset1, subset2 = subsets
+        if not (isinstance(subset1, list) and isinstance(subset2, list)):
+            return True, total_weight_sum, "each subset should be a list of nodes"
+    except Exception:
+        return True, total_weight_sum, "invalid answer format"
+
+    # Convert to sets
+    set1, set2 = set(subset1), set(subset2)
+
+    # 1. Check disjointness
+    if set1 & set2:
+        return True, total_weight_sum, f"subsets are not disjoint: common nodes {set1 & set2}"
+
+    # 2. Check coverage
+    all_nodes = set(int(node) for node in graph.keys())
+    union = set1 | set2
+    if union != all_nodes:
+        missing = all_nodes - union
+        extra = union - all_nodes
+        errors = []
+        if missing:
+            errors.append(f"missing nodes: {missing}")
+        if extra:
+            errors.append(f"extra nodes: {extra}")
+        return True, total_weight_sum, "; ".join(errors)
+
+    # 3. Check balance constraint
+    n = len(all_nodes)
+    diff = abs(len(set1) - len(set2))
+    if not (diff == 0 or (n % 2 == 1 and diff == 1)):
+        return True, total_weight_sum, (
+            f"balance constraint violated: subset sizes {len(set1)} and {len(set2)} "
+            f"for total nodes {n}"
+        )
+
+    # 4. Compute cut weight
+    cut_weight = 0
+    for node1 in set1:
+        for node2 in set2:
+            try:
+                weight = graph[str(node1)].get(str(node2), 0)
+                cut_weight += weight
+            except KeyError:
+                return True, total_weight_sum, f"invalid node in subset: {node1} or {node2}"
+
+    return False, cut_weight, f"Cut weight: {cut_weight}"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/tsp.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/tsp.py
new file mode 100644
index 0000000000000000000000000000000000000000..8ddba3e048cb44590ea4ae59cbdf3b0d7b4b45f6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/tsp.py
@@ -0,0 +1,88 @@
+def validation(graph, answer):
+    """
+    验证给定的答案是否是一个有效的TSP路线
+
+    参数:
+    graph: json中question1字段, 表示城市之间的距离
+    answer: 字符串答案，包含"Answer："后的路径
+
+    返回:
+    (bool, float, str): 布尔值表示答案是否有效，True代表无效，False代表有效，
+                       浮点数表示路径总距离。如果答案无效，返回一个比任何可能路径都大的值。
+                       字符串提供错误或成功信息。
+    """
+    # 计算一个大于任何可能路径长度的值，作为无效答案的得分。
+    # 这里使用所有边距离之和作为惩罚值，因为任何有效路径的长度都不会超过它。
+    total_distance_sum = 0
+    cities = list(graph.keys())
+    for i in range(len(cities)):
+        for j in range(i + 1, len(cities)):
+            u, v = cities[i], cities[j]
+            total_distance_sum += graph[u].get(v, 0)
+
+    # 如果图中没有距离，使用一个默认的大值来惩罚无效格式的答案。
+    if total_distance_sum == 0:
+        total_distance_sum = 1e9
+
+    # 解析answer字符串，提取"Answer："后面的内容
+    if "Answer:" in answer:
+        path_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, total_distance_sum, "invalid answer: no 'Answer:' in answer"
+
+    # 标准化路径字符串格式
+    path_str = path_str.strip()
+    if path_str.startswith('[') and path_str.endswith(']'):
+        path_str = path_str[1:-1]
+    else:
+        if '->' in path_str:
+            nodes = [node.strip() for node in path_str.split('->')]
+            path_str = ','.join(nodes)
+        else:
+            path_str = path_str.replace("'", "").replace('"', '')
+            import re
+            pattern = r'[{\[\(]([^)}\]]*)[}\])]'
+            match = re.search(pattern, path_str)
+            if match:
+                path_str = match.group(1)
+
+    try:
+        try:
+            path = [int(x.strip()) for x in path_str.split(',')]
+        except Exception:
+            return True, total_distance_sum, "path must be a list of integers"
+        if not path:
+            return True, total_distance_sum, "path cannot be empty"
+    except Exception:
+        return True, total_distance_sum, "invalid answer format"
+
+    #  TSP specific validation
+
+    # 1. 检查起点和终点是否相同
+    if not path or path[0] != path[-1]:
+        return True, total_distance_sum, f"path is not a cycle: start {path[0]} and end {path[-1]} are different"
+
+    # 2. 检查是否访问了所有城市且只访问一次（除了起点/终点）
+    num_cities = len(graph)
+    if len(path) != num_cities + 1:
+        return True, total_distance_sum, f"path length is incorrect. Expected {num_cities + 1} cities in path, but got {len(path)}"  # noqa: E501
+
+    visited = set()
+    for city in path[:-1]:  # Exclude the last city (which is the same as the first)
+        if city in visited or city < 0 or city >= num_cities:
+            return True, total_distance_sum, f"invalid city: {city} (either repeated, negative, or out of range)"
+        visited.add(city)
+    if len(visited) != num_cities:
+        return True, total_distance_sum, f"not all cities are visited. Expected {num_cities}, but visited {len(visited)}"  # noqa: E501
+    # 3. 计算总距离
+    total_distance = 0
+    for i in range(len(path) - 1):
+        current = str(path[i])
+        next_city = str(path[i + 1])
+        try:
+            distance = graph[current][next_city]  # 直接访问距离
+            total_distance += distance
+        except KeyError:
+            return True, total_distance_sum, f"no distance found between cities {current} and {next_city}"
+
+    return False, total_distance, f"Total distance: {total_distance}"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/vertex_cover.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/vertex_cover.py
new file mode 100644
index 0000000000000000000000000000000000000000..abe9c171519fcf80f82a8f2ac17f4295b3e87927
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/NPMM/vertex_cover.py
@@ -0,0 +1,88 @@
+import re
+
+
+def validation(graph, answer):
+    """
+    验证给定的答案是否是一个有效的顶点覆盖方案
+
+    参数:
+    graph: 图的表示，包含邻接表
+    answer: 字符串答案，包含"Answer："后的顶点列表
+
+    返回:
+    (bool, int, str): 布尔值表示答案是否有效，True代表无效，False代表有效，
+                       整数表示顶点覆盖的大小（越小越好）。如果答案无效，返回一个比任何可能解都大的值。
+                       字符串提供错误或成功信息。
+    """
+    num_vertices = len(graph)
+    # 定义一个惩罚值，用于无效答案
+    penalty_value = num_vertices + 1
+
+    # 解析answer字符串，提取"Answer:"后面的内容
+    if "Answer:" in answer:
+        vertices_str = answer.split("Answer:")[-1].strip()
+    else:
+        return True, penalty_value, "invalid answer: no 'Answer:' in answer"
+
+    # 清理字符串
+    vertices_str = vertices_str.strip().replace("'", "").replace('"', '')
+
+    # 提取顶点列表
+    pattern = r'\[([^\]]*)\]'
+    match = re.search(pattern, vertices_str)
+
+    if not match:
+        return True, penalty_value, "vertices must be in list format"
+
+    vertices_content = match.group(1)
+
+    # 解析顶点
+    try:
+        if vertices_content.strip() == '':
+            # 空列表可能对于没有边的图有效
+            vertices = []
+        else:
+            vertices = [int(x.strip()) for x in vertices_content.split(',') if x.strip() != '']
+
+        # 去除重复并排序
+        vertices = sorted(list(set(vertices)))
+
+    except Exception as e:
+        return True, penalty_value, f"vertices must be integers: {str(e)}"
+
+    # 检查所有顶点是否有效
+    for v in vertices:
+        if v < 0 or v >= num_vertices:
+            return True, penalty_value, f"invalid vertex {v}: must be in range [0, {num_vertices - 1}]"
+
+    # 检查是否是有效的顶点覆盖
+    vertex_cover = set(vertices)
+
+    # 检查所有边
+    for node_str, neighbors in graph.items():
+        node = int(node_str)
+
+        for neighbor in neighbors:
+            if isinstance(neighbor, str):
+                neighbor = int(neighbor)
+
+            # 只检查每条边一次（避免同时检查 (u,v) 和 (v,u)）
+            if node < neighbor:
+                # 检查这条边是否被覆盖
+                if node not in vertex_cover and neighbor not in vertex_cover:
+                    # 边 (node, neighbor) 没有被覆盖
+                    return True, penalty_value, f"invalid vertex cover: edge ({node}, {neighbor}) is not covered"
+
+    # 有效的顶点覆盖
+    cover_size = len(vertices)
+
+    # 特殊情况：空顶点覆盖
+    if cover_size == 0:
+        # 检查图是否有边
+        has_edges = any(len(neighbors) > 0 for neighbors in graph.values())
+        if has_edges:
+            return True, penalty_value, "invalid vertex cover: empty set cannot cover edges"
+        else:
+            return False, 0, "valid vertex cover: empty set for graph with no edges"
+
+    return False, cover_size, f"valid vertex cover with {cover_size} vertices"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/IoUscore_metric.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/IoUscore_metric.py
new file mode 100644
index 0000000000000000000000000000000000000000..f106ae90f0c32d66f0bfb14f3db0577675001cee
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/IoUscore_metric.py
@@ -0,0 +1,89 @@
+# flake8: noqa
+import ast
+import os
+import re
+
+import ipdb
+
+from vlmeval.dataset.utils.Ocrbench_v2.vqa_metric import vqa_evaluation
+
+
+def calculate_iou(box1, box2):
+
+    try:
+        box1 = [int(coordinate) for coordinate in box1]
+        box2 = [int(coordinate) for coordinate in box2]
+    except:
+        return 0
+
+    x1_inter = max(box1[0], box2[0])
+    y1_inter = max(box1[1], box2[1])
+    x2_inter = min(box1[2], box2[2])
+    y2_inter = min(box1[3], box2[3])
+    inter_area = max(0, x2_inter - x1_inter) * max(0, y2_inter - y1_inter)
+    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
+    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])
+    union_area = box1_area + box2_area - inter_area
+    iou = inter_area / union_area if union_area != 0 else 0
+    return iou
+
+
+def vqa_with_position_evaluation(predict, img_metas):
+
+    score_content, score_bbox = .0, .0
+    if "answer" in predict.keys():
+        score_content = vqa_evaluation(predict["answer"], img_metas["answers"])
+    if "bbox" in predict.keys():
+        gt_bbox = img_metas["bbox"]
+        try:
+            predict_bbox_list = ast.literal_eval(predict["bbox"])
+            score_bbox = calculate_iou(predict_bbox_list, gt_bbox)
+        except:
+            score_bbox = 0
+    return 0.5 * score_content + 0.5 * score_bbox
+
+
+def extract_coordinates(text):
+    # Regex pattern to match coordinates in either (x1, y1, x2, y2) or [x1, y1, x2, y2] format
+
+    pattern = r'[\(\[]\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*[\)\]]'
+
+    matches = list(re.finditer(pattern, text))
+    coords_list = []
+    coords_set = set()
+    for match in matches:
+
+        x1, y1, x2, y2 = map(int, match.groups())
+
+        if all(0 <= n <= 1000 for n in [x1, y1, x2, y2]):
+            coords = (x1, y1, x2, y2)
+
+            if coords in coords_set:
+                coords_list = [c for c in coords_list if c != coords]
+
+            coords_list.append(coords)
+            coords_set.add(coords)
+    if coords_list:
+        last_coords = coords_list[-1]
+        return list(last_coords)
+    else:
+        return None
+
+
+if __name__ == "__main__":
+
+    print("Example for Text Grounding task.")
+    box1 = [50, 50, 150, 150]
+    box2 = [60, 60, 140, 140]
+    iou_score = calculate_iou(box1, box2)
+    print(f"IoU score: {iou_score}")
+
+    print("Example for VQA with position task.")
+    pred = {"content": "The content is Hello Buddies", "bbox": box1}
+    gt = {"content": "Hello Buddies", "bbox": box2}
+
+    vqa_score = vqa_evaluation(pred["content"], gt["content"])
+    iou_score = calculate_iou(pred["bbox"], gt["bbox"])
+
+    print(f"VQA score: {vqa_score}")
+    print(f"IoU score: {iou_score}")
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py
new file mode 100644
index 0000000000000000000000000000000000000000..d46b5028d935d04f58da3c50435ea912179feb71
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py
@@ -0,0 +1,932 @@
+# flake8: noqa
+# Copyright 2020 IBM
+# Author: peter.zhong@au1.ibm.com
+#
+# This is free software; you can redistribute it and/or modify
+# it under the terms of the Apache 2.0 License.
+#
+# This software is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# Apache 2.0 License for more details.
+
+import ast
+import json
+import re
+import string
+from collections import deque
+from itertools import product
+from typing import Any, Callable, Optional, Sequence
+
+import distance
+import editdistance
+import ipdb
+import Levenshtein
+import numpy as np
+from apted import APTED, Config
+from apted.helpers import Tree
+from lxml import etree, html
+from tqdm import tqdm
+from zss import Node, simple_distance
+
+from vlmeval.dataset.utils.Ocrbench_v2.parallel import parallel_process
+
+
+class TableTree(Tree):
+    def __init__(self, tag, colspan=None, rowspan=None, content=None, *children):
+        self.tag = tag
+        self.colspan = colspan
+        self.rowspan = rowspan
+        self.content = content
+        self.children = list(children)
+
+    def bracket(self):
+        """Show tree using brackets notation"""
+        if self.tag == 'td':
+            result = '"tag": %s, "colspan": %d, "rowspan": %d, "text": %s' % \
+                     (self.tag, self.colspan, self.rowspan, self.content)
+        else:
+            result = '"tag": %s' % self.tag
+        for child in self.children:
+            result += child.bracket()
+        return "{{{}}}".format(result)
+
+
+class CustomConfig(Config):
+    @staticmethod
+    def maximum(*sequences):
+        """Get maximum possible value
+        """
+        return max(map(len, sequences))
+
+    def normalized_distance(self, *sequences):
+        """Get distance from 0 to 1
+        """
+        return float(distance.levenshtein(*sequences)) / self.maximum(*sequences)
+
+    def rename(self, node1, node2):
+        """Compares attributes of trees"""
+        if (node1.tag != node2.tag) or (node1.colspan != node2.colspan) or (node1.rowspan != node2.rowspan):
+            return 1.
+        if node1.tag == 'td':
+            if node1.content or node2.content:
+                return self.normalized_distance(node1.content, node2.content)
+        return 0.
+
+
+class TEDS(object):
+    ''' Tree Edit Distance basead Similarity
+    '''
+    def __init__(self, structure_only=False, n_jobs=1, ignore_nodes=None):
+        assert isinstance(n_jobs, int) and (n_jobs >= 1), 'n_jobs must be an integer greather than 1'
+        self.structure_only = structure_only
+        self.n_jobs = n_jobs
+        self.ignore_nodes = ignore_nodes
+        self.__tokens__ = []
+
+    def tokenize(self, node):
+        ''' Tokenizes table cells
+        '''
+        self.__tokens__.append('<%s>' % node.tag)
+        if node.text is not None:
+            self.__tokens__ += list(node.text)
+        for n in node.getchildren():
+            self.tokenize(n)
+        if node.tag != 'unk':
+            self.__tokens__.append('</%s>' % node.tag)
+        if node.tag != 'td' and node.tail is not None:
+            self.__tokens__ += list(node.tail)
+
+    def load_html_tree(self, node, parent=None):
+        ''' Converts HTML tree to the format required by apted
+        '''
+        global __tokens__
+        if node.tag == 'td':
+            if self.structure_only:
+                cell = []
+            else:
+                self.__tokens__ = []
+                self.tokenize(node)
+                cell = self.__tokens__[1:-1].copy()
+            new_node = TableTree(node.tag,
+                                 int(node.attrib.get('colspan', '1')),
+                                 int(node.attrib.get('rowspan', '1')),
+                                 cell, *deque())
+        else:
+            new_node = TableTree(node.tag, None, None, None, *deque())
+        if parent is not None:
+            parent.children.append(new_node)
+        if node.tag != 'td':
+            for n in node.getchildren():
+                self.load_html_tree(n, new_node)
+        if parent is None:
+            return new_node
+
+    def evaluate(self, pred, true):
+        ''' Computes TEDS score between the prediction and the ground truth of a
+            given sample
+        '''
+        if (not pred) or (not true):
+            return 0.0
+        parser = html.HTMLParser(remove_comments=True, encoding='utf-8')
+        pred = html.fromstring(pred, parser=parser)
+        true = html.fromstring(true, parser=parser)
+        if pred.xpath('body/table') and true.xpath('body/table'):
+            pred = pred.xpath('body/table')[0]
+            true = true.xpath('body/table')[0]
+            if self.ignore_nodes:
+                etree.strip_tags(pred, *self.ignore_nodes)
+                etree.strip_tags(true, *self.ignore_nodes)
+            n_nodes_pred = len(pred.xpath(".//*"))
+            n_nodes_true = len(true.xpath(".//*"))
+            n_nodes = max(n_nodes_pred, n_nodes_true)
+            tree_pred = self.load_html_tree(pred)
+            tree_true = self.load_html_tree(true)
+            distance = APTED(tree_pred, tree_true, CustomConfig()).compute_edit_distance()
+            return 1.0 - (float(distance) / n_nodes)
+        else:
+            return 0.0
+
+    def batch_evaluate(self, pred_json, true_json):
+        ''' Computes TEDS score between the prediction and the ground truth of
+            a batch of samples
+            @params pred_json: {'FILENAME': 'HTML CODE', ...}
+            @params true_json: {'FILENAME': {'html': 'HTML CODE'}, ...}
+            @output: {'FILENAME': 'TEDS SCORE', ...}
+        '''
+        samples = true_json.keys()
+        if self.n_jobs == 1:
+            scores = [self.evaluate(pred_json.get(filename, ''), true_json[filename]['html']) for filename in tqdm(samples)]
+        else:
+            #inputs = [{'pred': pred_json.get(filename, ''), 'true': true_json[filename]['html']} for filename in samples]
+            inputs = [{'pred': pred_json.get(filename, ''), 'true': true_json[filename]} for filename in samples]
+            scores = parallel_process(inputs, self.evaluate, use_kwargs=True, n_jobs=self.n_jobs, front_num=1)
+        scores = dict(zip(samples, scores))
+        return scores
+
+
+def convert_table_to_html_str(table_row_list=[]):
+    """
+    Given a list of table rows, build the corresponding html string, which is used to compute the TEDS score.
+    We use the official code of PubTabNet to compute TEDS score, it does not consider '<th>' label.
+    We also remove unneccessary spaces within a table cell and extra '\n' as they will influence the TEDS score.
+    """
+    html_table_str = "<html><body><table>" + '\n'
+    for data_row in table_row_list:
+        html_table_str += "<tr>"
+        for cell_str in data_row:
+            html_table_str += f"<td>{cell_str}</td>"
+        html_table_str += "</tr>"
+        html_table_str += '\n'
+    html_table_str += "</table></body></html>"
+    html_table_str = html_table_str.replace('\n','')
+    return html_table_str
+
+
+def convert_markdown_table_to_html(markdown_table):
+    """
+    Converts a markdown table to the corresponding html string for TEDS computation.
+    """
+    # remove extra code block tokens like '```markdown' and '```
+    markdown_table = markdown_table.strip('```markdown').strip('```').strip()
+    row_str_list = markdown_table.split('\n')
+    # extra the first header row and other data rows
+    valid_row_str_list = [row_str_list[0]]+row_str_list[2:]
+    table_rows = []
+    for row_str in valid_row_str_list:
+        one_row = []
+        for cell in row_str.strip().split('|')[1:-1]:
+            if set(cell) != set(' '):
+                one_row.append(cell.strip())
+            else:
+                one_row.append(' ')
+        table_rows.append(one_row)
+    # build html string based on table rows
+    html_str = convert_table_to_html_str(table_rows)
+    return html_str
+
+
+def dict_to_html(data):
+    html = "<html><body><table>\n"
+    for key, value in data.items():
+        if not isinstance(value, str):
+            value = str(value)
+        value_str = ' '.join(value)
+
+        html += f"  <tr><td>{key}</td><td>{value_str}</td></tr>\n"
+    html += "</table></body></html>"
+    return html
+
+
+def convert_str_to_dict(predict_str: str):
+    """
+    Parses the 'predict' string and returns a dictionary.
+    Missing or unparseable content is handled gracefully.
+
+    Parameters:
+    - predict_str (str): The prediction string containing the output dict.
+
+    Returns:
+    - dict: A dictionary extracted from the predict string.
+    """
+    # Remove code fences like ```python\n...\n```
+    code_fence_pattern = r'```(?:python|json)?\n(.*?)\n```'
+    match = re.search(code_fence_pattern, predict_str, re.DOTALL | re.IGNORECASE)
+    if match:
+        content = match.group(1)
+    else:
+        content = predict_str.strip()
+
+    data = {}
+    success = False
+
+    # try parsing with JSON
+    try:
+        data = json.loads(content)
+        success = True
+    except json.JSONDecodeError:
+        pass
+
+    # try parsing with ast.literal_eval
+    if not success:
+        try:
+            data = ast.literal_eval(content)
+            if isinstance(data, dict):
+                success = True
+        except (ValueError, SyntaxError):
+            pass
+
+    # try parsing with regex
+    if not success:
+        key_value_pattern = r'["\']?([\w\s]+)["\']?\s*[:=]\s*["\']?([^\n,"\'{}]+)["\']?'
+        matches = re.findall(key_value_pattern, content)
+        try:
+            for key, value in matches:
+                data[key.strip()] = value.strip()
+        except:
+            return {}
+
+    if not data:
+        return {}
+
+    try:
+        result = {k.strip(): str(v).strip() for k, v in data.items()}
+    except:
+        return {}
+    return result
+
+
+def convert_str_to_multi_dict(predict_str: str):
+    """
+    Parses the 'predict' string and returns a dictionary.
+    Handles nested dictionaries and missing or unparseable content gracefully.
+
+    Parameters:
+    - predict_str (str): The prediction string containing the output dict.
+
+    Returns:
+    - dict: A dictionary extracted from the predict string.
+    """
+    # Remove code fences like ```python\n...\n```
+    code_fence_pattern = r'```(?:python|json)?\n(.*?)\n```'
+    matches = re.findall(code_fence_pattern, predict_str, re.DOTALL | re.IGNORECASE)
+    if matches:
+        content = max(matches, key=len)
+    else:
+        content = predict_str.strip()
+
+    def strip_variable_assignment(s):
+        variable_assignment_pattern = r'^\s*\w+\s*=\s*'
+        return re.sub(variable_assignment_pattern, '', s.strip(), count=1)
+
+    content = strip_variable_assignment(content)
+
+    def remove_comments(s):
+        return re.sub(r'#.*', '', s)
+
+    content = remove_comments(content)
+
+    last_brace_pos = content.rfind('}')
+    if last_brace_pos != -1:
+        content = content[:last_brace_pos+1]
+
+    data = {}
+    success = False
+
+    # try parsing with ast.literal_eval
+    try:
+        data = ast.literal_eval(content)
+        if isinstance(data, dict):
+            success = True
+    except (ValueError, SyntaxError, TypeError):
+        pass
+
+    if not success:
+        return {}
+
+    def process_data(obj):
+        if isinstance(obj, dict):
+            return {k: process_data(v) for k, v in obj.items()}
+        elif isinstance(obj, list):
+            return [process_data(elem) for elem in obj]
+        else:
+            return obj
+
+    data = process_data(data)
+
+    return data
+
+
+def generate_combinations(input_dict):
+    """
+    Function to generate all possible combinations of values from a dictionary.
+    """
+    kie_answer = input_dict
+    if not isinstance(kie_answer, dict):
+        kie_answer = kie_answer.strip('"')
+        try:
+            kie_answer = json.loads(kie_answer)
+        except json.JSONDecodeError:
+            try:
+                kie_answer = ast.literal_eval(kie_answer)
+                if not isinstance(kie_answer, dict):
+                    kie_answer = ast.literal_eval(kie_answer)
+            except (ValueError, SyntaxError):
+                print(f"Unable to parse 'answers' field: {kie_answer}")
+                return {}
+
+        # Ensure the parsed result is a dictionary.
+        if not isinstance(kie_answer, dict):
+            print("Parsed 'answers' is still not a dictionary.")
+            raise ValueError("Input could not be parsed into a dictionary.")
+
+        keys = list(kie_answer.keys())
+
+        value_lists = []
+        for single_key in keys:
+            sinlge_value = kie_answer[single_key]
+            if not isinstance(sinlge_value, list):
+                sinlge_value = [sinlge_value]
+            value_lists.append(sinlge_value)
+
+        # Compute the Cartesian product of the value lists.
+        combinations = list(product(*value_lists))
+
+        # Create a dictionary for each combination of values.
+        result = [dict(zip(keys, values)) for values in combinations]
+
+        return result
+
+    else:
+        keys = list(input_dict.keys())
+        value_lists = [input_dict[key] for key in keys]
+
+        # Compute the Cartesian product of the value lists.
+        combinations = list(product(*value_lists))
+
+        # Create a dictionary for each combination of values.
+        result = [dict(zip(keys, values)) for values in combinations]
+
+        return result
+
+
+def compute_f1_score(preds, gts, ignores=[]):
+    """Compute the F1-score for KIE task between predicted and ground truth dictionaries.
+
+    Args:
+        preds (dict): The predicted key-value pairs.
+        gts (dict): The ground truth key-value pairs.
+        ignores (list): The list of keys to ignore during evaluation.
+
+    Returns:
+        dict: A dictionary where keys are field names and values are their corresponding F1-scores.
+    """
+    # Optionally remove ignored keys from predictions and ground truths
+    keys = set(preds.keys()).union(set(gts.keys())) - set(ignores)
+    f1_scores = {}
+
+    for key in keys:
+        pred_value = preds.get(key, None)
+        gt_value = gts.get(key, None)
+
+        if pred_value:
+            pred_value = pred_value.lower().strip().replace("\n"," ").replace(" ", "")
+        if gt_value:
+            gt_value = gt_value.lower().strip().replace("\n"," ").replace(" ", "")
+
+        if pred_value is None and gt_value is None:
+            continue
+        elif pred_value is None:
+            precision = 0.0
+            recall = 0.0
+        elif gt_value is None:
+            # false positive
+            precision = 0.0
+            recall = 0.0
+        else:
+            if pred_value == gt_value:
+                # True positive
+                precision = 1.0
+                recall = 1.0
+            else:
+                precision = 0.0
+                recall = 0.0
+
+        # Compute F1-score
+        f1_score = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
+        f1_scores[key] = f1_score
+
+    if len(f1_scores) == 0:
+        return 0
+    average_f1 = sum(f1_scores.values()) / len(f1_scores)
+
+    return average_f1
+
+
+def pre_clean(text):
+    text = re.sub(r'<bos>|<eos>|<pad>|<unk>', '', text)
+    text = re.sub(r'\s##(\S)', r'\1', text)
+    text = re.sub(r'\\\s', r'\\', text)
+    text = re.sub(r'\s\*\s\*\s', r'**', text)
+    text = re.sub(r'{\s', r'{', text)
+    text = re.sub(r'\s}', r'}', text)
+    text = re.sub(r'\s}', r'}', text)
+    text = re.sub(r'\\begin\s', r'\\begin', text)
+    text = re.sub(r'\\end\s', r'\\end', text)
+    text = re.sub(r'\\end{table}', r'\\end{table} \n\n', text)
+    text = text.replace('\n', ' ')
+    text = text.replace('*', ' ')
+    text = text.replace('_', ' ')
+    return text
+
+
+def get_tree(input_str):
+    tree = (Node('ROOT').addkid(Node('TITLE')))
+
+    lines = input_str.split("\n")
+    lines = [pre_clean(line) for line in lines]
+    last_title = ''
+    for line in lines:
+        if line.startswith('#'):
+            child = tree.get('ROOT')
+            line = line.replace('#', '')
+            child.addkid(Node(line))
+            last_title = line
+        else:
+            if last_title == '':
+                child = tree.get('TITLE')
+                child.addkid(Node(line))
+            else:
+                child = tree.get(last_title)
+                child.addkid(Node(line))
+    return tree
+
+def STEDS(pred_tree, ref_tree):
+    def my_distance(pred, ref):
+        if len(pred.split()) == 0 or len(ref.split()) == 0:
+            return 1
+        else:
+            return 0
+    total_distance = simple_distance(pred_tree, ref_tree, label_dist=my_distance)
+    num_of_nodes = max(len(list(pred_tree.iter())), len(list(ref_tree.iter())))
+    return 1-total_distance/num_of_nodes
+
+
+def doc_parsing_evaluation(pred, gt):
+    score = 0
+    if not isinstance(pred, str):
+        return 0
+    pred_tree = get_tree(pred)
+    gt_tree = get_tree(gt)
+    score = STEDS(pred_tree, gt_tree)
+
+    return score
+
+
+def wrap_html_table(html_table):
+    """
+    The TEDS computation from PubTabNet code requires that the input html table should have <html>, <body>, and <table> tags.
+    Add them if they are missing.
+    """
+    html_table = html_table.replace('\n','')
+    # add missing <table> tag if missing
+    if "<table" in html_table and "</table>" not in html_table:
+        html_table = html_table + "</table>"
+    elif "<table" not in html_table and "</table>" in html_table:
+        html_table = "<table>" + html_table
+    elif "<table" not in html_table and "</table>" not in html_table:
+        html_table = "<table>" + html_table + "</table>"
+    else:
+        pass
+    # add <body> and <html> tags if missing
+    if '<body>' not in html_table:
+        html_table = '<body>' + html_table + '</body>'
+    if '<html>' not in html_table:
+        html_table = '<html>' + html_table + '</html>'
+    return html_table
+
+
+def get_anls(s1, s2):
+    try:
+        s1 = s1.lower()
+        s2 = s2.lower()
+    except:
+        pass
+    if s1 == s2:
+        return 1.0
+    iou = 1 - editdistance.eval(s1, s2) / max(len(s1), len(s2))
+    anls = iou
+    return anls
+
+
+def ocr_eval(references,predictions):
+    socre_=0.0
+    None_num=0
+    for idx,ref_value in enumerate(references):
+        pred_value = predictions[idx]
+        pred_values, ref_values = [], []
+        if isinstance(pred_value, str):
+            pred_values.append(pred_value)
+        else:
+            pred_values = pred_value
+        if isinstance(ref_value, str):
+            ref_values.append(ref_value)
+        else:
+            ref_values = ref_value
+
+        temp_score = 0.0
+        temp_num = len(ref_values)
+
+        for tmpidx, tmpref in enumerate(ref_values):
+            tmppred = pred_values[tmpidx] if tmpidx < len(pred_values) else pred_values[0]
+            if len(pred_values) == 1 and tmppred != "None" and "None" not in ref_values:  # pred 1, and not None
+                temp_score = max(temp_score, get_anls(tmppred, tmpref))
+                temp_num = len(ref_values)
+            else:
+                if tmppred=='None' and tmpref!='None':
+                    temp_score += 0.0
+                elif tmpref=='None':
+                    temp_num -= 1
+                else:
+                    temp_score += get_anls(tmppred, tmpref)
+        if temp_num == 0:
+            ocr_score = 0.0
+            None_num += 1
+        else:
+            ocr_score = temp_score / (temp_num)
+        socre_ += ocr_score
+    if None_num == len(references):
+        return 9999
+    else:
+        return round(socre_ / (len(references)-None_num), 5)
+
+
+def csv_eval(predictions,references,easy, pred_type='json'):
+    predictions = predictions
+    labels = references
+    def is_int(val):
+        try:
+            int(val)
+            return True
+        except ValueError:
+            return False
+
+    def is_float(val):
+        try:
+            float(val)
+            return True
+        except ValueError:
+            return False
+
+    def convert_dict_to_list(data):
+        """
+        Convert a dictionary to a list of tuples, handling both simple and nested dictionaries.
+
+        Args:
+        data (dict): The input dictionary, which might be nested or simple.
+
+        Returns:
+        list: A list of tuples generated from the input dictionary.
+        """
+        # print(data)
+        converted_list = []
+        for key, value in data.items():
+            # Check if the value is a dictionary (indicating a nested structure)
+            if isinstance(value, dict):
+                # Handle nested dictionary
+                for subkey, subvalue in value.items():
+                    # converted_list.append((key, subkey, subvalue))
+                    converted_list.append((key, subkey, re.sub(r'[^\d.-]', '', str(subvalue))))
+
+            else:
+                # Handle simple key-value pair
+                # converted_list.append((key, "value", value))
+                converted_list.append((key, "value", re.sub(r'[^\d.-]', '', str(value))))
+        return converted_list
+
+
+    def csv2triples(csv, separator='\\t', delimiter='\\n'):
+        lines = csv.strip().split(delimiter)
+        header = lines[0].split(separator)
+        triples = []
+        for line in lines[1:]:
+            if not line:
+                continue
+            values = line.split(separator)
+            entity = values[0]
+            for i in range(1, len(values)):
+                if i >= len(header):
+                    break
+                #---------------------------------------------------------
+                temp = [entity.strip(), header[i].strip()]
+                temp = [x if len(x)==0 or x[-1] != ':' else x[:-1] for x in temp]
+                value = values[i].strip()
+                value = re.sub(r'[^\d.-]', '', str(value))
+                # value = value.replace("%","")
+                # value = value.replace("$","")
+                triples.append((temp[0], temp[1], value))
+                #---------------------------------------------------------
+        return triples
+
+    def csv2triples_noheader(csv, separator='\\t', delimiter='\\n'):
+        lines = csv.strip().split(delimiter)
+        maybe_header = [x.strip() for x in lines[0].split(separator)]
+        not_header = False
+        if len(maybe_header) > 2:
+            for c in maybe_header[1:]:
+                try:
+                    num = float(c)
+                    not_header = True
+                except:
+                    continue
+                if not_header:
+                    break
+        header = None if not_header else maybe_header
+        data_start = 0 if not_header and separator in lines[0] else 1
+        triples = []
+        for line in lines[data_start:]:
+            if not line:
+                continue
+            values = [x.strip() for x in line.split(separator)]
+            entity = values[0]
+            for i in range(1, len(values)):
+                try:
+                    temp = [entity if entity[-1]!=':' else entity[:-1], ""]
+                except:
+                    temp = [entity, ""]
+                if header is not None:
+                    try:
+                        this_header = header[i]
+                        temp = [entity, this_header]
+                        temp = [x if x[-1] != ':' else x[:-1] for x in temp]
+                    except:
+                        this_header = entity.strip()
+                value = values[i].strip()
+                value = re.sub(r'[^\d.-]', '', str(value))
+                # value = value.replace("%","")
+                # value = value.replace("$","")
+                triples.append((temp[0], temp[1], value))
+                #---------------------------------------------------------
+        return triples
+
+    def process_triplets(triplets):
+        new_triplets = []
+        for triplet in triplets:
+            new_triplet = []
+            triplet_temp = []
+            if len(triplet) > 2:
+                if is_int(triplet[2]) or is_float(triplet[2]):
+                    triplet_temp = (triplet[0].lower(), triplet[1].lower(), float(triplet[2]))
+                else:
+                    triplet_temp = (triplet[0].lower(), triplet[1].lower(), triplet[2].lower())
+            else:
+                triplet_temp = (triplet[0].lower(), triplet[1].lower(), "no meaning")
+            new_triplets.append(triplet_temp)
+        return new_triplets
+
+    def intersection_with_tolerance(a, b, tol_word, tol_num):
+        a = set(a)
+        b = set(b)
+        c = set()
+        for elem1 in a:
+            for elem2 in b:
+                if is_float(elem1[-1]) and is_float(elem2[-1]):
+                    if ((Levenshtein.distance(''.join(elem1[:-1]),''.join(elem2[:-1])) <= tol_word) and (abs(elem1[-1] - elem2[-1]) / (abs(elem2[-1])+0.000001) <= tol_num))or \
+                    ((''.join(elem1[:-1]) in ''.join(elem2[:-1])) and (abs(elem1[-1] - elem2[-1]) / (abs(elem2[-1])+0.000001) <= tol_num)) or \
+                    ((''.join(elem2[:-1]) in ''.join(elem1[:-1])) and (abs(elem1[-1] - elem2[-1]) / (abs(elem2[-1])+0.000001) <= tol_num)):
+                        c.add(elem1)
+                else:
+                    if (Levenshtein.distance(''.join([str(i) for i in elem1]),''.join([str(j) for j in elem2])) <= tol_word):
+                        c.add(elem1)
+        return list(c)
+
+    def union_with_tolerance(a, b, tol_word, tol_num):
+        c = set(a) | set(b)
+        d = set(a) & set(b)
+        e = intersection_with_tolerance(a, b, tol_word, tol_num)
+        f = set(e)
+        g = c-(f-d)
+        return list(g)
+
+    def get_eval_list(pred_csv, label_csv, separator='\\t', delimiter='\\n', tol_word=3, tol_num=0.05, pred_type='json'):
+
+        if pred_type == 'json':
+            pred_triple_list=[]
+            for it in pred_csv:
+                pred_triple_temp = convert_dict_to_list(it)
+                pred_triple_pre = process_triplets(pred_triple_temp)
+                pred_triple_list.append(pred_triple_pre)
+        else:
+            pred_triple_list=[]
+            for it in pred_csv:
+                pred_triple_temp = csv2triples(it, separator=separator, delimiter=delimiter)
+                # pred_triple_temp = csv2triples_noheader(it, separator=separator, delimiter=delimiter)
+                pred_triple_pre = process_triplets(pred_triple_temp)
+                pred_triple_list.append(pred_triple_pre)
+
+        label_triple_list=[]
+        for it in label_csv:
+            label_triple_temp = convert_dict_to_list(it)
+            label_triple_pre = process_triplets(label_triple_temp)
+            label_triple_list.append(label_triple_pre)
+
+
+        intersection_list=[]
+        union_list=[]
+        sim_list=[]
+        # for each chart image
+        for pred,label in zip(pred_triple_list, label_triple_list):
+            for idx in range(len(pred)):
+                try:
+                    if label[idx][1] == "value" and "value" not in pred[idx][:2]:
+                        pred[idx] = (pred[idx][0], "value", pred[idx][2])
+                    temp_pred_head = sorted(pred[idx][:2])
+                    temp_gt_head = sorted(label[idx][:2])
+                    pred[idx] = (temp_pred_head[0], temp_pred_head[1], pred[idx][2])
+                    label[idx] = (temp_gt_head[0], temp_gt_head[1], label[idx][2])
+                except:
+                    continue
+            intersection = intersection_with_tolerance(pred, label, tol_word = tol_word, tol_num=tol_num)
+            union = union_with_tolerance(pred, label, tol_word = tol_word, tol_num=tol_num)
+            sim = len(intersection)/len(union)
+            intersection_list.append(intersection)
+            union_list.append(union)
+            sim_list.append(sim)
+        return intersection_list, union_list, sim_list
+
+    def get_ap(predictions, labels, sim_threhold, tolerance, separator='\\t', delimiter='\\n', easy=1):
+        if tolerance == 'strict':
+            tol_word=0
+            if easy == 1:
+                tol_num=0
+            else:
+                tol_num=0.1
+
+        elif tolerance == 'slight':
+            tol_word=2
+            if easy == 1:
+                tol_num=0.05
+            else:
+                tol_num=0.3
+
+        elif tolerance == 'high':
+            tol_word= 5
+            if easy == 1:
+                tol_num=0.1
+            else:
+                tol_num=0.5
+        intersection_list, union_list, sim_list = get_eval_list(predictions, labels, separator=separator, delimiter=delimiter, tol_word=tol_word, tol_num=tol_num, pred_type=pred_type)
+        ap = len([num for num in sim_list if num >= sim_threhold])/(len(sim_list)+1e-16)
+        return ap
+
+    map_strict = 0
+    map_slight = 0
+    map_high = 0
+    s="\\t"
+    d="\\n"
+
+    for sim_threhold in np.arange (0.5, 1, 0.05):
+        map_temp_strict = get_ap(predictions, labels, sim_threhold=sim_threhold, tolerance='strict', separator=s, delimiter=d, easy=easy)
+        map_temp_slight = get_ap(predictions, labels, sim_threhold=sim_threhold, tolerance='slight', separator=s, delimiter=d, easy=easy)
+        map_temp_high = get_ap(predictions, labels, sim_threhold=sim_threhold, tolerance='high', separator=s, delimiter=d, easy=easy)
+        map_strict += map_temp_strict/10
+        map_slight += map_temp_slight/10
+        map_high += map_temp_high/10
+
+    em = get_ap(predictions, labels, sim_threhold=1, tolerance='strict', separator=s, delimiter=d, easy=easy)
+    ap_50_strict = get_ap(predictions, labels, sim_threhold=0.5, tolerance='strict', separator=s, delimiter=d, easy=easy)
+    ap_75_strict = get_ap(predictions, labels, sim_threhold=0.75, tolerance='strict', separator=s, delimiter=d, easy=easy)
+    ap_90_strict = get_ap(predictions, labels, sim_threhold=0.90, tolerance='strict', separator=s, delimiter=d, easy=easy)
+    ap_50_slight = get_ap(predictions, labels, sim_threhold=0.5, tolerance='slight', separator=s, delimiter=d, easy=easy)
+    ap_75_slight = get_ap(predictions, labels, sim_threhold=0.75, tolerance='slight', separator=s, delimiter=d, easy=easy)
+    ap_90_slight = get_ap(predictions, labels, sim_threhold=0.90, tolerance='slight', separator=s, delimiter=d, easy=easy)
+    ap_50_high = get_ap(predictions, labels, sim_threhold=0.5, tolerance='high', separator=s, delimiter=d, easy=easy)
+    ap_75_high = get_ap(predictions, labels, sim_threhold=0.75, tolerance='high', separator=s, delimiter=d, easy=easy)
+    ap_90_high = get_ap(predictions, labels, sim_threhold=0.90, tolerance='high', separator=s, delimiter=d, easy=easy)
+
+
+    return em, map_strict, map_slight, map_high, ap_50_strict, ap_75_strict, ap_90_strict, ap_50_slight, ap_75_slight, ap_90_slight, ap_50_high, ap_75_high, ap_90_high
+
+def draw_SCRM_table(em, map_strict, map_slight, map_high, ap_50_strict, ap_75_strict, ap_90_strict, ap_50_slight, ap_75_slight, ap_90_slight, ap_50_high, ap_75_high, ap_90_high,title_ocr_socre,source_ocr_socre,x_title_ocr_socre,y_title_ocr_socre,structure_accuracy):
+
+    result=f'''
+            -----------------------------------------------------------\n
+            |  Metrics   |  Sim_threshold  |  Tolerance  |    Value    |\n
+            -----------------------------------------------------------\n
+            |             |                 |   strict    |    {'%.4f' % map_strict}    |     \n
+            |             |                 ----------------------------\n
+            |  mPrecison  |  0.5:0.05:0.95  |   slight    |    {'%.4f' % map_slight}    |\n
+            |             |                  ---------------------------\n
+            |             |                 |    high     |    {'%.4f' % map_high}    |\n
+            -----------------------------------------------------------\n
+            |             |                 |   strict    |    {'%.4f' % ap_50_strict}    |\n
+            |             |                  ---------------------------\n
+            |  Precison   |       0.5       |   slight    |    {'%.4f' % ap_50_slight }    |\n
+            |             |                  ---------------------------\n
+            |             |                 |    high     |    {'%.4f' % ap_50_high }    |\n
+            -----------------------------------------------------------\n
+            |             |                 |   strict    |    {'%.4f' % ap_75_strict}    |\n
+            |             |                  ---------------------------\n
+            |  Precison   |      0.75       |   slight    |    {'%.4f' % ap_75_slight}    |\n
+            |             |                  ---------------------------\n
+            |             |                 |    high     |    {'%.4f' % ap_75_high}    |\n
+            -----------------------------------------------------------\n
+            |             |                 |   strict    |    {'%.4f' % ap_90_strict}    |\n
+            |             |                  ---------------------------\n
+            |  Precison   |       0.9       |   slight    |    {'%.4f' % ap_90_slight }    |\n
+            |             |                  ---------------------------\n
+            |             |                 |    high     |    {'%.4f' % ap_90_high}    |\n
+            -----------------------------------------------------------\n
+            |Precison(EM) |                                    {'%.4f' % em}    |\n
+            -----------------------------------------------------------\n
+            |Title(EM)    |                                    {'%.4f' % title_ocr_socre}    |\n
+            -----------------------------------------------------------\n
+            |Source(EM)   |                                    {'%.4f' % source_ocr_socre}    |\n
+            -----------------------------------------------------------\n
+            |X_title(EM)  |                                    {'%.4f' % x_title_ocr_socre}    |\n
+            -----------------------------------------------------------\n
+            |Y_title(EM)  |                                    {'%.4f' % y_title_ocr_socre}    |\n
+            -----------------------------------------------------------\n
+            |structure_acc|                                    {'%.4f' % structure_accuracy}    |\n
+            -----------------------------------------------------------\n
+
+
+            '''
+    return result
+
+
+if __name__ == '__main__':
+    import json
+    import pprint
+
+    # markdown structure for Table Parsing task
+    pred_markdown = "| 1 | august 5 , 1972 | detroit lions | l 23 - 31 | 0 - 1 |\n| 2 | august 12 , 1972 | green bay packers | l 13 - 14 | 0 - 2 |\n| 3 | august 19 , 1972 | cincinnati bengals | w 35 - 17 | 1 - 2 |\n| 4 | august 25 , 1972 | atlanta falcons | w 24 - 10 | 2 - 2 |\n| 5 | august 31 , 1972 | washington redskins | l 24 - 27 | 2 - 3 |\n| 6 | september 10 , 1972 | minnesota vikings | w 21 - 19 | 3 - 3 |"
+    true_markdown = "| week | date | opponent | result | record |\n| --- | --- | --- | --- | --- |\n| 1 | august 5 , 1972 | detroit lions | l 23 - 31 | 0 - 1 |\n| 2 | august 12 , 1972 | green bay packers | l 13 - 14 | 0 - 2 |\n| 3 | august 19 , 1972 | cincinnati bengals | w 35 - 17 | 1 - 2 |\n| 4 | august 25 , 1972 | atlanta falcons | w 24 - 10 | 2 - 2 |\n| 5 | august 31 , 1972 | washington redskins | l 24 - 27 | 2 - 3 |\n| 6 | september 10 , 1972 | minnesota vikings | w 21 - 19 | 3 - 3 |"
+    teds = TEDS(n_jobs=4)
+    pred_table_html = convert_markdown_table_to_html(pred_markdown)
+    true_table_html = convert_markdown_table_to_html(true_markdown)
+
+    scores = teds.evaluate(pred_table_html, true_table_html)
+
+    pp = pprint.PrettyPrinter()
+    pp.pprint(scores)
+
+    # dict structure for Key Information Extraction task
+    pred_dict = {
+            "company": [
+                "OLD TOWN "
+            ],
+            "date": [
+                "2024"
+            ],
+            "address": [
+                "SRI RAMPAI"
+            ],
+            "total": [
+                "30"
+            ]
+        }
+    true_dict = {
+            "company": [
+                "OLD TOWN KOPITAM SND BHD"
+            ],
+            "date": [
+                "2024/9/27"
+            ],
+            "address": [
+                "SRI RAMPAI"
+            ],
+            "total": [
+                "30"
+            ]
+        }
+    teds = TEDS(n_jobs=4)
+    pred_dict_html = dict_to_html(pred_dict)
+    true_dict_html = dict_to_html(true_dict)
+    print(pred_dict_html)
+    print(true_dict_html)
+
+    scores = teds.evaluate(pred_dict_html, true_dict_html)
+
+    pp = pprint.PrettyPrinter()
+    pp.pprint(scores)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/page_ocr_metric.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/page_ocr_metric.py
new file mode 100644
index 0000000000000000000000000000000000000000..fc19d111b335848af7d27a63de865dbea0352562
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/page_ocr_metric.py
@@ -0,0 +1,49 @@
+import re
+
+import jieba
+import nltk
+from nltk.metrics import f_measure, precision, recall
+from nltk.translate import meteor_score
+
+
+def contain_chinese_string(text):
+    chinese_pattern = re.compile(r'[\u4e00-\u9fa5]')
+    return bool(chinese_pattern.search(text))
+
+
+def cal_per_metrics(pred, gt):
+    metrics = {}
+
+    if contain_chinese_string(gt) or contain_chinese_string(pred):
+        reference = jieba.lcut(gt)
+        hypothesis = jieba.lcut(pred)
+    else:
+        reference = gt.split()
+        hypothesis = pred.split()
+
+    metrics["bleu"] = nltk.translate.bleu([reference], hypothesis)
+    metrics["meteor"] = meteor_score.meteor_score([reference], hypothesis)
+
+    reference = set(reference)
+    hypothesis = set(hypothesis)
+    metrics["f_measure"] = f_measure(reference, hypothesis)
+
+    metrics["precision"] = precision(reference, hypothesis)
+    metrics["recall"] = recall(reference, hypothesis)
+    metrics["edit_dist"] = nltk.edit_distance(pred, gt) / max(len(pred), len(gt))
+    return metrics
+
+
+if __name__ == "__main__":
+
+    # Examples for region text recognition and read all text tasks
+    predict_text = "metrics['edit_dist'] = nltk.edit_distance(pred, gt) / max(len(pred), len(gt))"
+    true_text = "metrics = nltk.edit_distance(pred, gt) / max(len(pred), len(gt))"
+
+    scores = cal_per_metrics(predict_text, true_text)
+
+    predict_text = "metrics['edit_dist'] len(gt))"
+    true_text = "metrics = nltk.edit_distance(pred, gt) / max(len(pred), len(gt))"
+
+    scores = cal_per_metrics(predict_text, true_text)
+    print(scores)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/parallel.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/parallel.py
new file mode 100644
index 0000000000000000000000000000000000000000..5692b950ebb1a8b5ce675331e175619e93b7b622
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/parallel.py
@@ -0,0 +1,52 @@
+from concurrent.futures import ProcessPoolExecutor, as_completed
+
+from tqdm import tqdm
+
+
+def parallel_process(array, function, n_jobs=16, use_kwargs=False, front_num=0):
+    """
+        A parallel version of the map function with a progress bar.
+
+        Args:
+            array (array-like): An array to iterate over.
+            function (function): A python function to apply to the elements of array
+            n_jobs (int, default=16): The number of cores to use
+            use_kwargs (boolean, default=False): Whether to consider the elements of array as dictionaries of
+                keyword arguments to function
+            front_num (int, default=3): The number of iterations to run serially before kicking off the parallel job.
+                Useful for catching bugs
+        Returns:
+            [function(array[0]), function(array[1]), ...]
+    """
+    # We run the first few iterations serially to catch bugs
+    if front_num > 0:
+        front = [function(**a) if use_kwargs else function(a) for a in array[:front_num]]
+    else:
+        front = []
+    # If we set n_jobs to 1, just run a list comprehension. This is useful for benchmarking and debugging.
+    if n_jobs == 1:
+        return front + [function(**a) if use_kwargs else function(a) for a in tqdm(array[front_num:])]
+    # Assemble the workers
+    with ProcessPoolExecutor(max_workers=n_jobs) as pool:
+        # Pass the elements of array into function
+        if use_kwargs:
+            futures = [pool.submit(function, **a) for a in array[front_num:]]
+        else:
+            futures = [pool.submit(function, a) for a in array[front_num:]]
+        kwargs = {
+            'total': len(futures),
+            'unit': 'it',
+            'unit_scale': True,
+            'leave': True
+        }
+        # Print out the progress as tasks complete
+        for f in tqdm(as_completed(futures), **kwargs):
+            pass
+    out = []
+    # Get the results from the futures.
+    for i, future in tqdm(enumerate(futures)):
+        try:
+            out.append(future.result())
+        except Exception as e:
+            out.append(e)
+    return front + out
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/requirements.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/requirements.txt
new file mode 100644
index 0000000000000000000000000000000000000000..ecdff4394adc7244bdf5256bb87948be6707d031
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/requirements.txt
@@ -0,0 +1,12 @@
+apted
+distance
+editdistance
+ipdb
+jieba
+Levenshtein
+lxml
+nltk
+numpy
+Polygon3
+tqdm
+zss
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/rrc_evaluation_funcs_1_1.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/rrc_evaluation_funcs_1_1.py
new file mode 100644
index 0000000000000000000000000000000000000000..28cf13561551b4e2256cf55ba797da92a887f77f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/rrc_evaluation_funcs_1_1.py
@@ -0,0 +1,458 @@
+# flake8: noqa
+#!/usr/bin/env python3
+
+#File: rrc_evaluation_funcs_1_1.py
+#Version: 1.1
+#Version info: changes for Python 3
+#Date: 2019-12-29
+#Description: File with useful functions to use by the evaluation scripts in the RRC website.
+
+import json
+import sys
+
+sys.path.append('./')
+import importlib
+import os
+import re
+import zipfile
+
+
+def print_help():
+    sys.stdout.write('Usage: python %s.py -g=<gtFile> -s=<submFile> [-o=<outputFolder> -p=<jsonParams>]' %sys.argv[0])
+    sys.exit(2)
+
+
+def load_zip_file_keys(file,fileNameRegExp=''):
+    """
+    Returns an array with the entries of the ZIP file that match with the regular expression.
+    The key's are the names or the file or the capturing group definied in the fileNameRegExp
+    """
+    try:
+        archive=zipfile.ZipFile(file, mode='r', allowZip64=True)
+    except :
+        raise Exception('Error loading the ZIP archive.')
+
+    pairs = []
+
+    for name in archive.namelist():
+        addFile = True
+        keyName = name
+        if fileNameRegExp!="":
+            m = re.match(fileNameRegExp,name)
+            if m == None:
+                addFile = False
+            else:
+                if len(m.groups())>0:
+                    keyName = m.group(1)
+
+        if addFile:
+            pairs.append( keyName )
+
+    return pairs
+
+
+def load_zip_file(file,fileNameRegExp='',allEntries=False):
+    """
+    Returns an array with the contents (filtered by fileNameRegExp) of a ZIP file.
+    The key's are the names or the file or the capturing group definied in the fileNameRegExp
+    allEntries validates that all entries in the ZIP file pass the fileNameRegExp
+    """
+    try:
+        archive=zipfile.ZipFile(file, mode='r', allowZip64=True)
+    except :
+        raise Exception('Error loading the ZIP archive')
+
+    pairs = []
+    for name in archive.namelist():
+        addFile = True
+        keyName = name
+        if fileNameRegExp!="":
+            m = re.match(fileNameRegExp,name)
+            if m == None:
+                addFile = False
+            else:
+                if len(m.groups())>0:
+                    keyName = m.group(1)
+
+        if addFile:
+            pairs.append( [ keyName , archive.read(name)] )
+        else:
+            if allEntries:
+                raise Exception('ZIP entry not valid: %s' %name)
+
+    return dict(pairs)
+
+def decode_utf8(raw):
+    """
+    Returns a Unicode object on success, or None on failure
+    """
+    try:
+        return raw.decode('utf-8-sig',errors = 'replace')
+    except:
+       return None
+
+def validate_lines_in_file(fileName,file_contents,CRLF=True,LTRB=True,withTranscription=False,withConfidence=False,imWidth=0,imHeight=0):
+    """
+    This function validates that all lines of the file calling the Line validation function for each line
+    """
+    utf8File = decode_utf8(file_contents)
+    if (utf8File is None) :
+        raise Exception("The file %s is not UTF-8" %fileName)
+
+    lines = utf8File.split( "\r\n" if CRLF else "\n" )
+    for line in lines:
+        line = line.replace("\r","").replace("\n","")
+        if(line != ""):
+            try:
+                validate_tl_line(line,LTRB,withTranscription,withConfidence,imWidth,imHeight)
+            except Exception as e:
+                raise Exception(("Line in sample not valid. Sample: %s Line: %s Error: %s" %(fileName,line,str(e))).encode('utf-8', 'replace'))
+
+
+
+def validate_tl_line(line,LTRB=True,withTranscription=True,withConfidence=True,imWidth=0,imHeight=0):
+    """
+    Validate the format of the line. If the line is not valid an exception will be raised.
+    If maxWidth and maxHeight are specified, all points must be inside the imgage bounds.
+    Posible values are:
+    LTRB=True: xmin,ymin,xmax,ymax[,confidence][,transcription]
+    LTRB=False: x1,y1,x2,y2,x3,y3,x4,y4[,confidence][,transcription]
+    """
+    get_tl_line_values(line,LTRB,withTranscription,withConfidence,imWidth,imHeight)
+
+
+def get_tl_line_values(line,LTRB=True,withTranscription=False,withConfidence=False,imWidth=0,imHeight=0):
+    """
+    Validate the format of the line. If the line is not valid an exception will be raised.
+    If maxWidth and maxHeight are specified, all points must be inside the imgage bounds.
+    Posible values are:
+    LTRB=True: xmin,ymin,xmax,ymax[,confidence][,transcription]
+    LTRB=False: x1,y1,x2,y2,x3,y3,x4,y4[,confidence][,transcription]
+    Returns values from a textline. Points , [Confidences], [Transcriptions]
+    """
+    confidence = 0.0
+    transcription = "";
+    points = []
+
+    numPoints = 4;
+
+    if LTRB:
+
+        numPoints = 4;
+
+        if withTranscription and withConfidence:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-1].?[0-9]*)\s*,(.*)$',line)
+            if m == None :
+                m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-1].?[0-9]*)\s*,(.*)$',line)
+                raise Exception("Format incorrect. Should be: xmin,ymin,xmax,ymax,confidence,transcription")
+        elif withConfidence:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-1].?[0-9]*)\s*$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: xmin,ymin,xmax,ymax,confidence")
+        elif withTranscription:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-9]+)\s*,(.*)$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: xmin,ymin,xmax,ymax,transcription")
+        else:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-9]+)\s*,\s*([0-9]+)\s*,?\s*$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: xmin,ymin,xmax,ymax")
+
+        xmin = int(m.group(1))
+        ymin = int(m.group(2))
+        xmax = int(m.group(3))
+        ymax = int(m.group(4))
+        if(xmax<xmin):
+                raise Exception("Xmax value (%s) not valid (Xmax < Xmin)." %(xmax))
+        if(ymax<ymin):
+                raise Exception("Ymax value (%s)  not valid (Ymax < Ymin)." %(ymax))
+
+        points = [ float(m.group(i)) for i in range(1, (numPoints+1) ) ]
+
+        if (imWidth>0 and imHeight>0):
+            validate_point_inside_bounds(xmin,ymin,imWidth,imHeight);
+            validate_point_inside_bounds(xmax,ymax,imWidth,imHeight);
+
+    else:
+
+        numPoints = 8;
+
+        if withTranscription and withConfidence:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-1].?[0-9]*)\s*,(.*)$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: x1,y1,x2,y2,x3,y3,x4,y4,confidence,transcription")
+        elif withConfidence:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*([0-1].?[0-9]*)\s*$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: x1,y1,x2,y2,x3,y3,x4,y4,confidence")
+        elif withTranscription:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,(.*)$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: x1,y1,x2,y2,x3,y3,x4,y4,transcription")
+        else:
+            m = re.match(r'^\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*,\s*(-?[0-9]+)\s*$',line)
+            if m == None :
+                raise Exception("Format incorrect. Should be: x1,y1,x2,y2,x3,y3,x4,y4")
+
+        points = [ float(m.group(i)) for i in range(1, (numPoints+1) ) ]
+
+        validate_clockwise_points(points)
+
+        if (imWidth>0 and imHeight>0):
+            validate_point_inside_bounds(points[0],points[1],imWidth,imHeight);
+            validate_point_inside_bounds(points[2],points[3],imWidth,imHeight);
+            validate_point_inside_bounds(points[4],points[5],imWidth,imHeight);
+            validate_point_inside_bounds(points[6],points[7],imWidth,imHeight);
+
+
+    if withConfidence:
+        try:
+            confidence = float(m.group(numPoints+1))
+        except ValueError:
+            raise Exception("Confidence value must be a float")
+
+    if withTranscription:
+        posTranscription = numPoints + (2 if withConfidence else 1)
+        transcription = m.group(posTranscription)
+        m2 = re.match(r'^\s*\"(.*)\"\s*$',transcription)
+        if m2 != None : #Transcription with double quotes, we extract the value and replace escaped characters
+            transcription = m2.group(1).replace("\\\\", "\\").replace("\\\"", "\"")
+
+    return points,confidence,transcription
+
+def get_tl_dict_values(detection,withTranscription=False,withConfidence=False,imWidth=0,imHeight=0,validNumPoints=[],validate_cw=True):
+    """
+    Validate the format of the dictionary. If the dictionary is not valid an exception will be raised.
+    If maxWidth and maxHeight are specified, all points must be inside the imgage bounds.
+    Posible values:
+    {"points":[[x1,y1],[x2,y2],[x3,x3],..,[xn,yn]]}
+    {"points":[[x1,y1],[x2,y2],[x3,x3],..,[xn,yn]],"transcription":"###","confidence":0.4,"illegibility":false}
+    {"points":[[x1,y1],[x2,y2],[x3,x3],..,[xn,yn]],"transcription":"###","confidence":0.4,"dontCare":false}
+    Returns values from the dictionary. Points , [Confidences], [Transcriptions]
+    """
+    confidence = 0.0
+    transcription = "";
+    points = []
+
+    if isinstance(detection, dict) == False :
+        raise Exception("Incorrect format. Object has to be a dictionary")
+
+    if not 'points' in detection:
+        raise Exception("Incorrect format. Object has no points key)")
+
+    if isinstance(detection['points'], list) == False :
+        raise Exception("Incorrect format. Object points key have to be an array)")
+
+    num_points = len(detection['points'])
+
+    if num_points<3 :
+        raise Exception("Incorrect format. Incorrect number of points. At least 3 points are necessary. Found: " +  str(num_points))
+
+    if(len(validNumPoints)>0 and num_points in validNumPoints == False ):
+        raise Exception("Incorrect format. Incorrect number of points. Only allowed 4,8 or 12 points)")
+
+    for i in range(num_points):
+        if isinstance(detection['points'][i], list) == False :
+            raise Exception("Incorrect format. Point #" + str(i+1) + " has to be an array)")
+
+        if len(detection['points'][i]) != 2 :
+            raise Exception("Incorrect format. Point #" + str(i+1) + " has to be an array with 2 objects(x,y) )")
+
+        if isinstance(detection['points'][i][0], (int,float) ) == False or isinstance(detection['points'][i][1], (int,float) ) == False :
+            raise Exception("Incorrect format. Point #" + str(i+1) + " childs have to be Integers)")
+
+        if (imWidth>0 and imHeight>0):
+            validate_point_inside_bounds(detection['points'][i][0],detection['points'][i][1],imWidth,imHeight);
+
+        points.append(float(detection['points'][i][0]))
+        points.append(float(detection['points'][i][1]))
+
+    if validate_cw :
+        validate_clockwise_points(points)
+
+    if withConfidence:
+        if not 'confidence' in detection:
+            raise Exception("Incorrect format. No confidence key)")
+
+        if isinstance(detection['confidence'], (int,float)) == False :
+            raise Exception("Incorrect format. Confidence key has to be a float)")
+
+        if detection['confidence']<0 or detection['confidence']>1 :
+            raise Exception("Incorrect format. Confidence key has to be a float between 0.0 and 1.0")
+
+        confidence = detection['confidence']
+
+    if withTranscription:
+        if not 'transcription' in detection:
+            raise Exception("Incorrect format. No transcription key)")
+
+        if isinstance(detection['transcription'], str) == False :
+            raise Exception("Incorrect format. Transcription has to be a string. Detected: " + type(detection['transcription']).__name__ )
+
+        transcription = detection['transcription']
+
+        if 'illegibility' in detection: #Ensures that if illegibility atribute is present and is True the transcription is set to ### (don't care)
+            if detection['illegibility'] == True:
+                transcription = "###"
+
+        if 'dontCare' in detection: #Ensures that if dontCare atribute is present and is True the transcription is set to ### (don't care)
+            if detection['dontCare'] == True:
+                transcription = "###"
+
+    return points,confidence,transcription
+
+def validate_point_inside_bounds(x,y,imWidth,imHeight):
+    if(x<0 or x>imWidth):
+            raise Exception("X value (%s) not valid. Image dimensions: (%s,%s)" %(xmin,imWidth,imHeight))
+    if(y<0 or y>imHeight):
+            raise Exception("Y value (%s)  not valid. Image dimensions: (%s,%s) Sample: %s Line:%s" %(ymin,imWidth,imHeight))
+
+def validate_clockwise_points(points):
+    """
+    Validates that the points are in clockwise order.
+    """
+    edge = []
+    for i in range(len(points)//2):
+        edge.append( (int(points[(i+1)*2 % len(points)]) - int(points[i*2])) * (int(points[ ((i+1)*2+1) % len(points)]) + int(points[i*2+1]))   )
+    if sum(edge)>0:
+        raise Exception("Points are not clockwise. The coordinates of bounding points have to be given in clockwise order. Regarding the correct interpretation of 'clockwise' remember that the image coordinate system used is the standard one, with the image origin at the upper left, the X axis extending to the right and Y axis extending downwards.")
+
+def get_tl_line_values_from_file_contents(content,CRLF=True,LTRB=True,withTranscription=False,withConfidence=False,imWidth=0,imHeight=0,sort_by_confidences=True):
+    """
+    Returns all points, confindences and transcriptions of a file in lists. Valid line formats:
+    xmin,ymin,xmax,ymax,[confidence],[transcription]
+    x1,y1,x2,y2,x3,y3,x4,y4,[confidence],[transcription]
+    """
+    pointsList = []
+    transcriptionsList = []
+    confidencesList = []
+
+    lines = content.split( "\r\n" if CRLF else "\n" )
+    for line in lines:
+        line = line.replace("\r","").replace("\n","")
+        if(line != "") :
+            points, confidence, transcription = get_tl_line_values(line,LTRB,withTranscription,withConfidence,imWidth,imHeight);
+            pointsList.append(points)
+            transcriptionsList.append(transcription)
+            confidencesList.append(confidence)
+
+    if withConfidence and len(confidencesList)>0 and sort_by_confidences:
+        import numpy as np
+        sorted_ind = np.argsort(-np.array(confidencesList))
+        confidencesList = [confidencesList[i] for i in sorted_ind]
+        pointsList = [pointsList[i] for i in sorted_ind]
+        transcriptionsList = [transcriptionsList[i] for i in sorted_ind]
+
+    return pointsList,confidencesList,transcriptionsList
+
+def get_tl_dict_values_from_array(array,withTranscription=False,withConfidence=False,imWidth=0,imHeight=0,sort_by_confidences=True,validNumPoints=[],validate_cw=True):
+    """
+    Returns all points, confindences and transcriptions of a file in lists. Valid dict formats:
+    {"points":[[x1,y1],[x2,y2],[x3,x3],..,[xn,yn]],"transcription":"###","confidence":0.4}
+    """
+    pointsList = []
+    transcriptionsList = []
+    confidencesList = []
+
+    for n in range(len(array)):
+        objectDict = array[n]
+        points, confidence, transcription = get_tl_dict_values(objectDict,withTranscription,withConfidence,imWidth,imHeight,validNumPoints,validate_cw);
+        pointsList.append(points)
+        transcriptionsList.append(transcription)
+        confidencesList.append(confidence)
+
+    if withConfidence and len(confidencesList)>0 and sort_by_confidences:
+        import numpy as np
+        sorted_ind = np.argsort(-np.array(confidencesList))
+        confidencesList = [confidencesList[i] for i in sorted_ind]
+        pointsList = [pointsList[i] for i in sorted_ind]
+        transcriptionsList = [transcriptionsList[i] for i in sorted_ind]
+
+    return pointsList,confidencesList,transcriptionsList
+
+def main_evaluation(p,default_evaluation_params_fn,validate_data_fn,evaluate_method_fn,show_result=True,per_sample=True):
+    """
+    This process validates a method, evaluates it and if it succed generates a ZIP file with a JSON entry for each sample.
+    Params:
+    p: Dictionary of parmeters with the GT/submission locations. If None is passed, the parameters send by the system are used.
+    default_evaluation_params_fn: points to a function that returns a dictionary with the default parameters used for the evaluation
+    validate_data_fn: points to a method that validates the corrct format of the submission
+    evaluate_method_fn: points to a function that evaluated the submission and return a Dictionary with the results
+    """
+
+    if (p == None):
+        p = dict([s[1:].split('=') for s in sys.argv[1:]])
+        if(len(sys.argv)<3):
+            print_help()
+
+    evalParams = default_evaluation_params_fn()
+    if 'p' in p.keys():
+        evalParams.update( p['p'] if isinstance(p['p'], dict) else json.loads(p['p']) )
+
+    resDict={'calculated':True,'Message':'','method':'{}','per_sample':'{}'}
+    try:
+        validate_data_fn(p['g'], p['s'], evalParams)
+        evalData = evaluate_method_fn(p['g'], p['s'], evalParams)
+        resDict.update(evalData)
+
+    except Exception as e:
+        resDict['Message']= str(e)
+        resDict['calculated']=False
+
+    if 'o' in p:
+        if not os.path.exists(p['o']):
+            os.makedirs(p['o'])
+
+        resultsOutputname = p['o'] + '/results.zip'
+        outZip = zipfile.ZipFile(resultsOutputname, mode='w', allowZip64=True)
+
+        del resDict['per_sample']
+        if 'output_items' in resDict.keys():
+            del resDict['output_items']
+
+        outZip.writestr('method.json',json.dumps(resDict))
+
+    if not resDict['calculated']:
+        if show_result:
+            sys.stderr.write('Error!\n'+ resDict['Message']+'\n\n')
+        if 'o' in p:
+            outZip.close()
+        return resDict
+
+    if 'o' in p:
+        if per_sample == True:
+            for k,v in evalData['per_sample'].items():
+                outZip.writestr( k + '.json',json.dumps(v))
+
+        if 'output_items' in evalData.keys():
+            for k, v in evalData['output_items'].items():
+                outZip.writestr( k,v)
+
+        outZip.close()
+
+    # if show_result:
+    #     #sys.stdout.write("Calculated!")
+    #     sys.stdout.write(json.dumps(resDict['method']))
+
+    return resDict
+
+
+def main_validation(default_evaluation_params_fn,validate_data_fn):
+    """
+    This process validates a method
+    Params:
+    default_evaluation_params_fn: points to a function that returns a dictionary with the default parameters used for the evaluation
+    validate_data_fn: points to a method that validates the corrct format of the submission
+    """
+    try:
+        p = dict([s[1:].split('=') for s in sys.argv[1:]])
+        evalParams = default_evaluation_params_fn()
+        if 'p' in p.keys():
+            evalParams.update( p['p'] if isinstance(p['p'], dict) else json.loads(p['p']) )
+
+        validate_data_fn(p['g'], p['s'], evalParams)
+        print ('SUCCESS')
+        sys.exit(0)
+    except Exception as e:
+        print (str(e))
+        sys.exit(101)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/spotting_metric.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/spotting_metric.py
new file mode 100644
index 0000000000000000000000000000000000000000..16267278642660f3c176035d91344f367405752b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/spotting_metric.py
@@ -0,0 +1,188 @@
+# flake8: noqa
+import ast
+import os
+import re
+import shutil
+import subprocess
+import zipfile
+
+import ipdb
+
+import vlmeval.dataset.utils.Ocrbench_v2.spotting_eval.rrc_evaluation_funcs_1_1 as rrc_evaluation_funcs
+from vlmeval.dataset.utils.Ocrbench_v2.spotting_eval.script import (default_evaluation_params,
+                                                                    evaluate_method, validate_data)
+
+
+def extract_bounding_boxes_robust(predict_str):
+    """
+    Extract coordinates and text content from the given prediction string,
+    handling potential format issues.
+
+    Args:
+    predict_str (str): Model prediction output as a string.
+
+    Returns:
+    list: Extracted data in the format [[x1, y1, x2, y2, text_content], ...].
+          Returns None if no valid data is extracted.
+    """
+    results = []
+    seen = set()
+
+    # try parsing with ast.literal_eval
+    try:
+        data = ast.literal_eval(predict_str)
+    except Exception:
+        data = None
+
+    if data is not None:
+        if isinstance(data, (list, tuple)):
+            for item in data:
+                if isinstance(item, (list, tuple)) and len(item) >= 5:
+                    x1_str, y1_str, x2_str, y2_str = item[:4]
+                    text_content = item[4]
+
+                    x1_str = str(x1_str).strip()
+                    y1_str = str(y1_str).strip()
+                    x2_str = str(x2_str).strip()
+                    y2_str = str(y2_str).strip()
+                    text_content = str(text_content).replace("\n", "").strip().strip('"').strip("'")
+
+                    try:
+                        x1 = int(x1_str)
+                        y1 = int(y1_str)
+                        x2 = int(x2_str)
+                        y2 = int(y2_str)
+
+                        if not (0 <= x1 <= 1000 and 0 <= y1 <= 1000 and 0 <= x2 <= 1000 and 0 <= y2 <= 1000):
+                            continue
+
+                        key = (x1, y1, x2, y2, text_content)
+                        if key in seen:
+                            continue
+
+                        seen.add(key)
+                        results.append([x1, y1, x2, y2, text_content])
+                    except ValueError:
+                        continue
+    else:
+        # try parsing with regular expression
+
+        list_content = predict_str
+        items = re.findall(r'[\[\(]\s*([^\[\]\(\)]*?)\s*[\]\)]', list_content)
+
+        if not items:
+            return None
+
+        for item in items:
+            parts = item.split(',', 4)
+            if len(parts) < 5:
+                continue
+
+            x1_str, y1_str, x2_str, y2_str, text_content = parts
+
+            x1_str = x1_str.strip()
+            y1_str = y1_str.strip()
+            x2_str = x2_str.strip()
+            y2_str = y2_str.strip()
+            text_content = text_content.replace("\n", "").strip().strip('"').strip("'")
+
+            try:
+                x1 = int(x1_str)
+                y1 = int(y1_str)
+                x2 = int(x2_str)
+                y2 = int(y2_str)
+
+                if not (0 <= x1 <= 1000 and 0 <= y1 <= 1000 and 0 <= x2 <= 1000 and 0 <= y2 <= 1000):
+                    continue
+
+                key = (x1, y1, x2, y2, text_content)
+                if key in seen:
+                    continue
+
+                seen.add(key)
+                results.append([x1, y1, x2, y2, text_content])
+            except ValueError:
+                continue
+
+    if not results:
+        return None
+
+    return results
+
+
+def zip_folder(source_folder, destination_zip):
+    abs_source = os.path.abspath(source_folder)
+    abs_destination = os.path.abspath(destination_zip)
+
+    with zipfile.ZipFile(abs_destination, 'w', zipfile.ZIP_DEFLATED) as zf:
+        for root, _, files in os.walk(abs_source):
+            for file in files:
+                abs_file_path = os.path.join(root, file)
+
+                relative_path = os.path.relpath(abs_file_path, abs_source)
+                zf.write(abs_file_path, relative_path)
+
+
+def spotting_evaluation(prediction_list, img_metas):
+    score = 0
+
+    submit_path = ".vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/submit"
+    gt_path = ".vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/gt"
+    submit_zip_path = ".vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/submit.zip"
+    gt_zip_path = ".vlmeval/dataset/utils/Ocrbench_v2/spotting_eval/gt.zip"
+    for file_path in [submit_path, gt_path, submit_zip_path, gt_zip_path]:
+        if "zip" in file_path:
+            if os.path.exists(file_path):
+                os.remove(file_path)
+        else:
+            if os.path.exists(file_path):
+                shutil.rmtree(file_path)
+            os.makedirs(file_path)
+
+    res_submit_list = []
+    for item in prediction_list:
+        if len(item) != 5:
+            ipdb.set_trace()
+        x1, y1, x2, y2, rec = item
+        if x1 >= x2 or y1 >= y2:
+            continue
+
+        res_submit_list.append(",".join([str(x1),str(y1),str(x2),str(y1),str(x2),str(y2),str(x1),str(y2),rec]))
+
+    res_gt_list = []
+    for bbox, rec in zip(img_metas["bbox"], img_metas["content"]):
+        x_coords = bbox[0::2]
+        y_coords = bbox[1::2]
+
+        x1, y1 = min(x_coords), min(y_coords)
+        x2, y2 = max(x_coords), max(y_coords)
+
+        res_gt_list.append(",".join([str(x1),str(y1),str(x2),str(y1),str(x2),str(y2),str(x1),str(y2),rec]))
+
+    if len(res_submit_list) == 0 or len(res_gt_list) == 0:
+        return 0
+
+    with open(os.path.join(submit_path,"res_img_0.txt"), "w") as f:
+        for item in res_submit_list[:-1]:
+            f.write(item + "\n")
+        f.write(res_submit_list[-1])
+
+    with open(os.path.join(gt_path,"gt_img_0.txt"), "w") as f:
+        for item in res_gt_list[:-1]:
+            f.write(item + "\n")
+        f.write(res_gt_list[-1])
+
+    zip_folder(submit_path, submit_zip_path)
+    zip_folder(gt_path, gt_zip_path)
+
+    command = {
+        'g': gt_zip_path,
+        's': submit_zip_path,
+        'o': './',
+        'p': '{"IOU_CONSTRAINT":0.5}'
+    }
+
+    # run rrc_evaluation_funcs
+    result = rrc_evaluation_funcs.main_evaluation(command,default_evaluation_params,validate_data,evaluate_method)
+    score = result["method"]["hmean"]
+    return score
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/vqa_metric.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/vqa_metric.py
new file mode 100644
index 0000000000000000000000000000000000000000..d3410f34338534503d83c42f569f881a0d6d7162
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/Ocrbench_v2/vqa_metric.py
@@ -0,0 +1,280 @@
+import math
+import re
+
+import ipdb
+
+
+def levenshtein_distance(s1, s2):
+    if len(s1) > len(s2):
+        s1, s2 = s2, s1
+
+    distances = range(len(s1) + 1)
+    for i2, c2 in enumerate(s2):
+        distances_ = [i2 + 1]
+        for i1, c1 in enumerate(s1):
+            if c1 == c2:
+                distances_.append(distances[i1])
+            else:
+                distances_.append(1 + min((distances[i1], distances[i1 + 1], distances_[-1])))
+        distances = distances_
+    return distances[-1]
+
+
+def vqa_evaluation(predict, answers):
+    score = 0
+    if isinstance(answers, list):
+        for j in range(len(answers)):
+            if isinstance(answers[j], (int, float)):
+                answers[j] = str(answers[j])
+            try:
+                answer = answers[j].lower().strip().replace("\n", " ")
+            except Exception:
+                ipdb.set_trace()
+            if isinstance(predict, (int, float)):
+                predict = str(predict)
+            predict = predict.lower().strip().replace("\n", " ")
+            if len(answer.split()) < 5:
+                if answer in predict:
+                    score = 1
+            else:
+                dist = levenshtein_distance(predict, answer)
+                length = max(len(predict), len(answer))
+                ANLS_value = 0.0 if length == 0 else float(dist) / float(length)
+                ANLS_value = 1 - ANLS_value
+
+                if ANLS_value >= 0.5 and ANLS_value > score:
+                    score = ANLS_value
+
+    else:
+        answers = answers.lower().strip().replace("\n", " ")
+        predict = predict.lower().strip().replace("\n", " ")
+        if len(answers.split()) < 5:
+            if answers in predict:
+                score = 1
+            else:
+                dist = levenshtein_distance(predict, answers)
+                length = max(len(predict), len(answers))
+                ANLS_value = 0.0 if length == 0 else float(dist) / float(length)
+                ANLS_value = 1 - ANLS_value
+
+                if ANLS_value >= 0.5 and ANLS_value > score:
+                    score = ANLS_value
+
+    return score
+
+
+def cn_vqa_evaluation(predict, answers):
+    score = 0
+    if isinstance(answers, list):
+        for j in range(len(answers)):
+            if isinstance(answers[j], (int, float)):
+                answers[j] = str(answers[j])
+            try:
+                answer = answers[j].lower().strip().replace("\n", " ").replace(" ", "")
+            except Exception:
+                ipdb.set_trace()
+            if isinstance(predict, (int, float)):
+                predict = str(predict)
+            predict = predict.lower().strip().replace("\n", " ").replace(" ", "")
+            if len(answer.split(",")) < 4:
+                if answer in predict:
+                    score = 1
+            else:
+                dist = levenshtein_distance(predict, answer)
+                length = max(len(predict), len(answer))
+                ANLS_value = 0.0 if length == 0 else float(dist) / float(length)
+                ANLS_value = 1 - ANLS_value
+
+                if ANLS_value >= 0.5 and ANLS_value > score:
+                    score = ANLS_value
+
+    else:
+        answers = answers.lower().strip().replace("\n", " ").replace(" ", "")
+        predict = predict.lower().strip().replace("\n", " ").replace(" ", "")
+        if len(answer.split(",")) < 4:
+            if answers in predict:
+                score = 1
+            else:
+                dist = levenshtein_distance(predict, answers)
+                length = max(len(predict), len(answers))
+                ANLS_value = 0.0 if length == 0 else float(dist) / float(length)
+                ANLS_value = 1 - ANLS_value
+
+                if ANLS_value >= 0.5 and ANLS_value > score:
+                    score = ANLS_value
+
+    return score
+
+
+def vqa_evaluation_case_sensitive(predict, answers):
+    score = 0
+    if isinstance(answers, list):
+        for j in range(len(answers)):
+            if isinstance(answers[j], (int, float)):
+                answers[j] = str(answers[j])
+            try:
+                answer = answers[j].strip().replace("\n", " ")
+            except Exception:
+                ipdb.set_trace()
+            predict = predict.strip().replace("\n", " ")
+            if len(answer.split()) < 5:
+                if answer in predict:
+                    score = 1
+            else:
+                dist = levenshtein_distance(predict, answer)
+                length = max(len(predict), len(answer))
+                ANLS_value = 0.0 if length == 0 else float(dist) / float(length)
+                ANLS_value = 1 - ANLS_value
+
+                if ANLS_value >= 0.5 and ANLS_value > score:
+                    score = ANLS_value
+
+    else:
+        answers = answers.strip().replace("\n", " ")
+        predict = predict.strip().replace("\n", " ")
+        if len(answers.split()) < 5:
+            if answers in predict:
+                score = 1
+            else:
+                dist = levenshtein_distance(predict, answers)
+                length = max(len(predict), len(answers))
+                ANLS_value = 0.0 if length == 0 else float(dist) / float(length)
+                ANLS_value = 1 - ANLS_value
+
+                if ANLS_value >= 0.5 and ANLS_value > score:
+                    score = ANLS_value
+
+    return score
+
+
+def extract_first_number(string):
+    match = re.search(r'\d+', string)
+    if match:
+        return int(match.group())
+    return None
+
+
+def counting_evaluation(predict, answers, eval_method):
+    score = 0
+
+    if isinstance(predict, str):
+        predict_processed = predict.lower().strip().replace("\n", " ")
+    elif math.isnan(predict):
+        return 0
+    else:
+        predict_processed = int(predict)
+    if isinstance(answers, list):
+        temp_score = 0
+        for j in range(len(answers)):
+            if isinstance(answers[j], (int, float)):
+                answers[j] = str(answers[j])
+            answer = answers[j].lower().strip().replace("\n", " ")
+            if eval_method == "exact match":
+                if answer in predict:
+                    score = 1
+                else:
+                    score = 0
+            elif eval_method == "regression":
+                predict_number = extract_first_number(predict_processed)
+                if predict_number:
+
+                    answer = int(answer)
+
+                    if predict_number <= 0 or predict_number >= 2 * answer:
+                        score = 0
+                    else:
+                        iou = 1 - abs(predict_number - answer) / answer
+                        if iou > 0.5:
+                            score = iou
+                        else:
+                            score = 0
+                else:
+                    score = 0
+            if score > temp_score:
+                temp_score = score
+        score = temp_score
+
+    else:
+        answers = answers.lower().strip().replace("\n", " ")
+        predict = predict.lower().strip().replace("\n", " ")
+        if eval_method == "exact match":
+            if answer in predict:
+                score = 1
+            else:
+                score = 0
+        elif eval_method == "regression":
+            predict = extract_first_number(predict)
+            if predict:
+                answer = int(answer)
+                if predict <= 0 or predict >= 2 * answer:
+                    score = 0
+                else:
+                    iou = 1 - abs(predict - answer) / answer
+
+                    if iou > 0.5:
+                        score = iou
+                    else:
+                        score = 0
+            else:
+                score = 0
+    return score
+
+
+def math_expression_evaluation(predict, answers):
+    score = 0
+    if isinstance(answers, list):
+        for j in range(len(answers)):
+            answer = answers[j].strip().replace("\n", " ").replace(" ", "")
+            predict = predict.strip().replace("\n", " ").replace(" ", "")
+            if answer in predict:
+                score = 1
+    else:
+        answers = answers.strip().replace("\n", " ").replace(" ", "")
+        predict = predict.strip().replace("\n", " ").replace(" ", "")
+        if answers in predict:
+            score = 1
+    return score
+
+
+def remove_text_tags(latex_str):
+    """
+    Removes LaTeX \text{...} tags while keeping their content.
+
+    :param latex_str: A string containing LaTeX expressions
+    :return: The processed string with \text{...} tags removed
+    """
+
+    pattern = r'\\text\{([^{}]*)\}'
+
+    processed_str = re.sub(pattern, r'\1', latex_str)
+
+    return processed_str
+
+
+def cn_math_expression_evaluation(predict, answers):
+    score = 0
+
+    assert len(answers) == 1
+    answers = [remove_text_tags(answers[0])]
+    predict = remove_text_tags(predict)
+
+    if isinstance(answers, list):
+        for j in range(len(answers)):
+            answer = answers[j].strip().replace("\n", " ").replace(" ", "")
+            predict = predict.strip().replace("\n", " ").replace(" ", "")
+            if answer in predict:
+                score = 1
+    else:
+        answers = answers.strip().replace("\n", " ").replace(" ", "")
+        predict = predict.strip().replace("\n", " ").replace(" ", "")
+        if answers in predict:
+            score = 1
+    return score
+
+
+if __name__ == "__main__":
+    test_predict = "apple pie and banana"
+    test_answers = ["apple", "banana pie", "apple pie and orange"]
+
+    vqa_score = vqa_evaluation(test_predict, test_answers)
+    print(f"VQA evaluation score for predict '{test_predict}' and answers {test_answers}: {vqa_score}")
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/CLIP_Score.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/CLIP_Score.py
new file mode 100644
index 0000000000000000000000000000000000000000..e03f98b58017d6700a71d3147856e9823872e946
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/CLIP_Score.py
@@ -0,0 +1,95 @@
+from typing import Literal
+
+import torch
+from torch.utils.data import DataLoader
+from torchmetrics.functional.multimodal.clip_score import _clip_score_update
+from torchmetrics.multimodal.clip_score import CLIPScore
+from torchvision.transforms import ToTensor
+from tqdm import tqdm
+
+from .base_metric import BaseMetric
+from .runtime import (get_available_cpu_count, get_in_memory_dataloader_workers,
+                      get_metric_batch_size)
+
+
+class CLIPScoreCalculator(BaseMetric):
+    def __init__(self, task_type: Literal['T2I', 'I2I']):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.clip_score = CLIPScore(model_name_or_path="openai/clip-vit-large-patch14")
+        self.clip_score.to(self.device)
+        self.task_type = task_type
+        default_cpu_batch = max(16, min(get_available_cpu_count(), 64))
+        self.batch_size = get_metric_batch_size(
+            "VLMEVAL_SARENA_CLIP_BATCH_SIZE",
+            cpu_default=default_cpu_batch,
+            cuda_default=64,
+            cpu_cap=64,
+        )
+        self.num_workers = get_in_memory_dataloader_workers()
+
+    def CLIP_Score(self, images, captions):
+        if isinstance(captions, tuple):
+            captions = list(captions)
+        all_scores = _clip_score_update(images, captions, self.clip_score.model, self.clip_score.processor)
+        return all_scores
+
+    def collate_fn(self, batch):
+        if self.task_type == 'T2I':
+            pred_imgs, captions = zip(*batch)
+            tensor_pred_imgs = [ToTensor()(img) for img in pred_imgs]
+            return tensor_pred_imgs, captions
+        else:
+            pred_imgs, gt_imgs = zip(*batch)
+            tensor_pred_imgs = [ToTensor()(img) for img in pred_imgs]
+            tensor_gt_imgs = [ToTensor()(img) for img in gt_imgs]
+            return tensor_pred_imgs, tensor_gt_imgs
+
+    def calculate_score(self, batch, batch_size=None, update=True):
+        effective_batch_size = batch_size or self.batch_size
+        if self.task_type == 'T2I':
+            pred_images = batch['pred_im']
+            captions = batch['caption']
+            data_loader = DataLoader(
+                list(zip(pred_images, captions)),
+                collate_fn=self.collate_fn,
+                batch_size=effective_batch_size,
+                shuffle=False,
+                num_workers=self.num_workers,
+                pin_memory=(self.device == "cuda"),
+            )
+        else:
+            pred_images = batch['pred_im']
+            gt_images = batch['gt_im']
+            data_loader = DataLoader(
+                list(zip(pred_images, gt_images)),
+                collate_fn=self.collate_fn,
+                batch_size=effective_batch_size,
+                shuffle=False,
+                num_workers=self.num_workers,
+                pin_memory=(self.device == "cuda"),
+            )
+
+        all_scores = []
+        for batch_eval in tqdm(data_loader):
+            if self.task_type == 'T2I':
+                images, captions = batch_eval
+                images = [img.to(self.device, non_blocking=True) * 255 for img in images]
+                list_scores = self.CLIP_Score(images, captions)[0].detach().cpu().tolist()
+                all_scores.extend(list_scores)
+            else:
+                pred_images, gt_images = batch_eval
+                pred_images = [img.to(self.device, non_blocking=True) * 255 for img in pred_images]
+                gt_images = [img.to(self.device, non_blocking=True) * 255 for img in gt_images]
+                list_scores = self.CLIP_Score(pred_images, gt_images)[0].detach().cpu().tolist()
+                all_scores.extend(list_scores)
+
+        if not all_scores:
+            print("No valid scores found for metric calculation.")
+            return float("nan"), []
+
+        avg_score = sum(all_scores) / len(all_scores)
+        if update:
+            self.meter.update(avg_score, len(all_scores))
+        return avg_score, all_scores
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/DINO_Score.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/DINO_Score.py
new file mode 100644
index 0000000000000000000000000000000000000000..800a119430d298358dc50a800abfd8682fe46b98
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/DINO_Score.py
@@ -0,0 +1,102 @@
+import torch
+import torch.nn.functional as F
+from PIL import Image
+from tqdm import tqdm
+from transformers import AutoImageProcessor, AutoModel
+
+from .base_metric import BaseMetric
+from .runtime import get_available_cpu_count, get_metric_batch_size
+
+
+class DINOScoreCalculator(BaseMetric):
+    def __init__(self, batch_size=None):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.model, self.processor = self.get_DINOv2_model("base")
+        self.model = self.model.to(self.device).eval()
+        self.metric = self.calculate_DINOv2_similarity_score
+        default_cpu_batch = max(8, min(get_available_cpu_count(), 32))
+        self.batch_size = batch_size or get_metric_batch_size(
+            "VLMEVAL_SARENA_DINO_BATCH_SIZE",
+            cpu_default=default_cpu_batch,
+            cuda_default=64,
+            cpu_cap=32,
+        )
+
+    def get_DINOv2_model(self, model_size):
+        model_map = {
+            "small": "facebook/dinov2-small",
+            "base": "facebook/dinov2-base",
+            "large": "facebook/dinov2-large",
+        }
+        name = model_map.get(model_size)
+        if not name:
+            raise ValueError(f"model_size should be either 'small', 'base' or 'large', got {model_size}")
+
+        model = AutoModel.from_pretrained(name)
+        processor = AutoImageProcessor.from_pretrained(name)
+
+        return model, processor
+
+    def _ensure_pil_image(self, image):
+        if isinstance(image, str):
+            with Image.open(image) as pil_image:
+                return pil_image.convert("RGB").copy()
+        if isinstance(image, Image.Image):
+            return image.convert("RGB")
+        raise ValueError("Input must be a file path or PIL Image")
+
+    def process_input(self, image, processor):
+        if isinstance(image, torch.Tensor):
+            return image.unsqueeze(0) if image.dim() == 1 else image
+        pil_image = self._ensure_pil_image(image)
+        return self._encode_batch([pil_image])
+
+    @torch.inference_mode()
+    def _encode_batch(self, images):
+        pil_images = [self._ensure_pil_image(image) for image in images]
+        inputs = self.processor(images=pil_images, return_tensors="pt")
+        pixel_values = inputs["pixel_values"].to(self.device, non_blocking=True)
+        outputs = self.model(pixel_values=pixel_values)
+        return outputs.last_hidden_state.mean(dim=1)
+
+    def calculate_DINOv2_similarity_score(self, **kwargs):
+        image1 = kwargs.get('gt_im')
+        image2 = kwargs.get('pred_im')
+        features1 = self.process_input(image1, self.processor)
+        features2 = self.process_input(image2, self.processor)
+
+        sim = F.cosine_similarity(features1, features2, dim=1).item()
+        sim = (sim + 1) / 2
+
+        return sim
+
+    @torch.inference_mode()
+    def calculate_score(self, batch, batch_size=None, update=True):
+        gt_images = batch["gt_im"]
+        pred_images = batch["pred_im"]
+
+        effective_batch_size = batch_size or self.batch_size
+        values = []
+
+        for start in tqdm(
+            range(0, len(gt_images), effective_batch_size),
+            desc="DINO batches",
+            leave=False,
+        ):
+            end = start + effective_batch_size
+            features1 = self._encode_batch(gt_images[start:end])
+            features2 = self._encode_batch(pred_images[start:end])
+            sims = F.cosine_similarity(features1, features2, dim=1)
+            sims = ((sims + 1.0) / 2.0).cpu().tolist()
+            values.extend(sims)
+
+        if not values:
+            print("No valid values found for metric calculation.")
+            return float("nan"), []
+
+        avg_score = sum(values) / len(values)
+        if update:
+            self.meter.update(avg_score, len(values))
+        return avg_score, values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/FID.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/FID.py
new file mode 100644
index 0000000000000000000000000000000000000000..a769cf80e0364c814956d9266fa58cf7a6bda7af
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/FID.py
@@ -0,0 +1,171 @@
+import os
+
+import clip
+import numpy as np
+import torch
+import torchvision.transforms as TF
+from scipy import linalg
+from torch.nn.functional import adaptive_avg_pool2d
+from tqdm import tqdm
+
+from vlmeval.smp.file import LMUDataRoot
+from .base_metric import BaseMetric
+from .inception import InceptionV3
+from .runtime import (get_available_cpu_count, get_in_memory_dataloader_workers,
+                      get_metric_batch_size)
+
+
+class FIDCalculator(BaseMetric):
+    """
+    Calculate FID and FID-C Score
+    """
+    def __init__(self, model_name='InceptionV3'):  # Fixed E251
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+        self.model_name = model_name
+
+        if self.model_name == 'ViT-B/32':
+            self.dims = 512
+            root = os.path.join(LMUDataRoot(), 'aux_models')
+            model, preprocess = clip.load('ViT-B/32', download_root=root)
+        elif self.model_name == 'InceptionV3':
+            self.dims = 2048
+            block_idx = InceptionV3.BLOCK_INDEX_BY_DIM[self.dims]
+            model = InceptionV3([block_idx]).to(self.device)
+            preprocess = TF.Compose([TF.ToTensor()])
+
+        self.model = model.to(self.device).eval()
+        self.preprocess = preprocess
+        default_cpu_batch = min(max(32, get_available_cpu_count() * 2), 96)
+        self.batch_size = get_metric_batch_size(
+            "VLMEVAL_SARENA_FID_BATCH_SIZE",
+            cpu_default=default_cpu_batch,
+            cuda_default=50,
+            cpu_cap=128,
+        )
+        self.num_workers = get_in_memory_dataloader_workers()
+
+    def calculate_frechet_distance(self, mu1, sigma1, mu2, sigma2, eps=1e-6):
+        """Numpy implementation of the Frechet Distance.
+        The Frechet distance between two multivariate Gaussians X_1 ~ N(mu_1, C_1)
+        and X_2 ~ N(mu_2, C_2) is
+                d^2 = ||mu_1 - mu_2||^2 + Tr(C_1 + C_2 - 2*sqrt(C_1*C_2)).
+
+        Stable version by Dougal J. Sutherland.
+
+        Params:
+        -- mu1   : Numpy array containing the activations of a layer of the
+                inception net (like returned by the function 'get_predictions')
+                for generated samples.
+        -- mu2   : The sample mean over activations, precalculated on an
+                representative data set.
+        -- sigma1: The covariance matrix over activations for generated samples.
+        -- sigma2: The covariance matrix over activations, precalculated on an
+                representative data set.
+
+        Returns:
+        --   : The Frechet Distance.
+        """
+
+        mu1 = np.atleast_1d(mu1)
+        mu2 = np.atleast_1d(mu2)
+
+        sigma1 = np.atleast_2d(sigma1)
+        sigma2 = np.atleast_2d(sigma2)
+
+        assert mu1.shape == mu2.shape, \
+            'Training and test mean vectors have different lengths'
+        assert sigma1.shape == sigma2.shape, \
+            'Training and test covariances have different dimensions'
+
+        diff = mu1 - mu2
+
+        # Product might be almost singular
+        covmean, _ = linalg.sqrtm(sigma1.dot(sigma2), disp=False)
+        if not np.isfinite(covmean).all():
+            msg = ('fid calculation produces singular product; '
+                   'adding %s to diagonal of cov estimates') % eps  # Fixed E128
+            print(msg)
+            offset = np.eye(sigma1.shape[0]) * eps
+            covmean = linalg.sqrtm((sigma1 + offset).dot(sigma2 + offset))
+
+        # Numerical error might give slight imaginary component
+        if np.iscomplexobj(covmean):
+            if not np.allclose(np.diagonal(covmean).imag, 0, atol=1e-3):
+                m = np.max(np.abs(covmean.imag))
+                raise ValueError('Imaginary component {}'.format(m))
+            covmean = covmean.real
+
+        tr_covmean = np.trace(covmean)
+
+        return (diff.dot(diff) + np.trace(sigma1)
+                + np.trace(sigma2) - 2 * tr_covmean)
+
+    @torch.inference_mode()
+    def get_activations(self, images):
+        dataset = ImageDataset(images, self.preprocess)
+        dataloader = torch.utils.data.DataLoader(
+            dataset,
+            batch_size=self.batch_size,
+            shuffle=False,
+            num_workers=self.num_workers,
+            pin_memory=(self.device.type == "cuda"),
+        )
+        pred_arr = np.empty((len(images), self.dims), dtype=np.float32)
+        start_idx = 0
+        for batch in tqdm(dataloader):
+            batch = batch.to(self.device)
+
+            if self.model_name == 'ViT-B/32':
+                pred = self.model.encode_image(batch).cpu().numpy()
+            elif self.model_name == 'InceptionV3':
+                pred = self.model(batch)[0]
+
+                # If model output is not scalar, apply global spatial average pooling.
+                # This happens if you choose a dimensionality not equal 2048.
+                if pred.size(2) != 1 or pred.size(3) != 1:
+                    pred = adaptive_avg_pool2d(pred, output_size=(1, 1))
+
+                pred = pred.squeeze(3).squeeze(2).cpu().numpy()
+            pred_arr[start_idx:start_idx + pred.shape[0]] = pred
+            start_idx = start_idx + pred.shape[0]
+
+        return pred_arr
+
+    def calculate_activation_statistics(self, activations):
+        mu = np.mean(activations, axis=0)
+        sigma = np.cov(activations, rowvar=False)
+        return mu, sigma
+
+    def pil_images_to_tensor(self, images_list):
+        """Convert a list of PIL Images to a torch.Tensor."""
+        tensors_list = [self.preprocess(img) for img in images_list]
+        return torch.stack(tensors_list).to(self.device)  # BxCxHxW format
+
+    def calculate_score(self, batch, update=True):
+        gt_images = batch['gt_im']
+        pred_images = batch['pred_im']
+        combined_activations = self.get_activations(gt_images + pred_images)
+        split_index = len(gt_images)
+        m1, s1 = self.calculate_activation_statistics(combined_activations[:split_index])
+        m2, s2 = self.calculate_activation_statistics(combined_activations[split_index:])
+        fid_value = self.calculate_frechet_distance(m1, s1, m2, s2)
+        if update:
+            self.meter.update(fid_value, len(batch['gt_im']))
+        return fid_value, []
+
+
+class ImageDataset(torch.utils.data.Dataset):
+    def __init__(self, images, processor=None):
+        self.images = images
+        self.processor = processor
+
+    def __len__(self):
+        return len(self.images)
+
+    def __getitem__(self, i):
+        img = self.images[i]
+        if self.processor is not None:
+            img = self.processor(img)
+        return img
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/LPIPS.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/LPIPS.py
new file mode 100644
index 0000000000000000000000000000000000000000..c88f2dc22db35606127c622cf8d94b5d8ca21a0f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/LPIPS.py
@@ -0,0 +1,96 @@
+import os
+import shutil
+
+import lpips
+import torch
+from torch.utils.data import DataLoader
+from torchvision.transforms import Normalize, ToTensor
+from tqdm import tqdm
+
+from vlmeval.smp.file import LMUDataRoot
+from .base_metric import BaseMetric
+from .runtime import (get_available_cpu_count, get_in_memory_dataloader_workers,
+                      get_metric_batch_size)
+
+
+def get_lpips_vgg_model(device):
+    """Load LPIPS VGG model, downloading to aux_models if needed."""
+    vgg_path = os.path.join(LMUDataRoot(), 'aux_models', 'vgg.pth')
+
+    if os.path.exists(vgg_path):
+        return lpips.LPIPS(net='vgg', model_path=vgg_path).to(device)
+
+    # Download model (lpips uses torch hub cache)
+    model = lpips.LPIPS(net='vgg').to(device)
+
+    # Copy from torch hub cache to aux_models for future offline use
+    aux_models_dir = os.path.dirname(vgg_path)
+    os.makedirs(aux_models_dir, exist_ok=True)
+
+    cache_path = os.path.expanduser('~/.cache/torch/hub/checkpoints/vgg_net_g.pth')
+    if os.path.exists(cache_path):
+        shutil.copy(cache_path, vgg_path)
+
+    return model
+
+
+class LPIPSCalculator(BaseMetric):
+    def __init__(self, batch_size=None):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.model = get_lpips_vgg_model(self.device).eval()
+        self.metric = self.LPIPS
+        self.to_tensor = ToTensor()
+        self.normalize = Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+        default_cpu_batch = max(4, min(get_available_cpu_count(), 16))
+        self.batch_size = batch_size or get_metric_batch_size(
+            "VLMEVAL_SARENA_LPIPS_BATCH_SIZE",
+            cpu_default=default_cpu_batch,
+            cuda_default=8,
+            cpu_cap=32,
+        )
+        self.num_workers = get_in_memory_dataloader_workers()
+
+    def LPIPS(self, tensor_image1, tensor_image2):
+        tensor_image1, tensor_image2 = tensor_image1.to(self.device), tensor_image2.to(self.device)
+        return self.model(tensor_image1, tensor_image2)
+
+    def to_tensor_transform(self, pil_img):
+        return self.normalize(self.to_tensor(pil_img))
+
+    def collate_fn(self, batch):
+        gt_imgs, pred_imgs = zip(*batch)
+        tensor_gt_imgs = torch.stack([self.to_tensor_transform(img) for img in gt_imgs])
+        tensor_pred_imgs = torch.stack([self.to_tensor_transform(img) for img in pred_imgs])
+        return tensor_gt_imgs, tensor_pred_imgs
+
+    @torch.inference_mode()
+    def calculate_score(self, batch, batch_size=None, update=True):
+        gt_images = batch['gt_im']
+        pred_images = batch['pred_im']
+        effective_batch_size = batch_size or self.batch_size
+
+        data_loader = DataLoader(
+            list(zip(gt_images, pred_images)),
+            batch_size=effective_batch_size,
+            collate_fn=self.collate_fn,
+            shuffle=False,
+            num_workers=self.num_workers,
+            pin_memory=(self.device == "cuda"),
+        )
+
+        values = []
+        for tensor_gt_batch, tensor_pred_batch in tqdm(data_loader):
+            lpips_values = self.LPIPS(tensor_gt_batch, tensor_pred_batch)
+            values.extend([lpips_values.squeeze().cpu().detach().tolist()]
+                          if lpips_values.numel() == 1 else lpips_values.squeeze().cpu().detach().tolist())
+
+        if not values:
+            print("No valid values found for metric calculation.")
+            return float("nan"), []
+
+        avg_score = sum(values) / len(values)
+        if update:
+            self.meter.update(avg_score, len(values))
+        return avg_score, values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/PSNR.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/PSNR.py
new file mode 100644
index 0000000000000000000000000000000000000000..bd75ea0e4be81e547a9f08adfee216a2f37733ab
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/PSNR.py
@@ -0,0 +1,27 @@
+import numpy as np
+from skimage.metrics import peak_signal_noise_ratio as psnr
+
+from .base_metric import BaseMetric
+
+
+class PSNRCalculator(BaseMetric):
+    def __init__(self):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.metric = self.compute_psnr
+
+    def compute_psnr(self, **kwargs):
+        gt_im = kwargs.get('gt_im')
+        pred_im = kwargs.get('pred_im')
+
+        gt_im = np.array(gt_im)
+        pred_im = np.array(pred_im)
+
+        assert gt_im.shape == pred_im.shape, "GT and predicted images must have the same shape"
+
+        psnr_score = psnr(gt_im, pred_im)
+
+        if np.isinf(psnr_score):
+            psnr_score = 100
+
+        return psnr_score
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/SSIM.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/SSIM.py
new file mode 100644
index 0000000000000000000000000000000000000000..5b5548b274cdf6c604ffd9972e95da6ea4609da3
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/SSIM.py
@@ -0,0 +1,36 @@
+import numpy as np
+from skimage.metrics import structural_similarity as ssim
+
+from .base_metric import BaseMetric
+
+
+class SSIMCalculator(BaseMetric):
+    def __init__(self):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.metric = self.compute_SSIM
+
+    def compute_SSIM(self, **kwargs):
+        image1 = kwargs.get('gt_im')
+        image2 = kwargs.get('pred_im')
+        win_size = kwargs.get('win_size', 11)  # Increase win_size for more accuracy
+        channel_axis = kwargs.get('channel_axis', -1)  # Default channel_axis to -1
+        sigma = kwargs.get('sigma', 1.5)  # Add sigma parameter for Gaussian filter
+
+        # Convert images to numpy arrays if they aren't already
+        img1_np = np.array(image1)
+        img2_np = np.array(image2)
+
+        # Check if images are grayscale or RGB
+        if len(img1_np.shape) == 3 and img1_np.shape[2] == 3:
+            # Compute SSIM for RGB images
+            score, _ = ssim(img1_np, img2_np, win_size=win_size, channel_axis=channel_axis, sigma=sigma, full=True)
+        else:
+            # Convert to grayscale if not already
+            if len(img1_np.shape) == 3:
+                img1_np = np.mean(img1_np, axis=2)
+                img2_np = np.mean(img2_np, axis=2)
+
+            score, _ = ssim(img1_np, img2_np, win_size=win_size, sigma=sigma, full=True)
+
+        return score
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/average_meter.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/average_meter.py
new file mode 100644
index 0000000000000000000000000000000000000000..7b5d792a2a8577bc3f5680aa2306d8ce0190103a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/average_meter.py
@@ -0,0 +1,16 @@
+class AverageMeter(object):
+    """Computes and stores the average and current value"""
+    def __init__(self):
+        self.reset()
+
+    def reset(self):
+        self.val = 0
+        self.avg = 0
+        self.sum = 0
+        self.count = 0
+
+    def update(self, val, n=1):
+        self.val = val
+        self.sum += val * n
+        self.count += n
+        self.avg = self.sum / self.count
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/base_metric.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/base_metric.py
new file mode 100644
index 0000000000000000000000000000000000000000..3c37f6c0a9bc3e1be767a90b5160baed222a6d81
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/base_metric.py
@@ -0,0 +1,51 @@
+import math
+
+from tqdm import tqdm
+
+from .average_meter import AverageMeter
+
+
+class BaseMetric:
+    def __init__(self):
+        self.meter = AverageMeter()
+
+    def reset(self):
+        self.meter.reset()
+
+    def calculate_score(self, batch, update=True):
+        """
+        Batch: {"gt_im": [PIL Image], "pred_im": [PIL Image]}
+        """
+        values = []
+        batch_size = len(next(iter(batch.values())))
+        for index in tqdm(range(batch_size)):
+            kwargs = {}
+            for key in ["gt_im", "pred_im", "gt_svg", "pred_svg", "gt_video", "pred_video", "caption"]:
+                if key in batch:
+                    kwargs[key] = batch[key][index]
+            try:
+                measure = self.metric(**kwargs)
+            except Exception as e:
+                print("Error calculating metric: {}".format(e))
+                continue
+            if math.isnan(measure):
+                continue
+            values.append(measure)
+
+        if not values:
+            print("No valid values found for metric calculation.")
+            return float("nan"), []
+
+        score = sum(values) / len(values)
+        if update:
+            self.meter.update(score, len(values))
+        return score, values
+
+    def metric(self, **kwargs):
+        """
+        This method should be overridden by subclasses to provide the specific metric computation.
+        """
+        raise NotImplementedError("The metric method must be implemented by subclasses.")
+
+    def get_average_score(self):
+        return self.meter.avg
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/inception.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/inception.py
new file mode 100644
index 0000000000000000000000000000000000000000..6685ad87f37c04f7208c796f469dc53ba672e87c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/inception.py
@@ -0,0 +1,350 @@
+import os
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torchvision
+from torch.hub import load_state_dict_from_url
+
+from vlmeval.smp.file import LMUDataRoot
+
+# Inception weights ported to Pytorch from
+# http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz
+FID_WEIGHTS_URL = 'https://github.com/mseitzer/pytorch-fid/releases/download/fid_weights/pt_inception-2015-12-05-6726825d.pth'  # noqa: E501
+
+
+class InceptionV3(nn.Module):
+    """Pretrained InceptionV3 network returning feature maps"""
+
+    # Index of default block of inception to return,
+    # corresponds to output of final average pooling
+    DEFAULT_BLOCK_INDEX = 3
+
+    # Maps feature dimensionality to their output blocks indices
+    BLOCK_INDEX_BY_DIM = {
+        64: 0,   # First max pooling features
+        192: 1,  # Second max pooling featurs
+        768: 2,  # Pre-aux classifier features
+        2048: 3  # Final average pooling features
+    }
+
+    def __init__(self,
+                 output_blocks=(DEFAULT_BLOCK_INDEX,),
+                 resize_input=True,
+                 normalize_input=True,
+                 requires_grad=False,
+                 use_fid_inception=True):
+        """Build pretrained InceptionV3
+
+        Parameters
+        ----------
+        output_blocks : list of int
+            Indices of blocks to return features of. Possible values are:
+                - 0: corresponds to output of first max pooling
+                - 1: corresponds to output of second max pooling
+                - 2: corresponds to output which is fed to aux classifier
+                - 3: corresponds to output of final average pooling
+        resize_input : bool
+            If true, bilinearly resizes input to width and height 299 before
+            feeding input to model. As the network without fully connected
+            layers is fully convolutional, it should be able to handle inputs
+            of arbitrary size, so resizing might not be strictly needed
+        normalize_input : bool
+            If true, scales the input from range (0, 1) to the range the
+            pretrained Inception network expects, namely (-1, 1)
+        requires_grad : bool
+            If true, parameters of the model require gradients. Possibly useful
+            for finetuning the network
+        use_fid_inception : bool
+            If true, uses the pretrained Inception model used in Tensorflow's
+            FID implementation. If false, uses the pretrained Inception model
+            available in torchvision. The FID Inception model has different
+            weights and a slightly different structure from torchvision's
+            Inception model. If you want to compute FID scores, you are
+            strongly advised to set this parameter to true to get comparable
+            results.
+        """
+        super(InceptionV3, self).__init__()
+
+        self.resize_input = resize_input
+        self.normalize_input = normalize_input
+        self.output_blocks = sorted(output_blocks)
+        self.last_needed_block = max(output_blocks)
+
+        assert self.last_needed_block <= 3, \
+            'Last possible output block index is 3'
+
+        self.blocks = nn.ModuleList()
+
+        if use_fid_inception:
+            inception = fid_inception_v3()
+        else:
+            inception = _inception_v3(weights='DEFAULT')
+
+        # Block 0: input to maxpool1
+        block0 = [
+            inception.Conv2d_1a_3x3,
+            inception.Conv2d_2a_3x3,
+            inception.Conv2d_2b_3x3,
+            nn.MaxPool2d(kernel_size=3, stride=2)
+        ]
+        self.blocks.append(nn.Sequential(*block0))
+
+        # Block 1: maxpool1 to maxpool2
+        if self.last_needed_block >= 1:
+            block1 = [
+                inception.Conv2d_3b_1x1,
+                inception.Conv2d_4a_3x3,
+                nn.MaxPool2d(kernel_size=3, stride=2)
+            ]
+            self.blocks.append(nn.Sequential(*block1))
+
+        # Block 2: maxpool2 to aux classifier
+        if self.last_needed_block >= 2:
+            block2 = [
+                inception.Mixed_5b,
+                inception.Mixed_5c,
+                inception.Mixed_5d,
+                inception.Mixed_6a,
+                inception.Mixed_6b,
+                inception.Mixed_6c,
+                inception.Mixed_6d,
+                inception.Mixed_6e,
+            ]
+            self.blocks.append(nn.Sequential(*block2))
+
+        # Block 3: aux classifier to final avgpool
+        if self.last_needed_block >= 3:
+            block3 = [
+                inception.Mixed_7a,
+                inception.Mixed_7b,
+                inception.Mixed_7c,
+                nn.AdaptiveAvgPool2d(output_size=(1, 1))
+            ]
+            self.blocks.append(nn.Sequential(*block3))
+
+        for param in self.parameters():
+            param.requires_grad = requires_grad
+
+    def forward(self, inp):
+        """Get Inception feature maps
+
+        Parameters
+        ----------
+        inp : torch.autograd.Variable
+            Input tensor of shape Bx3xHxW. Values are expected to be in
+            range (0, 1)
+
+        Returns
+        -------
+        List of torch.autograd.Variable, corresponding to the selected output
+        block, sorted ascending by index
+        """
+        outp = []
+        x = inp
+
+        if self.resize_input:
+            x = F.interpolate(x,
+                              size=(299, 299),
+                              mode='bilinear',
+                              align_corners=False)
+
+        if self.normalize_input:
+            x = 2 * x - 1  # Scale from range (0, 1) to range (-1, 1)
+
+        for idx, block in enumerate(self.blocks):
+            x = block(x)
+            if idx in self.output_blocks:
+                outp.append(x)
+
+            if idx == self.last_needed_block:
+                break
+
+        return outp
+
+
+def _inception_v3(*args, **kwargs):
+    """Wraps `torchvision.models.inception_v3`"""
+    try:
+        version = tuple(map(int, torchvision.__version__.split('.')[:2]))
+    except ValueError:
+        # Just a caution against weird version strings
+        version = (0,)
+
+    # Skips default weight inititialization if supported by torchvision
+    # version. See https://github.com/mseitzer/pytorch-fid/issues/28.
+    if version >= (0, 6):
+        kwargs['init_weights'] = False
+
+    # Backwards compatibility: `weights` argument was handled by `pretrained`
+    # argument prior to version 0.13.
+    if version < (0, 13) and 'weights' in kwargs:
+        if kwargs['weights'] == 'DEFAULT':
+            kwargs['pretrained'] = True
+        elif kwargs['weights'] is None:
+            kwargs['pretrained'] = False
+        else:
+            raise ValueError(
+                'weights=={} not supported in torchvision {}'.format(
+                    kwargs['weights'], torchvision.__version__
+                )
+            )
+        del kwargs['weights']
+
+    return torchvision.models.inception_v3(*args, **kwargs)
+
+
+def fid_inception_v3():
+    """Build pretrained Inception model for FID computation
+
+    The Inception model for FID computation uses a different set of weights
+    and has a slightly different structure than torchvision's Inception.
+
+    This method first constructs torchvision's Inception and then patches the
+    necessary parts that are different in the FID Inception model.
+    """
+    inception = _inception_v3(num_classes=1008,
+                              aux_logits=False,
+                              weights=None)
+    inception.Mixed_5b = FIDInceptionA(192, pool_features=32)
+    inception.Mixed_5c = FIDInceptionA(256, pool_features=64)
+    inception.Mixed_5d = FIDInceptionA(288, pool_features=64)
+    inception.Mixed_6b = FIDInceptionC(768, channels_7x7=128)
+    inception.Mixed_6c = FIDInceptionC(768, channels_7x7=160)
+    inception.Mixed_6d = FIDInceptionC(768, channels_7x7=160)
+    inception.Mixed_6e = FIDInceptionC(768, channels_7x7=192)
+    inception.Mixed_7b = FIDInceptionE_1(1280)
+    inception.Mixed_7c = FIDInceptionE_2(2048)
+
+    local_path = os.path.join(LMUDataRoot(), 'aux_models', 'pt_inception-2015-12-05-6726825d.pth')
+    if os.path.exists(local_path):
+        state_dict = torch.load(local_path, map_location='cpu', weights_only=True)
+    else:
+        # Ensure directory exists
+        os.makedirs(os.path.dirname(local_path), exist_ok=True)
+        # Download to aux_models directory
+        state_dict = load_state_dict_from_url(
+            FID_WEIGHTS_URL, progress=True, model_dir=os.path.dirname(local_path)
+        )
+    inception.load_state_dict(state_dict)
+    return inception
+
+
+class FIDInceptionA(torchvision.models.inception.InceptionA):
+    """InceptionA block patched for FID computation"""
+    def __init__(self, in_channels, pool_features):
+        super(FIDInceptionA, self).__init__(in_channels, pool_features)
+
+    def forward(self, x):
+        branch1x1 = self.branch1x1(x)
+
+        branch5x5 = self.branch5x5_1(x)
+        branch5x5 = self.branch5x5_2(branch5x5)
+
+        branch3x3dbl = self.branch3x3dbl_1(x)
+        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)
+        branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)
+
+        # Patch: Tensorflow's average pool does not use the padded zero's in
+        # its average calculation
+        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1,
+                                   count_include_pad=False)
+        branch_pool = self.branch_pool(branch_pool)
+
+        outputs = [branch1x1, branch5x5, branch3x3dbl, branch_pool]
+        return torch.cat(outputs, 1)
+
+
+class FIDInceptionC(torchvision.models.inception.InceptionC):
+    """InceptionC block patched for FID computation"""
+    def __init__(self, in_channels, channels_7x7):
+        super(FIDInceptionC, self).__init__(in_channels, channels_7x7)
+
+    def forward(self, x):
+        branch1x1 = self.branch1x1(x)
+
+        branch7x7 = self.branch7x7_1(x)
+        branch7x7 = self.branch7x7_2(branch7x7)
+        branch7x7 = self.branch7x7_3(branch7x7)
+
+        branch7x7dbl = self.branch7x7dbl_1(x)
+        branch7x7dbl = self.branch7x7dbl_2(branch7x7dbl)
+        branch7x7dbl = self.branch7x7dbl_3(branch7x7dbl)
+        branch7x7dbl = self.branch7x7dbl_4(branch7x7dbl)
+        branch7x7dbl = self.branch7x7dbl_5(branch7x7dbl)
+
+        # Patch: Tensorflow's average pool does not use the padded zero's in
+        # its average calculation
+        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1,
+                                   count_include_pad=False)
+        branch_pool = self.branch_pool(branch_pool)
+
+        outputs = [branch1x1, branch7x7, branch7x7dbl, branch_pool]
+        return torch.cat(outputs, 1)
+
+
+class FIDInceptionE_1(torchvision.models.inception.InceptionE):
+    """First InceptionE block patched for FID computation"""
+    def __init__(self, in_channels):
+        super(FIDInceptionE_1, self).__init__(in_channels)
+
+    def forward(self, x):
+        branch1x1 = self.branch1x1(x)
+
+        branch3x3 = self.branch3x3_1(x)
+        branch3x3 = [
+            self.branch3x3_2a(branch3x3),
+            self.branch3x3_2b(branch3x3),
+        ]
+        branch3x3 = torch.cat(branch3x3, 1)
+
+        branch3x3dbl = self.branch3x3dbl_1(x)
+        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)
+        branch3x3dbl = [
+            self.branch3x3dbl_3a(branch3x3dbl),
+            self.branch3x3dbl_3b(branch3x3dbl),
+        ]
+        branch3x3dbl = torch.cat(branch3x3dbl, 1)
+
+        # Patch: Tensorflow's average pool does not use the padded zero's in
+        # its average calculation
+        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1,
+                                   count_include_pad=False)
+        branch_pool = self.branch_pool(branch_pool)
+
+        outputs = [branch1x1, branch3x3, branch3x3dbl, branch_pool]
+        return torch.cat(outputs, 1)
+
+
+class FIDInceptionE_2(torchvision.models.inception.InceptionE):
+    """Second InceptionE block patched for FID computation"""
+    def __init__(self, in_channels):
+        super(FIDInceptionE_2, self).__init__(in_channels)
+
+    def forward(self, x):
+        branch1x1 = self.branch1x1(x)
+
+        branch3x3 = self.branch3x3_1(x)
+        branch3x3 = [
+            self.branch3x3_2a(branch3x3),
+            self.branch3x3_2b(branch3x3),
+        ]
+        branch3x3 = torch.cat(branch3x3, 1)
+
+        branch3x3dbl = self.branch3x3dbl_1(x)
+        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)
+        branch3x3dbl = [
+            self.branch3x3dbl_3a(branch3x3dbl),
+            self.branch3x3dbl_3b(branch3x3dbl),
+        ]
+        branch3x3dbl = torch.cat(branch3x3dbl, 1)
+
+        # Patch: The FID Inception model uses max pooling instead of average
+        # pooling. This is likely an error in this specific Inception
+        # implementation, as other Inception models use average pooling here
+        # (which matches the description in the paper).
+        branch_pool = F.max_pool2d(x, kernel_size=3, stride=1, padding=1)
+        branch_pool = self.branch_pool(branch_pool)
+
+        outputs = [branch1x1, branch3x3, branch3x3dbl, branch_pool]
+        return torch.cat(outputs, 1)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/metrics.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/metrics.py
new file mode 100644
index 0000000000000000000000000000000000000000..fc7a59a1521b57783f2c61531fde8b5adcf81082
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/metrics.py
@@ -0,0 +1,91 @@
+import math
+from dataclasses import dataclass
+from typing import Callable, Dict
+
+from .base_metric import BaseMetric
+from .CLIP_Score import CLIPScoreCalculator
+from .DINO_Score import DINOScoreCalculator
+from .FID import FIDCalculator
+from .LPIPS import LPIPSCalculator
+from .PSNR import PSNRCalculator
+from .SSIM import SSIMCalculator
+from .token_length import TokenLengthCalculator
+
+
+@dataclass
+class MetricsConfig:
+    use_FID: bool = False
+    use_FID_C: bool = False
+    use_CLIP_Score_T2I: bool = False
+    use_CLIP_Score_I2I: bool = False
+    use_DINO_Score: bool = False
+    use_LPIPS: bool = False
+    use_SSIM: bool = False
+    use_PSNR: bool = False
+    use_token_length: bool = False
+
+
+class InternSVGMetrics:
+    def __init__(self, config: MetricsConfig, tokenizer_path: str):
+        self.config = config
+
+        # flag -> (metric_name, builder)
+        _registry: Dict[str, tuple[str, Callable[[], BaseMetric]]] = {
+            'use_FID': ('FID', lambda: FIDCalculator(model_name='InceptionV3')),
+            'use_FID_C': ('FID-C', lambda: FIDCalculator(model_name='ViT-B/32')),
+            'use_CLIP_Score_T2I': ('CLIP-Score-T2I', lambda: CLIPScoreCalculator(task_type='T2I')),
+            'use_CLIP_Score_I2I': ('CLIP-Score-I2I', lambda: CLIPScoreCalculator(task_type='I2I')),
+            'use_DINO_Score': ('DINO-Score', lambda: DINOScoreCalculator()),
+            'use_LPIPS': ('LPIPS', lambda: LPIPSCalculator()),
+            'use_SSIM': ('SSIM', lambda: SSIMCalculator()),
+            'use_PSNR': ('PSNR', lambda: PSNRCalculator()),
+            'use_token_length': ('Token-Length', lambda: TokenLengthCalculator(tokenizer_path=tokenizer_path)),
+        }
+
+        self.active_metrics = {}
+        for flag, (metric_name, builder) in _registry.items():
+            if getattr(self.config, flag, False):
+                self.active_metrics[metric_name] = builder()
+
+    def reset(self):
+        for metric in self.active_metrics.values():
+            metric.reset()
+
+    @staticmethod
+    def _normalize_metric_result(result):
+        if isinstance(result, tuple):
+            return result[0]
+        return result
+
+    @staticmethod
+    def _is_valid_scalar(value):
+        try:
+            return not math.isnan(float(value))
+        except (TypeError, ValueError):
+            return False
+
+    def calculate_metrics(self, batch):
+        avg_results_dict = {}
+
+        for metric_name, metric in self.active_metrics.items():
+            print(f"Calculating {metric_name}...")
+            metric_result = self._normalize_metric_result(metric.calculate_score(batch))
+            if isinstance(metric_result, dict):
+                avg_results_dict[metric_name] = metric_result
+            elif self._is_valid_scalar(metric_result):
+                avg_results_dict[metric_name] = float(metric_result)
+
+        return avg_results_dict
+
+    def summarize_metrics(self):
+        summary_scores = {}
+        for name, calc in self.active_metrics.items():
+            summary_scores[name] = calc.get_average_score()
+        return summary_scores
+
+    def __len__(self) -> int:
+        return len(self.active_metrics)
+
+    def __repr__(self) -> str:
+        metrics_list = ", ".join(self.active_metrics.keys())
+        return f"<InternSVGMetrics active=[{metrics_list}]>"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/requirements_sarena.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/requirements_sarena.txt
new file mode 100644
index 0000000000000000000000000000000000000000..e68797eb3e9029503deb91bdae8ac8422f2d72df
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/requirements_sarena.txt
@@ -0,0 +1,18 @@
+av
+cairosvg
+cd-fvd
+evaluate==0.4.3
+ftfy
+hpsv2x
+lpips
+matplotlib
+moviepy
+nest_asyncio
+pyppeteer
+regex
+rich==13.9.4
+scikit-image
+svgpathtools==1.6.1
+timm==1.0.15
+torchmetrics
+vtracer
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/runtime.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/runtime.py
new file mode 100644
index 0000000000000000000000000000000000000000..8cf9e0eadcbfa7726262b872357824f57041d0d9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/runtime.py
@@ -0,0 +1,81 @@
+import os
+
+import torch
+
+try:
+    from ....utils.mp_util import cpu_count as quota_cpu_count
+except ImportError:
+    from os import cpu_count as os_cpu_count
+
+    def quota_cpu_count():
+        return os_cpu_count()
+
+
+def get_available_cpu_count() -> int:
+    count = quota_cpu_count()
+    try:
+        return max(int(count), 1)
+    except (TypeError, ValueError):
+        return 1
+
+
+def get_env_positive_int(name: str, default: int) -> int:
+    raw_value = os.getenv(name)
+    if raw_value is None:
+        return default
+
+    try:
+        return max(int(raw_value), 1)
+    except ValueError:
+        return default
+
+
+def get_metric_batch_size(
+    env_name: str,
+    cpu_default: int,
+    cuda_default: int,
+    cpu_cap: int | None = None,
+) -> int:
+    default = cuda_default if torch.cuda.is_available() else cpu_default
+    if not torch.cuda.is_available() and cpu_cap is not None:
+        default = min(default, cpu_cap)
+    return get_env_positive_int(env_name, default)
+
+
+def get_in_memory_dataloader_workers(
+    env_name: str = "VLMEVAL_SARENA_NUM_WORKERS",
+) -> int:
+    default = 0 if not torch.cuda.is_available() else min(4, get_available_cpu_count())
+    return get_env_positive_int(env_name, default)
+
+
+def maybe_configure_torch_cpu_threads() -> dict[str, int] | None:
+    if torch.cuda.is_available():
+        return None
+
+    available_cpus = get_available_cpu_count()
+    target_threads = get_env_positive_int(
+        "VLMEVAL_SARENA_TORCH_THREADS", available_cpus
+    )
+    target_interop = get_env_positive_int(
+        "VLMEVAL_SARENA_TORCH_INTEROP_THREADS",
+        min(max(1, target_threads // 2), 8),
+    )
+
+    try:
+        torch.set_num_threads(target_threads)
+    except RuntimeError:
+        pass
+
+    thread_config = {"threads": torch.get_num_threads()}
+
+    if hasattr(torch, "set_num_interop_threads") and hasattr(
+        torch, "get_num_interop_threads"
+    ):
+        try:
+            torch.set_num_interop_threads(target_interop)
+        except RuntimeError:
+            pass
+        thread_config["interop_threads"] = torch.get_num_interop_threads()
+
+    return thread_config
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/token_length.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/token_length.py
new file mode 100644
index 0000000000000000000000000000000000000000..f27c801b521cfe7a8e34a800cce48b835a818366
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/token_length.py
@@ -0,0 +1,85 @@
+import torch
+from tqdm import tqdm
+from transformers import AutoTokenizer
+
+from .average_meter import AverageMeter
+from .base_metric import BaseMetric
+
+
+class TokenLengthCalculator(BaseMetric):
+    def __init__(self, tokenizer_path: str):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.tokenizer = AutoTokenizer.from_pretrained(tokenizer_path)
+        self.metric = self.count_token_length
+
+        self.meter_gt_tokens = AverageMeter()
+        self.meter_pred_tokens = AverageMeter()
+        self.meter_diff = AverageMeter()
+
+    def count_token_length(self, **kwargs):
+        svg = kwargs.get('gt_svg')
+        tokens = self.tokenizer.encode(svg)
+
+        pred_svg = kwargs.get('pred_svg')
+        pred_tokens = self.tokenizer.encode(pred_svg)
+
+        diff = len(pred_tokens) - len(tokens)
+        return len(tokens), len(pred_tokens), diff
+
+    def calculate_score(self, batch, update=True):
+        gt_svgs = batch['gt_svg']
+        pred_svgs = batch['pred_svg']
+        values = []
+        valid_len = 0
+        for gt_svg, pred_svg in tqdm(zip(gt_svgs, pred_svgs), total=len(gt_svgs), desc='Processing SVGs'):
+            if gt_svg == '' or pred_svg == '':
+                continue
+            gt_tokens, pred_tokens, diff = self.count_token_length(gt_svg=gt_svg, pred_svg=pred_svg)
+            self.meter_gt_tokens.update(gt_tokens, 1)
+            self.meter_pred_tokens.update(pred_tokens, 1)
+            self.meter_diff.update(diff, 1)
+
+            values.append({
+                'gt_tokens': gt_tokens,
+                'pred_tokens': pred_tokens,
+                'diff': diff
+            })
+            valid_len += 1
+        avg_score = {
+            'gt_tokens': self.meter_gt_tokens.avg,
+            'pred_tokens': self.meter_pred_tokens.avg,
+            'diff': self.meter_diff.avg
+        }
+
+        if not values:
+            print("No valid SVGs found in the batch")
+            return float('nan'), []
+
+        if update:
+            self.meter.update(avg_score['pred_tokens'], valid_len)
+
+        return avg_score, values
+
+    def reset(self):
+        super().reset()
+        self.meter_gt_tokens.reset()
+        self.meter_pred_tokens.reset()
+        self.meter_diff.reset()
+
+    def get_avg_gt_tokens(self):
+        return self.meter_gt_tokens.avg
+
+    def get_avg_pred_tokens(self):
+        return self.meter_pred_tokens.avg
+
+    def get_avg_diff(self):
+        return self.meter_diff.avg
+
+    def get_average_score(self):
+        return {
+            "gt_tokens": self.get_avg_gt_tokens(),
+            "pred_tokens": self.get_avg_pred_tokens(),
+            "diff": self.get_avg_diff(),
+        }
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/utils/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/utils/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/utils/raster_svg.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/utils/raster_svg.py
new file mode 100644
index 0000000000000000000000000000000000000000..3d5ff0365a0623f756ea3fac5fe521bf99e3dae8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/utils/raster_svg.py
@@ -0,0 +1,34 @@
+import os
+import xml.etree.ElementTree as ET
+from dataclasses import dataclass
+
+import cairosvg
+
+
+@dataclass
+class InputData:
+    svg_path: str
+    output_dir: str
+    width: int
+    height: int
+
+
+def is_valid_svg(path: str) -> bool:
+    try:
+        ET.parse(path)
+        return True
+    except ET.ParseError:
+        return False
+
+
+def raster_svg(input_data: InputData):
+    try:
+        output_path = os.path.join(input_data.output_dir,
+                                   os.path.basename(input_data.svg_path).replace('.svg', '.png'))
+        if not is_valid_svg(input_data.svg_path):
+            print(f"Invalid SVG file: {input_data.svg_path}")
+            return
+        cairosvg.svg2png(url=input_data.svg_path, write_to=output_path,
+                         background_color='white', output_width=input_data.width, output_height=input_data.height)
+    except Exception as e:
+        print(f"Error rastering {input_data.svg_path}: {e}")
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/CLIP_video.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/CLIP_video.py
new file mode 100644
index 0000000000000000000000000000000000000000..024e42e4f741d543379785b055c28dba41179260
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/CLIP_video.py
@@ -0,0 +1,233 @@
+import os
+from typing import Literal
+
+import cv2
+import numpy as np
+import torch
+from torch.utils.data import DataLoader
+from tqdm import tqdm
+
+from vlmeval.smp.file import LMUDataRoot
+from ..base_metric import BaseMetric
+from .viclip.simple_tokenizer import SimpleTokenizer as _Tokenizer
+from .viclip.viclip import ViCLIP
+
+
+def get_clip(name="viclip", ckpt_path="sarena_ckpt/ViClip-InternVid-10M-FLT.pth"):
+    if name == "viclip":
+        tokenizer = _Tokenizer()
+        # Check aux_models first
+        local_path = os.path.join(LMUDataRoot(), 'aux_models', 'ViClip-InternVid-10M-FLT.pth')
+
+        if os.path.exists(local_path):
+            ckpt_path = local_path
+        else:
+            # Download from HuggingFace to aux_models
+            aux_models_dir = os.path.dirname(local_path)
+            os.makedirs(aux_models_dir, exist_ok=True)
+
+            from huggingface_hub import hf_hub_download
+            downloaded_path = hf_hub_download(
+                repo_id="OpenGVLab/ViCLIP",
+                filename="ViClip-InternVid-10M-FLT.pth",
+                local_dir=aux_models_dir,
+            )
+            ckpt_path = downloaded_path
+
+        vclip = ViCLIP(tokenizer, pretrain=ckpt_path)
+        m = (vclip, tokenizer)
+    else:
+        raise Exception("the target clip model is not found.")
+    return m
+
+
+def sample_frames_from_video(video_path, num_samples=8):
+    video = cv2.VideoCapture(video_path)
+    total_frames = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+
+    if total_frames <= 0:
+        video.release()
+        return []
+
+    indices = np.linspace(0, total_frames - 1, num_samples, dtype=int)
+
+    frames = []
+    for idx in indices:
+        video.set(cv2.CAP_PROP_POS_FRAMES, idx)
+        success, frame = video.read()
+        if success:
+            frames.append(frame)
+
+    video.release()
+    return frames
+
+
+v_mean = np.array([0.485, 0.456, 0.406]).reshape(1, 1, 3)
+v_std = np.array([0.229, 0.224, 0.225]).reshape(1, 1, 3)
+
+
+def normalize(data):
+    return (data / 255.0 - v_mean) / v_std
+
+
+def frames2tensor(
+    vid_list, fnum=8, target_size=(224, 224), device=torch.device("cuda")
+):
+    assert len(vid_list) >= fnum
+    step = len(vid_list) // fnum
+    vid_list = vid_list[::step][:fnum]
+    vid_list = [cv2.resize(x[:, :, ::-1], target_size) for x in vid_list]
+    vid_tube = [np.expand_dims(normalize(x), axis=(0, 1)) for x in vid_list]
+    vid_tube = np.concatenate(vid_tube, axis=1)
+    vid_tube = np.transpose(vid_tube, (0, 1, 4, 2, 3))
+    vid_tube = torch.from_numpy(vid_tube).to(device, non_blocking=True).float()
+    return vid_tube
+
+
+def get_text_feat_dict(texts, clip, tokenizer, text_feat_d={}):
+    for t in texts:
+        feat = clip.get_text_features(t, tokenizer, text_feat_d)
+        text_feat_d[t] = feat
+    return text_feat_d
+
+
+def get_vid_feat(frames, clip):
+    return clip.get_vid_features(frames)
+
+
+def retrieve_text(frames, texts, name="viclip", topk=5, device=torch.device("cuda")):
+    clip, tokenizer = get_clip(name)
+    clip = clip.to(device)
+    frames_tensor = frames2tensor(frames, device=device)
+    vid_feat = get_vid_feat(frames_tensor, clip)
+
+    text_feat_d = {}
+    text_feat_d = get_text_feat_dict(texts, clip, tokenizer, text_feat_d)
+    text_feats = [text_feat_d[t] for t in texts]
+    text_feats_tensor = torch.cat(text_feats, 0)
+
+    probs, idxs = clip.get_predict_label(vid_feat, text_feats_tensor, top=topk)
+
+    ret_texts = [texts[i] for i in idxs.numpy()[0].tolist()]
+    return ret_texts, probs.numpy()[0]
+
+
+class ViCLIPCalculator(BaseMetric):
+    def __init__(
+        self,
+        model_name: str = "viclip",
+        ckpt_path: str = "sarena_ckpt/ViClip-InternVid-10M-FLT.pth",
+        num_frames: int = 8,
+        target_size: tuple = (224, 224),
+        task_type: Literal["T2V", "V2V"] = "T2V",
+    ):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.model_name = model_name
+        self.ckpt_path = ckpt_path
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.clip, self.tokenizer = get_clip(model_name, ckpt_path)
+        self.clip = self.clip.to(self.device)
+        self.num_frames = num_frames
+        self.target_size = target_size
+        self.task_type = task_type
+
+    def collate_fn(self, batch):
+        if self.task_type == "T2V":
+            pred_videos, captions = zip(*batch)
+            pred_frames = [
+                sample_frames_from_video(video_path, num_samples=self.num_frames)
+                for video_path in pred_videos
+            ]
+            return pred_frames, captions
+        else:
+            pred_videos, gt_videos = zip(*batch)
+            pred_frames = [
+                sample_frames_from_video(video_path, num_samples=self.num_frames)
+                for video_path in pred_videos
+            ]
+            gt_frames = [
+                sample_frames_from_video(video_path, num_samples=self.num_frames)
+                for video_path in gt_videos
+            ]
+            return pred_frames, gt_frames
+
+    def calculate_score(self, batch, batch_size=64, update=True):
+        if self.task_type == "T2V":
+            pred_videos = batch["pred_video"]
+            texts = batch["caption"]
+            data_loader = DataLoader(
+                list(zip(pred_videos, texts)),
+                collate_fn=self.collate_fn,
+                batch_size=batch_size,
+                shuffle=False,
+                num_workers=16,
+                pin_memory=True,
+            )
+        else:
+            pred_videos = batch["pred_video"]
+            gt_videos = batch["gt_video"]
+            data_loader = DataLoader(
+                list(zip(pred_videos, gt_videos)),
+                collate_fn=self.collate_fn,
+                batch_size=batch_size,
+                shuffle=False,
+                num_workers=16,
+                pin_memory=True,
+            )
+
+        all_scores = []
+        for batch_eval in tqdm(data_loader):
+            if self.task_type == "T2V":
+                pred_frames, captions = batch_eval
+
+                for i, frames in enumerate(pred_frames):
+                    if len(frames) == 0:
+                        all_scores.append(float("nan"))
+                        continue
+                    vid_tensor = frames2tensor(
+                        frames,
+                        fnum=self.num_frames,
+                        target_size=self.target_size,
+                        device=torch.device(self.device),
+                    )
+                    vid_feat = self.clip.get_vid_features(vid_tensor)
+                    text_feat = self.clip.get_text_features(
+                        captions[i], self.tokenizer, {}
+                    )
+                    score = 100 * (vid_feat * text_feat).sum(axis=-1)
+                    all_scores.append(float(score.item()))
+            else:
+                pred_frames, gt_frames = batch_eval
+                for i, (pred_frames, gt_frames) in enumerate(
+                    zip(pred_frames, gt_frames)
+                ):
+                    if len(pred_frames) == 0 or len(gt_frames) == 0:
+                        all_scores.append(float("nan"))
+                        continue
+
+                    pred_vid_tensor = frames2tensor(
+                        pred_frames,
+                        fnum=self.num_frames,
+                        target_size=self.target_size,
+                        device=torch.device(self.device),
+                    )
+                    gt_vid_tensor = frames2tensor(
+                        gt_frames,
+                        fnum=self.num_frames,
+                        target_size=self.target_size,
+                        device=torch.device(self.device),
+                    )
+                    pred_vid_feat = self.clip.get_vid_features(pred_vid_tensor)
+                    gt_vid_feat = self.clip.get_vid_features(gt_vid_tensor)
+                    score = 100 * (pred_vid_feat * gt_vid_feat).sum(axis=-1)
+                    all_scores.append(float(score.item()))
+
+        if not all_scores:
+            print("No valid scores found for metric calculation.")
+            return float("nan"), []
+
+        avg_score = sum(all_scores) / len(all_scores)
+        if update:
+            self.meter.update(avg_score, len(all_scores))
+        return avg_score, all_scores
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/DINO_video.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/DINO_video.py
new file mode 100644
index 0000000000000000000000000000000000000000..9d12667c91fcea2c081f8e00f6fc23f89e133334
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/DINO_video.py
@@ -0,0 +1,86 @@
+import cv2
+import torch
+import torch.nn.functional as F
+from transformers import AutoImageProcessor, AutoModel
+
+from ..base_metric import BaseMetric
+from .CLIP_video import sample_frames_from_video
+
+
+class DINOVideoCalculator(BaseMetric):
+    def __init__(self, num_frames=16, model_size="base", batch_size=64, use_amp=True):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.model, self.processor = self.get_DINOv2_model(model_size)
+        self.model = self.model.to(self.device).eval()
+        self.metric = self.calculate_DINOv2_similarity_score
+        self.num_frames = num_frames
+        self.batch_size = batch_size
+        self.use_amp = use_amp and self.device == "cuda"
+
+    def get_DINOv2_model(self, model_size):
+        model_map = {
+            "small": "facebook/dinov2-small",
+            "base": "facebook/dinov2-base",
+            "large": "facebook/dinov2-large",
+        }
+        name = model_map.get(model_size)
+        if not name:
+            raise ValueError(
+                f"model_size should be either 'small', 'base' or 'large', got {model_size}"
+            )
+        model = AutoModel.from_pretrained(name)
+        processor = AutoImageProcessor.from_pretrained(name)
+        return model, processor
+
+    @torch.inference_mode()
+    def _frames_to_features(self, frames_bgr):
+        if len(frames_bgr) == 0:
+            return torch.empty(0, device=self.device)
+
+        frames_rgb = [cv2.cvtColor(f, cv2.COLOR_BGR2RGB) for f in frames_bgr]
+
+        feats = []
+        for i in range(0, len(frames_rgb), self.batch_size):
+            batch_imgs = frames_rgb[i: i + self.batch_size]
+            inputs = self.processor(images=batch_imgs, return_tensors="pt")
+            pixel_values = inputs["pixel_values"].to(self.device, non_blocking=True)
+
+            if self.use_amp:
+                with torch.autocast(device_type="cuda", dtype=torch.float16):
+                    outputs = self.model(pixel_values=pixel_values)
+            else:
+                outputs = self.model(pixel_values=pixel_values)
+
+            feat = outputs.last_hidden_state.mean(dim=1)
+            feat = F.normalize(feat, dim=1)
+            feats.append(feat)
+
+        return torch.cat(feats, dim=0)
+
+    @torch.inference_mode()
+    def calculate_DINOv2_similarity_score(self, **kwargs):
+        video1 = kwargs.get("gt_video")
+        video2 = kwargs.get("pred_video")
+        if video1 is None or video2 is None:
+            raise ValueError("Please provide 'gt_video' and 'pred_video'.")
+
+        frames1 = sample_frames_from_video(video1, self.num_frames)
+        frames2 = sample_frames_from_video(video2, self.num_frames)
+        if len(frames1) == 0 or len(frames2) == 0:
+            return float("nan")
+
+        feats1 = self._frames_to_features(frames1)
+        feats2 = self._frames_to_features(frames2)
+        if feats1.numel() == 0 or feats2.numel() == 0:
+            return float("nan")
+
+        T = min(feats1.size(0), feats2.size(0))
+        feats1 = feats1[:T]
+        feats2 = feats2[:T]
+
+        sims = (feats1 * feats2).sum(dim=1)
+        sims01 = (sims + 1.0) / 2.0
+        sim_mean = float(sims01.mean().item())
+        return sim_mean
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/FVD.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/FVD.py
new file mode 100644
index 0000000000000000000000000000000000000000..78112de2bd36fabe054df4f54f31e54d143c8644
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/FVD.py
@@ -0,0 +1,68 @@
+import torch
+from cdfvd import fvd
+
+from ..base_metric import BaseMetric
+
+
+class FVDCalculator(BaseMetric):
+    """
+    Calculate FVD Score
+    """
+
+    def __init__(
+        self, model_name: str = "i3d", resolution: int = 128, sequence_length: int = 16
+    ):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.model_name = model_name
+        self.resolution = resolution
+        self.sequence_length = sequence_length
+
+        if self.model_name == "i3d":
+            self.evaluator = fvd.cdfvd(
+                model="i3d", n_real="full", n_fake="full", device=self.device
+            )
+        elif self.model_name == "videomae":
+            self.evaluator = fvd.cdfvd(
+                model="videomae",
+                n_real="full",
+                n_fake="full",
+                device=self.device,
+                ckpt_path="PATH_TO_CKPT",
+            )
+        else:
+            raise ValueError(f"Invalid model name: {self.model_name}")
+
+    def calculate_score(self, batch, update: bool = True):
+        pred_videos = batch.get("pred_video_fvd", [])
+        overall_videos = batch.get("overall_video", [])
+        if not pred_videos or not overall_videos:
+            raise ValueError(
+                "batch must contain the key 'pred_video_fvd' and 'overall_video'"
+            )
+
+        # get dir of video
+        pred_video_dir = pred_videos[0]
+        overall_video_dir = overall_videos[0]
+        self.evaluator.compute_real_stats(
+            self.evaluator.load_videos(
+                overall_video_dir,
+                data_type="video_folder",
+                resolution=self.resolution,
+                sequence_length=self.sequence_length,
+            )
+        )
+        self.evaluator.compute_fake_stats(
+            self.evaluator.load_videos(
+                pred_video_dir,
+                data_type="video_folder",
+                resolution=self.resolution,
+                sequence_length=self.sequence_length,
+            )
+        )
+
+        fvd_score = self.evaluator.compute_fvd_from_stats()
+        if update:
+            self.meter.update(fvd_score, len(batch["gt_video"]))
+        return fvd_score, []
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/LPIPS_video.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/LPIPS_video.py
new file mode 100644
index 0000000000000000000000000000000000000000..410042e6d3dd1a432b1984e3bd62d9374cc78988
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/LPIPS_video.py
@@ -0,0 +1,121 @@
+import os
+import shutil
+
+import cv2
+import lpips
+import torch
+from PIL import Image
+from torch.utils.data import DataLoader
+from torchvision.transforms import Normalize, ToTensor
+from tqdm import tqdm
+
+from vlmeval.smp.file import LMUDataRoot
+from ..base_metric import BaseMetric
+from .CLIP_video import sample_frames_from_video
+
+
+def get_lpips_vgg_model(device):
+    """Load LPIPS VGG model, downloading to aux_models if needed."""
+    vgg_path = os.path.join(LMUDataRoot(), 'aux_models', 'vgg.pth')
+
+    if os.path.exists(vgg_path):
+        return lpips.LPIPS(net='vgg', model_path=vgg_path).to(device)
+
+    # Download model (lpips uses torch hub cache)
+    model = lpips.LPIPS(net='vgg').to(device)
+
+    # Copy from torch hub cache to aux_models for future offline use
+    aux_models_dir = os.path.dirname(vgg_path)
+    os.makedirs(aux_models_dir, exist_ok=True)
+
+    cache_path = os.path.expanduser('~/.cache/torch/hub/checkpoints/vgg_net_g.pth')
+    if os.path.exists(cache_path):
+        shutil.copy(cache_path, vgg_path)
+
+    return model
+
+
+class LPIPSVideoCalculator(BaseMetric):
+    def __init__(self, num_frames=32, batch_size=32):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.model = get_lpips_vgg_model(self.device).eval()
+        self.metric = self.LPIPS
+        self.to_tensor = ToTensor()
+        self.normalize = Normalize(
+            mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
+        )
+        self.num_frames = num_frames
+        self.batch_size = batch_size
+
+    def LPIPS(self, tensor_image1, tensor_image2):
+        tensor_image1, tensor_image2 = tensor_image1.to(self.device), tensor_image2.to(
+            self.device
+        )
+        return self.model(tensor_image1, tensor_image2)
+
+    def to_tensor_transform(self, pil_img):
+        return self.normalize(self.to_tensor(pil_img))
+
+    def collate_fn(self, batch):
+        gt_imgs, pred_imgs = zip(*batch)
+        tensor_gt_imgs = torch.stack([self.to_tensor_transform(img) for img in gt_imgs])
+        tensor_pred_imgs = torch.stack(
+            [self.to_tensor_transform(img) for img in pred_imgs]
+        )
+        return tensor_gt_imgs, tensor_pred_imgs
+
+    @torch.inference_mode()
+    def calculate_score(self, batch, batch_size=None, update=True):
+        if batch_size is None:
+            batch_size = self.batch_size
+
+        gt_videos = batch["gt_video"]
+        pred_videos = batch["pred_video"]
+
+        gt_imgs_pil = []
+        pred_imgs_pil = []
+
+        for gt_video, pred_video in zip(gt_videos, pred_videos):
+            gt_frames = sample_frames_from_video(gt_video, self.num_frames)
+            pred_frames = sample_frames_from_video(pred_video, self.num_frames)
+
+            T = min(len(gt_frames), len(pred_frames))
+            if T == 0:
+                continue
+            gt_frames = gt_frames[:T]
+            pred_frames = pred_frames[:T]
+
+            for g, p in zip(gt_frames, pred_frames):
+                g_pil = Image.fromarray(cv2.cvtColor(g, cv2.COLOR_BGR2RGB))
+                p_pil = Image.fromarray(cv2.cvtColor(p, cv2.COLOR_BGR2RGB))
+                gt_imgs_pil.append(g_pil)
+                pred_imgs_pil.append(p_pil)
+
+        if len(gt_imgs_pil) == 0:
+            print("No valid frames for metric calculation.")
+            return float("nan"), []
+
+        data_loader = DataLoader(
+            list(zip(gt_imgs_pil, pred_imgs_pil)),
+            batch_size=batch_size,
+            collate_fn=self.collate_fn,
+            shuffle=False,
+            pin_memory=(self.device == "cuda"),
+        )
+
+        values = []
+        for tensor_gt_batch, tensor_pred_batch in tqdm(data_loader):
+            lpips_values = self.LPIPS(tensor_gt_batch, tensor_pred_batch)
+            lpips_values = lpips_values.view(-1).detach().cpu().tolist()
+            values.extend(lpips_values)
+
+        if not values:
+            print("No valid values found for metric calculation.")
+            return float("nan"), []
+
+        avg_score = float(sum(values) / len(values))
+        if update:
+            self.meter.update(avg_score, len(values))
+        return avg_score, values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/PSNR_video.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/PSNR_video.py
new file mode 100644
index 0000000000000000000000000000000000000000..e38f934bcc821a0e5cf5560affaef7ce20f64d51
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/PSNR_video.py
@@ -0,0 +1,42 @@
+import cv2
+import numpy as np
+from skimage.metrics import peak_signal_noise_ratio as psnr
+
+from ..base_metric import BaseMetric
+from .CLIP_video import sample_frames_from_video
+
+
+class PSNRVideoCalculator(BaseMetric):
+    def __init__(self, num_frames=16):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.metric = self.compute_psnr
+        self.num_frames = num_frames
+
+    def compute_psnr(self, **kwargs):
+        gt_video = kwargs.get("gt_video")
+        pred_video = kwargs.get("pred_video")
+
+        gt_frames = sample_frames_from_video(gt_video, self.num_frames)
+        pred_frames = sample_frames_from_video(pred_video, self.num_frames)
+
+        scores = []
+        for gt_frame, pred_frame in zip(gt_frames, pred_frames):
+            gt_frame = cv2.cvtColor(gt_frame, cv2.COLOR_RGB2BGR)
+            pred_frame = cv2.cvtColor(pred_frame, cv2.COLOR_RGB2BGR)
+
+            gt_im = np.array(gt_frame)
+            pred_im = np.array(pred_frame)
+
+            assert (
+                gt_im.shape == pred_im.shape
+            ), "GT and predicted images must have the same shape"
+
+            psnr_score = psnr(gt_im, pred_im)
+
+            if np.isinf(psnr_score):
+                psnr_score = 100
+
+            scores.append(psnr_score)
+
+        return np.mean(scores)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/SSIM_video.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/SSIM_video.py
new file mode 100644
index 0000000000000000000000000000000000000000..6648c93f7ffd20bf5585d2a0427209532a59c369
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/SSIM_video.py
@@ -0,0 +1,57 @@
+import cv2
+import numpy as np
+from skimage.metrics import structural_similarity as ssim
+
+from ..base_metric import BaseMetric
+from .CLIP_video import sample_frames_from_video
+
+
+class SSIMVideoCalculator(BaseMetric):
+    def __init__(self, num_frames=16):
+        super().__init__()
+        self.class_name = self.__class__.__name__
+        self.metric = self.compute_SSIM
+        self.num_frames = num_frames
+
+    def compute_SSIM(self, **kwargs):
+        video1 = kwargs.get("gt_video")
+        video2 = kwargs.get("pred_video")
+        win_size = kwargs.get("win_size", 11)  # Increase win_size for more accuracy
+        channel_axis = kwargs.get("channel_axis", -1)  # Default channel_axis to -1
+        sigma = kwargs.get("sigma", 1.5)  # Add sigma parameter for Gaussian filter
+
+        frames1 = sample_frames_from_video(video1, self.num_frames)
+        frames2 = sample_frames_from_video(video2, self.num_frames)
+
+        scores = []
+        for frame1, frame2 in zip(frames1, frames2):
+            frame1 = cv2.cvtColor(frame1, cv2.COLOR_RGB2BGR)
+            frame2 = cv2.cvtColor(frame2, cv2.COLOR_RGB2BGR)
+
+            # Convert images to numpy arrays if they aren't already
+            img1_np = np.array(frame1)
+            img2_np = np.array(frame2)
+
+            # Check if images are grayscale or RGB
+            if len(img1_np.shape) == 3 and img1_np.shape[2] == 3:
+                # Compute SSIM for RGB images
+                cur_score, _ = ssim(
+                    img1_np,
+                    img2_np,
+                    win_size=win_size,
+                    channel_axis=channel_axis,
+                    sigma=sigma,
+                    full=True,
+                )
+            else:
+                # Convert to grayscale if not already
+                if len(img1_np.shape) == 3:
+                    img1_np = np.mean(img1_np, axis=2)
+                    img2_np = np.mean(img2_np, axis=2)
+
+                cur_score, _ = ssim(
+                    img1_np, img2_np, win_size=win_size, sigma=sigma, full=True
+                )
+            scores.append(cur_score)
+
+        return np.mean(scores)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/simple_tokenizer.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/simple_tokenizer.py
new file mode 100644
index 0000000000000000000000000000000000000000..9c4a3b315464bf536eaf13593ddf80b5eed52f68
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/simple_tokenizer.py
@@ -0,0 +1,159 @@
+"""
+Copy from https://huggingface.co/OpenGVLab/ViCLIP/blob/main/simple_tokenizer.py
+"""
+
+import gzip
+import html
+import os
+from functools import lru_cache
+
+import ftfy
+import regex as re
+
+
+@lru_cache()
+def default_bpe():
+    return os.path.join(
+        os.path.dirname(os.path.abspath(__file__)), "bpe_simple_vocab_16e6.txt.gz"
+    )
+
+
+# @lru_cache()
+# def default_bpe():
+#     return "bpe_simple_vocab_16e6.txt.gz"
+
+
+@lru_cache()
+def bytes_to_unicode():
+    """
+    Returns list of utf-8 byte and a corresponding list of unicode strings.
+    The reversible bpe codes work on unicode strings.
+    This means you need a large # of unicode characters in your vocab if you want to avoid UNKs.
+    When you're at something like a 10B token dataset you end up needing around 5K for decent coverage.
+    This is a signficant percentage of your normal, say, 32K bpe vocab.
+    To avoid that, we want lookup tables between utf-8 bytes and unicode strings.
+    And avoids mapping to whitespace/control characters the bpe code barfs on.
+    """
+    bs = (
+        list(range(ord("!"), ord("~") + 1))
+        + list(range(ord("¡"), ord("¬") + 1))
+        + list(range(ord("®"), ord("ÿ") + 1))
+    )
+    cs = bs[:]
+    n = 0
+    for b in range(2**8):
+        if b not in bs:
+            bs.append(b)
+            cs.append(2**8 + n)
+            n += 1
+    cs = [chr(n) for n in cs]
+    return dict(zip(bs, cs))
+
+
+def get_pairs(word):
+    """Return set of symbol pairs in a word.
+    Word is represented as tuple of symbols (symbols being variable-length strings).
+    """
+    pairs = set()
+    prev_char = word[0]
+    for char in word[1:]:
+        pairs.add((prev_char, char))
+        prev_char = char
+    return pairs
+
+
+def basic_clean(text):
+    text = ftfy.fix_text(text)
+    text = html.unescape(html.unescape(text))
+    return text.strip()
+
+
+def whitespace_clean(text):
+    text = re.sub(r"\s+", " ", text)
+    text = text.strip()
+    return text
+
+
+class SimpleTokenizer(object):
+    def __init__(self, bpe_path: str = default_bpe()):
+        self.byte_encoder = bytes_to_unicode()
+        self.byte_decoder = {v: k for k, v in self.byte_encoder.items()}
+        merges = gzip.open(bpe_path).read().decode("utf-8").split("\n")
+        merges = merges[1: 49152 - 256 - 2 + 1]
+        merges = [tuple(merge.split()) for merge in merges]
+        vocab = list(bytes_to_unicode().values())
+        vocab = vocab + [v + "</w>" for v in vocab]
+        for merge in merges:
+            vocab.append("".join(merge))
+        vocab.extend(["<|startoftext|>", "<|endoftext|>"])
+        self.encoder = dict(zip(vocab, range(len(vocab))))
+        self.decoder = {v: k for k, v in self.encoder.items()}
+        self.bpe_ranks = dict(zip(merges, range(len(merges))))
+        self.cache = {
+            "<|startoftext|>": "<|startoftext|>",
+            "<|endoftext|>": "<|endoftext|>",
+        }
+        self.pat = re.compile(
+            r"""<\|startoftext\|>|<\|endoftext\|>|'s|'t|'re|'ve|'m|'ll|'d|[\p{L}]+|[\p{N}]|[^\s\p{L}\p{N}]+""",
+            re.IGNORECASE,
+        )
+
+    def bpe(self, token):
+        if token in self.cache:
+            return self.cache[token]
+        word = tuple(token[:-1]) + (token[-1] + "</w>",)
+        pairs = get_pairs(word)
+
+        if not pairs:
+            return token + "</w>"
+
+        while True:
+            bigram = min(pairs, key=lambda pair: self.bpe_ranks.get(pair, float("inf")))
+            if bigram not in self.bpe_ranks:
+                break
+            first, second = bigram
+            new_word = []
+            i = 0
+            while i < len(word):
+                try:
+                    j = word.index(first, i)
+                    new_word.extend(word[i:j])
+                    i = j
+                except Exception:
+                    new_word.extend(word[i:])
+                    break
+
+                if word[i] == first and i < len(word) - 1 and word[i + 1] == second:
+                    new_word.append(first + second)
+                    i += 2
+                else:
+                    new_word.append(word[i])
+                    i += 1
+            new_word = tuple(new_word)
+            word = new_word
+            if len(word) == 1:
+                break
+            else:
+                pairs = get_pairs(word)
+        word = " ".join(word)
+        self.cache[token] = word
+        return word
+
+    def encode(self, text):
+        bpe_tokens = []
+        text = whitespace_clean(basic_clean(text)).lower()
+        for token in re.findall(self.pat, text):
+            token = "".join(self.byte_encoder[b] for b in token.encode("utf-8"))
+            bpe_tokens.extend(
+                self.encoder[bpe_token] for bpe_token in self.bpe(token).split(" ")
+            )
+        return bpe_tokens
+
+    def decode(self, tokens):
+        text = "".join([self.decoder[token] for token in tokens])
+        text = (
+            bytearray([self.byte_decoder[c] for c in text])
+            .decode("utf-8", errors="replace")
+            .replace("</w>", " ")
+        )
+        return text
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip.py
new file mode 100644
index 0000000000000000000000000000000000000000..c6f92578eed07a8f7dec87b1cc7382df892bf578
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip.py
@@ -0,0 +1,220 @@
+"""
+Copy from https://huggingface.co/OpenGVLab/ViCLIP/blob/main/viclip.py
+"""
+
+import logging
+import os
+
+import torch
+from torch import nn
+
+# from .criterions import VTC_VTM_Loss
+from .simple_tokenizer import SimpleTokenizer as _Tokenizer
+from .viclip_text import clip_text_l14
+from .viclip_vision import clip_joint_l14
+
+logger = logging.getLogger(__name__)
+
+
+class ViCLIP(nn.Module):
+    """docstring for ViCLIP"""
+
+    def __init__(
+        self,
+        tokenizer=None,
+        pretrain=os.path.join(
+            os.path.dirname(os.path.abspath(__file__)), "ViClip-InternVid-10M-FLT.pth"
+        ),
+        freeze_text=True,
+    ):
+        super(ViCLIP, self).__init__()
+        if tokenizer:
+            self.tokenizer = tokenizer
+        else:
+            self.tokenizer = _Tokenizer()
+        self.max_txt_l = 32
+
+        self.vision_encoder_name = "vit_l14"
+
+        self.vision_encoder_pretrained = False
+        self.inputs_image_res = 224
+        self.vision_encoder_kernel_size = 1
+        self.vision_encoder_center = True
+        self.video_input_num_frames = 8
+        self.vision_encoder_drop_path_rate = 0.1
+        self.vision_encoder_checkpoint_num = 24
+        self.is_pretrain = pretrain
+        self.vision_width = 1024
+        self.text_width = 768
+        self.embed_dim = 768
+        self.masking_prob = 0.9
+
+        self.text_encoder_name = "vit_l14"
+        self.text_encoder_pretrained = False
+        self.text_encoder_d_model = 768
+
+        self.text_encoder_vocab_size = 49408
+
+        # create modules.
+        self.vision_encoder = self.build_vision_encoder()
+        self.text_encoder = self.build_text_encoder()
+
+        self.temp = nn.parameter.Parameter(torch.ones([]) * 1 / 100.0)
+        self.temp_min = 1 / 100.0
+
+        if pretrain:
+            logger.info(f"Load pretrained weights from {pretrain}")
+            state_dict = torch.load(pretrain, map_location="cpu", weights_only=True)["model"]
+            self.load_state_dict(state_dict)
+
+        # Freeze weights
+        if freeze_text:
+            self.freeze_text()
+
+    def freeze_text(self):
+        """freeze text encoder"""
+        for p in self.text_encoder.parameters():
+            p.requires_grad = False
+
+    def no_weight_decay(self):
+        ret = {"temp"}
+        ret.update(
+            {"vision_encoder." + k for k in self.vision_encoder.no_weight_decay()}
+        )
+        ret.update({"text_encoder." + k for k in self.text_encoder.no_weight_decay()})
+
+        return ret
+
+    def forward(
+        self, image, text, raw_text, idx, log_generation=None, return_sims=False
+    ):
+        """forward and calculate loss.
+
+        Args:
+            image (torch.Tensor): The input images. Shape: [B,T,C,H,W].
+            text (dict): TODO
+            idx (torch.Tensor): TODO
+
+        Returns: TODO
+
+        """
+        self.clip_contrastive_temperature()
+
+        vision_embeds = self.encode_vision(image)
+        text_embeds = self.encode_text(raw_text)
+        if return_sims:
+            sims = torch.nn.functional.normalize(
+                vision_embeds, dim=-1
+            ) @ torch.nn.functional.normalize(text_embeds, dim=-1).transpose(0, 1)
+            return sims
+
+        loss_vtc = self.clip_loss.vtc_loss(
+            vision_embeds, text_embeds, idx, self.temp, all_gather=True
+        )
+
+        return dict(
+            loss_vtc=loss_vtc,
+        )
+
+    def encode_vision(self, image, test=False):
+        """encode image / videos as features.
+
+        Args:
+            image (torch.Tensor): The input images.
+            test (bool): Whether testing.
+
+        Returns: tuple.
+            - vision_embeds (torch.Tensor): The features of all patches. Shape: [B,T,L,C].
+            - pooled_vision_embeds (torch.Tensor): The pooled features. Shape: [B,T,C].
+
+        """
+        if image.ndim == 5:
+            image = image.permute(0, 2, 1, 3, 4).contiguous()
+        else:
+            image = image.unsqueeze(2)
+
+        if not test and self.masking_prob > 0.0:
+            return self.vision_encoder(image, masking_prob=self.masking_prob)
+
+        return self.vision_encoder(image)
+
+    def encode_text(self, text):
+        device = next(self.text_encoder.parameters()).device
+        text = self.text_encoder.tokenize(text, context_length=self.max_txt_l).to(
+            device
+        )
+        text_embeds = self.text_encoder(text)
+        return text_embeds
+
+    @torch.no_grad()
+    def clip_contrastive_temperature(self, min_val=0.001, max_val=0.5):
+        """Seems only used during pre-training"""
+        self.temp.clamp_(min=self.temp_min)
+
+    def build_vision_encoder(self):
+        """build vision encoder
+        Returns: (vision_encoder, vision_layernorm). Each is a `nn.Module`.
+
+        """
+        encoder_name = self.vision_encoder_name
+        if encoder_name != "vit_l14":
+            raise ValueError(f"Not implemented: {encoder_name}")
+        vision_encoder = clip_joint_l14(
+            pretrained=self.vision_encoder_pretrained,
+            input_resolution=self.inputs_image_res,
+            kernel_size=self.vision_encoder_kernel_size,
+            center=self.vision_encoder_center,
+            num_frames=self.video_input_num_frames,
+            drop_path=self.vision_encoder_drop_path_rate,
+            checkpoint_num=self.vision_encoder_checkpoint_num,
+        )
+        return vision_encoder
+
+    def build_text_encoder(self):
+        """build text_encoder and possiblly video-to-text multimodal fusion encoder.
+        Returns: nn.Module. The text encoder
+
+        """
+        encoder_name = self.text_encoder_name
+        if encoder_name != "vit_l14":
+            raise ValueError(f"Not implemented: {encoder_name}")
+        text_encoder = clip_text_l14(
+            pretrained=self.text_encoder_pretrained,
+            embed_dim=self.text_encoder_d_model,
+            context_length=self.max_txt_l,
+            vocab_size=self.text_encoder_vocab_size,
+            checkpoint_num=0,
+        )
+
+        return text_encoder
+
+    def get_text_encoder(self):
+        """get text encoder, used for text and cross-modal encoding"""
+        encoder = self.text_encoder
+        return encoder.bert if hasattr(encoder, "bert") else encoder
+
+    def get_text_features(self, input_text, tokenizer, text_feature_dict={}):
+        if input_text in text_feature_dict:
+            return text_feature_dict[input_text]
+        text_template = f"{input_text}"
+        with torch.no_grad():
+            # text_token = tokenizer.encode(text_template).cuda()
+            text_features = self.encode_text(text_template).float()
+            text_features /= text_features.norm(dim=-1, keepdim=True)
+            text_feature_dict[input_text] = text_features
+        return text_features
+
+    def get_vid_features(self, input_frames):
+        with torch.no_grad():
+            clip_feat = self.encode_vision(input_frames, test=True).float()
+            clip_feat /= clip_feat.norm(dim=-1, keepdim=True)
+        return clip_feat
+
+    def get_predict_label(self, clip_feature, text_feats_tensor, top=5):
+        label_probs = (100.0 * clip_feature @ text_feats_tensor.T).softmax(dim=-1)
+        top_probs, top_labels = label_probs.cpu().topk(top, dim=-1)
+        return top_probs, top_labels
+
+
+if __name__ == "__main__":
+    tokenizer = _Tokenizer()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip_text.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip_text.py
new file mode 100644
index 0000000000000000000000000000000000000000..cd9ae3084b9eaab8388be4726fbde85d0b32854c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip_text.py
@@ -0,0 +1,287 @@
+"""
+Copy from https://huggingface.co/OpenGVLab/ViCLIP/blob/main/viclip_text.py
+"""
+
+import functools
+import logging
+import os
+from collections import OrderedDict
+
+import torch
+import torch.nn.functional as F
+import torch.utils.checkpoint as checkpoint
+from pkg_resources import packaging
+from torch import nn
+
+from ..viclip.simple_tokenizer import SimpleTokenizer as _Tokenizer
+
+logger = logging.getLogger(__name__)
+
+
+# On P1, model extracted from https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K
+MODEL_PATH = "https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K"
+_MODELS = {
+    "ViT-L/14": os.path.join(MODEL_PATH, "vit_l14_text.pth"),
+}
+
+
+class LayerNorm(nn.LayerNorm):
+    """Subclass torch's LayerNorm to handle fp16."""
+
+    def forward(self, x: torch.Tensor):
+        orig_type = x.dtype
+        ret = super().forward(x.type(torch.float32))
+        return ret.type(orig_type)
+
+
+class QuickGELU(nn.Module):
+    def forward(self, x: torch.Tensor):
+        return x * torch.sigmoid(1.702 * x)
+
+
+class ResidualAttentionBlock(nn.Module):
+    def __init__(self, d_model: int, n_head: int, attn_mask: torch.Tensor = None):
+        super().__init__()
+
+        self.attn = nn.MultiheadAttention(d_model, n_head)
+        self.ln_1 = LayerNorm(d_model)
+        self.mlp = nn.Sequential(
+            OrderedDict(
+                [
+                    ("c_fc", nn.Linear(d_model, d_model * 4)),
+                    ("gelu", QuickGELU()),
+                    ("c_proj", nn.Linear(d_model * 4, d_model)),
+                ]
+            )
+        )
+        self.ln_2 = LayerNorm(d_model)
+        self.attn_mask = attn_mask
+
+    def attention(self, x: torch.Tensor):
+        self.attn_mask = (
+            self.attn_mask.to(dtype=x.dtype, device=x.device)
+            if self.attn_mask is not None
+            else None
+        )
+        return self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask)[0]
+
+    def forward(self, x: torch.Tensor):
+        x = x + self.attention(self.ln_1(x))
+        x = x + self.mlp(self.ln_2(x))
+        return x
+
+
+class Transformer(nn.Module):
+    def __init__(
+        self,
+        width: int,
+        layers: int,
+        heads: int,
+        attn_mask: torch.Tensor = None,
+        checkpoint_num: int = 0,
+    ):
+        super().__init__()
+        self.width = width
+        self.layers = layers
+        self.resblocks = nn.Sequential(
+            *[ResidualAttentionBlock(width, heads, attn_mask) for _ in range(layers)]
+        )
+
+        self.checkpoint_num = checkpoint_num
+
+    def forward(self, x: torch.Tensor):
+        if self.checkpoint_num > 0:
+            segments = min(self.checkpoint_num, len(self.resblocks))
+            return checkpoint.checkpoint_sequential(self.resblocks, segments, x)
+        else:
+            return self.resblocks(x)
+
+
+class CLIP_TEXT(nn.Module):
+    def __init__(
+        self,
+        embed_dim: int,
+        context_length: int,
+        vocab_size: int,
+        transformer_width: int,
+        transformer_heads: int,
+        transformer_layers: int,
+        checkpoint_num: int,
+    ):
+        super().__init__()
+
+        self.context_length = context_length
+        self._tokenizer = _Tokenizer()
+
+        self.transformer = Transformer(
+            width=transformer_width,
+            layers=transformer_layers,
+            heads=transformer_heads,
+            attn_mask=self.build_attention_mask(),
+            checkpoint_num=checkpoint_num,
+        )
+
+        self.vocab_size = vocab_size
+        self.token_embedding = nn.Embedding(vocab_size, transformer_width)
+        self.positional_embedding = nn.Parameter(
+            torch.empty(self.context_length, transformer_width)
+        )
+        self.ln_final = LayerNorm(transformer_width)
+
+        self.text_projection = nn.Parameter(torch.empty(transformer_width, embed_dim))
+
+    def no_weight_decay(self):
+        return {"token_embedding", "positional_embedding"}
+
+    @functools.lru_cache(maxsize=None)
+    def build_attention_mask(self):
+        # lazily create causal attention mask, with full attention between the vision tokens
+        # pytorch uses additive attention mask; fill with -inf
+        mask = torch.empty(self.context_length, self.context_length)
+        mask.fill_(float("-inf"))
+        mask.triu_(1)  # zero out the lower diagonal
+        return mask
+
+    def tokenize(self, texts, context_length=77, truncate=True):
+        """
+        Returns the tokenized representation of given input string(s)
+        Parameters
+        ----------
+        texts : Union[str, List[str]]
+            An input string or a list of input strings to tokenize
+        context_length : int
+            The context length to use; all CLIP models use 77 as the context length
+        truncate: bool
+            Whether to truncate the text in case its encoding is longer than the context length
+        Returns
+        -------
+        A two-dimensional tensor containing the resulting tokens, shape = [number of input strings, context_length].
+        We return LongTensor when torch version is <1.8.0, since older index_select requires indices to be long.
+        """
+        if isinstance(texts, str):
+            texts = [texts]
+
+        sot_token = self._tokenizer.encoder["<|startoftext|>"]
+        eot_token = self._tokenizer.encoder["<|endoftext|>"]
+        all_tokens = [
+            [sot_token] + self._tokenizer.encode(text) + [eot_token] for text in texts
+        ]
+        if packaging.version.parse(torch.__version__) < packaging.version.parse(
+            "1.8.0"
+        ):
+            result = torch.zeros(len(all_tokens), context_length, dtype=torch.long)
+        else:
+            result = torch.zeros(len(all_tokens), context_length, dtype=torch.int)
+
+        for i, tokens in enumerate(all_tokens):
+            if len(tokens) > context_length:
+                if truncate:
+                    tokens = tokens[:context_length]
+                    tokens[-1] = eot_token
+                else:
+                    raise RuntimeError(
+                        f"Input {texts[i]} is too long for context length {context_length}"
+                    )
+            result[i, : len(tokens)] = torch.tensor(tokens)
+
+        return result
+
+    def forward(self, text):
+        x = self.token_embedding(text)  # [batch_size, n_ctx, d_model]
+
+        x = x + self.positional_embedding
+        x = x.permute(1, 0, 2)  # NLD -> LND
+        x = self.transformer(x)
+        x = x.permute(1, 0, 2)  # LND -> NLD
+        x = self.ln_final(x)
+
+        # x.shape = [batch_size, n_ctx, transformer.width]
+        # take features from the eot embedding (eot_token is the highest number in each sequence)
+        x = x[torch.arange(x.shape[0]), text.argmax(dim=-1)] @ self.text_projection
+
+        return x
+
+
+def clip_text_b16(
+    embed_dim=512,
+    context_length=77,
+    vocab_size=49408,
+    transformer_width=512,
+    transformer_heads=8,
+    transformer_layers=12,
+):
+    raise NotImplementedError
+
+
+def clip_text_l14(
+    embed_dim=768,
+    context_length=77,
+    vocab_size=49408,
+    transformer_width=768,
+    transformer_heads=12,
+    transformer_layers=12,
+    checkpoint_num=0,
+    pretrained=True,
+):
+    model = CLIP_TEXT(
+        embed_dim,
+        context_length,
+        vocab_size,
+        transformer_width,
+        transformer_heads,
+        transformer_layers,
+        checkpoint_num,
+    )
+    if pretrained:
+        if isinstance(pretrained, str) and pretrained != "bert-base-uncased":
+            pretrained = _MODELS[pretrained]
+        else:
+            pretrained = _MODELS["ViT-L/14"]
+        logger.info(f"Load pretrained weights from {pretrained}")
+        state_dict = torch.load(pretrained, map_location="cpu", weights_only=True)
+        if context_length != state_dict["positional_embedding"].size(0):
+            # assert context_length < state_dict["positional_embedding"].size(0), "Cannot increase context length."
+            print(
+                f"Resize positional embedding from {state_dict['positional_embedding'].size(0)} to {context_length}"
+            )
+            if context_length < state_dict["positional_embedding"].size(0):
+                state_dict["positional_embedding"] = state_dict["positional_embedding"][
+                    :context_length
+                ]
+            else:
+                state_dict["positional_embedding"] = F.pad(
+                    state_dict["positional_embedding"],
+                    (
+                        0,
+                        0,
+                        0,
+                        context_length - state_dict["positional_embedding"].size(0),
+                    ),
+                    value=0,
+                )
+
+        message = model.load_state_dict(state_dict, strict=False)
+        print(f"Load pretrained weights from {pretrained}: {message}")
+    return model.eval()
+
+
+def clip_text_l14_336(
+    embed_dim=768,
+    context_length=77,
+    vocab_size=49408,
+    transformer_width=768,
+    transformer_heads=12,
+    transformer_layers=12,
+):
+    raise NotImplementedError
+
+
+def build_clip(config):
+    model_cls = config.text_encoder.clip_teacher
+    model_builders = {
+        "clip_text_l14": clip_text_l14,
+    }
+    if model_cls not in model_builders:
+        raise ValueError(f"Unknown model class: {model_cls}")
+    model = model_builders[model_cls]()
+    return model
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip_vision.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip_vision.py
new file mode 100644
index 0000000000000000000000000000000000000000..2fa8ee245f00c394112b967725ce04b9be6b8fe7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/viclip/viclip_vision.py
@@ -0,0 +1,464 @@
+"""
+Copy from https://huggingface.co/OpenGVLab/ViCLIP/blob/main/viclip_vision.py
+"""
+
+import logging
+import os
+from collections import OrderedDict
+
+import torch
+import torch.utils.checkpoint as checkpoint
+from einops import rearrange
+from timm.models.layers import DropPath
+from timm.models.registry import register_model
+from torch import nn
+
+# from models.utils import load_temp_embed_with_mismatch
+
+logger = logging.getLogger(__name__)
+
+
+def load_temp_embed_with_mismatch(temp_embed_old, temp_embed_new, add_zero=True):
+    """
+    Add/Remove extra temporal_embeddings as needed.
+    https://arxiv.org/abs/2104.00650 shows adding zero paddings works.
+
+    temp_embed_old: (1, num_frames_old, 1, d)
+    temp_embed_new: (1, num_frames_new, 1, d)
+    add_zero: bool, if True, add zero, else, interpolate trained embeddings.
+    """
+    # TODO zero pad
+    num_frms_new = temp_embed_new.shape[1]
+    num_frms_old = temp_embed_old.shape[1]
+    logger.info(f"Load temporal_embeddings, lengths: {num_frms_old}-->{num_frms_new}")
+    if num_frms_new > num_frms_old:
+        temp_embed_new[:, :num_frms_old] = (
+            temp_embed_old  # untrained embeddings are zeros.
+        )
+    elif num_frms_new < num_frms_old:
+        temp_embed_new = temp_embed_old[:, :num_frms_new]
+    else:  # =
+        temp_embed_new = temp_embed_old
+    return temp_embed_new
+
+
+# On P1, model extracted from https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K
+MODEL_PATH = ""
+_MODELS = {
+    "ViT-L/14": os.path.join(MODEL_PATH, "ViClip-InternVid-10M-FLT.pth"),
+}
+
+
+class QuickGELU(nn.Module):
+    def forward(self, x):
+        return x * torch.sigmoid(1.702 * x)
+
+
+class ResidualAttentionBlock(nn.Module):
+    def __init__(self, d_model, n_head, drop_path=0.0, attn_mask=None, dropout=0.0):
+        super().__init__()
+
+        self.drop_path1 = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+        self.drop_path2 = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+        # logger.info(f'Droppath: {drop_path}')
+        self.attn = nn.MultiheadAttention(d_model, n_head, dropout=dropout)
+        self.ln_1 = nn.LayerNorm(d_model)
+        self.mlp = nn.Sequential(
+            OrderedDict(
+                [
+                    ("c_fc", nn.Linear(d_model, d_model * 4)),
+                    ("gelu", QuickGELU()),
+                    ("drop1", nn.Dropout(dropout)),
+                    ("c_proj", nn.Linear(d_model * 4, d_model)),
+                    ("drop2", nn.Dropout(dropout)),
+                ]
+            )
+        )
+        self.ln_2 = nn.LayerNorm(d_model)
+        self.attn_mask = attn_mask
+
+    def attention(self, x):
+        self.attn_mask = (
+            self.attn_mask.to(dtype=x.dtype, device=x.device)
+            if self.attn_mask is not None
+            else None
+        )
+        return self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask)[0]
+
+    def forward(self, x):
+        x = x + self.drop_path1(self.attention(self.ln_1(x)))
+        x = x + self.drop_path2(self.mlp(self.ln_2(x)))
+        return x
+
+
+class Transformer(nn.Module):
+    def __init__(
+        self, width, layers, heads, drop_path=0.0, checkpoint_num=0, dropout=0.0
+    ):
+        super().__init__()
+        dpr = [x.item() for x in torch.linspace(0, drop_path, layers)]
+        self.resblocks = nn.ModuleList()
+        for idx in range(layers):
+            self.resblocks.append(
+                ResidualAttentionBlock(
+                    width, heads, drop_path=dpr[idx], dropout=dropout
+                )
+            )
+        self.checkpoint_num = checkpoint_num
+
+    def forward(self, x):
+        for idx, blk in enumerate(self.resblocks):
+            if idx < self.checkpoint_num:
+                x = checkpoint.checkpoint(blk, x)
+            else:
+                x = blk(x)
+        return x
+
+
+class VisionTransformer(nn.Module):
+    def __init__(
+        self,
+        input_resolution,
+        patch_size,
+        width,
+        layers,
+        heads,
+        output_dim=None,
+        kernel_size=1,
+        num_frames=8,
+        drop_path=0,
+        checkpoint_num=0,
+        dropout=0.0,
+        temp_embed=True,
+    ):
+        super().__init__()
+        self.output_dim = output_dim
+        self.conv1 = nn.Conv3d(
+            3,
+            width,
+            (kernel_size, patch_size, patch_size),
+            (kernel_size, patch_size, patch_size),
+            (0, 0, 0),
+            bias=False,
+        )
+
+        scale = width**-0.5
+        self.class_embedding = nn.Parameter(scale * torch.randn(width))
+        self.positional_embedding = nn.Parameter(
+            scale * torch.randn((input_resolution // patch_size) ** 2 + 1, width)
+        )
+        self.ln_pre = nn.LayerNorm(width)
+        if temp_embed:
+            self.temporal_positional_embedding = nn.Parameter(
+                torch.zeros(1, num_frames, width)
+            )
+
+        self.transformer = Transformer(
+            width,
+            layers,
+            heads,
+            drop_path=drop_path,
+            checkpoint_num=checkpoint_num,
+            dropout=dropout,
+        )
+
+        self.ln_post = nn.LayerNorm(width)
+        if output_dim is not None:
+            self.proj = nn.Parameter(torch.empty(width, output_dim))
+        else:
+            self.proj = None
+
+        self.dropout = nn.Dropout(dropout)
+
+    def get_num_layers(self):
+        return len(self.transformer.resblocks)
+
+    @torch.jit.ignore
+    def no_weight_decay(self):
+        return {
+            "positional_embedding",
+            "class_embedding",
+            "temporal_positional_embedding",
+        }
+
+    def mask_tokens(self, inputs, masking_prob=0.0):
+        B, L, _ = inputs.shape
+
+        # This is different from text as we are masking a fix number of tokens
+        Lm = int(masking_prob * L)
+        masked_indices = torch.zeros(B, L)
+        indices = torch.argsort(torch.rand_like(masked_indices), dim=-1)[:, :Lm]
+        batch_indices = (
+            torch.arange(masked_indices.shape[0]).unsqueeze(-1).expand_as(indices)
+        )
+        masked_indices[batch_indices, indices] = 1
+
+        masked_indices = masked_indices.bool()
+
+        return inputs[~masked_indices].reshape(B, -1, inputs.shape[-1])
+
+    def forward(self, x, masking_prob=0.0):
+        x = self.conv1(x)  # shape = [*, width, grid, grid]
+        B, C, T, H, W = x.shape
+        x = x.permute(0, 2, 3, 4, 1).reshape(B * T, H * W, C)
+
+        x = torch.cat(
+            [
+                self.class_embedding.to(x.dtype)
+                + torch.zeros(
+                    x.shape[0], 1, x.shape[-1], dtype=x.dtype, device=x.device
+                ),
+                x,
+            ],
+            dim=1,
+        )
+        x = x + self.positional_embedding.to(x.dtype)
+
+        # temporal pos
+        cls_tokens = x[:B, :1, :]
+        x = x[:, 1:]
+        x = rearrange(x, "(b t) n m -> (b n) t m", b=B, t=T)
+        if hasattr(self, "temporal_positional_embedding"):
+            if x.size(1) == 1:
+                # This is a workaround for unused parameter issue
+                x = x + self.temporal_positional_embedding.mean(1)
+            else:
+                x = x + self.temporal_positional_embedding
+        x = rearrange(x, "(b n) t m -> b (n t) m", b=B, t=T)
+
+        if masking_prob > 0.0:
+            x = self.mask_tokens(x, masking_prob)
+
+        x = torch.cat((cls_tokens, x), dim=1)
+
+        x = self.ln_pre(x)
+
+        x = x.permute(1, 0, 2)  # BND -> NBD
+        x = self.transformer(x)
+
+        x = self.ln_post(x)
+
+        if self.proj is not None:
+            x = self.dropout(x[0]) @ self.proj
+        else:
+            x = x.permute(1, 0, 2)  # NBD -> BND
+
+        return x
+
+
+def inflate_weight(weight_2d, time_dim, center=True):
+    logger.info(f"Init center: {center}")
+    if center:
+        weight_3d = torch.zeros(*weight_2d.shape)
+        weight_3d = weight_3d.unsqueeze(2).repeat(1, 1, time_dim, 1, 1)
+        middle_idx = time_dim // 2
+        weight_3d[:, :, middle_idx, :, :] = weight_2d
+    else:
+        weight_3d = weight_2d.unsqueeze(2).repeat(1, 1, time_dim, 1, 1)
+        weight_3d = weight_3d / time_dim
+    return weight_3d
+
+
+def load_state_dict(
+    model, state_dict, input_resolution=224, patch_size=16, center=True
+):
+    state_dict_3d = model.state_dict()
+    for k in state_dict.keys():
+        if k in state_dict_3d.keys() and state_dict[k].shape != state_dict_3d[k].shape:
+            if len(state_dict_3d[k].shape) <= 2:
+                logger.info(f"Ignore: {k}")
+                continue
+            logger.info(
+                f"Inflate: {k}, {state_dict[k].shape} => {state_dict_3d[k].shape}"
+            )
+            time_dim = state_dict_3d[k].shape[2]
+            state_dict[k] = inflate_weight(state_dict[k], time_dim, center=center)
+
+    pos_embed_checkpoint = state_dict["positional_embedding"]
+    embedding_size = pos_embed_checkpoint.shape[-1]
+    num_patches = (input_resolution // patch_size) ** 2
+    orig_size = int((pos_embed_checkpoint.shape[-2] - 1) ** 0.5)
+    new_size = int(num_patches**0.5)
+    if orig_size != new_size:
+        logger.info(f"Pos_emb from {orig_size} to {new_size}")
+        extra_tokens = pos_embed_checkpoint[:1]
+        pos_tokens = pos_embed_checkpoint[1:]
+        pos_tokens = pos_tokens.reshape(
+            -1, orig_size, orig_size, embedding_size
+        ).permute(0, 3, 1, 2)
+        pos_tokens = torch.nn.functional.interpolate(
+            pos_tokens, size=(new_size, new_size), mode="bicubic", align_corners=False
+        )
+        pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(0, 2)
+        new_pos_embed = torch.cat((extra_tokens, pos_tokens), dim=0)
+        state_dict["positional_embedding"] = new_pos_embed
+
+    message = model.load_state_dict(state_dict, strict=False)
+    logger.info(f"Load pretrained weights: {message}")
+
+
+@register_model
+def clip_joint_b16(
+    pretrained=True,
+    input_resolution=224,
+    kernel_size=1,
+    center=True,
+    num_frames=8,
+    drop_path=0.0,
+):
+    model = VisionTransformer(
+        input_resolution=input_resolution,
+        patch_size=16,
+        width=768,
+        layers=12,
+        heads=12,
+        output_dim=512,
+        kernel_size=kernel_size,
+        num_frames=num_frames,
+        drop_path=drop_path,
+    )
+    raise NotImplementedError
+    if pretrained:
+        logger.info("load pretrained weights")
+        state_dict = torch.load(_MODELS["ViT-B/16"], map_location="cpu", weights_only=True)
+        load_state_dict(
+            model,
+            state_dict,
+            input_resolution=input_resolution,
+            patch_size=16,
+            center=center,
+        )
+    return model.eval()
+
+
+@register_model
+def clip_joint_l14(
+    pretrained=False,
+    input_resolution=224,
+    kernel_size=1,
+    center=True,
+    num_frames=8,
+    drop_path=0.0,
+    checkpoint_num=0,
+    dropout=0.0,
+):
+    model = VisionTransformer(
+        input_resolution=input_resolution,
+        patch_size=14,
+        width=1024,
+        layers=24,
+        heads=16,
+        output_dim=768,
+        kernel_size=kernel_size,
+        num_frames=num_frames,
+        drop_path=drop_path,
+        checkpoint_num=checkpoint_num,
+        dropout=dropout,
+    )
+
+    if pretrained:
+        if isinstance(pretrained, str):
+            model_name = pretrained
+        else:
+            model_name = "ViT-L/14"
+        logger.info("load pretrained weights")
+        state_dict = torch.load(_MODELS[model_name], map_location="cpu", weights_only=True)
+        load_state_dict(
+            model,
+            state_dict,
+            input_resolution=input_resolution,
+            patch_size=14,
+            center=center,
+        )
+    return model.eval()
+
+
+@register_model
+def clip_joint_l14_336(
+    pretrained=True,
+    input_resolution=336,
+    kernel_size=1,
+    center=True,
+    num_frames=8,
+    drop_path=0.0,
+):
+    raise NotImplementedError
+    model = VisionTransformer(
+        input_resolution=input_resolution,
+        patch_size=14,
+        width=1024,
+        layers=24,
+        heads=16,
+        output_dim=768,
+        kernel_size=kernel_size,
+        num_frames=num_frames,
+        drop_path=drop_path,
+    )
+    if pretrained:
+        logger.info("load pretrained weights")
+        state_dict = torch.load(_MODELS["ViT-L/14_336"], map_location="cpu", weights_only=True)
+        load_state_dict(
+            model,
+            state_dict,
+            input_resolution=input_resolution,
+            patch_size=14,
+            center=center,
+        )
+    return model.eval()
+
+
+def interpolate_pos_embed_vit(state_dict, new_model):
+    key = "vision_encoder.temporal_positional_embedding"
+    if key in state_dict:
+        vision_temp_embed_new = new_model.state_dict()[key]
+        vision_temp_embed_new = vision_temp_embed_new.unsqueeze(
+            2
+        )  # [1, n, d] -> [1, n, 1, d]
+        vision_temp_embed_old = state_dict[key]
+        vision_temp_embed_old = vision_temp_embed_old.unsqueeze(2)
+
+        state_dict[key] = load_temp_embed_with_mismatch(
+            vision_temp_embed_old, vision_temp_embed_new, add_zero=False
+        ).squeeze(2)
+
+    key = "text_encoder.positional_embedding"
+    if key in state_dict:
+        text_temp_embed_new = new_model.state_dict()[key]
+        text_temp_embed_new = text_temp_embed_new.unsqueeze(0).unsqueeze(
+            2
+        )  # [n, d] -> [1, n, 1, d]
+        text_temp_embed_old = state_dict[key]
+        text_temp_embed_old = text_temp_embed_old.unsqueeze(0).unsqueeze(2)
+
+        state_dict[key] = (
+            load_temp_embed_with_mismatch(
+                text_temp_embed_old, text_temp_embed_new, add_zero=False
+            )
+            .squeeze(2)
+            .squeeze(0)
+        )
+    return state_dict
+
+
+if __name__ == "__main__":
+    import time
+
+    import numpy as np
+    from fvcore.nn import FlopCountAnalysis, flop_count_table
+
+    seed = 4217
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+    num_frames = 8
+
+    # model = clip_joint_b16(pretrained=True, kernel_size=1, num_frames=8, num_classes=400, drop_path=0.1)
+    # logger.info(model)
+    model = clip_joint_l14(pretrained=False)
+
+    flops = FlopCountAnalysis(model, torch.rand(1, 3, num_frames, 224, 224))
+    s = time.time()
+    logger.info(flop_count_table(flops, max_depth=1))
+    logger.info(time.time() - s)
+    # logger.info(model(torch.rand(1, 3, num_frames, 224, 224)).shape)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/video_metrics.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/video_metrics.py
new file mode 100644
index 0000000000000000000000000000000000000000..1a977bb5976bf22febfb03e80da91452baa8fca6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/SArena/video/video_metrics.py
@@ -0,0 +1,123 @@
+import math
+from dataclasses import dataclass
+from typing import Callable, Dict
+
+from ..base_metric import BaseMetric
+from ..token_length import TokenLengthCalculator
+from .CLIP_video import ViCLIPCalculator
+from .DINO_video import DINOVideoCalculator
+from .FVD import FVDCalculator
+from .LPIPS_video import LPIPSVideoCalculator
+from .PSNR_video import PSNRVideoCalculator
+from .SSIM_video import SSIMVideoCalculator
+
+
+@dataclass
+class VideoMetricsConfig:
+    use_FVD: bool = False
+    use_ViCLIP_T2V: bool = False
+    use_ViCLIP_V2V: bool = False
+    use_DINO_Video: bool = False
+    use_SSIM_Video: bool = False
+    use_LPIPS_Video: bool = False
+    use_PSNR_Video: bool = False
+    use_token_length: bool = False
+
+
+VICLIP_MODED_PATH = "https://huggingface.co/OpenGVLab/VBench_Used_Models/resolve/main/ViClip-InternVid-10M-FLT.pth"
+
+
+class InternSVGVideoMetrics:
+    def __init__(self, config: VideoMetricsConfig, tokenizer_path: str):
+        self.config = config
+
+        _registry: Dict[str, tuple[str, Callable[[], BaseMetric]]] = {
+            "use_FVD": (
+                "FVD",
+                lambda: FVDCalculator(
+                    model_name="i3d", resolution=128, sequence_length=16
+                ),
+            ),
+            "use_ViCLIP_T2V": (
+                "ViCLIP-T2V",
+                lambda: ViCLIPCalculator(
+                    model_name="viclip",
+                    ckpt_path=VICLIP_MODED_PATH,
+                    num_frames=8,
+                    target_size=(224, 224),
+                    task_type="T2V",
+                ),
+            ),
+            "use_ViCLIP_V2V": (
+                "ViCLIP-V2V",
+                lambda: ViCLIPCalculator(
+                    model_name="viclip",
+                    ckpt_path=VICLIP_MODED_PATH,
+                    num_frames=8,
+                    target_size=(224, 224),
+                    task_type="V2V",
+                ),
+            ),
+            "use_DINO_Video": (
+                "DINO-Video",
+                lambda: DINOVideoCalculator(num_frames=8, batch_size=64),
+            ),
+            "use_SSIM_Video": ("SSIM-Video", lambda: SSIMVideoCalculator(num_frames=8)),
+            "use_LPIPS_Video": (
+                "LPIPS-Video",
+                lambda: LPIPSVideoCalculator(num_frames=8, batch_size=32),
+            ),
+            "use_PSNR_Video": ("PSNR-Video", lambda: PSNRVideoCalculator(num_frames=8)),
+            "use_token_length": (
+                "Token-Length",
+                lambda: TokenLengthCalculator(tokenizer_path=tokenizer_path),
+            ),
+        }
+
+        self.active_metrics = {}
+        for flag, (metric_name, builder) in _registry.items():
+            if getattr(self.config, flag, False):
+                self.active_metrics[metric_name] = builder()
+
+    def reset(self):
+        for metric in self.active_metrics.values():
+            metric.reset()
+
+    @staticmethod
+    def _normalize_metric_result(result):
+        if isinstance(result, tuple):
+            return result[0]
+        return result
+
+    @staticmethod
+    def _is_valid_scalar(value):
+        try:
+            return not math.isnan(float(value))
+        except (TypeError, ValueError):
+            return False
+
+    def calculate_metrics(self, batch):
+        avg_results_dict = {}
+
+        for metric_name, metric in self.active_metrics.items():
+            print(f"Calculating {metric_name}...")
+            metric_result = self._normalize_metric_result(metric.calculate_score(batch))
+            if isinstance(metric_result, dict):
+                avg_results_dict[metric_name] = metric_result
+            elif self._is_valid_scalar(metric_result):
+                avg_results_dict[metric_name] = float(metric_result)
+
+        return avg_results_dict
+
+    def summarize_metrics(self):
+        summary_scores = {}
+        for name, calc in self.active_metrics.items():
+            summary_scores[name] = calc.get_average_score()
+        return summary_scores
+
+    def __len__(self) -> int:
+        return len(self.active_metrics)
+
+    def __repr__(self) -> str:
+        metrics_list = ", ".join(self.active_metrics.keys())
+        return f"<InternSVGVideoMetrics active=[{metrics_list}]>"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/README.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..99572ef587eb9e5689199ba965f399a16eeb4b1a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/README.md
@@ -0,0 +1,59 @@
+# CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy
+
+## Introduction
+
+Please refer to our [GitHub](https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/Benchmarks/CC-OCR) for more information.
+
+## Running Scripts
+
+Once the environment is ready, execute the following script from the root directory of VLMEvalKit
+to perform inference and evaluation tasks in batch.
+
+```shell
+MODEL_NAME="QwenVLMax"
+OUTPUT_DIR="/your/path/to/output_dir"
+
+SUB_OUTPUT_DIR=${OUTPUT_DIR}/multi_scene_ocr
+python run.py --data CCOCR_MultiSceneOcr_Cord CCOCR_MultiSceneOcr_Funsd CCOCR_MultiSceneOcr_Iam CCOCR_MultiSceneOcr_ZhDoc CCOCR_MultiSceneOcr_ZhHandwriting CCOCR_MultiSceneOcr_Hieragent CCOCR_MultiSceneOcr_Ic15 CCOCR_MultiSceneOcr_Inversetext CCOCR_MultiSceneOcr_Totaltext CCOCR_MultiSceneOcr_ZhScene CCOCR_MultiSceneOcr_UgcLaion CCOCR_MultiSceneOcr_ZhDense CCOCR_MultiSceneOcr_ZhVertical --model ${MODEL_NAME} --work-dir ${SUB_OUTPUT_DIR} --verbose
+python vlmeval/dataset/utils/ccocr_evaluator/common.py ${SUB_OUTPUT_DIR}
+
+SUB_OUTPUT_DIR=${OUTPUT_DIR}/multi_lan_ocr
+python run.py --data CCOCR_MultiLanOcr_Arabic CCOCR_MultiLanOcr_French CCOCR_MultiLanOcr_German CCOCR_MultiLanOcr_Italian CCOCR_MultiLanOcr_Japanese CCOCR_MultiLanOcr_Korean CCOCR_MultiLanOcr_Portuguese CCOCR_MultiLanOcr_Russian CCOCR_MultiLanOcr_Spanish CCOCR_MultiLanOcr_Vietnamese --model ${MODEL_NAME} --work-dir ${SUB_OUTPUT_DIR} --verbose
+python vlmeval/dataset/utils/ccocr_evaluator/common.py ${SUB_OUTPUT_DIR}
+
+SUB_OUTPUT_DIR=${OUTPUT_DIR}/doc_parsing
+python run.py --data CCOCR_DocParsing_DocPhotoChn CCOCR_DocParsing_DocPhotoEng CCOCR_DocParsing_DocScanChn CCOCR_DocParsing_DocScanEng CCOCR_DocParsing_TablePhotoChn CCOCR_DocParsing_TablePhotoEng CCOCR_DocParsing_TableScanChn CCOCR_DocParsing_TableScanEng CCOCR_DocParsing_MolecularHandwriting CCOCR_DocParsing_FormulaHandwriting --model ${MODEL_NAME} --work-dir ${SUB_OUTPUT_DIR} --verbose
+python vlmeval/dataset/utils/ccocr_evaluator/common.py ${SUB_OUTPUT_DIR}
+
+SUB_OUTPUT_DIR=${OUTPUT_DIR}/kie
+python run.py --data CCOCR_Kie_Sroie2019Word CCOCR_Kie_Cord CCOCR_Kie_EphoieScut CCOCR_Kie_Poie CCOCR_Kie_ColdSibr CCOCR_Kie_ColdCell --model ${MODEL_NAME} --work-dir ${SUB_OUTPUT_DIR} --verbose
+python vlmeval/dataset/utils/ccocr_evaluator/common.py ${SUB_OUTPUT_DIR}
+```
+
+## Example Output
+The evaluation results will be saved in `${SUB_OUTPUT_DIR}/summary.md`. For example, for the KIE subset,
+the output is as follows:
+
+| exp_name(f1_score) |   COLD_CELL |   COLD_SIBR |   CORD |   EPHOIE_SCUT |   POIE |   sroie2019_word |   summary |
+|:-------------------|------------:|------------:|-------:|--------------:|-------:|-----------------:|----------:|
+| QwenVLMax          |       81.01 |       72.46 |  69.33 |          71.2 |  60.85 |            76.37 |     71.87 |
+
+
+## Citation
+If you find our work helpful, feel free to give us a cite.
+
+```
+@misc{yang2024ccocr,
+      title={CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy},
+      author={Zhibo Yang and Jun Tang and Zhaohai Li and Pengfei Wang and Jianqiang Wan and Humen Zhong and Xuejing Liu and Mingkun Yang and Peng Wang and Shuai Bai and LianWen Jin and Junyang Lin},
+      year={2024},
+      eprint={2412.02210},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2412.02210},
+}
+```
+
+## Contact Us
+
+If you have any questions, feel free to send an email to: wpf272043@alibaba-inc.com or xixing.tj@alibaba-inc.com
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..b8cdab5d2101b16ee6e5d9af43fbe00ba77e2e07
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/__init__.py
@@ -0,0 +1,11 @@
+from .common import summary  # noqa: F401
+from .doc_parsing_evaluator import ParsingEvaluator
+from .kie_evaluator import KieEvaluator
+from .ocr_evaluator import OcrEvaluator
+
+evaluator_map_info = {
+    "kie": KieEvaluator("kie"),
+    "doc_parsing": ParsingEvaluator("doc_parsing"),
+    "multi_lan_ocr": OcrEvaluator("multi_lan_ocr"),
+    "multi_scene_ocr": OcrEvaluator("multi_scene_ocr")
+}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/common.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/common.py
new file mode 100644
index 0000000000000000000000000000000000000000..ee3e0ea35ff165931d5fcd294d9fdfd30aa493cf
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/common.py
@@ -0,0 +1,223 @@
+import json
+import os
+import sys
+import time
+from abc import abstractmethod
+
+from tabulate import tabulate
+
+
+def pick_response_text(json_path):
+    """
+    """
+    try:
+        with open(json_path, "r") as f:
+            json_data = json.load(f)
+    except Exception as e:
+        print("--> file error: msg: {}, path: {}".format(e, json_path))
+        return None
+
+    for required_key in ["model_name", "response"]:
+        if required_key not in json_data:
+            print("--> required key not exists, name: {}, path: {}".format(required_key, json_path))
+            return None
+
+    model_name = json_data["model_name"]
+    model_response = json_data["response"]
+
+    response_text = None
+    if model_name.startswith("gpt") or model_name.startswith("o1"):
+        response_text = model_response.get("data", {}).get("response", {}).get("choices", [{}])[0].get("message", {}).get("content", None)  # noqa: E501
+    elif model_name.startswith("local_"):
+        response_text = model_response
+    else:
+        if model_name.startswith("claude"):
+            content_list = model_response.get("content", None)
+        elif model_name.startswith("gemini"):
+            content_list = model_response.get("candidates", [{}])[0].get("content", {}).get("parts", None)
+        elif model_name.startswith("qwen"):
+            content_list = model_response.get("output", {}).get("choices", [{}])[0].get("message", {}).get("content", None)  # noqa: E501
+        else:
+            raise NotImplementedError("The pick_response_text NOT implemented for model: {}".format(model_name))
+
+        if isinstance(content_list, list) and len(content_list) > 0:
+            response_text = content_list[0].get("text", None)
+
+    if response_text is None:
+        print("--> [error][{}] text pick error, path: {}".format(model_name, json_path))
+    return response_text
+
+
+def load_response_from_dir(res_dir):
+    """
+    """
+    response_info = {}
+    for file_name in os.listdir(res_dir):
+        file_path = os.path.abspath(os.path.join(res_dir, file_name))
+        if not file_name.endswith(".json"):
+            print("--> skip: result file should be a json: but got: {}".format(file_path))
+            continue
+
+        response_text = pick_response_text(file_path)
+        if response_text is None:
+            continue
+
+        file_name_wo_ext, ext = os.path.splitext(file_name)
+        response_info[file_name_wo_ext] = response_text
+    return response_info
+
+
+class BaseMetric(object):
+    """ BaseMetric """
+    """ OCRMetric """
+    def __init__(self, group_name, **kwargs):
+        self.group_name = group_name
+        self.kwargs = kwargs
+
+    def response_post_func(self, response_text, **kwargs):
+        return response_text
+
+    @abstractmethod
+    # Given the prediction and gt, return the evaluation results in the format of a dictionary
+    # results should contain a 'summary' key, for example:
+    # {
+    #     "summary": {
+    #         "f1-score": 99.99,
+    #         "metric_name": "metric_value"  # used for summary，only metric info could be placed in this dict.
+    #     },
+    #     "your other info": "xxx"
+    # }
+    def evaluate(self, response_info, gt_info, normalize_func=None, **kwargs):
+        pass
+
+    def __call__(self, pdt_res_dir, gt_info, with_response_ratio=True, **kwargs):
+        if isinstance(pdt_res_dir, dict):
+            raw_response_info = pdt_res_dir
+        elif os.path.exists(pdt_res_dir) and os.path.isdir(pdt_res_dir):
+            raw_response_info = load_response_from_dir(pdt_res_dir)
+        else:
+            return ValueError("invalid input: response dict or folder are required, but got {}".format(pdt_res_dir))
+
+        post_error_list, response_info = [], {}
+        response_error_list = list(gt_info.keys() - raw_response_info.keys())
+        for file_name, single_pdt_str in raw_response_info.items():
+            single_pdt_str = self.response_post_func(single_pdt_str, **kwargs)
+            if single_pdt_str is None:
+                post_error_list.append(file_name)
+                continue
+            response_info[file_name] = single_pdt_str
+
+        meta_info = {
+            "gt_total_num": len(gt_info), "pdt_total_num": len(response_info),
+            "post_error_list": post_error_list, "response_error_list": response_error_list,
+        }
+        eval_info = self.evaluate(response_info, gt_info, **kwargs)
+
+        # add response_success_ratio
+        if "summary" in eval_info and with_response_ratio:
+            success_ratio = (len(response_info) + len(post_error_list)) / (len(gt_info) + 1e-9)
+            eval_info["summary"].update({"response_success_ratio": success_ratio})
+        return meta_info, eval_info
+
+
+def summary(index_path, exp_dir_base, is_weighted_sum=False):
+    """
+    """
+    with open(index_path, "r") as f:
+        data_list = json.load(f)
+
+    all_data_info = {}
+    for data_info_item in data_list:
+        data_name = data_info_item["dataset"]
+        if not data_info_item.get("release", True):
+            continue
+        all_data_info[data_name] = data_info_item
+    dataset_list = list(all_data_info.keys())
+    summary_path = summary_multi_exp(exp_dir_base, dataset_list, is_weighted_sum=is_weighted_sum)
+    return summary_path
+
+
+def summary_multi_exp(exp_dir_base, dataset_list=None, is_weighted_sum=False):
+    """
+    """
+    if dataset_list is None:
+        all_dataset_name = []
+        for exp_name in os.listdir(exp_dir_base):
+            dir_status_path = os.path.join(exp_dir_base, exp_name, "status.json")
+            if not os.path.exists(dir_status_path):
+                continue
+            with open(dir_status_path, "r") as f:
+                data_status_info = json.load(f)
+            all_dataset_name.extend(data_status_info.keys())
+        dataset_list = sorted(set(all_dataset_name))
+
+    # summary main code
+    all_evaluate_info, _ = {}, 0
+    for exp_name in os.listdir(exp_dir_base):
+        dir_status_path = os.path.join(exp_dir_base, exp_name, "status.json")
+        if not os.path.exists(dir_status_path):
+            print("--> skip: status.json not exist: {}".format(dir_status_path))
+            continue
+
+        with open(dir_status_path, "r") as f:
+            all_status_info = json.load(f)
+
+        for data_name in dataset_list:
+            total_num = all_status_info.get(data_name, {}).get("config", {}).get("num", "-1")
+            summary_info = all_status_info.get(data_name, {}).get("evaluation", {}).get("summary", {})
+            for metric_name, metric_value in summary_info.items():
+                if metric_name not in all_evaluate_info:
+                    all_evaluate_info[metric_name] = {}
+                if exp_name not in all_evaluate_info[metric_name]:
+                    all_evaluate_info[metric_name][exp_name] = {}
+                all_evaluate_info[metric_name][exp_name][data_name] = (metric_value, total_num)
+
+    all_table_md = []
+    for metric_name, metric_info in all_evaluate_info.items():
+        formatted_time = time.strftime("%Y-%m-%d %H:%M", time.localtime(time.time()))
+        summary_line_list = []
+        summary_key_name = "summary(weighted)" if is_weighted_sum else "summary"
+        summary_head = [f"exp_name({metric_name}_{formatted_time})"] + dataset_list + [summary_key_name]
+        for exp_name, data_eval_info in metric_info.items():
+            summary_line = [exp_name, ]
+
+            all_metric_value = 0
+            is_summary_valid, all_total_num, all_weighted_metric = True, 0, 0
+            for data_name in dataset_list:
+                metric_value, total_num = data_eval_info.get(data_name, ("-1", "-1"))
+                summary_line.append("{:.2f}".format(float(metric_value) * 100))
+                if str(metric_value) == "-1" or str(metric_value) == "-1":
+                    is_summary_valid = False
+                    continue
+
+                all_total_num += float(total_num)
+                all_weighted_metric += float(total_num) * float(metric_value)
+                all_metric_value += float(metric_value)
+
+            summary_value_valid = ((all_weighted_metric / (all_total_num + 1e-9)) * 100) if is_weighted_sum \
+                else (all_metric_value / (len(dataset_list) + 1e-9) * 100)
+            summary_value = "-" if not is_summary_valid else "{:.2f}".format(summary_value_valid)
+            summary_line.append(summary_value)
+            summary_line_list.append(summary_line)
+
+        md_table_info = tabulate(summary_line_list, headers=summary_head, tablefmt='pipe')
+        all_table_md.append(md_table_info)
+
+    print("\n\n".join(all_table_md))
+    summary_path = os.path.abspath(os.path.join(exp_dir_base, "summary.md"))
+    with open(summary_path, "w") as f:
+        f.write("\n\n".join(all_table_md))
+    return summary_path
+
+
+if __name__ == '__main__':
+    if len(sys.argv) != 2:
+        print("Usage: python {} exp_base_dir".format(__file__))
+        exit(-1)
+    else:
+        print('--> info: {}'.format(sys.argv))
+        exp_base_dir = sys.argv[1]
+
+    summary_path = summary_multi_exp(exp_base_dir, dataset_list=None, is_weighted_sum=False)
+    print("--> info: summary saved at : {}".format(summary_path))
+    print("happy coding.")
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/doc_parsing_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/doc_parsing_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..dbcb30b69b4990d2d3014f535339ae6215f4b4d1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/doc_parsing_evaluator.py
@@ -0,0 +1,256 @@
+import re
+from collections import deque
+
+import nltk
+from apted import APTED, Config
+from apted.helpers import Tree
+from tqdm import tqdm
+
+# local import
+from .common import BaseMetric
+
+# 移除指定的LaTeX命令
+patterns = [
+    r'\\documentclass\{.*?\}',
+    r'\\usepackage\[.*?\]\{.*?\}',
+    r'\\usepackage\{.*?\}',
+    r'\\geometry\{.*?\}',
+    r'\\begin\{document\}',
+    r'\\end\{document\}',
+    r'\\noindent'
+]
+
+
+class TableTree(Tree):
+    """
+    # Copyright 2020 IBM
+    # Author: peter.zhong@au1.ibm.com
+    # License:  Apache 2.0 License.
+    """
+    def __init__(self, tag, colspan=None, rowspan=None, content=None, *children):
+        self.tag = tag
+        self.colspan = colspan
+        self.rowspan = rowspan
+        self.content = content
+        self.children = list(children)
+
+    def bracket(self):
+        """Show tree using brackets notation"""
+        if self.tag == "td":
+            result = '"tag": %s, "colspan": %d, "rowspan": %d, "text": %s' % (
+                self.tag,
+                self.colspan,
+                self.rowspan,
+                self.content,
+            )
+        else:
+            result = '"tag": %s' % self.tag
+        for child in self.children:
+            result += child.bracket()
+        return "{{{}}}".format(result)
+
+
+class CustomConfig(Config):
+    """
+    # Copyright 2020 IBM
+    # Author: peter.zhong@au1.ibm.com
+    # License:  Apache 2.0 License.
+    """
+    def rename(self, node1, node2):
+        """Compares attributes of trees"""
+        # print(node1.tag)
+        if (
+            (node1.tag != node2.tag)
+            or (node1.colspan != node2.colspan)
+            or (node1.rowspan != node2.rowspan)
+        ):
+            return 1.0
+        if node1.tag == "td":
+            if node1.content or node2.content:
+                return nltk.edit_distance(node1.content, node2.content) / max(len(node1.content), len(node2.content))
+        return 0.0
+
+
+class TEDS(object):
+    """Tree Edit Distance basead Similarity
+    # Copyright 2020 IBM
+    # Author: peter.zhong@au1.ibm.com
+    # License:  Apache 2.0 License.
+    """
+    def __init__(self, structure_only=False, n_jobs=1, ignore_nodes=None):
+        assert isinstance(n_jobs, int) and (
+            n_jobs >= 1
+        ), "n_jobs must be an integer greather than 1"
+        self.structure_only = structure_only
+        self.n_jobs = n_jobs
+        self.ignore_nodes = ignore_nodes
+        self.__tokens__ = []
+
+    def tokenize(self, node):
+        """Tokenizes table cells"""
+        self.__tokens__.append("<%s>" % node.tag)
+        if node.text is not None:
+            self.__tokens__ += list(node.text)
+        for n in node.getchildren():
+            self.tokenize(n)
+        if node.tag != "unk":
+            self.__tokens__.append("</%s>" % node.tag)
+        if node.tag != "td" and node.tail is not None:
+            self.__tokens__ += list(node.tail)
+
+    def load_html_tree(self, node, parent=None):
+        """Converts HTML tree to the format required by apted"""
+        global __tokens__
+        if node.tag == "td":
+            if self.structure_only:
+                cell = []
+            else:
+                self.__tokens__ = []
+                self.tokenize(node)
+                cell = self.__tokens__[1:-1].copy()
+            new_node = TableTree(
+                node.tag,
+                int(node.attrib.get("colspan", "1")),
+                int(node.attrib.get("rowspan", "1")),
+                cell,
+                *deque(),
+            )
+        else:
+            new_node = TableTree(node.tag, None, None, None, *deque())
+        if parent is not None:
+            parent.children.append(new_node)
+        if node.tag != "td":
+            for n in node.getchildren():
+                self.load_html_tree(n, new_node)
+        if parent is None:
+            return new_node
+
+    def evaluate(self, pred, true):
+        """Computes TEDS score between the prediction and the ground truth of a
+        given sample
+        """
+        # try_import("lxml")
+        from lxml import etree, html
+        if (not pred) or (not true):
+            return 0.0
+
+        parser = html.HTMLParser(remove_comments=True, encoding="utf-8")
+        pred = html.fromstring(pred, parser=parser)
+        true = html.fromstring(true, parser=parser)
+        if pred.xpath("body/table") and true.xpath("body/table"):
+            pred = pred.xpath("body/table")[0]
+            true = true.xpath("body/table")[0]
+            if self.ignore_nodes:
+                etree.strip_tags(pred, *self.ignore_nodes)
+                etree.strip_tags(true, *self.ignore_nodes)
+            n_nodes_pred = len(pred.xpath(".//*"))
+            n_nodes_true = len(true.xpath(".//*"))
+            n_nodes = max(n_nodes_pred, n_nodes_true)
+            tree_pred = self.load_html_tree(pred)
+            tree_true = self.load_html_tree(true)
+            distance = APTED(
+                tree_pred, tree_true, CustomConfig()
+            ).compute_edit_distance()
+            return 1.0 - (float(distance) / n_nodes)
+        else:
+            return 0.0
+
+
+class ParsingEvaluator(BaseMetric):
+    def response_post_func(self, response_text, **kwargs):
+        return response_text
+
+    def evaluate(self, response_info, gt_info, **kwargs):
+        op = kwargs['op']
+        if op == 'doc':
+            score = self.eval_doc(response_info, gt_info)
+        elif op == 'table':
+            score = self.eval_table(response_info, gt_info)
+        elif op in ['molecular', "formula"]:
+            score = self.eval_formula(response_info, gt_info, op_name=op)
+        else:
+            raise ValueError(f'doc parsing unsupported op: {op}')
+
+        # summary info
+        eval_info = {"summary": {"score": score}}
+        return eval_info
+
+    def eval_doc(self, response_info, gt_info):
+        results = []
+        for img_name, gt in tqdm(gt_info.items()):
+            if img_name not in response_info:
+                results.append(0)
+                continue
+
+            pred = response_info[img_name]
+            for pattern in patterns:
+                pred = re.sub(pattern, '', pred)
+
+            try:
+                pred = pred.split('```')[1]
+            except Exception:
+                pass
+
+            pred = pred.replace('```latex', '')
+            pred = pred.replace('```', '')
+
+            pred = pred.replace(' ', '').replace('\n', '')
+            gt = gt.replace(' ', '').replace('\n', '')
+
+            edit_dist = nltk.edit_distance(pred, gt) / max(len(pred), len(gt))
+            results.append(1 - edit_dist)
+
+        score = sum(results) / len(results)
+        return score
+
+    def eval_table(self, response_info, gt_info):
+        teds = TEDS(structure_only=False, n_jobs=1)
+        results = []
+        for img_name, gt in tqdm(gt_info.items()):
+            if img_name not in response_info:
+                results.append(0)
+                continue
+
+            pred = response_info[img_name]
+            for pattern in patterns:
+                pred = re.sub(pattern, '', pred)
+
+            try:
+                pred = pred.split('```html')[1]
+            except Exception:
+                pass
+
+            pred = pred.replace('```', '')
+            pred = pred.replace(' ', '').replace('\n', '').replace('，', ',')
+            gt = gt.replace(' ', '').replace('\n', '')
+
+            pred_html = '<html><body>{}</body></html>'.format(pred)
+            gt_html = '<html><body>{}</body></html>'.format(gt)
+            results.append(teds.evaluate(pred_html, gt_html))
+
+        score = sum(results) / len(results)
+        return score
+
+    def eval_formula(self, response_info, gt_info, op_name='formula'):
+        results = []
+        for img_name, gt in tqdm(gt_info.items()):
+            if img_name not in response_info:
+                results.append(0)
+                continue
+
+            pred = response_info[img_name]
+
+            if op_name == 'formula':
+                pred = pred.replace("\n", " ").replace("```latex", "").replace("```", "").replace("\t", " ").replace(" ", "")  # noqa: E501
+                gt = gt.replace(" ", "")
+            elif op_name == 'molecular':
+                pred = pred.replace("\n", "").replace(" ", "").replace("<smiles>", "").replace("</smiles>", "")
+                gt = gt.replace(" ", "")
+            edit_dist = nltk.edit_distance(pred, gt) / max(len(pred), len(gt))
+            results.append(1 - edit_dist)
+        score = sum(results) / len(results)
+        return score
+
+
+if __name__ == '__main__':
+    pass
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..50a6d6fc6250743db10e8fad6fadfb58497a2844
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py
@@ -0,0 +1,407 @@
+
+"""
+Donut
+Copyright (c) 2022-present NAVER Corp.
+MIT License
+"""
+import ast
+import json
+import re
+from collections import Counter
+from typing import Any, Dict, List, Union
+
+import zss
+from json_repair import repair_json
+from nltk import edit_distance
+from zss import Node
+
+# local import
+from .common import BaseMetric
+
+
+def flatten(data: dict):
+    """
+    Convert Dictionary into Non-nested Dictionary
+    Example:
+        input(dict)
+            {
+                "menu": [
+                    {"name" : ["cake"], "count" : ["2"]},
+                    {"name" : ["juice"], "count" : ["1"]},
+                ]
+            }
+        output(list)
+            [
+                ("menu.name", "cake"),
+                ("menu.count", "2"),
+                ("menu.name", "juice"),
+                ("menu.count", "1"),
+            ]
+    """
+    flatten_data = list()
+
+    def _flatten(value, key=""):
+        if type(value) is dict:
+            for child_key, child_value in value.items():
+                _flatten(child_value, f"{key}.{child_key}" if key else child_key)
+        elif type(value) is list:
+            for value_item in value:
+                _flatten(value_item, key)
+        else:
+            flatten_data.append((key, value))
+
+    _flatten(data)
+    return flatten_data
+
+
+def update_cost(node1: Node, node2: Node):
+    """
+    Update cost for tree edit distance.
+    If both are leaf node, calculate string edit distance between two labels (special token '<leaf>' will be ignored).
+    If one of them is leaf node, cost is length of string in leaf node + 1.
+    If neither are leaf node, cost is 0 if label1 is same with label2 othewise 1
+    """
+    label1 = node1.label
+    label2 = node2.label
+    label1_leaf = "<leaf>" in label1
+    label2_leaf = "<leaf>" in label2
+    if label1_leaf and label2_leaf:
+        return edit_distance(label1.replace("<leaf>", ""), label2.replace("<leaf>", ""))
+    elif not label1_leaf and label2_leaf:
+        return 1 + len(label2.replace("<leaf>", ""))
+    elif label1_leaf and not label2_leaf:
+        return 1 + len(label1.replace("<leaf>", ""))
+    else:
+        return int(label1 != label2)
+
+
+def insert_and_remove_cost(node: Node):
+    """
+    Insert and remove cost for tree edit distance.
+    If leaf node, cost is length of label name.
+    Otherwise, 1
+    """
+    label = node.label
+    if "<leaf>" in label:
+        return len(label.replace("<leaf>", ""))
+    else:
+        return 1
+
+
+def normalize_dict(data: Union[Dict, List, Any]):
+    """
+    Sort by value, while iterate over element if data is list
+    """
+    # if not data:
+    #     return {}
+
+    if isinstance(data, dict):
+        new_data = dict()
+        for key in sorted(data.keys(), key=lambda k: (len(k), k)):
+            value = normalize_dict(data[key])
+            if value:
+                if not isinstance(value, list):
+                    value = [value]
+                new_data[key] = value
+
+    elif isinstance(data, list):
+        if all(isinstance(item, dict) for item in data):
+            new_data = []
+            for item in data:
+                item = normalize_dict(item)
+                if item:
+                    new_data.append(item)
+        else:
+            new_data = [str(item).strip() for item in data if type(item) in {str, int, float} and str(item).strip()]
+    else:
+        new_data = [str(data).strip()]
+    return new_data
+
+
+def _load_json_like(text: Any):
+    if isinstance(text, (dict, list)):
+        return text
+    if not isinstance(text, str):
+        return text
+
+    parsers = (
+        json.loads,
+        lambda s: repair_json(s, return_objects=True),
+        ast.literal_eval,
+    )
+    last_error = None
+    for parser in parsers:
+        try:
+            parsed = parser(text)
+            if isinstance(parsed, (dict, list)):
+                return parsed
+        except Exception as err:
+            last_error = err
+    if last_error is not None:
+        raise last_error
+    return text
+
+
+def cal_f1_all(preds, answers):
+    """
+    Calculate global F1 accuracy score (field-level, micro-averaged) by counting all true positives,
+    false negatives and false positives
+    """
+    metric_info, error_info = {}, {}
+    total_tp, total_fn_or_fp = 0, 0
+    for file_name, answer in answers.items():
+        sample_error_info = {"fp": [], "fn": [], "tp": []}
+        pred = preds.get(file_name, {})
+        pred, answer = flatten(normalize_dict(pred)), flatten(normalize_dict(answer))
+        for field in pred:
+            field_name = field[0]
+            if field_name not in metric_info:
+                metric_info[field_name] = {"total_tp": 0, "total_fn_or_fp": 0}
+            if field in answer:
+                total_tp += 1
+                metric_info[field_name]["total_tp"] += 1
+                sample_error_info["tp"].append(field)
+                answer.remove(field)
+            else:
+                total_fn_or_fp += 1
+                metric_info[field_name]["total_fn_or_fp"] += 1
+                sample_error_info["fp"].append(field)
+
+        total_fn_or_fp += len(answer)
+        for field in answer:
+            field_name = field[0]
+            if field_name not in metric_info:
+                metric_info[field_name] = {"total_tp": 0, "total_fn_or_fp": 0}
+            metric_info[field_name]["total_fn_or_fp"] += 1
+            sample_error_info["fn"].append(field)
+
+        sample_error_num = sum([len(v) for k, v in sample_error_info.items() if k != "tp"])
+        if sample_error_num > 0:
+            sample_error_info["error_num"] = sample_error_num
+            error_class_list = ["counter_" + x[0] for x in (sample_error_info["fn"] + sample_error_info["fp"])]
+            counter = Counter(error_class_list)
+            sample_error_info["error_info"] = dict(counter)
+            error_info[file_name] = sample_error_info
+
+    # summary
+    for field_name, field_info in metric_info.items():
+        field_tp, field_fn_or_fp = field_info["total_tp"], field_info["total_fn_or_fp"]
+        metric_info[field_name]["acc"] = field_tp / (field_tp + field_fn_or_fp / 2 + 1e-6)
+
+    print("donut_evaluator: total_tp: {}, total_fn_or_fp: {}, ptd_num: {}, gt_num: {}".format(total_tp, total_fn_or_fp,
+                                                                                              len(preds), len(answers)))
+    error_info = {k: v for k, v in
+                  sorted(error_info.items(), key=lambda item: item[1].get("error_num", 0), reverse=True)}
+    metric_info = {k: v for k, v in
+                   sorted(metric_info.items(), key=lambda item: item[1].get("total_fn_or_fp", 0), reverse=True)}
+    return total_tp / (total_tp + total_fn_or_fp / 2 + 1e-6), metric_info, error_info
+
+
+def construct_tree_from_dict(data: Union[Dict, List], node_name: str = None):
+    """
+    Convert Dictionary into Tree
+
+    Example:
+        input(dict)
+
+            {
+                "menu": [
+                    {"name" : ["cake"], "count" : ["2"]},
+                    {"name" : ["juice"], "count" : ["1"]},
+                ]
+            }
+
+        output(tree)
+                                 <root>
+                                   |
+                                 menu
+                                /    \
+                         <subtree>  <subtree>
+                        /      |     |      \
+                     name    count  name    count
+                    /         |     |         \
+              <leaf>cake  <leaf>2  <leaf>juice  <leaf>1
+     """
+    if node_name is None:
+        node_name = "<root>"
+
+    node = Node(node_name)
+
+    if isinstance(data, dict):
+        for key, value in data.items():
+            kid_node = construct_tree_from_dict(value, key)
+            node.addkid(kid_node)
+    elif isinstance(data, list):
+        if all(isinstance(item, dict) for item in data):
+            for item in data:
+                kid_node = construct_tree_from_dict(
+                    item,
+                    "<subtree>",
+                )
+                node.addkid(kid_node)
+        else:
+            for item in data:
+                node.addkid(Node(f"<leaf>{item}"))
+    else:
+        raise Exception(data, node_name)
+    return node
+
+
+def cal_acc(pred: dict, answer: dict):
+    """
+    Calculate normalized tree edit distance(nTED) based accuracy.
+    1) Construct tree from dict,
+    2) Get tree distance with insert/remove/update cost,
+    3) Divide distance with GT tree size (i.e., nTED),
+    4) Calculate nTED based accuracy. (= max(1 - nTED, 0 ).
+    """
+    pred = construct_tree_from_dict(normalize_dict(pred))
+    answer = construct_tree_from_dict(normalize_dict(answer))
+    val1 = zss.distance(
+        pred,
+        answer,
+        get_children=zss.Node.get_children,
+        insert_cost=insert_and_remove_cost,
+        remove_cost=insert_and_remove_cost,
+        update_cost=update_cost,
+        return_operations=False,
+    )
+    val2 = zss.distance(
+        construct_tree_from_dict(normalize_dict({})),
+        answer,
+        get_children=zss.Node.get_children,
+        insert_cost=insert_and_remove_cost,
+        remove_cost=insert_and_remove_cost,
+        update_cost=update_cost,
+        return_operations=False,
+    )
+    return max(0, 1 - val1 / val2)
+
+
+def cal_acc_all(pred_info, answer_info):
+    acc_info, error_info = {}, {}
+    for file_name, answer in answer_info.items():
+        # if file_name not in pred_info:
+        #     print("---> error: pdt not found: {}".format(file_name))
+        #     continue
+        pred = pred_info.get(file_name, {})
+        acc = cal_acc(pred, answer)
+        acc_info[file_name] = acc
+        if acc < 1.0:
+            error_info[file_name] = {"acc": acc, "pred": pred, "answer": answer}
+
+    error_info = {k: v for k, v in sorted(error_info.items(), key=lambda item: item[1].get("acc", 0))}
+    acc_averge = sum(list(acc_info.values())) / (len(acc_info) + 1e-6)
+    return acc_averge, error_info
+
+
+def normalize_values_of_nested_dict(d, normalize_func):
+    """
+    """
+    if isinstance(d, dict):
+        return {k: normalize_values_of_nested_dict(v, normalize_func) for k, v in d.items()}
+    elif isinstance(d, list):
+        return [normalize_values_of_nested_dict(x, normalize_func) if isinstance(x, dict) else x for x in d]
+    elif isinstance(d, str):
+        return normalize_func(d)
+    else:
+        return d
+
+
+def eval_donut(pdt_info, gt_info, normalize_func=None, data_name=None):
+    """
+    """
+    if normalize_func is not None:
+        print("--> info: normalize_func executed.")
+        pdt_info = normalize_values_of_nested_dict(pdt_info, normalize_func)
+        gt_info = normalize_values_of_nested_dict(gt_info, normalize_func)
+
+    f1_score, class_eval_info, error_info = cal_f1_all(pdt_info, gt_info)
+    acc_average, acc_error_info = cal_acc_all(pdt_info, gt_info)
+    eval_info = {"f1_score": f1_score, "acc": acc_average, "class_f1_score": class_eval_info,
+                 "f1_error_info": error_info, "acc_error_info": acc_error_info}
+    print(data_name, "f1_score", f1_score, "acc", acc_average)
+    return eval_info
+
+
+def post_process_to_json(qwen_info_str, file_name=None):
+    try:
+        if "```json" in qwen_info_str:
+            if "```" not in qwen_info_str:
+                qwen_info_str += "```"
+            qwen_info_group = re.search(r'```json(.*?)```', qwen_info_str, re.DOTALL)
+            json_str = qwen_info_group.group(1).strip().replace("\n", "")
+        else:
+            json_str = qwen_info_str.strip().replace("\n", "")
+        json_data = json.loads(json_str)
+        return json_data
+    except Exception as err:  # noqa: F841
+        return None
+
+
+def fullwidth_to_halfwidth(text):
+    # 全角转半角
+    result = ''
+    for char in text:
+        code_point = ord(char)
+        # 全角空格直接转化
+        if code_point == 0x3000:
+            code_point = 0x0020
+        # 其他全角字符（除空格）转换为半角
+        elif 0xFF01 <= code_point <= 0xFF5E:
+            code_point -= 0xFEE0
+        result += chr(code_point)
+    result = result.replace("、", ",")
+    return result
+
+
+def remove_unnecessary_spaces(text):
+    # 去掉中文字符之间的空格
+    text = re.sub(r'(?<=[\u4e00-\u9fff])\s+(?=[\u4e00-\u9fff])', '', text)
+    # 去掉中文和英文、数字之间的空格
+    text = re.sub(r'(?<=[\u4e00-\u9fff])\s+(?=[a-zA-Z0-9])', '', text)
+    text = re.sub(r'(?<=[a-zA-Z0-9])\s+(?=[\u4e00-\u9fff])', '', text)
+    # 去掉符号前的不必要空格，保留符号后的一个空格
+    text = re.sub(r'(?<![0-9])\s*([,.!?:;])\s*', r'\1 ', text)  # 非数字前后的符号
+    # 在数字和英文之间添加空格
+    text = re.sub(r'(?<=[0-9])(?=[a-zA-Z])', ' ', text)
+    text = re.sub(r'(?<=[a-zA-Z])(?=[0-9])', ' ', text)
+    text = re.sub(r'\s+', ' ', text)
+    return text
+
+
+class KieEvaluator(BaseMetric):
+    def response_post_func(self, response_text, **kwargs):
+        response_text = post_process_to_json(response_text, file_name=kwargs.get('file_name', None))
+        return response_text
+
+    def normalize_func(self, text, **kwargs):
+        halfwidth_text = fullwidth_to_halfwidth(str(text))
+        cleaned_text = remove_unnecessary_spaces(halfwidth_text)
+        return cleaned_text
+
+    def evaluate(self, response_info, gt_info, **kwargs):
+        """
+        response_info: dict: {"file_name_1": response, "file_name_2": gt}
+        gt_info: dict: {"file_name_1": gt, "file_name_2": gt}
+        kwargs: dataset index config: {'dataset': 'kie_benchmark_POIE', 'group': 'kie', 'op': 'poie', 'num': 250}
+        """
+        # gt should be a dict for kie task, fix for VLMEvalKit
+        for image_name, label_content in gt_info.items():
+            gt_info[image_name] = _load_json_like(label_content)
+
+        response_info = normalize_values_of_nested_dict(response_info, self.normalize_func)
+        gt_info = normalize_values_of_nested_dict(gt_info, self.normalize_func)
+
+        f1_score, class_eval_info, error_info = cal_f1_all(response_info, gt_info)
+        acc_average, acc_error_info = cal_acc_all(response_info, gt_info)
+
+        # summary info
+        summary_info = {"f1_score": f1_score, "acc": acc_average}
+        eval_info = {"summary": summary_info, "class_f1_score": class_eval_info,
+                     "f1_error_info": error_info, "acc_error_info": acc_error_info}
+        return eval_info
+
+
+if __name__ == '__main__':
+    pass
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/ocr_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/ocr_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..e596269a1d8ea4363b2e565a0c5a8d024e9651d3
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ccocr_evaluator/ocr_evaluator.py
@@ -0,0 +1,103 @@
+import re
+from collections import Counter
+
+# local import
+from .common import BaseMetric
+
+
+def token_normalize(token_text, is_lower=False, is_alphanum_only=False):
+    """
+    """
+    if is_lower:
+        token_text = token_text.lower()
+    if is_alphanum_only:
+        token_text = re.sub('[^A-Za-z0-9]+', '', token_text)
+    return token_text
+
+
+def text_normalize_and_tokenize(text, is_keep_blank=True, is_lower=True, is_alphanum_only=False):
+    text = text.replace("\t", " ").replace("\n", " ").replace("###", "").replace("***", "")
+    text = re.sub(r'\s+', ' ', text)
+    if not is_keep_blank:
+        text = text.replace(" ", "")
+    text_tokens = text.split(" ") if is_keep_blank else list(text)
+    text_token_normalized = [token_normalize(t, is_lower, is_alphanum_only) for t in text_tokens]
+    text_token_normalized = [x for x in text_token_normalized if len(x) > 0]
+    return text_token_normalized
+
+
+def evaluate_single_sample(gts, preds):
+    right_num = 0
+    gt_counter_info = dict(Counter(gts))
+    pdt_counter_info = dict(Counter(preds))
+    for gt_token, gt_count in gt_counter_info.items():
+        pred_count = pdt_counter_info.get(gt_token, 0)
+        right_num += min(gt_count, pred_count)
+    return right_num
+
+
+def calculate_metrics(response_info, gt_info, is_verbose=False):
+    """
+    """
+    macro_recall_list, macro_precision_list, macro_f1_list = [], [], []
+    total_gt_num, total_pred_num, total_right_num = 0, 0, 0
+    for file_name, fullbox_gts in gt_info.items():
+        fullbox_preds = response_info.get(file_name, [])
+        right_num = evaluate_single_sample(fullbox_gts, fullbox_preds)
+        total_right_num += right_num
+        total_gt_num += len(fullbox_gts)
+        total_pred_num += len(fullbox_preds)
+
+        macro_recall = right_num / (len(fullbox_gts) + 1e-9)
+        macro_precision = right_num / (len(fullbox_preds) + 1e-9)
+        macro_f1 = 2 * macro_recall * macro_precision / (macro_recall + macro_precision + 1e-9)
+        macro_recall_list.append(macro_recall)
+        macro_precision_list.append(macro_precision)
+        macro_f1_list.append(macro_f1)
+
+    # marco
+    final_macro_recall = sum(macro_recall_list) / (len(macro_recall_list) + 1e-9)
+    final_macro_precision = sum(macro_precision_list) / (len(macro_precision_list) + 1e-9)
+    final_macro_f1 = sum(macro_f1_list) / (len(macro_f1_list) + 1e-9)
+
+    # micro
+    recall_acc = total_right_num / (total_gt_num + 1e-9)
+    preci_acc = total_right_num / (total_pred_num + 1e-9)
+    hmean = 2 * recall_acc * preci_acc / (recall_acc + preci_acc + 1e-9)
+    vbs_eval_result = {
+        'macro_recall': final_macro_recall, 'macro_precision': final_macro_precision, 'macro_f1_score': final_macro_f1,
+        'micro_recall': recall_acc, 'micro_precision': preci_acc, 'mirco_f1_score': hmean
+    }
+    eval_result = vbs_eval_result if is_verbose else {'macro_f1_score': final_macro_f1, 'mirco_f1_score': hmean}
+    return eval_result
+
+
+class OcrEvaluator(BaseMetric):
+    def response_post_func(self, response_text, **kwargs):
+        return response_text
+
+    def evaluate(self, response_info, gt_info, **kwargs):
+        # hard code here
+        dataset_name = kwargs['dataset']
+        is_word_level, is_lower, is_alphanum_only = True, True, False
+        if dataset_name in ["Arabic", "Japanese", "Korean"] or "zh" in dataset_name:
+            is_word_level = False
+        if "multi_scene_ocr" in self.group_name and is_word_level:
+            is_alphanum_only = True
+        eval_config = {"word_level": is_word_level, "alphanum_only": is_alphanum_only, "lowercase": is_lower}
+
+        image_pdt_info, image_gt_info = {}, {}
+        for file_name, gt_src in gt_info.items():
+            pred_src = response_info.get(file_name, "")
+            pdt_token_list = text_normalize_and_tokenize(
+                str(pred_src).strip(), is_word_level, is_lower, is_alphanum_only)
+            gt_token_list = text_normalize_and_tokenize(
+                str(gt_src).strip(), is_word_level, is_lower, is_alphanum_only)
+            image_pdt_info[file_name] = pdt_token_list
+            image_gt_info[file_name] = gt_token_list
+        eval_result = calculate_metrics(image_pdt_info, image_gt_info, is_verbose=False)
+        return {"summary": eval_result, "metric_config": eval_config}
+
+
+if __name__ == '__main__':
+    pass
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_configs/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_configs/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_configs/global_config.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_configs/global_config.py
new file mode 100644
index 0000000000000000000000000000000000000000..6fcd158f3aa38667cd80cf690f8b1d76c721f4f3
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_configs/global_config.py
@@ -0,0 +1,62 @@
+import subprocess
+
+texts = []
+images = []
+markers = []
+
+
+def reset_texts():
+    texts.clear()
+
+
+def add_text(text):
+    texts.append(text)
+
+
+def get_raw_texts():
+    return [item[2] for item in texts]
+
+
+def get_texts():
+    return texts
+
+
+def reset_images():
+    images.clear()
+
+
+def add_image(image):
+    images.append(image)
+
+
+def get_images():
+    return images
+
+
+def reset_markers():
+    markers.clear()
+
+
+def add_marker(marker):
+    markers.append(marker)
+
+
+def get_markers():
+    return markers
+
+
+def run_script_safe(script_path):
+    try:
+        subprocess.run(
+            ["python3", script_path],
+            check=True,
+            capture_output=True,
+            text=True
+        )
+        return True  # success
+    except subprocess.CalledProcessError as e:
+        print(f"[ERROR] Failed to run {script_path}")
+        print(f"[Return Code]: {e.returncode}")
+        print(f"[Stdout]:\n{e.stdout}")
+        print(f"[Stderr]:\n{e.stderr}")
+        return False  # failed
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_req.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_req.txt
new file mode 100644
index 0000000000000000000000000000000000000000..c754c049d3e33936b75183ce6ae846392756fe12
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/eval_req.txt
@@ -0,0 +1,5 @@
+pdf2image
+colormath
+squarify
+matplotlib_venn
+Pillow
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_and_color.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_and_color.py
new file mode 100644
index 0000000000000000000000000000000000000000..32ca463ba57ca2f7dee49ca47e3240b3f7ff7cac
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_and_color.py
@@ -0,0 +1,972 @@
+# flake8: noqa
+import inspect
+import warnings
+
+import matplotlib
+import matplotlib.colors as mcolors
+import matplotlib.pyplot as plt
+import networkx.drawing.nx_pylab as nx_pylab
+import squarify
+from matplotlib.axes._axes import Axes
+from matplotlib.axes._base import _process_plot_var_args
+from matplotlib.image import NonUniformImage
+from matplotlib.patches import Ellipse
+from matplotlib.projections.polar import PolarAxes
+from mpl_toolkits.mplot3d import Axes3D
+
+warnings.filterwarnings("ignore", category=UserWarning)
+
+# sys.path.insert(0, f'{os.environ["PROJECT_PATH"]}')
+
+
+drawed_colors = []
+in_decorator = False
+
+
+def convert_color_to_hex(color):
+    'Convert color from name, RGBA, or hex to a hex format.'
+    try:
+        # First, try to convert from color name to RGBA to hex
+        if isinstance(color, str):
+            # Check if it's already a hex color (start with '#' and length
+            # either 7 or 9)
+            if color.startswith('#') and (len(color) == 7 or len(color) == 9):
+                return color.upper()
+            else:
+                return mcolors.to_hex(mcolors.to_rgba(color)).upper()
+        # Then, check if it's in RGBA format
+        elif isinstance(color, (list, tuple)) and len(color) == 4:
+            return mcolors.to_hex(color).upper()
+        else:
+            raise ValueError("Unsupported color format")
+    except ValueError as e:
+        print(color)
+        print("Error converting color:", e)
+        return None
+
+
+def log_function_specific_for_draw_networkx_labels(func):
+    def wrapper(
+        G,
+        pos,
+        labels=None,
+        font_size=12,
+        font_color="k",
+        font_family="sans-serif",
+        font_weight="normal",
+        alpha=None,
+        bbox=None,
+        horizontalalignment="center",
+        verticalalignment="center",
+        ax=None,
+        clip_on=True,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(
+                G,
+                pos,
+                labels=labels,
+                font_size=font_size,
+                font_color=font_color,
+                font_family=font_family,
+                font_weight=font_weight,
+                alpha=alpha,
+                bbox=bbox,
+                horizontalalignment=horizontalalignment,
+                verticalalignment=verticalalignment,
+                ax=ax,
+                clip_on=clip_on
+            )
+
+            for item in result.values():
+                color = convert_color_to_hex(item.get_color())
+                drawed_colors.append(func_name + "--" + color)
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                labels=labels,
+                font_size=font_size,
+                font_color=font_color,
+                font_family=font_family,
+                font_weight=font_weight,
+                alpha=alpha,
+                bbox=bbox,
+                horizontalalignment=horizontalalignment,
+                verticalalignment=verticalalignment,
+                ax=ax,
+                clip_on=clip_on
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_specific_for_draw_networkx_edges(func):
+    def wrapper(
+        G,
+        pos,
+        edgelist=None,
+        width=1.0,
+        edge_color="k",
+        style="solid",
+        alpha=None,
+        arrowstyle=None,
+        arrowsize=10,
+        edge_cmap=None,
+        edge_vmin=None,
+        edge_vmax=None,
+        ax=None,
+        arrows=None,
+        label=None,
+        node_size=300,
+        nodelist=None,
+        node_shape="o",
+        connectionstyle="arc3",
+        min_source_margin=0,
+        min_target_margin=0,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(
+                G,
+                pos,
+                edgelist=edgelist,
+                width=width,
+                edge_color=edge_color,
+                style=style,
+                alpha=alpha,
+                arrowstyle=arrowstyle,
+                arrowsize=arrowsize,
+                edge_cmap=edge_cmap,
+                edge_vmin=edge_vmin,
+                edge_vmax=edge_vmax,
+                ax=ax,
+                arrows=arrows,
+                label=label,
+                node_size=node_size,
+                nodelist=nodelist,
+                node_shape=node_shape,
+                connectionstyle=connectionstyle,
+                min_source_margin=min_source_margin,
+                min_target_margin=min_target_margin
+            )
+
+            for item in result.get_edgecolors().tolist():
+                color = convert_color_to_hex(item)
+                drawed_colors.append(func_name + "--" + color)
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                edgelist=edgelist,
+                width=width,
+                edge_color=edge_color,
+                style=style,
+                alpha=alpha,
+                arrowstyle=arrowstyle,
+                arrowsize=arrowsize,
+                edge_cmap=edge_cmap,
+                edge_vmin=edge_vmin,
+                edge_vmax=edge_vmax,
+                ax=ax,
+                arrows=arrows,
+                label=label,
+                node_size=node_size,
+                nodelist=nodelist,
+                node_shape=node_shape,
+                connectionstyle=connectionstyle,
+                min_source_margin=min_source_margin,
+                min_target_margin=min_target_margin
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_specific_for_draw_networkx_nodes(func):
+    def wrapper(
+        G,
+        pos,
+        nodelist=None,
+        node_size=300,
+        node_color="#1f78b4",
+        node_shape="o",
+        alpha=None,
+        cmap=None,
+        vmin=None,
+        vmax=None,
+        ax=None,
+        linewidths=None,
+        edgecolors=None,
+        label=None,
+        margins=None,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(
+                G,
+                pos,
+                nodelist=nodelist,
+                node_size=node_size,
+                node_color=node_color,
+                node_shape=node_shape,
+                alpha=alpha,
+                cmap=cmap,
+                vmin=vmin,
+                vmax=vmax,
+                ax=ax,
+                linewidths=linewidths,
+                edgecolors=edgecolors,
+                label=label,
+                margins=margins
+            )
+
+            for item in result.get_facecolors().tolist():
+                color = convert_color_to_hex(item)
+                drawed_colors.append(func_name + "--" + color)
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                nodelist=nodelist,
+                node_size=node_size,
+                node_color=node_color,
+                node_shape=node_shape,
+                alpha=alpha,
+                cmap=cmap,
+                vmin=vmin,
+                vmax=vmax,
+                ax=ax,
+                linewidths=linewidths,
+                edgecolors=edgecolors,
+                label=label,
+                margins=margins
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_for_3d(func):
+    def wrapper(*args, **kwargs):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(*args, **kwargs)
+
+            if func.__name__ == "scatter":
+                # check whether cmap is used
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    drawed_colors.append(func_name + "--" + kwargs["cmap"])
+                else:
+                    for item in result.get_facecolors().tolist():
+                        color = convert_color_to_hex(item)
+                        drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "plot":
+                for line in result:
+                    color = convert_color_to_hex(line.get_color())
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "plot_surface":
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    drawed_colors.append(func_name + "--" + kwargs["cmap"])
+                else:
+                    colors = result.get_facecolors().tolist()
+                    drawed_colors.append(
+                        func_name +
+                        "--" +
+                        convert_color_to_hex(
+                            colors[0]))
+            elif func.__name__ == "bar3d":
+                colors = result.get_facecolors().tolist()
+                drawed_colors.append(
+                    func_name +
+                    "--" +
+                    convert_color_to_hex(
+                        colors[0]))
+            elif func.__name__ == "bar":
+                for item in result:
+                    color = convert_color_to_hex(item.get_facecolor())
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "add_collection3d":
+                colors = result.get_facecolors().tolist()
+                for color in colors:
+                    drawed_colors.append(
+                        func_name + "--" + convert_color_to_hex(color))
+
+            in_decorator = False
+        else:
+            return func(*args, **kwargs)
+        return result
+
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function(func):
+    def wrapper(*args, **kwargs):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(*args, **kwargs)
+
+            if func.__name__ == "_makeline":
+                color = convert_color_to_hex(result[1]["color"])
+                drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "axhline":
+                color = convert_color_to_hex(result.get_color())
+                drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "axvline":
+                color = convert_color_to_hex(result.get_color())
+                drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "_fill_between_x_or_y":
+                color = convert_color_to_hex(list(result.get_facecolors()[0]))
+                drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "bar":
+                for item in result:
+                    color = convert_color_to_hex(
+                        list(item._original_facecolor))
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "scatter" and not isinstance(args[0], PolarAxes):
+                # check whether cmap is used
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    drawed_colors.append(func_name + "--" + kwargs["cmap"])
+                else:
+                    color = convert_color_to_hex(
+                        list(result.get_facecolor()[0]))
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "pie":
+                for item in result[0]:
+                    color = convert_color_to_hex(item.get_facecolor())
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "axvspan":
+                color = convert_color_to_hex(result.get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "axhspan":
+                color = convert_color_to_hex(result.get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "hlines":
+                for item in result.get_edgecolors():
+                    color = convert_color_to_hex(list(item))
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "vlines":
+                for item in result.get_edgecolors():
+                    color = convert_color_to_hex(list(item))
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "boxplot":
+                for item in result["boxes"]:
+                    if isinstance(item, matplotlib.patches.PathPatch):
+                        color = convert_color_to_hex(
+                            list(item.get_facecolor()))
+                        drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "violinplot":
+                for item in result["bodies"]:
+                    color = convert_color_to_hex(list(item.get_facecolor()[0]))
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "hist":
+                tops, bins, patches = result
+                if not isinstance(patches, matplotlib.cbook.silent_list):
+                    for item in patches:
+                        color = convert_color_to_hex(
+                            list(item.get_facecolor()))
+                        drawed_colors.append(func_name + "--" + color)
+                else:
+                    for container in patches:
+                        for item in container:
+                            color = convert_color_to_hex(
+                                list(item.get_facecolor()))
+                            drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "quiver":
+                for item in result.get_facecolors().tolist():
+                    color = convert_color_to_hex(item)
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "plot" and len(args) > 0 and isinstance(args[0], PolarAxes):
+                lines = result
+                for line in lines:
+                    color = convert_color_to_hex(line.get_color())
+                    drawed_colors.append(func_name + "_polar" + "--" + color)
+            elif func.__name__ == "scatter" and isinstance(args[0], PolarAxes):
+                # check whether cmap is used
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    drawed_colors.append(func_name + "--" + kwargs["cmap"])
+                else:
+                    color = convert_color_to_hex(
+                        list(result.get_facecolor()[0]))
+                    drawed_colors.append(func_name + "_polar" + "--" + color)
+            elif func.__name__ == "plot" and "squarify" in func_name:
+                # get ax
+                ax = result
+                # get container
+                containers = ax.containers
+                for container in containers:
+                    for item in container:
+                        color = convert_color_to_hex(
+                            list(item.get_facecolor()))
+                        drawed_colors.append(
+                            func_name + "_squarify" + "--" + color)
+            elif func.__name__ == "imshow":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+            elif func.__name__ == "pcolor":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+            elif func.__name__ == "contour":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+            elif func.__name__ == "contourf":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+            elif func.__name__ == "fill":
+                patches = result
+                for patch in patches:
+                    color = convert_color_to_hex(list(patch.get_facecolor()))
+                    drawed_colors.append(func_name + "--" + color)
+            elif func.__name__ == "__init__" and isinstance(args[0], NonUniformImage):
+                colormap = args[0].get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+            elif func.__name__ == "broken_barh":
+                colors = result.get_facecolors().tolist()
+                for color in colors:
+                    drawed_colors.append(
+                        func_name + "--" + convert_color_to_hex(color))
+            elif func.__name__ == "__init__" and isinstance(args[0], Ellipse):
+                color = convert_color_to_hex(args[0].get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+
+            in_decorator = False
+        else:
+            return func(*args, **kwargs)
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+_process_plot_var_args._makeline = log_function(
+    _process_plot_var_args._makeline)
+Axes.bar = log_function(Axes.bar)
+Axes.scatter = log_function(Axes.scatter)
+Axes.axhline = log_function(Axes.axhline)
+Axes.axvline = log_function(Axes.axvline)
+Axes._fill_between_x_or_y = log_function(Axes._fill_between_x_or_y)
+Axes.pie = log_function(Axes.pie)
+Axes.axvspan = log_function(Axes.axvspan)
+Axes.axhspan = log_function(Axes.axhspan)
+Axes.hlines = log_function(Axes.hlines)
+Axes.vlines = log_function(Axes.vlines)
+Axes.boxplot = log_function(Axes.boxplot)
+Axes.violinplot = log_function(Axes.violinplot)
+Axes.hist = log_function(Axes.hist)
+Axes.plot = log_function(Axes.plot)
+Axes.quiver = log_function(Axes.quiver)
+Axes.imshow = log_function(Axes.imshow)
+Axes.pcolor = log_function(Axes.pcolor)
+Axes.contour = log_function(Axes.contour)
+Axes.contourf = log_function(Axes.contourf)
+Axes.fill = log_function(Axes.fill)
+NonUniformImage.__init__ = log_function(NonUniformImage.__init__)
+Ellipse.__init__ = log_function(Ellipse.__init__)
+Axes.broken_barh = log_function(Axes.broken_barh)
+
+nx_pylab.draw_networkx_nodes = log_function_specific_for_draw_networkx_nodes(
+    nx_pylab.draw_networkx_nodes)
+nx_pylab.draw_networkx_edges = log_function_specific_for_draw_networkx_edges(
+    nx_pylab.draw_networkx_edges)
+nx_pylab.draw_networkx_labels = log_function_specific_for_draw_networkx_labels(
+    nx_pylab.draw_networkx_labels)
+
+
+squarify.plot = log_function(squarify.plot)
+
+Axes3D.scatter = log_function_for_3d(Axes3D.scatter)
+Axes3D.plot = log_function_for_3d(Axes3D.plot)
+Axes3D.plot_surface = log_function_for_3d(Axes3D.plot_surface)
+Axes3D.bar3d = log_function_for_3d(Axes3D.bar3d)
+Axes3D.bar = log_function_for_3d(Axes3D.bar)
+Axes3D.add_collection3d = log_function_for_3d(Axes3D.add_collection3d)
+
+# barh test
+# draw a simple barh plot
+# fig, ax = plt.subplots()
+# ax.barh(np.arange(5), np.random.rand(5))
+# ax.barh(np.arange(5), np.random.rand(5))
+# plt.show()
+
+# axhline test
+# fig, ax = plt.subplots()
+# ax.axhline(0.5)
+# ax.axhline(0.8)
+# plt.show()
+
+# axvline test
+# fig, ax = plt.subplots()
+# ax.axvline(0.5)
+# ax.axvline(0.8)
+# plt.show()
+
+# errorbar test
+# fig, ax = plt.subplots()
+# x = np.arange(10)
+# y = np.sin(x)
+#
+# ax.errorbar(x, y, yerr=0.1)
+# ax.errorbar(x, y, yerr=0.2)
+# plt.show()
+
+# squarify test
+# fig, ax = plt.subplots()
+# sizes = [50, 25, 25]
+# squarify.plot(sizes=sizes, ax=ax)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# loglog test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y = x**2
+# ax.loglog(x, y)
+# plt.show()
+
+# fill_between test
+# fig, ax = plt.subplots()
+# x = np.arange(10)
+# y1 = np.sin(x)
+# y2 = np.cos(x)
+# ax.fill_between(x, y1, y2, cmap='viridis')
+# plt.show()
+
+# fill_betweenx test
+# fig, ax = plt.subplots()
+# x = np.arange(10)
+# y1 = np.sin(x)
+# y2 = np.cos(x)
+# ax.fill_betweenx(x, y1, y2, cmap='viridis')
+# plt.show()
+
+# pie test
+# fig, ax = plt.subplots()
+# sizes = [50, 25, 25]
+# ax.pie(sizes)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# axvspan test
+# fig, ax = plt.subplots()
+# ax.axvspan(0.2, 0.3, color='red', alpha=0.5)
+# ax.axvspan(0.5, 0.7, color='blue', alpha=0.5)
+# plt.show()
+
+# axhspan test
+# fig, ax = plt.subplots()
+# ax.axhspan(0.2, 0.3, color='red', alpha=0.5)
+# ax.axhspan(0.5, 0.7, color='blue', alpha=0.5)
+# plt.show()
+
+
+# hlines test
+# fig, ax = plt.subplots()
+# y_values = [1, 2, 3, 4, 5]
+# xmin = 0
+# xmax = 10
+# ax.hlines(y=y_values, xmin=xmin, xmax=xmax, linestyles='dashed')
+# ax.set_xlabel('X-axis')
+# ax.set_ylabel('Y-axis')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# vlines test
+# fig, ax = plt.subplots()
+# x_values = [1, 2, 3, 4, 5]
+# ymin = 0
+# ymax = 10
+# ax.vlines(x=x_values, ymin=ymin, ymax=ymax, linestyles='dashed')
+# ax.set_xlabel('X-axis')
+# ax.set_ylabel('Y-axis')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# boxplot test
+# fig, ax = plt.subplots()
+# data = np.random.rand(10, 3)
+# ax.boxplot(data, patch_artist=True)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# violin test
+# fig, ax = plt.subplots()
+# data = np.random.rand(10, 3)
+# ax.violinplot(data)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# hist test
+# fig, ax = plt.subplots()
+# data = np.random.rand(100, 1)
+# ax.hist(data, bins=10)
+# plt.savefig("tmp.png")
+# plt.show()
+
+
+# networkx test
+# fig, ax = plt.subplots()
+# G = networkx.complete_graph(5)
+# draw the graph, give each node a different color, and a label. make the edges red and blue, with labels
+# networkx.draw(G, ax=ax, node_color='r', edge_color='b', labels={0: '0', 1: '1', 2: '2', 3: '3', 4: '4'})
+# plt.savefig("tmp.png")
+# plt.show()
+
+# quiver test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 10)
+# y = np.linspace(0, 10, 10)
+# u = np.zeros(10)
+# v = np.ones(10)
+# # draw the quiver plot, with color red
+# ax.quiver(x, y, u, v, color='r')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# 3d scatter test
+# fig, ax = plt.subplots(subplot_kw={'projection': '3d'})
+# x = np.random.rand(10)
+# y = np.random.rand(10)
+# z = np.random.rand(10)
+# draw the scatter plot, with color red
+# ax.scatter3D(x, y, z, c='#ff2395')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# 3d plot test
+# fig, ax = plt.subplots(subplot_kw={'projection': '3d'})
+# draw two lines in 3d, with color red and blue
+# ax.plot([0, 1], [0, 1], [0, 1], color='r')
+# ax.plot([0, 1], [0, 1], [1, 0], color='b')
+
+# 3d plot_surface test
+# fig, ax = plt.subplots(subplot_kw={'projection': '3d'})
+# draw a surface plot, with a beautiful colormap
+# X = np.linspace(-5, 5, 100)
+# Y = np.linspace(-5, 5, 100)
+# X, Y = np.meshgrid(X, Y)
+# Z = np.sin(np.sqrt(X**2 + Y**2))
+# ax.plot_surface(X, Y, Z, cmap='viridis')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# 3d bar test
+# fig, ax = plt.subplots(subplot_kw={'projection': '3d'})
+# x = np.arange(10)
+# y = np.random.rand(10)
+# z = np.zeros(10)
+# dx = np.ones(10)
+# dy = np.ones(10)
+# dz = np.random.rand(10)
+# # draw the 3d bar plot, with color red
+# ax.bar3d(x, y, z, dx, dy, dz)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# # bar2d in axes3d test
+# fig, ax = plt.subplots(subplot_kw={'projection': '3d'})
+# x = np.arange(10)
+# y = np.random.rand(10)
+# z = np.zeros(10)
+# dx = np.ones(10)
+# dy = np.ones(10)
+# dz = np.random.rand(10)
+# # draw the 2d bar plot, with color red
+# ax.bar(x, y, z, zdir='y', color=['r', 'b', 'g', 'y', 'm', 'c', 'k', 'w', 'r', 'b'])
+# plt.savefig("tmp.png")
+# plt.show()
+
+
+# plot in test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y = np.sin(x)
+# draw the plot, with color red
+# ax.plot(x, y, color='r')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# matshow in test
+# fig, ax = plt.subplots()
+# data = np.random.rand(10, 10)
+# draw the matshow plot, with a beautiful colormap
+# ax.imshow(data, cmap='pink')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# pcolor in test
+# fig, ax = plt.subplots()
+# data = np.random.rand(10, 10)
+# draw the pcolor plot, with a beautiful colormap
+# ax.pcolor(data)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# # contour in test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y = np.linspace(0, 10, 100)
+# X, Y = np.meshgrid(x, y)
+# Z = np.sin(X) * np.cos(Y)
+# # draw the contour plot, with a beautiful colormap
+# ax.contour(X, Y, Z)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# # contourf in test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y = np.linspace(0, 10, 100)
+# X, Y = np.meshgrid(x, y)
+# Z = np.sin(X) * np.cos(Y)
+# # draw the contourf plot, with a beautiful colormap
+# ax.contourf(X, Y, Z, cmap='viridis')
+# plt.savefig("tmp.png")
+# plt.show()
+
+# stackplot in test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y1 = np.sin(x)
+# y2 = np.cos(x)
+# y3 = np.tan(x)
+# draw the stackplot, with beautiful colors
+# ax.stackplot(x, y1, y2, y3, colors=['r', 'g', 'b'])
+# plt.savefig("tmp.png")
+# plt.show()
+
+# fill in test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y1 = np.sin(x)
+# y2 = np.cos(x)
+# draw the fill plot, with color red
+# ax.fill(x, y1, color='r')
+# plt.savefig("tmp.png")
+# plt.show()
+
+
+# # NonUniformImage in test
+# fig, ax = plt.subplots()
+# data = np.random.rand(10, 10)
+# x = np.linspace(-4, 4, 9)
+# y = np.linspace(-4, 4, 9)
+# z = np.sqrt(x[np.newaxis, :] ** 2 + y[:, np.newaxis] ** 2)
+# im = NonUniformImage(ax, interpolation='bilinear')
+# im.set_data(x, y , z)
+# ax.add_image(im)
+# plt.savefig("tmp.png")
+# plt.show()
+
+# broken_barh in test
+# fig, ax = plt.subplots()
+# x = np.linspace(0, 10, 100)
+# y = np.sin(x)
+# draw the broken_barh plot, with color red
+# ax.broken_barh([(1, 2), (3, 4)], (0, 1), facecolors='r')
+# plt.savefig("tmp.png")
+# plt.show()
+
+
+# Ellipse in test
+fig, ax = plt.subplots()
+e = matplotlib.patches.Ellipse((0.5, 0.5), 0.4, 0.2, color='r')
+ax.add_patch(e)
+plt.savefig("tmp.png")
+plt.show()
+
+
+# # radar plot in test
+# fig, ax = plt.subplots(subplot_kw={'projection': 'polar'})
+# theta = np.linspace(0, 2*np.pi, 100)
+# r = np.sin(3*theta)**2
+# # draw the radar plot, with color red
+# ax.plot(theta, r, color='r')
+# plt.savefig("tmp.png")
+# plt.show()
+
+
+# import numpy as np; np.random.seed(0)
+
+# import matplotlib.pyplot as plt
+# from matplotlib.lines import Line2D
+
+# # ===================
+# # Part 2: Data Preparation
+# # ===================
+# # Data for PC1 and PC2
+# values_pc1 = [0.8, 0.7, 0.6, 0.85, 0.9, 0.75, 0.7, 0.65, 0.8, 0.9]
+# values_pc2 = [0.6, 0.55, 0.5, 0.45, 0.4, 0.35, 0.3, 0.25, 0.2, 0.15]
+# num_vars = len(values_pc1)
+
+# # Compute angle for each axis
+# angles = np.linspace(0, 2 * np.pi, num_vars, endpoint=False).tolist()
+
+# # The plot is circular, so we need to "complete the loop" and append the start to the end.
+# values_pc1 += values_pc1[:1]
+# values_pc2 += values_pc2[:1]
+# angles += angles[:1]
+
+# # ===================
+# # Part 3: Plot Configuration and Rendering
+# # ===================
+# # Draw the radar chart
+# fig, ax = plt.subplots(figsize=(8, 8), subplot_kw=dict(polar=True))
+# ax.fill(angles, values_pc1, color="black", alpha=0.1)
+# ax.plot(angles, values_pc1, color="black", linewidth=2, label="Loadings PC1")
+# ax.scatter(angles[:-1], values_pc1[:-1], color="black", s=50)
+# ax.fill(angles, values_pc2, color="red", alpha=0.1)
+# ax.plot(angles, values_pc2, color="red", linewidth=2, label="Loadings PC2")
+# ax.scatter(angles[:-1], values_pc2[:-1], color="red", s=50)
+
+# # Add labels to the plot
+# ax.set_yticklabels([])
+# grid_angles = np.linspace(0, 2 * np.pi, 8, endpoint=False)
+# ax.set_xticks(grid_angles)
+# angle_labels = [f"{i*45}°" for i in range(8)]
+# ax.set_xticklabels(angle_labels)
+
+# # Add grid lines and labels for the concentric circles
+# ax.set_rgrids(
+#     [0.2, 0.4, 0.6, 0.8, 1.0],
+#     labels=["0.2", "0.4", "0.6", "0.8", "1.0"],
+#     angle=30,
+#     color="black",
+#     size=10,
+# )
+
+# # Create legend handles manually
+# legend_elements = [
+#     Line2D(
+#         [0],
+#         [0],
+#         color="black",
+#         linewidth=2,
+#         marker="o",
+#         markersize=8,
+#         label="Loadings PC1",
+#     ),
+#     Line2D(
+#         [0],
+#         [0],
+#         color="red",
+#         linewidth=2,
+#         marker="o",
+#         markersize=8,
+#         label="Loadings PC2",
+#     ),
+# ]
+
+# # Add legend and title
+# ax.legend(
+#     handles=legend_elements, loc="upper right", bbox_to_anchor=(1.1, 1.1), frameon=False
+# )
+
+# # ===================
+# # Part 4: Saving Output
+# # ===================
+# # Adjust layout and save the plot
+# plt.tight_layout()
+# plt.savefig('tmp.png')
+
+
+# poly3d in test
+# import math
+# import matplotlib.pyplot as plt
+# import numpy as np; np.random.seed(0)
+
+# from matplotlib.collections import PolyCollection
+
+# # ===================
+# # Part 2: Data Preparation
+# # ===================
+# # Fixing random state for reproducibility
+# def polygon_under_graph(x, y):
+#     """
+#     Construct the vertex list which defines the polygon filling the space under
+#     the (x, y) line graph. This assumes x is in ascending order.
+#     """
+#     return [(x[0], 0.0), *zip(x, y), (x[-1], 0.0)]
+
+
+# x = np.linspace(0.0, 10.0, 31)
+# vaccination_numbers = range(1, 4)
+
+# # verts[i] is a list of (x, y) pairs defining polygon i.
+# gamma = np.vectorize(math.gamma)
+# verts = [
+#     polygon_under_graph(x, v**x * np.exp(-v) / gamma(x + 1))
+#     for v in vaccination_numbers
+# ]
+
+# # ===================
+# # Part 3: Plot Configuration and Rendering
+# # ===================
+# ax = plt.figure(figsize=(8, 6)).add_subplot(projection="3d")
+# facecolors = plt.colormaps["viridis_r"](np.linspace(0, 1, len(verts)))
+
+# poly = PolyCollection(verts, facecolors=facecolors, alpha=0.7)
+# ax.add_collection3d(poly, zs=vaccination_numbers, zdir="y")
+
+# ax.set(
+#     xlim=(0, 10),
+#     ylim=(1, 4),
+#     zlim=(0, 0.35),
+#     xlabel="Age",
+#     ylabel="Vaccination Number",
+#     zlabel="Incidence Rate",
+# )
+
+# ax.set_yticks([1, 2, 3])
+# ax.set_box_aspect(aspect=None, zoom=0.8)
+
+# # ===================
+# # Part 4: Saving Output
+# # ===================
+# plt.tight_layout()
+# plt.savefig('3d_14.pdf', bbox_inches='tight')
+
+
+drawed_colors = set(drawed_colors)
+print("drawed_colors", drawed_colors)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..45e72dcba6cfa6f73464f7a594d78ba3e448acc5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_evaluator.py
@@ -0,0 +1,181 @@
+# flake8: noqa
+import os
+from typing import Dict
+
+from ..eval_configs.global_config import run_script_safe
+
+# from dotenv import load_dotenv
+# load_dotenv()
+
+
+
+class ChartTypeEvaluator:
+
+    def __init__(self):
+        self.metrics = {
+            "precision": 0,
+            "recall": 0,
+            "f1": 0
+        }
+
+    def __call__(self, generation_code_file, golden_code_file):
+        generation_chart_types = self._get_chart_types(generation_code_file)
+        golden_chart_types = self._get_chart_types(golden_code_file)
+
+        self.golden_code_file = golden_code_file
+
+        self._calculate_metrics(generation_chart_types, golden_chart_types)
+
+        # [TAG] What is this for?
+        # redunant_file = os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"] + "/" + os.path.basename(golden_code_file).replace(".py", ".pdf")
+        # print(f"redunant_file: {redunant_file}")
+        # breakpoint()
+        # # if os.path.exists(redunant_file) == True:
+        # os.remove(redunant_file)
+
+        # print(self.metrics)
+
+    def _get_chart_types(self, code_file):
+
+        with open(code_file, "r") as f:
+            lines = f.readlines()
+        code = "".join(lines)
+
+        prefix = self._get_prefix()
+        output_file = code_file.replace(".py", "_log_chart_types.txt")
+        suffix = self._get_suffix(output_file)
+        code = prefix + code + suffix
+
+        code_log_chart_types_file = code_file.replace(
+            ".py", "_log_chart_types.py")
+        with open(code_log_chart_types_file, "w") as f:
+            f.write(code)
+
+        # os.system(f"python {code_log_chart_types_file}")
+        success = run_script_safe(code_log_chart_types_file)
+        if not success:
+            print("Skip downstream logic due to previous failure.")
+            # optionally return default result or continue
+
+        if os.path.exists(output_file):
+            with open(output_file, "r") as f:
+                chart_types = f.read()
+                chart_types = eval(chart_types)
+            os.remove(output_file)
+        else:
+            chart_types = {}
+        os.remove(code_log_chart_types_file)
+
+        # pdf_file = re.findall(r"plt\.savefig\('(.*)'\)", code)
+        # if len(pdf_file) != 0:
+        # pdf_file = pdf_file[0].split(",")[0][:-1]
+        # print(pdf_file)
+        # if os.path.basename(pdf_file) == pdf_file:
+        # os.remove(pdf_file)
+
+        return chart_types
+
+    def _calculate_metrics(
+            self, generation_chart_types: Dict[str, int], golden_chart_types: Dict[str, int]):
+        """
+        Calculate precision, recall, and f1 score of the chart types.
+
+        Args:
+            - generation_chart_types: Dict[str, int]
+                - key: chart type
+                - value: number of times the chart type is called
+            - golden_chart_types: Dict[str, int]
+                - key: chart type
+                - value: number of times the chart type is called
+        """
+        if len(generation_chart_types) == 0:
+            return
+
+        n_correct = 0
+        total = sum(generation_chart_types.values())
+
+        for chart_type, count in generation_chart_types.items():
+            if chart_type in golden_chart_types:
+                n_correct += min(count, golden_chart_types[chart_type])
+
+        self.metrics["precision"] = n_correct / total
+        try:
+            self.metrics["recall"] = n_correct / \
+                sum(golden_chart_types.values())
+        except BaseException:
+            print(
+                "<<<<<<<<<<<<<<<<<<<<golden_code_file",
+                self.golden_code_file)
+        if self.metrics["precision"] + self.metrics["recall"] == 0:
+            self.metrics["f1"] = 0
+        else:
+            self.metrics["f1"] = 2 * self.metrics["precision"] * \
+                self.metrics["recall"] / (self.metrics["precision"] + self.metrics["recall"])
+        return
+
+    def _get_prefix(self):
+        with open(os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"] + "/evaluator/chart_type_evaluator_prefix.py", "r") as f:
+            prefix = f.read()
+        return prefix
+
+#     def _get_prefix(self):
+#         return f"""
+# import warnings
+# warnings.filterwarnings("ignore", category=UserWarning)
+
+# import sys
+# sys.path.insert(0, '{os.environ['PROJECT_PATH']}')
+# import matplotlib.pyplot as plt
+# from matplotlib.axes import Axes
+# import squarify
+# import inspect
+
+# called_functions = {{}}
+# in_decorator = False
+
+# def log_function(func):
+#     def wrapper(*args, **kwargs):
+#         global in_decorator
+#         if not in_decorator:
+#             in_decorator = True
+#             # name = func.__name__
+#             file_name = inspect.getfile(func)
+#             name = file_name + "-" + func.__name__
+#             called_functions[name] = called_functions.get(name, 0) + 1
+#             result = func(*args, **kwargs)
+#             in_decorator = False
+#             return result
+#         else:
+#             return func(*args, **kwargs)
+#     return wrapper
+
+# Axes.plot = log_function(Axes.plot)
+# Axes.loglog = log_function(Axes.loglog)
+# Axes.scatter = log_function(Axes.scatter)
+# Axes.bar = log_function(Axes.bar)
+# Axes.barh = log_function(Axes.barh)
+# Axes.axhline = log_function(Axes.axhline)
+# Axes.axvline = log_function(Axes.axvline)
+# Axes.errorbar = log_function(Axes.errorbar)
+# Axes.matshow = log_function(Axes.matshow)
+# Axes.hist = log_function(Axes.hist)
+# Axes.pie = log_function(Axes.pie)
+# Axes.boxplot = log_function(Axes.boxplot)
+# Axes.arrow = log_function(Axes.arrow)
+# Axes.fill_between = log_function(Axes.fill_between)
+# Axes.fill_betweenx = log_function(Axes.fill_betweenx)
+# Axes.imshow = log_function(Axes.imshow)
+# Axes.contour = log_function(Axes.contour)
+# Axes.contourf = log_function(Axes.contourf)
+# Axes.violinplot = log_function(Axes.violinplot)
+# Axes.violin = log_function(Axes.violin)
+
+# squarify.plot = log_function(squarify.plot)
+# """
+
+    def _get_suffix(self, output_file):
+        return f"""
+# print(called_functions)
+with open('{output_file}', 'w') as f:
+    f.write(str(called_functions))
+"""
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_evaluator_prefix.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_evaluator_prefix.py
new file mode 100644
index 0000000000000000000000000000000000000000..d32f5e0b5d21f71ee7003b7dde1a79ca03babb99
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/chart_type_evaluator_prefix.py
@@ -0,0 +1,375 @@
+import os
+import sys
+
+# sys.path.insert(0, os.environ['PROJECT_PATH'])
+if os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"] not in sys.path:
+    sys.path.insert(0, os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"])
+
+import inspect
+import warnings
+
+import matplotlib.pyplot as plt
+import networkx as nx
+import networkx.drawing.nx_pylab as nx_pylab
+import squarify
+from matplotlib.axes import Axes
+from matplotlib.axes._base import _process_plot_var_args
+from matplotlib.image import NonUniformImage
+from matplotlib.patches import Circle, Ellipse
+from matplotlib.projections.polar import PolarAxes
+from matplotlib.tri._tripcolor import tripcolor
+from matplotlib_venn._common import VennDiagram
+from mpl_toolkits.mplot3d import Axes3D
+
+warnings.filterwarnings("ignore", category=UserWarning)
+warnings.filterwarnings("ignore", category=DeprecationWarning)
+warnings.filterwarnings("ignore", category=FutureWarning)
+
+# sys.path.insert(0, os.environ['PROJECT_PATH'])
+
+called_functions = {}
+in_decorator = False
+
+
+def log_function_specific_for_draw_networkx_labels(func):
+    def wrapper(
+        G,
+        pos,
+        labels=None,
+        font_size=12,
+        font_color="k",
+        font_family="sans-serif",
+        font_weight="normal",
+        alpha=None,
+        bbox=None,
+        horizontalalignment="center",
+        verticalalignment="center",
+        ax=None,
+        clip_on=True,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            file_name = inspect.getfile(func) + "/" + func.__name__
+            name = file_name + "-" + func.__name__
+            called_functions[name] = called_functions.get(name, 0) + 1
+
+            result = func(
+                G,
+                pos,
+                labels=labels,
+                font_size=font_size,
+                font_color=font_color,
+                font_family=font_family,
+                font_weight=font_weight,
+                alpha=alpha,
+                bbox=bbox,
+                horizontalalignment=horizontalalignment,
+                verticalalignment=verticalalignment,
+                ax=ax,
+                clip_on=clip_on
+            )
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                labels=labels,
+                font_size=font_size,
+                font_color=font_color,
+                font_family=font_family,
+                font_weight=font_weight,
+                alpha=alpha,
+                bbox=bbox,
+                horizontalalignment=horizontalalignment,
+                verticalalignment=verticalalignment,
+                ax=ax,
+                clip_on=clip_on
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_specific_for_draw_networkx_edges(func):
+    def wrapper(
+        G,
+        pos,
+        edgelist=None,
+        width=1.0,
+        edge_color="k",
+        style="solid",
+        alpha=None,
+        arrowstyle=None,
+        arrowsize=10,
+        edge_cmap=None,
+        edge_vmin=None,
+        edge_vmax=None,
+        ax=None,
+        arrows=None,
+        label=None,
+        node_size=300,
+        nodelist=None,
+        node_shape="o",
+        connectionstyle="arc3",
+        min_source_margin=0,
+        min_target_margin=0,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            file_name = inspect.getfile(func) + "/" + func.__name__
+            name = file_name + "-" + func.__name__
+            called_functions[name] = called_functions.get(name, 0) + 1
+
+            result = func(
+                G,
+                pos,
+                edgelist=edgelist,
+                width=width,
+                edge_color=edge_color,
+                style=style,
+                alpha=alpha,
+                arrowstyle=arrowstyle,
+                arrowsize=arrowsize,
+                edge_cmap=edge_cmap,
+                edge_vmin=edge_vmin,
+                edge_vmax=edge_vmax,
+                ax=ax,
+                arrows=arrows,
+                label=label,
+                node_size=node_size,
+                nodelist=nodelist,
+                node_shape=node_shape,
+                connectionstyle=connectionstyle,
+                min_source_margin=min_source_margin,
+                min_target_margin=min_target_margin
+            )
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                edgelist=edgelist,
+                width=width,
+                edge_color=edge_color,
+                style=style,
+                alpha=alpha,
+                arrowstyle=arrowstyle,
+                arrowsize=arrowsize,
+                edge_cmap=edge_cmap,
+                edge_vmin=edge_vmin,
+                edge_vmax=edge_vmax,
+                ax=ax,
+                arrows=arrows,
+                label=label,
+                node_size=node_size,
+                nodelist=nodelist,
+                node_shape=node_shape,
+                connectionstyle=connectionstyle,
+                min_source_margin=min_source_margin,
+                min_target_margin=min_target_margin
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_specific_for_draw_networkx_nodes(func):
+    def wrapper(
+        G,
+        pos,
+        nodelist=None,
+        node_size=300,
+        node_color="#1f78b4",
+        node_shape="o",
+        alpha=None,
+        cmap=None,
+        vmin=None,
+        vmax=None,
+        ax=None,
+        linewidths=None,
+        edgecolors=None,
+        label=None,
+        margins=None,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            file_name = inspect.getfile(func) + "/" + func.__name__
+            name = file_name + "-" + func.__name__
+            called_functions[name] = called_functions.get(name, 0) + 1
+
+            result = func(
+                G,
+                pos,
+                nodelist=nodelist,
+                node_size=node_size,
+                node_color=node_color,
+                node_shape=node_shape,
+                alpha=alpha,
+                cmap=cmap,
+                vmin=vmin,
+                vmax=vmax,
+                ax=ax,
+                linewidths=linewidths,
+                edgecolors=edgecolors,
+                label=label,
+                margins=margins
+            )
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                nodelist=nodelist,
+                node_size=node_size,
+                node_color=node_color,
+                node_shape=node_shape,
+                alpha=alpha,
+                cmap=cmap,
+                vmin=vmin,
+                vmax=vmax,
+                ax=ax,
+                linewidths=linewidths,
+                edgecolors=edgecolors,
+                label=label,
+                margins=margins
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function(func):
+    def wrapper(*args, **kwargs):
+        global in_decorator
+        if not in_decorator:
+            in_decorator = True
+            if len(args) > 0 and isinstance(
+                    args[0], PolarAxes) and func.__name__ == "plot":
+                file_name = inspect.getfile(func)
+                file_name += "_polar"
+            else:
+                file_name = inspect.getfile(func)
+            name = file_name + "-" + func.__name__
+            called_functions[name] = called_functions.get(name, 0) + 1
+            result = func(*args, **kwargs)
+            in_decorator = False
+            return result
+        else:
+            return func(*args, **kwargs)
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+Axes.bar = log_function(Axes.bar)
+Axes.barh = log_function(Axes.barh)   # The same as the bar
+
+# _process_plot_var_args._makeline = log_function(_process_plot_var_args._makeline)
+Axes.plot = log_function(Axes.plot)     # Special Case for polar plot
+Axes.axhline = log_function(Axes.axhline)
+Axes.axvline = log_function(Axes.axvline)
+Axes.axvspan = log_function(Axes.axvspan)
+Axes.axhspan = log_function(Axes.axhspan)
+Axes.hlines = log_function(Axes.hlines)
+Axes.vlines = log_function(Axes.vlines)
+
+Axes.errorbar = log_function(Axes.errorbar)   # The same as the line
+
+Axes.boxplot = log_function(Axes.boxplot)
+
+Axes.violinplot = log_function(Axes.violinplot)
+Axes.violin = log_function(Axes.violin)
+
+Axes.hist = log_function(Axes.hist)
+
+# Axes._fill_between_x_or_y = log_function(Axes._fill_between_x_or_y)
+Axes.fill_between = log_function(Axes.fill_between)
+Axes.fill_betweenx = log_function(Axes.fill_betweenx)
+
+Axes.scatter = log_function(Axes.scatter)
+
+nx_pylab.draw_networkx_nodes = log_function_specific_for_draw_networkx_nodes(
+    nx_pylab.draw_networkx_nodes)
+nx_pylab.draw_networkx_edges = log_function_specific_for_draw_networkx_edges(
+    nx_pylab.draw_networkx_edges)
+nx_pylab.draw_networkx_labels = log_function_specific_for_draw_networkx_labels(
+    nx_pylab.draw_networkx_labels)
+
+# nx_pylab.draw_networkx_nodes = log_function_specific_for_draw_networkx_nodes(nx_pylab.draw_networkx_nodes)
+# nx_pylab.draw_networkx_edges = log_function_specific_for_draw_networkx_edges(nx_pylab.draw_networkx_edges)
+# nx_pylab.draw_networkx_labels = log_function_specific_for_draw_networkx_labels(nx_pylab.draw_networkx_labels)
+
+nx.draw_networkx_nodes = log_function_specific_for_draw_networkx_nodes(
+    nx.draw_networkx_nodes)
+nx.draw_networkx_edges = log_function_specific_for_draw_networkx_edges(
+    nx.draw_networkx_edges)
+nx.draw_networkx_labels = log_function_specific_for_draw_networkx_labels(
+    nx.draw_networkx_labels)
+
+Axes.quiver = log_function(Axes.quiver)
+
+Axes3D.scatter = log_function(Axes3D.scatter)
+Axes3D.plot = log_function(Axes3D.plot)
+Axes3D.plot_surface = log_function(Axes3D.plot_surface)
+Axes3D.bar3d = log_function(Axes3D.bar3d)
+Axes3D.bar = log_function(Axes3D.bar)
+Axes3D.add_collection3d = log_function(Axes3D.add_collection3d)
+
+Axes.pie = log_function(Axes.pie)
+
+Axes.fill = log_function(Axes.fill)
+
+squarify.plot = log_function(squarify.plot)
+
+Axes.imshow = log_function(Axes.imshow)
+Axes.pcolor = log_function(Axes.pcolor)
+NonUniformImage.__init__ = log_function(NonUniformImage.__init__)
+
+Axes.contour = log_function(Axes.contour)
+Axes.contourf = log_function(Axes.contourf)
+
+Ellipse.__init__ = log_function(Ellipse.__init__)
+Axes.broken_barh = log_function(Axes.broken_barh)
+
+Axes.tripcolor = log_function(Axes.tripcolor)
+
+VennDiagram.__init__ = log_function(VennDiagram.__init__)
+
+Circle.__init__ = log_function(Circle.__init__)
+
+# Axes.plot = log_function(Axes.plot)
+# Axes.loglog = log_function(Axes.loglog)
+# Axes.scatter = log_function(Axes.scatter)
+# Axes.bar = log_function(Axes.bar)
+# Axes.barh = log_function(Axes.barh)
+# Axes.axhline = log_function(Axes.axhline)
+# Axes.axvline = log_function(Axes.axvline)
+# Axes.errorbar = log_function(Axes.errorbar)
+# Axes.matshow = log_function(Axes.matshow)
+# Axes.hist = log_function(Axes.hist)
+# Axes.pie = log_function(Axes.pie)
+# Axes.boxplot = log_function(Axes.boxplot)
+# Axes.arrow = log_function(Axes.arrow)
+# Axes.fill_between = log_function(Axes.fill_between)
+# Axes.fill_betweenx = log_function(Axes.fill_betweenx)
+# Axes.imshow = log_function(Axes.imshow)
+# Axes.contour = log_function(Axes.contour)
+# Axes.contourf = log_function(Axes.contourf)
+# Axes.violinplot = log_function(Axes.violinplot)
+# Axes.violin = log_function(Axes.violin)
+
+# squarify.plot = log_function(squarify.plot)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..78b1efd1cca8c560dbaea82ea58e78268c963834
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_evaluator.py
@@ -0,0 +1,326 @@
+# flake8: noqa
+import os
+# from skimage.color import deltaE_cie76
+# from skimage.color import rgb2lab
+from itertools import permutations
+from multiprocessing import Pool, Process, cpu_count
+from typing import List, Tuple
+
+from ..eval_configs.global_config import run_script_safe
+from .color_utils import calculate_similarity_single, group_color
+
+# from dotenv import load_dotenv
+# load_dotenv()
+
+
+# sys.path.insert(0, os.environ["PROJECT_PATH"])
+
+
+
+
+
+
+# def hex_to_rgb(hex_color):
+#     hex_color = hex_color.lstrip('#')
+#     return tuple(int(hex_color[i:i+2], 16) / 255.0 for i in (0, 2, 4))
+
+# def calculate_similarity_single(c1, c2):
+#     c1_file = c1.split("--")[0]
+#     c2_file = c2.split("--")[0]
+
+#     c1_color = c1.split("--")[1]
+#     c2_color = c2.split("--")[1]
+
+#     if c1_file != c2_file:
+#         return 0
+#     elif c1_color.startswith("#") and c2_color.startswith("#"):
+
+#         c1_color = rgb2lab(np.array([hex_to_rgb(c1_color)]))
+#         c2_color = rgb2lab(np.array([hex_to_rgb(c2_color)]))
+
+#         return max(0, 1 - deltaE_cie76(c1_color, c2_color)[0] / 100)
+#     elif not c1_color.startswith("#") and not c2_color.startswith("#"):
+
+#         return 1 if c1_color == c2_color else 0
+#     else:
+#         return 0
+
+
+def calculate_similarity_for_permutation(args):
+    shorter, perm = args
+    current_similarity = sum(
+        calculate_similarity_single(c1, c2) for c1, c2 in zip(shorter, perm)
+    )
+    return current_similarity
+
+
+class ColorEvaluator:
+
+    def __init__(self) -> None:
+        self.metrics = {
+            "precision": 0,
+            "recall": 0,
+            "f1": 0,
+        }
+
+    def __call__(self, generation_code_file, golden_code_file):
+        # print("genearion_code_file", generation_code_file)
+        # print("golden_code_file", golden_code_file)
+
+        self.golden_code_file = golden_code_file
+
+        # print(f"generation_code_file: {generation_code_file}")
+        generation_colors = self._log_colors(generation_code_file)
+        # print(f"golden_code_file: {golden_code_file}")
+        golden_colors = self._log_colors(golden_code_file)
+        # print(f"len(generation_colors): {len(generation_colors)}")
+        # print(f"len(golden_colors): {len(golden_colors)}")
+
+        self._calculate_metrics(generation_colors, golden_colors)
+
+        # [TAG] What is this for?
+        # redunant_file = os.environ["PROJECT_PATH"] + "/" + os.path.basename(golden_code_file).replace(".py", ".pdf")
+        # os.remove(redunant_file)
+        # print(self.metrics)
+
+    def _log_colors(self, code_file):
+        """
+        Get text objects of the code
+        """
+
+        with open(code_file, "r") as f:
+            lines = f.readlines()
+        code = "".join(lines)
+
+        prefix = self._get_prefix()
+        output_file = code_file.replace(".py", "_log_colors.txt")
+        suffix = self._get_suffix(output_file)
+        code = prefix + code + suffix
+
+        code_log_texts_file = code_file.replace(".py", "_log_colors.py")
+        with open(code_log_texts_file, "w") as f:
+            f.write(code)
+
+        # os.system(f"python3 {code_log_texts_file}")
+        success = run_script_safe(code_log_texts_file)
+        if not success:
+            print("Skip downstream logic due to previous failure.")
+            # optionally return default result or continue
+
+        if os.path.exists(output_file):
+            with open(output_file, "r") as f:
+                colors = f.read()
+                try:
+                    colors = eval(colors)
+                except BaseException:
+                    colors = []
+            os.remove(output_file)
+        else:
+            colors = []
+
+        os.remove(code_log_texts_file)
+
+        # pdf_file = re.findall(r"plt\.savefig\('(.*)'\)", code)
+        # if len(pdf_file) != 0:
+        # pdf_file = pdf_file[0]
+        # if os.path.basename(pdf_file) == pdf_file:
+        # os.remove(pdf_file)
+
+        return colors
+
+    def _calculate_metrics(
+        self, generation_colors: List[Tuple], golden_colors: List[Tuple]
+    ):
+        generation_colors = list(generation_colors)
+        golden_colors = list(golden_colors)
+
+        if len(generation_colors) == 0 or len(golden_colors) == 0:
+            self.metrics["precision"] = 0
+            self.metrics["recall"] = 0
+            self.metrics["f1"] = 0
+            return
+
+        group_generation_colors = group_color(generation_colors)
+        group_golden_colors = group_color(golden_colors)
+
+        # print("group_generation_colors", group_generation_colors)
+        # print("group_golden_colors", group_golden_colors)
+
+        # print("generation_colors", generation_colors)
+        # print("golden_colors", golden_colors)
+
+        def calculate_similarity_serial(lst1, lst2):
+            if len(lst1) == 0 or len(lst2) == 0:
+                return 0
+
+            shorter, longer = (lst1, lst2) if len(lst1) <= len(lst2) else (lst2, lst1)
+
+            max_total_similarity = float("-inf")
+            best_index = None
+
+            for perm in permutations(longer, len(shorter)):
+                current_similarity = sum(
+                    calculate_similarity_single(c1, c2) for c1, c2 in zip(shorter, perm)
+                )
+                current_similarity /= len(shorter)
+
+                if current_similarity > max_total_similarity:
+                    max_total_similarity = current_similarity
+                    best_index = [shorter, perm]
+
+            # best_index[0] = sorted(best_index[0])
+            # best_index[1] = sorted(best_index[1])
+            # print("best_index", best_index)
+            for i1, i2 in zip(best_index[0], best_index[1]):
+                print(i1, i2)
+            tmp_similarity = sum(
+                calculate_similarity_single(c1, c2)
+                for c1, c2 in zip(best_index[0], best_index[1])
+            ) / len(shorter)
+            print("tmp_similarity", tmp_similarity)
+
+            return max_total_similarity
+
+        def calculate_similarity_parallel(lst1, lst2):
+            if len(lst1) == 0 or len(lst2) == 0:
+                return 0
+
+            shorter, longer = (lst1, lst2) if len(lst1) <= len(lst2) else (lst2, lst1)
+            perms = permutations(longer, len(shorter))
+
+            # create processes according to the number of CPUs
+            with Pool(processes=cpu_count()) as pool:
+                similarities = pool.map(
+                    calculate_similarity_for_permutation,
+                    [(shorter, perm) for perm in perms],
+                )
+
+            # print("length of similarities", len(similarities))
+
+            # indexes = [item[0] for item in similarities]
+            # similarities = [item[1] for item in similarities]
+
+            # get max similarity and its index
+            # max_total_similarity = max(similarities)
+            # max_index = similarities.index(max_total_similarity)
+            # index = indexes[max_index]
+
+            # max_total_similarity = max(similarities)
+            # index[0] = sorted(index[0])
+            # index[1] = sorted(index[1])
+            # for i1, i2 in zip(index[0], index[1]):
+            # print(i1, i2)
+
+            # tmp_similarity = sum( calculate_similarity_single(c1, c2) for c1, c2 in zip(index[0], index[1]) ) / len(shorter)
+            # print("tmp_similarity", tmp_similarity)
+            # print("best_index", index)
+
+            return max(similarities)
+
+        # merge keys in group_generation_colors and group_golden_colors
+        merged_color_group = list(
+            set(list(group_generation_colors.keys()) + list(group_golden_colors.keys()))
+        )
+        for color in merged_color_group:
+            if color not in group_generation_colors:
+                group_generation_colors[color] = []
+            if color not in group_golden_colors:
+                group_golden_colors[color] = []
+
+        max_set_similarity = 0
+
+        for color in merged_color_group:
+            max_set_similarity += calculate_similarity_parallel(
+                group_generation_colors[color], group_golden_colors[color]
+            )
+
+        # self.metrics["similarity"] = calculate_similarity_parallel(generation_colors, golden_colors)
+        # max_set_similarity = calculate_similarity_parallel(generation_colors, golden_colors)
+        self.metrics["precision"] = (
+            max_set_similarity / len(generation_colors)
+            if len(generation_colors) != 0
+            else 0
+        )
+        if "box" in self.golden_code_file:
+            self.metrics["recall"] = (
+                max_set_similarity / len(golden_colors)
+                if len(golden_colors) != 0
+                else 0
+            )
+        else:
+            self.metrics["recall"] = max_set_similarity / len(golden_colors)
+        if self.metrics["precision"] + self.metrics["recall"] == 0:
+            self.metrics["f1"] = 0
+        else:
+            self.metrics["f1"] = (
+                2
+                * self.metrics["precision"]
+                * self.metrics["recall"]
+                / (self.metrics["precision"] + self.metrics["recall"])
+            )
+
+        return
+
+    def _get_prefix(self):
+        with open(
+            os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"]
+            + "/evaluator/color_evaluator_prefix.py",
+            "r",
+        ) as f:
+            prefix = f.read()
+        return prefix
+
+    def _get_suffix(self, output_file):
+        return f"""
+drawed_colors = list(set(drawed_colors))
+drawed_colors = update_drawed_colors(drawed_objects)
+if len(drawed_colors) > 10:
+    drawed_colors = filter_color(drawed_colors)
+# print("drawed_colors", drawed_colors)
+# print("len(drawed_colors)", len(drawed_colors))
+# print("Length of drawed_obejcts", len(drawed_objects))
+# print("drawed_objects", drawed_objects)
+with open('{output_file}', 'w') as f:
+    f.write(str(drawed_colors))
+"""
+
+
+if __name__ == "__main__":
+    # sys.path.insert(0, '/home/yc21/project/Princess-s-CHI')
+
+    evaluator = ColorEvaluator()
+    # evaluator = TextEvaluator()
+
+    golden_code_dir = f"{os.environ['PROJECT_PATH']}/dataset/ori_500/"
+    generation_code_dir = f"{os.environ['PROJECT_PATH']}/results/chart2code_Phi-3-vision-128k-instruct_DirectAgent_results/direct/"
+
+    # list python files in the directory
+    golden_code_files = [f for f in os.listdir(golden_code_dir) if f.endswith(".py")]
+
+    # for golden_code_file in golden_code_files:
+    # print(golden_code_file)
+    # generation_code_file = generation_code_dir + golden_code_file
+    # evaluator(generation_code_file, golden_code_dir + golden_code_file)
+
+    # write a multi-processing version
+    def _muti_process_run(rank, data, num_processes):
+        for i in range(rank, len(data), num_processes):
+            golden_code_file = data[i]
+            generation_code_file = generation_code_dir + golden_code_file
+            evaluator(generation_code_file, golden_code_dir + golden_code_file)
+
+    evaluator = ColorEvaluator()
+    processes = []
+    num_processes = 20
+    for rank in range(num_processes):
+        p = Process(
+            target=_muti_process_run, args=(rank, golden_code_files, num_processes)
+        )
+        p.start()
+        processes.append(p)
+    for p in processes:
+        p.join()
+
+    # golden_code_file = f"{os.environ['PROJECT_PATH']}/dataset/ori_500/line_5.py"
+    # generation_code_file = f"{os.environ['PROJECT_PATH']}/results/chart2code_gpt-4-vision-preview_DirectAgent_results/direct/line_5.py"
+    # evaluator(generation_code_file, golden_code_file)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_evaluator_prefix.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_evaluator_prefix.py
new file mode 100644
index 0000000000000000000000000000000000000000..bb0b64ba9c6aa49b8af00a5644ba662990f85790
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_evaluator_prefix.py
@@ -0,0 +1,840 @@
+# # flake8: noqa
+import os
+import warnings
+
+import squarify
+
+warnings.filterwarnings("ignore", category=UserWarning)
+warnings.filterwarnings("ignore", category=DeprecationWarning)
+warnings.filterwarnings("ignore", category=FutureWarning)
+
+import sys
+
+if os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"] not in sys.path:
+    sys.path.insert(0, os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"])
+
+import inspect
+
+import matplotlib
+import matplotlib.colors as mcolors
+import matplotlib.pyplot as plt
+import networkx
+import networkx as nx
+import networkx.drawing.nx_pylab as nx_pylab
+import numpy as np
+from evaluator.color_utils import filter_color
+from matplotlib.axes._axes import Axes
+from matplotlib.axes._base import _process_plot_var_args
+from matplotlib.image import NonUniformImage
+from matplotlib.patches import Circle, Ellipse
+from matplotlib.projections.polar import PolarAxes
+from matplotlib_venn._common import VennDiagram
+from mpl_toolkits.mplot3d import Axes3D
+
+# from chart2code.utils.evaluator.color_utils import filter_color
+
+
+drawed_colors = []
+drawed_objects = {}
+in_decorator = False
+
+
+def convert_color_to_hex(color):
+    'Convert color from name, RGBA, or hex to a hex format.'
+    try:
+        # First, try to convert from color name to RGBA to hex
+        if isinstance(color, str):
+            # Check if it's already a hex color (start with '#' and length
+            # either 7 or 9)
+            if color.startswith('#') and (len(color) == 7 or len(color) == 9):
+                return color.upper()
+            else:
+                return mcolors.to_hex(mcolors.to_rgba(color)).upper()
+        # Then, check if it's in RGBA format
+        elif isinstance(color, (list, tuple, np.ndarray)) and (len(color) == 4 or len(color) == 3):
+            return mcolors.to_hex(color).upper()
+        else:
+            raise ValueError("Unsupported color format")
+    except ValueError as e:
+        print(color)
+        print("Error converting color:", e)
+        return None
+
+
+def log_function_specific_for_draw_networkx_labels(func):
+    def wrapper(
+        G,
+        pos,
+        labels=None,
+        font_size=12,
+        font_color="k",
+        font_family="sans-serif",
+        font_weight="normal",
+        alpha=None,
+        bbox=None,
+        horizontalalignment="center",
+        verticalalignment="center",
+        ax=None,
+        clip_on=True,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(
+                G,
+                pos,
+                labels=labels,
+                font_size=font_size,
+                font_color=font_color,
+                font_family=font_family,
+                font_weight=font_weight,
+                alpha=alpha,
+                bbox=bbox,
+                horizontalalignment=horizontalalignment,
+                verticalalignment=verticalalignment,
+                ax=ax,
+                clip_on=clip_on
+            )
+
+            for item in result.values():
+                color = convert_color_to_hex(item.get_color())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = item
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                labels=labels,
+                font_size=font_size,
+                font_color=font_color,
+                font_family=font_family,
+                font_weight=font_weight,
+                alpha=alpha,
+                bbox=bbox,
+                horizontalalignment=horizontalalignment,
+                verticalalignment=verticalalignment,
+                ax=ax,
+                clip_on=clip_on
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_specific_for_draw_networkx_edges(func):
+    def wrapper(
+        G,
+        pos,
+        edgelist=None,
+        width=1.0,
+        edge_color="k",
+        style="solid",
+        alpha=None,
+        arrowstyle=None,
+        arrowsize=10,
+        edge_cmap=None,
+        edge_vmin=None,
+        edge_vmax=None,
+        ax=None,
+        arrows=None,
+        label=None,
+        node_size=300,
+        nodelist=None,
+        node_shape="o",
+        connectionstyle="arc3",
+        min_source_margin=0,
+        min_target_margin=0,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(
+                G,
+                pos,
+                edgelist=edgelist,
+                width=width,
+                edge_color=edge_color,
+                style=style,
+                alpha=alpha,
+                arrowstyle=arrowstyle,
+                arrowsize=arrowsize,
+                edge_cmap=edge_cmap,
+                edge_vmin=edge_vmin,
+                edge_vmax=edge_vmax,
+                ax=ax,
+                arrows=arrows,
+                label=label,
+                node_size=node_size,
+                nodelist=nodelist,
+                node_shape=node_shape,
+                connectionstyle=connectionstyle,
+                min_source_margin=min_source_margin,
+                min_target_margin=min_target_margin
+            )
+
+            if isinstance(result, list):
+                for line in result:
+                    color = convert_color_to_hex(line.get_facecolor())
+                    drawed_colors.append(func_name + "--" + color)
+                if len(result) > 0:
+                    drawed_objects[func_name + "--" + color] = result
+            else:
+                for item in result.get_edgecolors().tolist():
+                    color = convert_color_to_hex(item)
+                    drawed_colors.append(func_name + "--" + color)
+                if len(result.get_edgecolors().tolist()) > 0:
+                    drawed_objects[func_name + "--" +
+                                   color] = result  # ! Attention
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                edgelist=edgelist,
+                width=width,
+                edge_color=edge_color,
+                style=style,
+                alpha=alpha,
+                arrowstyle=arrowstyle,
+                arrowsize=arrowsize,
+                edge_cmap=edge_cmap,
+                edge_vmin=edge_vmin,
+                edge_vmax=edge_vmax,
+                ax=ax,
+                arrows=arrows,
+                label=label,
+                node_size=node_size,
+                nodelist=nodelist,
+                node_shape=node_shape,
+                connectionstyle=connectionstyle,
+                min_source_margin=min_source_margin,
+                min_target_margin=min_target_margin
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_specific_for_draw_networkx_nodes(func):
+    def wrapper(
+        G,
+        pos,
+        nodelist=None,
+        node_size=300,
+        node_color="#1f78b4",
+        node_shape="o",
+        alpha=None,
+        cmap=None,
+        vmin=None,
+        vmax=None,
+        ax=None,
+        linewidths=None,
+        edgecolors=None,
+        label=None,
+        margins=None,
+    ):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(
+                G,
+                pos,
+                nodelist=nodelist,
+                node_size=node_size,
+                node_color=node_color,
+                node_shape=node_shape,
+                alpha=alpha,
+                cmap=cmap,
+                vmin=vmin,
+                vmax=vmax,
+                ax=ax,
+                linewidths=linewidths,
+                edgecolors=edgecolors,
+                label=label,
+                margins=margins
+            )
+
+            for item in result.get_facecolors().tolist():
+                color = convert_color_to_hex(item)
+                drawed_colors.append(func_name + "--" + color)
+            drawed_objects[func_name + "--" + color] = result
+
+            in_decorator = False
+        else:
+            return func(
+                G,
+                pos,
+                nodelist=nodelist,
+                node_size=node_size,
+                node_color=node_color,
+                node_shape=node_shape,
+                alpha=alpha,
+                cmap=cmap,
+                vmin=vmin,
+                vmax=vmax,
+                ax=ax,
+                linewidths=linewidths,
+                edgecolors=edgecolors,
+                label=label,
+                margins=margins
+            )
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function_for_3d(func):
+    def wrapper(*args, **kwargs):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(*args, **kwargs)
+
+            if func.__name__ == "scatter":
+                # check whether cmap is used
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    if isinstance(kwargs["cmap"], str):
+                        drawed_colors.append(
+                            func_name + "_3d--" + kwargs["cmap"])
+                        drawed_objects[func_name + "_3d--" +
+                                       kwargs["cmap"]] = result
+                    else:
+                        drawed_colors.append(
+                            func_name + "_3d--" + kwargs["cmap"].name)
+                        drawed_objects[func_name + "_3d--" +
+                                       kwargs["cmap"].name] = result
+                else:
+                    for item in result.get_facecolors().tolist():
+                        color = convert_color_to_hex(item)
+                        drawed_colors.append(func_name + "_3d--" + color)
+                    drawed_objects[func_name + "_3d--" +
+                                   color] = result  # ! Attention
+            elif func.__name__ == "plot":
+                for line in result:
+                    color = convert_color_to_hex(line.get_color())
+                    drawed_colors.append(func_name + "_3d--" + color)
+                    drawed_objects[func_name + "_3d--" + color] = line
+            elif func.__name__ == "plot_surface":
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    if isinstance(kwargs["cmap"], str):
+                        drawed_colors.append(
+                            func_name + "_3d--" + kwargs["cmap"])
+                        drawed_objects[func_name + "_3d--" +
+                                       kwargs["cmap"]] = result
+                    else:
+                        drawed_colors.append(
+                            func_name + "_3d--" + kwargs["cmap"].name)  # ! Attention
+                        drawed_objects[func_name + "_3d--" +
+                                       kwargs["cmap"].name] = result
+                else:
+                    colors = result.get_facecolors().tolist()
+                    drawed_colors.append(
+                        func_name +
+                        "_3d--" +
+                        convert_color_to_hex(
+                            colors[0]))
+                    # ! Attention
+                    drawed_objects[func_name + "_3d--" +
+                                   convert_color_to_hex(colors[0])] = result
+            elif func.__name__ == "bar3d":
+                colors = result.get_facecolors().tolist()
+                drawed_colors.append(
+                    func_name +
+                    "_3d--" +
+                    convert_color_to_hex(
+                        colors[0]))
+                # ! Attention
+                drawed_objects[func_name + "_3d--" +
+                               convert_color_to_hex(colors[0])] = result
+            elif func.__name__ == "bar":
+                for item in result:
+                    color = convert_color_to_hex(item.get_facecolor())
+                    drawed_colors.append(func_name + "_3d--" + color)
+                    drawed_objects[func_name + "_3d--" + color] = item
+            elif func.__name__ == "add_collection3d":
+                colors = result.get_facecolors().tolist()
+                for color in colors:
+                    drawed_colors.append(
+                        func_name + "_3d--" + convert_color_to_hex(color))
+                drawed_objects[func_name + "_3d--" +
+                               convert_color_to_hex(color)] = result
+
+            in_decorator = False
+        else:
+            return func(*args, **kwargs)
+        return result
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def log_function(func):
+    def wrapper(*args, **kwargs):
+        global drawed_colors
+        global in_decorator
+
+        if not in_decorator:
+            in_decorator = True
+
+            func_name = inspect.getfile(func) + "/" + func.__name__
+
+            result = func(*args, **kwargs)
+
+            if func.__name__ == "_makeline":
+                color = convert_color_to_hex(result[1]["color"])
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = result[0]
+            elif func.__name__ == "axhline":
+                color = convert_color_to_hex(result.get_color())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = result
+            elif func.__name__ == "axvline":
+                color = convert_color_to_hex(result.get_color())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = result
+            elif func.__name__ == "_fill_between_x_or_y":
+                color = convert_color_to_hex(list(result.get_facecolors()[0]))
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = result
+            elif func.__name__ == "bar":
+                for item in result:
+                    color = convert_color_to_hex(
+                        list(item._original_facecolor))
+                    drawed_colors.append(func_name + "--" + color)
+                    drawed_objects[func_name + "--" + color] = item
+            elif func.__name__ == "scatter" and not isinstance(args[0], PolarAxes):
+                # check whether cmap is used
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    if isinstance(kwargs["cmap"], str):
+                        drawed_colors.append(func_name + "--" + kwargs["cmap"])
+                        drawed_objects[func_name + "--" +
+                                       kwargs["cmap"]] = result
+                    else:
+                        drawed_colors.append(
+                            func_name + "--" + kwargs["cmap"].name)  # ! Attention
+                        drawed_objects[func_name + "--" +
+                                       kwargs["cmap"].name] = result
+                else:
+                    if len(result.get_facecolor()) != 0:
+                        color = convert_color_to_hex(
+                            list(result.get_facecolor()[0]))
+                        drawed_colors.append(func_name + "--" + color)
+                        drawed_objects[func_name + "--" + color] = result
+            elif func.__name__ == "pie":
+                for item in result[0]:
+                    color = convert_color_to_hex(item.get_facecolor())
+                    drawed_colors.append(func_name + "--" + color)
+                    drawed_objects[func_name + "--" + color] = item
+            elif func.__name__ == "axvspan":
+                color = convert_color_to_hex(result.get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = result
+            elif func.__name__ == "axhspan":
+                color = convert_color_to_hex(result.get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = result
+            elif func.__name__ == "hlines":
+                for item in result.get_edgecolors():
+                    color = convert_color_to_hex(list(item))
+                    drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" +
+                               color] = result   # ! Attention
+            elif func.__name__ == "vlines":
+                for item in result.get_edgecolors():
+                    color = convert_color_to_hex(list(item))
+                    drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" +
+                               color] = result   # ! Attention
+            elif func.__name__ == "boxplot":
+                for item in result["boxes"]:
+                    if isinstance(item, matplotlib.patches.PathPatch):
+                        color = convert_color_to_hex(
+                            list(item.get_facecolor()))
+                        drawed_colors.append(func_name + "--" + color)
+                        drawed_objects[func_name + "--" +
+                                       color] = item  # ! Attention
+            elif func.__name__ == "violinplot":
+                for item in result["bodies"]:
+                    color = convert_color_to_hex(list(item.get_facecolor()[0]))
+                    drawed_colors.append(func_name + "--" + color)
+                    drawed_objects[func_name + "--" +
+                                   color] = item  # ! Attention
+            elif func.__name__ == "hist":
+                tops, bins, patches = result
+                if not isinstance(patches, matplotlib.cbook.silent_list):
+                    for item in patches:
+                        color = convert_color_to_hex(
+                            list(item.get_facecolor()))
+                        drawed_colors.append(func_name + "--" + color)
+                        drawed_objects[func_name + "--" + color] = item
+                else:
+                    for container in patches:
+                        for item in container:
+                            color = convert_color_to_hex(
+                                list(item.get_facecolor()))
+                            drawed_colors.append(func_name + "--" + color)
+                            drawed_objects[func_name + "--" + color] = item
+            elif func.__name__ == "quiver":
+                for item in result.get_facecolors().tolist():
+                    color = convert_color_to_hex(item)
+                    drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" +
+                               color] = result  # ! Attention
+            elif func.__name__ == "plot" and len(args) > 0 and isinstance(args[0], PolarAxes):
+                lines = result
+                for line in lines:
+                    color = convert_color_to_hex(line.get_color())
+                    # print("color", color)
+                    drawed_colors.append(func_name + "_polar" + "--" + color)
+                    drawed_objects[func_name + "_polar" + "--" + color] = line
+            elif func.__name__ == "scatter" and isinstance(args[0], PolarAxes):
+                # check whether cmap is used
+                if "cmap" in kwargs and kwargs["cmap"] is not None:
+                    print("cmap is used", kwargs["cmap"])
+                    if isinstance(kwargs["cmap"], str):
+                        drawed_colors.append(
+                            func_name + "_polar" + "--" + kwargs["cmap"])
+                        drawed_objects[func_name +
+                                       "_polar--" + kwargs["cmap"]] = result
+                    else:
+                        drawed_colors.append(
+                            func_name + "_polar" + "--" + kwargs["cmap"].name)
+                        drawed_objects[func_name + "_polar" +
+                                       "--" + kwargs["cmap"].name] = result
+                else:
+                    if len(result.get_facecolor()) != 0:
+                        color = convert_color_to_hex(
+                            list(result.get_facecolor()[0]))
+                        drawed_colors.append(
+                            func_name + "_polar" + "--" + color)
+                        drawed_objects[func_name + "_polar" +
+                                       "--" + color] = result  # ! Attention
+            elif func.__name__ == "plot" and "squarify" in func_name:
+                # get ax
+                ax = result
+                # get container
+                containers = ax.containers
+                for container in containers:
+                    for item in container:
+                        color = convert_color_to_hex(
+                            list(item.get_facecolor()))
+                        drawed_colors.append(
+                            func_name + "_squarify" + "--" + color)
+                        drawed_objects[func_name +
+                                       "_squarify" + "--" + color] = item
+            elif func.__name__ == "imshow":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+                drawed_objects[func_name + "--" +
+                               colormap] = result  # ! Attention
+            elif func.__name__ == "pcolor":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+                drawed_objects[func_name + "--" +
+                               colormap] = result  # ! Attention
+            elif func.__name__ == "contour":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+                drawed_objects[func_name + "--" +
+                               colormap] = result  # ! Attention
+            elif func.__name__ == "contourf":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+                drawed_objects[func_name + "--" +
+                               colormap] = result  # ! Attention
+            elif func.__name__ == "fill":
+                patches = result
+                for patch in patches:
+                    color = convert_color_to_hex(list(patch.get_facecolor()))
+                    drawed_colors.append(func_name + "--" + color)
+                    drawed_objects[func_name + "--" + color] = patch
+            elif func.__name__ == "__init__" and isinstance(args[0], NonUniformImage):
+                colormap = args[0].get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+                drawed_objects[func_name + "--" + colormap] = args[0]
+            elif func.__name__ == "broken_barh":
+                colors = result.get_facecolors().tolist()
+                for color in colors:
+                    drawed_colors.append(
+                        func_name + "--" + convert_color_to_hex(color))
+                drawed_objects[func_name + "--" +
+                               convert_color_to_hex(color)] = result
+            elif func.__name__ == "__init__" and isinstance(args[0], Ellipse):
+                color = convert_color_to_hex(args[0].get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = args[0]
+            elif func.__name__ == "tripcolor":
+                colormap = result.get_cmap().name
+                drawed_colors.append(func_name + "--" + colormap)
+                drawed_objects[func_name + "--" +
+                               colormap] = result  # ! Attention
+            elif func.__name__ == "__init__" and isinstance(args[0], VennDiagram):
+                for item in args[0].patches:
+                    color = convert_color_to_hex(item.get_facecolor())
+                    drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = args[0]
+            elif func.__name__ == "__init__" and isinstance(args[0], Circle):
+                color = convert_color_to_hex(args[0].get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+                drawed_objects[func_name + "--" + color] = args[0]
+            in_decorator = False
+        else:
+            return func(*args, **kwargs)
+        return result
+
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+
+def update_drawed_colors(drawed_obejcts):
+    drawed_colors = []
+    for name, obj in drawed_objects.items():
+        func_name = name.split("--")[0]
+        color = name.split("--")[1]
+
+        if "/_makeline" in func_name:
+            color = convert_color_to_hex(obj.get_color())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/axhline" in func_name:
+            color = convert_color_to_hex(obj.get_color())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/axvline" in func_name:
+            color = convert_color_to_hex(obj.get_color())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/_fill_between_x_or_y" in func_name:
+            color = convert_color_to_hex(list(obj.get_facecolors()[0]))
+            drawed_colors.append(func_name + "--" + color)
+        elif "/bar" in func_name and "_3d" not in func_name:
+            color = convert_color_to_hex(list(obj._original_facecolor))
+            if color is not None:
+                drawed_colors.append(func_name + "--" + color)
+        elif "/scatter" in func_name and "polar" not in func_name and "3d" not in func_name:
+            # check whether cmap is used by checking whether color is hex
+            if color.startswith("#") is False:
+                drawed_colors.append(func_name + "--" + color)
+            else:
+                if len(obj.get_facecolor()) != 0:
+                    color = convert_color_to_hex(list(obj.get_facecolor()[0]))
+                    drawed_colors.append(func_name + "--" + color)
+        elif "/pie" in func_name:
+            color = convert_color_to_hex(obj.get_facecolor())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/axvspan" in func_name:
+            color = convert_color_to_hex(obj.get_facecolor())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/axhspan" in func_name:
+            color = convert_color_to_hex(obj.get_facecolor())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/hlines" in func_name:
+            for item in obj.get_edgecolors():
+                color = convert_color_to_hex(list(item))
+                drawed_colors.append(func_name + "--" + color)
+        elif "/vlines" in func_name:
+            for item in obj.get_edgecolors():
+                color = convert_color_to_hex(list(item))
+                drawed_colors.append(func_name + "--" + color)
+        elif "/boxplot" in func_name:
+            color = convert_color_to_hex(list(obj.get_facecolor()))
+            drawed_colors.append(func_name + "--" + color)
+        elif "/violinplot" in func_name:
+            color = convert_color_to_hex(list(obj.get_facecolor()[0]))
+            drawed_colors.append(func_name + "--" + color)
+        elif "/hist" in func_name:
+            color = convert_color_to_hex(list(obj.get_facecolor()))
+            drawed_colors.append(func_name + "--" + color)
+        elif "/quiver" in func_name:
+            for item in obj.get_facecolors().tolist():
+                color = convert_color_to_hex(item)
+                drawed_colors.append(func_name + "--" + color)
+        elif "/plot" in func_name and "polar" in func_name:
+            color = convert_color_to_hex(obj.get_color())
+            drawed_colors.append(func_name + "_polar--" + color)
+        elif "/scatter" in func_name and "polar" in func_name:
+            # check whether cmap is used by checking whether color is hex
+            if color.startswith("#") is False:
+                drawed_colors.append(func_name + "_polar--" + color)
+            else:
+                if len(obj.get_facecolor()) != 0:
+                    color = convert_color_to_hex(list(obj.get_facecolor()[0]))
+                    drawed_colors.append(func_name + "_polar--" + color)
+        elif "/plot" in func_name and "_squarify" in func_name:
+            color = convert_color_to_hex(list(obj.get_facecolor()))
+            drawed_colors.append(func_name + "--" + color)
+        elif "/imshow" in func_name:
+            colormap = obj.get_cmap().name
+            drawed_colors.append(func_name + "--" + colormap)
+        elif "/pcolor" in func_name:
+            colormap = obj.get_cmap().name
+            drawed_colors.append(func_name + "--" + colormap)
+        elif "/contour" in func_name:
+            colormap = obj.get_cmap().name
+            drawed_colors.append(func_name + "--" + colormap)
+        elif "/contourf" in func_name:
+            colormap = obj.get_cmap().name
+            drawed_colors.append(func_name + "--" + colormap)
+        elif "/fill" in func_name:
+            color = convert_color_to_hex(list(obj.get_facecolor()))
+            drawed_colors.append(func_name + "--" + color)
+        elif "/__init__" in func_name and isinstance(obj, NonUniformImage):
+            colormap = obj.get_cmap().name
+            drawed_colors.append(func_name + "--" + colormap)
+        elif "/broken_barh" in func_name:
+            colors = obj.get_facecolors().tolist()
+            for color in colors:
+                drawed_colors.append(
+                    func_name + "--" + convert_color_to_hex(color))
+        elif "/__init__" in func_name and isinstance(obj, Ellipse):
+            color = convert_color_to_hex(obj.get_facecolor())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/tripcolor" in func_name:
+            colormap = obj.get_cmap().name
+            drawed_colors.append(func_name + "--" + colormap)
+        elif "/__init__" in func_name and isinstance(obj, VennDiagram):
+            for item in obj.patches:
+                color = convert_color_to_hex(item.get_facecolor())
+                drawed_colors.append(func_name + "--" + color)
+        elif "/__init__" in func_name and isinstance(obj, Circle):
+            color = convert_color_to_hex(obj.get_facecolor())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/scatter" in func_name and "3d" in func_name:
+            # check whether cmap is used by checking whether color is hex
+            if color.startswith("#") is False:
+                drawed_colors.append(func_name + "_3d--" + color)
+            else:
+                for item in obj.get_facecolors().tolist():
+                    color = convert_color_to_hex(item)
+                    drawed_colors.append(func_name + "_3d--" + color)
+        elif "/plot" in func_name and "3d" in func_name and "plot_surface" not in func_name:
+            color = convert_color_to_hex(obj.get_color())
+            drawed_colors.append(func_name + "_3d--" + color)
+        elif "/plot_surface" in func_name:
+            if color.startswith("#") is False:
+                drawed_colors.append(func_name + "_3d--" + color)
+            else:
+                colors = obj.get_facecolors().tolist()
+                drawed_colors.append(
+                    func_name +
+                    "_3d--" +
+                    convert_color_to_hex(
+                        colors[0]))
+        elif "/bar3d" in func_name:
+            colors = obj.get_facecolors().tolist()
+            drawed_colors.append(
+                func_name +
+                "_3d--" +
+                convert_color_to_hex(
+                    colors[0]))
+        elif "/bar" in func_name and "3d" in func_name:
+            color = convert_color_to_hex(obj.get_facecolor())
+            drawed_colors.append(func_name + "_3d--" + color)
+        elif "/add_collection3d" in func_name:
+            colors = obj.get_facecolors().tolist()
+            for color in colors:
+                drawed_colors.append(
+                    func_name + "_3d--" + convert_color_to_hex(color))
+        elif "/draw_networkx_labels" in func_name:
+            color = convert_color_to_hex(obj.get_color())
+            drawed_colors.append(func_name + "--" + color)
+        elif "/draw_networkx_edges" in func_name:
+            if isinstance(obj, list):
+                for line in obj:
+                    color = convert_color_to_hex(line.get_facecolor())
+                    drawed_colors.append(func_name + "--" + color)
+            else:
+                for item in obj.get_edgecolors().tolist():
+                    color = convert_color_to_hex(item)
+                    drawed_colors.append(func_name + "--" + color)
+        elif "/draw_networkx_nodes" in func_name:
+            for item in obj.get_facecolors().tolist():
+                color = convert_color_to_hex(item)
+                drawed_colors.append(func_name + "--" + color)
+
+    drawed_colors = list(set(drawed_colors))
+
+    return drawed_colors
+
+try:
+    _process_plot_var_args._makeline = log_function(
+        _process_plot_var_args._makeline)
+except:
+    _process_plot_var_args._make_line = log_function(
+        _process_plot_var_args._make_line)
+Axes.bar = log_function(Axes.bar)
+Axes.scatter = log_function(Axes.scatter)
+Axes.axhline = log_function(Axes.axhline)
+Axes.axvline = log_function(Axes.axvline)
+Axes._fill_between_x_or_y = log_function(Axes._fill_between_x_or_y)
+Axes.pie = log_function(Axes.pie)
+Axes.axvspan = log_function(Axes.axvspan)
+Axes.axhspan = log_function(Axes.axhspan)
+Axes.hlines = log_function(Axes.hlines)
+Axes.vlines = log_function(Axes.vlines)
+Axes.boxplot = log_function(Axes.boxplot)
+Axes.violinplot = log_function(Axes.violinplot)
+Axes.hist = log_function(Axes.hist)
+# Axes.plot = log_function(Axes.plot)
+PolarAxes.plot = log_function(PolarAxes.plot)
+Axes.quiver = log_function(Axes.quiver)
+Axes.imshow = log_function(Axes.imshow)
+Axes.pcolor = log_function(Axes.pcolor)
+Axes.contour = log_function(Axes.contour)
+Axes.contourf = log_function(Axes.contourf)
+Axes.fill = log_function(Axes.fill)
+NonUniformImage.__init__ = log_function(NonUniformImage.__init__)
+Ellipse.__init__ = log_function(Ellipse.__init__)
+Axes.broken_barh = log_function(Axes.broken_barh)
+
+nx_pylab.draw_networkx_nodes = log_function_specific_for_draw_networkx_nodes(
+    nx_pylab.draw_networkx_nodes)
+nx_pylab.draw_networkx_edges = log_function_specific_for_draw_networkx_edges(
+    nx_pylab.draw_networkx_edges)
+nx_pylab.draw_networkx_labels = log_function_specific_for_draw_networkx_labels(
+    nx_pylab.draw_networkx_labels)
+
+nx.draw_networkx_nodes = log_function_specific_for_draw_networkx_nodes(
+    nx.draw_networkx_nodes)
+nx.draw_networkx_edges = log_function_specific_for_draw_networkx_edges(
+    nx.draw_networkx_edges)
+nx.draw_networkx_labels = log_function_specific_for_draw_networkx_labels(
+    nx.draw_networkx_labels)
+
+
+squarify.plot = log_function(squarify.plot)
+
+Axes3D.scatter = log_function_for_3d(Axes3D.scatter)
+Axes3D.plot = log_function_for_3d(Axes3D.plot)
+Axes3D.plot_surface = log_function_for_3d(Axes3D.plot_surface)
+Axes3D.bar3d = log_function_for_3d(Axes3D.bar3d)
+Axes3D.bar = log_function_for_3d(Axes3D.bar)
+Axes3D.add_collection3d = log_function_for_3d(Axes3D.add_collection3d)
+
+Axes.tripcolor = log_function(Axes.tripcolor)
+
+VennDiagram.__init__ = log_function(VennDiagram.__init__)
+
+Circle.__init__ = log_function(Circle.__init__)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..d2f504dd49f5cfb6830ff7bd17e463f93cbbbef2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/color_utils.py
@@ -0,0 +1,85 @@
+import numpy as np
+
+# This is a patch for color map, which is not updated for newer version of
+# numpy
+
+
+def patch_asscalar(a):
+    return a.item()
+
+
+setattr(np, "asscalar", patch_asscalar)
+
+
+def hex_to_rgb(hex_color):
+    hex_color = hex_color.lstrip('#')
+    return tuple(int(hex_color[i:i + 2], 16) for i in (0, 2, 4))
+
+
+def rgb_to_lab(rgb):
+    """
+    Convert an RGB color to Lab color space.
+    RGB values should be in the range [0, 255].
+    """
+    # Create an sRGBColor object from RGB values
+    from colormath.color_conversions import convert_color
+    from colormath.color_objects import LabColor, sRGBColor
+    rgb_color = sRGBColor(rgb[0], rgb[1], rgb[2], is_upscaled=True)
+
+    # Convert to Lab color space
+    lab_color = convert_color(rgb_color, LabColor)
+
+    return lab_color
+
+
+def calculate_similarity_single(c1, c2):
+    if c1.startswith("#") and c2.startswith("#"):
+        # c1 = rgb2lab(np.array([hex_to_rgb(c1)]))
+        # c2 = rgb2lab(np.array([hex_to_rgb(c2)]))
+        c1 = hex_to_rgb(c1)
+        c2 = hex_to_rgb(c2)
+        lab1 = rgb_to_lab(c1)
+        lab2 = rgb_to_lab(c2)
+        # return max(0, 1 - deltaE_cie76(c1, c2)[0] / 100)
+        from colormath.color_diff import delta_e_cie2000
+        return max(0, 1 - (delta_e_cie2000(lab1, lab2) / 100))
+    elif not c1.startswith("#") and not c2.startswith("#"):
+
+        return 1 if c1 == c2 else 0
+    else:
+        return 0
+
+
+def filter_color(color_list):
+    filtered_color_list = []
+    len_color_list = len(color_list)
+    for i in range(len_color_list):
+        if i != 0:
+            put_in = True
+            for item in filtered_color_list:
+                similarity = calculate_similarity_single(
+                    color_list[i].split("--")[1], item.split("--")[1])
+                if similarity > 0.7:
+                    put_in = False
+                    break
+            if put_in:
+                filtered_color_list.append(color_list[i])
+        else:
+            filtered_color_list.append(color_list[i])
+    # print("Filtered color list: ", filtered_color_list)
+    return filtered_color_list
+
+
+def group_color(color_list):
+    color_dict = {}
+
+    for color in color_list:
+        chart_type = color.split("--")[0]
+        color = color.split("--")[1]
+
+        if chart_type not in color_dict:
+            color_dict[chart_type] = [color]
+        else:
+            color_dict[chart_type].append(color)
+
+    return color_dict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/grid_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/grid_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..67b5ed12589ce84502df54ce78c84f60f917b73d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/grid_evaluator.py
@@ -0,0 +1,183 @@
+# flake8: noqa
+import os
+from typing import List, Tuple
+
+from ..eval_configs.global_config import run_script_safe
+
+# from dotenv import load_dotenv
+# load_dotenv()
+
+# sys.path.insert(0, os.environ["PROJECT_PATH"])
+
+
+
+
+class GridEvaluator:
+
+    def __init__(self) -> None:
+        self.metrics = {
+            "precision": 0,
+            "recall": 0,
+            "f1": 0
+        }
+
+    def __call__(self, generation_code_file, golden_code_file):
+        generation_grids = self._log_legends(generation_code_file)
+        golden_grids = self._log_legends(golden_code_file)
+
+        self._calculate_metrics(generation_grids, golden_grids)
+
+        # redunant_file = os.environ["PROJECT_PATH"] + "/" + os.path.basename(golden_code_file).replace(".py", ".pdf")
+        # os.remove(redunant_file)
+        # print(self.metrics)
+
+    def _log_legends(self, code_file):
+        """
+        Get legend objects of the code
+        """
+
+        with open(code_file, 'r') as f:
+            lines = f.readlines()
+        code = ''.join(lines)
+
+        prefix = self._get_prefix()
+        output_file = code_file.replace(".py", ".txt")
+        suffix = self._get_suffix(output_file)
+        code = prefix + code + suffix
+
+        code_log_texts_file = code_file.replace(".py", "_log_legends.py")
+        with open(code_log_texts_file, 'w') as f:
+            f.write(code)
+
+        # os.system(f"python3 {code_log_texts_file}")
+        success = run_script_safe(code_log_texts_file)
+        if not success:
+            print("Skip downstream logic due to previous failure.")
+            # optionally return default result or continue
+
+        with open(output_file, 'r') as f:
+            texts = f.read()
+            texts = eval(texts)
+
+        os.remove(code_log_texts_file)
+        os.remove(output_file)
+
+        # pdf_file = re.findall(r"plt\.savefig\('(.*)'\)", code)
+        # if len(pdf_file) != 0:
+        # pdf_file = pdf_file[0]
+        # if os.path.basename(pdf_file) == pdf_file:
+        # os.remove(pdf_file)
+
+        return texts
+
+    def _calculate_metrics(
+            self,
+            generation_grids: List[Tuple],
+            golden_grids: List[Tuple]):
+        """
+        Calculate the metrics
+
+        Args:
+            - generation_grids: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+            - golden_grids: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+        """
+        if len(generation_grids) == 0 or len(golden_grids) == 0:
+            self.metrics["precision"] = 0
+            self.metrics["recall"] = 0
+            self.metrics["f1"] = 0
+            return
+
+        len_generation = len(generation_grids)
+        len_golden = len(golden_grids)
+
+        n_correct = 0
+        for t in golden_grids:
+            if t in generation_grids:
+                n_correct += 1
+                generation_grids.remove(t)
+
+        self.metrics["precision"] = n_correct / len_generation
+        self.metrics["recall"] = n_correct / len_golden
+        if self.metrics["precision"] + self.metrics["recall"] == 0:
+            self.metrics["f1"] = 0
+        else:
+            self.metrics["f1"] = 2 * self.metrics["precision"] * \
+                self.metrics["recall"] / (self.metrics["precision"] + self.metrics["recall"])
+
+        return
+
+    def _get_prefix(self):
+        sys_to_add = os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"]
+        # assert sys_to_add not empty
+        assert sys_to_add != "", "VLMEVAL_CHARTMIMIC_UTILS_PATH is not set"
+        return f"""
+import warnings
+warnings.filterwarnings("ignore", category=UserWarning)
+
+if "{sys_to_add}" not in sys.path:
+    sys.path.insert(0, "{sys_to_add}")
+
+import eval_configs.global_config as global_config
+global_config.reset_texts()
+from matplotlib.backends.backend_pdf import RendererPdf
+
+grid_visibility = []
+"""
+
+    def _get_suffix(self, output_file):
+        return f"""
+
+all_axes = plt.gcf().get_axes()
+
+for ax in all_axes:
+    subplot_spec = ax.get_subplotspec()
+    row = subplot_spec.rowspan.start
+    col = subplot_spec.colspan.start
+    x_grid_visible = any(line.get_visible() for line in ax.get_xgridlines())
+    y_grid_visible = any(line.get_visible() for line in ax.get_ygridlines())
+
+    grid_visibility.append(
+        dict(
+            row=row,
+            col=col,
+            x_grid_visible=x_grid_visible,
+            y_grid_visible=y_grid_visible
+        )
+    )
+
+# sort the grid visibility by row and col
+grid_visibility = sorted(grid_visibility, key=lambda x: (x['row'], x['col']))
+
+# Since there can be twin axes, we need to merge the grid visibility, if they are in the same row and col, use "or" to merge
+grid_visibility_merged = []
+for i, grid in enumerate(grid_visibility):
+    if i == 0:
+        grid_visibility_merged.append(grid)
+        continue
+
+    last_grid = grid_visibility_merged[-1]
+    if last_grid['row'] == grid['row'] and last_grid['col'] == grid['col']:
+        last_grid['x_grid_visible'] = last_grid['x_grid_visible'] or grid['x_grid_visible']
+        last_grid['y_grid_visible'] = last_grid['y_grid_visible'] or grid['y_grid_visible']
+    else:
+        grid_visibility_merged.append(grid)
+
+grid_visibility = grid_visibility_merged
+
+# print(grid_visibility)
+with open('{output_file}', 'w') as f:
+    f.write(str(grid_visibility))
+"""
+
+
+if __name__ == "__main__":
+    # sys.path.insert(0, '/home/yc21/project/Princess-s-CHI')
+
+    evaluator = GridEvaluator()
+
+    for idx in range(1, 40):
+        print(f"Processing {idx}")
+        generation_code_file = f"/home/yc21/project/Princess-s-CHI/dataset/line/line_{idx}.py"
+        golden_code_file = f"/home/yc21/project/Princess-s-CHI/results/chart2code_gpt_DirectAgent_results/direct/line_{idx}.py"
+        evaluator(generation_code_file, golden_code_file)
+        print()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/layout_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/layout_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..03fd4ad96d5af2d4d0ee5092249df89f6f525f3d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/layout_evaluator.py
@@ -0,0 +1,168 @@
+# flake8: noqa
+import os
+from typing import List, Tuple
+
+from ..eval_configs.global_config import run_script_safe
+
+# from dotenv import load_dotenv
+# load_dotenv()
+
+# sys.path.insert(0, os.environ["PROJECT_PATH"])
+
+
+
+class LayoutEvaluator:
+
+    def __init__(self) -> None:
+        self.metrics = {
+            "precision": 0,
+            "recall": 0,
+            "f1": 0
+        }
+
+    def __call__(self, generation_code_file, golden_code_file):
+        generation_layouts = self._log_layouts(generation_code_file)
+        golden_layouts = self._log_layouts(golden_code_file)
+
+        self._calculate_metrics(generation_layouts, golden_layouts)
+
+        # redunant_file = os.environ["PROJECT_PATH"] + "/" + os.path.basename(golden_code_file).replace(".py", ".pdf")
+        # os.remove(redunant_file)
+
+        # print(self.metrics)
+
+    def _log_layouts(self, code_file):
+        """
+        Get objects of the code
+        """
+
+        with open(code_file, 'r') as f:
+            lines = f.readlines()
+        code = ''.join(lines)
+
+        prefix = self._get_prefix()
+        output_file = code_file.replace(".py", "_log_layouts.txt")
+        if "/graph" in code_file:
+            suffix = self._get_suffix_special_for_graph(output_file)
+        else:
+            suffix = self._get_suffix(output_file)
+
+        code = prefix + code + suffix
+
+        code_log_texts_file = code_file.replace(".py", "_log_layouts.py")
+        with open(code_log_texts_file, 'w') as f:
+            f.write(code)
+
+        # os.system(f"python3 {code_log_texts_file}")
+        success = run_script_safe(code_log_texts_file)
+        if not success:
+            print("Skip downstream logic due to previous failure.")
+            # optionally return default result or continue
+
+        if os.path.exists(output_file):
+            with open(output_file, 'r') as f:
+                texts = f.read()
+                texts = eval(texts)
+            os.remove(output_file)
+        else:
+            texts = []
+        os.remove(code_log_texts_file)
+
+        return texts
+
+    def _calculate_metrics(
+            self,
+            generation_layouts: List[Tuple],
+            golden_layouts: List[Tuple]):
+        """
+        Calculate the metrics
+
+        Args:
+            - generation_layouts: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+            - golden_layouts: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+        """
+        if len(generation_layouts) == 0 or len(golden_layouts) == 0:
+            self.metrics["precision"] = 0
+            self.metrics["recall"] = 0
+            self.metrics["f1"] = 0
+            return
+
+        len_generation = len(generation_layouts)
+        len_golden = len(golden_layouts)
+
+        n_correct = 0
+        for t in golden_layouts:
+            if t in generation_layouts:
+                n_correct += 1
+                generation_layouts.remove(t)
+
+        self.metrics["precision"] = n_correct / len_generation
+        self.metrics["recall"] = n_correct / len_golden
+        if self.metrics["precision"] + self.metrics["recall"] == 0:
+            self.metrics["f1"] = 0
+        else:
+            self.metrics["f1"] = 2 * self.metrics["precision"] * \
+                self.metrics["recall"] / (self.metrics["precision"] + self.metrics["recall"])
+
+        return
+
+    def _get_prefix(self):
+        return """
+import warnings
+warnings.filterwarnings("ignore", category=UserWarning)
+warnings.filterwarnings("ignore", category=DeprecationWarning)
+warnings.filterwarnings("ignore", category=FutureWarning)
+
+"""
+
+    def _get_suffix(self, output_file):
+        return f"""
+
+def get_gridspec_layout_info(fig):
+    layout_info = {{}}
+    for ax in fig.axes:
+        spec = ax.get_subplotspec()
+        if spec is None:
+            continue
+        gs = spec.get_gridspec()
+        nrows, ncols = gs.get_geometry()
+        row_start, row_end = spec.rowspan.start, spec.rowspan.stop - 1  # Zero-based and inclusive
+        col_start, col_end = spec.colspan.start, spec.colspan.stop - 1  # Zero-based and inclusive
+        layout_info[ax] = dict(nrows=nrows, ncols=ncols, row_start=row_start, row_end=row_end, col_start=col_start, col_end=col_end)
+    # print(layout_info)
+    layout_info = list(layout_info.values())
+    return layout_info
+
+layout_info = get_gridspec_layout_info(fig=plt.gcf())
+with open('{output_file}', 'w') as f:
+    f.write(str(layout_info))
+"""
+
+    def _get_suffix_special_for_graph(self, output_file):
+        return f"""
+def get_gridspec_layout_info(fig):
+    layout_info = {{}}
+    for ax in fig.axes:
+        layout_info[ax] = dict(nrows=1, ncols=1, row_start=0, row_end=1, col_start=0, col_end=1)
+    # print(layout_info)
+    layout_info = list(layout_info.values())
+    return layout_info
+
+layout_info = get_gridspec_layout_info(fig=plt.gcf())
+with open('{output_file}', 'w') as f:
+    f.write(str(layout_info))
+"""
+
+
+if __name__ == "__main__":
+
+    evaluator = LayoutEvaluator()
+
+    for idx in range(60, 61):
+        print(f"Processing {idx}")
+        # print("Processing Golden Code")
+        golden_code_file = f"{os.environ['PROJECT_PATH']}/dataset/ori/line_{idx}.py"
+        # print("Processing Generation Code")
+        generation_code_file = f"{os.environ['PROJECT_PATH']}/results/chart2code_gpt_ScaffoldAgent_results/scaffold/line_{idx}.py"
+        evaluator(generation_code_file, golden_code_file)
+        print()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/legend_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/legend_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..b9489e4dd1b6d9b1e5e68d570d18895ab13424db
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/legend_evaluator.py
@@ -0,0 +1,196 @@
+# flake8: noqa
+import os
+from typing import List, Tuple
+
+from ..eval_configs.global_config import run_script_safe
+
+# from dotenv import load_dotenv
+# load_dotenv()
+
+# sys.path.insert(0, os.environ["PROJECT_PATH"])
+
+
+
+
+class LegendEvaluator:
+
+    def __init__(self, use_position=True) -> None:
+        self.use_position = use_position
+        self.metrics = {
+            "precision": 0,
+            "recall": 0,
+            "f1": 0
+        }
+
+    def __call__(self, generation_code_file, golden_code_file):
+        generation_texts = self._log_legends(generation_code_file)
+        golden_texts = self._log_legends(golden_code_file)
+
+        self._calculate_metrics(generation_texts, golden_texts)
+
+        # redunant_file = os.environ["PROJECT_PATH"] + "/" + os.path.basename(golden_code_file).replace(".py", ".pdf")
+        # os.remove(redunant_file)
+        # print(self.metrics)
+
+    def _log_legends(self, code_file):
+        """
+        Get legend objects of the code
+        """
+
+        with open(code_file, 'r') as f:
+            lines = f.readlines()
+        code = ''.join(lines)
+
+        prefix = self._get_prefix()
+        output_file = code_file.replace(".py", ".txt")
+        suffix = self._get_suffix(output_file)
+        code = prefix + code + suffix
+
+        code_log_texts_file = code_file.replace(".py", "_log_legends.py")
+        with open(code_log_texts_file, 'w') as f:
+            f.write(code)
+
+        # os.system(f"python3 {code_log_texts_file}")
+        success = run_script_safe(code_log_texts_file)
+        if not success:
+            print("Skip downstream logic due to previous failure.")
+            # optionally return default result or continue
+
+        with open(output_file, 'r') as f:
+            texts = f.read()
+            texts = eval(texts)
+
+        os.remove(code_log_texts_file)
+        os.remove(output_file)
+
+        # pdf_file = re.findall(r"plt\.savefig\('(.*)'\)", code)
+        # if len(pdf_file) != 0:
+        # pdf_file = pdf_file[0]
+        # if os.path.basename(pdf_file) == pdf_file:
+        # os.remove(pdf_file)
+
+        return texts
+
+    def _calculate_metrics(
+            self,
+            generation_texts: List[Tuple],
+            golden_texts: List[Tuple]):
+        """
+        Calculate the metrics
+
+        Args:
+            - generation_texts: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+            - golden_texts: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+        """
+        if len(generation_texts) == 0 or len(golden_texts) == 0:
+            self.metrics["precision"] = 0
+            self.metrics["recall"] = 0
+            self.metrics["f1"] = 0
+            return
+
+        len_generation = len(generation_texts)
+        len_golden = len(golden_texts)
+
+        if not self.use_position:
+            generation_texts = [t[-1] for t in generation_texts]
+            golden_texts = [t[-1] for t in golden_texts]
+
+            n_correct = 0
+            for t in golden_texts:
+                if t in generation_texts:
+                    n_correct += 1
+                    generation_texts.remove(t)
+
+        else:
+            generation_texts = [t[2:] for t in generation_texts]
+            golden_texts = [t[2:] for t in golden_texts]
+
+            n_correct = 0
+            for t1 in golden_texts:
+                for t2 in generation_texts:
+                    # text must be equal, but x_rel and y_rel can be in a range
+                    if t1[-1] == t2[-1] and abs(t1[0] - t2[0]
+                                                ) <= 10 and abs(t1[1] - t2[1]) <= 10:
+                        # print("matched:", t2)
+                        n_correct += 1
+                        generation_texts.remove(t2)
+                        break
+
+        self.metrics["precision"] = n_correct / len_generation
+        self.metrics["recall"] = n_correct / len_golden
+        if self.metrics["precision"] + self.metrics["recall"] == 0:
+            self.metrics["f1"] = 0
+        else:
+            self.metrics["f1"] = 2 * self.metrics["precision"] * \
+                self.metrics["recall"] / (self.metrics["precision"] + self.metrics["recall"])
+
+        return
+
+    def _get_prefix(self):
+        sys_to_add = os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"]
+        # assert sys_to_add not empty
+        assert sys_to_add != "", "VLMEVAL_CHARTMIMIC_UTILS_PATH is not set"
+        return f"""
+import warnings
+warnings.filterwarnings("ignore", category=UserWarning)
+
+if "{sys_to_add}" not in sys.path:
+    sys.path.insert(0, "{sys_to_add}")
+
+import eval_configs.global_config as global_config
+global_config.reset_texts()
+from matplotlib.backends.backend_pdf import RendererPdf
+
+drawed_legend_texts = []
+drawed_texts = []
+
+def log_function(func):
+    def wrapper(*args, **kwargs):
+        global drawed_texts
+
+        object = args[0]
+        x = args[2]
+        y = args[3]
+        x_rel = ( x / object.width / 72 ) * 100
+        y_rel = ( y / object.height / 72 ) * 100
+        s = args[4]
+
+        drawed_texts.append( (x, y, x_rel, y_rel, s) )
+        return func(*args, **kwargs)
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+RendererPdf.draw_text = log_function(RendererPdf.draw_text)
+"""
+
+    def _get_suffix(self, output_file):
+        return f"""
+
+all_axes = plt.gcf().get_axes()
+legends = [ax.get_legend() for ax in all_axes if ax.get_legend() is not None]
+for legend in legends:
+    for t in legend.get_texts():
+        drawed_legend_texts.append(t.get_text())
+
+new_drawed_legend_texts = []
+for t1 in drawed_legend_texts:
+    for t2 in drawed_texts:
+        if t1 == t2[-1]:
+            new_drawed_legend_texts.append(t2)
+            break
+drawed_legend_texts = new_drawed_legend_texts
+
+with open('{output_file}', 'w') as f:
+    f.write(str(drawed_legend_texts))
+"""
+
+
+if __name__ == "__main__":
+    # sys.path.insert(0, '/home/yc21/project/Princess-s-CHI')
+
+    evaluator = LegendEvaluator()
+
+    generation_code_file = "/home/yc21/project/Princess-s-CHI/dataset/line/line_9.py"
+    golden_code_file = "/home/yc21/project/Princess-s-CHI/results/chart2code_gpt_DirectAgent_results/direct/line_9.py"
+
+    evaluator(generation_code_file, golden_code_file)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/text_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/text_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..2f19a23e87671e817a1fa9bd7ac031b9d5dea249
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/evaluator/text_evaluator.py
@@ -0,0 +1,208 @@
+# flake8: noqa
+import os
+from typing import List, Tuple
+
+from ..eval_configs.global_config import run_script_safe
+
+# from dotenv import load_dotenv
+# load_dotenv()
+
+
+
+
+class TextEvaluator:
+
+    def __init__(self, use_position=False, use_axs=True) -> None:
+        self.metrics = {
+            "precision": 0,
+            "recall": 0,
+            "f1": 0
+        }
+        self.use_position = use_position
+        self.use_axs = use_axs
+
+    def __call__(self, generation_code_file, golden_code_file):
+        generation_texts = self._log_texts(generation_code_file)
+        golden_texts = self._log_texts(golden_code_file)
+
+        self._calculate_metrics(generation_texts, golden_texts)
+
+        # [TAG] What is this for?
+        # print(f"os.getcwd(): {os.getcwd()}")
+        # breakpoint()
+        # redunant_file = os.environ["PROJECT_PATH"] + "/" + os.path.basename(golden_code_file).replace(".py", ".pdf")
+        # os.remove(redunant_file)
+        # print(self.metrics)
+
+    def _log_texts(self, code_file):
+        """
+        Get text objects of the code
+        """
+
+        with open(code_file, 'r') as f:
+            lines = f.readlines()
+        code = ''.join(lines)
+
+        prefix = self._get_prefix()
+        output_file = code_file.replace(".py", "_log_texts.txt")
+        suffix = self._get_suffix(output_file)
+        code = prefix + code + suffix
+
+        if not self.use_axs:
+            # find plt.savefig and append code before it
+            savefig_idx = code.find("plt.savefig")
+            ax_ticks_deletion_code = self._get_ax_ticks_deletion_code()
+            code = code[:savefig_idx] + \
+                ax_ticks_deletion_code + code[savefig_idx:]
+
+        code_log_texts_file = code_file.replace(".py", "_log_texts.py")
+        with open(code_log_texts_file, 'w') as f:
+            f.write(code)
+
+        # os.system(f"python3 {code_log_texts_file}")
+        success = run_script_safe(code_log_texts_file)
+        if not success:
+            print("Skip downstream logic due to previous failure.")
+            # optionally return default result or continue
+
+        if os.path.exists(output_file):
+            with open(output_file, 'r') as f:
+                texts = f.read()
+                try:
+                    texts = eval(texts)
+                except Exception as e:
+                    print(f"Error evaluating texts: {e}")
+                    print(f"Texts: {texts}")
+            os.remove(output_file)
+        else:
+            texts = []
+        os.remove(code_log_texts_file)
+
+        # pdf_file = re.findall(r"plt\.savefig\('(.*)'\)", code)
+        # if len(pdf_file) != 0:
+        # pdf_file = pdf_file[0]
+        # if os.path.basename(pdf_file) == pdf_file:
+        # os.remove(pdf_file)
+
+        return texts
+
+    def _calculate_metrics(
+            self,
+            generation_texts: List[Tuple],
+            golden_texts: List[Tuple]):
+        """
+        Calculate the metrics
+
+        Args:
+            - generation_texts: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+            - golden_texts: List of tuples of texts, [(x, y, x_rel, y_rel, text), ...]
+        """
+        if len(generation_texts) == 0 or len(golden_texts) == 0:
+            self.metrics["precision"] = 0
+            self.metrics["recall"] = 0
+            self.metrics["f1"] = 0
+            return
+
+        len_generation = len(generation_texts)
+        len_golden = len(golden_texts)
+
+        if not self.use_position:
+            generation_texts = [t[-1] for t in generation_texts]
+            golden_texts = [t[-1] for t in golden_texts]
+
+            n_correct = 0
+            for t in golden_texts:
+                if t in generation_texts:
+                    n_correct += 1
+                    generation_texts.remove(t)
+
+        else:
+            generation_texts = [t[2:] for t in generation_texts]
+            golden_texts = [t[2:] for t in golden_texts]
+
+            n_correct = 0
+            for t1 in golden_texts:
+                for t2 in generation_texts:
+                    # text must be equal, but x_rel and y_rel can be in a range
+                    if t1[-1] == t2[-1] and abs(t1[0] - t2[0]
+                                                ) <= 10 and abs(t1[1] - t2[1]) <= 10:
+                        # print("matched:", t2)
+                        n_correct += 1
+                        generation_texts.remove(t2)
+                        break
+
+        self.metrics["precision"] = n_correct / len_generation
+        self.metrics["recall"] = n_correct / len_golden
+        if self.metrics["precision"] + self.metrics["recall"] == 0:
+            self.metrics["f1"] = 0
+        else:
+            self.metrics["f1"] = 2 * self.metrics["precision"] * \
+                self.metrics["recall"] / (self.metrics["precision"] + self.metrics["recall"])
+
+        return
+
+    def _get_prefix(self):
+        sys_to_add = os.environ["VLMEVAL_CHARTMIMIC_UTILS_PATH"]
+        # assert sys_to_add not empty
+        assert sys_to_add != "", "VLMEVAL_CHARTMIMIC_UTILS_PATH is not set"
+        return f"""
+import warnings
+warnings.filterwarnings("ignore", category=UserWarning)
+warnings.filterwarnings("ignore", category=DeprecationWarning)
+warnings.filterwarnings("ignore", category=FutureWarning)
+
+import sys
+if "{sys_to_add}" not in sys.path:
+    sys.path.insert(0, "{sys_to_add}")
+
+import eval_configs.global_config as global_config
+global_config.reset_texts()
+from matplotlib.backends.backend_pdf import RendererPdf
+
+drawed_texts = []
+
+def log_function(func):
+    def wrapper(*args, **kwargs):
+        global drawed_texts
+
+        object = args[0]
+        x = args[2]
+        y = args[3]
+        x_rel = ( x / object.width / 72 ) * 100
+        y_rel = ( y / object.height / 72 ) * 100
+        s = args[4]
+
+        drawed_texts.append( (float(x), float(y), float(x_rel), float(y_rel), s) )
+        return func(*args, **kwargs)
+    wrapper.__name__ = func.__name__
+    return wrapper
+
+RendererPdf.draw_text = log_function(RendererPdf.draw_text)
+"""
+
+    def _get_suffix(self, output_file):
+        return f"""
+# print("drawed_texts", drawed_texts)
+with open('{output_file}', 'w') as f:
+    f.write(str(drawed_texts))
+"""
+
+    def _get_ax_ticks_deletion_code(self):
+        return """
+all_axes = plt.gcf().get_axes()
+for ax in all_axes:
+    ax.set_xticks([])
+    ax.set_yticks([])
+"""
+
+
+if __name__ == "__main__":
+    # sys.path.insert(0, '/home/yc21/project/Princess-s-CHI')
+
+    evaluator = TextEvaluator(use_axs=False)
+    # evaluator = TextEvaluator()
+
+    generation_code_file = "/home/yc21/project/Princess-s-CHI/dataset/line/line_7.py"
+    golden_code_file = "/home/yc21/project/Princess-s-CHI/results/chart2code_gpt_DirectAgent_results/direct/line_7.py"
+
+    evaluator(generation_code_file, golden_code_file)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/mp_util.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/mp_util.py
new file mode 100644
index 0000000000000000000000000000000000000000..30612f5a9644353efa51132edcf1bad70380a985
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/chartmimic/mp_util.py
@@ -0,0 +1,78 @@
+import os
+import traceback
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from typing import Callable, Iterable
+
+from tqdm import tqdm
+
+from vlmeval.smp import dump, get_logger, load
+
+logger = get_logger(__name__)
+
+
+def track_progress_rich_new(
+    func: Callable,
+    tasks: Iterable = tuple(),
+    nproc: int = 1,
+    save=None,
+    keys=None,
+    **kwargs
+) -> list:
+    """
+    Parallel execution with progress tracking and safe interim saving.
+    """
+    # Prepare persistent storage
+    if save:
+        os.makedirs(os.path.dirname(save), exist_ok=True)
+        if not os.path.exists(save):
+            dump({}, save)
+        res = load(save)
+    else:
+        res = {}
+
+    results = [None] * len(tasks)
+    future_to_idx = {}
+
+    # Use process pool to bypass GIL for CPU-bound tasks
+    with ProcessPoolExecutor(max_workers=nproc) as executor:
+        for idx, inp in enumerate(tasks):
+            # Support dict, tuple/list, or single-value tasks
+            if isinstance(inp, dict):
+                future = executor.submit(func, **inp)
+            elif isinstance(inp, (list, tuple)):
+                future = executor.submit(func, *inp)
+            else:
+                future = executor.submit(func, inp)
+            future_to_idx[future] = idx
+
+        # Display progress bar as tasks complete
+        with tqdm(total=len(tasks)) as pbar:
+            for future in as_completed(future_to_idx):
+                idx = future_to_idx[future]
+                key = keys[idx] if keys else None
+                try:
+                    result = future.result()
+                except Exception as e:
+                    exc_type = type(e).__name__
+                    err_msg = f"[{exc_type}] Exception in task {key or idx}: {str(e)}"
+                    logger.error(err_msg)
+                    logger.error("Full traceback:")
+                    logger.error(traceback.format_exc())
+
+                    # Optional: attach traceback to result for downstream
+                    # reference
+                    result = getattr(e, 'result', (-1, {
+                        'msg': err_msg,
+                        'traceback': traceback.format_exc(),
+                    }))
+
+                results[idx] = result
+                # Update persistent results
+                if keys and key is not None:
+                    res[key] = result
+                if save:
+                    dump(res, save)  # save after each task
+
+                pbar.update(1)
+
+    return results
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/dedup_post_gen.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/dedup_post_gen.py
new file mode 100644
index 0000000000000000000000000000000000000000..31906cd999d4cbd52a17b92f03b85d3cdebad78d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/dedup_post_gen.py
@@ -0,0 +1,78 @@
+import difflib
+import os
+import re
+
+
+def map_positions(clean_text, original_text):
+    """Map positions from clean text back to original text."""
+    map_clean_to_original = []
+    original_idx = 0
+
+    for clean_char in clean_text:
+        while original_text[original_idx] != clean_char:
+            original_idx += 1
+        map_clean_to_original.append(original_idx)
+        original_idx += 1
+
+    return map_clean_to_original
+
+
+def check_repetitive_content(
+    file_path,
+    chunk_size=100,
+    repetition_threshold=5,
+    similarity_threshold=0.8,
+    debug=False,
+):
+    """
+    Check repetitive content in a text file, ignoring HTML tags.
+
+    It compares fixed-size chunks in cleaned text and maps the first repetitive
+    position back to the original HTML string for optional truncation.
+    """
+    with open(file_path, "r", encoding="utf-8") as file:
+        content = file.read()
+
+    content_no_html = re.sub("<.*?>", "", content)
+    position_map = map_positions(content_no_html, content)
+
+    chunks = [
+        content_no_html[i:i + chunk_size]
+        for i in range(0, len(content_no_html), chunk_size)
+    ]
+
+    seen = {}
+    repetitive_start = len(content_no_html)
+    for i, chunk in enumerate(chunks):
+        for seen_chunk, indexes in seen.items():
+            similarity = difflib.SequenceMatcher(None, chunk, seen_chunk).ratio()
+            if similarity >= similarity_threshold:
+                indexes.append(i)
+                if len(indexes) >= repetition_threshold:
+                    clean_start = min(repetitive_start, indexes[0] * chunk_size)
+                    c_repetitive_start = (
+                        position_map[clean_start]
+                        if clean_start < len(position_map)
+                        else len(content)
+                    )
+                    if c_repetitive_start < repetitive_start:
+                        repetitive_start = c_repetitive_start
+                break
+        else:
+            seen[chunk] = [i]
+
+    repetitive = repetitive_start != len(content_no_html)
+    start_position = repetitive_start
+
+    if repetitive:
+        print(f"[Warning] Repetitive content found in {file_path}, start at {start_position}")
+        print(
+            "[Warning] You might want to manually check whether the automatic repetition removal is correct."
+        )
+        if not debug:
+            os.rename(file_path, file_path.replace(".html", "_old.txt"))
+            with open(file_path, "w", encoding="utf-8") as file:
+                file.write(content[:start_position])
+        else:
+            with open(file_path.replace(".html", "_new.html"), "w", encoding="utf-8") as file:
+                file.write(content[:start_position])
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/ocr_free_utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/ocr_free_utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..e07595145fb7a54c312f9ee6718988e96a874470
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/ocr_free_utils.py
@@ -0,0 +1,291 @@
+import os
+from pathlib import Path
+
+import cv2
+import numpy as np
+from bs4 import BeautifulSoup, Comment, NavigableString, Tag
+from PIL import Image, ImageColor
+
+
+def rgb_to_hex(rgb):
+    """Convert an RGB tuple to hexadecimal format."""
+    return "{:02X}{:02X}{:02X}".format(*rgb)
+
+
+class ColorPool:
+    def __init__(self, offset=0):
+        color_values = list(range(10, 251, 16))
+        color_list = [
+            (
+                (r + offset) % 256,
+                (g + offset) % 256,
+                (b + offset) % 256,
+            )
+            for r in color_values
+            for g in color_values
+            for b in color_values
+        ]
+        self.color_pool = [rgb_to_hex(color) for color in color_list]
+
+    def pop_color(self):
+        if self.color_pool:
+            return self.color_pool.pop()
+        raise NotImplementedError
+
+
+def process_html(input_file_path, output_file_path, offset=0):
+    with open(input_file_path, "r") as file:
+        soup = BeautifulSoup(file, "html.parser")
+
+    def update_style(element, property_name, value):
+        important_value = f"{value} !important"
+        styles = element.attrs.get("style", "").split(";")
+        updated_styles = [
+            s
+            for s in styles
+            if not s.strip().startswith(property_name) and len(s.strip()) > 0
+        ]
+        updated_styles.append(f"{property_name}: {important_value}")
+        element["style"] = "; ".join(updated_styles).strip()
+
+    for element in soup.find_all(True):
+        update_style(element, "background-color", "rgba(255, 255, 255, 0.0)")
+
+    color_pool = ColorPool(offset)
+    text_tags = [
+        "p",
+        "h1",
+        "h2",
+        "h3",
+        "h4",
+        "h5",
+        "h6",
+        "div",
+        "span",
+        "a",
+        "b",
+        "li",
+        "table",
+        "td",
+        "th",
+        "button",
+        "footer",
+        "header",
+        "figcaption",
+    ]
+    for tag in soup.find_all(text_tags):
+        color = f"#{color_pool.pop_color()}"
+        update_style(tag, "color", color)
+        update_style(tag, "opacity", 1.0)
+
+    with open(output_file_path, "w") as file:
+        file.write(str(soup))
+
+
+def similar(n1, n2):
+    return abs(n1 - n2) <= 8
+
+
+def find_different_pixels(image1_path, image2_path):
+    img1 = Image.open(image1_path)
+    img2 = Image.open(image2_path)
+
+    if img1.size != img2.size:
+        print(f"[Warning] Images are not the same size, {image1_path}, {image2_path}")
+        return None
+
+    img1 = img1.convert("RGB")
+    img2 = img2.convert("RGB")
+
+    pixels1 = img1.load()
+    pixels2 = img2.load()
+    different_pixels = []
+
+    for x in range(img1.size[0]):
+        for y in range(img1.size[1]):
+            r1, g1, b1 = pixels1[x, y]
+            r2, g2, b2 = pixels2[x, y]
+            if (
+                similar((r1 + 50) % 256, r2)
+                and similar((g1 + 50) % 256, g2)
+                and similar((b1 + 50) % 256, b2)
+            ):
+                different_pixels.append((y, x))
+
+    if len(different_pixels) > 0:
+        return np.stack(different_pixels)
+    return None
+
+
+def extract_text_with_color(html_file):
+    def get_color(tag):
+        if "style" in tag.attrs:
+            styles = tag["style"].split(";")
+            color_style = [s for s in styles if "color" in s and "background-color" not in s]
+            if color_style:
+                color = color_style[-1].split(":")[1].strip().replace(" !important", "")
+                if color[0] == "#":
+                    return color
+                try:
+                    if color.startswith("rgb"):
+                        color = tuple(map(int, color[4:-1].split(",")))
+                    else:
+                        color = ImageColor.getrgb(color)
+                    return "#{:02x}{:02x}{:02x}".format(*color)
+                except ValueError:
+                    print(f"Warning: unable to identify or convert color in {html_file}...", color)
+                    return None
+        return None
+
+    def extract_text_recursive(element, parent_color="#000000"):
+        if isinstance(element, Comment):
+            return None
+        if isinstance(element, NavigableString):
+            text = element.strip()
+            return (text, parent_color) if text else None
+        if isinstance(element, Tag):
+            current_color = get_color(element) or parent_color
+            children_texts = filter(
+                None,
+                [extract_text_recursive(child, current_color) for child in element.children],
+            )
+            return list(children_texts)
+        return None
+
+    with open(html_file, "r", encoding="utf-8") as file:
+        soup = BeautifulSoup(file, "html.parser")
+        body = soup.body
+        return extract_text_recursive(body) if body else []
+
+
+def flatten_tree(tree):
+    flat_list = []
+
+    def flatten(node):
+        if isinstance(node, list):
+            for item in node:
+                flatten(item)
+        else:
+            flat_list.append(node)
+
+    flatten(tree)
+    return flat_list
+
+
+def average_color(image_path, coordinates):
+    """
+    Calculate the average color of the specified coordinates in the image.
+
+    :param coordinates: A 2D numpy array with rows in [x, y] format.
+    :return: A tuple representing the average color (R, G, B).
+    """
+    image_array = np.array(Image.open(image_path).convert("RGB"))
+    colors = [image_array[x, y] for x, y in coordinates]
+    avg_color = np.mean(colors, axis=0)
+    return tuple(avg_color.astype(int))
+
+
+def robust_cv2_imread(img_name):
+    image = Image.open(img_name)
+    # Convert Image to numpy array
+    # It's not the most efficient way, but it works. *(link¹)
+    image = np.asarray(image)
+    # Remove alpha channel if existent
+    if len(image.shape) == 3 and image.shape[2] == 4:
+        image = image[:, :, : 3]
+    # Restore RGB colors
+    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
+    return image
+
+
+def get_blocks_from_image_diff_pixels(image_path, html_text_color_tree, different_pixels):
+    image = robust_cv2_imread(image_path)
+    x_w = image.shape[0]
+    y_w = image.shape[1]
+
+    def hex_to_bgr(hex_color):
+        """Convert a hex color string to a BGR color tuple."""
+        hex_color = hex_color.lstrip("#")
+        rgb = tuple(int(hex_color[i:i + 2], 16) for i in (0, 2, 4))
+        return rgb[::-1]
+
+    def get_intersect(arr1, arr2):
+        arr1_reshaped = arr1.view([("", arr1.dtype)] * arr1.shape[1])
+        arr2_reshaped = arr2.view([("", arr2.dtype)] * arr2.shape[1])
+        common_rows = np.intersect1d(arr1_reshaped, arr2_reshaped)
+        return common_rows.view(arr1.dtype).reshape(-1, arr1.shape[1])
+
+    blocks = []
+    for item in html_text_color_tree:
+        try:
+            color = np.array(hex_to_bgr(item[1]), dtype="uint8")
+        except Exception:
+            continue
+
+        lower = color - 4
+        upper = color + 4
+        mask = cv2.inRange(image, lower, upper)
+        coords = np.column_stack(np.where(mask > 0))
+        coords = get_intersect(coords, different_pixels)
+
+        if coords.size == 0:
+            continue
+
+        x_min, y_min = np.min(coords, axis=0)
+        x_max, y_max = np.max(coords, axis=0)
+        color = average_color(image_path.replace("_p.png", ".png"), coords)
+
+        blocks.append(
+            {
+                "text": item[0].lower(),
+                "bbox": (
+                    y_min / y_w,
+                    x_min / x_w,
+                    (y_max - y_min + 1) / y_w,
+                    (x_max - x_min + 1) / x_w,
+                ),
+                "color": color,
+            }
+        )
+    return blocks
+
+
+def get_itermediate_names(name):
+    return (
+        name.replace(".png", ".html"),
+        name.replace(".png", "_p.html"),
+        name.replace(".png", "_p_1.html"),
+        name.replace(".png", "_p.png"),
+        name.replace(".png", "_p_1.png"),
+    )
+
+
+def get_blocks_ocr_free(image_path):
+    html, p_html, p_html_1, p_png, p_png_1 = get_itermediate_names(image_path)
+    process_html(html, p_html)
+    process_html(html, p_html_1, offset=50)
+
+    os.system(f"python3 {Path(__file__).parent}/screenshot_single.py --html {p_html} --png {p_png}")
+    os.system(
+        f"python3 {Path(__file__).parent}/screenshot_single.py --html {p_html_1} --png {p_png_1}"
+    )
+
+    different_pixels = find_different_pixels(p_png, p_png_1)
+
+    if different_pixels is None:
+        print(f"[Warning] Unable to get pixels with different colors from {p_png}, {p_png_1}...")
+        os.system(f"rm {p_html} {p_png} {p_html_1} {p_png_1}")
+        return []
+
+    html_text_color_tree = flatten_tree(extract_text_with_color(p_html))
+    try:
+        blocks = get_blocks_from_image_diff_pixels(
+            p_png, html_text_color_tree, different_pixels
+        )
+    except Exception:
+        print(f"[Warning] Unable to get blocks from {p_png}...")
+        os.system(f"rm {p_html} {p_png} {p_html_1} {p_png_1}")
+        return []
+
+    os.system(f"rm {p_html} {p_png} {p_html_1} {p_png_1}")
+    return blocks
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/screenshot_single.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/screenshot_single.py
new file mode 100644
index 0000000000000000000000000000000000000000..c8b210e2672d2159562e34ea4c7d6ea311daced2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/screenshot_single.py
@@ -0,0 +1,73 @@
+import os
+
+try:
+    from playwright.sync_api import sync_playwright
+except ImportError:
+    error_msg = """
+playwright not installed. Please install it with the following commands:
+pip install playwright
+playwright install-deps
+playwright install --no-shell chromium-headless-shell
+"""
+    print(error_msg)
+    exit(1)
+import argparse
+
+from PIL import Image
+
+
+def take_screenshot(url, output_file="screenshot.png", do_it_again=False):
+    # Convert local path to file:// URL if it's a file
+    if os.path.exists(url):
+        url = "file://" + os.path.abspath(url)
+
+    if os.path.exists(output_file) and not do_it_again:
+        print(f"{output_file} exists!")
+        return
+
+    try:
+        with sync_playwright() as p:
+            # Choose a browser, e.g., Chromium, Firefox, or WebKit
+            browser = p.chromium.launch(
+                headless=True,
+                args=[
+                    "--no-sandbox",
+                    "--disable-dev-shm-usage",
+                    "--disable-web-security",
+                    "--disable-features=VizDisplayCompositor",
+                    "--disable-gpu",
+                    "--no-first-run",
+                    "--disable-background-timer-throttling",
+                    "--disable-renderer-backgrounding",
+                    "--disable-backgrounding-occluded-windows",
+                ],
+            )
+            page = browser.new_page()
+
+            # Navigate to the URL
+            page.goto(url, timeout=60000)
+
+            # Take the screenshot
+            page.screenshot(path=output_file, full_page=True, animations="disabled", timeout=60000)
+
+            browser.close()
+    except Exception as e:
+        print(f"Failed to take screenshot due to: {e}. Generating a blank image.")
+        # Generate a blank image
+        img = Image.new('RGB', (1280, 960), color='white')
+        img.save(output_file)
+
+
+if __name__ == "__main__":
+
+    # Initialize the parser
+    parser = argparse.ArgumentParser(description='Process two path strings.')
+
+    # Define the arguments
+    parser.add_argument('--html', type=str)
+    parser.add_argument('--png', type=str)
+
+    # Parse the arguments
+    args = parser.parse_args()
+
+    take_screenshot(args.html, args.png, do_it_again=True)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/visual_score.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/visual_score.py
new file mode 100644
index 0000000000000000000000000000000000000000..a301b08a080a77fd99367546f8cf7e795b52bf21
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/design2code/visual_score.py
@@ -0,0 +1,619 @@
+import os
+import random
+import re
+from collections import Counter
+from copy import deepcopy
+from difflib import SequenceMatcher
+
+import cv2
+import numpy as np
+import torch
+from bs4 import BeautifulSoup, Comment, NavigableString
+from colormath.color_conversions import convert_color
+from colormath.color_diff import delta_e_cie2000
+from colormath.color_objects import LabColor, sRGBColor
+from PIL import Image
+from scipy.optimize import linear_sum_assignment
+
+from .dedup_post_gen import check_repetitive_content
+from .ocr_free_utils import get_blocks_ocr_free
+
+
+def patch_asscalar(a):
+    return a.item()
+
+
+setattr(np, "asscalar", patch_asscalar)
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+_CLIP_MODEL = None
+_CLIP_PREPROCESS = None
+
+
+def get_clip_model():
+    global _CLIP_MODEL, _CLIP_PREPROCESS
+    if _CLIP_MODEL is None or _CLIP_PREPROCESS is None:
+        import clip
+        _CLIP_MODEL, _CLIP_PREPROCESS = clip.load("ViT-B/32", device=device)
+    return _CLIP_MODEL, _CLIP_PREPROCESS
+
+
+def calculate_similarity(block1, block2, max_distance=1.42):
+    del max_distance
+    text_similarity = SequenceMatcher(None, block1["text"], block2["text"]).ratio()
+    return text_similarity
+
+
+def adjust_cost_for_context(cost_matrix, consecutive_bonus=1.0, window_size=20):
+    if window_size <= 0:
+        return cost_matrix
+
+    n, m = cost_matrix.shape
+    adjusted_cost_matrix = np.copy(cost_matrix)
+
+    for i in range(n):
+        for j in range(m):
+            if adjusted_cost_matrix[i][j] >= -0.5:
+                continue
+            nearby_matrix = cost_matrix[
+                max(0, i - window_size): min(n, i + window_size + 1),
+                max(0, j - window_size): min(m, j + window_size + 1),
+            ]
+            flattened_array = nearby_matrix.flatten()
+            sorted_array = np.sort(flattened_array)[::-1]
+            sorted_array = np.delete(
+                sorted_array, np.where(sorted_array == cost_matrix[i, j])[0][0]
+            )
+            top_k_elements = sorted_array[-window_size * 2:]
+            sum_top_k = np.sum(top_k_elements)
+            bonus = consecutive_bonus * sum_top_k
+            adjusted_cost_matrix[i][j] += bonus
+    return adjusted_cost_matrix
+
+
+def create_cost_matrix(blocks_a, blocks_b):
+    n = len(blocks_a)
+    m = len(blocks_b)
+    cost_matrix = np.zeros((n, m))
+    for i in range(n):
+        for j in range(m):
+            cost_matrix[i, j] = -calculate_similarity(blocks_a[i], blocks_b[j])
+    return cost_matrix
+
+
+def draw_matched_bboxes(img1, img2, matched_bboxes):
+    img1_drawn = img1.copy()
+    img2_drawn = img2.copy()
+
+    h1, w1, _ = img1.shape
+    h2, w2, _ = img2.shape
+
+    for bbox_pair in matched_bboxes:
+        color = (
+            random.randint(0, 255),
+            random.randint(0, 255),
+            random.randint(0, 255),
+        )
+
+        bbox1 = [
+            int(bbox_pair[0][0] * w1),
+            int(bbox_pair[0][1] * h1),
+            int(bbox_pair[0][2] * w1),
+            int(bbox_pair[0][3] * h1),
+        ]
+        bbox2 = [
+            int(bbox_pair[1][0] * w2),
+            int(bbox_pair[1][1] * h2),
+            int(bbox_pair[1][2] * w2),
+            int(bbox_pair[1][3] * h2),
+        ]
+
+        top_left_1 = (bbox1[0], bbox1[1])
+        bottom_right_1 = (bbox1[0] + bbox1[2], bbox1[1] + bbox1[3])
+        img1_drawn = cv2.rectangle(img1_drawn, top_left_1, bottom_right_1, color, 2)
+
+        top_left_2 = (bbox2[0], bbox2[1])
+        bottom_right_2 = (bbox2[0] + bbox2[2], bbox2[1] + bbox2[3])
+        img2_drawn = cv2.rectangle(img2_drawn, top_left_2, bottom_right_2, color, 2)
+
+    return img1_drawn, img2_drawn
+
+
+def calculate_distance_max_1d(x1, y1, x2, y2):
+    return max(abs(x2 - x1), abs(y2 - y1))
+
+
+def calculate_ratio(h1, h2):
+    return max(h1, h2) / min(h1, h2)
+
+
+def rgb_to_lab(rgb):
+    """Convert an RGB color to Lab color space."""
+    rgb_color = sRGBColor(rgb[0], rgb[1], rgb[2], is_upscaled=True)
+    return convert_color(rgb_color, LabColor)
+
+
+def color_similarity_ciede2000(rgb1, rgb2):
+    """
+    Calculate color similarity using CIEDE2000.
+
+    Returns a score in [0, 1], where 1 means identical.
+    """
+    lab1 = rgb_to_lab(rgb1)
+    lab2 = rgb_to_lab(rgb2)
+    delta_e = delta_e_cie2000(lab1, lab2)
+    return max(0, 1 - (delta_e / 100))
+
+
+def merge_blocks_wo_check(block1, block2):
+    merged_text = block1["text"] + " " + block2["text"]
+
+    x_min = min(block1["bbox"][0], block2["bbox"][0])
+    y_min = min(block1["bbox"][1], block2["bbox"][1])
+    x_max = max(
+        block1["bbox"][0] + block1["bbox"][2],
+        block2["bbox"][0] + block2["bbox"][2],
+    )
+    y_max = max(
+        block1["bbox"][1] + block1["bbox"][3],
+        block2["bbox"][1] + block2["bbox"][3],
+    )
+    merged_bbox = (x_min, y_min, x_max - x_min, y_max - y_min)
+
+    merged_color = tuple(
+        (color1 + color2) // 2 for color1, color2 in zip(block1["color"], block2["color"])
+    )
+
+    return {"text": merged_text, "bbox": merged_bbox, "color": merged_color}
+
+
+def calculate_current_cost(cost_matrix, row_ind, col_ind):
+    return cost_matrix[row_ind, col_ind].tolist()
+
+
+def find_maximum_matching(blocks_a, blocks_b, consecutive_bonus, window_size):
+    cost_matrix = create_cost_matrix(blocks_a, blocks_b)
+    cost_matrix = adjust_cost_for_context(cost_matrix, consecutive_bonus, window_size)
+    row_ind, col_ind = linear_sum_assignment(cost_matrix)
+    current_cost = calculate_current_cost(cost_matrix, row_ind, col_ind)
+    return list(zip(row_ind, col_ind)), current_cost, cost_matrix
+
+
+def remove_indices(lst, indices):
+    for index in sorted(indices, reverse=True):
+        if index < len(lst):
+            lst.pop(index)
+    return lst
+
+
+def merge_blocks_by_list(blocks, merge_list):
+    pop_list = []
+    while True:
+        if len(merge_list) == 0:
+            remove_indices(blocks, pop_list)
+            return blocks
+
+        i = merge_list[0][0]
+        j = merge_list[0][1]
+
+        blocks[i] = merge_blocks_wo_check(blocks[i], blocks[j])
+        pop_list.append(j)
+
+        merge_list.pop(0)
+        if len(merge_list) > 0:
+            new_merge_list = []
+            for k in range(len(merge_list)):
+                if (
+                    merge_list[k][0] != i
+                    and merge_list[k][1] != i
+                    and merge_list[k][0] != j
+                    and merge_list[k][1] != j
+                ):
+                    new_merge_list.append(merge_list[k])
+            merge_list = new_merge_list
+
+
+def print_matching(matching, blocks1, blocks2, cost_matrix):
+    for i, j in matching:
+        print(f"{blocks1[i]} matched with {blocks2[j]}, cost {cost_matrix[i][j]}")
+
+
+def difference_of_means(list1, list2):
+    counter1 = Counter(list1)
+    counter2 = Counter(list2)
+
+    for element in set(list1) & set(list2):
+        common_count = min(counter1[element], counter2[element])
+        counter1[element] -= common_count
+        counter2[element] -= common_count
+
+    unique_list1 = [item for item in counter1.elements()]
+    unique_list2 = [item for item in counter2.elements()]
+
+    mean_list1 = sum(unique_list1) / len(unique_list1) if unique_list1 else 0
+    mean_list2 = sum(unique_list2) / len(unique_list2) if unique_list2 else 0
+
+    if mean_list1 - mean_list2 > 0:
+        if min(unique_list1) > min(unique_list2):
+            return mean_list1 - mean_list2
+        return 0.0
+    return mean_list1 - mean_list2
+
+
+def find_possible_merge(blocks_a, blocks_b, consecutive_bonus, window_size, debug=False):
+    merge_bonus = 0.0
+    merge_windows = 1
+
+    def sort_fn(value):
+        return value[2]
+
+    while True:
+        a_changed = False
+        b_changed = False
+
+        matching, current_cost, cost_matrix = find_maximum_matching(
+            blocks_a, blocks_b, merge_bonus, merge_windows
+        )
+        if debug:
+            print("Current cost of the solution:", current_cost)
+            print_matching(matching, blocks_a, blocks_b, cost_matrix)
+
+        if len(blocks_a) >= 2:
+            merge_list = []
+            for i in range(len(blocks_a) - 1):
+                new_a = deepcopy(blocks_a)
+                new_a[i] = merge_blocks_wo_check(new_a[i], new_a[i + 1])
+                new_a.pop(i + 1)
+
+                _, updated_cost, _ = find_maximum_matching(
+                    new_a, blocks_b, merge_bonus, merge_windows
+                )
+                diff = difference_of_means(current_cost, updated_cost)
+                if diff > 0.05:
+                    merge_list.append([i, i + 1, diff])
+                    if debug:
+                        print(new_a[i]["text"], diff)
+
+            merge_list.sort(key=sort_fn, reverse=True)
+            if len(merge_list) > 0:
+                a_changed = True
+                blocks_a = merge_blocks_by_list(blocks_a, merge_list)
+                _, current_cost, _ = find_maximum_matching(
+                    blocks_a, blocks_b, merge_bonus, merge_windows
+                )
+                if debug:
+                    print("Cost after optimization A:", current_cost)
+
+        if len(blocks_b) >= 2:
+            merge_list = []
+            for i in range(len(blocks_b) - 1):
+                new_b = deepcopy(blocks_b)
+                new_b[i] = merge_blocks_wo_check(new_b[i], new_b[i + 1])
+                new_b.pop(i + 1)
+
+                _, updated_cost, _ = find_maximum_matching(
+                    blocks_a, new_b, merge_bonus, merge_windows
+                )
+                diff = difference_of_means(current_cost, updated_cost)
+                if diff > 0.05:
+                    merge_list.append([i, i + 1, diff])
+                    if debug:
+                        print(new_b[i]["text"], diff)
+
+            merge_list.sort(key=sort_fn, reverse=True)
+            if len(merge_list) > 0:
+                b_changed = True
+                blocks_b = merge_blocks_by_list(blocks_b, merge_list)
+                _, current_cost, _ = find_maximum_matching(
+                    blocks_a, blocks_b, merge_bonus, merge_windows
+                )
+                if debug:
+                    print("Cost after optimization B:", current_cost)
+
+        if not a_changed and not b_changed:
+            break
+
+    matching, _, _ = find_maximum_matching(
+        blocks_a, blocks_b, consecutive_bonus, window_size
+    )
+    return blocks_a, blocks_b, matching
+
+
+def merge_blocks_by_bbox(blocks):
+    merged_blocks = {}
+    for block in blocks:
+        bbox = tuple(block["bbox"])
+        if bbox in merged_blocks:
+            existing_block = merged_blocks[bbox]
+            existing_block["text"] += " " + block["text"]
+            existing_block["color"] = [
+                (ec + c) / 2 for ec, c in zip(existing_block["color"], block["color"])
+            ]
+        else:
+            merged_blocks[bbox] = block
+    return list(merged_blocks.values())
+
+
+def mask_bounding_boxes_with_inpainting(image, bounding_boxes):
+    image_cv = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
+    mask = np.zeros(image_cv.shape[:2], dtype=np.uint8)
+    height, width = image_cv.shape[:2]
+
+    for bbox in bounding_boxes:
+        x_ratio, y_ratio, w_ratio, h_ratio = bbox
+        x = int(x_ratio * width)
+        y = int(y_ratio * height)
+        w = int(w_ratio * width)
+        h = int(h_ratio * height)
+        mask[y:y + h, x:x + w] = 255
+
+    inpainted_image = cv2.inpaint(image_cv, mask, 3, cv2.INPAINT_TELEA)
+    return Image.fromarray(cv2.cvtColor(inpainted_image, cv2.COLOR_BGR2RGB))
+
+
+def rescale_and_mask(image_path, blocks):
+    with Image.open(image_path) as img:
+        if len(blocks) > 0:
+            img = mask_bounding_boxes_with_inpainting(img, blocks)
+
+        width, height = img.size
+        if width < height:
+            new_size = (width, width)
+        else:
+            new_size = (height, height)
+
+        img_resized = img.resize(new_size, Image.LANCZOS)
+        return img_resized
+
+
+def calculate_clip_similarity_with_blocks(image_path1, image_path2, blocks1, blocks2):
+    model, preprocess = get_clip_model()
+    image1 = preprocess(
+        rescale_and_mask(image_path1, [block["bbox"] for block in blocks1])
+    ).unsqueeze(0).to(device)
+    image2 = preprocess(
+        rescale_and_mask(image_path2, [block["bbox"] for block in blocks2])
+    ).unsqueeze(0).to(device)
+
+    with torch.no_grad():
+        image_features1 = model.encode_image(image1)
+        image_features2 = model.encode_image(image2)
+
+    image_features1 /= image_features1.norm(dim=-1, keepdim=True)
+    image_features2 /= image_features2.norm(dim=-1, keepdim=True)
+    similarity = (image_features1 @ image_features2.T).item()
+    return similarity
+
+
+def truncate_repeated_html_elements(soup, max_count=50):
+    content_counts = {}
+
+    for element in soup.find_all(True):
+        if isinstance(element, (NavigableString, Comment)):
+            continue
+
+        try:
+            element_html = str(element)
+        except Exception:
+            element.decompose()
+            continue
+        content_counts[element_html] = content_counts.get(element_html, 0) + 1
+
+        if content_counts[element_html] > max_count:
+            element.decompose()
+
+    return str(soup)
+
+
+def make_html(filename):
+    with open(filename, "r") as file:
+        content = file.read()
+
+    if not re.search(r"<html[^>]*>", content, re.IGNORECASE):
+        new_content = f"<html><body><p>{content}</p></body></html>"
+        with open(filename, "w") as file:
+            file.write(new_content)
+
+
+def pre_process(html_file):
+    check_repetitive_content(html_file)
+    make_html(html_file)
+    with open(html_file, "r") as file:
+        soup = BeautifulSoup(file, "html.parser")
+    soup_str = truncate_repeated_html_elements(soup)
+    with open(html_file, "w") as file:
+        file.write(soup_str)
+
+
+def visual_eval_v3_multi(input_list, debug=False):
+    current_dir = os.path.abspath(os.path.dirname(__file__))
+    predict_html_list, original_html = input_list[0], input_list[1]
+    predict_img_list = [html.replace(".html", ".png") for html in predict_html_list]
+
+    predict_blocks_list = []
+    for predict_html in predict_html_list:
+        predict_img = predict_html.replace(".html", ".png")
+        pre_process(predict_html)
+        os.system(f"python3 {current_dir}/screenshot_single.py --html {predict_html} --png {predict_img}")
+        predict_blocks = get_blocks_ocr_free(predict_img)
+        predict_blocks_list.append(predict_blocks)
+
+    original_img = original_html.replace(".html", ".png")
+    os.system(f"python3 {current_dir}/screenshot_single.py --html {original_html} --png {original_img}")
+    original_blocks = get_blocks_ocr_free(original_img)
+    original_blocks = merge_blocks_by_bbox(original_blocks)
+
+    consecutive_bonus, window_size = 0.1, 1
+    return_score_list = []
+
+    for k, predict_blocks in enumerate(predict_blocks_list):
+        if len(predict_blocks) == 0:
+            print("[Warning] No detected blocks in: ", predict_img_list[k])
+            final_clip_score = calculate_clip_similarity_with_blocks(
+                predict_img_list[k],
+                original_img,
+                predict_blocks,
+                original_blocks,
+            )
+            return_score_list.append(
+                [0.0, 0.2 * final_clip_score, (0.0, 0.0, 0.0, 0.0, final_clip_score)]
+            )
+            continue
+        if len(original_blocks) == 0:
+            print("[Warning] No detected blocks in: ", original_img)
+            final_clip_score = calculate_clip_similarity_with_blocks(
+                predict_img_list[k],
+                original_img,
+                predict_blocks,
+                original_blocks,
+            )
+            return_score_list.append(
+                [0.0, 0.2 * final_clip_score, (0.0, 0.0, 0.0, 0.0, final_clip_score)]
+            )
+            continue
+
+        if debug:
+            print(predict_blocks)
+            print(original_blocks)
+
+        predict_blocks = merge_blocks_by_bbox(predict_blocks)
+        predict_blocks_m, original_blocks_m, matching = find_possible_merge(
+            predict_blocks,
+            deepcopy(original_blocks),
+            consecutive_bonus,
+            window_size,
+            debug=debug,
+        )
+
+        filtered_matching = []
+        for i, j in matching:
+            text_similarity = SequenceMatcher(
+                None, predict_blocks_m[i]["text"], original_blocks_m[j]["text"]
+            ).ratio()
+            if text_similarity < 0.5:
+                continue
+            filtered_matching.append([i, j, text_similarity])
+        matching = filtered_matching
+
+        indices1 = [item[0] for item in matching]
+        indices2 = [item[1] for item in matching]
+
+        matched_list = []
+        sum_areas = []
+        matched_areas = []
+        matched_text_scores = []
+        position_scores = []
+        text_color_scores = []
+
+        unmatched_area_1 = 0.0
+        for i in range(len(predict_blocks_m)):
+            if i not in indices1:
+                unmatched_area_1 += predict_blocks_m[i]["bbox"][2] * predict_blocks_m[i]["bbox"][3]
+        unmatched_area_2 = 0.0
+        for j in range(len(original_blocks_m)):
+            if j not in indices2:
+                unmatched_area_2 += original_blocks_m[j]["bbox"][2] * original_blocks_m[j]["bbox"][3]
+        sum_areas.append(unmatched_area_1 + unmatched_area_2)
+
+        for i, j, text_similarity in matching:
+            sum_block_area = (
+                predict_blocks_m[i]["bbox"][2] * predict_blocks_m[i]["bbox"][3]
+                + original_blocks_m[j]["bbox"][2] * original_blocks_m[j]["bbox"][3]
+            )
+            position_similarity = 1 - calculate_distance_max_1d(
+                predict_blocks_m[i]["bbox"][0] + predict_blocks_m[i]["bbox"][2] / 2,
+                predict_blocks_m[i]["bbox"][1] + predict_blocks_m[i]["bbox"][3] / 2,
+                original_blocks_m[j]["bbox"][0] + original_blocks_m[j]["bbox"][2] / 2,
+                original_blocks_m[j]["bbox"][1] + original_blocks_m[j]["bbox"][3] / 2,
+            )
+            text_color_similarity = color_similarity_ciede2000(
+                predict_blocks_m[i]["color"],
+                original_blocks_m[j]["color"],
+            )
+            matched_list.append([predict_blocks_m[i]["bbox"], original_blocks_m[j]["bbox"]])
+
+            if min(
+                predict_blocks_m[i]["bbox"][2],
+                original_blocks_m[j]["bbox"][2],
+                predict_blocks_m[i]["bbox"][3],
+                original_blocks_m[j]["bbox"][3],
+            ) == 0:
+                print(f"{predict_blocks_m[i]} matched with {original_blocks_m[j]}")
+            assert (
+                calculate_ratio(
+                    predict_blocks_m[i]["bbox"][2],
+                    original_blocks_m[j]["bbox"][2],
+                )
+                > 0
+                and calculate_ratio(
+                    predict_blocks_m[i]["bbox"][3],
+                    original_blocks_m[j]["bbox"][3],
+                )
+                > 0
+            ), f"{predict_blocks_m[i]} matched with {original_blocks_m[j]}"
+
+            sum_areas.append(sum_block_area)
+            matched_areas.append(sum_block_area)
+            matched_text_scores.append(text_similarity)
+            position_scores.append(position_similarity)
+            text_color_scores.append(text_color_similarity)
+
+            if debug:
+                print(f"{predict_blocks_m[i]} matched with {original_blocks_m[j]}")
+                print(
+                    SequenceMatcher(
+                        None,
+                        predict_blocks_m[i]["text"],
+                        original_blocks_m[j]["text"],
+                    ).ratio()
+                )
+                print("text similarity score", text_similarity)
+                print("position score", position_similarity)
+                print("color score", text_color_similarity)
+                print("----------------------------------")
+
+        if len(matched_areas) > 0:
+            sum_sum_areas = np.sum(sum_areas)
+            final_size_score = np.sum(matched_areas) / np.sum(sum_areas)
+            final_matched_text_score = np.mean(matched_text_scores)
+            final_position_score = np.mean(position_scores)
+            final_text_color_score = np.mean(text_color_scores)
+            final_clip_score = calculate_clip_similarity_with_blocks(
+                predict_img_list[k],
+                original_img,
+                predict_blocks,
+                original_blocks,
+            )
+            final_score = 0.2 * (
+                final_size_score
+                + final_matched_text_score
+                + final_position_score
+                + final_text_color_score
+                + final_clip_score
+            )
+            return_score_list.append(
+                [
+                    sum_sum_areas,
+                    final_score,
+                    (
+                        final_size_score,
+                        final_matched_text_score,
+                        final_position_score,
+                        final_text_color_score,
+                        final_clip_score,
+                    ),
+                ]
+            )
+        else:
+            print("[Warning] No matched blocks in: ", predict_img_list[k])
+            final_clip_score = calculate_clip_similarity_with_blocks(
+                predict_img_list[k],
+                original_img,
+                predict_blocks,
+                original_blocks,
+            )
+            return_score_list.append(
+                [0.0, 0.2 * final_clip_score, (0.0, 0.0, 0.0, 0.0, final_clip_score)]
+            )
+
+    return return_score_list
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/README.md b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d55840e98c4f156025486e647f97d81c3351b16a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/README.md
@@ -0,0 +1,51 @@
+# MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks [ICLR 2025]
+
+![image](https://github.com/user-attachments/assets/5fd44fa9-0ec2-4298-ad0c-e883cb1edf7f)
+
+MEGA-Bench contains 505 multimodal tasks with diverse data sources, input/output formats, and skill requirements. The taxonomy tree is derived from the application dimension, which guides and calibrates the annotation process. The benchmark is equiped with a suite of 45 evaluation metrics to handle various output formats beyond multiple-choice questions.
+
+Following this doc, the evaluation result contains the final scores and multi-dimensional breakdown, which has a consistent format as [MEGA-Bench Leaderboard](https://huggingface.co/spaces/TIGER-Lab/MEGA-Bench). Below is an example from evaluating `Qwen-2-VL-7B-Instruct` on the core set.
+
+
+## Step-1: Install requirements for MEGA-Bench metrics to obtain the evaluation scores and breakdown analysis
+
+```bash
+pip install -r vlmeval/dataset/utils/megabench/requirements.txt
+```
+
+
+## Step-2: Get the model response and evaluation score files with VLMEvalKit
+
+```bash
+# Core set (440 tasks, in 16-frame setting)
+python3 run.py \
+    --data MEGABench_core_16frame \
+    --model Qwen2-VL-7B-Instruct \
+    --work-dir your/work/dir \
+
+# Open-ended set (65 tasks, in 16-frame setting)
+python3 run.py \
+    --data MEGABench_open_16frame \
+    --model Qwen2-VL-7B-Instruct \
+    --work-dir your/work/dir \
+```
+Note: please set up the `OPENAI_API_KEY` in the .env file to evaluate the open set.
+
+Then you can have 2 score files in the directory like: 
+
+```bash
+your/work/dir/Qwen-2-VL-7B-Instruct/T20250706_Gbf63ab2c/megabench_score_core.json
+your/work/dir/Qwen-2-VL-7B-Instruct/T20250707_Gbf63ab2c/megabench_score_open.json
+```
+
+## Step-3(Optional): Run MEGA-Bench scripts to obtain the breakdown analysis
+
+Move the 2 score files into the same directory, then run the script:
+
+```bash
+# Run the metrics for the open-ended set
+cd vlmeval/dataset/utils/megabench/tools
+python3 derive_breakdown_results.py  --input_dir your/dir/to/megabench_scores
+```
+
+The results in `your/dir/to/megabench_scores/analysis` are what used by [MEGA-Bench leaderboard](https://huggingface.co/spaces/TIGER-Lab/MEGA-Bench). The leaderboard can be updated by putting the files in the results directory of the leadboard's [HuggingFace space](https://huggingface.co/spaces/TIGER-Lab/MEGA-Bench/tree/main/static/eval_results/Default).
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..5b136359db1d036614f0b5c63e26c1b4a4f63b6e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/__init__.py
@@ -0,0 +1,5 @@
+from .aggregation_type import AggregationType
+from .metric_type import MetricType
+from .response_parse_type import ResponseParseType
+
+__all__ = [AggregationType, MetricType, ResponseParseType]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/mean_agg.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/mean_agg.py
new file mode 100644
index 0000000000000000000000000000000000000000..8bffc7228f64642a3ca0d89f1fd0f604dd49bc53
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/mean_agg.py
@@ -0,0 +1,23 @@
+from numbers import Number
+from typing import Dict
+
+import numpy as np
+
+
+class MeanAggregation:
+    """Take the mean of all valid scores."""
+
+    @staticmethod
+    def aggregate(scores: Dict[str, Number], weights: Dict[str, Number]) -> Number:
+        """Exact match between targets and responses."""
+        filtered_scores = {f: s for f, s in scores.items() if s >= 0}
+        if not filtered_scores:
+            return -1
+
+        # Align the key order
+        flattened_scores = []
+        flattened_weights = []
+        for field in filtered_scores:
+            flattened_scores.append(filtered_scores[field])
+            flattened_weights.append(weights[field])
+        return np.average(flattened_scores, weights=flattened_weights)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/min_agg.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/min_agg.py
new file mode 100644
index 0000000000000000000000000000000000000000..1558991ffda6dfdd0d12663e8ef31950bc521ce7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/min_agg.py
@@ -0,0 +1,14 @@
+from numbers import Number
+from typing import Dict
+
+
+class MinAggregation:
+    """Take the minimum of all valid scores."""
+
+    @staticmethod
+    def aggregate(scores: Dict[str, Number], weights: Dict[str, Number]) -> Number:
+        """Exact match between targets and responses."""
+        filtered_scores = [s for s in scores.values() if s >= 0]
+        if not filtered_scores:
+            return -1
+        return min(filtered_scores)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/unsupported_agg.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/unsupported_agg.py
new file mode 100644
index 0000000000000000000000000000000000000000..1cd9991b67130a7be359a75c32d953ecda68a4b8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation/unsupported_agg.py
@@ -0,0 +1,8 @@
+from numbers import Number
+from typing import Dict
+
+
+class UnsupportedAggregation:
+    @staticmethod
+    def aggregate(scores: Dict[str, Number], weights: Dict[str, Number]) -> Number:
+        return -1
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation_type.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation_type.py
new file mode 100644
index 0000000000000000000000000000000000000000..15ee66d543cfd222794cdd83c9e12fee194ac3e3
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/aggregation_type.py
@@ -0,0 +1,26 @@
+from enum import Enum
+
+
+class AggregationType(Enum):
+    MEAN = 0
+
+    @classmethod
+    def from_string(cls, s):
+        return cls.MEAN
+
+    def aggregate(self, field_scores, field_weights):
+        if not field_scores:
+            return 0.0
+
+        total_score = 0.0
+        total_weight = 0.0
+
+        for field, score in field_scores.items():
+            weight = field_weights.get(field, 1.0)
+            try:
+                total_score += score * weight
+            except:
+                total_score += score[0] * weight
+            total_weight += weight
+
+        return total_score / total_weight if total_weight > 0 else 0.0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..977b1645520d4908fe789dca51c270aeb5c6c7a2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/evaluator.py
@@ -0,0 +1,398 @@
+import argparse
+import ast
+import json
+import os
+from typing import Any, Dict, List
+
+from vlmeval.smp import dump, load
+from . import AggregationType, MetricType, ResponseParseType
+from .parsing.common.utils import evaluate_as_string
+
+
+class MEGABenchEvaluator:
+    def __init__(
+        self,
+        subset_name: str,
+        responses_file: str,
+        output_file: str,
+    ):
+        """
+        :param hf_data_file: Path to a file containing HF dataset tasks + their metric configs
+        :param model_responses_file: Path to a JSON file with tasks + model responses
+        :param output_file: Path to store evaluated results
+        """
+        self.hf_data = self._load_hf(subset_name)  # e.g. same structure used previously
+        self.data = self._load_json(responses_file)  # The model's output
+        self.output_file = output_file
+        self.tmp_output_file = output_file.replace(".json", "_tmp.pkl")
+
+        # Build a dict of {task_name -> metric configuration} for quick lookup
+        self.scoring_functions = {}
+        for task_name, task_samples in self.hf_data.items():
+            self.scoring_functions[task_name] = ast.literal_eval(
+                task_samples[0]["metric_info"]
+            )
+
+    def _load_hf(self, subset_name: str) -> List[Dict[str, Any]]:
+        """
+        Load the HF dataset for the given subset name.
+        """
+        from datasets import load_dataset
+        dataset = load_dataset("TIGER-Lab/MEGA-Bench", subset_name)["test"]
+        task_dict = {}
+        for sample in dataset:
+            task_name = sample["task_name"]
+            if task_name not in task_dict:
+                task_dict[task_name] = []
+            task_dict[task_name].append(sample)
+
+        return task_dict
+
+    def _get_eval_context(self, task_name, query):
+        if "query_idx" in query:
+            query_idx = query["query_idx"]
+            eval_context = self.hf_data[task_name][query_idx]["eval_context"]
+        else:
+            global_idx = query["global_idx"]
+            global_idx_to_sample = {sample["id"]: sample for sample in self.hf_data[task_name]}
+            eval_context = global_idx_to_sample[global_idx]["eval_context"]
+
+        eval_context = ast.literal_eval(eval_context)
+        return eval_context
+    
+    def _determine_eval_style(self, task):
+        metric_info = self.scoring_functions[task["task_name"]]
+        all_task_metrics = list(metric_info["field_score_function"].values())
+        eval_type = (
+            "rule"
+            if (
+                "gpt_4o_as_judge" not in all_task_metrics
+                and "ascii_art_gpt4o_judge" not in all_task_metrics
+            )
+            else "llm"
+        )
+        return eval_type
+
+    def evaluate(self):
+        """
+        The main entry point to evaluate all tasks in self.data based on the HF dataset’s metric info.
+        """
+        if os.path.exists(self.tmp_output_file):
+            exist_records = load(self.tmp_output_file)
+        else:
+            exist_records = {}
+        num_tasks = 0
+        num_queries = 0
+        total_query_score = 0.0
+        total_task_score = 0.0
+
+        # Evaluate each task
+        for task in self.data:
+            task_name = task.get("task_name", "")
+            if task_name not in exist_records:
+                exist_records[task_name] = {}
+
+            # If no scoring config is found for the given task_name, skip
+            score_config = self.scoring_functions.get(
+                task_name,
+                {
+                    "field_score_function": {},
+                    "aggregation": {"function": None, "field_weights": {}},
+                    "response_parse_function": None,
+                },
+            )
+            if not task.get("query_response"):
+                # No queries to score
+                continue
+
+            num_tasks += 1
+            task_score_sum = 0.0
+            # Prepare the aggregator
+            aggregator = AggregationType.from_string(score_config["aggregation"]["function"])
+            field_weights = score_config["aggregation"]["field_weights"]
+
+            # Parse the metric definitions
+            field_score_functions = score_config.get("field_score_function", {})
+            global_aux_metrics = score_config.get("global_aux_metrics", {})
+            parser_type_str = score_config.get("response_parse_function", "dummy")
+            parser = ResponseParseType.from_string(parser_type_str)
+
+            # Extract the fields from the first correct_answer (assuming uniform)
+            first_correct = task["query_response"][0]["correct_answer"]
+            all_fields = list(first_correct.keys())
+            # Usually, we only treat “##something” fields as metadata, so skip them:
+            answer_fields = [f for f in all_fields if not f.startswith("##")]
+
+            # For each query in the task
+            for idx, query in enumerate(task["query_response"]):
+                num_queries += 1
+                response_text = query.get("response", "")
+                correct_answer = query["correct_answer"]
+
+                # 1) Parse the response according to the specified parser
+                response_obj = self._parse_response(
+                    task_name,
+                    parser,
+                    response_text,
+                    correct_answer,
+                    answer_fields,
+                    query,
+                    task,
+                )
+
+                if idx in exist_records[task_name]:
+                    query["scores"] = exist_records[task_name][idx]
+                else:
+                    # Initialize scores for this query
+                    query["scores"] = {"field": {}, "info": {}}
+
+                    # 2) Evaluate each field
+                    for fld, fld_metric_name in field_score_functions.items():
+                        metric = self._build_metric(fld_metric_name, score_config)
+                        self._evaluate_field(
+                            task_name,
+                            metric,
+                            fld,
+                            response_obj,
+                            correct_answer,
+                            query
+                        )
+
+                    # Evaluate global auxiliary metrics (if any)
+                    for fld, fld_metric_name in global_aux_metrics.items():
+                        metric = self._build_metric(fld_metric_name, score_config)
+                        # Some tasks want the entire response object to do an additional check
+                        # So, pass original `response_obj` under `fld` key:
+                        tmp_obj = {fld: response_obj}
+                        self._evaluate_field(
+                            task_name,
+                            metric,
+                            fld,
+                            tmp_obj,
+                            correct_answer,
+                            query,
+                            is_aux=True,
+                        )
+
+                    exist_records[task_name][idx] = query["scores"]
+                    if idx % 10 == 0 or idx == len(task["query_response"]) - 1:
+                        dump(exist_records, self.tmp_output_file)
+
+                # 3) Aggregate the query-level score
+                query["scores"]["query"] = aggregator.aggregate(
+                    query["scores"]["field"],
+                    field_weights,
+                )
+
+                if query["scores"]["query"] >= 0:
+                    task_score_sum += query["scores"]["query"]
+
+            # Calculate overall task score
+            if task["query_response"]:
+                mean_score = task_score_sum / len(task["query_response"])
+            else:
+                mean_score = 0.0
+            task["task_score"] = task_score_sum
+            task["mean_task_score"] = mean_score
+            task["eval_type"] = self._determine_eval_style(task)
+
+            total_query_score += task_score_sum
+            total_task_score += mean_score
+
+            print(f"[Task: {task_name}] Score = {task_score_sum} / {len(task['query_response'])}")
+
+        # Produce overall summary stats
+        summary = {}
+        if num_tasks > 0:
+            macro_mean_score = total_task_score / num_tasks
+            summary["macro_mean_score"] = macro_mean_score
+        else:
+            summary["macro_mean_score"] = 0.0
+
+        if num_queries > 0:
+            micro_mean_score = total_query_score / num_queries
+            summary["micro_mean_score"] = micro_mean_score
+        else:
+            summary["micro_mean_score"] = 0.0
+
+        summary["num_tasks"] = num_tasks
+        summary["num_queries"] = num_queries
+        # print(f"\n=== Evaluation Summary ===\n{json.dumps(summary, indent=4)}\n")
+
+        # Write back final data + summary
+        output_data = {
+            "data": self.data,
+            "summary": summary,
+        }
+        self._save_results(self.output_file, output_data)
+        print(f"Evaluation complete! Results saved to {self.output_file}")
+
+    def _evaluate_field(
+        self,
+        task_name: str,
+        metric: Any,
+        field: str,
+        response_obj: Dict[str, Any],
+        correct_answer: Dict[str, Any],
+        query: Dict[str, Any],
+        is_aux: bool = False,
+    ) -> float:
+        """Compute score for a single field using the given metric."""
+        eval_context = self._get_eval_context(task_name, query)
+
+        if metric == MetricType.UNSUPPORTED:
+            print(f"The metric for {field} in task {task_name} is not supported")
+            return 0.0
+        elif metric == MetricType.SYMBOLIC_PLANNING_TEST or metric == MetricType.PROGRAM_JUDGE:
+            query["scores"]["field"][field] = metric.match(
+                response_obj.get(field),
+                eval_context,
+            )
+        elif metric == MetricType.CONSTRAINED_GENERATION:
+            score, eval_info = metric.match(response_obj, eval_context)
+            query["scores"]["field"][field] = score
+            query["scores"]["info"][field] = eval_info
+        elif metric == MetricType.XML_NORM_POINT_IN_BBOX:
+            score, eval_info = metric.match(response_obj.get(field), eval_context)
+            query["scores"]["field"][field] = score
+            query["scores"]["info"][field] = eval_info
+        elif isinstance(metric, MetricType.VLM_AS_JUDGE.class_impl):
+            images = query.get("images", [])
+            question = query.get("question", "")
+            correct_val = correct_answer.get(field, "") if not is_aux else correct_answer
+            response_info = (
+                response_obj.get(field)
+                if isinstance(response_obj, dict)
+                else response_obj
+            )
+            query["scores"]["field"][field] = metric.match(
+                response_info,
+                correct_val,
+                images=images,
+                question=question,
+                eval_context=eval_context,
+            )
+        else:
+            correct_val = correct_answer.get(field, "") if not is_aux else correct_answer
+            correct_val = evaluate_as_string(correct_val)  # remove extra formatting
+            predicted_val = response_obj.get(field, "")
+            query["scores"]["field"][field] = metric.match(predicted_val, correct_val)
+
+    def _parse_response(
+        self,
+        task_name: str,
+        parser,
+        response_text: str,
+        correct_answer: Dict[str, Any],
+        answer_fields: List[str],
+        query: Dict[str, Any],
+        task: Dict[str, Any],
+    ) -> Dict[str, Any]:
+        """
+        Parse the raw response into a structured object, depending on the parser.
+        """
+        res_parsing_pass = True
+        if parser.is_single_field_parser():
+            # single field
+            assert (
+                len(answer_fields) == 1
+            ), "The answer_string parse must be used when the answer has a single field"
+            answer_key = answer_fields[0]
+
+            global_description = task["task_description"]
+            query_question = query["question"]
+            is_single_line_ans = "\n" not in correct_answer[answer_key]
+
+            response_obj = parser.parse(
+                response_text,
+                answer_key,
+                global_description=global_description,
+                query_question=query_question,
+                is_single_line_ans=is_single_line_ans,
+            )
+            assert isinstance(response_obj[answer_key], str), "Single-field parsing results must be string"
+        else:
+            # Structural output (using JSON parser or other specified parsing func) or dummy parse (return all)
+            response_obj = parser.parse(response_text)
+
+            if parser == ResponseParseType.JSON and (
+                not isinstance(response_obj, dict) or not response_obj
+            ):
+                # Expect a JSON, but parsing failed,
+                # Record the failure parsing, and use the raw string for each field of the answer
+                res_parsing_pass = False
+                response_obj = {}
+                for field in correct_answer:
+                    response_obj[field] = response_text
+
+        if not res_parsing_pass:
+            print(
+                f"Task:{task_name}, cannot parse query with global idx {query['global_idx']}"
+            )
+        return response_obj
+
+    def _build_metric(self, metric_name: str, score_config: Dict[str, Any]):
+        """
+        Given a string for the metric (e.g. 'gpt_4o_as_judge'),
+        return the actual MetricType or a specialized metric class.
+        """
+        metric = MetricType.from_string(metric_name)
+        if metric == MetricType.VLM_AS_JUDGE:
+            # Build the GPT4O metric using the provided config
+            gpt4o_configs = score_config.get("gpt4o_eval_configs", {})
+            metric = metric.class_impl(gpt4o_configs)
+        elif metric == MetricType.ASCII_ART_GPT4O_JUDGE:
+            # Build the ASCII Art metric using the provided config
+            ascii_art_configs = score_config.get("ascii_art_eval_configs", {})
+            metric = metric.class_impl(ascii_art_configs)
+        return metric
+
+    @staticmethod
+    def _load_json(file_path: str) -> Any:
+        with open(file_path, "r", encoding="utf-8") as f:
+            return json.load(f)
+
+    @staticmethod
+    def _save_results(file_path: str, data: Any) -> None:
+        """
+        Safe-write a JSON file via temp file + replace.
+        Since the results file is long, this avoid breaking the file in case of a crash.
+        """
+        temp_filename = f"{file_path}.tmp"
+        with open(temp_filename, "w", encoding="utf-8") as f:
+            json.dump(data, f, ensure_ascii=False, indent=4)
+        os.replace(temp_filename, file_path)
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Simple Evaluator")
+    parser.add_argument(
+        "--subset_name",
+        type=str,
+        required=True,
+        help="The subset of MEGA-Bench to evaluate.",
+    )
+    parser.add_argument(
+        "--submission_file",
+        type=str,
+        required=True,
+        help="Path to a JSON file containing model responses.",
+    )
+    parser.add_argument(
+        "--output_file",
+        type=str,
+        required=True,
+        help="Where to store the evaluation results (JSON).",
+    )
+
+    args = parser.parse_args()
+    evaluator = MEGABenchEvaluator(
+        subset_name=args.subset_name,
+        responses_file=args.submission_file,
+        output_file=args.output_file,
+    )
+    evaluator.evaluate()
+
+
+if __name__ == "__main__":
+    main()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/metric_type.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/metric_type.py
new file mode 100644
index 0000000000000000000000000000000000000000..b3b06a96c75d31b3c40df33be4c04a2215ddca9e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/metric_type.py
@@ -0,0 +1,260 @@
+import logging
+from enum import Enum
+from functools import cached_property
+
+from .utils import lazy_import
+
+
+class MetricType(Enum):
+    """The types of metrics."""
+
+    EXACT_STR_MATCH = "exact_str_match"
+    SIMPLE_STR_MATCH = "simple_str_match"
+    CODE_RESULT_EXACT_STR_MATCH = "code_result_exact_str_match"
+    DICT_EXACT_STR_MATCH_AGG_RECALL = "dict_exact_str_match_agg_recall"
+    EXACT_STR_MATCH_CASE_INSENSITIVE = "exact_str_match_case_insensitive"
+    NORM_SIM_DAMERAU_LEVENSHTEIN = "normalized_similarity_damerau_levenshtein"
+    NEAR_STR_MATCH = "near_str_match"
+    NUMBER_RELATIVE_DIFF_RATIO = "number_rel_diff_ratio"
+    SET_EQUALITY = "set_equality"
+    SET_EQUALITY_CASE_INSENSITIVE = "set_equality_case_insensitive"
+    DICT_SET_EQUALITY_AGG_JACCARD = "dict_set_equality_agg_jaccard"
+    DICT_PRECISION = "dict_precision"
+    JACCARD_INDEX = "jaccard_index"
+    JACCARD_INDEX_CASE_INSENSITIVE = "jaccard_index_case_insensitive"
+    DICT_JACCARD_AGG_JACCARD = "dict_jaccard_agg_jaccard"
+    DICT_EQUALITY = "dict_equality"
+    SET_PRECISION = "set_precision"
+    POSITIVE_INT_MATCH = "positive_int_match"
+    CHESS_MOVE_LIST_JACCARD_INDEX = "chess_move_list_jaccard_index"
+    LONGEST_COMMON_LIST_PREFIX_RATIO = "longest_common_list_prefix_ratio"
+    ASCII_ART_GPT4O_JUDGE = "ascii_art_gpt4o_judge"
+    NLI_ENTAILMENT = "nli_entailment"
+    BLEU = "bleu"
+    GLEU_CN = "gleu_cn"
+    XML_NORM_BBOX_IOU_SINGLE = "xml_nbbox_iou_single"
+    LATEX_EXPR_EQUALITY = "latex_expr_equality"
+    TEXT_WITH_LATEX_EXPR_EQUALITY = "text_with_latex_expr_equality"
+    NORM_BBOX_IOU_TUPLE = "nbbox_iou_tuple"
+    NORM_BBOX_IOU_SINGLE = "nbbox_iou_single"
+    NORM_BBOX_IOU_SEQUENCE = "nbbox_iou_sequence"
+    DICT_NORM_BBOX_IOU_TUPLE_AGG_JACCARD = "dict_nbbox_iou_tuple_agg_jaccard"
+    XML_NORM_POINT_IN_BBOX = "xml_norm_point_in_bbox"
+    XML_NORM_POINT_DISTANCE = "xml_norm_point_distance"
+    GEO_PROXIMITY_LOCATION_DICT = "geo_proximity_location_dict"
+    NORMALIZED_RMSE = "normalized_rmse"
+    PROGRAM_JUDGE = "program_judge"
+    STR_SET_EQUALITY_LINE_BREAK = "str_set_equality_line_break"
+    STR_SET_EQUALITY_COMMA = "str_set_equality_comma"
+    SEQUENCE_EQUALITY = "sequence_equality"
+    SEQUENCE_EQUALITY_CASE_INSENSITIVE = "sequence_equality_case_insensitive"
+    SEQUENCE_ACCURACY_CASE_INSENSITIVE = "sequence_accuracy_case_insensitive"
+    ANGLE_SEQ_FLOAT_RMSE = "angle_seq_float_rmse"
+    SYMBOLIC_PLANNING_TEST = "symbolic_planning_test"
+    MULTI_REF_PHRASE_EVAL = "multi_ref_phrase"
+    GENERAL_SINGLE_NUMERICAL_MATCH = "general_single_numerical_match"
+    BOXED_SINGLE_NUMERICAL_MATCH = "boxed_single_numerical_match"
+    SEQUENCE_COORDS_SIMILARITY = "sequence_coords_similarity"
+    CONSTRAINED_GENERATION = "constrained_generation"
+    VLM_AS_JUDGE = "gpt_4o_as_judge"
+    UNSUPPORTED = "unsupported"
+
+    @cached_property
+    def class_impl(self):
+        lazy_imports = {
+            MetricType.SIMPLE_STR_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.simple_str_match", "SimpleStrMatch"
+            ),
+            MetricType.EXACT_STR_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.exact_str_match", "ExactStrMatch"
+            ),
+            MetricType.CODE_RESULT_EXACT_STR_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.exact_str_match", "CodeResultExactStrMatch"
+            ),
+            MetricType.DICT_EXACT_STR_MATCH_AGG_RECALL: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.dict_exact_match_agg_recall",
+                "DictExactStrMatchAggRecall",
+            ),
+            MetricType.EXACT_STR_MATCH_CASE_INSENSITIVE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.exact_str_match_case_insensitive",
+                "ExactStrMatchCaseInsensitive",
+            ),
+            MetricType.NORM_SIM_DAMERAU_LEVENSHTEIN: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.normalized_similarity_damerau_levenshtein",
+                "NormalizedSimilarityDamerauLevenshtein",
+            ),
+            MetricType.NEAR_STR_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.near_str_match", "NearStrMatch"
+            ),
+            MetricType.NUMBER_RELATIVE_DIFF_RATIO: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.number_rel_diff_ratio", "NumberRelDiffRatio"
+            ),
+            MetricType.SET_EQUALITY: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.set_equality", "SetEquality"
+            ),
+            MetricType.SET_EQUALITY_CASE_INSENSITIVE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.set_equality", "SetEqualityCaseInsensitive"
+            ),
+            MetricType.DICT_SET_EQUALITY_AGG_JACCARD: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.dict_set_equality_agg_jaccard",
+                "DictSetEqualityAggJaccard",
+            ),
+            MetricType.DICT_EQUALITY: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.dict_equality",
+                "DictEquality",
+            ),
+            MetricType.DICT_PRECISION: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.dict_equality",
+                "DictPrecision",
+            ),
+            MetricType.JACCARD_INDEX: lazy_import("vlmeval.dataset.utils.megabench.scoring.jaccard", "Jaccard"),
+            MetricType.JACCARD_INDEX_CASE_INSENSITIVE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.jaccard", "JaccardCaseInsensitive"
+            ),
+            MetricType.DICT_JACCARD_AGG_JACCARD: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.dict_jaccard_agg_jaccard", "DictJaccardAggJaccard"
+            ),
+            MetricType.SET_PRECISION: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.set_precision", "SetPrecision"
+            ),
+            MetricType.POSITIVE_INT_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.positive_int_match", "PositiveIntMatch"
+            ),
+            MetricType.CHESS_MOVE_LIST_JACCARD_INDEX: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.chess_jaccard", "ChessMoveJaccard"
+            ),
+            MetricType.LONGEST_COMMON_LIST_PREFIX_RATIO: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.longest_common_list_prefix_ratio",
+                "LongestCommonListPrefixRatio",
+            ),
+            MetricType.ASCII_ART_GPT4O_JUDGE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.ascii_art_gpt4o_judge",
+                "AsciiArtVLMJudgeScore",
+            ),
+            MetricType.NLI_ENTAILMENT: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.nli_entailment", "NliEntailment"
+            ),
+            MetricType.BLEU: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.sacrebleu_bleu",
+                "Bleu",
+            ),
+            MetricType.GLEU_CN: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.gleu",
+                "GLEUChinese",
+            ),
+            MetricType.XML_NORM_BBOX_IOU_SINGLE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.xml_nbbox_iou", "XmlNbboxIouSingle"
+            ),
+            MetricType.BOXED_SINGLE_NUMERICAL_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.general_numerical_match", "BoxedSingleNumericalMatch"
+            ),
+            MetricType.GENERAL_SINGLE_NUMERICAL_MATCH: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.general_numerical_match", "GeneralSingleNumericalMatch"
+            ),
+            MetricType.SEQUENCE_COORDS_SIMILARITY: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.coordinate_sequence_match", "CoordsSequenceSimilarity"
+            ),
+            MetricType.LATEX_EXPR_EQUALITY: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.latex_expr_equality",
+                "LatexExprEquality",
+            ),
+            MetricType.TEXT_WITH_LATEX_EXPR_EQUALITY: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.latex_expr_equality",
+                "TextLatexExprEquality",
+            ),
+            MetricType.NORM_BBOX_IOU_TUPLE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.nbbox_iou", "NbboxIouTuple"
+            ),
+            MetricType.NORM_BBOX_IOU_SINGLE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.nbbox_iou", "NbboxIouSingle"
+            ),
+            MetricType.NORM_BBOX_IOU_SEQUENCE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.nbbox_iou", "NbboxIouSequence"
+            ),
+            MetricType.DICT_NORM_BBOX_IOU_TUPLE_AGG_JACCARD: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.dict_nbbox_iou_tuple_agg_jaccard",
+                "DictNbboxIouTupleAggJaccard",
+            ),
+            MetricType.XML_NORM_POINT_IN_BBOX: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.xml_norm_point_in_bbox",
+                "XmlNormPointInBbox",
+            ),
+            MetricType.XML_NORM_POINT_DISTANCE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.xml_norm_point_distance",
+                "XmlNormPointDistance",
+            ),
+            MetricType.GEO_PROXIMITY_LOCATION_DICT: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.geo_proximity", "GeoProximityLocationDict"
+            ),
+            MetricType.NORMALIZED_RMSE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.mse", "NormalizedRMSE"
+            ),
+            MetricType.PROGRAM_JUDGE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.program_judge", "ProgramJudge"
+            ),
+            MetricType.STR_SET_EQUALITY_LINE_BREAK: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.set_equality", "StringSetEqualityLineSplit"
+            ),
+            MetricType.STR_SET_EQUALITY_COMMA: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.set_equality", "StringSetEqualityCommaSplit"
+            ),
+            MetricType.SEQUENCE_EQUALITY: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.sequence_equality", "SequenceEquality"
+            ),
+            MetricType.SEQUENCE_EQUALITY_CASE_INSENSITIVE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.sequence_equality", "SequenceEqualityCaseInsensitive"
+            ),
+            MetricType.SEQUENCE_ACCURACY_CASE_INSENSITIVE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.sequence_equality", "SequenceAccuracyCaseInsensitive"
+            ),
+            MetricType.ANGLE_SEQ_FLOAT_RMSE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.mse", "AngleSeqFloatRMSE"
+            ),
+            MetricType.SYMBOLIC_PLANNING_TEST: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.symbolic_planning", "SymbolicPlanningMetricTest"
+            ),
+            MetricType.MULTI_REF_PHRASE_EVAL: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.multi_ref_phrase", "MultipleReferencePhraseEval"
+            ),
+            MetricType.CONSTRAINED_GENERATION: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.constrained_generation", "ConstrainedGenerationEval"
+            ),
+            MetricType.VLM_AS_JUDGE: lazy_import(
+                "vlmeval.dataset.utils.megabench.scoring.vlm_as_judge", "VLMJudgeScore"
+            ),
+        }
+
+        if self not in lazy_imports:
+            logging.error(f"Metric {self} not implemented...")
+
+        importer = lazy_imports.get(
+            self,
+            lazy_import("vlmeval.dataset.utils.megabench.scoring.unsupported_scoring", "UnsupportedScoring"),
+        )
+        return importer()
+
+    def match(self, response: str, correct_answer: str, task_info=None):
+        if not task_info:
+            return self.class_impl.match(response, correct_answer)
+        else:
+            return self.class_impl.match(response, correct_answer, task_info)
+
+    @classmethod
+    def from_string(cls, s):
+        try:
+            if s is None:
+                return cls("unsupported")
+            return cls(s.lower())
+        except KeyError as exc:
+            raise ValueError(f"Invalid metric type: {s}") from exc
+
+    @classmethod
+    def get_all_values(cls):
+        return list(cls)
+
+
+# List all of the supported metrics:
+if __name__ == "__main__":
+    print("All MetricType values:")
+    for metric_type in MetricType.get_all_values():
+        print(f"{metric_type.name}: {metric_type.value}")
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/answer_str_parse.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/answer_str_parse.py
new file mode 100644
index 0000000000000000000000000000000000000000..ff119075ab0a4c278360689a705e4265d8c83a37
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/answer_str_parse.py
@@ -0,0 +1,130 @@
+from .common.parsers import parse_json
+from .common.utils import (drop_additional_text, evaluate_as_string, extract_answer_content,
+                           extract_code_block_content)
+
+
+class AnswerStrParse:
+    """Parse the response for the single answer field."""
+
+    @classmethod
+    def _parse(
+        cls,
+        response: str,
+        *,
+        is_ascii_art: bool = False,
+        should_remove_surrounding_whitespace=True,
+        global_description: str = "",
+        query_question: str = "",
+        is_single_line_ans: bool = None,
+    ) -> dict:
+        """Try to parse a single answer."""
+        if response is None:
+            response = ""
+
+        # Extract the answer content based on "Answer: ..." format
+        answer_content = extract_answer_content(
+            response,
+            is_ascii_art=is_ascii_art,
+            should_remove_surrounding_whitespace=should_remove_surrounding_whitespace,
+        )
+
+        # Extract things from the code block if response is wrapped by a code block
+        answer_content, is_code = extract_code_block_content(
+            answer_content,
+            is_ascii_art=is_ascii_art,
+            should_remove_surrounding_whitespace=should_remove_surrounding_whitespace,
+        )
+
+        if not is_code and is_single_line_ans and not is_ascii_art:
+            answer_content = drop_additional_text(answer_content)
+
+        # Check if the content is a potential dict or list.
+        if answer_content.startswith("{") or answer_content.startswith("["):
+            # Attempt to parse the content as JSON
+            response_obj = parse_json(answer_content)
+            if response_obj == {}:
+                if "{}" not in answer_content:
+                    return answer_content
+            elif response_obj == []:
+                # logger.error(
+                #    f"Unexpected answer parsing error:\n{response=}\n{global_description=}\n{query_question=}\n{is_ascii_art=}"
+                # )
+                if "[]" not in answer_content:
+                    return answer_content
+            return str(response_obj) # make sure the response to the metric is always a string
+        else:
+            # drop the redundant string quotes
+            answer_content = evaluate_as_string(answer_content)
+            return answer_content
+
+    @classmethod
+    def parse(
+        cls,
+        response: str,
+        answer_key: str,
+        *,
+        global_description: str = "",
+        query_question: str = "",
+        is_single_line_ans: bool = None,
+    ) -> dict:
+        """Try to parse a single answer."""
+        response_parsed = cls._parse(
+            response,
+            is_ascii_art=False,
+            global_description=global_description,
+            query_question=query_question,
+            is_single_line_ans=is_single_line_ans,
+        )
+        results = {answer_key: response_parsed}
+        return results
+
+
+class AsciiAnswerStrParse(AnswerStrParse):
+    """Parse the response for the single ASCII answer field."""
+
+    @classmethod
+    def parse(
+        cls,
+        response: str,
+        answer_key: str,
+        *,
+        global_description: str = "",
+        query_question: str = "",
+        is_single_line_ans: bool = None,
+    ) -> dict:
+        """Try to parse a single answer."""
+        response_parsed = cls._parse(
+            response,
+            is_ascii_art=True,
+            global_description=global_description,
+            query_question=query_question,
+            is_single_line_ans=is_single_line_ans,
+        )
+        results = {answer_key: response_parsed}
+        return results
+
+
+class VerbatimAnswerStrParse(AnswerStrParse):
+    """Parse the response for a single answer field that should not have preceding or trailing whitespace removed."""
+
+    @classmethod
+    def parse(
+        cls,
+        response: str,
+        answer_key: str,
+        *,
+        global_description: str = "",
+        query_question: str = "",
+        is_single_line_ans: bool = None,
+    ) -> dict:
+        """Try to parse a single answer."""
+        response_parsed = cls._parse(
+            response,
+            is_ascii_art=True,
+            should_remove_surrounding_whitespace=False,
+            global_description=global_description,
+            query_question=query_question,
+            is_single_line_ans=is_single_line_ans,
+        )
+        results = {answer_key: response_parsed}
+        return results
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/common/parsers.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/common/parsers.py
new file mode 100644
index 0000000000000000000000000000000000000000..6b88d1af5f555ef237c8759a2e39c49ff6bb61aa
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/common/parsers.py
@@ -0,0 +1,146 @@
+import ast
+import json
+import re
+from typing import List
+
+import regex  # Supports the non-standard ?R regex operator
+
+from .utils import extract_answer_at_beginning_of_line, extract_code_block_content
+
+PARSING_TIMEOUT = 0.1
+
+
+def parse_json(response: str):
+    """Parse the JSON object, including nested JSON strings."""
+
+    response_ = extract_answer_at_beginning_of_line(response)
+
+    # If it's wrapped in code block like json, drop it
+    response_, _ = extract_code_block_content(response_, "json")
+
+    # Regular expression to match JSON-like structures, including nested quotes
+    json_pattern = r"(\{(?:[^{}]|(?R))*\}|\[(?:[^{}]|(?R))*\])"
+    string_pattern = r'"(?:\\.|[^"\\])*"'
+
+    # Find all potential JSON objects
+    try:
+        potential_jsons = regex.findall(
+            json_pattern, response_, timeout=PARSING_TIMEOUT
+        )
+    except TimeoutError:
+        if response_.startswith("["):
+            return []
+        return {}
+
+    valid_jsons = []
+
+    for potential_json in potential_jsons:
+        # Replace escaped quotes with a placeholder
+        potential_json = potential_json.replace('\\"', "__DOUBLE_QUOTE__")
+        potential_json = potential_json.replace("\\'", "__SINGLE_QUOTE__")
+
+        # Find all string literals
+        strings = regex.findall(string_pattern, potential_json)
+
+        # Process each string literal
+        for s in strings:
+            # Unescape the string content
+            unescaped = (
+                s[1:-1]
+                .replace("__DOUBLE_QUOTE__", '"')
+                .replace("__SINGLE_QUOTE__", "'")
+            )
+            # Try to parse it as JSON
+            try:
+                parsed = json.loads(unescaped)
+                if isinstance(parsed, (dict, list)):
+                    # If it's a valid JSON object or array, replace it in the original string
+                    potential_json = potential_json.replace(s, json.dumps(parsed))
+            except json.JSONDecodeError:
+                pass
+
+        # Restore escaped quotes
+        potential_json = potential_json.replace("__DOUBLE_QUOTE__", '\\"')
+        potential_json = potential_json.replace("__SINGLE_QUOTE__", "\\'")
+
+        try:
+            # Attempt to parse the potential JSON
+            json_object = json.loads(potential_json)
+            valid_jsons.append(json_object)
+        except json.JSONDecodeError:
+            # try to update single quote to double quote for some special failure case
+            # caused by quote's type
+            potential_json_ = re.sub(r"(?<!\w)\'|\'(?!\w)", '"', potential_json)
+            try:
+                json_object = json.loads(potential_json_)
+                valid_jsons.append(json_object)
+            except json.JSONDecodeError:
+                # If parsing still fails, it's not a valid JSON object
+                pass
+
+        try:
+            # Attempt to parse the Python structure
+            valid_jsons.append(ast.literal_eval(potential_json))
+            continue
+        except (SyntaxError, ValueError):
+            pass
+        # Escape the backslashes.
+        potential_json = potential_json.replace('\\"', '\\\\"')
+        potential_json = potential_json.replace("\\'", "\\\\'")
+        try:
+            # Attempt to parse the Python structure
+            valid_jsons.append(ast.literal_eval(potential_json))
+        except (SyntaxError, ValueError):
+            pass
+
+    # Return the last valid JSON if any, otherwise an empty dict or list
+    if valid_jsons:
+        return valid_jsons[-1]
+    if response_.startswith("["):
+        return []
+    return {}
+
+
+def parse_nested_str_list(input_string):
+    # Add quotation marks around the words in the string
+    quoted_string = re.sub(r"(\w+)", r'"\1"', input_string)
+
+    # Safely evaluate the string as a Python object
+    try:
+        python_object = ast.literal_eval(quoted_string)
+        return python_object
+    except (ValueError, SyntaxError) as e:
+        print(f"Failed to convert string to Python object: {e}")
+        return input_string
+
+
+def parse_syllable_ranges(input_str: str) -> List[List[int]]:
+    """Convert a bunch of syllable ranges into a list of intervals.
+
+    Examples:
+        parse_syllable_ranges('[7,10][7, 10][5,7][5,7][7,10]')
+        >>> [[7, 10], [7, 10], [5, 7], [5, 7], [7, 10]]
+        parse_syllable_ranges('575 575')
+        >>> [[5, 5], [7, 7], [5, 5], [0, 0], [5, 5], [7, 7], [5, 5]]
+        parse_syllable_ranges('[11]5')
+        >>> [[11, 11], [5, 5]]
+    """
+
+    def convert_to_range(match):
+        match = match.strip("[]")
+        if "," in match:
+            start, end = map(int, match.split(","))
+            return [start, end]
+        elif match == " ":
+            return [0, 0]
+        else:
+            num = int(match)
+            return [num, num]
+
+    # Split the input string into chunks
+    chunks = re.findall(r"(?:\[\d+(?:,\s*\d+)?\]|\d| )", input_str.strip())
+
+    # Convert each chunk to a range and create the result list
+    result = [convert_to_range(chunk) for chunk in chunks]
+
+    return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/common/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/common/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..45ee61ada498b7f4da41ee2e957707569c19d6b7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/common/utils.py
@@ -0,0 +1,138 @@
+import ast
+import re
+
+
+def extract_code_block_content(
+    response,
+    code_type=None,
+    is_ascii_art: bool = False,
+    should_remove_surrounding_whitespace=True,
+):
+    # If code_type is specified, construct the pattern to match that specific code block
+    if code_type:
+        pattern = rf"```{code_type}\s*\n*(.*?)\s*```"
+    elif is_ascii_art:
+        if not response.strip() or len(response) > 10000:
+            # handle the special case of pure whitespace or super long empty string
+            response = response.rstrip()
+        if should_remove_surrounding_whitespace:
+            pattern = r"```\w*(?:\s*\n+)?(.*?)\s*```"
+        else:
+            pattern = r"```\w*(?:\s*\n+)?(.*?)(?:\n+\s*)?```"
+    else:
+        # If code_type is None, match any code block
+        pattern = r"```\w*\s*\n*(.*?)\s*```"
+
+    # Search for the code block in the response
+    match = re.search(pattern, response, flags=re.DOTALL)
+
+    if match:
+        # If a match is found, return the content inside the code block
+        if is_ascii_art:
+            return match.group(1), True
+        else:
+            return match.group(1).strip(), True
+    else:
+        # If no code block is found, return the original string
+        return response, False
+
+
+def extract_answer_content(
+    response, is_ascii_art=False, should_remove_surrounding_whitespace=True
+):
+    if is_ascii_art:
+        match = re.search(r"\*\*?Answer:(.*?)\*\*?|\bAnswer:(.*)", response, re.DOTALL)
+    else:
+        match = re.search(
+            r"\*\*?Answer:\s*(.*?)\*\*?|\bAnswer:\s*(.*)", response, re.DOTALL
+        )
+    if match:
+        # Extract the content after "Answer:"
+        response = match.group(1) or match.group(
+            2
+        )  # Return the first capturing group or second if the first is None
+        if response is None:
+            response = ""
+    if is_ascii_art:
+        # Reduce anything that is more than one blank line to a single blank line.
+        response = re.sub(r"^\s*$(\n^\s*$)+", "", response, flags=re.MULTILINE)
+
+        if should_remove_surrounding_whitespace:
+            # Remove trailing whitespace
+            response = response.rstrip()
+        else:
+            # Remove trailing blank lines
+            response = re.sub(r"(\n\s*)+$", "", response)
+        # Remove leading blank lines
+        response = re.sub(r"^(\s*\n)+", "", response)
+    else:
+        response = response.strip()
+
+    return response
+
+
+def extract_answer_at_beginning_of_line(response):
+    # Regular expression to match either "Answer:" or "**Answer:**" at the beginning of a new line
+    match = re.search(r"^(?:\*\*Answer:|Answer:)\s*(.+)", response, re.MULTILINE)
+
+    if match:
+        # Return the content after "Answer:" or "**Answer: **"
+        return match.group(1).strip()
+    else:
+        # Return None if no match is found
+        return response.strip()
+
+
+def drop_additional_text(result):
+    # Heuristic to catch multiple-choice queries. Does not use metadata.json.
+    result_first_paragraph = result.split("\n\n")[0].strip()
+    potential_ans_in_single_line = re.search(
+        r"^(?:(?:[a-zA-Z0-9_-]+)(?:,\s*[a-zA-Z0-9_-]+)*|(?:[a-zA-Z0-9_-]+)\.|\((?:[a-zA-Z0-9_-]+)\)$)",
+        result_first_paragraph,
+    )
+
+    only_return_first_paragraph = (
+        potential_ans_in_single_line
+        and result_first_paragraph.strip() != ""
+        and not _is_multiline_answer(result)
+    )
+
+    if only_return_first_paragraph:
+        return result_first_paragraph
+    else:
+        return result
+
+
+def _is_multiline_answer(text):
+    # Split the text into lines
+    lines = text.splitlines()
+
+    # Find the "Answer:" line
+    for i, line in enumerate(lines):
+        stripped_line = line.strip()
+        if stripped_line != "":
+            # Check if the next line (second line after "Answer:") is blank
+            if i + 1 < len(lines) and lines[i + 1].strip() == "":
+                return False  # Second line is blank, single-line answer,
+                # remaining parts are additional
+            return True  # Second line is not blank, multi-line answer
+
+    return False  # empty result found, treat as single-line
+
+
+def evaluate_as_string(s):
+    try:
+        # Try to evaluate the string using ast.literal_eval
+        evaluated = ast.literal_eval(s)
+        # If it's a valid Python string, return it
+        if isinstance(evaluated, str):
+            return evaluated
+        else:
+            # If it's not a string, return the original input
+            return s
+    except (ValueError, SyntaxError):
+        # If it's not valid, return the original input
+        return s
+    except MemoryError:
+        # the result overflows, simply return an empty string
+        return ""
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/dummy_parse.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/dummy_parse.py
new file mode 100644
index 0000000000000000000000000000000000000000..21b5a2b15148cbd5416f61a1d72d39a46eb6b35a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/dummy_parse.py
@@ -0,0 +1,6 @@
+class DummyParse:
+
+    @staticmethod
+    def parse(response: str, *args, **kwargs) -> dict:
+        """return the raw string without doing anything"""
+        return response.strip()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/json_parse.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/json_parse.py
new file mode 100644
index 0000000000000000000000000000000000000000..1bc91313a4ce9c49e586f7b81acc92198fa68fe8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/parsing/json_parse.py
@@ -0,0 +1,17 @@
+from .common.parsers import parse_json
+from .common.utils import evaluate_as_string
+
+
+class JsonParse:
+    """Load the response as a JSON object."""
+
+    @staticmethod
+    def parse(response: str):
+        """Parse the JSON object, including nested JSON strings."""
+        parsed_res = parse_json(response)
+        # Drop the potentially duplicated string quotes
+        if isinstance(parsed_res, dict):
+            for key, val in parsed_res.items():
+                parsed_res[key] = evaluate_as_string(val)
+
+        return parsed_res
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/requirements.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/requirements.txt
new file mode 100644
index 0000000000000000000000000000000000000000..bc7e2d9758a7c1e2b97ab147aa16189e320ac912
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/requirements.txt
@@ -0,0 +1,15 @@
+antlr4-python3-runtime==4.11.0
+filelock==3.16.1
+geopy==2.4.1
+jieba==0.42.1
+nltk==3.9.1
+numpy==1.26.4
+pronouncing==0.2.0
+rapidfuzz==3.9.5
+regex==2024.7.24
+requests==2.32.3
+requests_cache==1.2.1
+sacrebleu==2.4.3
+sympy==1.13.2
+tqdm==4.66.4
+Unidecode==1.3.8
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/response_parse_type.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/response_parse_type.py
new file mode 100644
index 0000000000000000000000000000000000000000..e3affa8b88a0c8d7eb7b899d5cab6dc02316de41
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/response_parse_type.py
@@ -0,0 +1,51 @@
+from enum import Enum
+from functools import cached_property
+
+from vlmeval.dataset.utils.megabench.parsing.dummy_parse import DummyParse
+from .parsing.answer_str_parse import AnswerStrParse, AsciiAnswerStrParse, VerbatimAnswerStrParse
+from .parsing.json_parse import JsonParse
+
+
+class ResponseParseType(Enum):
+    """Parse the response."""
+
+    JSON = "json"
+    ANSWER_STR = "answer_string"
+    ASCII_ANSWER_STR = "ascii_answer_string"
+    VERBATIM_ANSWER_STR = "verbatim_answer_string"
+    DUMMY = "dummy"
+    UNSUPPORTED = "unsupported"
+
+    @cached_property
+    def class_impl(self):
+        if self == ResponseParseType.ANSWER_STR:
+            return AnswerStrParse
+        elif self == ResponseParseType.ASCII_ANSWER_STR:
+            return AsciiAnswerStrParse
+        elif self == ResponseParseType.VERBATIM_ANSWER_STR:
+            return VerbatimAnswerStrParse
+        elif self == ResponseParseType.DUMMY:
+            return DummyParse
+        else:
+            return JsonParse
+
+    def is_single_field_parser(self):
+        return self in [
+            ResponseParseType.ANSWER_STR,
+            ResponseParseType.ASCII_ANSWER_STR,
+            ResponseParseType.VERBATIM_ANSWER_STR,
+        ]
+
+    def parse(self, response: str, *args, **kwargs):
+        """Parse the response."""
+        return self.class_impl.parse(response, *args, **kwargs)
+
+    @staticmethod
+    def from_string(s):
+        """Initialize the response parsing type from a string."""
+        try:
+            if s is None:
+                return ResponseParseType("unsupported")
+            return ResponseParseType(s.lower())
+        except KeyError as exc:
+            raise ValueError(f"Invalid metric type: {s}") from exc
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/ascii_art_gpt4o_judge.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/ascii_art_gpt4o_judge.py
new file mode 100644
index 0000000000000000000000000000000000000000..07e29e325ffe2da6cff595587b8492253df2b229
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/ascii_art_gpt4o_judge.py
@@ -0,0 +1,129 @@
+"""Return if two ASCII art images depict the same thing."""
+
+import logging
+import os
+from numbers import Number
+
+import requests
+
+from .common.conversions import ascii_text_to_image
+from .vlm_as_judge import OpenAIVLMJudger
+
+"""Return if two ASCII art images depict the same thing."""
+
+
+class AsciiArtGPT4OJudge(OpenAIVLMJudger):
+    """A GPT-4o judge for assessing if two ASCII art images depict the same thing."""
+
+    def __init__(self, metric_config, model="gpt-4o-2024-08-06"):
+        self.eval_prompt = """Determine if the following two ASCII art images depict the same object.
+        Your answer should be either "yes" or "no", but without the quotation marks."""
+        super().__init__(
+            metric_config,
+            model,
+        )
+
+    def encode_image(self, image):
+        """Encode an image into base64 and return its mime type."""
+        mime_type = "image/jpeg"
+        image_format = "JPEG"
+
+        if image.mode == "RGBA":
+            image = self._rgba_to_rgb(image)
+
+        if self.resize and max(image.size) > self.max_side:
+            image = self._resize_image(image)
+            encoded_image = self._encode_image(image, image_format)
+        else:
+            encoded_image = self._encode_image(image, image_format)
+
+        return encoded_image, mime_type
+
+    def create_image_content(self, image):
+        base64_image, mime_type = self.encode_image(image)
+        return {
+            "type": "image_url",
+            "image_url": {"url": f"data:{mime_type};base64,{base64_image}"},
+        }
+
+    def prepare_eval_prompt(self, images):
+        """Prepare the evaluation prompt."""
+        content = []
+        for image_path in images:
+            content.append(self.create_image_content(image_path))
+
+        content.append({"type": "text", "text": self.eval_prompt})
+        return content
+
+    def query(self, images):
+        """Query GPT4o to determine if the ASCII images show the same thing."""
+        headers = {
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {self.api_key}",
+        }
+
+        context = self.prepare_eval_prompt(images)
+
+        query_payload = {
+            "model": self.model,
+            "messages": [{"role": "user", "content": context}],
+            "temperature": 0.0,
+        }
+
+        response_data = None
+        while response_data is None:
+            try:
+                response = requests.post(
+                    self.url,
+                    headers=headers,
+                    json=query_payload,
+                )
+            except (requests.exceptions.JSONDecodeError, requests.exceptions.ConnectionError) as e:
+                print(f'Error in requests: {e}')
+                print('Retry...')
+                continue
+
+            response_ = response.json()
+            if "error" in response_:
+                error_info = response_["error"]
+                print(
+                    f"Got error with type: {error_info['type']}. Message: {error_info['message']}"
+                )
+                print("Retry...")
+            else:
+                response_data = response_
+                break
+
+        total_tokens = response_data.get("usage", {}).get("total_tokens", "N/A")
+
+        if response_data and "choices" in response_data:
+            choices = response_data["choices"]
+            if choices and "message" in choices[0]:
+                message_content = choices[0]["message"]["content"]
+                print(
+                    f"gpt-4o judge results: {message_content}; tokens:{total_tokens}"
+                )
+        else:
+            print("gpt-4o judge query failed...")
+            message_content = ""
+
+        return message_content
+
+
+class AsciiArtVLMJudgeScore:
+    """Compute the cosine similarity between two pieces of ASCII art."""
+
+    def __init__(self, metric_config):
+        self.model = AsciiArtGPT4OJudge(metric_config)
+
+    def match(self, response, correct_answer) -> Number:
+        """Compute the cosine similarity between two pieces of ASCII art."""
+        if not isinstance(response, str) or not isinstance(correct_answer, str):
+            return 0
+        if not response:
+            return 0
+        response_image = ascii_text_to_image(response, 224, 224)
+        correct_answer_image = ascii_text_to_image(correct_answer, 224, 224)
+
+        eval_results = self.model.query([response_image, correct_answer_image])
+        return 1 if "yes" in eval_results.lower() else 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/chess_jaccard.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/chess_jaccard.py
new file mode 100644
index 0000000000000000000000000000000000000000..8ab0ac6799e590fae50d385f8e6d802d95feaf19
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/chess_jaccard.py
@@ -0,0 +1,25 @@
+import logging
+from typing import Any, Dict
+
+from .common.conversions import str_to_set
+from .common.metrics import jaccard_index
+
+
+def chess_transform(move_sequence: str) -> set:
+    """Transform a sequence of chess moves encoded in SAN into a set."""
+    move_sequence = str_to_set(move_sequence)
+    return {move_san.removesuffix("!").removesuffix("#") for move_san in move_sequence}
+
+
+class ChessMoveJaccard:
+    """Calculates the Jacard index for chess moves."""
+
+    @classmethod
+    def match(cls, responses: str | None, targets: str) -> float:
+        """Exact match between targets and responses."""
+        if responses is None:
+            return 0
+        responses = chess_transform(responses)
+        targets = chess_transform(targets)
+
+        return jaccard_index(responses, targets)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/conversions.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/conversions.py
new file mode 100644
index 0000000000000000000000000000000000000000..774b0ebd9f81964564646fe87d274f5ef6d9a669
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/conversions.py
@@ -0,0 +1,246 @@
+import ast
+import json
+import re
+from numbers import Number
+from typing import Tuple, Union
+
+from matplotlib import font_manager
+from PIL import Image, ImageDraw, ImageFont
+
+from vlmeval.dataset.utils.megabench.parsing.common.parsers import parse_json
+
+
+def freeze_structure(obj):
+    """Freeze a structure and make it hashable."""
+    if isinstance(obj, dict):
+        return frozenset((k, freeze_structure(v)) for k, v in obj.items())
+    elif isinstance(obj, (list, tuple)):
+        return tuple(freeze_structure(item) for item in obj)
+    elif isinstance(obj, set):
+        return frozenset(obj)
+    else:
+        return obj
+
+
+def cast_to_set(object) -> set:
+    """Try to cast an object as a set."""
+    object = freeze_structure(object)
+    if isinstance(object, (frozenset, set, tuple)):
+        return set(object)
+    return str_to_set(object)
+
+
+def cast_to_dict(object) -> dict:
+    """Try to cast an object as a dict."""
+    if isinstance(object, dict):
+        return {key: cast_to_dict(val) for key, val in object.items()}
+    elif isinstance(object, str):
+        extract_json_attempt = parse_json(object)
+        if extract_json_attempt:
+            return extract_json_attempt
+        return object
+    else:
+        return object
+
+
+def str_to_iterable(func, iterable_str):
+    """Converts a string representation of an iterable to an iterable."""
+    if not isinstance(iterable_str, str):
+        return func()
+
+    iterable_str = iterable_str.strip(" ")
+    if not iterable_str:
+        return func()
+
+    is_in_iterable = True
+    if iterable_str[0] == "(":
+        if not iterable_str.endswith(")"):
+            return func()
+    elif iterable_str[0] == "{":
+        if not iterable_str.endswith("}"):
+            return func()
+    elif iterable_str[0] == "[":
+        if not iterable_str.endswith("]"):
+            return func()
+    else:
+        is_in_iterable = False
+
+    # We may have a nested object, so try to use eval first
+    try:
+        eval_ = ast.literal_eval(iterable_str)
+        if eval_ is None:
+            return ""
+        if isinstance(eval_, (int, float)):
+            eval_ = [
+                eval_,
+            ]
+        return func(eval_)
+    except (SyntaxError, ValueError):
+        if is_in_iterable:
+            iterable_str = iterable_str[1:-1]
+        items = [item.strip() for item in iterable_str.split(",")]
+        return func(items)
+
+
+def str_to_set(iterable_str) -> set:
+    """Converts a string representation of an iterable to a set."""
+    return str_to_iterable(set, iterable_str)
+
+
+def str_to_list(iterable_str) -> set:
+    """Converts a string representation of an iterable to a set."""
+    return str_to_iterable(list, iterable_str)
+
+
+def str_to_bboxes(bbox_list) -> list:
+    if not isinstance(bbox_list, str):
+        return []
+    try:
+        bboxes = ast.literal_eval(bbox_list)
+    except (SyntaxError, ValueError):
+        try:
+            bboxes = json.loads(bbox_list)
+        except json.JSONDecodeError:
+            return []
+
+    if len(bboxes) == 4 and isinstance(bboxes[0], Number):
+        bboxes = [bboxes]
+
+    if not isinstance(bboxes, (tuple | list)):
+        return []
+
+    new_bboxes = []
+    for bbox in bboxes:
+        if not isinstance(bbox, (tuple, list)) or len(bbox) != 4:
+            continue
+        if any(not isinstance(coord, (float, int)) for coord in bbox):
+            continue
+        new_bboxes.append(bbox)
+    return new_bboxes
+
+
+def str_to_coords(coord_list, dim=2) -> list:
+    if not isinstance(coord_list, str):
+        return []
+    try:
+        coords = ast.literal_eval(coord_list)
+    except SyntaxError:
+        try:
+            coords = json.loads(coord_list)
+        except json.JSONDecodeError:
+            return []
+
+    new_coords = []
+    for coord in coords:
+        if not isinstance(coord, (tuple, list)) or len(coord) != dim:
+            continue
+        if any(not isinstance(coord, (float, int)) for coord in coord):
+            continue
+        new_coords.append(coord)
+    return new_coords
+
+
+def parse_point_2d_from_xml(xml_string) -> Union[Tuple[float, float], None]:
+    """Parse an (x, y) point from XML formatted like this: <point>x, y</point>"""
+    if not isinstance(xml_string, str):
+        return None
+
+    point_pattern = re.compile(r"<point>(.*?)<\/point>")
+    matches = point_pattern.findall(xml_string)
+    if len(matches) >= 2:
+        return None
+
+    if matches:
+        coords = matches[0].split(",")
+        if len(coords) != 2:
+            return None
+        try:
+            return tuple(float(coord.strip()) for coord in coords)
+        except ValueError:
+            return None
+
+
+def parse_bboxes_from_xml(xml_string: str) -> list:
+
+    if not isinstance(xml_string, str):
+        return []
+
+    bbox_pattern = re.compile(r"<box>(.*?)<\/box>")
+    matches = bbox_pattern.findall(xml_string)
+
+    new_bboxes = []
+    for match in matches:
+
+        coords = match.split(",")
+        if len(coords) != 4:
+            continue
+        try:
+            bbox = tuple(float(coord.strip()) for coord in coords)
+        except ValueError:
+            continue
+
+        if len(bbox) == 4 and all(isinstance(coord, float) for coord in bbox):
+            new_bboxes.append(bbox)
+
+    return new_bboxes
+
+
+MONOSPACE_FONTS = ("Courier New", "DejaVu Sans Mono", "Consolas", "SF Mono")
+
+MONOSPACE_FONT_FILES = []
+for font_name in MONOSPACE_FONTS:
+    try:
+        MONOSPACE_FONT_FILES.append(
+            font_manager.findfont(font_name, fallback_to_default=False)
+        )
+    except ValueError:
+        continue
+
+
+def ascii_text_to_image(
+    text,
+    width,
+    height,
+    font_size=20,
+    padding=10,
+    line_spacing=1,
+    bg_color="white",
+    text_color="black",
+):
+    """Convert ASCII text into an image."""
+    # Split the text into lines
+    lines = text.splitlines()
+
+    # Calculate initial image size based on text
+    char_width = font_size * 0.6  # Approximate width of a character
+    init_width = int(max(len(line) for line in lines) * char_width + 2 * padding)
+    init_height = int(
+        (len(lines) * font_size * line_spacing) + 2 * padding
+    )  # 1.2 for line spacing
+
+    # Create a new image with the calculated size
+    image = Image.new("RGB", (init_width, init_height), color=bg_color)
+    draw = ImageDraw.Draw(image)
+
+    # Load a monospace font
+    font = None
+    for font_name in MONOSPACE_FONT_FILES:
+        try:
+            font = ImageFont.truetype(font_name, font_size)
+            break
+        except IOError:
+            continue
+    if font is None:
+        raise ValueError("Cannot properly render ASCII art: missing monospace font.")
+
+    # Draw each line of text
+    y_text = padding
+    for line in lines:
+        draw.text((padding, y_text), line, font=font, fill=text_color)
+        y_text += font_size * line_spacing  # Move to the next line
+
+    # Resize the image to the specified dimensions
+    image = image.resize((width, height), Image.Resampling.LANCZOS)
+
+    # Convert the image to a NumPy array
+    return image
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/metrics.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/metrics.py
new file mode 100644
index 0000000000000000000000000000000000000000..2a9b7910a9ccb5e1cfc329615dcb6f52a1e50bd5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/metrics.py
@@ -0,0 +1,102 @@
+import math
+from collections.abc import Iterable
+from numbers import Number
+
+
+def calculate_iou(predicted: Iterable[Number], target: Iterable[Number]):
+    """Calculate the IoU between predicted and target bounding boxes."""
+
+    def box_area(box):
+        return (box[2] - box[0]) * (box[3] - box[1])
+
+    def box_iou(box1, box2):
+        # Calculate intersection coordinates
+        x1 = max(box1[0], box2[0])
+        y1 = max(box1[1], box2[1])
+        x2 = min(box1[2], box2[2])
+        y2 = min(box1[3], box2[3])
+
+        # Calculate intersection area
+        intersection = max(0, x2 - x1) * max(0, y2 - y1)
+
+        # Calculate union area
+        box1_area = box_area(box1)
+        box2_area = box_area(box2)
+        union = box1_area + box2_area - intersection
+
+        # Calculate IoU
+        iou = intersection / union if union > 0 else 0
+        return iou
+
+    # Calculate IoU for each pair of predicted and target boxes
+    iou_scores = []
+    for pred_box in predicted:
+        best_iou = 0
+        for target_box in target:
+            iou = box_iou(pred_box, target_box)
+            best_iou = max(best_iou, iou)
+        iou_scores.append(best_iou)
+
+    return iou_scores
+
+
+def set_relevance_score(denominator_fn, predicted: Iterable, target: Iterable) -> float:
+    """Calculate the relevance score."""
+    pred = set(predicted)
+    tget = set(target)
+    denominator = denominator_fn(pred, tget)
+    if not denominator:
+        return 1
+    return len(pred & tget) / denominator
+
+
+def _union_denominator(pred: set, tget: set) -> int:
+    return len(pred | tget)
+
+
+def _pred_denominator(pred: set, _: set) -> int:
+    return len(pred)
+
+
+def _tget_denominator(_: set, tget: set) -> int:
+    return len(tget)
+
+
+def jaccard_index(predicted: Iterable, target: Iterable) -> float:
+    """Calculate the Jaccard Index."""
+    return set_relevance_score(_union_denominator, predicted, target)
+
+
+def set_precision(predicted: Iterable, target: Iterable) -> float:
+    """Calculate the precision, using sets."""
+    return set_relevance_score(_pred_denominator, predicted, target)
+
+
+def set_recall(predicted: Iterable, target: Iterable) -> float:
+    """Calculate the recall, using sets."""
+    return set_relevance_score(_tget_denominator, predicted, target)
+
+
+def longest_common_prefix(list1: list, list2: list) -> list:
+    """Return the longest common prefix."""
+    index_first_difference = next(
+        (i for i, (a, b) in enumerate(zip(list1, list2)) if a != b),
+        min(len(list1), len(list2)),
+    )
+    return list1[:index_first_difference]
+
+
+def mse(predicted: Number, target: Number) -> Number:
+    """Return the mean squared error."""
+    return (predicted - target) ** 2
+
+
+def point_distance(predicted: tuple[float, ...], target: tuple[float, ...]):
+    """Return the distance between two points."""
+    if len(predicted) != len(target):
+        raise ValueError(
+            "point_distance: Predicted and target points have different dimensions."
+        )
+    return math.sqrt(
+        sum((comp_res - comp_tar) ** 2 for comp_res, comp_tar in zip(predicted, target))
+    )
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/transformations.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/transformations.py
new file mode 100644
index 0000000000000000000000000000000000000000..380d26e061de532af3fd75705c07a896d5b08b63
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/common/transformations.py
@@ -0,0 +1,120 @@
+"""Like-to-like data transformations."""
+
+import re
+import unicodedata
+
+
+def remove_def_indef_articles(text: str) -> str:
+    """Remove definite and indefinite articles."""
+    text_list = [t for t in text.split(" ") if t.lower() not in {"the", "a"}]
+    return " ".join(text_list)
+
+
+def replace_macrons_with_latex_overline(text: str) -> str:
+    """Replace letters with macrons with the LaTeX bar."""
+    result = []
+    for char in text:
+        if char.isalpha():
+            decomposed = unicodedata.normalize("NFD", char)
+            if len(decomposed) > 1 and decomposed[1] == "\u0304":  # Macron accent
+                result.append(f"\\overline{{{decomposed[0]}}}")
+            else:
+                result.append(char)
+        elif char != "\u0304":
+            result.append(char)
+        else:
+            result[-1] = f"\\overline{{{result[-1]}}}"
+
+    return "".join(result)
+
+
+def fix_overline_underscores(text: str) -> str:
+    """Puts underscores that are outside \overline within overline."""
+    pattern = r"\\overline\{([^}]*)\}_([^{}\\ ]*)"
+    return re.sub(pattern, r"\\overline{\1_\2}", text)
+
+
+# Dictionary mapping Unicode Greek letters to LaTeX equivalents
+greek_to_latex = {
+    # Lowercase Greek letters
+    "α": "\\alpha",
+    "β": "\\beta",
+    "γ": "\\gamma",
+    "δ": "\\delta",
+    "ε": "\\epsilon",
+    "ζ": "\\zeta",
+    "η": "\\eta",
+    "θ": "\\theta",
+    "ι": "\\iota",
+    "κ": "\\kappa",
+    "λ": "\\lambda",
+    "μ": "\\mu",
+    "ν": "\\nu",
+    "ξ": "\\xi",
+    "ο": "\\omicron",
+    "π": "\\pi",
+    "ρ": "\\rho",
+    "σ": "\\sigma",
+    "τ": "\\tau",
+    "υ": "\\upsilon",
+    "φ": "\\phi",
+    "χ": "\\chi",
+    "ψ": "\\psi",
+    "ω": "\\omega",
+    # Uppercase Greek letters
+    "Α": "\\Alpha",
+    "Β": "\\Beta",
+    "Γ": "\\Gamma",
+    "Δ": "\\Delta",
+    "Ε": "\\Epsilon",
+    "Ζ": "\\Zeta",
+    "Η": "\\Eta",
+    "Θ": "\\Theta",
+    "Ι": "\\Iota",
+    "Κ": "\\Kappa",
+    "Λ": "\\Lambda",
+    "Μ": "\\Mu",
+    "Ν": "\\Nu",
+    "Ξ": "\\Xi",
+    "Ο": "\\Omicron",
+    "Π": "\\Pi",
+    "Ρ": "\\Rho",
+    "Σ": "\\Sigma",
+    "Τ": "\\Tau",
+    "Υ": "\\Upsilon",
+    "Φ": "\\Phi",
+    "Χ": "\\Chi",
+    "Ψ": "\\Psi",
+    "Ω": "\\Omega",
+}
+
+
+def replace_greek_letters(text: str) -> str:
+    """Replace Greek letters in Unicode with their LaTeX equivalents."""
+    return re.sub(r"[α-ωΑ-Ω]", lambda match: greek_to_latex[match.group()] + " ", text)
+
+
+def remove_latex_math_delimiters(latex_str):
+    # Pattern to match \begin{...}[...] and \end{...}[...] commands
+    env_pattern = r"\\(begin|end)\{.*?\}(?:\[[^\[\]]*\])?"
+    latex_str = re.sub(env_pattern, "", latex_str)
+
+    # Remove \( and \)
+    inline_math_pattern = r"\\\(|\\\)"
+    latex_str = re.sub(inline_math_pattern, "", latex_str)
+
+    # Remove \[ and \]
+    display_math_pattern = r"\\\[|\\\]"
+    latex_str = re.sub(display_math_pattern, "", latex_str)
+
+    return latex_str
+
+
+def normalize_latex(text: str) -> str:
+    """Normalize the LaTeX expression."""
+    text = text.replace("\\bar", "\\overline")
+    text = replace_macrons_with_latex_overline(text)
+    text = fix_overline_underscores(text)
+    text = replace_greek_letters(text)
+    text = remove_latex_math_delimiters(text)
+    return text
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/constrained_generation.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/constrained_generation.py
new file mode 100644
index 0000000000000000000000000000000000000000..89d026cef5bd590da2faf844db822285d34a7ee5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/constrained_generation.py
@@ -0,0 +1,520 @@
+import collections
+import itertools
+import logging
+import re
+import signal
+from numbers import Number
+
+import pronouncing
+from nltk.corpus import wordnet
+from nltk.stem import WordNetLemmatizer
+from nltk.tokenize import sent_tokenize, word_tokenize
+
+from ..parsing.common.parsers import parse_nested_str_list, parse_syllable_ranges
+from .common.conversions import str_to_iterable
+
+
+def custom_lemmatize(word, lemmatizer):
+    """
+    Custom lemmatization to handle special cases like 'puppies' -> 'puppy'.
+    """
+    lemma = lemmatizer.lemmatize(word, wordnet.NOUN)
+
+    # Handle irregular plural forms manually
+    if word.endswith("ies") and lemma.endswith("y"):
+        lemma = lemma[:-1] + "y"
+    elif word.endswith("ves") and lemma.endswith("f"):
+        lemma = lemma[:-1] + "f"
+
+    return lemma
+
+
+def lemmatize_phrase(phrase, lemmatizer):
+    """
+    Lemmatizes a phrase (multiple words).
+    """
+    words = word_tokenize(phrase.lower())
+    lemmatized_words = [custom_lemmatize(word, lemmatizer) for word in words]
+    return " ".join(lemmatized_words)
+
+
+custom_phones_for_word = {
+    "'gainst": ["G EH1 N S T", "G EY1 N S T"],
+    "'midst": ["M IH1 D S T"],
+    "'mongst": ["M AH1 NG S T"],
+    "'neath": ["N IY1 TH"],
+    "beguiles": ["B IH0 G AY1 L Z"],
+    "cerulean": ["S ER0 UW1 L IY0 AH0 N"],
+    "doggo": ["D AO1 G OW0"],
+    "downtorn": ["D AW1 N T AO2 R N"],
+    "enthrall": ["EH0 N TH R AO1 L"],
+    "fam'ly": ["F AE1 M L IY0"],
+    "fiery": ["F AY1 ER0 IY0", "F AY1 R IY0"],
+    "flits": ["F L IH1 T S"],
+    "furred": ["F ER1 D"],
+    "kneels": ["N IY1 L Z"],
+    "o'er": ["OW1 ER0"],
+    "orbs": ["AO1 R B Z"],
+    "quenched": ["K W EH1 N CH D"],
+    "quietude": ["K W AY1 AH0 T UW0 D"],
+    "retold": ["R IY0 T OW1 L D"],
+    "scurries": ["S K ER1 IY0 Z"],
+    "sunbeams": ["S AH1 N B IY2 M Z"],
+    "syncs": ["S IH1 NG K S"],
+    "'twixt": ["T W IH1 K S T"],
+}
+
+
+file_logger = logging.getLogger("errorLogger")
+
+
+def phones_for_word(text: str) -> list[str]:
+    """A wrapper for pronouncingpy's phones_for_word to handle out-of-vocab issues."""
+    text = text.replace("’", "'").lower()
+
+    suffixes = [""]
+    prefixes = [""]
+    prefixes_to_remove = [""]
+    if text.endswith("'s"):
+        suffixes = [" Z"]
+        text = text.removesuffix("'s")
+
+    if text in custom_phones_for_word:
+        return [
+            pr + suffix
+            for pr, suffix in itertools.product(custom_phones_for_word[text], suffixes)
+        ]
+
+    pronunciations = pronouncing.phones_for_word(text)
+
+    # Guess pronunciation from word stem
+    if not pronunciations:
+        if suffixes[0] != "":  # "'s doesn't really work with the rest."
+            file_logger.error(f"OOV: {text}")
+            return []
+
+        if text.endswith("ed"):
+            suffixes = [" D", " T", " AH0 D", " IH0 D"]
+            text = text.removesuffix("ed")
+        elif text.endswith("s"):
+            # Some words, like bustles, end with es but the plural suffix is s.
+            if pronouncing.phones_for_word(text.removesuffix("s")):
+                # On the other hand, pierces is two syllables but pierce is one.
+                if text.endswith("es"):
+                    suffixes = [" S", " Z", " AH0 Z", " IH0 Z"]
+                    text = text.removesuffix("s")
+                else:
+                    suffixes = [" S", " Z"]
+                    text = text.removesuffix("s")
+            elif text.endswith("es"):
+                suffixes = [" AH0 Z", " IH0 Z"]
+                text = text.removesuffix("es")
+        if text.startswith("un"):
+            prefixes = ["AH0 N "]
+            text = text.removeprefix("un")
+        elif text.startswith("'"):
+            if pronouncing.phones_for_word("a" + text.removeprefix("'")):
+                prefixes_to_remove = ["AH0 "]
+                text = "a" + text.removeprefix("'")
+        pronunciations = pronouncing.phones_for_word(text)
+    pronunciations = [
+        (prefix + pr + suffix).removeprefix(prefix_to_remove)
+        for prefix, pr, suffix, prefix_to_remove in itertools.product(
+            prefixes, pronunciations, suffixes, prefixes_to_remove
+        )
+    ]
+
+    if not pronunciations:
+        file_logger.error(f"OOV: {text}")
+    return pronunciations
+
+
+def rhyming_part_include_unstressed(phones: str) -> str:
+    """Get the "rhyming part" of a string with CMUdict phones.
+
+    "Rhyming part" here means everything from the vowel in the
+    last syllable up to the end of the word.
+
+    Example:
+        >>> import pronouncing
+        >>> phones = pronouncing.phones_for_word("purple")
+        >>> rhyming_part_include_unstressed(phones[0])
+        'AH0 L'
+
+    Args:
+        phones: a string containing space-separated CMUdict phones
+
+    Returns:
+        a string with just the "rhyming part" of those phones
+    """
+    phones_list = phones.split()
+    for i in range(len(phones_list) - 1, 0, -1):
+        if phones_list[i][-1] in "012":
+            phones = " ".join(phones_list[i:])
+            break
+    return re.sub(r"\d", "", phones)
+
+
+def count_syllables(text: str) -> list[int]:
+    """Count the number of syllables in a piece of text."""
+    pronunciations = [phones_for_word(p) for p in text.split()]
+    syllable_counts = []
+    for pronun_possibility in itertools.product(*pronunciations):
+        syllable_counts.append(
+            sum([pronouncing.syllable_count(p) for p in pronun_possibility])
+        )
+    return syllable_counts
+
+
+def find_string_occurrences_with_variations(text, search_string):
+    lemmatizer = WordNetLemmatizer()
+
+    # Lemmatize the entire search phrase
+    search_lemma = lemmatize_phrase(search_string, lemmatizer)
+
+    # Tokenize the text into sentences
+    sentences = sent_tokenize(text)
+
+    occurrences = []
+    total_count = 0
+
+    # Iterate over each sentence
+    for i, sentence in enumerate(sentences, 1):  # Sentence numbers start from 1
+        # Lemmatize the entire sentence
+        lemmatized_sentence = lemmatize_phrase(sentence, lemmatizer)
+
+        # Count occurrences of the lemmatized search phrase in the lemmatized sentence
+        count_in_sentence = lemmatized_sentence.count(search_lemma)
+        if count_in_sentence > 0:
+            occurrences.append((i, count_in_sentence))
+            total_count += count_in_sentence
+
+    return total_count, occurrences
+
+
+def word_to_stresses(word: str) -> list[list[int]]:
+    """Convert a word to a list of stresses, for each valid pronunciation."""
+    pronunciations = phones_for_word(word)
+    stresses = {
+        tuple(int(stress) for stress in pronouncing.stresses(pronunc))
+        for pronunc in pronunciations
+    }
+    return [list(pronunc_stresses) for pronunc_stresses in stresses]
+
+
+def is_iambic_pair(stress1: int, stress2: int) -> bool:
+    """Whether the pair of stresses is a valid iambic pair."""
+    valid_pairs = {(2, 1), (0, 2), (0, 1), (0, 0), (1, 1), (2, 2)}
+    return (stress1, stress2) in valid_pairs
+
+
+def grouper_ignore_last(iterable, n):
+    "Collect data into fixed-length chunks or blocks"
+    # grouper('ABCDEFG', 3) --> ABC DEF
+    args = [iter(iterable)] * n
+    return zip(*args)
+
+
+def is_line_iambic(line: str) -> bool:
+    """Determine if a line is iambic."""
+    words = line.split()
+    stress_options = [word_to_stresses(word) for word in words]
+
+    def backtrack(word_index: int, syllable_index: int, prev_stress: int) -> bool:
+        """Using backtracking, determine if there is a sequence of pronunciations that is in iambic pentameter."""
+        if word_index == len(words):
+            # At this point, syllable_index is the number of syllables
+            return syllable_index % 2 == 0
+
+        for stress_pattern in stress_options[word_index]:
+            word_syllable_index = 0
+            if syllable_index % 2 != 0:
+                current_stress = stress_pattern[word_syllable_index]
+                if not is_iambic_pair(prev_stress, current_stress):
+                    continue
+                word_syllable_index += 1
+
+            word_valid_iambic_pairs = True
+            for stress1, stress2 in grouper_ignore_last(
+                stress_pattern[word_syllable_index:], 2
+            ):
+                if not is_iambic_pair(stress1, stress2):
+                    word_valid_iambic_pairs = False
+                    break
+                word_syllable_index += 2
+            if not word_valid_iambic_pairs:
+                continue
+
+            if word_syllable_index < len(stress_pattern):
+                assert word_syllable_index + 1 == len(stress_pattern)
+                next_stress = stress_pattern[word_syllable_index]
+                if backtrack(
+                    word_index + 1,
+                    syllable_index + word_syllable_index + 1,
+                    next_stress,
+                ):
+                    return True
+            else:
+                assert word_syllable_index == len(stress_pattern)
+                if backtrack(word_index + 1, syllable_index + word_syllable_index, -1):
+                    return True
+
+        return False
+
+    return backtrack(
+        0, 0, -1
+    )  # Start with -1 as prev_stress as a placeholder for the first syllable
+
+
+def parse_constraints(key_string, value_string):
+    key_components = key_string.strip().split("##")
+    # Remove trailing numbers from each key
+    key_components = [re.sub(r"\d+$", "", key) for key in key_components]
+    # Extract value components by splitting on ##
+    value_components = value_string.strip().split("##")
+    # Clean value components by removing brackets and spaces
+    value_components = [comp.strip().strip('"').strip() for comp in value_components]
+
+    # Handle cases where we expect integers
+    for i, value in enumerate(value_components):
+        if value.isdigit():
+            value_components[i] = int(value)
+
+    # Combine keys and values into a dictionary
+    if len(key_components) == len(value_components):
+        result = {
+            key.lower(): value for key, value in zip(key_components, value_components)
+        }
+    elif len(key_components) == 1 and len(value_components) == 1:
+        result = {key_components[0].lower(): value_components[0]}
+    else:
+        raise ValueError("Mismatch between number of keys and values.")
+
+    return result
+
+
+def check_constraint(response, constraint, constraint_val):
+    if constraint_val.strip() == "":
+        # empty contraint (placeholder), directly return 1
+        return 1
+    elif "contain" in constraint:
+        occurs_records = {}
+        parsed_constraint = parse_constraints(constraint, constraint_val)
+        response = response.replace("**", "")  # Remove markdown around bolded letters
+        if "contain_only" in parsed_constraint:
+            num_satisfied = 0
+            conditions = parse_nested_str_list(parsed_constraint["contain_only"])
+            for cond in conditions:
+                count, occurs = 0, []
+                for item in cond:  # check one condition
+                    count_, occurs_ = find_string_occurrences_with_variations(
+                        response, item
+                    )
+                    if count_ > 0:
+                        count += count_
+                        occurs.extend(occurs_)
+                if count > 0:
+                    num_satisfied += 1
+                    occurs_records[tuple(cond)] = occurs
+            score = 1 if num_satisfied == 1 else 0
+        else:  # the vanilla "contain" constraint
+            items = str_to_iterable(list, parsed_constraint["contain"])
+            count, occurs = 0, []
+            for item in items:
+                count_, occurs_ = find_string_occurrences_with_variations(
+                    response, item
+                )
+                if count_ > 0:
+                    count += count_
+                    occurs.extend(occurs_)
+            if count > 0:
+                occurs_records[tuple(items)] = occurs
+            score = 0 if count == 0 else 1
+
+        ## Other logics like position or repeat, only check when
+        ## previous "contain" consraint passes
+        if score > 0:
+            occurs = list(occurs_records.values())[0]
+            if "position_only" in parsed_constraint:
+                pos = parsed_constraint["position_only"]
+                score = 1 if len(occurs) == 1 and occurs[0][0] == pos else 0
+                return score
+            elif "position" in parsed_constraint:
+                pos = parsed_constraint["position"]
+                occurs_sent_ids = [item[0] for item in occurs]
+                score = 1 if pos in occurs_sent_ids else 0
+
+            # check occurance times
+            if "times" in parsed_constraint:
+                repeat_times = parsed_constraint["times"]
+                total_occurs = sum([item[1] for item in occurs])
+                score = 1 if total_occurs == repeat_times else 0
+
+    elif "length" in constraint:
+        try:
+            len_constraint = int(constraint_val[1:])
+            words = re.findall(r"\b\w+\b", response)
+            if constraint_val.strip() == "":
+                score = 1  ## dummy placeholder constraint, score is 1
+            elif constraint_val[0] == "<":
+                score = 1 if len(words) < len_constraint else 0
+            elif constraint_val[0] == ">":
+                score = 1 if len(words) > len_constraint else 0
+            else:
+                file_logger.warning(f"Unknown length info {constraint_val}")
+        except ValueError:
+            file_logger.warning(f"Wrong length info {constraint_val}")
+            score = 0
+    elif "acrostic" in constraint:
+        response = response.replace("**", "")
+
+        lines = response.strip().lower().split("\n")
+        if len(lines) != len(constraint_val):
+            return 0
+        all_match = True
+        if "acrostic_alliteration" in constraint:
+            for line, letter in zip(lines, constraint_val.lower()):
+                line = line.strip()
+                if letter == " ":
+                    if line != "":
+                        all_match = False
+                        break
+                elif not line or not all(word[0] == letter for word in line.split(" ")):
+                    all_match = False
+                    break
+        else:
+            for line, letter in zip(lines, constraint_val.lower()):
+                line = line.strip()
+                if letter == " ":
+                    if line != "":
+                        all_match = False
+                        break
+                elif not line or not line[0] == letter:
+                    all_match = False
+                    break
+        score = 1 if all_match else 0
+    else:
+        response = response.strip()
+        response = response.replace(".", "")
+        response = response.replace(",", "")
+        response = response.replace("!", "")
+        response = response.replace("?", "")
+        response = response.replace(":", "")
+        response = response.replace(";", "")
+        response = response.replace('"', "")
+        response = response.replace("-", " ")
+        response = response.replace("—", " ")
+        response = re.sub(
+            " *\(\w\) *(?=\n|$)", "", response
+        )  # The parenthesized letter in the rhyming scheme
+
+        lines = response.lower().split("\n")
+        match constraint:
+            case "syllables":
+                syllable_count_intervals = parse_syllable_ranges(constraint_val)
+                if len(lines) != len(syllable_count_intervals):
+                    return 0
+                try:
+                    all_match = all(
+                        any(
+                            min_count <= syll_count <= max_count
+                            for syll_count in count_syllables(line)
+                        )
+                        for line, (min_count, max_count) in zip(
+                            lines, syllable_count_intervals
+                        )
+                    )
+                except IndexError:
+                    all_match = None
+                score = 1 if all_match else 0
+            case "rhyming_scheme":
+                # Ensure that the number of lines is the same as the number in the rhyming scheme
+                if len(lines) != len(constraint_val):
+                    return 0
+                last_words = []
+                for line in lines:
+                    if line.strip():  # Check if line has non-whitespace content
+                        words = line.split()
+                        last_words.append(words[-1] if words else "")
+                    else:
+                        last_words.append("")
+
+                # Map each rhyming scheme letter to the last word of a line
+                letter_to_words = collections.defaultdict(set)
+                for rhyme_letter, word in zip(constraint_val, last_words):
+                    if rhyme_letter == " ":
+                        if word != "":
+                            return 0
+                    else:
+                        letter_to_words[rhyme_letter].add(word)
+
+                # Check that 1. The words for the same letter all rhyme
+                letter_to_rhyming_parts = {}
+                for letter, words in letter_to_words.items():
+                    rhyming_parts: list[set[str]] = [
+                        {
+                            rhyming_part_include_unstressed(pronunciations)
+                            for pronunciations in phones_for_word(word)
+                        }
+                        for word in words
+                    ]
+                    common_rhyming_parts = set.intersection(*rhyming_parts)
+                    if not common_rhyming_parts:
+                        return 0
+                    letter_to_rhyming_parts[letter] = common_rhyming_parts
+                # Check that 2. The words for different letters do not rhyme
+                for a, b in itertools.combinations(letter_to_rhyming_parts, 2):
+                    # To simplify things, if there are any shared pronunciations between two different letters, we reject it
+                    if letter_to_rhyming_parts[a] & letter_to_rhyming_parts[b]:
+                        return 0
+                score = 1
+            case "poetry_meter":
+                all_match = all(is_line_iambic(line) for line in lines)
+                score = 1 if all_match else 0
+            case _:
+                file_logger.warning(f"Unknown constraint type {constraint}")
+                score = 0
+
+    return score
+
+
+class ConstrainedGenerationEval:
+    """
+    Constrained generation metric
+    """
+
+    timeout = 10
+
+    @classmethod
+    def match(cls, response, constraints) -> Number:
+        scores = []
+        eval_results = {}
+
+        def handler(signum, frame):
+            raise TimeoutError()
+
+        def check_with_timeout(constraint, constraint_val):
+            # Set the signal handler and a timeout
+            signal.signal(signal.SIGALRM, handler)
+            signal.alarm(cls.timeout)  # Set the timeout
+
+            try:
+                # Try to check the constraint
+                score = check_constraint(response, constraint, constraint_val)
+            except TimeoutError:
+                print(f"Timeout reached for constraint: {constraint}")
+                score = 0  # Set score to 0 if timeout occurs
+            finally:
+                signal.alarm(0)  # Reset the alarm
+
+            return score
+
+        for constraint, constraint_val in constraints.items():
+            score = check_with_timeout(constraint, constraint_val)
+            scores.append(score)
+            eval_results[constraint] = score
+
+        final_score = min(scores)
+        eval_info = "\t".join([f"{key}: {val}" for key, val in eval_results.items()])
+
+        return final_score, eval_info
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/coordinate_sequence_match.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/coordinate_sequence_match.py
new file mode 100644
index 0000000000000000000000000000000000000000..35df44b6a2ada67acbac2b2325945a9696ac7055
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/coordinate_sequence_match.py
@@ -0,0 +1,67 @@
+import logging
+
+import numpy as np
+
+from .common.conversions import str_to_coords
+
+
+class CoordsSequenceSimilarity:
+    """
+    Measure the similarity between two list of coordinates, used for keypoint estimation tasks
+    """
+
+    @staticmethod
+    def compute_score(pred_keypoints, gt_keypoints, k=10):
+        """
+        Compute the evaluation score for keypoint estimation.
+
+        Args:
+            pred_keypoints (list or np.ndarray): List or array of predicted keypoint coordinates,
+                                                 each as (x, y), normalized to [0, 1].
+            gt_keypoints (list or np.ndarray): List or array of ground truth keypoint coordinates,
+                                               each as (x, y), normalized to [0, 1].
+
+        Returns:
+            float: A score between 0 and 1, where 1 indicates perfect accuracy,
+                   and 0 indicates completely wrong.
+        """
+        # Convert inputs to NumPy arrays
+        try:
+            pred_keypoints = np.array(pred_keypoints)
+        except ValueError:
+            # Format is not a correct
+            return 0
+
+        gt_keypoints = np.array(gt_keypoints)
+
+        # shape mismatch, directly assign 0 score
+        if pred_keypoints.shape != gt_keypoints.shape:
+            return 0
+
+        # Compute Euclidean distances between corresponding keypoints
+        distances = np.linalg.norm(pred_keypoints - gt_keypoints, axis=1)
+
+        # Maximum possible distance in normalized coordinate space
+        max_distance = np.sqrt(2)
+
+        # Normalize distances
+        normalized_distances = distances / max_distance
+
+        # Compute per-keypoint scores using exponential decay
+        per_keypoint_scores = np.exp(-k * normalized_distances)
+
+        # Compute the average score across all keypoints
+        score = np.mean(per_keypoint_scores)
+
+        return score
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        logging.debug(f"{responses=}, {targets=}")
+        if not isinstance(responses, (tuple | list)):
+            responses = str_to_coords(responses, dim=2)
+        if not isinstance(targets, (tuple | list)):
+            targets = str_to_coords(targets, dim=2)
+
+        return cls.compute_score(responses, targets)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_equality.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_equality.py
new file mode 100644
index 0000000000000000000000000000000000000000..326e0bd34a31a70f574d2dfa46b08f19cd6e9e96
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_equality.py
@@ -0,0 +1,44 @@
+from .common.conversions import cast_to_dict
+from .simple_str_match import ExactStrMatch
+
+
+class DictEquality:
+    """Calculates the exact string match across the dict.
+
+    1. Calculates the exact match for all keys in the solution
+    2. Calculates the total, then divides by the size of the solution
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return the aggregated Jaccard index between targets and responses."""
+        responses = cast_to_dict(responses)
+        targets = cast_to_dict(targets)
+
+        if not isinstance(responses, dict):
+            return 0
+
+        return 1 if responses == targets else 0
+
+
+class DictPrecision:
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return the aggregated Jaccard index between targets and responses."""
+        responses = cast_to_dict(responses)
+        targets = cast_to_dict(targets)
+
+        if not isinstance(responses, dict):
+            return 0
+
+        if len(responses) == 0:
+            return 0
+
+        matched = 0
+        for key, val in responses.items():
+            if key in targets:
+                if ExactStrMatch.match(val, targets[key]):
+                    matched += 1
+
+        return matched / len(responses)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_exact_match_agg_recall.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_exact_match_agg_recall.py
new file mode 100644
index 0000000000000000000000000000000000000000..5f1901e56df6348cc8a08857aaec8dc069252637
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_exact_match_agg_recall.py
@@ -0,0 +1,27 @@
+from .common.conversions import cast_to_dict
+from .exact_str_match import ExactStrMatch
+
+
+class DictExactStrMatchAggRecall:
+    """Calculates the exact string match across the dict.
+
+    1. Calculates the exact match for all keys in the solution
+    2. Calculates the total, then divides by the size of the solution
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return the aggregated Jaccard index between targets and responses."""
+        responses = cast_to_dict(responses)
+        targets = cast_to_dict(targets)
+
+        if not isinstance(responses, dict):
+            return 0
+
+        num_keys = 0
+        total_score = 0
+        for key, answer in targets.items():
+            total_score += ExactStrMatch.match(responses.get(key), answer)
+            num_keys += 1
+
+        return total_score / num_keys
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_jaccard_agg_jaccard.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_jaccard_agg_jaccard.py
new file mode 100644
index 0000000000000000000000000000000000000000..87ec776609e818bf8c992ade02378db6976c1c82
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_jaccard_agg_jaccard.py
@@ -0,0 +1,28 @@
+from .common.conversions import cast_to_dict
+from .jaccard import Jaccard
+
+
+class DictJaccardAggJaccard:
+    """Calculates the Jaccard index, dividing by the union of the predictions.
+
+    1. Calculates the Jaccard index for all sets with the same key,
+    if it appears in either pred or targets
+    2. Calculates the total, then divides by the size of the union
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return the aggregated Jaccard index between targets and responses."""
+        responses = cast_to_dict(responses)
+        if not isinstance(responses, dict):
+            return 0
+
+        all_keys = set(responses) | set(targets)
+
+        num_keys = 0
+        total_score = 0
+        for key in all_keys:
+            total_score += Jaccard.match(responses.get(key, []), targets.get(key, []))
+            num_keys += 1
+
+        return total_score / num_keys
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_nbbox_iou_tuple_agg_jaccard.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_nbbox_iou_tuple_agg_jaccard.py
new file mode 100644
index 0000000000000000000000000000000000000000..9a225d44b4a6dce7c587d04c21aba6f6f0ce0b22
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_nbbox_iou_tuple_agg_jaccard.py
@@ -0,0 +1,27 @@
+from .nbbox_iou import NbboxIouTuple
+
+
+class DictNbboxIouTupleAggJaccard:
+    """Calculates the average precision IoU across the dict.
+
+    1. Calculates the precision IoU for all sets with the same key,
+    if it appears in either pred or targets
+    2. Calculates the total, then divides by the size of the union
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return the aggregated Jaccard index between targets and responses."""
+        if not isinstance(responses, dict):
+            return 0
+        all_keys = set(responses) | set(targets)
+
+        num_keys = 0
+        total_score = 0
+        for key in all_keys:
+            total_score += NbboxIouTuple.match(
+                responses.get(key, []), targets.get(key, [])
+            )
+            num_keys += 1
+
+        return total_score / num_keys
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_set_equality_agg_jaccard.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_set_equality_agg_jaccard.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc5cf47e492bd80a9f7c1506caec09c576c684eb
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/dict_set_equality_agg_jaccard.py
@@ -0,0 +1,28 @@
+from vlmeval.dataset.utils.megabench.scoring.set_equality import SetEquality
+
+
+class DictSetEqualityAggJaccard:
+    """Calculates the average set equality across the dict.
+
+    1. Calculates the set equality for all sets with the same key,
+    if it appears in either pred or targets
+    2. Calculates the total, then divides by the size of the union
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return the aggregated Jaccard index between targets and responses."""
+        if not isinstance(responses, dict):
+            return 0
+
+        all_keys = set(responses) | set(targets)
+
+        num_keys = 0
+        total_score = 0
+        for key in all_keys:
+            total_score += SetEquality.match(
+                responses.get(key, []), targets.get(key, [])
+            )
+            num_keys += 1
+
+        return total_score / num_keys
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/exact_str_match.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/exact_str_match.py
new file mode 100644
index 0000000000000000000000000000000000000000..40dbdb383e7e6eb996b56f4e61e7ac57410a4d8a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/exact_str_match.py
@@ -0,0 +1,49 @@
+import re
+
+from ..parsing.common.utils import extract_code_block_content
+
+
+def parse_single_letter(s):
+    # Regular expression to match (A)XXXXX, A . XXXXXXX, or A.XXXXXX
+    match = re.match(r"^\(?([A-Za-z])\)?(?:\s*\.\s*|\.)?(.*)", s)
+
+    if match:
+        # Extract and return the single letter
+        return match.group(1)
+    else:
+        # Return the original string if no match is found
+        return s
+
+
+class ExactStrMatch:
+    """Exact string matching."""
+
+    @staticmethod
+    def match(response: str, correct_answer: str) -> int:
+        """Exact match between targets and responses."""
+        if not isinstance(response, str):
+            response = str(response)
+        if not isinstance(correct_answer, str):
+            correct_answer = str(correct_answer)
+
+        if len(correct_answer) == 1 and correct_answer.isalpha() and len(response) > 1:
+            # handle special case of choice letter,
+            # drop the potential parenthesis
+            response = parse_single_letter(response)
+
+        return 1 if response == correct_answer else 0
+
+
+class CodeResultExactStrMatch:
+    """Exact string matching, with the results from a results code block."""
+
+    @staticmethod
+    def match(response: str, correct_answer: str) -> int:
+        """Exact match between targets and responses."""
+        correct_answer, is_code = extract_code_block_content(
+            correct_answer,
+            is_ascii_art=True,
+            should_remove_surrounding_whitespace=False,
+        )
+        # assert is_code
+        return ExactStrMatch.match(response, correct_answer)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/exact_str_match_case_insensitive.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/exact_str_match_case_insensitive.py
new file mode 100644
index 0000000000000000000000000000000000000000..7dee848b6a0c2086f090823653a36e6b93ba7339
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/exact_str_match_case_insensitive.py
@@ -0,0 +1,12 @@
+from .exact_str_match import ExactStrMatch
+
+
+class ExactStrMatchCaseInsensitive:
+    """Case-insensitive exact string matching."""
+
+    @staticmethod
+    def match(response, correct_answer) -> int:
+        """Case-insensitive exact match between targets and responses."""
+        if not isinstance(response, str) and isinstance(correct_answer, str):
+            return 0
+        return ExactStrMatch.match(response.lower(), correct_answer.lower())
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/general_numerical_match.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/general_numerical_match.py
new file mode 100644
index 0000000000000000000000000000000000000000..232ac310a109325b05cfd3c4cc939af83194381e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/general_numerical_match.py
@@ -0,0 +1,253 @@
+import math
+import multiprocessing
+import re
+import signal
+
+from sympy.parsing.latex import parse_latex
+
+from .simple_str_match import SimpleStrMatch
+
+
+class TimeoutException(Exception):
+    pass
+
+
+def timeout_handler(signum, frame):
+    raise TimeoutException()
+
+
+E = 2.718
+
+############## Begin
+# Numerical comparison from https://github.com/TIGER-AI-Lab/MAmmoTH/blob/main/math_eval/number_utils.py
+
+
+def run_eval(expression, output):
+    try:
+        # Safely evaluate the expression
+        result = eval(expression)
+        output.put(result)
+    except Exception as e:
+        output.put(e)
+
+
+def eval_with_timeout(expression, timeout=5):
+    # Create a multiprocessing.Queue to receive the output
+    output = multiprocessing.Queue()
+
+    # Define and start the process
+    process = multiprocessing.Process(target=run_eval, args=(expression, output))
+    process.start()
+
+    # Wait for the process to complete or timeout
+    process.join(timeout)
+
+    if process.is_alive():
+        # Terminate the process
+        process.terminate()
+        process.join()
+        return "Timeout or error during evaluation"
+
+    # Get result from the queue
+    try:
+        return output.get_nowait()
+    except Exception as e:
+        return "Error retrieving result"
+
+
+def compare_two_list(pred, gt):
+    if not isinstance(pred, list):
+        return False
+    elif len(pred) != len(gt):
+        return False
+    elif any([not isinstance(x, (int, float)) for x in pred]):
+        return False
+    else:
+        pred = sorted(pred)
+        gt = sorted(gt)
+        return all([compare_two_numbers(p, g) for p, g in zip(pred, gt)])
+
+
+def compare_two_numbers(p, gt):
+    try:
+        if math.isnan(p):
+            return False
+        else:
+            return within_eps(pred=p, gt=gt)
+    except Exception:
+        return False
+
+
+def within_eps(pred: float, gt: float):
+    eps = abs(gt) * 0.01
+    if pred >= gt - eps and pred <= gt + eps:
+        return True
+    else:
+        return False
+
+
+def clean_units(pred_str: str):
+    """Clean the units in the number."""
+
+    def convert_pi_to_number(code_string):
+        code_string = code_string.replace("\\pi", "π")
+        # Replace \pi or π not preceded by a digit or } with 3.14
+        code_string = re.sub(r"(?<![\d}])\\?π", "3.14", code_string)
+        # Replace instances where π is preceded by a digit but without a multiplication symbol, e.g., "3π" -> "3*3.14"
+        code_string = re.sub(r"(\d)(\\?π)", r"\1*3.14", code_string)
+        # Handle cases where π is within braces or followed by a multiplication symbol
+        # This replaces "{π}" with "3.14" directly and "3*π" with "3*3.14"
+        code_string = re.sub(r"\{(\\?π)\}", "3.14", code_string)
+        code_string = re.sub(r"\*(\\?π)", "*3.14", code_string)
+        return code_string
+
+    pred_str = convert_pi_to_number(pred_str)
+    pred_str = pred_str.replace("%", "/100")
+    pred_str = pred_str.replace("$", "")
+    pred_str = pred_str.replace("¥", "")
+    pred_str = pred_str.replace("°C", "")
+    pred_str = pred_str.replace(" C", "")
+    pred_str = pred_str.replace("°", "")
+    return pred_str
+
+
+def number_it(num):
+    if isinstance(num, (int, float)):
+        return num
+
+    num = clean_units(num)
+    try:
+        num = str(parse_latex(num))
+    except Exception:
+        pass
+
+    if floatify(num) is not None:
+        return floatify(num)
+    else:
+        try:
+            num = eval_with_timeout(num)
+            if isinstance(num, list) or isinstance(num, tuple):
+                return num  # return num list
+            if floatify(num) is not None:
+                return floatify(num)
+            else:
+                return None
+        except Exception:
+            return None
+
+
+def floatify(num: str):
+    try:
+        num = float(num)
+        if num.is_integer():
+            return round(num)
+        else:
+            return num
+    except Exception:
+        return None
+
+
+def remove_latex_math_brackets(latex_str):
+    """
+    Removes LaTeX math mode delimiters (\( ... \) and \[ ... \]) from a string
+    while preserving the contents inside the delimiters.
+    If no such delimiters are found, the original string is returned.
+    """
+    # Regex pattern for inline math \( ... \)
+    inline_pattern = re.compile(r"\\\((.*?)\\\)")
+    # Regex pattern for TeX inline math $...$
+    tex_inline_pattern = re.compile(r"$(.*?)$")
+    # Regex pattern for display math \[ ... \]
+    display_pattern = re.compile(r"\\\[(.*?)\\\]")
+
+    latex_patterns = (inline_pattern, tex_inline_pattern, display_pattern)
+
+    if any(pattern.search(latex_str) for pattern in latex_patterns):
+        # Remove inline math mode brackets
+        latex_str = inline_pattern.sub(r"\1", latex_str)
+        # Remove display math mode brackets
+        latex_str = display_pattern.sub(r"\1", latex_str)
+    return latex_str
+
+
+def parse_assignment(expression):
+    # match the content after "=", "≈", or "\approx"
+    pattern = r"(?:=|≈|\\approx)\s*(.+)"
+
+    match = re.search(pattern, expression)
+    if match:
+        # Return the content after the sign
+        return match.group(1).strip()
+    else:
+        return expression
+
+
+############## End
+
+
+class GeneralSingleNumericalMatch:
+    """
+    Extract the results from ```\\boxed{xxxx}``` and match with the anaswer
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        if not isinstance(responses, str):
+            responses = str(responses)
+        responses = remove_latex_math_brackets(responses)
+        responses = parse_assignment(responses)
+        targets = remove_latex_math_brackets(targets)
+        targets = parse_assignment(targets)
+        res = number_it(responses)
+        tgt = number_it(targets)
+
+        if res is not None and tgt is not None:
+            if (
+                isinstance(res, list)
+                and isinstance(tgt, list)
+                or isinstance(res, tuple)
+                and isinstance(tgt, tuple)
+            ):
+                score = float(compare_two_list(res, tgt))
+            else:
+                score = float(compare_two_numbers(res, tgt))
+        else:
+            score = SimpleStrMatch.match(responses, targets)
+
+        return score
+
+
+class BoxedSingleNumericalMatch:
+    """
+    Extract the results from ```\\boxed{xxxx}``` and match with the anaswer
+    """
+
+    @staticmethod
+    def parse_boxed_content(text):
+        ###
+        #   Pattern: r'\\boxed\{((?:[^\{\}]+|\{[^\{\}]*\})*)\}':
+        #    \\boxed\{: Matches the literal \boxed{.
+        #    ((?:[^\{\}]+|\{[^\{\}]*\})*): This part matches the content inside the \boxed{}.
+        #    (?:...): A non-capturing group that allows us to match both non-brace content and brace-enclosed content.
+        #    [^\{\}]+: Matches any content that is not an opening { or closing } brace.
+        #    \{[^\{\}]*\}: Matches balanced braces containing non-nested content (e.g., {5} or {3} in the LaTeX expression \frac{5}{3}).
+        ###
+        pattern = r"\\boxed\{((?:[^\{\}]+|\{[^\{\}]*\})*)\}"
+        match = re.search(pattern, text)
+        return match.group(1) if match else text
+
+    @classmethod
+    def match(cls, responses, targets, timeout_duration=10) -> float:
+        if not isinstance(responses, str):
+            responses = str(responses)
+        signal.signal(signal.SIGALRM, timeout_handler)
+        signal.alarm(timeout_duration)  # Set the timeout duration in seconds
+        try:
+            parsed_res = cls.parse_boxed_content(responses)
+            targets = cls.parse_boxed_content(targets)
+            score = GeneralSingleNumericalMatch.match(parsed_res, targets)
+            return score
+        except TimeoutException:
+            return SimpleStrMatch.match(responses, targets)
+        finally:
+            signal.alarm(0)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/geo_proximity.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/geo_proximity.py
new file mode 100644
index 0000000000000000000000000000000000000000..d0b80546e0184b775693904ed6399cfffb383cf2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/geo_proximity.py
@@ -0,0 +1,100 @@
+import functools
+import logging
+import math
+import random
+
+from geopy.distance import distance
+from geopy.extra.rate_limiter import RateLimiter
+from geopy.geocoders import Nominatim
+
+USER_AGENT_SUFFIX = hex(random.getrandbits(128))[2:]
+geolocator = Nominatim(user_agent=f"vlm-mega-benchmark_{USER_AGENT_SUFFIX}")
+
+
+error_logger = logging.getLogger("errorLogger")
+
+
+def calculate_proximity_score(guess_coords, actual_coords, k=100):
+    """Calculate the proximity score based on the location.
+
+    Exponentially decreases depending on the distance.
+
+    Args:
+        guess_coords (float, float): The longitude and latitude of the guessed coordinates.
+        actual_coords (float, float): The longitude and latitude of the actual coordinates.
+        k (numbers.Number): The threshold (in km) at which we get a score of 0.5.
+    """
+    dist = distance(guess_coords, actual_coords).km
+    proximity_score = math.exp(-dist / k)
+    return proximity_score
+
+
+GEOLOCATION_TIMEOUT = 1
+MAX_RETRIES = 30
+
+
+geocode = RateLimiter(
+    geolocator.geocode, min_delay_seconds=GEOLOCATION_TIMEOUT, max_retries=MAX_RETRIES
+)
+
+
+@functools.cache
+def try_geolocate(query):
+    """Try to look up the location."""
+    location = geocode(query)
+    if location is None:
+        error_logger.error(
+            f"Geolocation API request failed due to timeout: exceeded {MAX_RETRIES} retries!"
+        )
+    return location
+
+
+def location_to_coords(
+    country: str, province_or_state: str, municipality: str
+) -> tuple[float, float] | None:
+    if country == "" or province_or_state == "" or municipality == "":
+        return None
+    """Convert the location to longitude and latitude."""
+    location = geolocator.geocode(
+        query={"country": country, "state": province_or_state, "city": municipality}
+    )
+    if location is not None:
+        return (location.latitude, location.longitude)
+    # Try searching without the province/state, as it can be non-standard for some questions
+    location = geolocator.geocode(query={"country": country, "city": municipality})
+    if location is None:
+        return None
+    return (location.latitude, location.longitude)
+
+
+class GeoProximityLocationDict:
+    """Return a score based on the distance between two locations."""
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Return a score based on how far two targets are away from each other,
+        where each field is a dict with the following schema:
+        {
+            country: str,
+            province_or_state: str,
+            municipality: str
+        }
+        """
+        try:
+            guess_coords = location_to_coords(**responses)
+        except:
+            return 0
+
+        if guess_coords is None:
+            error_logger.error(
+                f"GeoProximityLocationDict: could not load co-ordinates for {responses=}"
+            )
+            return 0
+        actual_coords = location_to_coords(**targets)
+        if actual_coords is None:
+            error_logger.error(
+                f"GeoProximityLocationDict: could not load co-ordinates for {targets=}"
+            )
+            return 0
+
+        return calculate_proximity_score(guess_coords, actual_coords)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/gleu.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/gleu.py
new file mode 100644
index 0000000000000000000000000000000000000000..7390febbb7ebb756384572a30ac6e137361440b5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/gleu.py
@@ -0,0 +1,18 @@
+from numbers import Number
+
+import jieba
+from nltk.translate.gleu_score import sentence_gleu
+
+
+class GLEUChinese:
+    """Compute GLEU score for Chinese text."""
+
+    @staticmethod
+    def match(response, correct_answer) -> Number:
+        """Compute the BLEU scores between two strings."""
+        if isinstance(response, str) and isinstance(correct_answer, str):
+            reference_tokens = list(jieba.cut_for_search(response))
+            translation_tokens = list(jieba.cut_for_search(correct_answer))
+        else:
+            return 0
+        return sentence_gleu([reference_tokens], translation_tokens)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/jaccard.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/jaccard.py
new file mode 100644
index 0000000000000000000000000000000000000000..7695e473ae66060b02795e370f94fc67642ee14a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/jaccard.py
@@ -0,0 +1,75 @@
+from .common.conversions import cast_to_set
+from .common.metrics import jaccard_index
+
+
+class Jaccard:
+    """Calculates the Jacard index for iterables."""
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        if responses is None:
+            return 0
+        responses = cast_to_set(responses)
+        targets = cast_to_set(targets)
+
+        return jaccard_index(responses, targets)
+
+
+class JaccardCaseInsensitive:
+    """Calculates the Jacard index for iterables of strings,
+    Do not consider the case
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        if responses is None:
+            return 0
+        responses = cast_to_set(responses)
+        targets = cast_to_set(targets)
+
+        if isinstance(list(targets)[0], str):
+            new_responses = {
+                item.lower() if isinstance(item, str) else str(item).lower()
+                for item in responses
+            }
+            new_targets = {item.lower() for item in targets}
+        elif isinstance(list(targets)[0], tuple):
+            new_responses = set()
+            new_targets = set()
+            try:
+                for res in responses:
+                    new_res = tuple(
+                        [
+                            item.lower()
+                            .replace(" ", "")
+                            .replace("-", "")
+                            .replace("\n", "")
+                            .replace("\t", "")
+                            .replace("_", "")
+                            .replace(".", "")
+                            for item in res
+                        ]
+                    )
+                    new_responses.add(new_res)
+            except:  # the data type of the response might be wrong, return 0 in this case
+                return 0
+            for tgt in targets:
+                new_tgt = tuple(
+                    [
+                        item.lower()
+                        .replace(" ", "")
+                        .replace("-", "")
+                        .replace("\n", "")
+                        .replace("\t", "")
+                        .replace("_", "")
+                        .replace(".", "")
+                        for item in tgt
+                    ]
+                )
+                new_targets.add(new_tgt)
+        else:
+            return 0
+
+        return jaccard_index(new_responses, new_targets)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/latex_expr_equality.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/latex_expr_equality.py
new file mode 100644
index 0000000000000000000000000000000000000000..74d49add9f68f9743699a11748b32c7ee0e40eb4
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/latex_expr_equality.py
@@ -0,0 +1,98 @@
+import re
+import signal
+
+from sympy.core.sympify import SympifyError
+from sympy.parsing.latex import parse_latex
+from sympy.parsing.latex.errors import LaTeXParsingError
+
+from .common.transformations import normalize_latex
+from .simple_str_match import SimpleStrMatch
+
+
+class TimeoutException(Exception):
+    pass
+
+
+def timeout_handler(signum, frame):
+    raise TimeoutException()
+
+
+class LatexExprEquality:
+    """Determines if two LaTeX expressions are equal."""
+
+    @classmethod
+    def match(cls, responses, targets, timeout_duration=15) -> int:
+        """Whether two LaTeX expressions are equal."""
+        if not isinstance(responses, str) or not isinstance(targets, str):
+            return 0
+        signal.signal(signal.SIGALRM, timeout_handler)
+        signal.alarm(timeout_duration)  # Set the timeout duration in seconds
+        try:
+            # seems that this eval can get stuck when evaluating all tasks..
+            responses = normalize_latex(responses)
+            targets = normalize_latex(targets)
+            responses_expr = parse_latex(responses)
+            targets_expr = parse_latex(targets)
+            result = 1 if responses_expr.equals(targets_expr) else 0
+            return result
+        except (
+            LaTeXParsingError,
+            SympifyError,
+            TypeError,
+            TimeoutException,
+            NotImplementedError,
+        ):
+            return SimpleStrMatch.match(responses, targets)
+        finally:
+            signal.alarm(0)  # Cancel the alarm if it completes successfully
+
+
+def separate_text_and_latex(text):
+    # Regular expression to match LaTeX content between $ symbols
+    pattern = r"(\$[^$]*\$)"
+
+    # Split the text based on LaTeX parts
+    parts = re.split(pattern, text)
+
+    # Separate plain text and LaTeX
+    latex_content = []
+    plain_text = []
+
+    for part in parts:
+        if part.startswith("$") and part.endswith("$"):
+            latex_content.append(part)
+        else:
+            plain_text.append(part.strip())
+
+    return plain_text, latex_content
+
+
+def join_latex(latex_exps):
+    result = []
+    for exp in latex_exps:
+        result.append(exp[1:-1].strip().replace(",", ""))
+    result = f"{' '.join(result)}"
+    return result
+
+
+class TextLatexExprEquality:
+    """Determines if two LaTeX expressions are equal."""
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Whether two LaTeX expressions are equal."""
+        if not isinstance(responses, str) or not isinstance(targets, str):
+            return 0
+
+        tgt_texts, tgt_latex = separate_text_and_latex(targets)
+        res_texts, res_latex = separate_text_and_latex(responses)
+
+        res_text_join = "".join(res_texts).replace(",", "")
+        tgt_text_join = "".join(tgt_texts).replace(",", "")
+        text_match = SimpleStrMatch.match(res_text_join, tgt_text_join)
+
+        res_latex_join = join_latex(res_latex)
+        tgt_latex_join = join_latex(tgt_latex)
+        latex_match = LatexExprEquality.match(res_latex_join, tgt_latex_join)
+
+        return 1 if text_match and latex_match else 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/longest_common_list_prefix_ratio.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/longest_common_list_prefix_ratio.py
new file mode 100644
index 0000000000000000000000000000000000000000..ebe652dda47c7525c2a78e0b6496a374fca8b100
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/longest_common_list_prefix_ratio.py
@@ -0,0 +1,15 @@
+from .common.conversions import str_to_list
+from .common.metrics import longest_common_prefix
+
+
+class LongestCommonListPrefixRatio:
+    """Determines how much of the first part of the list
+    was predicted correctly.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Exact match between targets and responses."""
+        responses = str_to_list(responses)
+        targets = str_to_list(targets)
+        return len(longest_common_prefix(responses, targets)) / len(targets)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/mse.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/mse.py
new file mode 100644
index 0000000000000000000000000000000000000000..0ef9c39955e305b5ccc3531f95010c17c20fb9ac
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/mse.py
@@ -0,0 +1,66 @@
+import ast
+import math
+
+import numpy as np
+
+from .common.conversions import str_to_list
+from .common.metrics import mse
+
+
+class MSE:
+    """Mean Squared Error."""
+
+    @staticmethod
+    def match(response: str, correct_answer: str) -> int:
+        """Return the mean squared error."""
+        try:
+            return mse(ast.literal_eval(response), ast.literal_eval(correct_answer))
+        except (SyntaxError, ValueError):
+            return 0
+
+
+class NormalizedRMSE:
+    """Mean Squared Error."""
+
+    MIN = 0.0
+    MAX = 0.1
+
+    @classmethod
+    def match(cls, response: str, correct_answer: str) -> int:
+        """Return the mean squared error."""
+        try:
+            mse_val = mse(ast.literal_eval(response), ast.literal_eval(correct_answer))
+            rmse = np.clip(np.sqrt(mse_val), cls.MIN, cls.MAX)
+            norm_rmse = 1 - (rmse - cls.MIN) / (cls.MAX - cls.MIN)
+            return norm_rmse
+        except (SyntaxError, ValueError):
+            return 0
+
+
+class AngleSeqFloatRMSE:
+    """Whether the sequence of numbers is close enough to the real answer."""
+
+    MIN = 0.0
+    MAX = 10.0
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Determines whether the sequence of floats are close enough to the real answer."""
+        responses = str_to_list(responses)
+        targets = str_to_list(targets)
+
+        if len(responses) != len(targets):
+            return 0
+
+        try:
+            res = np.array(responses)
+            tgt = np.array(targets)
+            rmse = np.sqrt(mse(res, tgt)).sum() / len(targets)
+        except:  # cannot obtain the rmse from the response, return 0
+            return 0
+
+        rmse = np.clip(rmse, cls.MIN, cls.MAX)
+        norm_rmse = 1 - (rmse - cls.MIN) / (cls.MAX - cls.MIN)
+        if math.isnan(norm_rmse):
+            return 0
+        return norm_rmse
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/multi_ref_phrase.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/multi_ref_phrase.py
new file mode 100644
index 0000000000000000000000000000000000000000..1f7c2a9426dc0157fe5ff760dbfe353435e9f380
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/multi_ref_phrase.py
@@ -0,0 +1,27 @@
+from numbers import Number
+
+from .common.conversions import str_to_iterable
+from .simple_str_match import SimpleStrMatch
+
+
+def replace_potential_chinese_comma(input_string):
+    return input_string.replace("，", ",")
+
+
+class MultipleReferencePhraseEval:
+    """
+    Check the response with multiple correct references
+    As long as one is matched, the score is 1, otherwise the score is 0
+    """
+
+    @staticmethod
+    def match(response, targets) -> Number:
+        targets = replace_potential_chinese_comma(targets)
+        refs = str_to_iterable(list, targets)
+        matched = False
+        for ref in refs:
+            str_ref = ref if isinstance(ref, str) else str(ref)
+            if SimpleStrMatch.match(response, str_ref):
+                matched = True
+                break
+        return 1 if matched else 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/nbbox_iou.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/nbbox_iou.py
new file mode 100644
index 0000000000000000000000000000000000000000..820fa316087c7d92302105b09dcfeb78a2982535
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/nbbox_iou.py
@@ -0,0 +1,106 @@
+import ast
+import logging
+
+import numpy as np
+
+from .common.conversions import str_to_bboxes
+from .common.metrics import calculate_iou
+
+
+class NbboxIouTuple:
+    """Calculates the IoU, for all bounding boxes, for all predicted bounding boxes.
+    For each predicted bounding box, it uses the IoU with the target bounding box with
+    the highest IoU.
+
+    Assumes that co-ordinates are normalized between 0 and 1 and that the bounding boxes
+    are of the form (x1, y1, x2, y2), where (x1, y1) is the top-left corner and (x2, y2)
+    is the bottom-right.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        logging.debug(f"{responses=}, {targets=}")
+        if not isinstance(responses, (tuple | list)):
+            responses = str_to_bboxes(responses)
+        if not isinstance(targets, (tuple | list)):
+            targets = str_to_bboxes(targets)
+
+        try:
+            iou_scores = calculate_iou(responses, targets)
+        except:
+            return 0
+
+        if not iou_scores:
+            return 0
+
+        # Take the mean IoU score for now.
+        return sum(iou_scores) / len(iou_scores)
+
+
+class NbboxIouSingle:
+    """
+    Single bbox IoU metric
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        logging.debug(f"{responses=}, {targets=}")
+        targets = ast.literal_eval(targets)
+        try:
+            responses = ast.literal_eval(responses)
+        except SyntaxError:
+            return 0
+
+        try:
+            iou_scores = calculate_iou(
+                [
+                    responses,
+                ],
+                [
+                    targets,
+                ],
+            )
+            if not iou_scores:
+                return 0
+        except:
+            return 0
+
+        # Take the mean IoU score for now.
+        return sum(iou_scores) / len(iou_scores)
+
+
+class NbboxIouSequence:
+    """
+    Metric for a sequence of bboxes (used for single object tracking).
+    The number of predicted boxes must match the ground truth.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        if not isinstance(responses, (tuple | list)):
+            responses = str(responses) if not isinstance(responses, str) else responses
+            responses = str_to_bboxes(responses)
+        if not isinstance(targets, (tuple | list)):
+            targets = str_to_bboxes(targets)
+
+        if len(targets) != len(responses):
+            return 0
+
+        scores = []
+        for res, tgt in zip(responses, targets):
+            scores.append(
+                calculate_iou(
+                    [
+                        res,
+                    ],
+                    [
+                        tgt,
+                    ],
+                )
+            )
+        avg_iou = np.mean(scores)
+
+        return avg_iou
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/near_str_match.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/near_str_match.py
new file mode 100644
index 0000000000000000000000000000000000000000..8deedf9d1f395fcab3768a8413ee4fb7e65efd51
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/near_str_match.py
@@ -0,0 +1,24 @@
+import rapidfuzz
+import unidecode
+
+from .common.transformations import remove_def_indef_articles
+
+
+def approximate(text: str) -> str:
+    """Return an approximation of the original string."""
+    return unidecode.unidecode(remove_def_indef_articles(text)).lower()
+
+
+class NearStrMatch:
+    """Near string matching."""
+
+    @staticmethod
+    def match(response, correct_answer: str, threshold=0.9) -> int:
+        """Simple string match between response and correct_answer."""
+        if not isinstance(response, str) or not isinstance(correct_answer, str):
+            return 0
+        response = approximate(response)
+        correct_answer = approximate(correct_answer)
+        return rapidfuzz.distance.DamerauLevenshtein.normalized_similarity(
+            response, correct_answer, score_cutoff=threshold
+        )
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/nli_entailment.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/nli_entailment.py
new file mode 100644
index 0000000000000000000000000000000000000000..686336ee862d425adc432638990b79e9ef8639ce
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/nli_entailment.py
@@ -0,0 +1,19 @@
+import torch
+from transformers import pipeline
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+pipe = pipeline(
+    "text-classification", model="microsoft/deberta-large-mnli", device=device
+)
+
+
+class NliEntailment:
+    """NLI entailment, where the correct answer is used as the premise."""
+
+    @staticmethod
+    def match(response, correct_answer) -> int:
+        """Return whether the response and correct answer agree with each other."""
+        if not isinstance(response, str) or isinstance(correct_answer, str):
+            return 0
+        resp = pipe(f"[CLS] {correct_answer.strip()} [SEP] {response.strip()} [SEP]")
+        return 1 if resp[0]["label"] == "ENTAILMENT" else 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/normalized_similarity_damerau_levenshtein.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/normalized_similarity_damerau_levenshtein.py
new file mode 100644
index 0000000000000000000000000000000000000000..110ed84665d541dd9bc510b3c9faaad4bdd12ddc
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/normalized_similarity_damerau_levenshtein.py
@@ -0,0 +1,14 @@
+import rapidfuzz
+
+
+class NormalizedSimilarityDamerauLevenshtein:
+    """Normalized Damerau-Levenshtein Similarity."""
+
+    @staticmethod
+    def match(response, correct_answer) -> int:
+        """Normalized indel similarityuiio do between targets and responses."""
+        if not isinstance(response, str) and isinstance(correct_answer, str):
+            return 0
+        return rapidfuzz.distance.DamerauLevenshtein.normalized_similarity(
+            response, correct_answer
+        )
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/number_rel_diff_ratio.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/number_rel_diff_ratio.py
new file mode 100644
index 0000000000000000000000000000000000000000..070023f2ab3516866eb121cd1121df67eac6ae55
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/number_rel_diff_ratio.py
@@ -0,0 +1,22 @@
+import ast
+import math
+from numbers import Number
+
+
+class NumberRelDiffRatio:
+    """Number relative difference ratio scoring = min(0, 1 - |pred - gt| / gt)"""
+
+    @staticmethod
+    def match(response: str | Number, correct_answer: str) -> int:
+        """Return the relative difference ratio."""
+        try:
+            if isinstance(response, Number):
+                pred = response
+            else:
+                pred = ast.literal_eval(response)
+            if not isinstance(pred, Number):
+                return 0
+            gt = ast.literal_eval(correct_answer)
+            return max(0, 1 - math.fabs((pred - gt) / gt))
+        except (SyntaxError, ValueError):
+            return 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/positive_int_match.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/positive_int_match.py
new file mode 100644
index 0000000000000000000000000000000000000000..bd72d416dd22de6228a00e530174038cd9bbd08d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/positive_int_match.py
@@ -0,0 +1,31 @@
+import ast
+
+
+class PositiveIntMatch:
+    """Positive int matching."""
+
+    @staticmethod
+    def match(response: str, correct_answer: str) -> int:
+        """If the correct answer or response is a positive integer, then it returns if the predicted and correct answers are identical.
+
+        Otherwise, it returns -1.
+        """
+        try:
+            response_obj = ast.literal_eval(response)
+        except (SyntaxError, ValueError):
+            return 0
+
+        if not correct_answer:
+            return 0
+
+        correct_answer_obj = ast.literal_eval(correct_answer)
+
+        assert isinstance(correct_answer_obj, int)
+        if not isinstance(response_obj, int):
+            return 0
+
+        # We only want to score the fields with a positive amount
+        if correct_answer_obj <= 0 and response_obj <= 0:
+            return -1
+
+        return 1 if response_obj == correct_answer_obj else 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/program_judge.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/program_judge.py
new file mode 100644
index 0000000000000000000000000000000000000000..0ad324af064393c5b9d8cb668c8c7d2d5be58cb5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/program_judge.py
@@ -0,0 +1,141 @@
+import io
+import json
+import multiprocessing
+import pathlib
+from multiprocessing.queues import Empty
+from unittest.mock import patch
+
+BIG_BENCH_PATH = pathlib.Path(__file__).resolve().parent.parent.parent
+
+
+class ProgramJudge:
+    """Program Judging."""
+
+    # Check if results have been saved for this metric instance
+    # prevent duplicate saving results
+    task_saved = {}
+
+    @classmethod
+    def save_test_results(cls, task_name, results, query_file):
+        query_base = pathlib.Path(query_file).parent
+        output_dir = query_base / "code_eval"
+        output_dir.mkdir(parents=True, exist_ok=True)
+        output_file = output_dir / f"{task_name}_test_case.json"
+
+        saved = cls.task_saved.get(task_name, False)
+        if output_file.is_file() and saved:
+            with open(output_file, "r") as f:
+                existing_data = json.load(f)
+            existing_data.extend(results)
+        else:
+            existing_data = results
+            cls.task_saved[task_name] = True
+        with open(output_file, "w") as f:
+            json.dump(existing_data, f, indent=4)
+
+    @staticmethod
+    def match(response: str, eval_context: str, task_info: str = None) -> int:
+        # Load all test cases from the benchmark_tasks directory
+        # task_name = task_info["task_name"]
+        # task_folder = task_info["task_folder"]
+        # query_results_file = task_info["results_file"]
+
+        test_cases = eval_context["test_case"]
+
+        # Create a CodeTester instance with the response and the found test cases
+        tester = CodeTester(response, test_cases)
+        score, results = tester.run_tests()
+
+        # ProgramJudge.save_test_results(task_name, results, query_results_file)
+        return score
+
+
+#########################################################
+### Implementation of the automatic code tester
+#########################################################
+
+
+class CodeTester:
+    def __init__(self, user_code, test_cases, timeout=2, verbose=True):
+        self.user_code = user_code
+        self.test_cases = test_cases
+        self.timeout = timeout
+        self.verbose = verbose
+
+    def run_user_code(self, input_data):
+        input_str = "\n".join(input_data) + "\n"
+        output_queue = multiprocessing.Queue()
+        process = multiprocessing.Process(
+            target=self.target, args=(output_queue, input_str)
+        )
+        process.start()
+
+        process.join(self.timeout)
+        if process.is_alive():
+            process.terminate()
+            return f"ERROR: Code execution exceeded the time limit."
+
+        try:
+            result = output_queue.get(timeout=10)  # Add timeout for queue retrieval
+        except Empty:
+            return "ERROR: No output was produced before timeout."
+        finally:
+            output_queue.close()  # Close the queue to release resources
+            output_queue.join_thread()  # Ensure all items in the queue are processed
+
+        return result
+
+    def target(self, output_queue, input_str):
+        contains_main_block = 'if __name__ == "__main__":' in self.user_code
+        stdout = io.StringIO()
+        try:
+            with patch("builtins.input", side_effect=input_str.splitlines()):
+                with patch("sys.stdout", new=stdout):
+                    if contains_main_block:
+                        # If the user code contains the main block, execute in the context of __name__ == "__main__"
+                        exec(self.user_code, {"__name__": "__main__"})
+                    else:
+                        # Otherwise, just execute the user code directly
+                        exec(self.user_code)
+        except Exception as e:
+            output_queue.put(f"ERROR during execution: {e}")
+        else:
+            output_queue.put(stdout.getvalue().rstrip())
+
+    def evaluate_test_case(self, input_data, expected_output):
+        output = self.run_user_code(input_data)
+        return output == expected_output.rstrip(), output
+
+    def run_tests(self):
+        if isinstance(self.test_cases, dict):
+            self.test_cases = [self.test_cases]
+        total_tests = len(self.test_cases)
+        passed_tests = 0
+        results = []
+
+        for i, test_case in enumerate(self.test_cases, 1):
+            result, output = self.evaluate_test_case(
+                test_case["input"], test_case["expected"]
+            )
+
+            test_result = {
+                "response": self.user_code,
+                "test_case": test_case["input"],
+                "output": output,
+                "expected": test_case["expected"],
+                "result": "Passed" if result else "Failed",
+            }
+            results.append(test_result)
+
+            if result:
+                if self.verbose:
+                    print(f"Test case {i}: Passed")
+                passed_tests += 1
+            else:
+                if self.verbose:
+                    print(
+                        f"Test case {i}: Failed - Expected {test_case['expected']} but got {output}"
+                    )
+
+        score = passed_tests / total_tests if total_tests > 0 else 0
+        return score, results
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/sacrebleu_bleu.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/sacrebleu_bleu.py
new file mode 100644
index 0000000000000000000000000000000000000000..3d11be6f2baf195bb616ed8f422bd6e681fa334f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/sacrebleu_bleu.py
@@ -0,0 +1,23 @@
+from numbers import Number
+
+import sacrebleu
+
+
+class Bleu:
+    """Compute BLEU score, using SacreBLEU."""
+
+    @staticmethod
+    def match(response, correct_answer) -> Number:
+        """Compute the BLEU scores between two strings."""
+        if isinstance(response, str) and isinstance(correct_answer, str):
+            resp = [response]
+            corr = [correct_answer]
+        elif isinstance(response, (list, tuple)) and isinstance(
+            correct_answer, (list, tuple)
+        ):
+            resp = tuple(response)
+            corr = tuple(correct_answer)
+        else:
+            return 0
+        result = sacrebleu.corpus_bleu(corr, [resp]).score / 100
+        return result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/sequence_equality.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/sequence_equality.py
new file mode 100644
index 0000000000000000000000000000000000000000..7a3e82cff673277c3985d1f7d71660ae49ed1a06
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/sequence_equality.py
@@ -0,0 +1,63 @@
+from numbers import Number
+
+from .common.conversions import str_to_list
+
+
+class SequenceEquality:
+    """Determines how much of the first part of the list
+    was predicted correctly.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Exact match between targets and responses."""
+        if not isinstance(responses, str):
+            responses = str(responses)
+        responses = str_to_list(responses)
+        targets = str_to_list(targets)
+        return 1 if responses == targets else 0
+
+
+class SequenceEqualityCaseInsensitive:
+    """Determines how much of the first part of the list
+    was predicted correctly.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Exact match between targets and responses."""
+        if not isinstance(responses, str):
+            responses = str(responses)
+        responses = str_to_list(responses)
+        targets = str_to_list(targets)
+
+        responses = [
+            item.lower() if isinstance(item, str) else str(item) for item in responses
+        ]
+        targets = [item.lower() for item in targets]
+        return 1 if responses == targets else 0
+
+
+class SequenceAccuracyCaseInsensitive:
+    """Determines how much of the first part of the list
+    was predicted correctly.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Exact match between targets and responses."""
+        responses = str_to_list(responses)
+        targets = str_to_list(targets)
+        if len(targets) != len(responses):
+            return 0
+        correct = 0
+        for res, tgt in zip(responses, targets):
+            if isinstance(tgt, str):
+                if res.lower() == tgt.lower():
+                    correct += 1
+            elif isinstance(tgt, Number) and isinstance(res, Number):
+                if res == tgt:
+                    correct += 1
+            else:
+                pass
+        return correct / len(targets)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/set_equality.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/set_equality.py
new file mode 100644
index 0000000000000000000000000000000000000000..01745bcf7cffe32c04573eae89edbbeba521033d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/set_equality.py
@@ -0,0 +1,74 @@
+from .common.conversions import cast_to_set, str_to_set
+
+
+def _convert_to_hashable(item):
+    """将不可哈希的类型转换为可哈希类型"""
+    if isinstance(item, (list, tuple)):
+        return tuple(item)  # 将列表转换为元组
+    return item
+
+
+class SetEquality:
+    """Determines whether two sets are equal."""
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Exact match between targets and responses."""
+        if isinstance(responses, (list, tuple)):
+            responses = {_convert_to_hashable(item) for item in responses}
+        if isinstance(targets, (list, tuple)):
+            targets = {_convert_to_hashable(item) for item in targets}
+        return 1 if responses == targets else 0
+
+
+class SetEqualityCaseInsensitive:
+    """Determines whether two sets are equal, ignoring string case."""
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        """Exact match between targets and responses."""
+        try:
+            responses: set[str] = {text.upper() for text in cast_to_set(responses)}
+            targets: set[str] = {text.upper() for text in cast_to_set(targets)}
+        except AttributeError:
+            return 0
+        return 1 if responses == targets else 0
+
+
+class StringSetEqualityLineSplit:
+    """Determines whether two sets are equal, for string inputs, separated by line breaks"""
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        if "\\n" in targets:
+            targets = targets.replace("\\n", "\n")
+        if "\\n" in responses:
+            responses = responses.replace("\\n", "\n")
+        responses_set = set(responses.split("\n"))
+        targets_set = set(targets.split("\n"))
+        responses_set = {
+            item.lower() if isinstance(item, str) else item for item in responses_set
+        }
+        targets_set = {
+            item.lower() if isinstance(item, str) else item for item in targets_set
+        }
+        return 1 if responses_set == targets_set else 0
+
+
+class StringSetEqualityCommaSplit:
+    """Determines whether two sets are equal, for string inputs, separated by commas
+    Handles some corner cases that would fail the general SetEquality metric, like the string
+    with "None", which fails the eval. Also do case-insensitive eval.
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> int:
+        responses_set = str_to_set(responses)
+        targets_set = str_to_set(targets)
+        responses_set = {
+            item.lower() if isinstance(item, str) else item for item in responses_set
+        }
+        targets_set = {
+            item.lower() if isinstance(item, str) else item for item in targets_set
+        }
+        return 1 if responses_set == targets_set else 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/set_precision.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/set_precision.py
new file mode 100644
index 0000000000000000000000000000000000000000..b030131f3fc04c2aa619195f5df385721ac0678f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/set_precision.py
@@ -0,0 +1,16 @@
+from .common.conversions import cast_to_set
+from .common.metrics import set_precision
+
+
+class SetPrecision:
+    """Calculates the set precision for iterables."""
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Exact match between targets and responses."""
+        if responses is None:
+            return 0
+        responses = cast_to_set(responses)
+        targets = cast_to_set(targets)
+
+        return set_precision(responses, targets)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/simple_str_match.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/simple_str_match.py
new file mode 100644
index 0000000000000000000000000000000000000000..e117410d7cfe4ca7498514839d344fbacf297ef4
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/simple_str_match.py
@@ -0,0 +1,29 @@
+from .exact_str_match import ExactStrMatch
+
+
+class SimpleStrMatch:
+    """Basic string matching, without spaces or hyphens."""
+
+    @staticmethod
+    def match(response, correct_answer: str) -> int:
+        """Simple string match between response and correct_answer."""
+        if not isinstance(response, str):
+            response = str(response)  # If it is JSON-like
+        response = (
+            response.replace(" ", "")
+            .replace("-", "")
+            .replace("\n", "")
+            .replace("\t", "")
+            .replace(".", "")
+            .lower()
+        )
+        correct_answer = (
+            correct_answer.replace(" ", "")
+            .replace("-", "")
+            .replace("\n", "")
+            .replace("\t", "")
+            .replace(".", "")
+            .lower()
+        )
+
+        return ExactStrMatch.match(response, correct_answer)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/symbolic_planning.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/symbolic_planning.py
new file mode 100644
index 0000000000000000000000000000000000000000..165a43d4ed4d01e2a27cd49155d433c43d650b07
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/symbolic_planning.py
@@ -0,0 +1,266 @@
+import re
+
+PARAM_LIST_MATCHER = re.compile(r"((?:\?\S+\s*)+)(?:-\s+([^\?$]+)\s*)?")
+PARAM_NAME_MATCHER = re.compile(r"\?([^\s\?\)]+)\s*")
+
+
+##### Parsing functions and parentheses matching
+def parse_pddl_param_list(s):
+    s = s.strip()
+    assert s[0] == "(" and s[-1] == ")"
+    s = s[1:-1]
+    param_type_dict = {}
+    for params, p_type in PARAM_LIST_MATCHER.findall(s):
+        for p in PARAM_NAME_MATCHER.findall(params):
+            p_type = p_type.strip()
+            if p_type.startswith("("):
+                p_type = p_type[1:-1].strip()
+                assert "either"
+                param_type_dict[p] = re.split(r"\s+", p_type)[1:]
+            else:
+                param_type_dict[p] = p_type
+    return s.split("?")[0].strip(), param_type_dict
+
+
+def parse_outer_inner_str(s, str_ender, inner_starter, inner_ender):
+    inner_count = 0
+    start_id = 0
+    matched_str = []
+    for i, c in enumerate(s):
+        if inner_count == 0 and c == str_ender:
+            return s[: i + 1], matched_str, i + 1
+        elif c == inner_starter:
+            if inner_count == 0:
+                start_id = i
+            inner_count += 1
+        elif c == inner_ender:
+            inner_count -= 1
+            if inner_count == 0:
+                matched_str.append(s[start_id : i + 1])
+    return s, matched_str, len(s)
+
+
+def parse_pddl_attr_from_string(
+    s,
+    attr_starter="(:",
+    attr_ender=")",
+    inner_starter="(",
+    inner_ender=")",
+    overlap=False,
+):
+    s_attr = s.split(attr_starter)
+    if len(s_attr) == 1:
+        return "", []
+    elif len(s_attr) == 2:
+        outer_str, inner_str, _ = parse_outer_inner_str(
+            s_attr[1], attr_ender, inner_starter, inner_ender
+        )
+        return attr_starter + outer_str, inner_str
+    else:
+        matched_dict = {}
+        outer_list = []
+        if not overlap:
+            while len(s.split(attr_starter)) > 1:
+                s = s.split(attr_starter, 1)[1]
+                name = re.split(r"\s+", s.strip())[0]
+                outer_str, inner_str, end_point = parse_outer_inner_str(
+                    s, attr_ender, inner_starter, inner_ender
+                )
+                outer_list.append(attr_starter + outer_str)
+                matched_dict[name] = inner_str
+                s = s[end_point:]
+        else:
+            for seg in s_attr[1:]:
+                name = re.split(r"\s+", seg.strip())[0]
+                outer_str, inner_str, _ = parse_outer_inner_str(
+                    seg, attr_ender, inner_starter, inner_ender
+                )
+                outer_list.append(attr_starter + outer_str)
+                matched_dict[name] = inner_str
+        return outer_list, matched_dict
+
+
+def remove_type_in_cnf(s):
+    s_split_type = s.split(" - ")
+    if len(s_split_type) > 1:
+        for i in range(1, len(s_split_type)):
+            if len(s_split_type[i].strip().split(")")[0].split()) == 1:
+                s_split_type[i] = ")" + s_split_type[i].strip().split(")", 1)[1]
+            else:
+                s_split_type[i] = " " + s_split_type[i].strip().split(" ", 1)[1]
+        return "".join(s_split_type).strip()
+    else:
+        return s
+
+
+def split_cnf_by_parentheses(s):
+    assert s.startswith("(and")
+    matches = set()
+    p_count = 0
+    clause_start_id = 0
+    for i in range(len(s)):
+        if s[i] == "(":
+            p_count += 1
+            if p_count == 2:
+                clause_start_id = i
+        elif s[i] == ")":
+            p_count -= 1
+            if p_count == 0:
+                break
+            elif p_count == 1:
+                matches.add(remove_type_in_cnf(s[clause_start_id : i + 1]))
+    return matches
+
+
+##### End of parsing functions
+
+
+####### Domain (the env for each planning task)
+class Domain:
+    def __init__(self, name, domain_pddl):
+        # self.name = name
+
+        # Domain files
+        self.domain_pddl = domain_pddl
+        self.action_name, self.action_params, self.action_params_dict = (
+            self.get_domain_action()
+        )
+        self.gt_cond_dict = self.parse_gt_pre_post_cond()
+
+    def get_domain_action(self):
+        action_pddl_str_list, all_actions = parse_pddl_attr_from_string(
+            self.domain_pddl, attr_starter="(:action"
+        )
+        action_name, action_params, action_params_dict = [], [], []
+        for action_pddl_str, (name, action_attr) in zip(
+            action_pddl_str_list, all_actions.items()
+        ):
+            assert len(action_attr) == 3
+            param_str, pre_cond_str, post_cond_str = action_attr
+            action_name.append(name)
+            action_params.append(param_str)
+            action_params_dict.append(parse_pddl_param_list(param_str)[1])
+        return action_name, action_params, action_params_dict
+
+    def parse_gt_pre_post_cond(self):
+        cond_dict = {}
+        for a in self.action_name:
+            act_str = self.domain_pddl.split(f"(:action {a}")[1]
+            for postfix in ["pre", "post"]:
+                split_tag = ":precondition" if postfix == "pre" else ":effect"
+                cond_str = act_str.split(split_tag)[1].strip()
+                if cond_str.startswith("(and"):
+                    cond_dict[f"{a}_{postfix}"] = split_cnf_by_parentheses(cond_str)
+                else:
+                    cond_dict[f"{a}_{postfix}"] = {cond_str.split(")")[0].strip() + ")"}
+                cond_dict[f"{a}_{postfix}"] = sorted(
+                    list(cond_dict[f"{a}_{postfix}"]),
+                    key=lambda x: 0 if x.startswith("(not ") else 1,
+                )
+        return cond_dict
+
+
+##### Transition functions
+def construct_param_to_obj(domain, action):
+    action = action[1:-1]
+    a_name = action.split(" ")[0].strip()
+    objs = action.split(" ")[1:]
+    a_index = domain.action_name.index(a_name)
+    assert len(objs) == len(domain.action_params_dict[a_index])
+    return {p: obj for p, obj in zip(domain.action_params_dict[a_index], objs)}, a_name
+
+
+def state_transition(current_state, effects, param_to_obj):
+    for obj_cond in effects:
+        for param in param_to_obj:
+            obj_cond = re.sub(
+                r"\?{}(?=[^\w-])".format(param), param_to_obj[param], obj_cond
+            )
+        _, reversed_cond = parse_pddl_attr_from_string(obj_cond, attr_starter="(not ")
+        if reversed_cond:
+            assert len(reversed_cond) == 1
+            if reversed_cond[0] in current_state:
+                current_state.remove(reversed_cond[0])
+        elif obj_cond.strip() not in current_state:
+            current_state.append(obj_cond)
+    return current_state
+
+
+def check_pre_conds_satisfy(current_state, pre_conds, param_to_obj):
+    for obj_cond in pre_conds:
+        for param in param_to_obj:
+            obj_cond = re.sub(
+                r"\?{}(?=[^\w-])".format(param), param_to_obj[param], obj_cond
+            )
+        if (obj_cond.startswith("(not ") and obj_cond in current_state) or (
+            not obj_cond.startswith("(not ") and obj_cond not in current_state
+        ):
+            return False
+    return True
+
+
+##### End of transition functions
+
+
+class SymbolicPlanningMetricTest:
+    """An example metric for symbolic planning tasks"""
+
+    @classmethod
+    def match(cls, response, eval_context, task_info=None):
+        ## Initialize domain
+        # task_name = task_info["task_name"]
+        domain_pddl = eval_context["domain_pddl"]
+        domain = Domain(" ", domain_pddl)
+
+        ## Parse trajectory, setup initial and goal state
+        # response = eval_context["gt_plan"]  # for debug
+        match response:
+            case str():
+                candidates = response.split("\n")
+            case tuple() | list():
+                candidates = list(response)
+            case _:
+                raise ValueError(
+                    f"`response` has unsupported type: {type(response)=}, {response=}"
+                )
+        cand_traj = [cand_a.strip() for cand_a in candidates if cand_a.startswith("(")]
+        try:
+            task_pddl = eval_context["task_pddl"]
+            cur_state = parse_pddl_attr_from_string(task_pddl, attr_starter="(:init")[1]
+            goal_state = parse_pddl_attr_from_string(task_pddl, attr_starter="(and")[1]
+        except IndexError:
+            score = 0
+            return score
+
+        score = 1
+        try:
+            ## State transitions and check if satisfy the preconditions
+            for cand_a in cand_traj:
+                param_to_obj, a_name = construct_param_to_obj(domain, cand_a)
+                if not check_pre_conds_satisfy(
+                    cur_state, domain.gt_cond_dict[f"{a_name}_pre"], param_to_obj
+                ):
+                    print(f"precondition of the action {cand_a} is not satisfied!")
+                    score = 0
+                    break
+                cur_state = state_transition(
+                    cur_state, domain.gt_cond_dict[f"{a_name}_post"], param_to_obj
+                )
+
+            ## Check if goal conditions are reached in the final state
+            if score == 1:
+                for g_state in goal_state:
+                    if (g_state.startswith("(not ") and g_state in cur_state) or (
+                        not g_state.startswith("(not ") and g_state not in cur_state
+                    ):
+                        print(f"goal state {g_state} is not reached!")
+                        score = 0
+                        break
+        except ValueError:
+            # grammar error in execution
+            score = 0
+        except AssertionError:
+            # assertion error in functions
+            score = 0
+
+        return score
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/unsupported_scoring.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/unsupported_scoring.py
new file mode 100644
index 0000000000000000000000000000000000000000..baebd51e48a2e78f85c34f595b83a87c6d417889
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/unsupported_scoring.py
@@ -0,0 +1,7 @@
+class UnsupportedScoring:
+    """Unsupported scoring."""
+
+    @staticmethod
+    def match(response: str, correct_answer: str) -> int:
+        """Default response for unimplemented metrics."""
+        return -1
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/vlm_as_judge.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/vlm_as_judge.py
new file mode 100644
index 0000000000000000000000000000000000000000..08764b5908c8b567f643be5788ecfd330db860a5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/vlm_as_judge.py
@@ -0,0 +1,237 @@
+import abc
+import base64
+import os
+import re
+from io import BytesIO
+from mimetypes import guess_type
+
+import requests
+from PIL import Image
+
+
+class OpenAIVLMJudger(abc.ABC):
+    """
+        The OpenAI model class for calling GPT4o or textonly gpt as the juedge
+    for open-ended generation tasks
+    """
+
+    def __init__(
+        self,
+        metric_config,
+        model="gpt-4o-2024-08-06",
+        resize=True,
+        max_side=1000,
+    ):
+        if metric_config is not None and metric_config != {}:
+            self.judge_model_type = metric_config["judge_model_type"]
+            self.eval_prompt = metric_config["eval_criteria_prompt"]
+            self.reference_type = metric_config["reference_type"]
+            self.template_mapping = metric_config["template_mapping"]
+
+        self.api_key = os.getenv("OPENAI_API_KEY")
+        self.url = os.getenv("OPENAI_API_BASE")
+        self.model = model
+        self.resize = resize
+        self.max_side = max_side
+
+        # if os.getenv("MEGABENCH_OPEN_API_KEY") is not None:
+        #     self.api_key = os.getenv("MEGABENCH_OPEN_API_KEY")
+        #     self.url = os.getenv("MEGABENCH_OPEN_API_URL")
+        #     if os.getenv("MEGABENCH_OPEN_API_MODEL") is not None:
+        #         self.model = os.getenv("MEGABENCH_OPEN_API_MODEL")
+        #     assert self.url, "You must set up the API URL for evaluating the Open tasks using your own API"
+
+    @staticmethod
+    def _update_image_path(image_path):
+        hf_home = os.getenv("HF_HOME", "~/.cache/huggingface")
+        base_cache_dir = os.path.expanduser(hf_home)
+        image_path = image_path.replace('./data/', f'{base_cache_dir}/megabench_data/data/')
+        return image_path
+
+    def create_image_content(self, image_path):
+        image_path = self._update_image_path(image_path)
+        base64_image, mime_type = self.encode_image(image_path)
+        return {
+            "type": "image_url",
+            "image_url": {"url": f"data:{mime_type};base64,{base64_image}"},
+        }
+
+    @property
+    def url(self) -> str:
+        """The server URL. We use OpenAI API by default. """
+        return self._url if hasattr(self, '_url') else "https://api.openai.com/v1/chat/completions"
+
+    @url.setter
+    def url(self, value: str) -> None:
+        """Set the server URL."""
+        self._url = value
+
+    @staticmethod
+    def _rgba_to_rgb(image):
+        background = Image.new("RGBA", image.size, (255, 255, 255, 255))
+        return Image.alpha_composite(background, image).convert("RGB")
+
+    def _resize_image(self, image):
+        resize_scale = self.max_side / max(image.size)
+        new_size = (
+            int(image.size[0] * resize_scale),
+            int(image.size[1] * resize_scale),
+        )
+        return image.resize(new_size)
+
+    def _encode_image(self, image, image_format):
+        with BytesIO() as output:
+            image.convert("RGB").save(output, format=image_format)
+            base64_encoded_data = base64.b64encode(output.getvalue()).decode("utf-8")
+        return base64_encoded_data
+
+    def encode_image(self, image_path, max_side=None):
+        mime_type, _ = guess_type(image_path)
+        if mime_type is None:
+            mime_type = "image/jpeg"
+        image_format = mime_type.split("/")[-1].upper() if mime_type else "JPEG"
+
+        image = Image.open(image_path)
+        # Handle the alpha channel
+        if image.mode == "RGBA":
+            image = self._rgba_to_rgb(image)
+        if not max_side and self.max_side:
+            max_side = self.max_side
+
+        if self.resize and max(image.size) > self.max_side:
+            image = self._resize_image(image)
+        encoded_image = self._encode_image(image, image_format)
+
+        return encoded_image, mime_type
+
+    def prepare_eval_prompt(
+        self, reference, response, images, question, eval_context=None
+    ):
+        content = []
+        if self.judge_model_type == "with image":
+            for image_path in images:
+                content.append(self.create_image_content(image_path))
+
+        prompt_mapping = {}
+        for key, val in self.template_mapping.items():
+            if val == "model_output":
+                prompt_mapping[key] = response
+            elif val == "example_question":
+                prompt_mapping[key] = question
+            elif val.split(".")[0] == "answers":
+                if isinstance(reference, str):
+                    prompt_mapping[key] = reference
+                else:
+                    # 如果是字典，则按原来的逻辑处理
+                    key_name = val.split(".")[1]
+                    prompt_mapping[key] = reference.get(key_name, '')
+            elif val.split(".")[0] == "eval_context":
+                key_name = val.split(".")[1]
+                prompt_mapping[key] = eval_context[key_name]
+
+        full_eval_prompt = self.eval_prompt.format(**prompt_mapping)
+
+        content.append({"type": "text", "text": full_eval_prompt})
+        return content
+
+    def query(self, reference_info, response, images, question, eval_context=None):
+        headers = {
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {self.api_key}",
+        }
+
+        context = self.prepare_eval_prompt(
+            reference_info, response, images, question, eval_context
+        )
+
+        query_payload = {
+            "model": self.model,
+            "messages": [{"role": "user", "content": context}],
+            "temperature": 0.0,
+        }
+
+        response_data = None
+        while response_data is None:
+            try:
+                response = requests.post(
+                    self.url,
+                    headers=headers,
+                    json=query_payload,
+                )
+                response_ = response.json()
+            except (requests.exceptions.JSONDecodeError, requests.exceptions.ConnectionError) as e:
+                print(f'Error in requests: {e}')
+                print('Retry...')
+                continue
+
+            if "error" in response_:
+                error_info = response_["error"]
+                print(
+                    f"Got error with type: {error_info['type']}. Message: {error_info['message']}"
+                )
+                if (
+                    error_info["message"]
+                    == """Sorry! We've encountered an issue with repetitive patterns in your prompt.
+                    Please try again with a different prompt."""
+                ):
+                    print(query_payload)
+                    # If the model's response has too many repetitive tokens, then we give it a score of 0.
+                    print("gpt-4o judge query failed...")
+                    return f"**Score explanation**: {error_info['message']}\n\n**Score**: 0"
+                print("Retry...")
+            else:
+                response_data = response_
+                break
+
+        total_tokens = response_data.get("usage", {}).get("total_tokens", "N/A")
+
+        # Extracting the 'content' field from the response
+        if response_data and "choices" in response_data:
+            choices = response_data["choices"]
+            if choices and "message" in choices[0]:
+                message_content = choices[0]["message"]["content"]
+                print(
+                    f"gpt-4o judge results: {message_content}; tokens:{total_tokens}"
+                )
+        else:
+            print("gpt-4o judge query failed...")
+            message_content = ""
+
+        return message_content
+
+
+class VLMJudgeScore:
+    """Using GPT-4o as a adjuge to evaluate open-ended generation tasks"""
+
+    def __init__(self, metric_config):
+        self.model = OpenAIVLMJudger(metric_config)
+
+    def parse_results(self, eval_results):
+        """
+        This parsing function is based on the output prompt setting in the
+        file "gpt4o_judge_prompt.json"
+        """
+        score_pattern = r"\*\*Score\*\*\s*:\s*(\d+)"
+        explanation_pattern = r"\*\*Score explanation\*\*\s*:\s*(.*)"
+
+        # Extract the score
+        score_match = re.search(score_pattern, eval_results)
+        score = int(score_match.group(1)) if score_match else None
+
+        # Extract the score explanation
+        explanation_match = re.search(explanation_pattern, eval_results, re.DOTALL)
+        explanation = explanation_match.group(1).strip() if explanation_match else ""
+        info_str = f"Score: {score}; Explanation: {explanation}"
+        if score is None:
+            return 0, f"Score is NULL: {eval_results};"
+
+        return score / 10.0, info_str
+
+    def match(
+        self, response, reference_dict, images, question, eval_context=None
+    ) -> int:
+        eval_results = self.model.query(
+            reference_dict, response, images, question, eval_context
+        )
+        score = self.parse_results(eval_results)
+        return score
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_nbbox_iou.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_nbbox_iou.py
new file mode 100644
index 0000000000000000000000000000000000000000..b1ed749242bca44bfb417aaac4d6e55ce0ca60a9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_nbbox_iou.py
@@ -0,0 +1,34 @@
+import logging
+from numbers import Number
+
+from .common.conversions import parse_bboxes_from_xml
+from .common.metrics import calculate_iou
+
+
+class XmlNbboxIouSingle:
+    """Calculates the IoU of bounding box.
+
+    Assumes that co-ordinates are normalized between 0 and 1 and that the bounding boxes
+    are of the form <box>top_left_x, top_left_y, bottom_right_x, bottom_right_y</box>
+    """
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+
+        logging.debug(f"{responses=}, {targets=}")
+        if not isinstance(responses, (tuple | list)):
+            responses = parse_bboxes_from_xml(responses)
+        if not isinstance(targets, (tuple | list)):
+            targets = parse_bboxes_from_xml(targets)
+
+        if len(responses) == 0:
+            return 0
+        elif isinstance(responses[0], Number) and len(responses) == 4:
+            responses = [responses]
+
+        iou_scores = calculate_iou(responses, targets)
+        if not iou_scores:
+            return 0
+
+        # Take the mean IoU score for now.
+        return sum(iou_scores) / len(iou_scores)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_norm_point_distance.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_norm_point_distance.py
new file mode 100644
index 0000000000000000000000000000000000000000..e189596c8205ec0556c5b3aeebdee42894f0a16a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_norm_point_distance.py
@@ -0,0 +1,37 @@
+"""Return the normalized point distance."""
+
+from .common.conversions import parse_point_2d_from_xml
+from .common.metrics import point_distance
+
+
+class XmlNormPointDistance:
+    """Determines the distance between two points in XML notation.
+
+    Assumes that co-ordinates are normalized between 0 and 1 and that the 2D point is
+    of the form <point>x, y</point>.
+    """
+
+    @classmethod
+    def parse_2d_point(cls, point) -> tuple[float, float]:
+        """Parse a 2D point encoded in XML as <point>x, y</point>."""
+        if not isinstance(point, (tuple | list)):
+            point = parse_point_2d_from_xml(point)
+            if not point:
+                raise ValueError("Point could not be parsed from XML string.")
+        elif len(point) != 2:
+            raise ValueError("Point is not 2D.")
+        if not all(0 <= comp <= 1 for comp in point):
+            raise ValueError("Point is not normalized.")
+        return tuple(point)
+
+    @classmethod
+    def match(cls, responses, targets) -> float:
+        """Determine the normalized distance between two points."""
+        try:
+            responses = cls.parse_2d_point(responses)
+            targets = cls.parse_2d_point(targets)
+        except ValueError:
+            return 0
+
+        # Instead of normalizing by 1/sqrt(2), we just set it to 0 if the distance is above 1.
+        return max(0, 1 - point_distance(responses, targets))
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_norm_point_in_bbox.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_norm_point_in_bbox.py
new file mode 100644
index 0000000000000000000000000000000000000000..929a2dcf4923718f1baba48941905aa126246cc9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/scoring/xml_norm_point_in_bbox.py
@@ -0,0 +1,35 @@
+from .common.conversions import parse_point_2d_from_xml, str_to_bboxes
+
+
+class XmlNormPointInBbox:
+    """Determines whether a point is located in a bounding box.
+
+    Assumes that co-ordinates are normalized between 0 and 1 and that the 2D point is
+    of the form <point>x, y</point>
+    """
+
+    @classmethod
+    def match(cls, responses, eval_context) -> int:
+        """Determine if the point is in the bounding box
+        and return which bounding box was matched, if any."""
+        bounding_box_has_match = {
+            bbox: False for bbox in eval_context["bounding_boxes"]
+        }
+        bounding_boxes = [
+            str_to_bboxes(bbox_str)[0] for bbox_str in eval_context["bounding_boxes"]
+        ]
+        assert bounding_boxes
+
+        if not isinstance(responses, (tuple | list)):
+            responses = parse_point_2d_from_xml(responses)
+            if not responses:
+                return 0, bounding_box_has_match
+        elif len(responses) != 2:
+            return 0, bounding_box_has_match
+
+        x, y = responses
+        for min_x, min_y, max_x, max_y in bounding_boxes:
+            if min_x <= x <= max_x and min_y <= y <= max_y:
+                bounding_box_has_match[str((min_x, min_y, max_x, max_y))] = True
+                return 1, bounding_box_has_match
+        return 0, bounding_box_has_match
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/analysis_utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/analysis_utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..0e34b9cfee3d216ddf4aefc02264eb20a507c035
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/analysis_utils.py
@@ -0,0 +1,182 @@
+import ast
+import json
+from collections import defaultdict
+from typing import Any, Dict, List
+
+_DATASET_CACHE = {}
+_SCORING_FUNCTIONS_CACHE = {}
+
+def _load_hf(subset_name: str) -> List[Dict[str, Any]]:
+    """
+    Load the HF dataset for the given subset name.
+    """
+    if subset_name in _DATASET_CACHE:
+        return _DATASET_CACHE[subset_name]
+    
+    from datasets import load_dataset
+    dataset = load_dataset("TIGER-Lab/MEGA-Bench", subset_name)["test"]
+    task_dict = {}
+    for sample in dataset:
+        task_name = sample["task_name"]
+        if task_name not in task_dict:
+            task_dict[task_name] = []
+        task_dict[task_name].append(sample)
+
+    _DATASET_CACHE[subset_name] = task_dict
+    return task_dict
+
+
+def _get_scoring_functions():
+    if _SCORING_FUNCTIONS_CACHE:
+        return _SCORING_FUNCTIONS_CACHE
+    
+    core_data = _load_hf("core")
+    open_data = _load_hf("open")
+    
+    core_scoring_functions = {}
+    open_scoring_functions = {}
+    
+    for task_name, task_samples in core_data.items():
+        core_scoring_functions[task_name] = ast.literal_eval(
+            task_samples[0]["metric_info"]
+        )
+    
+    for task_name, task_samples in open_data.items():
+        open_scoring_functions[task_name] = ast.literal_eval(
+            task_samples[0]["metric_info"]
+        )
+    
+    _SCORING_FUNCTIONS_CACHE["core"] = core_scoring_functions
+    _SCORING_FUNCTIONS_CACHE["open"] = open_scoring_functions
+    
+    return _SCORING_FUNCTIONS_CACHE
+
+
+def _determine_eval_style(task):
+    """
+    Determine the evaluation style (rule or llm) for a task.
+    """
+    scoring_functions = _get_scoring_functions()
+    core_scoring_functions = scoring_functions["core"]
+    open_scoring_functions = scoring_functions["open"]
+    
+    task_name = task["task_name"]
+    if task_name in core_scoring_functions:
+        metric_info = core_scoring_functions[task_name]
+    elif task_name in open_scoring_functions:
+        metric_info = open_scoring_functions[task_name]
+    else:
+        raise ValueError(f"Task '{task_name}' not found in either core or open datasets")
+    
+    all_task_metrics = list(metric_info["field_score_function"].values())
+    eval_type = (
+        "rule"
+        if (
+            "gpt_4o_as_judge" not in all_task_metrics
+            and "ascii_art_gpt4o_judge" not in all_task_metrics
+        )
+        else "llm"
+    )
+    return eval_type
+
+
+def clear_cache():
+    """
+    Clear the cache and force re-loading the dataset.
+    """
+    global _DATASET_CACHE, _SCORING_FUNCTIONS_CACHE
+    _DATASET_CACHE.clear()
+    _SCORING_FUNCTIONS_CACHE.clear()
+
+
+def task_list_refine(task_list):
+    task_results = []
+    for task in task_list:
+        if "mean_task_score" in task and task["mean_task_score"] != -1:
+            num_demo = 1 if len(task["example_contents"]) > 0 else 0
+            task_results.append(
+                {
+                    "name": task["task_name"],
+                    "score": task["mean_task_score"],
+                    "eval_type": task.get("eval_type", _determine_eval_style(task)),
+                    "num_demo": num_demo,
+                    "num_query": len(task["query_response"]),
+                }
+            )
+    return task_results
+
+
+def derive_keyword_stats(task_results_with_meta, include_per_task_info=False):
+    """
+    Calculate keyword-based statistics for skills, input_format, and output_format.
+    """
+    skills_stats = defaultdict(lambda: {"count": 0, "total_score": 0.0, "num_samples": 0, "tasks": []})
+    input_format_stats = defaultdict(lambda: {"count": 0, "total_score": 0.0, "num_samples": 0, "tasks": []})
+    output_format_stats = defaultdict(lambda: {"count": 0, "total_score": 0.0, "num_samples": 0, "tasks": []})
+    input_num_stats = defaultdict(lambda: {"count": 0, "total_score": 0.0, "num_samples": 0, "tasks": []})
+    app_stats = defaultdict(lambda: {"count": 0, "total_score": 0.0, "num_samples": 0, "tasks": []})
+
+    for task_name, task in task_results_with_meta.items():
+        task_name = task.get("original_task_name", "Unknown Task")
+        score = task.get("score", 0.0)
+        num_samples = task.get("num_query", 0) + task.get("num_demo", 0)
+
+        if score == -1:
+            continue
+
+        for skill in task.get("skills", []):
+            skills_stats[skill]["count"] += 1
+            skills_stats[skill]["total_score"] += score
+            skills_stats[skill]["num_samples"] += num_samples
+            if include_per_task_info:
+                skills_stats[skill]["tasks"].append((task_name, score))
+
+        for stat_dict, key in [
+            (input_format_stats, "input_format"),
+            (output_format_stats, "output_format"),
+            (input_num_stats, "num_input"),
+            (app_stats, "app")
+        ]:
+            if value := task.get(key):
+                stat_dict[value]["count"] += 1
+                stat_dict[value]["total_score"] += score
+                stat_dict[value]["num_samples"] += num_samples
+                if include_per_task_info:
+                    stat_dict[value]["tasks"].append((task_name, score))
+
+    all_stats = {
+        "skills": skills_stats,
+        "input_format": input_format_stats,
+        "output_format": output_format_stats,
+        "input_num": input_num_stats,
+        "app": app_stats,
+    }
+
+    for stats_dict in all_stats.values():
+        for keyword, data in stats_dict.items():
+            data["average_score"] = data["total_score"] / data["count"] if data["count"] > 0 else 0.0
+            del data["total_score"]
+
+    return dict(all_stats)
+
+
+def collect_task_metadata(model_results, all_task_meta_path):
+    """
+    Collect task metadata for a model's results using the all_task_meta.json file
+    """
+    # Load the complete task metadata
+    with open(all_task_meta_path, "r") as f:
+        all_meta = json.load(f)
+    
+    # Create result dictionary
+    all_task_meta = {}
+    
+    # Match results with metadata
+    for task_result in model_results:
+        task_name = task_result["name"]
+        if task_name in all_meta:
+            meta = all_meta[task_name].copy()  # Create a copy to avoid modifying original
+            meta.update(task_result)
+            all_task_meta[task_name] = meta
+    
+    return all_task_meta
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/derive_breakdown_results.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/derive_breakdown_results.py
new file mode 100644
index 0000000000000000000000000000000000000000..1a68be6fd102b37e5519e5cea18f9c6307ff8182
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/derive_breakdown_results.py
@@ -0,0 +1,139 @@
+import argparse
+import json
+from pathlib import Path
+
+from analysis_utils import collect_task_metadata, derive_keyword_stats, task_list_refine
+
+
+def calculate_model_summary(task_results_with_meta):
+    """
+    Re-calculate model performance summary statistics across core and open tasks.
+    
+    Args:
+        task_results: List of task results with scores
+        task_metadata: Dictionary containing task metadata including task types
+    
+    Returns:
+        Dictionary containing summary statistics for core and open tasks
+    """
+    core_tasks = []
+    open_tasks = []
+
+    # Separate core and open tasks
+    for task in task_results_with_meta.values():
+        if task['eval_type'] == 'llm':
+            open_tasks.append(task)
+        else:
+            core_tasks.append(task)
+    
+    def calculate_stats(tasks):
+        if not tasks:
+            return None
+        
+        total_samples = sum(task.get('num_query', 0) for task in tasks)
+        macro_scores = [task.get('score', 0) for task in tasks]
+        
+        return {
+            "num_eval_tasks": len(tasks),
+            "num_eval_samples": total_samples,
+            "macro_mean_score": sum(macro_scores) / len(tasks) if tasks else 0,
+        }
+    
+    core_stats = calculate_stats(core_tasks)
+    open_stats = calculate_stats(open_tasks)
+    
+    # Calculate overall score (weighted average based on number of tasks)
+    total_tasks = (core_stats["num_eval_tasks"] + open_stats["num_eval_tasks"])
+    overall_score = (
+        (core_stats["macro_mean_score"] * core_stats["num_eval_tasks"] + 
+         open_stats["macro_mean_score"] * open_stats["num_eval_tasks"]) / total_tasks
+        if core_stats and open_stats
+        else 0
+    )
+    
+    return {
+        "core": core_stats,
+        "open": open_stats,
+        "overall_score": overall_score
+    }
+
+def merge_json_files(input_dir, output_path, key="name"):
+    """
+    Merge multiple JSON files containing evaluation results from a directory.
+    Looks for all files matching pattern 'data_with_scores*.json'.
+    Prioritizes LLM evaluations over rule-based ones when duplicates exist.
+    """
+    data_dict = {}  # Using name as key for easy lookup and updates
+    
+    # Find all matching JSON files in the directory
+    json_paths = list(Path(input_dir).glob("megabench_score*.json"))
+    print(f"Found {len(json_paths)} files to merge")
+    
+    # Load and merge all JSON files
+    for path in json_paths:
+        print(f"Processing {path}")
+        with open(path, "r") as f:
+            data = json.load(f)
+            if isinstance(data, dict) and "data" in data:
+                data = task_list_refine(data["data"])
+            
+            # Update or add entries
+            for item in data:
+                item_key = item[key]
+                # If new item or if new item is LLM-evaluated (prioritize LLM eval)
+                if item_key not in data_dict or (
+                    item.get("eval_type") == "llm" and data_dict[item_key].get("eval_type") != "llm"
+                ):
+                    data_dict[item_key] = item
+
+    # Convert back to list
+    merged_data = list(data_dict.values())
+    
+    # Save the merged result
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    with open(output_path, "w") as f:
+        json.dump(merged_data, f, indent=4)
+    
+    print(f"Merged file with {len(merged_data)} tasks saved to {output_path}")
+    return merged_data
+
+def main():
+    # Parse command line arguments
+    parser = argparse.ArgumentParser(description='Merge and process evaluation score files.')
+    parser.add_argument('--input_dir', type=str, help='Directory containing score files')
+    args = parser.parse_args()
+
+    # Convert path to Path object
+    input_dir = Path(args.input_dir)
+    
+    # Create analysis directory under input directory
+    output_dir = input_dir / "analysis"
+    output_dir.mkdir(parents=True, exist_ok=True)
+    
+    # Merge files
+    output_path = output_dir / "task_results.json"
+    task_results = merge_json_files(input_dir, output_path)
+    
+    # Collect metadata and derive keyword stats
+    task_results_with_meta = collect_task_metadata(task_results, all_task_meta_path="all_task_meta.json")
+    keyword_stats = derive_keyword_stats(task_results_with_meta)
+    
+    # Calculate model summary
+    model_summary = calculate_model_summary(task_results_with_meta)
+
+    summary_results = {
+        "model_summary": model_summary,
+        "keyword_stats": keyword_stats
+    }
+    
+    # Save keyword stats
+    stats_output = output_dir / "summary_and_keyword_stats.json"
+    with open(stats_output, "w") as f:
+        json.dump(summary_results, f, indent=4)
+    
+    print(f"\nResults saved in {output_dir}:")
+    print(f"- Merged data: {output_path}")
+    print(f"- Multi-dimensional keywords stats: {stats_output}")
+
+if __name__ == "__main__":
+    main()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/get_si_subset_from_full.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/get_si_subset_from_full.py
new file mode 100644
index 0000000000000000000000000000000000000000..f2bdd265d4d05a5b8608971b5b71a88bd350bec5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/tools/get_si_subset_from_full.py
@@ -0,0 +1,90 @@
+"""
+For propietary models that naturally suport multi-image or video inputs, we don't run the single-image setting, 
+instead, we directly compute the SI results by extracting the subset results from the full task set to compute the stats.
+"""
+
+
+import argparse
+import json
+from pathlib import Path
+
+from analysis_utils import collect_task_metadata, derive_keyword_stats
+from derive_breakdown_results import calculate_model_summary
+
+
+def process_subset_results(input_dir, eval_type):
+    """Read results from the full results directory structure"""
+    task_results_path = input_dir / "analysis" / "task_results.json"
+    
+    # Load task results
+    with open(task_results_path, "r") as f:
+        task_results = json.load(f)
+    
+    results_with_meta = collect_task_metadata(task_results, all_task_meta_path="all_task_meta.json")
+    
+    # Filter tasks by eval_type
+    filtered_results_with_meta = {task_name:task for task_name, task in results_with_meta.items() if task["eval_type"] == eval_type and task["num_input"] == "1-image"}
+    filtered_results = [task for task in task_results if task["name"] in filtered_results_with_meta]
+    
+    if not filtered_results:
+        print(f"Warning: No tasks found in {input_dir} with eval_type {eval_type}")
+        return None, None, None
+    
+    # Calculate summary statistics
+    num_tasks = len(filtered_results)
+    total_queries = sum(task["num_query"] for task in filtered_results)
+    total_correct = sum(round(task["score"] * task["num_query"]) for task in filtered_results)
+    
+    summary = {
+        "num_eval_tasks": num_tasks,
+        "num_eval_samples": total_queries,
+        "macro_mean_score": sum(task["score"] for task in filtered_results) / num_tasks,
+    }
+    
+    return filtered_results, filtered_results_with_meta, summary
+
+
+def main(input_dir, output_dir):
+    # Process core and open set results
+    filtered_tasks_core, filtered_tasks_core_with_meta, _ = process_subset_results(input_dir, "rule")
+    filtered_tasks_open, filtered_tasks_open_with_meta, _ = process_subset_results(input_dir, "llm")
+
+    if filtered_tasks_core and filtered_tasks_open:
+        task_results = filtered_tasks_core + filtered_tasks_open
+        task_results_with_meta = {**filtered_tasks_core_with_meta, **filtered_tasks_open_with_meta}
+
+        # Save task results
+        with open(output_dir / "task_results.json", "w") as f:
+            json.dump(task_results, f, indent=4)
+
+        # Collect metadata and derive keyword stats
+        keyword_stats = derive_keyword_stats(task_results_with_meta)
+
+        # Calculate model summary
+        model_summary = calculate_model_summary(task_results_with_meta)
+
+        summary_results = {
+            "model_summary": model_summary,
+            "keyword_stats": keyword_stats
+        }
+    
+        # Save keyword stats
+        stats_output = output_dir / "summary_and_keyword_stats.json"
+        with open(stats_output, "w") as f:
+            json.dump(summary_results, f, indent=4)
+    
+        print(f"\nResults saved in {output_dir}")
+
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input_dir", type=str, required=True, help="Path to the input directory containing full results")
+    parser.add_argument("--output_dir", type=str, required=True, help="Path to the output directory")
+    args = parser.parse_args()
+
+    input_dir = Path(args.input_dir)
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    main(input_dir, output_dir)
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..8c32d27d6bf6c3624d2dc2e8bf32250bbb38ba95
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/megabench/utils.py
@@ -0,0 +1,69 @@
+import importlib
+from mimetypes import guess_type
+
+
+def lazy_import(module_name, class_name):
+    """Import the module lazily."""
+
+    def importer():
+        module = importlib.import_module(module_name)
+        return getattr(module, class_name)
+
+    return importer
+
+
+def is_video_file(file_path):
+    mime_type, _ = guess_type(file_path)
+    if not mime_type:
+        return False
+    return mime_type.startswith("video")
+
+
+
+def prepare_megabench_data(dataset_name, dataset_subset_name):
+    """
+        Prepare the MEGA-Bench dataset for evaluation.
+        Return:
+            subset_dataset: The organized data of the specified subset
+            all_dataset: The organized data of all tasks, used for evaluation
+    """
+    from datasets import load_dataset
+    if "single_image" in dataset_subset_name:
+        core_data = load_dataset(dataset_name, "core_single_image")
+        open_data = load_dataset(dataset_name, "open_single_image")
+    else:
+        core_data = load_dataset(dataset_name, "core")
+        open_data = load_dataset(dataset_name, "open")
+    core_test_samples = list(core_data["test"])
+    organized_core_dataset = organize_hf_dataset(core_test_samples)
+    open_test_samples = list(open_data["test"])
+    organized_open_dataset = organize_hf_dataset(open_test_samples)
+    subset_dataset = organized_core_dataset if "core" in dataset_subset_name else organized_open_dataset
+    all_dataset = organized_core_dataset + organized_open_dataset
+    return subset_dataset, all_dataset
+
+
+def organize_hf_dataset(dataset):
+    """
+    Organize the dataset with task-based manner
+
+    Return:
+        organized_dataset: list, each item is a dict, with the following keys:
+            - task_name: str
+            - task_query_samples: list of dicts, each dict contains the sample information
+    """
+    task_dict = {}
+    for sample in dataset:
+        task_name = sample["task_name"]
+        if task_name not in task_dict:
+            task_dict[task_name] = []
+        task_dict[task_name].append(sample)
+
+    organized_dataset = []
+    for task_name, samples in task_dict.items():
+        organized_dataset.append({
+            "task_name": task_name,
+            "task_samples": samples
+        })
+
+    return organized_dataset
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..2322577cea1218e51a9c4a88a502081af051ae28
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluator.py
@@ -0,0 +1,51 @@
+import re
+from typing import Any
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        # Extract content within <answer></answer> tags
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+        if match:
+            return match.group(1).strip()
+        return model_output.strip()  # Fallback to full output if no tags found
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        raise NotImplementedError
+
+
+class SimpleStrMatch(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        return question
+
+    def evaluate(self, predicted_answer: str, ground_truth: str, initial_state: Any) -> bool:
+        clean_answer = re.sub(r'\s+', '', str(ground_truth).lower())
+        clean_response = re.sub(r'\s+', '', str(predicted_answer).lower())
+        return clean_answer == clean_response
+
+
+class MatchFromList(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        return question
+
+    def evaluate(self, predicted_answer: str, ground_truth: list, initial_state: Any) -> bool:
+        """
+        Check if the response matches any of the multiple possible answers
+        """
+        if not ground_truth:
+            return False
+
+        # Clean the response text
+        clean_response = re.sub(r'\s+', '', str(predicted_answer).lower())
+
+        # Check if it matches any of the answers
+        for answer in ground_truth:
+            clean_answer = re.sub(r'\s+', '', str(answer).lower())
+            if clean_answer == clean_response:
+                return True
+
+        return False
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/aquarium_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/aquarium_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..7e19a8435ddcafddde0aebc9079279eb556fe52c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/aquarium_eval.py
@@ -0,0 +1,526 @@
+import json
+import re
+from typing import Any, Dict, List, Optional, Union
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str,
+                 params: Dict[str, Any] = None) -> bool:
+        raise NotImplementedError
+
+
+class AquariumEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        return question
+
+    def extract_answer(self, model_output: str) -> Optional[List[tuple]]:
+        """
+        Extract the model's answer from the output string with enhanced robustness.
+
+        Supports multiple formats:
+        - Answer blocks: [answer][(1,2), (3,4)][/answer]
+        - Direct lists: [(1,2), (3,4)]
+        - Empty lists: []
+        - Scattered coordinates throughout text
+        - Various bracket styles and spacing patterns
+        """
+
+        if not isinstance(model_output, str):
+            return None
+
+        # Clean and normalize the input
+        normalized_output = self._normalize_text(model_output)
+
+        # Strategy 0: Check for explicit empty list (highest priority)
+        if self._is_empty_list(normalized_output):
+            return []
+
+        # Strategy 1: Find answer blocks first (highest priority)
+        coordinates = self._extract_from_answer_blocks(normalized_output)
+        if coordinates is not None:
+            return coordinates
+
+        # Strategy 2: Parse standard list formats
+        coordinates = self._parse_list_format(normalized_output)
+        if coordinates is not None:
+            return coordinates
+
+        # Strategy 3: Find scattered coordinate pairs
+        coordinates = self._parse_scattered_coordinates(normalized_output)
+        if coordinates is not None:
+            return coordinates
+
+        # Strategy 4: Parse comma-separated values
+        coordinates = self._parse_csv_format(normalized_output)
+        if coordinates is not None:
+            return coordinates
+
+        # Strategy 5: Parse structured text patterns
+        coordinates = self._parse_structured_text(normalized_output)
+        if coordinates is not None:
+            return coordinates
+
+        return None
+
+    def _normalize_text(self, text: str) -> str:
+        """Normalize text for better parsing."""
+        # Remove excessive whitespace
+        text = re.sub(r'\s+', ' ', text.strip())
+        # Normalize common variations
+        text = text.replace('，', ',')  # Chinese comma
+        text = text.replace('（', '(').replace('）', ')')  # Chinese parentheses
+        text = text.replace('【', '[').replace('】', ']')  # Chinese brackets
+        return text
+
+    def _is_empty_list(self, text: str) -> bool:
+        """Check if the text explicitly represents an empty list."""
+        # Clean the text and check for empty list patterns
+        _ = re.sub(r'\s+', '', text.lower())
+        empty_patterns = [
+            r'^\[\]$',  # Exact empty list
+            r'\[\s*\]',  # Empty list with spaces
+            r'answer:\s*\[\]',  # Answer: []
+            r'\[answer\]\[\]\[/answer\]',  # [answer][][/answer]
+        ]
+
+        for pattern in empty_patterns:
+            if re.search(pattern, text, re.IGNORECASE):
+                return True
+        return False
+
+    def _extract_from_answer_blocks(self, text: str) -> Optional[List[tuple]]:
+        """Extract coordinates from answer blocks like [answer]...[/answer]."""
+        try:
+            # Multiple patterns for answer blocks
+            patterns = [
+                r'\[answer\](.*?)\[/answer\]',
+                r'<answer>(.*?)</answer>',
+                r'Answer:\s*(.*?)(?:\n|$)',
+                r'答案[：:]\s*(.*?)(?:\n|$)',
+                r'Final answer:\s*(.*?)(?:\n|$)'
+            ]
+
+            for pattern in patterns:
+                matches = re.findall(pattern, text, re.DOTALL | re.IGNORECASE)
+                if matches:
+                    # Use the last match (most likely the final answer)
+                    last_match = matches[-1].strip()
+                    coordinates = self._parse_coordinates_from_text(last_match)
+                    if coordinates:
+                        return coordinates
+
+        except Exception:
+            pass
+
+        return None
+
+    def _parse_list_format(self, text: str) -> Optional[List[tuple]]:
+        """Parse coordinates in standard list format: [(1, 0), (2, 1), ...] or similar."""
+        try:
+            # Look for list-like structures with various bracket types
+            list_patterns = [
+                r'\[([^\[\]]*(?:\([^)]*\)[^\[\]]*)*)\]',  # Square brackets with parentheses inside
+                r'\{([^\{\}]*(?:\([^)]*\)[^\{\}]*)*)\}',  # Curly brackets
+                r'list\s*[:\=]\s*\[([^\[\]]*)\]',  # "list: [...]"
+                r'coordinates?\s*[:\=]\s*\[([^\[\]]*)\]',  # "coordinates: [...]"
+            ]
+            if len(text) > 500:
+                text = text[-100:]
+            for pattern in list_patterns:
+                matches = re.findall(pattern, text, re.IGNORECASE)
+                for match in matches:
+                    coordinates = self._parse_coordinates_from_text(match)
+                    if coordinates and len(coordinates) > 0:
+                        return coordinates
+
+        except Exception:
+            pass
+
+        return None
+
+    def _parse_scattered_coordinates(self, text: str) -> Optional[List[tuple]]:
+        """Parse coordinates scattered throughout text."""
+        try:
+            # Enhanced patterns for coordinate detection
+            patterns = [
+                r'\((\d+)\s*,\s*(\d+)\)',          # (x, y)
+                r'\((\d+)\s+(\d+)\)',              # (x y)
+                r'(\d+)\s*,\s*(\d+)',              # x, y
+                r'cell\s*\((\d+),\s*(\d+)\)',      # cell(x, y)
+                r'position\s*\((\d+),\s*(\d+)\)',  # position(x, y)
+                r'(\d+)-(\d+)',                    # x-y format
+            ]
+
+            all_coordinates = []
+            for pattern in patterns:
+                matches = re.findall(pattern, text, re.IGNORECASE)
+                for x_str, y_str in matches:
+                    try:
+                        x, y = int(x_str), int(y_str)
+                        # Validation with reasonable bounds
+                        if 0 <= x <= 50 and 0 <= y <= 50:
+                            all_coordinates.append((x, y))
+                    except ValueError:
+                        continue
+
+            # Remove duplicates while preserving order
+            unique_coordinates = self._remove_duplicates(all_coordinates)
+            return unique_coordinates if unique_coordinates else None
+
+        except Exception:
+            pass
+
+        return None
+
+    def _parse_csv_format(self, text: str) -> Optional[List[tuple]]:
+        """Parse coordinates in CSV-like format: x1,y1,x2,y2,... or x1 y1 x2 y2..."""
+        try:
+            # Extract sequences of numbers
+            number_sequences = [
+                re.findall(r'\d+', line) for line in text.split('\n')
+                if re.search(r'\d+', line)
+            ]
+
+            for numbers in number_sequences:
+                if len(numbers) >= 2 and len(numbers) % 2 == 0:
+                    coordinates = []
+                    for i in range(0, len(numbers), 2):
+                        x, y = int(numbers[i]), int(numbers[i + 1])
+                        if 0 <= x <= 50 and 0 <= y <= 50:
+                            coordinates.append((x, y))
+
+                    if coordinates:
+                        return coordinates
+
+        except Exception:
+            pass
+
+        return None
+
+    def _parse_structured_text(self, text: str) -> Optional[List[tuple]]:
+        """Parse coordinates from structured text descriptions."""
+        try:
+            # Look for patterns like "fill cells at (1,2), (3,4)"
+            patterns = [
+                r'(?:fill|water|cells?)\s+(?:at|in)?\s*[:\s]*([^\n.!?]*(?:\(\d+,\s*\d+\)[^\n.!?]*)+)',
+                r'(?:solution|answer)(?:\s+is)?[:\s]*([^\n.!?]*(?:\(\d+,\s*\d+\)[^\n.!?]*)+)',
+                r'(?:coordinates?|positions?)[:\s]*([^\n.!?]*(?:\(\d+,\s*\d+\)[^\n.!?]*)+)',
+            ]
+
+            for pattern in patterns:
+                matches = re.findall(pattern, text, re.IGNORECASE)
+                for match in matches:
+                    coordinates = self._parse_coordinates_from_text(match)
+                    if coordinates:
+                        return coordinates
+
+        except Exception:
+            pass
+
+        return None
+
+    def _parse_coordinates_from_text(self, text: str) -> Optional[List[tuple]]:
+        """Extract coordinate pairs from a text string."""
+        try:
+            # Multiple patterns to catch different coordinate formats
+            coordinate_patterns = [
+                r'\((\d+)\s*,\s*(\d+)\)',      # (x, y)
+                r'\((\d+)\s+(\d+)\)',          # (x y)
+                r'(\d+)\s*,\s*(\d+)',          # x, y
+                r'(\d+)\s*-\s*(\d+)',          # x-y
+            ]
+
+            coordinates = []
+            for pattern in coordinate_patterns:
+                matches = re.findall(pattern, text)
+                for x_str, y_str in matches:
+                    try:
+                        x, y = int(x_str), int(y_str)
+                        if 0 <= x <= 50 and 0 <= y <= 50:
+                            coordinates.append((x, y))
+                    except ValueError:
+                        continue
+
+            return self._remove_duplicates(coordinates) if coordinates else None
+
+        except Exception:
+            pass
+
+        return None
+
+    def _remove_duplicates(self, coordinates: List[tuple]) -> List[tuple]:
+        """Remove duplicates while preserving order."""
+        seen = set()
+        unique_coordinates = []
+        for coord in coordinates:
+            if coord not in seen:
+                seen.add(coord)
+                unique_coordinates.append(coord)
+        return unique_coordinates
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Union[str, dict],
+                 params: Dict[str, Any] = None) -> bool:
+        """
+        Evaluate if the predicted answer is correct based SOLELY on aquarium game rules and initial_state.
+
+        Args:
+            predicted_answer: Model's predicted answer (can be string or list of coordinates)
+            ground_truth: Ground truth answer (IGNORED - not used in evaluation)
+            initial_state: JSON string or dict containing puzzle state (regions, clues, grid_size)
+            params: Additional parameters (optional)
+
+        Returns:
+            bool: True if the predicted answer satisfies all game rules, False otherwise
+
+        Note: This function deliberately ignores ground_truth and evaluates purely based on
+              game rules and initial_state to test logical consistency.
+        """
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                try:
+                    state_data = json.loads(initial_state)
+                except json.JSONDecodeError:
+                    return False
+            elif isinstance(initial_state, dict):
+                state_data = initial_state
+            else:
+                return False
+
+            # Extract required components
+            required_keys = ['regions', 'row_clues', 'col_clues', 'grid_size']
+            for key in required_keys:
+                if key not in state_data:
+                    return False
+
+            regions = state_data['regions']
+            row_clues = state_data['row_clues']
+            col_clues = state_data['col_clues']
+            grid_rows, grid_cols = state_data['grid_size']
+
+        except (KeyError, TypeError, ValueError):
+            return False
+
+        # Extract and validate predicted answer
+        if isinstance(predicted_answer, str):
+            extracted_answer = self.extract_answer(predicted_answer)
+            if extracted_answer is None:
+                return False
+            predicted_coordinates = extracted_answer
+        elif isinstance(predicted_answer, list):
+            predicted_coordinates = predicted_answer
+        else:
+            return False
+
+        # Validate coordinate format
+        if not self._validate_coordinate_format(predicted_coordinates):
+            return False
+
+        # Validate coordinates are within grid bounds
+        if not self._validate_coordinate_bounds(predicted_coordinates, grid_rows, grid_cols):
+            return False
+
+        # Create solution grid from predicted coordinates
+        solution_grid = self._create_solution_grid(predicted_coordinates, grid_rows, grid_cols)
+
+        # Validate all game rules
+        if not self._validate_row_clues(solution_grid, row_clues, grid_rows,
+                                        grid_cols):
+            return False
+
+        if not self._validate_column_clues(solution_grid, col_clues, grid_rows,
+                                           grid_cols):
+            return False
+
+        if not self._validate_aquarium_rules(solution_grid, regions, grid_rows,
+                                             grid_cols):
+            return False
+
+        return True
+
+    def _validate_coordinate_format(self, coordinates: Any) -> bool:
+        """Validate that coordinates is a list of tuples with two integers each."""
+        if not isinstance(coordinates, list):
+            return False
+
+        if len(coordinates) == 0:
+            return True  # Empty list is valid (no water)
+
+        for i, item in enumerate(coordinates):
+            if not isinstance(item, (tuple, list)) or len(item) != 2:
+                return False
+            try:
+                x, y = item
+                int(x), int(y)  # Ensure they can be converted to int
+            except (ValueError, TypeError):
+                return False
+
+        return True
+
+    def _validate_coordinate_bounds(self, coordinates: List[tuple], grid_rows: int, grid_cols: int) -> bool:
+        """Validate that all coordinates are within grid bounds."""
+        for x, y in coordinates:
+            if not (0 <= x < grid_cols and 0 <= y < grid_rows):
+                return False
+        return True
+
+    def _create_solution_grid(self, coordinates: List[tuple], grid_rows: int, grid_cols: int) -> List[List[bool]]:
+        """Create a 2D boolean grid from coordinate list."""
+        if grid_rows == 0 or grid_cols == 0:
+            return []
+        solution_grid = [[False for _ in range(grid_cols)] for _ in range(grid_rows)]
+        for x, y in coordinates:
+            solution_grid[y][x] = True
+        return solution_grid
+
+    def _validate_row_clues(self, solution_grid: List[List[bool]], row_clues: List[int],
+                            grid_rows: int, grid_cols: int) -> bool:
+        """Validate that row clues match the solution."""
+        if grid_rows == 0:
+            return len(row_clues) == 0
+        for row_idx in range(grid_rows):
+            filled_count = sum(1 for col_idx in range(grid_cols) if solution_grid[row_idx][col_idx])
+            expected_count = row_clues[row_idx]
+            if filled_count != expected_count:
+                return False
+        return True
+
+    def _validate_column_clues(self, solution_grid: List[List[bool]],
+                               col_clues: List[int],
+                               grid_rows: int, grid_cols: int) -> bool:
+        """Validate that column clues match the solution."""
+        if grid_cols == 0:
+            return len(col_clues) == 0
+        for col_idx in range(grid_cols):
+            filled_count = sum(1 for row_idx in range(grid_rows) if solution_grid[row_idx][col_idx])
+            expected_count = col_clues[col_idx]
+            if filled_count != expected_count:
+                return False
+        return True
+
+    def _validate_aquarium_rules(self, solution_grid: List[List[bool]],
+                                 regions: List[List[int]],
+                                 grid_rows: int, grid_cols: int) -> bool:
+        """
+        Verify aquarium game rules:
+        1. Each region must be filled to a uniform water level (from bottom up)
+        2. Water cannot float - if a cell is filled, the cell directly below it
+           (if any, in same region) must also be filled
+        """
+
+        if grid_rows == 0 or grid_cols == 0:
+            return len(regions) == 0
+
+        # Get all unique region IDs
+        unique_regions = set()
+        for row in regions:
+            for cell in row:
+                unique_regions.add(cell)
+
+        # For each region, verify aquarium rules
+        for region_id in unique_regions:
+            if not self._check_region_rules(region_id, solution_grid, regions, grid_rows, grid_cols):
+                return False
+
+        return True
+
+    def _check_region_rules(self, region_id: int, solution_grid: List[List[bool]],
+                            regions: List[List[int]], grid_rows: int,
+                            grid_cols: int) -> bool:
+        """Check aquarium rules for a specific region."""
+
+        # Get all cells in this region
+        region_cells = []
+        for row_idx in range(grid_rows):
+            for col_idx in range(grid_cols):
+                if regions[row_idx][col_idx] == region_id:
+                    region_cells.append((col_idx, row_idx))  # (x, y) format
+
+        if not region_cells:
+            return True  # Empty region is valid
+
+        # Rule 1: Water cannot float - check gravity rule
+        if not self._check_gravity_rule(region_cells, solution_grid, regions,
+                                        region_id, grid_rows, grid_cols):
+            return False
+
+        # Rule 2: Uniform water level - group by column and check water levels
+        if not self._check_uniform_water_level(region_cells, solution_grid, region_id):
+            return False
+
+        return True
+
+    def _check_gravity_rule(self, region_cells: List[tuple],
+                            solution_grid: List[List[bool]],
+                            regions: List[List[int]], region_id: int,
+                            grid_rows: int, grid_cols: int) -> bool:
+        """Check that water cannot float: if a cell is filled, the cell directly
+        below it (if any, in same region) must also be filled."""
+
+        for x, y in region_cells:
+            if solution_grid[y][x]:  # If this cell is filled
+                # Check the cell directly below it
+                below_y = y + 1
+                if below_y < grid_rows:  # If there is a cell below
+                    # Check if the cell below is in the same region
+                    if regions[below_y][x] == region_id:
+                        # The cell below must also be filled
+                        if not solution_grid[below_y][x]:
+                            return False
+
+        return True
+
+    def _check_uniform_water_level(self, region_cells: List[tuple],
+                                   solution_grid: List[List[bool]],
+                                   region_id: int) -> bool:
+        """
+        Check that each region has uniform water level.
+
+        Rule: In the same region, if there's water at a certain height (y coordinate),
+        then ALL cells in that region at that height should have water.
+        This simulates a real aquarium where water surface is horizontal.
+        """
+
+        # Group cells by y coordinate (height level)
+        levels = {}
+        for x, y in region_cells:
+            if y not in levels:
+                levels[y] = []
+            levels[y].append(x)
+
+        # For each level, check if water is consistent
+        for y_level, x_coords in levels.items():
+            # Check how many cells at this level have water
+            filled_count = sum(1 for x in x_coords if solution_grid[y_level][x])
+
+            # Either all cells at this level should have water, or none should
+            if 0 < filled_count < len(x_coords):
+                return False
+
+        # Also check that water levels are contiguous (from bottom up)
+        # Find all levels that have water
+        water_levels = []
+        for y_level, x_coords in levels.items():
+            if any(solution_grid[y_level][x] for x in x_coords):
+                water_levels.append(y_level)
+
+        if water_levels:
+            water_levels.sort()  # Sort from top to bottom (smallest y to largest y)
+
+            # Check that water levels are contiguous from bottom
+            # Find the bottom-most water level
+            bottom_water_level = max(water_levels)
+
+            # All levels from bottom_water_level up to top water level should have water
+            expected_levels = list(range(min(water_levels), bottom_water_level + 1))
+
+            if water_levels != expected_levels:
+                return False
+
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/binario_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/binario_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..c64eb05e95c882fb4a2489bf96c109820841ff72
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/binario_eval.py
@@ -0,0 +1,1012 @@
+import re
+from typing import Any, Dict, List, Optional
+
+
+class BaseEvaluator:
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any = None,
+                 initial_state: List[List[Any]] = None) -> bool:
+        raise NotImplementedError
+
+
+class BinarioEvaluator(BaseEvaluator):
+    """
+    Enhanced evaluator for the Binairo task.
+    This task involves evaluating a matrix of 0s and 1s with improved robustness.
+    """
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """
+        Prepares the prompt for the model.
+
+        Args:
+            question: The question to be asked
+            params: Additional parameters for customizing the prompt
+
+        Returns:
+            The prepared prompt string
+        """
+        return question
+
+    def extract_answer(self, model_output: str) -> Optional[List[List[int]]]:
+        """
+        Enhanced extraction of matrix from model output with improved robustness.
+
+        Args:
+            model_output: The raw output from the model
+
+        Returns:
+            A list of lists representing the matrix, or None if no valid matrix is found
+        """
+        if not model_output or not isinstance(model_output, str):
+            return None
+
+        # Clean the output
+        output = model_output.strip()
+
+        # Multiple extraction strategies, ordered by preference
+        strategies = [
+            self._extract_from_literal_newlines,      # Handle \n literals
+            self._extract_from_code_blocks,
+            self._extract_from_answer_section,
+            self._extract_from_solution_section,
+            self._extract_from_result_section,
+            self._extract_from_matrix_keywords,       # Enhanced matrix keyword detection
+            self._extract_matrix_patterns,
+            self._extract_from_table_format,
+            self._extract_from_bracket_format,        # [1,0,1] format
+            self._extract_from_space_separated,       # Space-separated numbers
+            self._extract_from_json_like,             # JSON-like format
+            self._extract_from_numeric_blocks,        # Pure numeric blocks
+            self._extract_from_grid_format,           # Grid-like formats
+            self._extract_any_grid_pattern,
+            self._extract_from_single_line,           # All numbers in one line
+            self._extract_from_mixed_format,          # Mixed formats
+            self._extract_from_coordinate_format,     # R1C1=0 format
+            self._extract_from_quoted_strings,        # "0 1 0\n1 0 1" format
+            self._extract_from_enumerated_format,     # 1. 0 1 0, 2. 1 0 1 format
+        ]
+
+        for strategy in strategies:
+            matrix = strategy(output)
+            if matrix is not None and self._is_valid_matrix(matrix):
+                return matrix
+
+        return None
+
+    def _extract_from_literal_newlines(self, output: str) -> Optional[List[List[int]]]:
+        """Extract matrix from strings with literal \\n characters - enhanced version"""
+        # Handle various literal newline representations
+        patterns = [
+            r'([01](?:\s+[01])+(?:\\n[01](?:\s+[01])+)*)',  # Numbers with literal \n
+            r'([01](?:[,\s]+[01])+(?:\\n[01](?:[,\s]+[01])+)*)',  # With commas
+            r'([01](?:[,\s]+[01])+(?:[\s]*\\n[\s]*[01](?:[,\s]+[01])+)*)',  # More flexible spacing
+            r'([01](?:\s*[01])+(?:\\n[01](?:\s*[01])+)*)',  # Tight spacing
+        ]
+
+        for pattern in patterns:
+            matches = re.finditer(pattern, output)
+            for match in matches:
+                matrix_str = match.group(1)
+                # Handle multiple literal newline formats
+                for nl_format in ['\\n', '\\r\\n', '\\r']:
+                    matrix_str = matrix_str.replace(nl_format, '\n')
+
+                matrix = self._parse_matrix_string(matrix_str)
+                if matrix is not None:
+                    return matrix
+
+        # Also check for the exact format with various newline representations
+        for nl_format in ['\\n', '\\r\\n', '\\r']:
+            if nl_format in output:
+                cleaned_output = output.replace(nl_format, '\n')
+                matrix = self._parse_matrix_string(cleaned_output)
+                if matrix is not None:
+                    return matrix
+
+        return None
+
+    def _extract_from_code_blocks(self, output: str) -> Optional[List[List[int]]]:
+        """Extract matrix from code blocks (```...```)"""
+        patterns = [
+            r'```(?:python|text|matrix|grid|answer|solution)?\s*\n?([\d\s\n,|\t\\]+?)```',
+            r'```\s*([\d\s\n,|\t\\]+?)\s*```',
+        ]
+
+        for pattern in patterns:
+            matches = re.finditer(pattern, output, re.MULTILINE | re.DOTALL)
+            for match in matches:
+                content = match.group(1).replace('\\n', '\n')
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_answer_section(self, output: str) -> Optional[List[List[int]]]:
+        """Extract matrix from answer sections - enhanced version"""
+        patterns = [
+            # Standard answer patterns
+            r'(?:answer|solution|result):\s*([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+            r'(?:answer|solution|result)\s*[=:]\s*([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+            r'(?:final\s+)?(?:answer|solution|result)\s*[=:]?\s*\n?([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+
+            # Enhanced patterns with more keywords
+            r'(?:my\s+)?(?:answer|solution|result)\s*[=:]?\s*\n?([01][\d\s\n,|\t\\]+?)'
+            r'(?:\n\s*\n|\Z)',
+            r'(?:the\s+)?(?:correct\s+)?(?:answer|solution|result)\s*[=:]?\s*\n?'
+            r'([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+            r'(?:here\s+is\s+)?(?:my\s+)?(?:answer|solution|result)\s*[=:]?\s*\n?'
+            r'([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+
+            # Pattern for answers at the end of text
+            r'(?:answer|solution|result).*?([01](?:\s+[01])+(?:[\\n\n][01](?:\s+[01])+)*)\s*$',
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, output, re.IGNORECASE | re.MULTILINE | re.DOTALL)
+            if match:
+                content = match.group(1).replace('\\n', '\n')
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_solution_section(self, output: str) -> Optional[List[List[int]]]:
+        """Extract matrix from solution sections"""
+        patterns = [
+            r'(?:the\s+)?(?:solution|matrix|grid)\s+is:?\s*\n?([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+            r'(?:completed|filled|final)\s+(?:matrix|grid):?\s*\n?([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+            r'(?:solution|matrix|grid).*?([01](?:\s+[01])+(?:[\\n\n][01](?:\s+[01])+)*)',
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, output, re.IGNORECASE | re.MULTILINE | re.DOTALL)
+            if match:
+                content = match.group(1).replace('\\n', '\n')
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_result_section(self, output: str) -> Optional[List[List[int]]]:
+        """Extract matrix from result sections"""
+        patterns = [
+            r'(?:here\s+is\s+the\s+)?(?:result|output):?\s*\n?([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+            r'(?:the\s+)?(?:final|complete)\s+(?:result|output):?\s*\n?([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|\Z)',
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, output, re.IGNORECASE | re.MULTILINE | re.DOTALL)
+            if match:
+                content = match.group(1).replace('\\n', '\n')
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_bracket_format(self, output: str) -> Optional[List[List[int]]]:
+        """Extract matrix from bracket formats like [1,0,1] or [[1,0,1],[0,1,0]]"""
+        patterns = [
+            # [[1,0,1],[0,1,0]]
+            r'\[\s*\[\s*([01](?:\s*,\s*[01])*)\s*\](?:\s*,\s*\[\s*([01](?:\s*,\s*[01])*)\s*\])*\s*\]',
+            # [1,0,1]\n[0,1,0]
+            r'(?:\[\s*([01](?:\s*,\s*[01])*)\s*\](?:\s*,?\s*\n?)){2,}',
+        ]
+
+        for pattern in patterns:
+            matches = re.finditer(pattern, output)
+            for match in matches:
+                # Extract all bracket contents
+                bracket_contents = re.findall(r'\[\s*([01](?:\s*,\s*[01])*)\s*\]', match.group(0))
+                if len(bracket_contents) >= 2:
+                    matrix = []
+                    for content in bracket_contents:
+                        row = [int(x.strip()) for x in content.split(',') if x.strip().isdigit()]
+                        if all(cell in [0, 1] for cell in row) and len(row) > 0:
+                            matrix.append(row)
+
+                    if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+                        return matrix
+
+        return None
+
+    def _extract_from_space_separated(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from simple space-separated format - enhanced version"""
+        # Split by various line separators
+        lines = re.split(r'[\n\r]+', output)
+        matrix = []
+
+        for line in lines:
+            line = line.strip()
+            if not line:
+                continue
+
+            # Enhanced pattern matching for binary rows
+            # Check if line contains primarily 0s, 1s, and separators
+            if re.match(r'^[01\s,\|\-\+\.\t]*$', line):
+                # Extract only 0s and 1s
+                binary_chars = re.findall(r'[01]', line)
+                if len(binary_chars) >= 2:  # At least 2 binary digits
+                    row = [int(char) for char in binary_chars]
+                    matrix.append(row)
+
+        if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+            return matrix
+
+        return None
+
+    def _extract_from_json_like(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from JSON-like formats"""
+        patterns = [
+            r'\[\s*\[([01](?:\s*,\s*[01])*)\](?:\s*,\s*\[([01](?:\s*,\s*[01])*)\])*\s*\]',
+            r'(?:matrix|grid|answer)\s*[:=]\s*(\[\s*\[[\d\s,\]]+\])',
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, output)
+            if match:
+                try:
+                    # Try to evaluate as Python list
+                    content = match.group(1) if len(match.groups()) >= 1 else match.group(0)
+                    # Clean and make it safe for eval
+                    content = re.sub(r'[^\[\]01,\s]', '', content)
+                    matrix = eval(content)
+                    if isinstance(matrix, list) and all(isinstance(row, list) for row in matrix):
+                        if self._is_valid_matrix(matrix):
+                            return matrix
+                except Exception:
+                    pass
+
+        return None
+
+    def _extract_from_mixed_format(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from mixed formats with separators like | or tabs"""
+        patterns = [
+            r'([01](?:[\s|,\t]+[01])+(?:\n[01](?:[\s|,\t]+[01])+)*)',  # Mixed separators
+            r'([01](?:[|]+[01])+(?:\n[01](?:[|]+[01])+)*)',            # Pipe separated
+        ]
+
+        for pattern in patterns:
+            matches = re.finditer(pattern, output, re.MULTILINE)
+            for match in matches:
+                content = match.group(1)
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_coordinate_format(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from coordinate-based format like R1C1=0, R1C2=1"""
+        # Look for patterns like R1C1=0, R1C2=1, etc.
+        coord_pattern = r'R(\d+)C(\d+)\s*[=:]\s*([01])'
+        matches = re.findall(coord_pattern, output, re.IGNORECASE)
+
+        if matches:
+            # Determine grid size
+            max_row = max(int(m[0]) for m in matches)
+            max_col = max(int(m[1]) for m in matches)
+
+            # Check if we have enough coordinates for a complete grid
+            if len(matches) >= max_row * max_col * 0.5:  # At least half filled
+                matrix = [[None for _ in range(max_col)] for _ in range(max_row)]
+
+                for row_str, col_str, val_str in matches:
+                    row_idx = int(row_str) - 1  # Convert to 0-based
+                    col_idx = int(col_str) - 1
+                    if 0 <= row_idx < max_row and 0 <= col_idx < max_col:
+                        matrix[row_idx][col_idx] = int(val_str)
+
+                # Convert None to appropriate values if the grid is mostly filled
+                filled_count = sum(1 for row in matrix for cell in row if cell is not None)
+                if filled_count >= max_row * max_col * 0.8:  # 80% filled
+                    # This might be a complete solution, return as is
+                    return [[cell if cell is not None else 0 for cell in row] for row in matrix]
+
+        return None
+
+    def _extract_from_single_line(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from single line with all numbers"""
+        # Look for long sequences of 0s and 1s that might represent a flattened matrix
+        pattern = r'([01](?:[,\s]*[01]){5,})'  # At least 6 numbers (could be 2x3 or 3x2 etc.)
+
+        matches = re.finditer(pattern, output)
+        for match in matches:
+            numbers_str = match.group(1)
+            numbers = [int(x) for x in re.findall(r'[01]', numbers_str)]
+
+            if len(numbers) >= 4:  # At least 2x2
+                # Try different matrix dimensions
+                for rows in range(2, int(len(numbers)**0.5) + 2):
+                    if len(numbers) % rows == 0:
+                        cols = len(numbers) // rows
+                        if cols >= 2:  # At least 2 columns
+                            matrix = []
+                            for i in range(rows):
+                                row = numbers[i * cols:(i + 1) * cols]
+                                matrix.append(row)
+
+                            if self._is_valid_matrix(matrix):
+                                return matrix
+
+        return None
+
+    def _extract_matrix_patterns(self, output: str) -> Optional[List[List[int]]]:
+        """Extract using general matrix patterns"""
+        # Look for consecutive lines that look like matrix rows
+        lines = output.replace('\\n', '\n').split('\n')
+        matrix_candidates = []
+        current_matrix = []
+
+        for line in lines:
+            line = line.strip()
+            # Check if line looks like a matrix row
+            if self._is_matrix_row(line):
+                row = self._parse_matrix_row(line)
+                if row is not None:
+                    current_matrix.append(row)
+                else:
+                    if len(current_matrix) >= 2:  # At least 2 rows to be considered a matrix
+                        matrix_candidates.append(current_matrix[:])
+                    current_matrix = []
+            else:
+                if len(current_matrix) >= 2:
+                    matrix_candidates.append(current_matrix[:])
+                current_matrix = []
+
+        # Don't forget the last matrix if it exists
+        if len(current_matrix) >= 2:
+            matrix_candidates.append(current_matrix)
+
+        # Return the largest valid matrix
+        for matrix in sorted(matrix_candidates, key=len, reverse=True):
+            if self._is_valid_matrix(matrix):
+                return matrix
+
+        return None
+
+    def _extract_from_table_format(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from table-like formats with |"""
+        patterns = [
+            r'\|[\d\s\|]+\|',  # |0 1 0|
+            r'[\d\s]+\|[\d\s\|]+',  # 0 1 0|1 0 1|
+        ]
+
+        for pattern in patterns:
+            matches = re.findall(pattern, output, re.MULTILINE)
+            if matches:
+                matrix = []
+                for match in matches:
+                    # Clean up the match and extract numbers
+                    cleaned = re.sub(r'[|\s]+', ' ', match).strip()
+                    row = self._parse_matrix_row(cleaned)
+                    if row is not None:
+                        matrix.append(row)
+
+                if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+                    return matrix
+
+        return None
+
+    def _extract_any_grid_pattern(self, output: str) -> Optional[List[List[int]]]:
+        """Last resort: extract any pattern that looks like a grid"""
+        # Very permissive pattern for any sequence of 0s and 1s that could form a grid
+        output = output.replace('\\n', '\n')
+        pattern = r'(?:^|\n)\s*([01](?:[\s,|\t]+[01]){1,20})\s*(?:\n|$)'
+        matches = re.findall(pattern, output, re.MULTILINE)
+
+        if len(matches) >= 2:
+            matrix = []
+            for match in matches:
+                row = self._parse_matrix_row(match)
+                if row is not None and len(row) > 1:  # At least 2 elements per row
+                    matrix.append(row)
+
+            if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+                return matrix
+
+        return None
+
+    def _is_matrix_row(self, line: str) -> bool:
+        """Check if a line looks like a matrix row"""
+        if not line.strip():
+            return False
+
+        # Should contain only digits, spaces, commas, tabs, and pipes
+        if not re.match(r'^[\d\s,|\t]+$', line.strip()):
+            return False
+
+        # Should have at least 2 numbers
+        numbers = re.findall(r'\d+', line)
+        return len(numbers) >= 2 and all(num in ['0', '1'] for num in numbers)
+
+    def _parse_matrix_row(self, row_string: str) -> Optional[List[int]]:
+        """Parse a single row string into a list of integers with enhanced robustness"""
+        if not row_string:
+            return None
+
+        # Clean the string and extract numbers
+        cleaned = re.sub(r'[,|\t\-\+\=]+', ' ', row_string)  # Replace various separators with spaces
+        cleaned = re.sub(r'[^\d\s]', ' ', cleaned)  # Remove any non-digit, non-space characters
+        numbers = cleaned.strip().split()
+
+        try:
+            row = []
+            for num_str in numbers:
+                if num_str.isdigit():
+                    num = int(num_str)
+                    if num in [0, 1]:
+                        row.append(num)
+                    # Skip invalid numbers (not 0 or 1)
+
+            # Return only if we have at least 2 valid binary digits
+            if len(row) >= 2:
+                return row
+        except (ValueError, TypeError):
+            pass
+
+        return None
+
+    def _parse_matrix_string(self, matrix_string: str) -> Optional[List[List[int]]]:
+        """Parse a multi-line matrix string with enhanced robustness"""
+        if not matrix_string:
+            return None
+
+        matrix = []
+        # Replace various newline representations
+        matrix_string = matrix_string.replace('\\n', '\n').replace('\\r\\n', '\n').replace('\\r', '\n')
+
+        # Split by various line separators
+        rows = re.split(r'[\n\r]+', matrix_string.strip())
+
+        expected_row_length = None
+        for row_str in rows:
+            row_str = row_str.strip()
+            if not row_str:  # Skip empty rows
+                continue
+
+            row = self._parse_matrix_row(row_str)
+            if row is not None and len(row) > 0:
+                # Ensure consistent row length
+                if expected_row_length is None:
+                    expected_row_length = len(row)
+                elif len(row) != expected_row_length:
+                    # Skip rows with inconsistent length
+                    continue
+
+                matrix.append(row)
+
+        return matrix if len(matrix) >= 2 else None
+
+    def _is_valid_matrix(self, matrix: List[List[int]]) -> bool:
+        """Validate that the matrix is well-formed - enhanced version"""
+        if not matrix or not isinstance(matrix, list):
+            return False
+
+        if len(matrix) == 0:
+            return False
+
+        # Check that all rows have the same length
+        row_length = len(matrix[0])
+        if row_length == 0:
+            return False
+
+        for row in matrix:
+            if not isinstance(row, list) or len(row) != row_length:
+                return False
+
+            # Check that all elements are valid (0 or 1)
+            for cell in row:
+                if not isinstance(cell, int) or cell not in [0, 1]:
+                    return False
+
+        # Matrix should be at least 2x2 and dimensions should make sense for Binairo
+        if len(matrix) < 2 or row_length < 2:
+            return False
+
+        # For Binairo, dimensions should typically be even (though not strictly required)
+        # This is a soft check - we'll accept odd dimensions but prefer even
+        return True
+
+    def _extract_from_matrix_keywords(self, output: str) -> Optional[List[List[int]]]:
+        """Enhanced extraction from text with matrix-related keywords"""
+        # More comprehensive patterns for matrix keywords
+        patterns = [
+            r'(?:the\s+)?(?:completed|final|solved|answer)\s+(?:matrix|grid|puzzle)'
+            r'\s*[:\-]?\s*\n?([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|$)',
+            r'(?:matrix|grid|puzzle)\s+(?:solution|answer|result)\s*[:\-]?\s*\n?'
+            r'([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|$)',
+            r'(?:here\s+is\s+the\s+)?(?:matrix|grid|solution|answer)\s*[:\-]?\s*\n'
+            r'([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|$)',
+            r'(?:binairo|takuzu)\s+(?:solution|answer)\s*[:\-]?\s*\n?'
+            r'([01][\d\s\n,|\t\\]+?)(?:\n\s*\n|$)',
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, output, re.IGNORECASE | re.MULTILINE | re.DOTALL)
+            if match:
+                content = match.group(1).replace('\\n', '\n')
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_numeric_blocks(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from blocks of numbers separated by clear delimiters"""
+        # Look for blocks of binary digits with clear separations
+        patterns = [
+            r'(?:^|\n)\s*([01](?:[,\s]+[01])+)\s*(?:\n|$)',  # Simple space/comma separated
+            r'(?:^|\n)\s*([01](?:\s*[01])+)\s*(?:\n|$)',     # Tightly packed digits
+        ]
+
+        for pattern in patterns:
+            matches = re.findall(pattern, output, re.MULTILINE)
+            if len(matches) >= 2:
+                matrix = []
+                for match in matches:
+                    row = self._parse_matrix_row(match)
+                    if row is not None and len(row) >= 2:
+                        matrix.append(row)
+
+                if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+                    return matrix
+        return None
+
+    def _extract_from_grid_format(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from grid-like visual formats"""
+        # Handle ASCII grid formats with borders
+        patterns = [
+            r'(?:\+[\-\+]+\+\s*\n)([01\s\|]+)(?:\n\+[\-\+]+\+)?',  # +---+ bordered grids
+            r'(?:\|[01\s\|]+\|\s*\n){2,}',                          # Simple | delimited rows
+        ]
+
+        for pattern in patterns:
+            matches = re.finditer(pattern, output, re.MULTILINE)
+            for match in matches:
+                content = match.group(0)
+                # Extract rows from the grid
+                rows = re.findall(r'\|([01\s]+)\|', content)
+                if len(rows) >= 2:
+                    matrix = []
+                    for row_str in rows:
+                        row = self._parse_matrix_row(row_str)
+                        if row is not None:
+                            matrix.append(row)
+
+                    if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+                        return matrix
+        return None
+
+    def _extract_from_quoted_strings(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from quoted string formats"""
+        # Handle quoted matrix strings
+        patterns = [
+            r'"([01](?:[\s,]+[01])+(?:[\\n\n][01](?:[\s,]+[01])+)*)"',      # Double quotes
+            r"'([01](?:[\s,]+[01])+(?:[\\n\n][01](?:[\s,]+[01])+)*)'",      # Single quotes
+            r'`([01](?:[\s,]+[01])+(?:[\\n\n][01](?:[\s,]+[01])+)*)`',      # Backticks
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, output)
+            if match:
+                content = match.group(1).replace('\\n', '\n')
+                matrix = self._parse_matrix_string(content)
+                if matrix is not None:
+                    return matrix
+        return None
+
+    def _extract_from_enumerated_format(self, output: str) -> Optional[List[List[int]]]:
+        """Extract from enumerated row formats like '1. 0 1 0', '2. 1 0 1'"""
+        # Handle numbered rows
+        pattern = r'(?:^|\n)\s*\d+[\.\)]\s*([01](?:[\s,]+[01])+)\s*(?:\n|$)'
+        matches = re.findall(pattern, output, re.MULTILINE)
+
+        if len(matches) >= 2:
+            matrix = []
+            for match in matches:
+                row = self._parse_matrix_row(match)
+                if row is not None:
+                    matrix.append(row)
+
+            if len(matrix) >= 2 and self._is_valid_matrix(matrix):
+                return matrix
+        return None
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any = None, initial_state: List[List[Any]] = None) -> bool:
+        """
+        Evaluate predicted answer based ONLY on initial_state and Binairo game rules.
+        This function validates if the predicted answer is a correct completion of the initial puzzle
+        according to Binairo rules, completely ignoring the ground_truth parameter.
+
+        Args:
+            predicted_answer: The matrix extracted from the model's output or raw string
+            ground_truth: IGNORED - kept for interface compatibility only
+            initial_state: The initial puzzle state with None/null for empty cells
+
+        Returns:
+            True if the prediction is a valid solution to the initial_state according to Binairo rules, False otherwise
+        """
+        # Explicitly ignore ground_truth - this evaluation is rule-based only
+        _ = ground_truth  # Explicitly mark as unused
+
+        try:
+            # Extract predicted answer if it's a string
+            if isinstance(predicted_answer, str):
+                extracted_matrix = self.extract_answer(predicted_answer)
+                if extracted_matrix is None:
+                    print(f"Failed to extract matrix from string: {predicted_answer}")
+                    return False
+                predicted_answer = extracted_matrix
+
+            # Validate the matrix structure
+            if not self._is_valid_matrix(predicted_answer):
+                print(f"Invalid matrix structure: {predicted_answer}")
+                return False
+
+            # If no initial_state provided, validate only against Binairo rules
+            if initial_state is None:
+                print(f"Initial state is None: {predicted_answer}")
+                return self._validate_binairo_rules(predicted_answer)
+
+            # Normalize initial_state (handle different null representations)
+            normalized_initial_state = self._normalize_initial_state(initial_state)
+
+            # # Check if predicted answer correctly completes the initial state
+            if not self._validate_completion(predicted_answer, normalized_initial_state):
+                print(f"Predicted answer does not complete initial state: {predicted_answer}")
+                return False
+
+            # Check if the completed puzzle follows all Binairo rules
+            return self._validate_binairo_rules(predicted_answer)
+
+        except Exception:
+            # Log the exception for debugging if needed
+            return False
+
+    def _normalize_initial_state(self, initial_state: List[List[Any]]) -> List[List[Any]]:
+        """
+        Normalize initial state to handle different representations of empty cells.
+
+        Args:
+            initial_state: The initial puzzle state
+
+        Returns:
+            Normalized initial state with consistent None representation for empty cells
+        """
+        if initial_state is None:
+            return initial_state
+
+        # If provided as a string, try to parse into a list of lists first
+        if isinstance(initial_state, str):
+            parsed = None
+            try:
+                import json
+                parsed = json.loads(initial_state)
+            except Exception:
+                try:
+                    import ast
+                    parsed = ast.literal_eval(initial_state)
+                except Exception:
+                    parsed = None
+
+            if isinstance(parsed, list):
+                initial_state = parsed
+            else:
+                # Fallback: parse plain text grid with separators
+                lines = [line.strip() for line in initial_state.replace(
+                    '\r\n', '\n').replace('\r', '\n').split('\n') if line.strip()]
+                grid: List[List[Any]] = []
+                for line in lines:
+                    # Split on common separators while preserving empty markers
+                    tokens = [t for t in re.split(r'[\s,|]+', line)
+                              if t is not None]
+                    row: List[Any] = []
+                    for tok in tokens:
+                        tok_str = str(tok).strip()
+                        if tok_str in ['0', '1']:
+                            row.append(int(tok_str))
+                        elif tok_str in ['', '.', '_', '-', '*', 'x', 'X']:
+                            row.append(None)
+                        else:
+                            # Unknown token: ignore or treat as empty
+                            row.append(None)
+                    if len(row) > 0:
+                        grid.append(row)
+                initial_state = grid
+
+        if not initial_state:
+            return initial_state
+
+        normalized = []
+        for row in initial_state:
+            normalized_row = []
+            for cell in row:
+                # Normalize different empty cell representations to None
+                if (cell is None or cell == "" or cell == " "
+                        or cell == "." or cell == "_"
+                        or cell == 0 and isinstance(cell, str)):
+                    normalized_row.append(None)
+                else:
+                    # Ensure numeric values are integers
+                    try:
+                        if isinstance(cell, str) and cell.strip() in ['0', '1']:
+                            normalized_row.append(int(cell.strip()))
+                        elif isinstance(cell, (int, float)) and cell in [0, 1]:
+                            normalized_row.append(int(cell))
+                        else:
+                            normalized_row.append(None)  # Treat invalid values as empty
+                    except (ValueError, TypeError):
+                        normalized_row.append(None)
+            normalized.append(normalized_row)
+
+        return normalized
+
+    def str_to_dict(self, input_str):
+        if isinstance(input_str, str):
+            try:
+                import json
+                input_str = json.loads(input_str)
+            except (json.JSONDecodeError, TypeError):
+                try:
+                    import ast
+                    input_str = ast.literal_eval(input_str)
+                except (ValueError, SyntaxError):
+                    pass
+        return input_str
+
+    def _validate_completion(self, predicted_answer: List[List[int]], initial_state: List[List[Any]]) -> bool:
+        """
+        Validate that the predicted answer correctly completes the initial state.
+
+        Args:
+            predicted_answer: The complete solution matrix
+            initial_state: The initial puzzle state with None for empty cells
+
+        Returns:
+            True if predicted_answer is a valid completion of initial_state
+        """
+        # Ensure predicted_answer is a proper matrix (handle string inputs robustly)
+        if isinstance(predicted_answer, str):
+            extracted = self.extract_answer(predicted_answer)
+            if extracted is not None:
+                predicted_answer = extracted
+            else:
+                predicted_answer = self.str_to_dict(predicted_answer)
+
+        # If still not a list of lists, or empty, fail fast
+        if (not isinstance(predicted_answer, list) or len(predicted_answer) == 0
+                or not isinstance(predicted_answer[0], list)):
+            return False
+
+        # Coerce string digits to integers if needed
+        for i in range(len(predicted_answer)):
+            row = predicted_answer[i]
+            coerced_row = []
+            for cell in row:
+                if isinstance(cell, str):
+                    cell_str = cell.strip()
+                    if cell_str in ['0', '1']:
+                        coerced_row.append(int(cell_str))
+                    else:
+                        # Leave as-is; will fail validation below
+                        coerced_row.append(cell)
+                else:
+                    coerced_row.append(cell)
+            predicted_answer[i] = coerced_row
+
+        # Check dimensions match
+        if len(predicted_answer) != len(initial_state):
+            print(f"Row count mismatch: predicted {len(predicted_answer)} vs initial {len(initial_state)}")
+            return False
+
+        if len(predicted_answer) == 0:
+            return False
+
+        if len(predicted_answer[0]) != len(initial_state[0]):
+            print(f"Column count mismatch: predicted {len(predicted_answer[0])} vs initial {len(initial_state[0])}")
+            return False
+
+        # Check that all pre-filled cells match
+        for i in range(len(initial_state)):
+            for j in range(len(initial_state[0])):
+                initial_cell = initial_state[i][j]
+                predicted_cell = predicted_answer[i][j]
+
+                # If the initial cell was filled (not None), it must match the prediction
+                if initial_cell is not None:
+                    if initial_cell != predicted_cell:
+                        print(f"Prefilled mismatch at ({i},{j}): "
+                              f"initial={initial_cell}, predicted={predicted_cell}")
+                        return False
+
+                # Predicted cell must be 0 or 1
+                if predicted_cell not in [0, 1]:
+                    print(f"Invalid predicted cell at ({i},{j}): value={predicted_cell}")
+                    return False
+
+        return True
+
+    def _validate_binairo_rules(self, matrix: List[List[int]]) -> bool:
+        """
+        Validate that the matrix follows all Binairo (Takuzu) rules.
+
+        Rules:
+        1. Each row and column must contain equal numbers of 0s and 1s
+        2. No more than two consecutive identical digits in any row or column
+        3. All rows must be unique
+        4. All columns must be unique
+
+        Args:
+            matrix: The completed matrix to validate
+
+        Returns:
+            True if all rules are satisfied
+        """
+        if not matrix or len(matrix) == 0:
+            return False
+
+        # Rule 1: Equal numbers of 0s and 1s in each row and column
+        if not self._check_equal_distribution(matrix):
+            print(f"Equal distribution check failed: {matrix}")
+            return False
+
+        # Rule 2: No more than two consecutive identical digits
+        if not self._check_no_three_consecutive(matrix):
+            print(f"Three consecutive check failed: {matrix}")
+            return False
+
+        # Rule 3: All rows must be unique
+        if not self._check_unique_rows(matrix):
+            print(f"Unique rows check failed: {matrix}")
+            return False
+
+        # Rule 4: All columns must be unique
+        if not self._check_unique_columns(matrix):
+            print(f"Unique columns check failed: {matrix}")
+            return False
+
+        return True
+
+    def _check_equal_distribution(self, matrix: List[List[int]]) -> bool:
+        """Check if each row and column has equal numbers of 0s and 1s"""
+        rows = len(matrix)
+        cols = len(matrix[0])
+
+        # For even-sized grids, each row/column should have equal 0s and 1s
+        if rows % 2 == 0:
+            expected_count = rows // 2
+
+            # Check columns
+            for col_idx in range(cols):
+                column = [matrix[row_idx][col_idx] for row_idx in range(rows)]
+                if column.count(0) != expected_count or column.count(1) != expected_count:
+                    return False
+
+        if cols % 2 == 0:
+            expected_count = cols // 2
+
+            # Check rows
+            for row in matrix:
+                if row.count(0) != expected_count or row.count(1) != expected_count:
+                    return False
+
+        return True
+
+    def _check_no_three_consecutive(self, matrix: List[List[int]]) -> bool:
+        """Check that no row or column has three consecutive identical digits"""
+        rows = len(matrix)
+        cols = len(matrix[0])
+
+        # Check rows
+        for row in matrix:
+            for i in range(len(row) - 2):
+                if row[i] == row[i + 1] == row[i + 2]:
+                    return False
+
+        # Check columns
+        for col_idx in range(cols):
+            column = [matrix[row_idx][col_idx] for row_idx in range(rows)]
+            for i in range(len(column) - 2):
+                if column[i] == column[i + 1] == column[i + 2]:
+                    return False
+
+        return True
+
+    def _check_unique_rows(self, matrix: List[List[int]]) -> bool:
+        """Check that all rows are unique"""
+        row_tuples = [tuple(row) for row in matrix]
+        return len(set(row_tuples)) == len(row_tuples)
+
+    def _check_unique_columns(self, matrix: List[List[int]]) -> bool:
+        """Check that all columns are unique"""
+        rows = len(matrix)
+        cols = len(matrix[0])
+
+        columns = []
+        for col_idx in range(cols):
+            column = tuple(matrix[row_idx][col_idx] for row_idx in range(rows))
+            columns.append(column)
+
+        return len(set(columns)) == len(columns)
+
+    def get_validation_details(self, predicted_answer: Any, initial_state: List[List[Any]] = None) -> Dict[str, Any]:
+        """
+        Get detailed validation results for debugging purposes.
+
+        Args:
+            predicted_answer: The matrix to validate
+            initial_state: The initial puzzle state
+
+        Returns:
+            Dictionary with detailed validation results
+        """
+        details = {
+            'valid': False,
+            'errors': [],
+            'extraction_successful': False,
+            'matrix_well_formed': False,
+            'completion_valid': False,
+            'rules_satisfied': {
+                'equal_distribution': False,
+                'no_three_consecutive': False,
+                'unique_rows': False,
+                'unique_columns': False
+            }
+        }
+
+        try:
+            # Extract predicted answer if it's a string
+            if isinstance(predicted_answer, str):
+                predicted_answer = self.extract_answer(predicted_answer)
+
+            if predicted_answer is None:
+                details['errors'].append('Failed to extract matrix from input')
+                return details
+
+            details['extraction_successful'] = True
+
+            # Validate the matrix structure
+            if not self._is_valid_matrix(predicted_answer):
+                details['errors'].append('Matrix is not well-formed')
+                return details
+
+            details['matrix_well_formed'] = True
+
+            # Check completion if initial_state provided
+            if initial_state is not None:
+                normalized_initial = self._normalize_initial_state(initial_state)
+                if not self._validate_completion(predicted_answer, normalized_initial):
+                    details['errors'].append('Predicted answer does not correctly complete initial state')
+                    return details
+                details['completion_valid'] = True
+
+            # Check individual rules
+            details['rules_satisfied']['equal_distribution'] = self._check_equal_distribution(predicted_answer)
+            details['rules_satisfied']['no_three_consecutive'] = self._check_no_three_consecutive(predicted_answer)
+            details['rules_satisfied']['unique_rows'] = self._check_unique_rows(predicted_answer)
+            details['rules_satisfied']['unique_columns'] = self._check_unique_columns(predicted_answer)
+
+            # Add specific error messages for failed rules
+            if not details['rules_satisfied']['equal_distribution']:
+                details['errors'].append('Not all rows/columns have equal numbers of 0s and 1s')
+            if not details['rules_satisfied']['no_three_consecutive']:
+                details['errors'].append('Found three or more consecutive identical digits')
+            if not details['rules_satisfied']['unique_rows']:
+                details['errors'].append('Not all rows are unique')
+            if not details['rules_satisfied']['unique_columns']:
+                details['errors'].append('Not all columns are unique')
+
+            # Overall validation
+            details['valid'] = all(details['rules_satisfied'].values()) and (
+                initial_state is None or details['completion_valid']
+            )
+
+        except Exception as e:
+            details['errors'].append(f'Validation error: {str(e)}')
+
+        return details
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/bridges_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/bridges_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..c28c7b6e533e5c0b6ca4669a1ef7751116af9832
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/bridges_eval.py
@@ -0,0 +1,391 @@
+import json
+import re
+from collections import defaultdict
+from typing import Any, Dict, List, Set, Tuple
+
+
+class BaseEvaluator:
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any,
+                 initial_state: Any) -> bool:
+        raise NotImplementedError
+
+
+class BridgesEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """
+        Prepare the prompt for the bridges task.
+        Returns the question with additional instructions if needed.
+        """
+        return question
+
+    def extract_answer(self, model_output: str) -> str:
+        """
+        Enhanced extraction of bridge specifications from model output.
+        Handles various formats and edge cases with robust regex patterns.
+        """
+        if not model_output or not isinstance(model_output, str):
+            return ""
+
+        # Clean the input first
+        model_output = model_output.strip()
+
+        # Multiple patterns to handle different answer formats
+        patterns = [
+            # Standard format: <answer>content</answer>
+            r'<answer>\s*(.*?)\s*</answer>',
+            # Alternative format: [answer]content[/answer]
+            r'\[answer\]\s*(.*?)\s*\[/answer\]',
+            # JSON-like format: "answer": "content" (handle escaped newlines)
+            r'"answer"\s*:\s*"(.*?)"',
+            # Direct format without tags (Answer: or Solution:)
+            r'(?:Answer|Solution|答案)[:\s]*\n?(.*?)(?:\n\n|$)',
+            # Chinese answer tags format
+            r'答案[：:\s]*\n?(.*?)(?:\n\n|$)',
+            # Bridge: or Bridges: format
+            r'(?:Bridge|Bridges)[:\s]*\n?(.*?)(?:\n\n|$)',
+            # Solution in backticks
+            r'```(?:text|bridge|bridges)?\s*(.*?)\s*```',
+            # Final answer format
+            r'(?:Final answer|最终答案)[:\s]*\n?(.*?)(?:\n\n|$)',
+        ]
+
+        extracted_content = ""
+
+        # Try each pattern
+        for pattern in patterns:
+            match = re.search(pattern, model_output, re.DOTALL | re.IGNORECASE)
+            if match:
+                extracted_content = match.group(1).strip()
+                # Handle escaped newlines in JSON format
+                if '\\n' in extracted_content:
+                    extracted_content = extracted_content.replace('\\n', '\n')
+                # Remove extra quotes if present
+                extracted_content = extracted_content.strip('"\'')
+                break
+
+        # If no tags found, search for bridge patterns directly in the entire output
+        if not extracted_content:
+            bridge_lines = []
+            lines = model_output.split('\n')
+
+            for line in lines:
+                line = line.strip()
+                # Match bridge patterns with various formats
+                if self._is_bridge_line(line):
+                    bridge_lines.append(line)
+
+            extracted_content = '\n'.join(bridge_lines)
+
+        # Clean and normalize the extracted content
+        return self._normalize_bridge_format(extracted_content)
+
+    def _is_bridge_line(self, line: str) -> bool:
+        """Check if a line contains a valid bridge specification"""
+        if not line or len(line.strip()) == 0:
+            return False
+
+        # Various bridge patterns - more comprehensive
+        bridge_patterns = [
+            # Standard format: (x1,y1)-(x2,y2):count
+            r'\(\s*\d+\s*,\s*\d+\s*\)\s*[-–—]\s*\(\s*\d+\s*,\s*\d+\s*\)\s*[:\s]*\s*\d+',
+            # With "to" connector: (x1,y1) to (x2,y2): count
+            r'\(\s*\d+\s*,\s*\d+\s*\)\s*(?:to|TO)\s*\(\s*\d+\s*,\s*\d+\s*\)\s*[:\s]*\s*\d+',
+            # With arrow: (x1,y1) → (x2,y2): count
+            r'\(\s*\d+\s*,\s*\d+\s*\)\s*[→>]\s*\(\s*\d+\s*,\s*\d+\s*\)\s*[:\s]*\s*\d+',
+            # With bridge prefix: bridge: (x1,y1)-(x2,y2):count
+            r'(?:bridge|Bridge|BRIDGE)\s*[:\s]*\s*\(\s*\d+\s*,\s*\d+\s*\)\s*'
+            r'[-–—]\s*\(\s*\d+\s*,\s*\d+\s*\)\s*[:\s]*\s*\d+',
+            # Connect format: connect (x1,y1) to (x2,y2) with count bridge(s)
+            r'(?:connect|Connect)\s*\(\s*\d+\s*,\s*\d+\s*\)\s*(?:to|TO)\s*'
+            r'\(\s*\d+\s*,\s*\d+\s*\)\s*(?:with|using)\s*\d+',
+            # Number at the end format: (x1,y1)-(x2,y2) count
+            r'\(\s*\d+\s*,\s*\d+\s*\)\s*[-–—]\s*\(\s*\d+\s*,\s*\d+\s*\)\s+\d+',
+        ]
+
+        for pattern in bridge_patterns:
+            if re.search(pattern, line, re.IGNORECASE):
+                return True
+        return False
+
+    def _normalize_bridge_format(self, content: str) -> str:
+        """Normalize bridge specifications to standard format"""
+        if not content:
+            return ""
+
+        lines = [line.strip() for line in content.split('\n') if line.strip()]
+        normalized_lines = []
+
+        for line in lines:
+            # Extract coordinates and count using flexible regex patterns
+            patterns = [
+                # Standard format: (x1,y1)-(x2,y2):count
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[-–—]\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)'
+                r'\s*[:\s]*\s*(\d+)',
+                # With "to": (x1,y1) to (x2,y2): count
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*(?:to|TO)\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)'
+                r'\s*[:\s]*\s*(\d+)',
+                # With arrow: (x1,y1) → (x2,y2): count
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[→>]\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)'
+                r'\s*[:\s]*\s*(\d+)',
+                # Connect format
+                r'(?:connect|Connect)\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*(?:to|TO)\s*'
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*(?:with|using)\s*(\d+)',
+                # Number at end: (x1,y1)-(x2,y2) count
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[-–—]\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s+(\d+)',
+            ]
+
+            match = None
+            for pattern in patterns:
+                match = re.search(pattern, line, re.IGNORECASE)
+                if match:
+                    break
+
+            if match:
+                x1, y1, x2, y2, count = match.groups()
+                # Validate count is a positive integer
+                try:
+                    count_int = int(count)
+                    if count_int > 0:
+                        normalized = f"({x1},{y1})-({x2},{y2}):{count_int}"
+                        normalized_lines.append(normalized)
+                except ValueError:
+                    continue  # Skip invalid count
+
+        return '\n'.join(normalized_lines)
+
+    def evaluate(self, predicted_answer: str, ground_truth: str, initial_state: str) -> bool:
+        """
+        Evaluate if the predicted answer is correct based ONLY on game rules and initial state.
+        The ground_truth parameter is ignored - evaluation is purely rule-based.
+
+        Args:
+            predicted_answer: Model's predicted bridge connections
+            ground_truth: Reference answer (IGNORED - not used in evaluation)
+            initial_state: JSON string or dict containing the puzzle's initial island configuration
+
+        Returns:
+            bool: True if the predicted answer satisfies all game rules
+        """
+        try:
+            # Parse the initial state
+            if isinstance(initial_state, str):
+                puzzle_data = json.loads(initial_state)
+            elif isinstance(initial_state, dict):
+                puzzle_data = initial_state
+            else:
+                return False
+
+            islands = puzzle_data.get('islands', [])
+            if not islands:
+                return False
+
+            # Extract predicted answer if it hasn't been processed yet
+            if not self._is_normalized_answer_format(predicted_answer):
+                predicted_answer = self.extract_answer(predicted_answer)
+
+            if not predicted_answer.strip():
+                return False
+
+            # Extract and parse predicted bridges
+            predicted_bridges = self._parse_bridges(predicted_answer)
+
+            if not predicted_bridges:
+                return False
+
+            # Validate the solution against all game rules
+            return self._validate_solution(islands, predicted_bridges)
+
+        except (json.JSONDecodeError, KeyError, ValueError, TypeError):
+            return False
+
+    def _is_normalized_answer_format(self, answer: str) -> bool:
+        """Check if the answer is already in normalized format"""
+        if not answer or not isinstance(answer, str):
+            return False
+
+        lines = [line.strip() for line in answer.split('\n') if line.strip()]
+        if not lines:
+            return False
+
+        # Check if all lines match the normalized format
+        for line in lines:
+            if not re.match(r'\(\d+,\d+\)-\(\d+,\d+\):\d+', line):
+                return False
+        return True
+
+    def _parse_bridges(self, answer: str) -> List[Tuple[Tuple[int, int], Tuple[int, int], int]]:
+        """Parse bridge specifications into a list of tuples"""
+        if not answer:
+            return []
+
+        bridges = []
+        lines = [line.strip() for line in answer.split('\n') if line.strip()]
+
+        for line in lines:
+            match = re.match(r'\((\d+),(\d+)\)-\((\d+),(\d+)\):(\d+)', line)
+            if match:
+                try:
+                    x1, y1, x2, y2, count = map(int, match.groups())
+                    # Validate count is positive
+                    if count > 0:
+                        bridges.append(((x1, y1), (x2, y2), count))
+                except ValueError:
+                    continue  # Skip invalid lines
+
+        return bridges
+
+    def _validate_solution(self, islands: List[Dict], bridges: List[Tuple]) -> bool:
+        """
+        Validate the bridge solution against all game rules.
+
+        Rules:
+        1. Each island's bridge count must match its number
+        2. Bridges must be horizontal or vertical
+        3. No more than 2 bridges between any pair of islands
+        4. Bridges cannot cross each other or pass through islands
+        5. All islands must be connected in a single network
+        6. Bridges can only connect existing islands
+        """
+        if not islands or not bridges:
+            return False
+
+        # Create island lookup by coordinates
+        island_dict = {}
+        island_positions = set()
+
+        for island in islands:
+            if not isinstance(island, dict):
+                return False
+            if 'x' not in island or 'y' not in island or 'num' not in island:
+                return False
+            try:
+                x, y, num = int(island['x']), int(island['y']), int(island['num'])
+                if num <= 0:  # Island number must be positive
+                    return False
+                island_dict[(x, y)] = num
+                island_positions.add((x, y))
+            except (ValueError, TypeError):
+                return False
+
+        # Rule validation
+        island_bridge_counts = defaultdict(int)
+        bridge_connections = defaultdict(int)  # Track bridges between pairs
+
+        for (x1, y1), (x2, y2), count in bridges:
+            # Rule 6: Check if both endpoints are valid islands
+            if (x1, y1) not in island_positions or (x2, y2) not in island_positions:
+                return False
+
+            # Rule 2: Check if bridge is horizontal or vertical
+            if not (x1 == x2 or y1 == y2):
+                return False
+
+            # Don't allow bridges to the same island
+            if (x1, y1) == (x2, y2):
+                return False
+
+            # Count bridges for each island
+            island_bridge_counts[(x1, y1)] += count
+            island_bridge_counts[(x2, y2)] += count
+
+            # Count bridges between pairs - normalize pair order
+            pair = tuple(sorted([(x1, y1), (x2, y2)]))
+            bridge_connections[pair] += count
+
+        # Rule 1: Verify each island has correct number of bridges
+        for pos, required_count in island_dict.items():
+            if island_bridge_counts[pos] != required_count:
+                return False
+
+        # Rule 3: No more than 2 bridges between any pair
+        for count in bridge_connections.values():
+            if count > 2:
+                return False
+
+        # Rule 4: Check for bridge crossings and islands in path
+        if not self._check_no_crossings(bridges, island_positions):
+            return False
+
+        # Rule 5: Check connectivity
+        if not self._check_connectivity(island_positions, bridges):
+            return False
+
+        return True
+
+    def _check_no_crossings(self, bridges: List[Tuple], island_positions: Set[Tuple[int, int]]) -> bool:
+        """Check that bridges don't cross and don't pass through islands"""
+
+        for i, bridge1 in enumerate(bridges):
+            (x1a, y1a), (x1b, y1b), _ = bridge1
+
+            # Check no islands in path (except endpoints)
+            if x1a == x1b:  # Vertical bridge
+                for y in range(min(y1a, y1b) + 1, max(y1a, y1b)):
+                    if (x1a, y) in island_positions:
+                        return False
+            else:  # Horizontal bridge
+                for x in range(min(x1a, x1b) + 1, max(x1a, x1b)):
+                    if (x, y1a) in island_positions:
+                        return False
+
+            # Check for crossings with other bridges
+            for j, bridge2 in enumerate(bridges[i + 1:], i + 1):
+                if self._bridges_cross(bridge1, bridge2):
+                    return False
+
+        return True
+
+    def _bridges_cross(self, bridge1: Tuple, bridge2: Tuple) -> bool:
+        """Check if two bridges cross each other"""
+        (x1a, y1a), (x1b, y1b), _ = bridge1
+        (x2a, y2a), (x2b, y2b), _ = bridge2
+
+        # One bridge is horizontal, other is vertical
+        if x1a == x1b and y2a == y2b:  # bridge1 vertical, bridge2 horizontal
+            # Check if they intersect
+            if (min(y1a, y1b) < y2a < max(y1a, y1b)
+                    and min(x2a, x2b) < x1a < max(x2a, x2b)):
+                return True
+        elif y1a == y1b and x2a == x2b:  # bridge1 horizontal, bridge2 vertical
+            # Check if they intersect
+            if (min(x1a, x1b) < x2a < max(x1a, x1b)
+                    and min(y2a, y2b) < y1a < max(y2a, y2b)):
+                return True
+
+        return False
+
+    def _check_connectivity(self, island_positions: Set[Tuple[int, int]], bridges: List[Tuple]) -> bool:
+        """Check that all islands are connected through bridges"""
+        if not island_positions:
+            return True
+
+        # Build adjacency graph
+        graph = defaultdict(set)
+        for (x1, y1), (x2, y2), _ in bridges:
+            graph[(x1, y1)].add((x2, y2))
+            graph[(x2, y2)].add((x1, y1))
+
+        # DFS to check connectivity
+        visited = set()
+        start = next(iter(island_positions))
+        stack = [start]
+
+        while stack:
+            current = stack.pop()
+            if current in visited:
+                continue
+            visited.add(current)
+
+            for neighbor in graph[current]:
+                if neighbor not in visited:
+                    stack.append(neighbor)
+
+        # All islands should be reachable
+        return len(visited) == len(island_positions)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/calcudoku_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/calcudoku_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..5df9e17c849c78de24f8195cae0102ed7a2c8805
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/calcudoku_eval.py
@@ -0,0 +1,218 @@
+#!/usr/bin/env python3
+
+import json
+import re
+from functools import reduce
+from typing import Any, Dict, List
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class CalcudokuEvaluator(BaseEvaluator):
+    """
+    评估Calcudoku（计算数独）解答的评估器
+    验证模型输出的解答是否：
+    1. 符合Calcudoku的基本规则（每行每列包含1到n的数字且不重复）
+    2. 符合每个区域的数学运算规则
+    """
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """准备发送给模型的提示词"""
+        size = params.get("size", 3)
+        regions = params.get("regions", [])
+
+        prompt = (
+            f"This is a {size}x{size} Calcudoku puzzle. Each row and column must contain the numbers 1 to {size} "
+            f"exactly once.\n"
+            f"The grid is divided into regions, each with a target number and a specified operation.\n"
+            f"The numbers within each region must be combined using the given operation to achieve the "
+            f"target number.\n\n"
+        )
+
+        # 添加区域信息
+        for i, region in enumerate(regions):
+            cells = region.get('cells', [])
+            operator = region.get('operator', '+')
+            target = region.get('target', 0)
+
+            # 将乘法*符号转换为×以便显示
+            display_op = '×' if operator == '*' else operator
+
+            cell_str = ', '.join([f"({r},{c})" for r, c in cells])
+            prompt += f"Region {i+1}: Cells {cell_str}, Operation: {display_op}, Target: {target}\n"
+
+        prompt += (
+            "\nPlease solve the puzzle and provide the solution as a two-dimensional array.\n"
+            "Example answer format: [[1, 2, 3], [3, 1, 2], [2, 3, 1]]"
+        )
+
+        return prompt
+
+    def extract_answer(self, model_output: str) -> List[List[int]]:
+        """从模型输出中提取Calcudoku解答"""
+        if isinstance(model_output, dict) and "text" in model_output:
+            model_output = model_output["text"]
+
+        # 尝试查找完整的二维数组格式
+        # 匹配 [[数字, 数字, ...], [数字, 数字, ...], ...]
+        array_pattern = r'\[\s*\[(?:\s*\d+\s*,\s*)*\s*\d+\s*\](?:\s*,\s*\[\s*(?:\d+\s*,\s*)*\d+\s*\])*\s*\]'
+        matches = re.findall(array_pattern, model_output)
+
+        if matches:
+            # 取最后一个匹配的数组（可能是最终答案）
+            try:
+                # 尝试解析匹配到的字符串为JSON格式的数组
+                return json.loads(matches[-1])
+            except json.JSONDecodeError:
+                pass
+
+        # 如果无法直接解析为JSON，尝试手动解析
+        # 首先检查是否有明显的二维数组表示
+        lines = model_output.split('\n')
+        grid_lines = []
+
+        for line in lines:
+            # 查找包含多个数字的行
+            if re.search(r'\[\s*\d+.*\d+\s*\]', line):
+                grid_lines.append(line)
+
+        if grid_lines:
+            # 尝试构建一个有效的二维数组字符串
+            grid_str = '[' + ','.join(grid_lines) + ']'
+            grid_str = re.sub(r'[^\[\],\d\s]', '', grid_str)  # 移除不应出现在JSON数组中的字符
+            try:
+                return json.loads(grid_str)
+            except json.JSONDecodeError:
+                pass
+
+        # 最后尝试提取所有数字序列，根据问题规模构建网格
+        all_numbers = re.findall(r'\d+', model_output)
+
+        # 猜测网格大小（假设网格是方形的）
+        grid_size = int(len(all_numbers) ** 0.5) if all_numbers else 0
+
+        if grid_size > 0 and grid_size ** 2 == len(all_numbers):
+            grid = []
+            for i in range(0, len(all_numbers), grid_size):
+                row = [int(num) for num in all_numbers[i:i + grid_size]]
+                grid.append(row)
+            return grid
+
+        return []
+
+    def evaluate(self, model_output: str, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        """
+        评估预测的Calcudoku解答是否正确，不直接比对ground_truth，而是验证解是否满足所有规则
+
+        参数:
+        model_output: 模型生成的文本输出
+        ground_truth: 不再直接使用，但保留参数以保持接口一致性
+        params: 包含谜题信息的参数
+
+        返回:
+        是否正确（布尔值）
+        """
+        # 从模型输出中提取答案
+        extracted_answer = self.extract_answer(model_output)
+
+        # 如果无法提取有效答案，直接返回False
+        if not extracted_answer or not isinstance(extracted_answer, list):
+            return False
+
+        # 提取谜题信息
+        size = params.get("size", len(extracted_answer))
+        regions = params.get("regions", [])
+
+        # 1. 验证网格尺寸
+        if len(extracted_answer) != size:
+            return False
+
+        for row in extracted_answer:
+            if len(row) != size or not isinstance(row, list):
+                return False
+
+        # 2. 验证每行每列包含1到n的数字且不重复
+        expected_set = set(range(1, size + 1))
+
+        # 检查每行
+        for row in extracted_answer:
+            if set(row) != expected_set:
+                return False
+
+        # 检查每列
+        for col in range(size):
+            column_values = [extracted_answer[row][col] for row in range(size)]
+            if set(column_values) != expected_set:
+                return False
+
+        # 3. 验证每个区域的运算规则
+        for region in regions:
+            cells = region.get('cells', [])
+            operator = region.get('operator', '+')
+            target = region.get('target', 0)
+
+            # 提取区域中的值
+            region_values = []
+            for r, c in cells:
+                # 注意：cells坐标可能是1-indexed，需要转换为0-indexed
+                row_idx = r - 1
+                col_idx = c - 1
+
+                # 确保索引在有效范围内
+                if 0 <= row_idx < len(extracted_answer) and 0 <= col_idx < len(extracted_answer[row_idx]):
+                    region_values.append(extracted_answer[row_idx][col_idx])
+                else:
+                    # 索引超出范围，说明解答有问题
+                    return False
+
+            # 确保提取了正确数量的值
+            if len(region_values) != len(cells):
+                return False
+
+            # 根据运算符验证
+            result = self._calculate_region(region_values, operator)
+            if result != target:
+                return False
+
+        # 所有规则验证通过
+        return True
+
+    def _calculate_region(self, values: List[int], operator: str) -> int:
+        """
+        根据指定的操作符计算区域的结果值
+
+        参数:
+            values: 区域内的数值列表
+            operator: 操作符（+, -, *, ÷）
+
+        返回:
+            计算结果
+        """
+        if not values:
+            return 0
+
+        if operator == '+':
+            return sum(values)
+        elif operator == '*':
+            return reduce(lambda x, y: x * y, values)
+        elif operator == '-':
+            # 减法适用于两个数字的情况，取绝对值
+            if len(values) == 2:
+                return abs(values[0] - values[1])
+            return 0
+        elif operator == '÷':
+            # 除法适用于两个数字的情况，取最大值除以最小值
+            if len(values) == 2:
+                return max(values) // min(values) if min(values) != 0 else 0
+            return 0
+        else:
+            return 0
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/campsite_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/campsite_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..3f7abc5af9f61121f2ddf5a68c435d8b408dfef3
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/campsite_eval.py
@@ -0,0 +1,595 @@
+import ast
+import json
+import re
+from typing import Any, Dict, List, Optional, Union
+
+
+class CampsiteEvaluator:
+    """
+    Campsite puzzle evaluator that validates tent placement solutions.
+
+    Rules:
+    1. Each tent must be orthogonally adjacent to at least one tree
+    2. No tents can be adjacent to each other, even diagonally
+    3. The number of tents in each row and column must match the given constraints
+
+    Coordinate System: 1-based indexing (top-left corner is [1,1])
+    """
+
+    def __init__(self):
+        """Initialize the evaluator"""
+        pass
+
+    def extract_answer(self, model_output: str) -> Optional[List[List[int]]]:
+        """
+        从模型输出中提取答案坐标，具有极强的鲁棒性
+        支持多种格式和边界情况
+        """
+        if not isinstance(model_output, str):
+            # 如果输入已经是列表，直接验证格式
+            if isinstance(model_output, list):
+                return self._validate_coordinate_format(model_output)
+            return None
+
+        # 去除首尾空白字符和常见的干扰字符
+        output = model_output.strip()
+        output = output.replace('\n', ' ').replace('\t', ' ')
+        output = re.sub(r'\s+', ' ', output)  # 合并多个空格
+
+        # 方法1: 尝试直接解析完整的列表格式
+        result = self._try_direct_parsing(output)
+        if result is not None:
+            return result
+
+        # 方法2: 使用多种正则表达式模式匹配
+        result = self._try_regex_patterns(output)
+        if result is not None:
+            return result
+
+        # 方法3: 提取所有数字并尝试配对
+        result = self._try_number_pairing(output)
+        if result is not None:
+            return result
+
+        # 方法4: 处理特殊格式和语言描述
+        result = self._try_special_formats(output)
+        if result is not None:
+            return result
+
+        # 方法5: 尝试更激进的数字提取
+        result = self._try_aggressive_extraction(output)
+        if result is not None:
+            return result
+
+        # 方法6: 处理混合格式和特殊边界情况
+        result = self._try_mixed_formats(output)
+        if result is not None:
+            return result
+
+        # 如果所有方法都失败，返回None
+        return None
+
+    def _validate_coordinate_format(self, coords: List) -> Optional[List[List[int]]]:
+        """验证坐标格式是否正确"""
+        try:
+            if not isinstance(coords, list):
+                return None
+
+            result = []
+            for coord in coords:
+                if isinstance(coord, (list, tuple)) and len(coord) == 2:
+                    try:
+                        row, col = int(coord[0]), int(coord[1])
+                        if row > 0 and col > 0:  # 1-based indexing
+                            result.append([row, col])
+                        else:
+                            return None
+                    except (ValueError, TypeError):
+                        return None
+                else:
+                    return None
+
+            return result if result else []  # 允许空列表
+        except Exception:
+            return None
+
+    def _try_direct_parsing(self, output: str) -> Optional[List[List[int]]]:
+        """尝试直接解析完整的列表格式"""
+        try:
+            # 清理常见的格式问题
+            cleaned = output.strip()
+
+            # 处理标准格式: [[1,2], [3,4]]
+            if cleaned.startswith('[') and cleaned.endswith(']'):
+                # 尝试直接解析
+                try:
+                    parsed = ast.literal_eval(cleaned)
+                    return self._validate_coordinate_format(parsed)
+                except Exception:
+                    pass
+
+                # 如果失败，尝试清理后再解析
+                cleaned = re.sub(r'\s+', ' ', cleaned)
+                cleaned = re.sub(r',\s*]', ']', cleaned)
+                cleaned = re.sub(r'\[\s*,', '[', cleaned)
+                cleaned = re.sub(r',,+', ',', cleaned)  # 移除多余的逗号
+                try:
+                    parsed = ast.literal_eval(cleaned)
+                    return self._validate_coordinate_format(parsed)
+                except Exception:
+                    pass
+
+            # 处理JSON格式
+            try:
+                parsed = json.loads(cleaned)
+                return self._validate_coordinate_format(parsed)
+            except Exception:
+                pass
+
+        except Exception:
+            pass
+        return None
+
+    def _try_regex_patterns(self, output: str) -> Optional[List[List[int]]]:
+        """使用多种正则表达式模式匹配坐标"""
+        patterns = [
+            # 标准格式: [[1,2], [3,4]]
+            r'\[\s*\[\s*(\d+)\s*,\s*(\d+)\s*\](?:\s*,\s*\[\s*(\d+)\s*,\s*(\d+)\s*\])*\s*\]',
+            # 元组格式: [(1,2), (3,4)]
+            r'\[\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)(?:\s*,\s*\(\s*(\d+)\s*,\s*(\d+)\s*\))*\s*\]',
+            # 混合格式: [1,2], [3,4] (无外层括号)
+            r'\[\s*(\d+)\s*,\s*(\d+)\s*\](?:\s*,?\s*\[\s*(\d+)\s*,\s*(\d+)\s*\])*',
+            # 元组格式: (1,2), (3,4) (无外层括号)
+            r'\(\s*(\d+)\s*,\s*(\d+)\s*\)(?:\s*,?\s*\(\s*(\d+)\s*,\s*(\d+)\s*\))*',
+        ]
+
+        for pattern in patterns:
+            try:
+                # 使用findall找到所有数字对
+                numbers = re.findall(r'(\d+)\s*,\s*(\d+)', output)
+                if numbers:
+                    coords = []
+                    for num_pair in numbers:
+                        try:
+                            row, col = int(num_pair[0]), int(num_pair[1])
+                            if row > 0 and col > 0:  # 验证1-based indexing
+                                coords.append([row, col])
+                        except (ValueError, TypeError):
+                            continue
+
+                    if len(coords) >= 0:  # 允许空答案
+                        return coords
+            except Exception:
+                continue
+
+        return None
+
+    def _try_number_pairing(self, output: str) -> Optional[List[List[int]]]:
+        """提取所有数字并尝试配对成坐标"""
+        try:
+            # 提取所有数字
+            numbers = re.findall(r'\d+', output)
+            if len(numbers) >= 2 and len(numbers) % 2 == 0:
+                coords = []
+                for i in range(0, len(numbers), 2):
+                    try:
+                        row, col = int(numbers[i]), int(numbers[i + 1])
+                        if row > 0 and col > 0:  # 验证1-based indexing
+                            coords.append([row, col])
+                    except (ValueError, TypeError, IndexError):
+                        continue
+
+                # 如果提取的坐标数量合理（通常0-30个帐篷）
+                if 0 <= len(coords) <= 30:
+                    return coords
+            elif len(numbers) == 0:
+                # 没有数字，可能是空答案
+                return []
+        except Exception:
+            pass
+
+        return None
+
+    def _try_special_formats(self, output: str) -> Optional[List[List[int]]]:
+        """处理特殊格式和语言描述"""
+        special_patterns = [
+            # 处理 "坐标为: (1,2), (3,4)" 格式
+            r'坐标[为是]?\s*[:：]?\s*(.+)',
+            # 处理 "答案是: [[1,2], [3,4]]" 格式
+            r'答案[是为]?\s*[:：]?\s*(.+)',
+            # 处理 "positions are: (1,2), (3,4)" 格式
+            r'positions?\s+(?:are|is)\s*[:：]?\s*(.+)',
+            # 处理 "coordinates: [[1,2], [3,4]]" 格式
+            r'coordinates?\s*[:：]?\s*(.+)',
+            # 处理 "result: [[1,2], [3,4]]" 格式
+            r'result\s*[:：]?\s*(.+)',
+            # 处理 "answer: [[1,2], [3,4]]" 格式
+            r'answer\s*[:：]?\s*(.+)',
+            # 处理 "solution: [[1,2], [3,4]]" 格式
+            r'solution\s*[:：]?\s*(.+)',
+            # 处理 "tents: [[1,2], [3,4]]" 格式
+            r'tents?\s*[:：]?\s*(.+)',
+            # 处理 "final answer: [[1,2], [3,4]]" 格式
+            r'final\s+answer\s*[:：]?\s*(.+)',
+            # 处理 "tent coordinates: [[1,2], [3,4]]" 格式
+            r'tent\s+coordinates?\s*[:：]?\s*(.+)',
+            # 处理 "tent positions: [[1,2], [3,4]]" 格式
+            r'tent\s+positions?\s*[:：]?\s*(.+)',
+        ]
+
+        for pattern in special_patterns:
+            match = re.search(pattern, output, re.IGNORECASE)
+            if match:
+                extracted_part = match.group(1).strip()
+                # 递归调用其他方法处理提取的部分
+                result = self._try_direct_parsing(extracted_part)
+                if result is not None:
+                    return result
+                result = self._try_regex_patterns(extracted_part)
+                if result is not None:
+                    return result
+
+        return None
+
+    def _try_aggressive_extraction(self, output: str) -> Optional[List[List[int]]]:
+        """更激进的数字提取方法，处理各种边界情况"""
+        try:
+            # 检查是否包含"空"、"无"、"none"等关键词
+            empty_keywords = ['空', '无', 'none', 'empty', 'null', '没有', 'no tents', 'no tent', 'zero tent']
+            for keyword in empty_keywords:
+                if keyword.lower() in output.lower():
+                    return []
+
+            # 尝试找到任何看起来像坐标的模式
+            # 模式：数字 数字 数字 数字... (连续的偶数个数字)
+            number_sequences = re.findall(r'(?:\d+\s*,?\s*){2,}', output)
+            for seq in number_sequences:
+                numbers = re.findall(r'\d+', seq)
+                if len(numbers) >= 2 and len(numbers) % 2 == 0:
+                    coords = []
+                    for i in range(0, len(numbers), 2):
+                        try:
+                            row, col = int(numbers[i]), int(numbers[i + 1])
+                            if 1 <= row <= 30 and 1 <= col <= 30:  # 合理的范围
+                                coords.append([row, col])
+                        except Exception:
+                            continue
+                    if coords:
+                        return coords
+
+            # 查找行列描述格式："第1行第2列"
+            chinese_pattern = r'第(\d+)行第(\d+)列'
+            matches = re.findall(chinese_pattern, output)
+            if matches:
+                coords = []
+                for match in matches:
+                    try:
+                        row, col = int(match[0]), int(match[1])
+                        if row > 0 and col > 0:
+                            coords.append([row, col])
+                    except Exception:
+                        continue
+                if coords:
+                    return coords
+
+            # 查找行列描述格式："row 1 column 2"
+            english_pattern = r'row\s+(\d+)\s+column\s+(\d+)'
+            matches = re.findall(english_pattern, output, re.IGNORECASE)
+            if matches:
+                coords = []
+                for match in matches:
+                    try:
+                        row, col = int(match[0]), int(match[1])
+                        if row > 0 and col > 0:
+                            coords.append([row, col])
+                    except Exception:
+                        continue
+                if coords:
+                    return coords
+
+            # 查找 R1C1 格式（R代表行，C代表列）
+            rc_pattern = r'R(\d+)C(\d+)'
+            matches = re.findall(rc_pattern, output, re.IGNORECASE)
+            if matches:
+                coords = []
+                for match in matches:
+                    try:
+                        row, col = int(match[0]), int(match[1])
+                        if row > 0 and col > 0:
+                            coords.append([row, col])
+                    except Exception:
+                        continue
+                if coords:
+                    return coords
+
+        except Exception:
+            pass
+
+        return None
+
+    def _try_mixed_formats(self, output: str) -> Optional[List[List[int]]]:
+        """处理混合格式和特殊边界情况"""
+        try:
+            # 处理用分号、管道符等分隔的格式
+            separators = [';', '|', '\n', '\t', '  ']
+            for sep in separators:
+                if sep in output:
+                    parts = output.split(sep)
+                    coords = []
+                    for part in parts:
+                        part = part.strip()
+                        if not part:
+                            continue
+                        # 尝试从每个部分提取坐标
+                        numbers = re.findall(r'\d+', part)
+                        if len(numbers) == 2:
+                            try:
+                                row, col = int(numbers[0]), int(numbers[1])
+                                if row > 0 and col > 0:
+                                    coords.append([row, col])
+                            except Exception:
+                                continue
+                    if coords:
+                        return coords
+
+            # 处理表格格式（尝试识别类似表格的结构）
+            lines = output.split('\n')
+            if len(lines) > 1:
+                coords = []
+                for line in lines:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    # 查找每行中的数字对
+                    numbers = re.findall(r'\d+', line)
+                    if len(numbers) == 2:
+                        try:
+                            row, col = int(numbers[0]), int(numbers[1])
+                            if row > 0 and col > 0:
+                                coords.append([row, col])
+                        except Exception:
+                            continue
+                if coords:
+                    return coords
+
+            # 处理单个数字对分隔的情况（没有括号）
+            # 例如："1 2 3 4" -> [[1,2], [3,4]]
+            numbers_only = re.findall(r'\b\d+\b', output)
+            if len(numbers_only) >= 2 and len(numbers_only) % 2 == 0 and len(numbers_only) <= 60:  # 最多30个坐标
+                coords = []
+                for i in range(0, len(numbers_only), 2):
+                    try:
+                        row, col = int(numbers_only[i]), int(numbers_only[i + 1])
+                        if 1 <= row <= 30 and 1 <= col <= 30:
+                            coords.append([row, col])
+                    except Exception:
+                        continue
+                # 只有当所有数字都能成功转换为合理坐标时才返回
+                if len(coords) == len(numbers_only) // 2:
+                    return coords
+
+        except Exception:
+            pass
+
+        return None
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Union[str, Dict]) -> bool:
+        """
+        验证预测答案是否正确，仅基于initial_state和游戏规则进行判断
+
+        Args:
+            predicted_answer: 模型的预测答案（可以是字符串或列表）
+            ground_truth: 标准答案（不使用，仅为接口兼容性保留）
+            initial_state: 初始状态包含网格和约束条件
+
+        Returns:
+            bool: 答案是否正确
+
+        Note:
+            此函数完全不依赖ground_truth参数，仅根据游戏规则和约束条件验证答案正确性
+        """
+        try:
+            # 解析初始状态
+            if isinstance(initial_state, str):
+                state = json.loads(initial_state)
+            else:
+                state = initial_state
+
+            # 提取游戏状态
+            input_grid = state["input_grid"]
+            row_constraints = state["row_constraints"]
+            col_constraints = state["col_constraints"]
+
+            # 提取预测答案中的坐标
+            if isinstance(predicted_answer, str):
+                predicted_coords = self.extract_answer(predicted_answer)
+                if predicted_coords is None:
+                    return False
+            else:
+                predicted_coords = self._validate_coordinate_format(predicted_answer)
+                if predicted_coords is None:
+                    return False
+
+            # 验证游戏规则
+            return self._validate_game_rules(predicted_coords, input_grid, row_constraints, col_constraints)
+
+        except Exception:
+            return False
+
+    def _validate_game_rules(
+            self, tent_coords: List[List[int]], input_grid: List[List[str]],
+            row_constraints: List[int], col_constraints: List[int]) -> bool:
+        """
+        验证帐篷位置是否符合游戏规则
+
+        Args:
+            tent_coords: 帐篷坐标列表 (1-based indexing)
+            input_grid: 游戏网格
+            row_constraints: 行约束
+            col_constraints: 列约束
+
+        Returns:
+            bool: 是否符合所有规则
+        """
+        if not tent_coords:
+            # 检查是否所有约束都为0
+            return all(c == 0 for c in row_constraints) and all(c == 0 for c in col_constraints)
+
+        rows, cols = len(input_grid), len(input_grid[0])
+
+        # 转换为0-based索引用于内部处理
+        tent_positions_0based = set()
+        for coord in tent_coords:
+            row, col = coord[0] - 1, coord[1] - 1  # 转换为0-based
+            tent_positions_0based.add((row, col))
+
+        # 规则验证1：检查所有帐篷位置是否在网格边界内
+        for row, col in tent_positions_0based:
+            if row < 0 or row >= rows or col < 0 or col >= cols:
+                return False
+
+        # 规则验证2：每个帐篷必须与至少一棵树正交相邻
+        directions = [(-1, 0), (1, 0), (0, -1), (0, 1)]  # 上下左右
+        for row, col in tent_positions_0based:
+            adjacent_to_tree = False
+            for dr, dc in directions:
+                nr, nc = row + dr, col + dc
+                if 0 <= nr < rows and 0 <= nc < cols:
+                    if input_grid[nr][nc] == 'T':
+                        adjacent_to_tree = True
+                        break
+            if not adjacent_to_tree:
+                return False
+
+        # 规则验证3：帐篷之间不能相邻（包括对角线）
+        all_directions = [(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 1), (1, -1), (1, 0), (1, 1)]
+        for row, col in tent_positions_0based:
+            for dr, dc in all_directions:
+                nr, nc = row + dr, col + dc
+                if (nr, nc) in tent_positions_0based:
+                    return False
+
+        # 规则验证4：检查行约束
+        actual_row_counts = [0] * rows
+        for row, col in tent_positions_0based:
+            actual_row_counts[row] += 1
+
+        if actual_row_counts != row_constraints:
+            return False
+
+        # 规则验证5：检查列约束
+        actual_col_counts = [0] * cols
+        for row, col in tent_positions_0based:
+            actual_col_counts[col] += 1
+
+        if actual_col_counts != col_constraints:
+            return False
+
+        return True
+
+    def get_detailed_feedback(
+            self, predicted_answer: Any, ground_truth: Any,
+            initial_state: Union[str, Dict]) -> Dict[str, Any]:
+        """
+        获取详细的验证反馈，用于调试和分析
+
+        Returns:
+            Dict包含详细的验证结果和错误信息
+        """
+        feedback = {
+            "is_correct": False,
+            "extracted_coords": None,
+            "extraction_successful": False,
+            "rule_violations": [],
+            "error_message": None
+        }
+
+        try:
+            # 解析初始状态
+            if isinstance(initial_state, str):
+                state = json.loads(initial_state)
+            else:
+                state = initial_state
+
+            input_grid = state["input_grid"]
+            row_constraints = state["row_constraints"]
+            col_constraints = state["col_constraints"]
+            rows, cols = len(input_grid), len(input_grid[0])
+
+            # 提取答案
+            if isinstance(predicted_answer, str):
+                predicted_coords = self.extract_answer(predicted_answer)
+            else:
+                predicted_coords = self._validate_coordinate_format(predicted_answer)
+
+            feedback["extracted_coords"] = predicted_coords
+            feedback["extraction_successful"] = predicted_coords is not None
+
+            if predicted_coords is None:
+                feedback["error_message"] = "无法从答案中提取有效的坐标"
+                return feedback
+
+            # 详细规则验证
+            if not predicted_coords:
+                # 空答案的情况
+                all_constraints_zero = (all(c == 0 for c in row_constraints)
+                                        and all(c == 0 for c in col_constraints))
+                if not all_constraints_zero:
+                    feedback["rule_violations"].append("空答案但约束条件不为0")
+                feedback["is_correct"] = all_constraints_zero
+                return feedback
+
+            tent_positions_0based = set()
+            for coord in predicted_coords:
+                row, col = coord[0] - 1, coord[1] - 1
+                tent_positions_0based.add((row, col))
+
+            # 检查边界
+            for row, col in tent_positions_0based:
+                if row < 0 or row >= rows or col < 0 or col >= cols:
+                    feedback["rule_violations"].append(f"帐篷位置 ({row+1}, {col+1}) 超出网格边界")
+
+            # 检查树邻接
+            directions = [(-1, 0), (1, 0), (0, -1), (0, 1)]
+            for row, col in tent_positions_0based:
+                adjacent_to_tree = False
+                for dr, dc in directions:
+                    nr, nc = row + dr, col + dc
+                    if 0 <= nr < rows and 0 <= nc < cols and input_grid[nr][nc] == 'T':
+                        adjacent_to_tree = True
+                        break
+                if not adjacent_to_tree:
+                    feedback["rule_violations"].append(f"帐篷 ({row+1}, {col+1}) 没有与树相邻")
+
+            # 检查帐篷相邻
+            all_directions = [(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 1), (1, -1), (1, 0), (1, 1)]
+            for row, col in tent_positions_0based:
+                for dr, dc in all_directions:
+                    nr, nc = row + dr, col + dc
+                    if (nr, nc) in tent_positions_0based:
+                        feedback["rule_violations"].append(f"帐篷 ({row+1}, {col+1}) 与帐篷 ({nr+1}, {nc+1}) 相邻")
+
+            # 检查行约束
+            actual_row_counts = [0] * rows
+            for row, col in tent_positions_0based:
+                actual_row_counts[row] += 1
+
+            for i, (actual, expected) in enumerate(zip(actual_row_counts, row_constraints)):
+                if actual != expected:
+                    feedback["rule_violations"].append(f"第{i+1}行帐篷数量错误: 实际{actual}, 期望{expected}")
+
+            # 检查列约束
+            actual_col_counts = [0] * cols
+            for row, col in tent_positions_0based:
+                actual_col_counts[col] += 1
+
+            for i, (actual, expected) in enumerate(zip(actual_col_counts, col_constraints)):
+                if actual != expected:
+                    feedback["rule_violations"].append(f"第{i+1}列帐篷数量错误: 实际{actual}, 期望{expected}")
+
+            feedback["is_correct"] = len(feedback["rule_violations"]) == 0
+
+        except Exception as e:
+            feedback["error_message"] = f"验证过程中发生错误: {str(e)}"
+
+        return feedback
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/cryptomath_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/cryptomath_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..aa07e5817792bc3262693954490dccf6882fdec2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/cryptomath_eval.py
@@ -0,0 +1,196 @@
+import ast
+import json
+import re
+from typing import Any, Dict, Union
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class CryptoMathEvaluator(BaseEvaluator):
+    """
+    评估字母算术谜题(CryptoMath)解答的评估器
+    验证模型输出的字母到数字映射是否：
+    1. 每个字母对应唯一数字(0-9)
+    2. 满足等式计算
+    3. 确保首位字母不为零
+    """
+
+    def extract_answer(self, model_output: str) -> Dict[str, int]:
+        """
+        从模型输出中提取字母到数字的映射
+
+        Args:
+            model_output: 模型生成的字符串输出
+
+        Returns:
+            字母到数字的映射字典，如 {'A': 1, 'B': 2, ...}
+        """
+        # 方法1：尝试直接使用ast.literal_eval解析Python字典格式
+        dict_pattern = r'\{[^{}]*\}'
+        dict_matches = re.search(dict_pattern, model_output)
+
+        if dict_matches:
+            try:
+                dict_str = dict_matches.group(0)
+                mapping = ast.literal_eval(dict_str)
+                # 确保是字典且值是整数类型
+                if isinstance(mapping, dict):
+                    mapping = {k: int(v) for k, v in mapping.items() if isinstance(k, str) and str(k).isalpha()}
+                    return mapping
+            except Exception:
+                pass
+
+        # 方法2：尝试找到JSON对象格式
+        json_pattern = r'\{(?:\s*[\'\"]([A-Za-z])[\'\"]:\s*(\d+)\s*,?\s*)+\}'
+        json_matches = re.search(json_pattern, model_output)
+
+        if json_matches:
+            # 尝试提取完整的JSON对象并解析
+            try:
+                json_str = json_matches.group(0)
+                mapping = json.loads(json_str)
+                # 确保值是整数类型
+                mapping = {k: int(v) for k, v in mapping.items()}
+                return mapping
+            except Exception:
+                pass
+
+        # 方法3：查找形如 "S"=9,"E"=5,... 的格式
+        bracket_pattern = r'\[\s*(?:[\'\"]([A-Za-z])[\'\"]=\"?(\d+)\"?\s*,?\s*)+\]'
+        bracket_matches = re.search(bracket_pattern, model_output)
+
+        if bracket_matches:
+            mapping = {}
+            for match in re.finditer(r'[\'\"]([A-Za-z])[\'\"]=\"?(\d+)\"?', model_output):
+                letter, digit = match.groups()
+                mapping[letter] = int(digit)
+            return mapping
+
+        # 方法4：查找形如 "A=1, B=2, ..." 的格式
+        eq_pattern = r'([A-Za-z])\s*=\s*(\d+)'
+        eq_matches = re.findall(eq_pattern, model_output)
+
+        if eq_matches:
+            mapping = {letter: int(digit) for letter, digit in eq_matches}
+            return mapping
+
+        # 方法5：尝试在文本中查找明确的赋值语句
+        text_pattern = r'([A-Za-z])\s+is\s+(\d+)'
+        text_matches = re.findall(text_pattern, model_output)
+
+        if text_matches:
+            mapping = {letter: int(digit) for letter, digit in text_matches}
+            return mapping
+
+        # 如果没有找到有效的映射，返回空字典
+        return {}
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """
+        准备用于解决字母算术谜题的提示词
+
+        Args:
+            question: 问题描述
+            params: 包含谜题信息的参数，如等式
+
+        Returns:
+            格式化的提示词
+        """
+        equation = params.get("equation", "")
+        if not equation:
+            return question
+
+        # 提取等式中的单词（不需要用到，只是为了文档说明）
+        # words = re.findall(r'[A-Za-z]+', equation)
+
+        prompt = f"""请解决以下字母算术谜题（也称为字谜算术或cryptarithmetic）:
+
+    {equation}
+
+    在这个谜题中:
+    1. 每个字母代表0到9之间的一个唯一数字
+    2. 没有两个字母可以代表相同的数字
+    3. 等式必须在数学上成立
+    4. 每个单词的第一个字母不能为0
+
+    你的任务是找出每个字母对应的数字，使得等式成立。
+
+    请给出字母到数字的映射，格式如: {{"A": 1, "B": 2, ...}}
+
+    解题过程:
+    1. 分析等式中的限制条件
+    2. 确定每个字母可能的取值
+    3. 列出你的推理步骤
+    4. 提供最终的字母到数字映射
+    """
+
+        return prompt
+
+    def evaluate(self, output: Union[str, Dict[str, int]], ground_truth: Dict[str, int],
+                 params: Dict[str, Any]) -> bool:
+        """
+        评估模型的解答是否正确，仅基于规则验证
+
+        Args:
+            output: 模型的原始输出字符串或已提取的字母到数字映射
+            ground_truth: 正确的字母到数字映射（仅用于参考，不直接比对）
+            params: 其他参数，包括等式
+
+        Returns:
+            解答是否正确的布尔值
+        """
+        if isinstance(output, str):
+            predicted_mapping = self.extract_answer(output)
+        else:
+            predicted_mapping = output
+
+        # 提取等式
+        equation = params.get("equation", "")
+        if not equation:
+            return False  # 如果没有提供等式，无法验证
+
+        # 提取等式中的所有字母
+        all_letters = set(re.findall(r'[A-Za-z]', equation))
+
+        # 1. 检查是否包含等式中的所有字母
+        if not all(letter in predicted_mapping for letter in all_letters):
+            return False
+
+        # 2. 检查每个字母对应唯一数字(0-9)
+        if len(set(predicted_mapping.values())) != len(predicted_mapping):
+            return False  # 有重复数字
+
+        if any(not isinstance(digit, int) or digit < 0 or digit > 9 for digit in predicted_mapping.values()):
+            return False  # 有非法数字
+
+        # 3. 检查首位字母不为零
+        words = re.findall(r'[A-Za-z]+', equation)
+        leading_letters = [word[0] for word in words]
+
+        for letter in leading_letters:
+            if letter in predicted_mapping and predicted_mapping[letter] == 0:
+                return False
+
+        # 4. 验证等式是否成立
+        eval_equation = equation
+        for letter, digit in predicted_mapping.items():
+            eval_equation = eval_equation.replace(letter, str(digit))
+
+        try:
+            left_side, right_side = eval_equation.split('=')
+            if eval(left_side) != eval(right_side):
+                return False
+        except Exception:
+            return False  # 等式计算出错
+
+        # 所有条件均满足，返回True
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/eulero_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/eulero_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..2fece8efb2e9bbca201fecb683b5fa5e6061be49
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/eulero_eval.py
@@ -0,0 +1,348 @@
+import json
+import re
+from typing import Any, Dict, List, Optional
+
+
+def _default_prompt_tpl(q: str) -> str:
+    return (f"Please solve the Eulero puzzle. Return the grid in rows separated by newlines and cells by '|'.\n"
+            f"Question:\n{q}\n<answer>...</answer>")
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self,
+        predicted_answer: Any,
+        ground_truth: Any,
+        initial_state: Any,
+        params: Dict[str, Any] = None
+    ) -> bool:
+        raise NotImplementedError
+
+
+class EuleroEvaluator(BaseEvaluator):
+    # -----------------------
+    # Public API
+    # -----------------------
+    def prepare_prompt(self, question: str, params: Dict[str, Any] = None) -> str:
+        if params and isinstance(params, dict) and "prompt_template" in params:
+            return str(params["prompt_template"]).format(question=question)
+        try:
+            from utils.constants import PROMPT_EULERO
+            return PROMPT_EULERO.format(question)
+        except Exception:
+            return _default_prompt_tpl(question)
+
+    def extract_answer(self, model_output: str) -> str:
+        """
+        从模型输出中提取网格（支持 <answer> 标签、任意空白、大小写、任意 N×N）。
+        返回标准化的以 '|' 分隔、行以 '\n' 分隔的文本。
+        """
+        if not isinstance(model_output, str):
+            model_output = str(model_output)
+
+        # 优先从 <answer>...</answer> 中取
+        m = re.search(r"<answer>(.*?)</answer>", model_output, re.DOTALL | re.IGNORECASE)
+        content = (m.group(1) if m else model_output).strip()
+
+        # 若文本中存在明确的按行-按'|'分隔的结构，则直接规范化
+        if self._looks_like_piped_grid(content):
+            return self._normalize_grid(content)
+
+        # 否则从全文提取 A1/B2 等对并按行聚合（尽力）
+        pairs = re.findall(r"[A-Za-z]\d+", content)
+        if not pairs:
+            return ""
+
+        # 估计网格大小（letters 与 numbers 的基数的 max）
+        letters = {p[0].upper() for p in pairs}
+        numbers = {p[1:] for p in pairs}
+        n = max(len(letters), len(numbers))
+        if n == 0:
+            return ""
+
+        expected = n * n
+        if len(pairs) < expected:
+            # 数据不足，尽力而为但返回空表示无法构出完整网格
+            return ""
+
+        # 取前 n^2 个，按行输出（大写化字母）
+        canon_rows = []
+        idx = 0
+        for _ in range(n):
+            row_pairs = [pairs[idx + j] for j in range(n)]
+            row_pairs = [rp[0].upper() + rp[1:] for rp in row_pairs]
+            canon_rows.append("|".join(row_pairs))
+            idx += n
+        return "\n".join(canon_rows)
+
+    def evaluate(
+        self,
+        predicted_answer: str,
+        ground_truth: Any,
+        initial_state: Any,
+        params: Dict[str, Any] = None
+    ) -> bool:
+        """
+        规则：
+        1) 每格均为合法的 Letter+Number；
+        2) 每行/列 字母各出现一次；
+        3) 每行/列 数字各出现一次；
+        4) 全局 pair 唯一；
+        5) 尊重预填（initial_state）。
+        """
+        # 1) 解析预测网格
+        predicted_grid = self._parse_grid(predicted_answer)
+        if not predicted_grid:
+            return False
+
+        n = len(predicted_grid)
+
+        # 2) 解析/矫正 初始网格
+        initial_grid = self._parse_initial_state(initial_state)
+
+        if not initial_grid:
+            # 解析不到时，默认同尺寸全 None
+            initial_grid = [[None] * n for _ in range(n)]
+        else:
+            # 若尺寸不一致，尝试矫正到 n×n
+            initial_grid = self._coerce_to_size(initial_grid, n)
+
+        if len(initial_grid) != n or any(len(r) != n for r in initial_grid):
+            return False
+
+        # 3) 规则校验
+        if not self._check_valid_pairs(predicted_grid, n):
+            return False
+
+        if not self._check_letters_unique_in_rows_and_columns(predicted_grid):
+            return False
+
+        if not self._check_numbers_unique_in_rows_and_columns(predicted_grid):
+            return False
+
+        if not self._check_unique_pairs(predicted_grid):
+            return False
+
+        if not self._check_respects_initial_state(predicted_grid, initial_grid):
+            return False
+
+        return True
+
+    # -----------------------
+    # Parsing helpers
+    # -----------------------
+    def _looks_like_piped_grid(self, s: str) -> bool:
+        lines = [ln for ln in re.split(r"\r?\n", s) if ln.strip() != ""]
+        return any("|" in ln for ln in lines)
+
+    def _normalize_grid(self, grid_str: str) -> str:
+        """
+        规范化：去掉行首尾空白、删除纯空行、保留 '|' 分隔。
+        """
+        if not grid_str:
+            return ""
+        lines = [line.strip() for line in re.split(r"\r?\n", grid_str) if line.strip() != ""]
+        return "\n".join(lines)
+
+    def _parse_grid(self, grid_str: str) -> List[List[str]]:
+        """
+        解析预测解网格（严格：每行必须能解析出 A1、B2... 且每行等长）。
+        """
+        if not grid_str:
+            return []
+
+        normalized = self._normalize_grid(grid_str)
+        if not normalized:
+            return []
+
+        rows: List[List[str]] = []
+        for raw in normalized.split("\n"):
+            # 优先用 '|' 分割，兼容无 '|' 时用正则抽取
+            if "|" in raw:
+                cells = [c.strip() for c in re.split(r"\s*\|\s*", raw)]
+            else:
+                cells = re.findall(r"[A-Za-z]\d+", raw)
+
+            # 每个单元需是合法 pair
+            parsed = []
+            for cell in cells:
+                if re.fullmatch(r"[A-Za-z]\d+", cell):
+                    parsed.append(cell[0].upper() + cell[1:])  # 统一大写字母
+                else:
+                    # 预测解中出现非 pair，视为解析失败
+                    parsed = []
+                    break
+            if not parsed:
+                return []
+            rows.append(parsed)
+
+        # 行等长校验
+        if not rows or not all(len(r) == len(rows[0]) for r in rows):
+            return []
+        return rows
+
+    def _parse_initial_state(self, initial_state: Any) -> List[List[Optional[str]]]:
+        """
+        解析初始网格：优先逐格（'|'）解析，保留空格行与空单元。
+        接受：
+          - 带 '|' 的字符串（空位可为 '', '_', '__', '-', '--', ' '）
+          - JSON 字符串（会递归解析）
+          - 2D 列表（元素为 None/''/合法 pair）
+        """
+        # 2D 列表
+        if isinstance(initial_state, list) and all(isinstance(r, list) for r in initial_state):
+            return self._normalize_initial_list(initial_state)
+
+        # 字符串
+        if isinstance(initial_state, str):
+            s = initial_state.strip("\n\r")
+
+            # 1) 优先：逐格按 '|' 解析（保留空）
+            lines = [ln.strip() for ln in s.splitlines() if True]  # 不丢空行
+            if any("|" in ln for ln in lines):
+                grid: List[List[Optional[str]]] = []
+                for ln in lines:
+                    cells = re.split(r"\s*\|\s*", ln.strip())
+                    row: List[Optional[str]] = []
+                    for cell in cells:
+                        token = cell.strip()
+                        if re.fullmatch(r"[A-Za-z]\d+", token):
+                            row.append(token[0].upper() + token[1:])
+                        elif token in {"", "_", "__", "-", "--", " "}:
+                            row.append(None)
+                        else:
+                            # 未知记号按空处理
+                            row.append(None)
+                    if row:
+                        grid.append(row)
+                # 行长不一致先返回，后续会做尺寸矫正
+                return grid if grid else []
+
+            # 2) 尝试 JSON
+            try:
+                data = json.loads(s)
+                return self._parse_initial_state(data)
+            except Exception:
+                pass
+
+            # 3) 退回：像解析预测解那样的网格（但空位不可识别）
+            # 若按此法，空行会被丢弃；建议仅作为最后兜底
+            grid_like = self._parse_grid(s)
+            if grid_like:
+                # 将所有单元保留（预测式解析不会产生空），符合“全预填”的含义
+                return grid_like
+
+        # 其它类型或失败
+        return []
+
+    def _normalize_initial_list(self, grid: List[List[Any]]) -> List[List[Optional[str]]]:
+        """
+        将任意 2D 列表中的元素规范化为合法 pair 或 None。
+        """
+        norm: List[List[Optional[str]]] = []
+        for row in grid:
+            new_row: List[Optional[str]] = []
+            for cell in row:
+                if cell is None:
+                    new_row.append(None)
+                else:
+                    token = str(cell).strip()
+                    if re.fullmatch(r"[A-Za-z]\d+", token):
+                        new_row.append(token[0].upper() + token[1:])
+                    elif token in {"", "_", "__", "-", "--"}:
+                        new_row.append(None)
+                    else:
+                        new_row.append(None)
+            norm.append(new_row)
+        return norm
+
+    def _coerce_to_size(self, grid: List[List[Optional[str]]], n: int) -> List[List[Optional[str]]]:
+        """
+        将初始网格强制矫正为 n×n：
+        - 行数不足：在末尾补 None 行；
+        - 列数不足：各行右侧补 None；
+        - 行/列超出：裁剪（并打印提示）。
+        """
+        rows = len(grid)
+        if rows < n:
+            grid = grid + [[None] * (max(len(r) for r in grid) if grid else n) for _ in range(n - rows)]
+        elif rows > n:
+            grid = grid[:n]
+
+        # 统一列数到 n
+        coerced: List[List[Optional[str]]] = []
+        for r in grid:
+            cols = len(r)
+            if cols < n:
+                coerced.append(r + [None] * (n - cols))
+            elif cols > n:
+                coerced.append(r[:n])
+            else:
+                coerced.append(r)
+        return coerced
+
+    # -----------------------
+    # Rule checks
+    # -----------------------
+    def _check_valid_pairs(self, grid: List[List[str]], n: int) -> bool:
+        valid_letters = {chr(ord('A') + i) for i in range(n)}
+        valid_numbers = {str(i + 1) for i in range(n)}
+        for row in grid:
+            for cell in row:
+                if not re.fullmatch(r"[A-Z]\d+", cell):
+                    return False
+                if cell[0] not in valid_letters or cell[1:] not in valid_numbers:
+                    return False
+        return True
+
+    def _check_letters_unique_in_rows_and_columns(self, grid: List[List[str]]) -> bool:
+        n = len(grid)
+        # rows
+        for row in grid:
+            letters = [c[0] for c in row]
+            if len(letters) != len(set(letters)):
+                return False
+        # cols
+        for c in range(n):
+            letters = [grid[r][c][0] for r in range(n)]
+            if len(letters) != len(set(letters)):
+                return False
+        return True
+
+    def _check_numbers_unique_in_rows_and_columns(self, grid: List[List[str]]) -> bool:
+        n = len(grid)
+        # rows
+        for row in grid:
+            nums = [c[1:] for c in row]
+            if len(nums) != len(set(nums)):
+                return False
+        # cols
+        for c in range(n):
+            nums = [grid[r][c][1:] for r in range(n)]
+            if len(nums) != len(set(nums)):
+                return False
+        return True
+
+    def _check_unique_pairs(self, grid: List[List[str]]) -> bool:
+        flat = [c for row in grid for c in row]
+        return len(flat) == len(set(flat))
+
+    def _check_respects_initial_state(
+        self,
+        predicted_grid: List[List[str]],
+        initial_grid: List[List[Optional[str]]]
+    ) -> bool:
+        n = len(predicted_grid)
+        for r in range(n):
+            for c in range(n):
+                v0 = initial_grid[r][c]
+                if v0 is not None and v0 != "":
+                    if predicted_grid[r][c] != v0:
+                        return False
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/futoshiki_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/futoshiki_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..2a04999dc69faa7329daa26835035404094c2dee
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/futoshiki_eval.py
@@ -0,0 +1,509 @@
+import ast
+import json
+import re
+from typing import Any, Dict, List
+
+
+class FutoshikiEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """Format the question for the model."""
+        prompt = (
+            "Solve the Futoshiki puzzle described below. A Futoshiki puzzle uses a grid where:\n"
+            "1. Each row and column must contain each number exactly once (like Sudoku)\n"
+            "2. Inequality signs (< and >) between cells must be satisfied\n\n"
+            f"{question}\n\n"
+            "Provide your answer as a grid of numbers. Format your answer as a list of lists, "
+            "where each inner list represents a row of the grid."
+        )
+        return prompt
+
+    def extract_answer(self, model_output: str) -> List[List[int]]:
+        """Extract the grid solution from the model's output with enhanced robustness."""
+        if not isinstance(model_output, str):
+            model_output = str(model_output)
+
+        # Method 1: Try to find and parse nested list structures
+        grid = self._extract_nested_lists(model_output)
+        if grid:
+            return grid
+
+        # Method 2: Try to parse as Python literal
+        grid = self._extract_python_literal(model_output)
+        if grid:
+            return grid
+
+        # Method 3: Extract from table-like format
+        grid = self._extract_table_format(model_output)
+        if grid:
+            return grid
+
+        # Method 4: Extract from comma/space separated format
+        grid = self._extract_separated_format(model_output)
+        if grid:
+            return grid
+
+        # Method 5: Extract all numbers and try to infer grid structure
+        grid = self._extract_inferred_grid(model_output)
+        if grid:
+            return grid
+
+        # Method 6: Handle single line formats
+        grid = self._extract_single_line_format(model_output)
+        if grid:
+            return grid
+
+        return []
+
+    def _extract_nested_lists(self, text: str) -> List[List[int]]:
+        """Extract nested list structures like [[1,2,3],[4,5,6]]"""
+        # Enhanced pattern to match various nested list formats
+        patterns = [
+            r'\[\s*(?:\[\s*(?:\d+(?:\s*,\s*\d+)*)\s*\](?:\s*,\s*)?)+\s*\]',  # Standard format
+            r'\[(?:\s*\[\s*\d+(?:\s*,\s*\d+)*\s*\](?:\s*,\s*)?)+\s*\]',      # Compact format
+            r'\[\s*\[[\d\s,]+\](?:\s*,\s*\[[\d\s,]+\])*\s*\]'                # Flexible spacing
+        ]
+
+        for pattern in patterns:
+            matches = re.findall(pattern, text)
+            for match in matches:
+                try:
+                    grid = ast.literal_eval(match)
+                    if self._is_valid_grid(grid):
+                        return grid
+                except Exception:
+                    continue
+        return None
+
+    def _extract_python_literal(self, text: str) -> List[List[int]]:
+        """Try to extract and evaluate Python literal expressions"""
+        # Find potential list structures
+        bracket_matches = []
+        stack = []
+        start = -1
+
+        for i, char in enumerate(text):
+            if char == '[':
+                if not stack:
+                    start = i
+                stack.append(char)
+            elif char == ']':
+                if stack:
+                    stack.pop()
+                    if not stack and start != -1:
+                        bracket_matches.append(text[start:i + 1])
+
+        # Try to evaluate each match
+        for match in bracket_matches:
+            try:
+                grid = ast.literal_eval(match)
+                if self._is_valid_grid(grid):
+                    return grid
+            except Exception:
+                continue
+
+        return None
+
+    def _extract_table_format(self, text: str) -> List[List[int]]:
+        """Extract from table-like formats with | or other separators"""
+        lines = text.strip().split('\n')
+        grid = []
+
+        for line in lines:
+            # Skip empty lines and lines without numbers
+            if not line.strip():
+                continue
+
+            # Handle different table formats
+            row_numbers = []
+
+            # Try different separators
+            for separator in ['|', '\t', '  ', ' ']:
+                if separator in line:
+                    parts = line.split(separator)
+                    numbers = []
+                    for part in parts:
+                        nums = re.findall(r'\d+', part.strip())
+                        numbers.extend([int(n) for n in nums])
+                    if numbers:
+                        row_numbers = numbers
+                        break
+
+            # If no separator worked, extract all numbers from the line
+            if not row_numbers:
+                row_numbers = [int(n) for n in re.findall(r'\d+', line)]
+
+            if row_numbers:
+                grid.append(row_numbers)
+
+        if self._is_valid_grid(grid):
+            return grid
+
+        return None
+
+    def _extract_separated_format(self, text: str) -> List[List[int]]:
+        """Extract from comma or space separated values"""
+        lines = text.strip().split('\n')
+        grid = []
+
+        for line in lines:
+            # Skip lines that don't contain numbers
+            if not re.search(r'\d', line):
+                continue
+
+            # Remove brackets and extract numbers
+            clean_line = re.sub(r'[\[\]()]', '', line)
+
+            # Try comma separation first
+            if ',' in clean_line:
+                numbers = []
+                for part in clean_line.split(','):
+                    nums = re.findall(r'\d+', part)
+                    numbers.extend([int(n) for n in nums])
+            else:
+                # Space separation
+                numbers = [int(n) for n in re.findall(r'\d+', clean_line)]
+
+            if numbers:
+                grid.append(numbers)
+
+        if self._is_valid_grid(grid):
+            return grid
+
+        return None
+
+    def _extract_single_line_format(self, text: str) -> List[List[int]]:
+        """Extract from single line formats like '1 2 3 4 5 1 2 3 4 5 ...'"""
+        # Extract all numbers from the text
+        all_numbers = [int(n) for n in re.findall(r'\d+', text)]
+
+        if not all_numbers:
+            return None
+
+        # Try different grid sizes
+        possible_sizes = []
+        total_numbers = len(all_numbers)
+
+        # Check perfect squares
+        for size in range(2, int(total_numbers**0.5) + 2):
+            if size * size == total_numbers:
+                possible_sizes.append(size)
+
+        # Check rectangles
+        for rows in range(2, total_numbers // 2 + 1):
+            if total_numbers % rows == 0:
+                cols = total_numbers // rows
+                if cols >= 2:
+                    possible_sizes.append((rows, cols))
+
+        # Try each possible size
+        for size in possible_sizes:
+            if isinstance(size, int):
+                # Square grid
+                grid = []
+                for i in range(size):
+                    grid.append(all_numbers[i * size:(i + 1) * size])
+                if self._is_valid_grid(grid):
+                    return grid
+            else:
+                # Rectangular grid
+                rows, cols = size
+                grid = []
+                for i in range(rows):
+                    grid.append(all_numbers[i * cols:(i + 1) * cols])
+                if self._is_valid_grid(grid):
+                    return grid
+
+        return None
+
+    def _extract_inferred_grid(self, text: str) -> List[List[int]]:
+        """Extract numbers and try to infer the grid structure from context"""
+        # Look for hints about grid size in the text
+        size_hints = re.findall(r'(\d+)\s*[×x]\s*(\d+)', text)
+        if size_hints:
+            rows, cols = int(size_hints[0][0]), int(size_hints[0][1])
+        else:
+            # Look for patterns that suggest grid size
+            lines_with_numbers = []
+            for line in text.split('\n'):
+                numbers = re.findall(r'\d+', line)
+                if numbers:
+                    lines_with_numbers.append(len(numbers))
+
+            if lines_with_numbers:
+                # Use the most common line length as columns
+                cols = max(set(lines_with_numbers), key=lines_with_numbers.count)
+                rows = len(lines_with_numbers)
+            else:
+                # Extract all numbers and assume square grid
+                all_numbers = [int(n) for n in re.findall(r'\d+', text)]
+                if not all_numbers:
+                    return None
+
+                size = int(len(all_numbers) ** 0.5)
+                if size * size == len(all_numbers):
+                    rows = cols = size
+                else:
+                    return None
+
+        # Extract all numbers and arrange in grid
+        all_numbers = [int(n) for n in re.findall(r'\d+', text)]
+        if len(all_numbers) != rows * cols:
+            return None
+
+        grid = []
+        for i in range(rows):
+            grid.append(all_numbers[i * cols:(i + 1) * cols])
+
+        if self._is_valid_grid(grid):
+            return grid
+
+        return None
+
+    def _is_valid_grid(self, grid) -> bool:
+        """Check if the extracted grid is valid"""
+        if not isinstance(grid, list) or not grid:
+            return False
+
+        if not all(isinstance(row, list) for row in grid):
+            return False
+
+        if not all(len(row) == len(grid[0]) for row in grid):
+            return False
+
+        # Check if all elements are integers
+        try:
+            for row in grid:
+                for cell in row:
+                    int(cell)
+        except (ValueError, TypeError):
+            return False
+
+        # Grid should be at least 2x2
+        if len(grid) < 2 or len(grid[0]) < 2:
+            return False
+
+        return True
+
+    def _normalize_grid(self, answer_str_or_list):
+        """Normalize the input to a grid of integers."""
+        if isinstance(answer_str_or_list, list):
+            # If it's already a list, check if it's a valid grid
+            if all(isinstance(row, list) for row in answer_str_or_list):
+                return answer_str_or_list
+            return None
+
+        # If it's a string, use extract_answer method
+        return self.extract_answer(str(answer_str_or_list))
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Dict[str, Any]) -> bool:
+        """
+        Evaluate the predicted answer based on Futoshiki game rules and initial state.
+
+        Args:
+            predicted_answer: The model's output (string or list)
+            ground_truth: The ground truth solution (not used in rule-based evaluation)
+            initial_state: Dictionary containing:
+                - 'grid': 2D list with initial numbers (0 for empty cells)
+                - 'inequalities': List of inequality constraints
+                - 'size': Grid size
+
+        Returns:
+            bool: True if the predicted answer satisfies all Futoshiki rules
+        """
+        try:
+            # Extract the predicted grid using robust parsing
+            if isinstance(predicted_answer, str):
+                predicted_grid = self.extract_answer(predicted_answer)
+            elif isinstance(predicted_answer, list):
+                if self._is_valid_grid(predicted_answer):
+                    predicted_grid = predicted_answer
+                else:
+                    return False
+            else:
+                predicted_grid = self.extract_answer(str(predicted_answer))
+
+            # If extraction failed, return False
+            if not predicted_grid:
+                return False
+
+            # Parse initial state (support dict or JSON/string input)
+            if not isinstance(initial_state, dict):
+                try:
+                    if isinstance(initial_state, str):
+                        try:
+                            initial_state = json.loads(initial_state)
+                        except json.JSONDecodeError:
+                            # Fallback to Python literal (handles single quotes)
+                            initial_state = ast.literal_eval(initial_state)
+                    else:
+                        # Unsupported type
+                        return False
+                except Exception:
+                    return False
+
+            initial_grid = initial_state.get('grid', [])
+            inequalities = initial_state.get('inequalities', [])
+            expected_size = initial_state.get('size', len(initial_grid))
+
+            # Validate grid dimensions
+            if len(predicted_grid) != expected_size:
+                return False
+
+            if any(len(row) != expected_size for row in predicted_grid):
+                return False
+
+            # Convert to integers and validate range
+            try:
+                predicted_grid = [[int(cell) for cell in row] for row in predicted_grid]
+            except (ValueError, TypeError):
+                return False
+
+            # Check if all numbers are in valid range [1, n]
+            for row in predicted_grid:
+                for cell in row:
+                    if cell < 1 or cell > expected_size:
+                        return False
+
+            # Rule 1: Check if predicted grid preserves initial numbers
+            if not self._check_initial_numbers(predicted_grid, initial_grid):
+                return False
+
+            # Rule 2: Check if each row contains each number exactly once
+            if not self._check_rows_unique(predicted_grid):
+                return False
+
+            # Rule 3: Check if each column contains each number exactly once
+            if not self._check_columns_unique(predicted_grid):
+                return False
+
+            # Rule 4: Check if all inequality constraints are satisfied
+            if not self._check_inequalities(predicted_grid, inequalities):
+                return False
+
+            return True
+
+        except Exception:
+            return False
+
+    def _check_initial_numbers(self, predicted_grid: List[List[int]], initial_grid: List[List[int]]) -> bool:
+        """Check if the predicted grid preserves all initial numbers."""
+        if len(predicted_grid) != len(initial_grid):
+            return False
+
+        for i in range(len(initial_grid)):
+            if len(predicted_grid[i]) != len(initial_grid[i]):
+                return False
+            for j in range(len(initial_grid[i])):
+                # If there was an initial number (not 0), it must be preserved
+                if initial_grid[i][j] != 0 and predicted_grid[i][j] != initial_grid[i][j]:
+                    return False
+
+        return True
+
+    def _check_rows_unique(self, grid: List[List[int]]) -> bool:
+        """Check if each row contains each number from 1 to n exactly once."""
+        n = len(grid)
+        expected_set = set(range(1, n + 1))
+
+        for row in grid:
+            if set(row) != expected_set:
+                return False
+
+        return True
+
+    def _check_columns_unique(self, grid: List[List[int]]) -> bool:
+        """Check if each column contains each number from 1 to n exactly once."""
+        n = len(grid)
+        expected_set = set(range(1, n + 1))
+
+        for j in range(n):
+            column = [grid[i][j] for i in range(n)]
+            if set(column) != expected_set:
+                return False
+
+        return True
+
+    def _check_inequalities(self, grid: List[List[int]], inequalities: List[Dict]) -> bool:
+        """Check if all inequality constraints are satisfied."""
+        for inequality in inequalities:
+            try:
+                # Get cell positions
+                cell1 = inequality['cell1']
+                cell2 = inequality['cell2']
+                symbol = inequality['symbol']
+
+                # Get cell values
+                i1, j1 = cell1[0], cell1[1]
+                i2, j2 = cell2[0], cell2[1]
+
+                value1 = grid[i1][j1]
+                value2 = grid[i2][j2]
+
+                # Check inequality constraint
+                if symbol == '<':
+                    if value1 >= value2:
+                        return False
+                elif symbol == '>':
+                    if value1 <= value2:
+                        return False
+                else:
+                    # Invalid symbol
+                    return False
+
+            except (KeyError, IndexError, TypeError):
+                # Invalid inequality format
+                return False
+
+        return True
+
+    # Legacy method for backward compatibility
+    def old_evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state=None) -> bool:
+        """
+        Legacy evaluation method that compares predicted answer with ground truth.
+        """
+        # Parse ground truth
+        try:
+            if isinstance(ground_truth, str):
+                gt_grid = ast.literal_eval(ground_truth)
+            else:
+                gt_grid = ground_truth
+        except (SyntaxError, ValueError):
+            return False
+
+        if not isinstance(gt_grid, list) or not all(isinstance(row, list) for row in gt_grid):
+            return False
+
+        # Extract predicted answer using the robust extract_answer method
+        if isinstance(predicted_answer, str):
+            predicted_grid = self.extract_answer(predicted_answer)
+        elif isinstance(predicted_answer, list):
+            # If it's already a list, validate it
+            if self._is_valid_grid(predicted_answer):
+                predicted_grid = predicted_answer
+            else:
+                predicted_grid = []
+        else:
+            # Convert to string and extract
+            predicted_grid = self.extract_answer(str(predicted_answer))
+
+        # If extraction failed, return False
+        if not predicted_grid:
+            return False
+
+        # Check dimensions
+        if len(predicted_grid) != len(gt_grid):
+            return False
+
+        if any(len(row) != len(gt_grid[0]) for row in predicted_grid):
+            return False
+
+        # Compare grids element by element
+        try:
+            for i in range(len(gt_grid)):
+                for j in range(len(gt_grid[0])):
+                    if int(predicted_grid[i][j]) != int(gt_grid[i][j]):
+                        return False
+        except (ValueError, TypeError, IndexError):
+            return False
+
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/graph_problems_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/graph_problems_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..8beb57686df2ed7f9b5b5425097ba21e90ea3fbd
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/graph_problems_eval.py
@@ -0,0 +1,984 @@
+import ast
+import json
+from typing import Any, Dict, List
+
+
+def safe_parse_answer(answer_str: str, verbose: bool = False):
+    """
+    Safely parse answer string that could be JSON or Python list format
+    """
+    if answer_str.strip().lower() == "no":
+        return None
+
+    # First try JSON parsing
+    try:
+        return json.loads(answer_str)
+    except json.JSONDecodeError:
+        if verbose:
+            print(f"JSON parsing failed, trying literal evaluation for: {repr(answer_str)}")
+
+        # Try ast.literal_eval for safe evaluation of Python literals
+        try:
+            result = ast.literal_eval(answer_str)
+            if verbose:
+                print(f"Literal evaluation successful: {result}")
+            return result
+        except (ValueError, SyntaxError) as e:
+            if verbose:
+                print(f"Literal evaluation failed: {e}")
+            return None
+
+
+class HamiltonianPathEvaluator:
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate if the predicted answer is a valid Hamiltonian path
+
+        Args:
+            predicted_answer: Predicted path, could be a list (e.g., [4,5,3,1,2,0]) or string "No"
+            ground_truth: Ground truth answer (for reference)
+            initial_state: Graph adjacency list representation, format like {'0': [2], '1': [2, 3], '2': [0, 1], ...}
+
+        Returns:
+            bool: Whether the predicted answer is correct
+        """
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                graph = safe_parse_answer(initial_state, self.verbose)
+            else:
+                graph = initial_state
+
+            # Parse predicted answer
+            if isinstance(predicted_answer, str):
+                predicted_path = safe_parse_answer(predicted_answer, self.verbose)
+                if predicted_path is None and predicted_answer.strip().lower() != "no":
+                    if self.verbose:
+                        print(f"❌ Hamiltonian Path: Cannot parse predicted answer '{predicted_answer}'")
+                    return False
+            else:
+                predicted_path = predicted_answer
+
+            # Parse ground truth
+            if isinstance(ground_truth, str):
+                expected_path = safe_parse_answer(ground_truth, self.verbose)
+            else:
+                expected_path = ground_truth
+
+            # If ground truth is "No", check if predicted answer is also "No"
+            if expected_path is None:
+                if predicted_path is not None:
+                    if self.verbose:
+                        print(f"❌ Hamiltonian Path: Ground truth is 'No', but predicted answer is {predicted_path}")
+                    return False
+                return True
+
+            # If predicted answer is "No" but ground truth is not, then error
+            if predicted_path is None:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Path: Predicted answer is 'No', but ground truth is {expected_path}")
+                return False
+
+            # Validate if predicted path is a valid Hamiltonian path
+            validation_result = self._is_valid_hamiltonian_path(predicted_path, graph)
+            if not validation_result:
+                return False
+
+            if self.verbose:
+                print(f"✅ Hamiltonian Path Evaluation Passed: {predicted_path}")
+            return True
+
+        except Exception as e:
+            if self.verbose:
+                print(f"❌ Hamiltonian Path Evaluation Exception: {str(e)}")
+            return False
+
+    def _is_valid_hamiltonian_path(self, path: List, graph: Dict) -> bool:
+        """
+        Validate if the given path is a valid Hamiltonian path
+        """
+        if not path or not isinstance(path, list):
+            if self.verbose:
+                print("❌ Hamiltonian Path: Path is empty or not a list")
+            return False
+
+        # Get all nodes in the graph
+        all_nodes = set()
+        for node in graph.keys():
+            all_nodes.add(str(node))
+        for node, neighbors in graph.items():
+            for neighbor in neighbors:
+                all_nodes.add(str(neighbor))
+
+        # Convert path nodes to strings (for consistency)
+        path_str = [str(node) for node in path]
+
+        # Check if path visits all nodes exactly once
+        if set(path_str) != all_nodes:
+            if self.verbose:
+                missing_nodes = all_nodes - set(path_str)
+                extra_nodes = set(path_str) - all_nodes
+                if missing_nodes:
+                    print(f"❌ Hamiltonian Path: Path missing nodes {missing_nodes}")
+                if extra_nodes:
+                    print(f"❌ Hamiltonian Path: Path contains non-existent nodes {extra_nodes}")
+            return False
+
+        if len(path_str) != len(set(path_str)):
+            if self.verbose:
+                duplicates = [node for node in path_str if path_str.count(node) > 1]
+                print(f"❌ Hamiltonian Path: Path contains duplicate nodes {set(duplicates)}")
+            return False
+
+        # Check if adjacent nodes in path are connected in the graph
+        for i in range(len(path) - 1):
+            current_node = str(path[i])
+            next_node = str(path[i + 1])
+
+            # Check if there's an edge from current_node to next_node
+            if current_node not in graph:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Path: Node {current_node} does not exist in graph")
+                return False
+
+            neighbors = [str(neighbor) for neighbor in graph[current_node]]
+            if next_node not in neighbors:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Path: No edge between nodes {current_node} and {next_node}")
+                return False
+
+        return True
+
+
+class HamiltonianCycleEvaluator:
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate if the predicted answer is a valid Hamiltonian cycle
+
+        Args:
+            predicted_answer: Predicted cycle, could be a list (e.g., [0,1,2,3,0]) or string "No"
+            ground_truth: Ground truth answer (for reference)
+            initial_state: Graph adjacency list representation, format like {'0': [2], '1': [2, 3], '2': [0, 1], ...}
+
+        Returns:
+            bool: Whether the predicted answer is correct
+        """
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                graph = safe_parse_answer(initial_state, self.verbose)
+            else:
+                graph = initial_state
+
+            # Parse predicted answer
+            if isinstance(predicted_answer, str):
+                predicted_cycle = safe_parse_answer(predicted_answer, self.verbose)
+                if predicted_cycle is None and predicted_answer.strip().lower() != "no":
+                    if self.verbose:
+                        print(f"❌ Hamiltonian Cycle: Cannot parse predicted answer '{predicted_answer}'")
+                    return False
+            else:
+                predicted_cycle = predicted_answer
+
+            # Parse ground truth
+            if isinstance(ground_truth, str):
+                expected_cycle = safe_parse_answer(ground_truth, self.verbose)
+            else:
+                expected_cycle = ground_truth
+
+            # If ground truth is "No", check if predicted answer is also "No"
+            if expected_cycle is None:
+                if predicted_cycle is not None:
+                    if self.verbose:
+                        print(f"❌ Hamiltonian Cycle: Ground truth is 'No', but predicted answer is {predicted_cycle}")
+                    return False
+                return True
+
+            # If predicted answer is "No" but ground truth is not, then error
+            if predicted_cycle is None:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Cycle: Predicted answer is 'No', but ground truth is {expected_cycle}")
+                return False
+
+            # Validate if predicted cycle is a valid Hamiltonian cycle
+            if not self._is_valid_hamiltonian_cycle(predicted_cycle, graph):
+                return False
+
+            if self.verbose:
+                print(f"✅ Hamiltonian Cycle Evaluation Passed: {predicted_cycle}")
+            return True
+
+        except Exception as e:
+            if self.verbose:
+                print(f"❌ Hamiltonian Cycle Evaluation Exception: {str(e)}")
+            return False
+
+    def _is_valid_hamiltonian_cycle(self, cycle: List, graph: Dict) -> bool:
+        """
+        Validate if the given cycle is a valid Hamiltonian cycle
+        """
+        if not cycle or not isinstance(cycle, list) or len(cycle) < 2:
+            if self.verbose:
+                print("❌ Hamiltonian Cycle: Cycle is empty, not a list, or too short")
+            return False
+
+        # Get all nodes in the graph
+        all_nodes = set()
+        for node in graph.keys():
+            all_nodes.add(str(node))
+        for node, neighbors in graph.items():
+            for neighbor in neighbors:
+                all_nodes.add(str(neighbor))
+
+        # Convert cycle nodes to strings (for consistency)
+        cycle_str = [str(node) for node in cycle]
+
+        # Check if first and last nodes are the same (forms a cycle)
+        if cycle_str[0] == cycle_str[-1]:
+            cycle_without_last = cycle_str[:-1]
+        else:
+            cycle_without_last = cycle_str
+
+        # Check if all nodes are visited exactly once (excluding the duplicate last node)
+        if set(cycle_without_last) != all_nodes:
+            if self.verbose:
+                missing_nodes = all_nodes - set(cycle_without_last)
+                extra_nodes = set(cycle_without_last) - all_nodes
+                if missing_nodes:
+                    print(f"❌ Hamiltonian Cycle: Cycle missing nodes {missing_nodes}")
+                if extra_nodes:
+                    print(f"❌ Hamiltonian Cycle: Cycle contains non-existent nodes {extra_nodes}")
+            return False
+
+        if len(cycle_without_last) != len(set(cycle_without_last)):
+            if self.verbose:
+                duplicates = [node for node in cycle_without_last if cycle_without_last.count(node) > 1]
+                print(f"❌ Hamiltonian Cycle: Cycle contains duplicate nodes {set(duplicates)}")
+            return False
+
+        # Check if adjacent nodes in cycle are connected in the graph
+        for i in range(len(cycle_str) - 1):
+            current_node = cycle_str[i]
+            next_node = cycle_str[i + 1]
+
+            # Check if there's an edge from current_node to next_node
+            if current_node not in graph:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Cycle: Node {current_node} does not exist in graph")
+                return False
+
+            neighbors = [str(neighbor) for neighbor in graph[current_node]]
+            if next_node not in neighbors:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Cycle: No edge between nodes {current_node} and {next_node}")
+                return False
+
+        # For cycles that don't repeat the first node at the end (e.g., [0,1,2,3] instead of [0,1,2,3,0])
+        # we need to check if the last node connects back to the first node
+        if cycle_str[0] != cycle_str[-1]:
+            last_node = cycle_str[-1]
+            first_node = cycle_str[0]
+
+            if last_node not in graph:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Cycle: Last node {last_node} does not exist in graph")
+                return False
+
+            neighbors = [str(neighbor) for neighbor in graph[last_node]]
+            if first_node not in neighbors:
+                if self.verbose:
+                    print(f"❌ Hamiltonian Cycle: No edge between last node {last_node} and first node {first_node}")
+                return False
+
+        return True
+
+
+class EulerianPathEvaluator:
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate if the predicted answer is a valid Eulerian path
+
+        Args:
+            predicted_answer: Predicted path, could be a list or string "No"
+            ground_truth: Ground truth answer (for reference)
+            initial_state: Graph adjacency list representation
+
+        Returns:
+            bool: Whether the predicted answer is correct
+        """
+        if self.verbose:
+            print("predicted_answer: ", predicted_answer[:255])
+            print("ground_truth: ", ground_truth[:255])
+            print("initial_state: ", initial_state)
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                graph = safe_parse_answer(initial_state, self.verbose)
+            else:
+                graph = initial_state
+
+            # Parse predicted answer
+            if isinstance(predicted_answer, str):
+                predicted_path = safe_parse_answer(predicted_answer, self.verbose)
+                if predicted_path is None and predicted_answer.strip().lower() != "no":
+                    if self.verbose:
+                        print(f"❌ Eulerian Path: Cannot parse predicted answer '{predicted_answer}'")
+                    return False
+            else:
+                predicted_path = predicted_answer
+
+            # Parse ground truth
+            if isinstance(ground_truth, str):
+                expected_path = safe_parse_answer(ground_truth, self.verbose)
+            else:
+                expected_path = ground_truth
+
+            # If ground truth is "No", check if predicted answer is also "No"
+            if expected_path is None:
+                if predicted_path is not None:
+                    if self.verbose:
+                        print(f"❌ Eulerian Path: Ground truth is 'No', but predicted answer is {predicted_path[:120]}")
+                    return False
+                return True
+
+            # If predicted answer is "No" but ground truth is not, then error
+            if predicted_path is None:
+                if self.verbose:
+                    print(f"❌ Eulerian Path: Predicted answer is 'No', but ground truth is {expected_path[:120]}")
+                return False
+
+            # Validate if predicted path is a valid Eulerian path
+            if not self._is_valid_eulerian_path(predicted_path, graph):
+                return False
+
+            if self.verbose:
+                print(f"✅ Eulerian Path Evaluation Passed: {predicted_path}")
+            return True
+
+        except Exception as e:
+            if self.verbose:
+                print(f"❌ Eulerian Path Evaluation Exception: {str(e)}")
+            return False
+
+    def _is_valid_eulerian_path(self, path: List, graph: Dict) -> bool:
+        """
+        Validate if the given path is a valid Eulerian path (traverses each edge exactly once)
+        """
+        if not path or not isinstance(path, list) or len(path) < 2:
+            if self.verbose:
+                print("❌ Eulerian Path: Path is empty, not a list, or too short")
+            return False
+
+        # Build set of edges
+        edges = set()
+        for node, neighbors in graph.items():
+            for neighbor in neighbors:
+                # For undirected graphs, standardize edges (smaller node first)
+                edge = tuple(sorted([str(node), str(neighbor)]))
+                edges.add(edge)
+
+        # Check edges in the path
+        path_edges = set()
+        path_str = [str(node) for node in path]
+
+        for i in range(len(path_str) - 1):
+            current_node = path_str[i]
+            next_node = path_str[i + 1]
+            edge = tuple(sorted([current_node, next_node]))
+
+            # Check if edge exists
+            if edge not in edges:
+                if self.verbose:
+                    print(f"❌ Eulerian Path: Edge ({current_node}, {next_node}) does not exist in graph")
+                return False
+
+            # Check if edge has already been used
+            if edge in path_edges:
+                if self.verbose:
+                    print(f"❌ Eulerian Path: Edge ({current_node}, {next_node}) is used more than once")
+                return False
+
+            path_edges.add(edge)
+
+        # Check if all edges are traversed
+        if path_edges != edges:
+            if self.verbose:
+                missing_edges = edges - path_edges
+                extra_edges = path_edges - edges
+                if missing_edges:
+                    print(f"❌ Eulerian Path: Path missing edges {missing_edges}")
+                if extra_edges:
+                    print(f"❌ Eulerian Path: Path contains non-existent edges {extra_edges}")
+            return False
+
+        return True
+
+
+class EulerianCycleEvaluator:
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate if the predicted answer is a valid Eulerian cycle
+
+        Args:
+            predicted_answer: Predicted cycle, could be a list or string "No"
+            ground_truth: Ground truth answer (for reference)
+            initial_state: Graph adjacency list representation
+
+        Returns:
+            bool: Whether the predicted answer is correct
+        """
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                graph = safe_parse_answer(initial_state, self.verbose)
+            else:
+                graph = initial_state
+
+            # Parse predicted answer
+            if isinstance(predicted_answer, str):
+                predicted_cycle = safe_parse_answer(predicted_answer, self.verbose)
+                if predicted_cycle is None and predicted_answer.strip().lower() != "no":
+                    if self.verbose:
+                        print(f"❌ Eulerian Cycle: Cannot parse predicted answer '{predicted_answer}'")
+                    return False
+            else:
+                predicted_cycle = predicted_answer
+
+            # Parse ground truth
+            if isinstance(ground_truth, str):
+                expected_cycle = safe_parse_answer(ground_truth, self.verbose)
+            else:
+                expected_cycle = ground_truth
+
+            # If ground truth is "No", check if predicted answer is also "No"
+            if expected_cycle is None:
+                if predicted_cycle is not None:
+                    if self.verbose:
+                        print(
+                            f"❌ Eulerian Cycle: Ground truth is 'No', but predicted answer is {predicted_cycle[:120]}"
+                        )
+                    return False
+                return True
+
+            # If predicted answer is "No" but ground truth is not, then error
+            if predicted_cycle is None:
+                if self.verbose:
+                    print(
+                        f"❌ Eulerian Cycle: Predicted answer is 'No', but ground truth is {expected_cycle[:120]}"
+                    )
+                return False
+
+            # Validate if predicted cycle is a valid Eulerian cycle
+            if not self._is_valid_eulerian_cycle(predicted_cycle, graph):
+                return False
+
+            if self.verbose:
+                print(f"✅ Eulerian Cycle Evaluation Passed: {predicted_cycle}")
+            return True
+
+        except Exception as e:
+            if self.verbose:
+                print(f"❌ Eulerian Cycle Evaluation Exception: {str(e)}")
+            return False
+
+    def _is_valid_eulerian_cycle(self, cycle: List, graph: Dict) -> bool:
+        """
+        Validate if the given cycle is a valid Eulerian cycle (traverses each edge exactly once and returns to start)
+        """
+        if not cycle or not isinstance(cycle, list) or len(cycle) < 3:
+            if self.verbose:
+                print("❌ Eulerian Cycle: Cycle is empty, not a list, or too short")
+            return False
+
+        cycle_str = [str(node) for node in cycle]
+
+        # Check if it forms a cycle (start and end nodes are the same)
+        # If not, we need to check if the last node connects back to the first node
+        if cycle_str[0] != cycle_str[-1]:
+            # For cycles that don't repeat the first node at the end (e.g., [0,1,2] instead of [0,1,2,0])
+            # we need to verify that the last node connects back to the first node
+            last_node = cycle_str[-1]
+            first_node = cycle_str[0]
+
+            if last_node not in graph:
+                if self.verbose:
+                    print(f"❌ Eulerian Cycle: Last node {last_node} does not exist in graph")
+                return False
+
+            neighbors = [str(neighbor) for neighbor in graph[last_node]]
+            if first_node not in neighbors:
+                if self.verbose:
+                    print(f"❌ Eulerian Cycle: No edge between last node {last_node} and first node {first_node}")
+                return False
+
+            # Create a new cycle with the first node repeated at the end for validation
+            extended_cycle = cycle + [cycle[0]]
+            return self._is_valid_eulerian_path_internal(extended_cycle, graph)
+
+        # Use Eulerian path validation logic
+        return self._is_valid_eulerian_path_internal(cycle, graph)
+
+    def _is_valid_eulerian_path_internal(self, path: List, graph: Dict) -> bool:
+        """
+        Internal method: Validate Eulerian path (for use by Eulerian cycle)
+        """
+        if not path or not isinstance(path, list) or len(path) < 2:
+            if self.verbose:
+                print("❌ Eulerian Path Internal: Path is empty, not a list, or too short")
+            return False
+
+        # Build set of edges
+        edges = set()
+        for node, neighbors in graph.items():
+            for neighbor in neighbors:
+                edge = tuple(sorted([str(node), str(neighbor)]))
+                edges.add(edge)
+
+        # Check edges in the path
+        path_edges = set()
+        path_str = [str(node) for node in path]
+
+        for i in range(len(path_str) - 1):
+            current_node = path_str[i]
+            next_node = path_str[i + 1]
+            edge = tuple(sorted([current_node, next_node]))
+
+            if edge not in edges:
+                if self.verbose:
+                    print(f"❌ Eulerian Path Internal: Edge ({current_node}, {next_node}) does not exist in graph")
+                return False
+
+            if edge in path_edges:
+                if self.verbose:
+                    print(f"❌ Eulerian Path Internal: Edge ({current_node}, {next_node}) is used more than once")
+                return False
+
+            path_edges.add(edge)
+
+        if path_edges != edges:
+            if self.verbose:
+                missing_edges = edges - path_edges
+                extra_edges = path_edges - edges
+                if missing_edges:
+                    print(f"❌ Eulerian Path Internal: Path missing edges {missing_edges}")
+                if extra_edges:
+                    print(f"❌ Eulerian Path Internal: Path contains non-existent edges {extra_edges}")
+            return False
+
+        return True
+
+
+class ConnectivityEvaluator:
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate if the predicted answer is correct for connectivity problem
+
+        Args:
+            predicted_answer: Predicted answer, could be a path list (e.g., [0,3,7,9]) or number (e.g., "3")
+            ground_truth: Ground truth answer (for reference)
+            initial_state: Graph information including adjacency list, start_node, target_node
+
+        Returns:
+            bool: Whether the predicted answer is correct
+        """
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                graph_info = safe_parse_answer(initial_state, self.verbose)
+            else:
+                graph_info = initial_state
+
+            # Extract graph components
+            if isinstance(graph_info, dict) and 'adjacency_list' in graph_info:
+                # New format with complete graph info
+                adjacency_list = graph_info['adjacency_list']
+                start_node = graph_info.get('start_node')
+                target_node = graph_info.get('target_node')
+            else:
+                # Old format - just adjacency list
+                adjacency_list = graph_info
+                start_node = None
+                target_node = None
+
+            # Parse predicted answer
+            if isinstance(predicted_answer, str):
+                try:
+                    # Try to parse as integer (connected components count)
+                    predicted_result = int(predicted_answer)
+                except ValueError:
+                    # Try to parse as list (path)
+                    predicted_result = safe_parse_answer(predicted_answer, self.verbose)
+                    if predicted_result is None:
+                        if self.verbose:
+                            print(f"❌ Connectivity: Cannot parse predicted answer '{predicted_answer}'")
+                        return False
+            else:
+                predicted_result = predicted_answer
+
+            # Parse ground truth
+            if isinstance(ground_truth, str):
+                try:
+                    # Try to parse as integer (connected components count)
+                    expected_result = int(ground_truth)
+                except ValueError:
+                    # Try to parse as list (path)
+                    expected_result = safe_parse_answer(ground_truth, self.verbose)
+            else:
+                expected_result = ground_truth
+
+            # Case 1: Both are integers (connected components count)
+            if isinstance(predicted_result, int) and isinstance(expected_result, int):
+                if predicted_result == expected_result:
+                    # Also verify that the predicted count is actually correct
+                    actual_components = self._count_connected_components(adjacency_list)
+                    if predicted_result == actual_components:
+                        if self.verbose:
+                            print(
+                                "✅ Connectivity Evaluation Passed"
+                            )
+                        return True
+                    else:
+                        if self.verbose:
+                            print(
+                                f"❌ Connectivity: Predicted {predicted_result} components, \
+                                but actual count is {actual_components}"
+                            )
+                        return False
+                else:
+                    if self.verbose:
+                        print(
+                            f"❌ Connectivity: Predicted {predicted_result} components, expected {expected_result}"
+                        )
+                    return False
+
+            # Case 2: Both are lists (paths)
+            elif isinstance(predicted_result, list) and isinstance(expected_result, list):
+                # Validate the predicted path
+                if start_node is not None and target_node is not None:
+                    if self._is_valid_path(predicted_result, adjacency_list, start_node, target_node):
+                        if self.verbose:
+                            print(f"✅ Connectivity Evaluation Passed: Valid path {predicted_result}")
+                        return True
+                    else:
+                        return False
+                else:
+                    # If start/target nodes are not provided, just check if it's a valid path in the graph
+                    if self._is_valid_path_general(predicted_result, adjacency_list):
+                        if self.verbose:
+                            print(f"✅ Connectivity Evaluation Passed: Valid path {predicted_result}")
+                        return True
+                    else:
+                        return False
+
+            # Case 3: Type mismatch
+            else:
+                if self.verbose:
+                    print(
+                        f"❌ Connectivity: Type mismatch. "
+                        f"Predicted: {type(predicted_result)}, Expected: {type(expected_result)}"
+                    )
+                return False
+
+        except Exception as e:
+            if self.verbose:
+                print(f"❌ Connectivity Evaluation Exception: {str(e)}")
+            return False
+
+    def _count_connected_components(self, adjacency_list: Dict) -> int:
+        """Count the number of connected components in the graph"""
+        visited = set()
+        components = 0
+
+        # Get all nodes
+        all_nodes = set()
+        for node in adjacency_list.keys():
+            all_nodes.add(str(node))
+        for node, neighbors in adjacency_list.items():
+            for neighbor in neighbors:
+                all_nodes.add(str(neighbor))
+
+        for node in all_nodes:
+            if node not in visited:
+                components += 1
+                # DFS to mark all nodes in this component
+                stack = [node]
+                while stack:
+                    current = stack.pop()
+                    if current not in visited:
+                        visited.add(current)
+                        # Add unvisited neighbors
+                        current_neighbors = adjacency_list.get(current, [])
+                        for neighbor in current_neighbors:
+                            neighbor_str = str(neighbor)
+                            if neighbor_str not in visited:
+                                stack.append(neighbor_str)
+
+        return components
+
+    def _is_valid_path(self, path: List, adjacency_list: Dict, start_node: Any, target_node: Any) -> bool:
+        """Validate if the path is valid and connects start_node to target_node"""
+        if not path or not isinstance(path, list) or len(path) < 1:
+            if self.verbose:
+                print("❌ Connectivity Path: Path is empty or invalid")
+            return False
+
+        path_str = [str(node) for node in path]
+        start_str = str(start_node)
+        target_str = str(target_node)
+
+        # Check if path starts with start_node and ends with target_node
+        if path_str[0] != start_str:
+            if self.verbose:
+                print(f"❌ Connectivity Path: Path starts with {path_str[0]}, expected {start_str}")
+            return False
+
+        if path_str[-1] != target_str:
+            if self.verbose:
+                print(f"❌ Connectivity Path: Path ends with {path_str[-1]}, expected {target_str}")
+            return False
+
+        # Check if adjacent nodes in path are connected
+        for i in range(len(path) - 1):
+            current_node = str(path[i])
+            next_node = str(path[i + 1])
+
+            if current_node not in adjacency_list:
+                if self.verbose:
+                    print(f"❌ Connectivity Path: Node {current_node} not in graph")
+                return False
+
+            neighbors = [str(neighbor) for neighbor in adjacency_list[current_node]]
+            if next_node not in neighbors:
+                if self.verbose:
+                    print(f"❌ Connectivity Path: No edge between {current_node} and {next_node}")
+                return False
+
+        return True
+
+    def _is_valid_path_general(self, path: List, adjacency_list: Dict) -> bool:
+        """Validate if the path is valid in the graph (without specific start/end requirements)"""
+        if not path or not isinstance(path, list) or len(path) < 1:
+            if self.verbose:
+                print("❌ Connectivity Path: Path is empty or invalid")
+            return False
+
+        # Check if adjacent nodes in path are connected
+        for i in range(len(path) - 1):
+            current_node = str(path[i])
+            next_node = str(path[i + 1])
+
+            if current_node not in adjacency_list:
+                if self.verbose:
+                    print(f"❌ Connectivity Path: Node {current_node} not in graph")
+                return False
+
+            neighbors = [str(neighbor) for neighbor in adjacency_list[current_node]]
+            if next_node not in neighbors:
+                if self.verbose:
+                    print(f"❌ Connectivity Path: No edge between {current_node} and {next_node}")
+                return False
+
+        return True
+
+
+class TopologicalSortEvaluator:
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def extract_array(self, text):
+        """
+        从文本中提取数组，支持嵌套数组和单个数组的情况
+
+        参数:
+            text: 包含数组的字符串，如"[1,2,3]"或"[[1,2,3]]"
+
+        返回:
+            提取出的数组对象
+        """
+        import ast
+        import re
+
+        # 确保输入是字符串类型
+        if not isinstance(text, str):
+            return text
+
+        # 去除可能的空白字符
+        text = text.strip()
+
+        try:
+            # 尝试直接解析
+            parsed = ast.literal_eval(text)
+
+            # 如果是嵌套数组且只有一个元素，返回内部数组
+            if isinstance(parsed, list) and len(parsed) == 1 and isinstance(parsed[0], list):
+                return parsed[0]
+
+            # 如果是普通数组或其他情况，直接返回
+            return parsed
+        except (SyntaxError, ValueError):
+            # 使用正则表达式尝试提取
+            array_pattern = r'\[(.*)\]'
+            match = re.search(array_pattern, text)
+
+            if match:
+                inner_content = match.group(1).strip()
+
+                # 检查是否是嵌套数组
+                if inner_content.startswith('[') and inner_content.endswith(']'):
+                    try:
+                        return ast.literal_eval(inner_content)
+                    except Exception:
+                        pass
+
+                # 尝试作为单个数组解析
+                try:
+                    return ast.literal_eval(f'[{inner_content}]')
+                except Exception:
+                    # 如果还是失败，可能是格式不规范，使用更宽松的解析方式
+                    numbers = re.findall(r'-?\d+', inner_content)
+                    if numbers:
+                        return [int(num) for num in numbers]
+
+        return None
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate if the predicted answer is a valid topological sort
+
+        Args:
+            predicted_answer: Predicted sort order, could be a list or string "No"
+            ground_truth: Ground truth answer (for reference)
+            initial_state: Directed graph adjacency list representation
+
+        Returns:
+            bool: Whether the predicted answer is correct
+        """
+        # predicted_answer = self.extract_array(predicted_answer)
+        if self.verbose:
+            print("predicted_answer: ", predicted_answer[:255])
+            print("ground_truth: ", ground_truth[:255])
+            print("initial_state: ", initial_state)
+
+        try:
+            # Parse initial_state
+            if isinstance(initial_state, str):
+                graph = safe_parse_answer(initial_state, self.verbose)
+            else:
+                graph = initial_state
+
+            # Parse predicted answer
+            if isinstance(predicted_answer, str):
+                predicted_sort = safe_parse_answer(predicted_answer, self.verbose)
+                if predicted_sort is None and predicted_answer.strip().lower() != "no":
+                    if self.verbose:
+                        print(f"❌ Topological Sort: Cannot parse predicted answer '{predicted_answer}'")
+                    return False
+            else:
+                predicted_sort = predicted_answer
+
+            # Parse ground truth
+            if isinstance(ground_truth, str):
+                expected_sort = safe_parse_answer(ground_truth, self.verbose)
+            else:
+                expected_sort = ground_truth
+
+            # If ground truth is "No", check if predicted answer is also "No"
+            if expected_sort is None:
+                if predicted_sort is not None:
+                    if self.verbose:
+                        print(
+                            f"❌ Topological Sort: Ground truth is 'No', "
+                            f"but predicted answer is {predicted_sort[:120]}"
+                        )
+                    return False
+                return True
+
+            # If predicted answer is "No" but ground truth is not, then error
+            if predicted_sort is None:
+                if self.verbose:
+                    print(f"❌ Topological Sort: Predicted answer is 'No', but ground truth is {expected_sort[:120]}")
+                return False
+
+            # Validate if predicted sort is a valid topological sort
+            if not self._is_valid_topological_sort(predicted_sort, graph):
+                return False
+
+            if self.verbose:
+                print(f"✅ Topological Sort Evaluation Passed: {predicted_sort}")
+            return True
+
+        except Exception as e:
+            if self.verbose:
+                print(f"❌ Topological Sort Evaluation Exception: {str(e)}")
+            return False
+
+    def _is_valid_topological_sort(self, sort_order: List, graph: Dict) -> bool:
+        """
+        Validate if the given order is a valid topological sort
+        """
+        if not sort_order or not isinstance(sort_order, list):
+            if self.verbose:
+                print("❌ Topological Sort: Sort order is empty or not a list")
+            return False
+
+        # Get all nodes in the graph
+        all_nodes = set()
+        for node in graph.keys():
+            all_nodes.add(str(node))
+        for node, neighbors in graph.items():
+            for neighbor in neighbors:
+                all_nodes.add(str(neighbor))
+
+        sort_str = [str(node) for node in sort_order]
+
+        # Check if all nodes are included and no duplicates
+        if set(sort_str) != all_nodes:
+            if self.verbose:
+                missing_nodes = all_nodes - set(sort_str)
+                extra_nodes = set(sort_str) - all_nodes
+                if missing_nodes:
+                    print(f"❌ Topological Sort: Sort missing nodes {missing_nodes}")
+                if extra_nodes:
+                    print(f"❌ Topological Sort: Sort contains non-existent nodes {extra_nodes}")
+            return False
+
+        if len(sort_str) != len(set(sort_str)):
+            if self.verbose:
+                duplicates = [node for node in sort_str if sort_str.count(node) > 1]
+                print(f"❌ Topological Sort: Sort contains duplicate nodes {set(duplicates)}")
+            return False
+
+        # Create node position mapping
+        position = {node: i for i, node in enumerate(sort_str)}
+
+        # Check if all directed edges satisfy topological order
+        for node, neighbors in graph.items():
+            node_str = str(node)
+            for neighbor in neighbors:
+                neighbor_str = str(neighbor)
+                # For directed edge node -> neighbor, node should come before neighbor
+                if position[node_str] >= position[neighbor_str]:
+                    if self.verbose:
+                        print(
+                            f"❌ Topological Sort: Edge ({node_str} -> {neighbor_str}) "
+                            f"violates topological order. Position of {node_str}: {position[node_str]}, "
+                            f"Position of {neighbor_str}: {position[neighbor_str]}"
+                        )
+                    return False
+
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/hanoi_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/hanoi_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..52933bc1180b13e63df1576b620825419b08c502
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/hanoi_eval.py
@@ -0,0 +1,42 @@
+import ast
+import re
+from typing import Any, Dict
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        # Extract content within <answer></answer> tags
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+        if match:
+            return match.group(1).strip()
+        return model_output.strip()  # Fallback to full output if no tags found
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class TowerOfHanoiEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        from utils.constants import PROMPT_HANOI
+
+        if isinstance(question, list):
+            question_str = str(question)
+        else:
+            question_str = question
+
+        return PROMPT_HANOI.format(question_str)
+
+    def extract_answer(self, model_output: str) -> str:
+        answer = model_output.strip()
+        return answer
+
+    def evaluate(self, predicted_answer: str, ground_truth: Any, initial_state: Any) -> bool:
+        from vlmeval.dataset.utils.mmhelix.utils.validation import hanoi_check
+        if not initial_state:
+            return False
+        lst = ast.literal_eval(initial_state) if isinstance(initial_state, str) else initial_state
+        return hanoi_check(lst, predicted_answer)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/hitori_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/hitori_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..b13bec0e194906a0c9461db7a6cd708aa0374330
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/hitori_eval.py
@@ -0,0 +1,430 @@
+import json
+import re
+from typing import Any, Dict, List, Set, Tuple, Union
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        raise NotImplementedError
+
+
+class HitoriEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """
+        准备提示，指导模型如何解答Hitori拼图
+
+        Args:
+            question: 问题描述
+            params: 包含拼图数据的参数字典
+
+        Returns:
+            格式化的提示
+        """
+        grid = params.get("grid", [])
+        grid_size = len(grid)
+
+        prompt = f"{question}\n\n"
+        prompt += "Hitori拼图说明：\n"
+        prompt += "1. 需要将一些单元格涂黑\n"
+        prompt += "2. 每行每列不能有重复的数字(在非黑色单元格中)\n"
+        prompt += "3. 黑色单元格不能相邻(不能共享边)\n"
+        prompt += "4. 所有白色单元格必须连通(通过上下左右移动)\n\n"
+
+        prompt += f"网格尺寸: {grid_size}x{grid_size}\n"
+        prompt += "拼图网格:\n"
+
+        # 构建网格可视化
+        for row in grid:
+            prompt += " ".join(str(cell) for cell in row) + "\n"
+
+        prompt += "\n请列出应该涂黑的单元格坐标，格式为(row, col)的集合。"
+        prompt += "\n坐标从(0,0)开始计数，即左上角为(0,0)。"
+        prompt += "\n请使用标准格式返回答案: {(r1, c1), (r2, c2), ...}"
+
+        return prompt
+
+    def extract_answer(self, model_output: str) -> Set[Tuple[int, int]]:
+        """
+        从模型输出中提取涂黑单元格的坐标集合，支持多种格式
+        增强鲁棒性，处理各种边界情况
+
+        Args:
+            model_output: 模型生成的输出
+
+        Returns:
+            涂黑单元格的坐标集合
+        """
+        if not isinstance(model_output, str):
+            return set()
+
+        coords = set()
+
+        # 预处理：移除常见的无关字符和噪音
+        cleaned_output = re.sub(r'["""''`]', '', model_output)  # 移除引号
+        cleaned_output = re.sub(r'\\n|\\t', ' ', cleaned_output)  # 移除转义字符
+
+        # 方法1: 尝试匹配花括号包围的坐标集合
+        # 支持格式: {(1,2), (3,4)} 或 {(1, 2), (3, 4)} 等
+        brace_pattern = r'\{([^}]*)\}'
+        brace_matches = re.findall(brace_pattern, cleaned_output)
+
+        for match in brace_matches:
+            # 在花括号内容中查找所有坐标对
+            coord_matches = re.findall(r'[(\[]?\s*(\d+)\s*[,，]\s*(\d+)\s*[)\]]?', match)
+            for row_str, col_str in coord_matches:
+                try:
+                    row, col = int(row_str), int(col_str)
+                    if self._is_valid_coordinate(row, col):
+                        coords.add((row, col))
+                except ValueError:
+                    continue
+
+        # 方法2: 尝试匹配方括号包围的数组格式
+        # 支持格式: [(1,2), (3,4)] 或 [[1,2], [3,4]]
+        if not coords:
+            bracket_patterns = [
+                r'\[\s*([^]]*)\s*\]',  # 外层方括号
+                r'\(\s*\[\s*([^]]*)\s*\]\s*\)',  # 圆括号包围的方括号
+            ]
+
+            for pattern in bracket_patterns:
+                matches = re.findall(pattern, cleaned_output)
+                for match in matches:
+                    # 查找坐标对
+                    coord_matches = re.findall(r'[(\[]?\s*(\d+)\s*[,，]\s*(\d+)\s*[)\]]?', match)
+                    for row_str, col_str in coord_matches:
+                        try:
+                            row, col = int(row_str), int(col_str)
+                            if self._is_valid_coordinate(row, col):
+                                coords.add((row, col))
+                        except ValueError:
+                            continue
+
+        # 方法3: 如果前面没找到，尝试直接在整个文本中查找坐标
+        if not coords:
+            # 增强的坐标识别模式
+            coordinate_patterns = [
+                r'\(\s*(\d+)\s*[,，]\s*(\d+)\s*\)',  # (1,2) 格式
+                r'\[\s*(\d+)\s*[,，]\s*(\d+)\s*\]',  # [1,2] 格式
+                r'(?:坐标|位置|cell|点)\s*[：:]\s*\(?(\d+)\s*[,，]\s*(\d+)\s*\)?',  # 坐标:1,2
+                r'(?:row|行)\s*[=：:]\s*(\d+)\s*[,，\s]+(?:col|column|列)\s*[=：:]\s*(\d+)',  # row=1,col=2
+                r'(?:col|column|列)\s*[=：:]\s*(\d+)\s*[,，\s]+(?:row|行)\s*[=：:]\s*(\d+)',  # col=2,row=1
+                r'(?:第|第\s*)?(\d+)\s*行[，,\s]*(?:第|第\s*)?(\d+)\s*列',  # 第1行第2列
+                r'(?:第|第\s*)?(\d+)\s*列[，,\s]*(?:第|第\s*)?(\d+)\s*行',  # 第2列第1行
+            ]
+
+            for pattern in coordinate_patterns:
+                matches = re.findall(pattern, cleaned_output, re.IGNORECASE)
+                for match in matches:
+                    try:
+                        if len(match) == 2:
+                            # 处理行列可能互换的情况
+                            if 'col' in pattern.lower() and pattern.index('col') < pattern.index('row'):
+                                row, col = int(match[1]), int(match[0])
+                            elif '列' in pattern and pattern.index('列') < pattern.index('行'):
+                                row, col = int(match[1]), int(match[0])
+                            else:
+                                row, col = int(match[0]), int(match[1])
+
+                            if self._is_valid_coordinate(row, col):
+                                coords.add((row, col))
+                    except (ValueError, AttributeError):
+                        continue
+
+        # 方法4: 尝试解析JSON格式
+        if not coords:
+            try:
+                # 尝试找到JSON数组格式 [[1,2], [3,4]]
+                json_patterns = [
+                    r'\[\s*\[.*?\]\s*\]',  # [[1,2], [3,4]]
+                    r'\[\s*\(.*?\)\s*\]',  # [(1,2), (3,4)]
+                ]
+
+                for json_pattern in json_patterns:
+                    json_matches = re.findall(json_pattern, cleaned_output)
+                    for json_str in json_matches:
+                        try:
+                            # 预处理JSON字符串
+                            json_str = re.sub(r'\(\s*(\d+)\s*,\s*(\d+)\s*\)', r'[\1,\2]', json_str)
+                            coord_list = json.loads(json_str)
+                            if isinstance(coord_list, list):
+                                for item in coord_list:
+                                    if isinstance(item, list) and len(item) == 2:
+                                        try:
+                                            row, col = int(item[0]), int(item[1])
+                                            if self._is_valid_coordinate(row, col):
+                                                coords.add((row, col))
+                                        except (ValueError, TypeError):
+                                            continue
+                        except (json.JSONDecodeError, ValueError, TypeError):
+                            continue
+            except Exception:
+                pass
+
+        # 方法5: 处理纯数字序列（如果有明确的分隔符）
+        if not coords:
+            # 匹配形如: 1,2 3,4 5,6 的格式
+            number_pairs = re.findall(r'(\d+)\s*[,，]\s*(\d+)', cleaned_output)
+            if number_pairs:
+                # 只有在找到合理数量的坐标对时才使用这种方法
+                valid_pairs = []
+                for row_str, col_str in number_pairs:
+                    try:
+                        row, col = int(row_str), int(col_str)
+                        if self._is_valid_coordinate(row, col):
+                            valid_pairs.append((row, col))
+                    except ValueError:
+                        continue
+
+                # 如果找到的坐标对数量合理（不会包含太多错误匹配）
+                if 1 <= len(valid_pairs) <= 50:  # 假设合理的坐标数量范围
+                    coords.update(valid_pairs)
+
+        # 方法6: 处理表格形式的输出
+        if not coords:
+            # 查找表格形式的坐标列表
+            table_patterns = [
+                r'(?:shaded|black|涂黑|选择)[^:]*[:：]\s*([^.\n]+)',
+                r'(?:answer|答案|结果)[^:]*[:：]\s*([^.\n]+)',
+                r'(?:coordinates|坐标)[^:]*[:：]\s*([^.\n]+)',
+            ]
+
+            for pattern in table_patterns:
+                matches = re.findall(pattern, cleaned_output, re.IGNORECASE | re.DOTALL)
+                for match in matches:
+                    # 在匹配的内容中查找坐标
+                    coord_matches = re.findall(r'[(\[]?\s*(\d+)\s*[,，]\s*(\d+)\s*[)\]]?', match)
+                    for row_str, col_str in coord_matches:
+                        try:
+                            row, col = int(row_str), int(col_str)
+                            if self._is_valid_coordinate(row, col):
+                                coords.add((row, col))
+                        except ValueError:
+                            continue
+
+        return coords
+
+    def _is_valid_coordinate(self, row: int, col: int, max_size: int = 100) -> bool:
+        """
+        检查坐标是否有效
+
+        Args:
+            row, col: 坐标值
+            max_size: 最大网格大小限制
+
+        Returns:
+            是否为有效坐标
+        """
+        return 0 <= row < max_size and 0 <= col < max_size
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        评估预测答案是否正确，仅根据initial_state和Hitori拼图规则验证
+        不与ground_truth进行比较，只验证游戏规则
+
+        Args:
+            predicted_answer: 预测答案（可以是字符串、集合等格式）
+            ground_truth: 真实答案（保留参数但不使用，仅为兼容性）
+            initial_state: 初始网格状态（JSON字符串或二维数组）
+
+        Returns:
+            布尔值，表示预测答案是否满足游戏规则
+        """
+        # 提取预测答案中的坐标
+        if isinstance(predicted_answer, str):
+            pred_coords = self.extract_answer(predicted_answer)
+        else:
+            pred_coords = self._parse_coordinates(predicted_answer)
+
+        if pred_coords is None:
+            return False
+
+        # 解析初始状态
+        grid = self._parse_initial_state(initial_state)
+        if grid is None:
+            return False
+
+        # 验证预测答案是否满足Hitori拼图规则
+        return self._validate_hitori_rules(pred_coords, grid)
+
+    def _parse_initial_state(self, initial_state: Any) -> List[List[int]]:
+        """
+        解析initial_state为网格，支持多种输入格式
+
+        Args:
+            initial_state: 初始状态（字符串、列表等）
+
+        Returns:
+            二维数组网格或None（如果解析失败）
+        """
+        try:
+            # 如果已经是列表，直接使用
+            if isinstance(initial_state, list):
+                grid = initial_state
+            else:
+                # 尝试解析JSON格式的字符串
+                grid = json.loads(str(initial_state))
+
+            # 验证是否为有效的二维数组
+            if (isinstance(grid, list) and len(grid) > 0 and all(isinstance(row, list) for row in grid)
+                    and all(len(row) == len(grid[0]) for row in grid)
+                    and all(isinstance(cell, (int, float)) for row in grid for cell in row)):
+
+                # 确保所有元素都是整数
+                return [[int(cell) for cell in row] for row in grid]
+
+        except (json.JSONDecodeError, ValueError, TypeError):
+            pass
+
+        return None
+
+    def _parse_coordinates(self, coords_input: Union[str, Set[Tuple[int, int]], List]) -> Set[Tuple[int, int]]:
+        """
+        将坐标输入解析为集合
+
+        Args:
+            coords_input: 坐标输入（字符串、集合或列表）
+
+        Returns:
+            坐标集合或None（如果解析失败）
+        """
+        # 如果已经是集合，验证格式
+        if isinstance(coords_input, set):
+            if all(isinstance(item, tuple) and len(item) == 2
+                   and all(isinstance(coord, int) for coord in item) for item in coords_input):
+                return coords_input
+            return None
+
+        # 如果是列表，尝试转换为集合
+        if isinstance(coords_input, list):
+            try:
+                result = set()
+                for item in coords_input:
+                    if isinstance(item, (tuple, list)) and len(item) == 2:
+                        result.add((int(item[0]), int(item[1])))
+                    else:
+                        return None
+                return result
+            except (ValueError, TypeError):
+                return None
+
+        # 如果是字符串，使用extract_answer方法
+        if isinstance(coords_input, str):
+            return self.extract_answer(coords_input)
+
+        return None
+
+    def _validate_hitori_rules(self, shaded_cells: Set[Tuple[int, int]], grid: List[List[int]]) -> bool:
+        """
+        验证答案是否满足Hitori拼图的所有规则
+
+        Args:
+            shaded_cells: 涂黑单元格的坐标集合
+            grid: 初始网格
+
+        Returns:
+            布尔值，表示答案是否有效
+        """
+        if not grid:
+            return False
+
+        size = len(grid)
+
+        # 验证所有坐标都在网格范围内
+        for r, c in shaded_cells:
+            if not (0 <= r < size and 0 <= c < len(grid[r])):
+                return False
+
+        # 规则1：每行每列不能有重复的数字(在非黑色单元格中)
+        if not self._check_no_duplicates(shaded_cells, grid):
+            return False
+
+        # 规则2：黑色单元格不能相邻
+        if not self._check_no_adjacent_shaded(shaded_cells, size):
+            return False
+
+        # 规则3：所有白色单元格必须连通
+        if not self._check_connectivity(shaded_cells, grid):
+            return False
+
+        return True
+
+    def _check_no_duplicates(self, shaded_cells: Set[Tuple[int, int]], grid: List[List[int]]) -> bool:
+        """
+        检查每行每列在非黑色单元格中没有重复数字
+        """
+        size = len(grid)
+
+        # 检查每行
+        for i in range(size):
+            row_values = []
+            for j in range(size):
+                if (i, j) not in shaded_cells:  # 如果不是黑色单元格
+                    row_values.append(grid[i][j])
+
+            if len(row_values) != len(set(row_values)):
+                return False
+
+        # 检查每列
+        for j in range(size):
+            col_values = []
+            for i in range(size):
+                if (i, j) not in shaded_cells:  # 如果不是黑色单元格
+                    col_values.append(grid[i][j])
+
+            if len(col_values) != len(set(col_values)):
+                return False
+
+        return True
+
+    def _check_no_adjacent_shaded(self, shaded_cells: Set[Tuple[int, int]], size: int) -> bool:
+        """
+        检查黑色单元格不相邻
+        """
+        for r, c in shaded_cells:
+            neighbors = [(r + 1, c), (r - 1, c), (r, c + 1), (r, c - 1)]
+            for nr, nc in neighbors:
+                if 0 <= nr < size and 0 <= nc < size and (nr, nc) in shaded_cells:
+                    return False
+        return True
+
+    def _check_connectivity(self, shaded_cells: Set[Tuple[int, int]], grid: List[List[int]]) -> bool:
+        """
+        检查所有白色单元格连通
+        """
+        size = len(grid)
+
+        # 获取所有白色单元格
+        white_cells = []
+        for r in range(size):
+            for c in range(len(grid[r])):
+                if (r, c) not in shaded_cells:
+                    white_cells.append((r, c))
+
+        if not white_cells:
+            return False
+
+        # 使用BFS检查连通性
+        visited = set()
+        queue = [white_cells[0]]
+        visited.add(white_cells[0])
+
+        while queue:
+            r, c = queue.pop(0)
+            neighbors = [(r + 1, c), (r - 1, c), (r, c + 1), (r, c - 1)]
+            for nr, nc in neighbors:
+                if (
+                    0 <= nr < size and 0 <= nc < len(grid[nr])
+                    and (nr, nc) not in shaded_cells
+                    and (nr, nc) not in visited
+                ):
+                    visited.add((nr, nc))
+                    queue.append((nr, nc))
+
+        return len(visited) == len(white_cells)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..f4bbe4fb8c0a4ffd4f5e5b5263b5798f237506c4
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py
@@ -0,0 +1,598 @@
+import json
+import re
+from ast import literal_eval
+from typing import Any, Dict, List, Optional, Tuple
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str) -> bool:
+        """
+        Evaluate if the predicted answer is correct.
+
+        Args:
+            predicted_answer: The model's predicted answer
+            ground_truth: The ground truth answer (may not be used depending on implementation)
+            initial_state: The initial state of the puzzle
+
+        Returns:
+            bool: True if the answer is correct
+        """
+        raise NotImplementedError
+
+
+class KakuroEvaluator(BaseEvaluator):
+
+    def extract_answer(self, model_output: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Enhanced robust extraction of answer from model output with comprehensive patterns."""
+        if not model_output or not isinstance(model_output, str):
+            return None
+
+        # Clean the input text
+        text = model_output.strip()
+
+        # Strategy 1: Look for answer blocks first
+        answer_blocks = re.findall(r'\[answer\](.*?)\[/answer\]', text, re.DOTALL | re.IGNORECASE)
+
+        # Strategy 2: Look for other common answer delimiters
+        if not answer_blocks:
+            delimiters = [
+                r'<answer>(.*?)</answer>',
+                r'answer:\s*(.*?)(?:\n\n|\n$|$)',
+                r'solution:\s*(.*?)(?:\n\n|\n$|$)',
+                r'final answer:\s*(.*?)(?:\n\n|\n$|$)',
+                r'my answer:\s*(.*?)(?:\n\n|\n$|$)',
+            ]
+
+            for delimiter in delimiters:
+                matches = re.findall(delimiter, text, re.DOTALL | re.IGNORECASE)
+                if matches:
+                    answer_blocks = matches
+                    break
+
+        # Strategy 3: If no delimited blocks found, use the entire text
+        if not answer_blocks:
+            answer_blocks = [text]
+
+        # Process the last (most likely) answer block
+        last_block = answer_blocks[-1].strip()
+
+        # Multiple extraction strategies for maximum robustness
+        strategies = [
+            self._extract_coordinate_value_patterns,
+            self._extract_dict_literal,
+            self._extract_structured_formats,
+            self._extract_table_format,
+            self._extract_loose_patterns,
+            self._extract_json_like_formats,
+        ]
+
+        for strategy in strategies:
+            try:
+                result = strategy(last_block)
+                if self._is_valid_answer_format(result):
+                    return result
+            except Exception:
+                # Continue to next strategy if current one fails
+                continue
+
+        return None
+
+    def _is_valid_answer_format(self, result: Any) -> bool:
+        """Check if the extracted result is in valid answer format."""
+        return (
+            result is not None
+            and isinstance(result, dict)
+            and len(result) > 0
+            and all(
+                isinstance(k, tuple)
+                and len(k) == 2
+                and all(isinstance(coord, int) for coord in k) for k in result.keys()
+            )
+            and all(isinstance(v, int) and 1 <= v <= 9 for v in result.values())
+        )
+
+    def _extract_coordinate_value_patterns(self, text: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Extract coordinate-value patterns with comprehensive regex."""
+        result = {}
+
+        # Multiple coordinate-value patterns to handle various formats
+        patterns = [
+            # Standard format: (row,col):value
+            r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*:\s*(\d+)',
+            # With equals: (row,col)=value
+            r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*=\s*(\d+)',
+            # With arrow: (row,col)->value or (row,col) -> value
+            r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*->\s*(\d+)',
+            # Bracket format: [row,col]:value
+            r'\[\s*(\d+)\s*,\s*(\d+)\s*\]\s*:\s*(\d+)',
+            # Cell format: cell(row,col):value or Cell (row,col): value
+            r'(?:cell|Cell)\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[:=]\s*(\d+)',
+            # Position format: pos(row,col):value
+            r'(?:pos|position|Position)\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[:=]\s*(\d+)',
+            # Coordinate format: coord(row,col):value
+            r'(?:coord|coordinate|Coordinate)\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[:=]\s*(\d+)',
+            # R,C format: R0C1:value or r0c1=value
+            r'[Rr](\d+)[Cc](\d+)\s*[:=]\s*(\d+)',
+        ]
+
+        for pattern in patterns:
+            matches = re.findall(pattern, text, re.IGNORECASE)
+            for row, col, value in matches:
+                try:
+                    result[(int(row), int(col))] = int(value)
+                except ValueError:
+                    continue
+
+        return result if result else None
+
+    def _extract_dict_literal(self, text: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Extract Python dictionary literal format with error handling."""
+        # Try to find dictionary-like structures
+        dict_patterns = [
+            r'\{[^}]*\}',  # Basic dictionary pattern
+            r'{\s*[\s\S]*?\s*}',  # Multi-line dictionary
+        ]
+
+        for pattern in dict_patterns:
+            dict_matches = re.findall(pattern, text, re.DOTALL)
+
+            for dict_str in dict_matches:
+                try:
+                    # Try direct evaluation
+                    answer_dict = literal_eval(dict_str)
+                    if isinstance(answer_dict, dict):
+                        result = {}
+                        for k, v in answer_dict.items():
+                            if isinstance(k, str) and k.startswith('(') and k.endswith(')'):
+                                # Handle "(0,1)" format string keys
+                                coord_str = k.strip('()')
+                                row, col = map(int, coord_str.split(','))
+                                result[(row, col)] = int(v)
+                            elif isinstance(k, tuple) and len(k) == 2:
+                                result[k] = int(v)
+                            elif isinstance(k, str) and ',' in k:
+                                # Handle "0,1" format
+                                row, col = map(int, k.split(','))
+                                result[(row, col)] = int(v)
+
+                        if result:
+                            return result
+
+                except (ValueError, SyntaxError):
+                    # Try manual parsing for malformed dictionaries
+                    result = self._parse_malformed_dict(dict_str)
+                    if result:
+                        return result
+
+        return None
+
+    def _parse_malformed_dict(self, dict_str: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Parse malformed dictionary strings manually."""
+        result = {}
+
+        # Remove outer braces and split by commas
+        content = dict_str.strip('{}')
+
+        # Split by commas, but be careful about commas within coordinates
+        items = re.split(r',(?=\s*["\'\(])', content)
+
+        for item in items:
+            item = item.strip()
+            # Look for key-value pairs
+            kv_patterns = [
+                r'["\']?\(\s*(\d+)\s*,\s*(\d+)\s*\)["\']?\s*:\s*(\d+)',
+                r'["\']?(\d+)\s*,\s*(\d+)["\']?\s*:\s*(\d+)',
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*:\s*(\d+)',
+            ]
+
+            for pattern in kv_patterns:
+                match = re.search(pattern, item)
+                if match:
+                    row, col, value = match.groups()
+                    try:
+                        result[(int(row), int(col))] = int(value)
+                        break
+                    except ValueError:
+                        continue
+
+        return result if result else None
+
+    def _extract_structured_formats(self, text: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Extract structured formats like lists, comma-separated, newline-separated."""
+        result = {}
+
+        # Format 1: List-like formats
+        list_patterns = [
+            r'\[\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*,\s*(\d+)\s*\]',  # [(0,1), 3]
+            r'\(\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*,\s*(\d+)\s*\)',  # ((0,1), 3)
+            r'\[\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*\]',  # [0, 1, 3]
+        ]
+
+        for pattern in list_patterns:
+            matches = re.findall(pattern, text)
+            for match in matches:
+                if len(match) == 3:
+                    try:
+                        row, col, value = map(int, match)
+                        result[(row, col)] = value
+                    except ValueError:
+                        continue
+
+        # Format 2: Space-separated entries
+        lines = text.split('\n')
+        for line in lines:
+            line = line.strip()
+            if line:
+                # Try to extract from each line
+                coord_matches = re.findall(r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[:=]\s*(\d+)', line)
+                for row, col, value in coord_matches:
+                    try:
+                        result[(int(row), int(col))] = int(value)
+                    except ValueError:
+                        continue
+
+        return result if result else None
+
+    def _extract_table_format(self, text: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Extract table-like formats."""
+        result = {}
+        lines = text.split('\n')
+
+        # Look for table-like structures
+        for line in lines:
+            line = line.strip()
+            if not line or line.startswith(('|', '-', '+')):
+                continue
+
+            # Table row format: | (0,1) | 3 | or similar
+            table_patterns = [
+                r'\|\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*\|\s*(\d+)\s*\|',
+                r'(\d+)\s*,\s*(\d+)\s*\|\s*(\d+)',
+                r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*\|\s*(\d+)',
+            ]
+
+            for pattern in table_patterns:
+                matches = re.findall(pattern, line)
+                for match in matches:
+                    if len(match) == 3:
+                        try:
+                            row, col, value = map(int, match)
+                            result[(row, col)] = value
+                        except ValueError:
+                            continue
+
+        return result if result else None
+
+    def _extract_loose_patterns(self, text: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Loose pattern matching for edge cases and unusual formats."""
+        result = {}
+
+        # Very flexible patterns
+        lines = text.split('\n')
+        for line in lines:
+            line = line.strip()
+            if not line:
+                continue
+
+            # Pattern 1: Any format with "row" and "col" keywords
+            row_col_patterns = [
+                r'(?:row|r)\s*[:=]?\s*(\d+).*?(?:col|column|c)\s*[:=]?\s*(\d+).*?(?:value|val|v)\s*[:=]?\s*(\d+)',
+                r'(?:col|column|c)\s*[:=]?\s*(\d+).*?(?:row|r)\s*[:=]?\s*(\d+).*?(?:value|val|v)\s*[:=]?\s*(\d+)',
+            ]
+
+            for pattern in row_col_patterns:
+                matches = re.findall(pattern, line, re.IGNORECASE)
+                for match in matches:
+                    try:
+                        if pattern.startswith('(?:row'):
+                            row, col, value = map(int, match)
+                        else:
+                            col, row, value = map(int, match)
+                        result[(row, col)] = value
+                    except ValueError:
+                        continue
+
+            # Pattern 2: Simple number sequences that might represent coordinates and values
+            # Format: "row col value" on separate lines or separated by spaces/commas
+            number_sequences = re.findall(r'\b(\d+)\s*[,\s]\s*(\d+)\s*[,\s:=]\s*(\d+)\b', line)
+            for row, col, value in number_sequences:
+                try:
+                    r, c, v = int(row), int(col), int(value)
+                    # Basic sanity check: coordinates shouldn't be too large and value should be 1-9
+                    if 0 <= r <= 20 and 0 <= c <= 20 and 1 <= v <= 9:
+                        result[(r, c)] = v
+                except ValueError:
+                    continue
+
+        return result if result else None
+
+    def _extract_json_like_formats(self, text: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Extract JSON-like formats that might not be valid Python."""
+        result = {}
+
+        # Find JSON-like structures
+        json_patterns = [
+            r'\{[^}]*\}',
+            r'{\s*[\s\S]*?\s*}',
+        ]
+
+        for pattern in json_patterns:
+            json_matches = re.findall(pattern, text, re.DOTALL)
+
+            for json_str in json_matches:
+                try:
+                    # Try JSON parsing first
+                    import json as json_module
+                    data = json_module.loads(json_str)
+                    if isinstance(data, dict):
+                        for k, v in data.items():
+                            if isinstance(k, str):
+                                # Handle various key formats
+                                if k.startswith('(') and k.endswith(')'):
+                                    coord_str = k.strip('()')
+                                    row, col = map(int, coord_str.split(','))
+                                    result[(row, col)] = int(v)
+                                elif ',' in k:
+                                    row, col = map(int, k.split(','))
+                                    result[(row, col)] = int(v)
+
+                        if result:
+                            return result
+                except (json.JSONDecodeError, ValueError):
+                    continue
+
+        return result if result else None
+
+    def parse_solution_string(self, solution_str: str) -> Optional[Dict[Tuple[int, int], int]]:
+        """Parse a solution string using the enhanced extraction methods."""
+        return self.extract_answer(solution_str)
+
+    def validate_kakuro_solution(self, predicted_answer: Dict[Tuple[int, int], int], initial_state: str) -> bool:
+        """
+        Validate a Kakuro solution based only on the initial state and game rules.
+
+        Args:
+            predicted_answer: Dictionary mapping (row, col) tuples to digit values
+            initial_state: JSON string representing the grid structure
+
+        Returns:
+            bool: True if the solution is valid, False otherwise
+        """
+        # Support both JSON string and already-parsed Python structures
+        try:
+            if isinstance(initial_state, str):
+                grid = json.loads(initial_state)
+            else:
+                grid = initial_state  # assume it's already a parsed list[list[dict]]
+        except (json.JSONDecodeError, TypeError):
+            return False
+
+        if not isinstance(predicted_answer, dict):
+            return False
+
+        if not predicted_answer:  # Empty answer
+            return False
+
+        rows = len(grid)
+        cols = len(grid[0]) if rows > 0 else 0
+
+        # 1. Check that all values are digits 1-9
+        for coord, value in predicted_answer.items():
+            if not isinstance(value, int) or value < 1 or value > 9:
+                return False
+
+            # Check that coordinate is valid
+            if not isinstance(coord, tuple) or len(coord) != 2:
+                return False
+
+            row, col = coord
+            if row < 0 or row >= rows or col < 0 or col >= cols:
+                return False
+
+            # Check that the cell is actually a white cell
+            if grid[row][col]['type'] != 'white':
+                return False
+
+        # 2. Check that all white cells are filled
+        white_cells = set()
+        for i in range(rows):
+            for j in range(cols):
+                if grid[i][j]['type'] == 'white':
+                    white_cells.add((i, j))
+
+        if set(predicted_answer.keys()) != white_cells:
+            return False
+
+        # 3. Validate all constraint runs
+        for i in range(rows):
+            for j in range(cols):
+                cell = grid[i][j]
+                if cell['type'] == 'black':
+                    # Check right constraint
+                    if 'right' in cell:
+                        right_hint = cell['right']
+                        # Determine target sum and run length policy
+                        target_sum: Optional[int] = None
+                        expected_length: Optional[int] = None
+                        if isinstance(right_hint, (list, tuple)) and len(right_hint) == 2:
+                            try:
+                                target_sum = int(right_hint[0])
+                                expected_length = int(right_hint[1])
+                            except (ValueError, TypeError):
+                                return False
+                        else:
+                            try:
+                                target_sum = int(right_hint)
+                            except (ValueError, TypeError):
+                                return False
+
+                        # Collect cells in the right run until a non-white cell or boundary
+                        run_cells: List[Tuple[int, int]] = []
+                        k = j + 1
+                        while k < cols and grid[i][k]['type'] == 'white':
+                            run_cells.append((i, k))
+                            k += 1
+
+                        # If length was provided, enforce it; otherwise infer from layout
+                        if expected_length is not None and len(run_cells) != expected_length:
+                            return False
+
+                        # Check all cells are in prediction
+                        if not all(rc in predicted_answer for rc in run_cells):
+                            return False
+
+                        # Check sum
+                        actual_sum = sum(predicted_answer[rc] for rc in run_cells)
+                        if actual_sum != target_sum:
+                            return False
+
+                        # Check uniqueness (no repeated digits)
+                        values = [predicted_answer[rc] for rc in run_cells]
+                        if len(set(values)) != len(values):
+                            return False
+
+                    # Check down constraint
+                    if 'down' in cell:
+                        down_hint = cell['down']
+                        target_sum: Optional[int] = None
+                        expected_length: Optional[int] = None
+                        if isinstance(down_hint, (list, tuple)) and len(down_hint) == 2:
+                            try:
+                                target_sum = int(down_hint[0])
+                                expected_length = int(down_hint[1])
+                            except (ValueError, TypeError):
+                                return False
+                        else:
+                            try:
+                                target_sum = int(down_hint)
+                            except (ValueError, TypeError):
+                                return False
+
+                        # Collect cells in the down run until a non-white cell or boundary
+                        run_cells: List[Tuple[int, int]] = []
+                        k = i + 1
+                        while k < rows and grid[k][j]['type'] == 'white':
+                            run_cells.append((k, j))
+                            k += 1
+
+                        # If length was provided, enforce it; otherwise infer from layout
+                        if expected_length is not None and len(run_cells) != expected_length:
+                            return False
+
+                        # Check all cells are in prediction
+                        if not all(rc in predicted_answer for rc in run_cells):
+                            return False
+
+                        # Check sum
+                        actual_sum = sum(predicted_answer[rc] for rc in run_cells)
+                        if actual_sum != target_sum:
+                            return False
+
+                        # Check uniqueness (no repeated digits)
+                        values = [predicted_answer[rc] for rc in run_cells]
+                        if len(set(values)) != len(values):
+                            return False
+
+        return True
+
+    def _normalize_answer(self, ans: Any) -> Optional[Dict[Tuple[int, int], int]]:
+        """Convert various answer formats to {(row, col): val} dict; return None if invalid/empty."""
+        if ans is None:
+            return None
+
+        # 字符串：用已有的 extract_answer 做鲁棒解析
+        if isinstance(ans, str):
+            ans = self.extract_answer(ans)
+        # 依然可能是字符串解析失败，或已经是 dict
+        if not isinstance(ans, dict) or not ans:
+            return None
+
+        # 统一把字符串坐标转成 tuple
+        normalized: Dict[Tuple[int, int], int] = {}
+        for k, v in ans.items():
+            try:
+                if isinstance(k, tuple) and len(k) == 2:
+                    r, c = int(k[0]), int(k[1])
+                elif isinstance(k, str) and k.startswith('(') and k.endswith(')'):
+                    r, c = map(int, k.strip('()').split(','))
+                elif isinstance(k, str) and ',' in k:
+                    r, c = map(int, k.split(','))
+                else:
+                    return None
+                v = int(v)
+                if not (1 <= v <= 9):
+                    return None
+                normalized[(r, c)] = v
+            except Exception:
+                return None
+        return normalized if normalized else None
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str) -> bool:
+        """
+        如果提供了 ground_truth，则优先做“答案对答案”的精确匹配；
+        否则才进行基于初始盘面规则的完整解校验。
+        """
+        # 1) 若同时提供了 GT：做精确匹配（支持单格/部分答案评测）
+        gt_norm = self._normalize_answer(ground_truth) if ground_truth is not None else None
+        if gt_norm is not None:
+            pred_norm = self._normalize_answer(predicted_answer)
+            return pred_norm is not None and pred_norm == gt_norm
+
+        # 2) 无 GT：按规则校验完整解
+        pred_norm = self._normalize_answer(predicted_answer)
+        if pred_norm is None:
+            return False
+        return self.validate_kakuro_solution(pred_norm, initial_state)
+
+    # def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str) -> bool:
+    #     """
+    #     Evaluate if the predicted answer is correct based only on initial state and game rules.
+    #     Note: ground_truth is ignored in this implementation as evaluation is rule-based.
+
+    #     Args:
+    #         predicted_answer: The model's predicted answer (dict, string, or raw model output)
+    #         ground_truth: The ground truth answer (ignored in this implementation)
+    #         initial_state: JSON string representing the grid structure
+
+    #     Returns:
+    #         bool: True if the solution satisfies all game constraints
+    #     """
+    #     if predicted_answer is None:
+    #         return False
+
+    #     # If it's a string, try to extract the answer using robust parsing
+    #     if isinstance(predicted_answer, str):
+    #         predicted_answer = self.extract_answer(predicted_answer)
+
+    #     if predicted_answer is None:
+    #         return False
+
+    #     if not isinstance(predicted_answer, dict):
+    #         return False
+
+    #     # Convert string coordinates to tuples if needed
+    #     if predicted_answer and all(isinstance(k, str) for k in predicted_answer.keys()):
+    #         converted_answer = {}
+    #         for k, v in predicted_answer.items():
+    #             try:
+    #                 if isinstance(k, str) and k.startswith('(') and k.endswith(')'):
+    #                     coord_str = k.strip('()')
+    #                     row, col = map(int, coord_str.split(','))
+    #                     converted_answer[(row, col)] = int(v)
+    #                 elif isinstance(k, str) and ',' in k:
+    #                     row, col = map(int, k.split(','))
+    #                     converted_answer[(row, col)] = int(v)
+    #                 else:
+    #                     return False
+    #             except (ValueError, AttributeError):
+    #                 return False
+    #         predicted_answer = converted_answer
+
+    #     # Validate the solution using only initial_state and game rules
+    #     return self.validate_kakuro_solution(predicted_answer, initial_state)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/kukurasu_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/kukurasu_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..2afc901da1ce6e9edf515576c8f9eed3036f5a6c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/kukurasu_eval.py
@@ -0,0 +1,156 @@
+import json
+import re
+from typing import Any, Dict, List, Union
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class KukurasuEvaluator(BaseEvaluator):
+    """
+    评估Kukurasu谜题解答的评估器
+    Kukurasu是一种填充黑白格子的谜题，要求：
+    1. 每行黑格子的列位置之和等于给定的行约束
+    2. 每列黑格子的行位置之和等于给定的列约束
+    """
+
+    def extract_answer(self, model_output: str) -> List[List[int]]:
+        """
+        从模型输出中提取Kukurasu解答矩阵
+
+        Args:
+            model_output: 模型生成的字符串输出
+
+        Returns:
+            提取的二维列表，表示Kukurasu的解答
+        """
+        # 使用正则表达式寻找类似 [[1, 1, 1], [1, 0, 0], [1, 1, 0]] 的模式
+        pattern = r'\[\s*\[(?:\s*\d+\s*,\s*)*\s*\d+\s*\](?:\s*,\s*\[\s*(?:\d+\s*,\s*)*\d+\s*\])*\s*\]'
+        matches = re.findall(pattern, model_output)
+
+        if not matches:
+            # 如果没有找到符合格式的答案，返回空列表
+            return []
+
+        try:
+            # 尝试解析找到的第一个匹配项
+            answer_matrix = json.loads(matches[0])
+            # 确保它是一个二维列表，且每个元素都是整数
+            if (
+                isinstance(answer_matrix, list)
+                and all(
+                    isinstance(row, list)
+                    and all(isinstance(item, int) for item in row)
+                    for row in answer_matrix
+                )
+            ):
+                return answer_matrix
+            return []
+        except json.JSONDecodeError:
+            # 如果JSON解析失败，返回空列表
+            return []
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """
+        准备用于解决Kukurasu谜题的提示词
+
+        Args:
+            question: 问题描述
+            params: 包含谜题信息的参数，如行列约束和尺寸
+
+        Returns:
+            格式化的提示词
+        """
+        row_sums = params.get("row_sums", [])
+        col_sums = params.get("col_sums", [])
+        size = params.get("size", len(row_sums))
+
+        prompt = f"""请解决以下Kukurasu谜题:
+
+    这是一个 {size}x{size} 的网格，需要填充黑色格子(1)和白色格子(0)。
+
+    规则:
+    1. 每行黑格子的列位置之和等于行约束
+    2. 每列黑格子的行位置之和等于列约束
+
+    列位置指的是从左到右的列索引(从1开始)，行位置指的是从上到下的行索引(从1开始)。
+
+    行约束: {row_sums}
+    列约束: {col_sums}
+
+    例如，如果一行的约束是6，且该行在第1、2、3列有黑格子，那么 1+2+3=6，满足约束。
+    同样，如果一列的约束是4，且该列在第1、3行有黑格子，那么 1+3=4，满足约束。
+
+    请给出网格的填充方案，格式为二维数组，其中1表示黑格子，0表示白格子。
+    例如: [[1, 1, 1], [1, 0, 0], [1, 1, 0]]
+
+    解题过程:
+    1. 分析行列约束
+    2. 确定每个格子的颜色(黑/白)
+    3. 验证所有约束是否满足
+    4. 提供最终的网格填充方案
+    """
+
+        return prompt
+
+    def evaluate(self, output: Union[str, List[List[int]]], ground_truth: List[List[int]],
+                 params: Dict[str, Any]) -> bool:
+        """
+        评估模型的解答是否正确，基于Kukurasu规则验证
+
+        Args:
+            output: 模型的原始输出字符串或已提取的答案
+            ground_truth: 正确的解答（仅用于参考，不直接比对）
+            params: 包含行列约束和尺寸的参数
+
+        Returns:
+            解答是否正确的布尔值
+        """
+        # 检查output类型，如果是字符串，则提取答案；如果已经是列表，则直接使用
+        if isinstance(output, str):
+            predicted_answer = self.extract_answer(output)
+        else:
+            predicted_answer = output
+
+        # 如果无法提取有效答案，直接返回False
+        if not predicted_answer:
+            return False
+
+        # 获取行列约束和尺寸
+        row_sums = params.get("row_sums", [])
+        col_sums = params.get("col_sums", [])
+        size = params.get("size", len(row_sums))
+
+        # 检查答案维度是否正确
+        if len(predicted_answer) != size:
+            return False
+
+        if any(len(row) != size for row in predicted_answer):
+            return False
+
+        # 检查格子值是否只包含0和1
+        if any(cell not in [0, 1] for row in predicted_answer for cell in row):
+            return False
+
+        # 验证每行的约束
+        for i, row in enumerate(predicted_answer):
+            row_sum = sum((j + 1) for j, cell in enumerate(row) if cell == 1)
+            if row_sum != row_sums[i]:
+                return False
+
+        # 验证每列的约束
+        for j in range(size):
+            col_sum = sum((i + 1) for i, row in enumerate(predicted_answer) if row[j] == 1)
+            if col_sum != col_sums[j]:
+                return False
+
+        # 所有约束都满足，返回True
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/maze_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/maze_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..1591c4c7dc4bd1a891343f1398fec84eef2c6710
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/maze_eval.py
@@ -0,0 +1,34 @@
+from typing import Any, Dict
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self, predicted_answer: Any, ground_truth: Any,
+        initial_state: str, params: Dict[str, Any] = None
+    ) -> bool:
+        raise NotImplementedError
+
+
+class MazeEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        from utils.constants import PROMPT_MAZE
+        return PROMPT_MAZE.format(question)
+
+    def extract_answer(self, model_output: str) -> str:
+        answer = model_output.strip().lower()
+
+        valid_directions = ['up', 'down', 'left', 'right']
+        words = answer.split()
+        directions = [word for word in words if word in valid_directions]
+
+        return " ".join(directions)
+
+    def evaluate(self, predicted_answer: str, ground_truth: Any, initial_state: Any) -> bool:
+        from vlmeval.dataset.utils.mmhelix.utils.validation import maze_check
+        return maze_check(initial_state, predicted_answer)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/minesweeper_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/minesweeper_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..ac6ba00eb525d5183d8e20a7df3ce9b5854c8b1b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/minesweeper_eval.py
@@ -0,0 +1,86 @@
+import re
+from typing import Any, Dict
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str,
+                 params: Dict[str, Any] = None) -> bool:
+        raise NotImplementedError
+
+
+class MinesweeperEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        from utils.constants import PROMPT_MINESWEEPER
+        return PROMPT_MINESWEEPER.format(question)
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate minesweeper solution by comparing predicted mine coordinates with ground truth.
+        Both predicted_answer and ground_truth should be lists of (row, col) tuples.
+        """
+        # Extract coordinates from predicted answer if it's a string
+        if isinstance(predicted_answer, str):
+            pred_coordinates = self._extract_coordinates(predicted_answer)
+        elif isinstance(predicted_answer, list):
+            pred_coordinates = set(predicted_answer)
+        else:
+            return False
+
+        # Extract coordinates from ground truth
+        if isinstance(ground_truth, str):
+            truth_coordinates = self._extract_coordinates(ground_truth)
+        elif isinstance(ground_truth, list):
+            truth_coordinates = set(ground_truth)
+        else:
+            return False
+
+        # Compare the coordinate sets
+        return pred_coordinates == truth_coordinates
+
+    def _extract_coordinates(self, coord_str: str) -> set:
+        """
+        Extract coordinates from string format like "(0,5),(0,7),(1,1),(1,2)"
+        Returns a set of (row, col) tuples
+        """
+        coordinates = set()
+
+        # Pattern to match coordinates like (0,5) or (0, 5)
+        pattern = r'\((\d+)\s*,\s*(\d+)\)'
+        matches = re.findall(pattern, coord_str)
+
+        for match in matches:
+            row, col = int(match[0]), int(match[1])
+            coordinates.add((row, col))
+
+        return coordinates
+
+    def extract_answer(self, model_output: str) -> str:
+        """
+        Extract coordinate list from the model output.
+        Look for content within <answer></answer> tags or coordinate patterns.
+        """
+        # First try to extract content within <answer></answer> tags
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+
+        if match:
+            content = match.group(1).strip()
+        else:
+            # If no tags found, use the whole output
+            content = model_output.strip()
+
+        # Look for coordinate patterns like (0,5),(0,7),(1,1),(1,2)
+        coord_pattern = r'(\(\d+\s*,\s*\d+\)(?:\s*,\s*\(\d+\s*,\s*\d+\))*)'
+        coord_match = re.search(coord_pattern, content)
+
+        if coord_match:
+            return coord_match.group(1).strip()
+
+        # If no clear coordinate pattern found, return the content as is
+        return content
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/nibbles_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/nibbles_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..5412ad09d3fbea80a96d43fe5b35d6cfebe7ba02
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/nibbles_eval.py
@@ -0,0 +1,255 @@
+import re
+from typing import Any, Dict
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str,
+                 params: Dict[str, Any] = None) -> bool:
+        raise NotImplementedError
+
+
+class NibblesEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        pass
+
+    def extract_answer(self, model_output: str) -> str:
+        """Extract movement directions from model output"""
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+        if match:
+            content = match.group(1).strip()
+        else:
+            content = model_output.strip()  # Fallback to full output if no tags found
+
+        # Extract valid movement directions
+        directions = []
+        words = content.lower().split()
+        valid_moves = {'up', 'down', 'left', 'right'}
+
+        for word in words:
+            if word in valid_moves:
+                directions.append(word)
+
+        return ' '.join(directions)
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        """
+        Evaluate Snake game solution by simulating the game
+
+        Args:
+            predicted_answer: String of movement directions (e.g., "up down left right")
+            ground_truth: Expected answer (for reference, but we verify by game simulation)
+            initial_state: String containing grid size, snake position, direction, and apples
+        """
+        try:
+            # Parse initial state
+            game_state = self._parse_initial_state(initial_state)
+            if not game_state:
+                return False
+
+            # Strictly validate that provided Direction matches snake geometry
+            if not self._is_direction_consistent_with_snake(game_state):
+                return False
+
+            # Parse predicted moves
+            if isinstance(predicted_answer, str):
+                predicted_moves = predicted_answer.strip().split()
+            else:
+                predicted_moves = str(predicted_answer).strip().split()
+
+            # Validate moves
+            valid_moves = {'up', 'down', 'left', 'right'}
+            predicted_moves = [move.lower() for move in predicted_moves if move.lower() in valid_moves]
+
+            if not predicted_moves:
+                return False
+
+            # Simulate the game
+            result = self._simulate_snake_game(game_state, predicted_moves)
+
+            if result['success']:
+                return True
+            else:
+                return False
+
+        except Exception:
+            return False
+
+    def _parse_initial_state(self, initial_state: str) -> dict:
+        """Parse the initial state string to extract game information"""
+        try:
+            lines = initial_state.strip().split('\n')
+            game_info = {}
+
+            for line in lines:
+                line = line.strip()
+                if line.startswith('Grid:'):
+                    # Parse grid size, e.g., "Grid: 7x6"
+                    size_str = line.split(':')[1].strip()
+                    if 'x' in size_str:
+                        rows, cols = map(int, size_str.split('x'))
+                        game_info['rows'] = rows
+                        game_info['cols'] = cols
+
+                elif line.startswith('Snake:'):
+                    # Parse snake positions, e.g., "Snake: (4,2) (4,1)"
+                    coords_str = line.split(':')[1].strip()
+                    coords_pattern = r'\((\d+),(\d+)\)'
+                    matches = re.findall(coords_pattern, coords_str)
+                    game_info['snake'] = [(int(r), int(c)) for r, c in matches]
+
+                elif line.startswith('Direction:'):
+                    # Parse initial direction, e.g., "Direction: left"
+                    direction = line.split(':')[1].strip().lower()
+                    game_info['direction'] = direction
+
+                elif line.startswith('Apples:'):
+                    # Parse apple positions, e.g., "Apples: (5,4)"
+                    coords_str = line.split(':')[1].strip()
+                    coords_pattern = r'\((\d+),(\d+)\)'
+                    matches = re.findall(coords_pattern, coords_str)
+                    game_info['apples'] = set((int(r), int(c)) for r, c in matches)
+
+                elif line.startswith('Goal:'):
+                    # Parse goal information for validation
+                    goal_str = line.split(':')[1].strip()
+                    # Extract number of apples to eat
+                    num_match = re.search(r'(\d+)', goal_str)
+                    if num_match:
+                        game_info['target_apples'] = int(num_match.group(1))
+
+            # Validate required fields
+            required_fields = ['rows', 'cols', 'snake', 'direction', 'apples']
+            if all(field in game_info for field in required_fields):
+                return game_info
+            else:
+                return None
+
+        except Exception:
+            return None
+
+    def _simulate_snake_game(self, game_state: dict, moves: list) -> dict:
+        """
+        Simulate the Snake game with given moves
+
+        Returns:
+            dict with success status, error message, and statistics
+        """
+        # Initialize game state
+        rows, cols = game_state['rows'], game_state['cols']
+        snake = list(game_state['snake'])  # List of (row, col) positions
+        apples = set(game_state['apples'])  # Set of (row, col) positions
+        total_apples = len(apples)
+        apples_eaten = 0
+
+        # Use provided initial direction strictly
+        direction = game_state['direction']
+
+        # Direction vectors
+        direction_vectors = {
+            'up': (-1, 0),
+            'down': (1, 0),
+            'left': (0, -1),
+            'right': (0, 1)
+        }
+
+        # Opposite directions (can't reverse)
+        opposites = {
+            'up': 'down',
+            'down': 'up',
+            'left': 'right',
+            'right': 'left'
+        }
+
+        for i, move in enumerate(moves):
+            # Check if trying to reverse direction
+            if move == opposites.get(direction):
+                return {
+                    'success': False,
+                    'error': f"Invalid move at step {i+1}: Cannot reverse direction from {direction} to {move}",
+                    'apples_eaten': apples_eaten,
+                    'total_apples': total_apples
+                }
+
+            # Update direction
+            direction = move
+
+            # Calculate new head position
+            head_row, head_col = snake[0]
+            dr, dc = direction_vectors[direction]
+            new_head = (head_row + dr, head_col + dc)
+
+            # Check bounds
+            if not (0 <= new_head[0] < rows and 0 <= new_head[1] < cols):
+                return {
+                    'success': False,
+                    'error': f"Hit boundary at step {i+1}: position {new_head} out of bounds ({rows}x{cols})",
+                    'apples_eaten': apples_eaten,
+                    'total_apples': total_apples
+                }
+
+            # Check self-collision
+            if new_head in snake:
+                return {
+                    'success': False,
+                    'error': f"Self-collision at step {i+1}: position {new_head} already occupied by snake",
+                    'apples_eaten': apples_eaten,
+                    'total_apples': total_apples
+                }
+
+            # Move snake
+            snake.insert(0, new_head)
+
+            # Check if apple eaten
+            if new_head in apples:
+                apples.remove(new_head)
+                apples_eaten += 1
+                # Snake grows (don't remove tail)
+            else:
+                # No apple eaten, remove tail
+                snake.pop()
+
+        # Check if all apples eaten
+        if apples_eaten == total_apples:
+            return {
+                'success': True,
+                'error': None,
+                'apples_eaten': apples_eaten,
+                'total_apples': total_apples
+            }
+        else:
+            return {
+                'success': False,
+                'error': f"Not all apples eaten: {apples_eaten}/{total_apples}",
+                'apples_eaten': apples_eaten,
+                'total_apples': total_apples
+            }
+
+    def _is_direction_consistent_with_snake(self, game_state: dict) -> bool:
+        """Check that the provided initial direction matches the orientation implied by the first two snake segments."""
+        snake = game_state.get('snake', [])
+        provided = game_state.get('direction')
+        if not snake or len(snake) < 2:
+            # With length 1, we cannot infer; accept provided
+            return provided in {'up', 'down', 'left', 'right'}
+        head_row, head_col = snake[0]
+        neck_row, neck_col = snake[1]
+        dr, dc = head_row - neck_row, head_col - neck_col
+        if dr == -1 and dc == 0:
+            inferred = 'up'
+        elif dr == 1 and dc == 0:
+            inferred = 'down'
+        elif dr == 0 and dc == -1:
+            inferred = 'left'
+        elif dr == 0 and dc == 1:
+            inferred = 'right'
+        else:
+            # Non-adjacent segments; invalid geometry
+            return False
+        return inferred == provided
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/nonogram_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/nonogram_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..26591dd412229a2b88b1b4ac71d77f84f8bb4d86
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/nonogram_eval.py
@@ -0,0 +1,85 @@
+import re
+from typing import Any, Dict, List
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class NonogramsEvaluator(BaseEvaluator):
+
+    def extract_answer(self, model_output: str) -> List[List[bool]]:
+        """Extract the model's answer from its output"""
+        matches = re.findall(r'\[answer\](.*?)\[/answer\]', model_output, re.DOTALL)
+        if not matches:
+            return None
+
+        grid_str = matches[-1].strip()
+        solution = []
+        for line in grid_str.split('\n'):
+            line = line.strip()
+            if not line:
+                continue
+            # Consider 'X' or 'x' as filled cells, anything else as empty
+            solution.append([c.upper() == 'X' for c in line])
+        return solution
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state=None) -> bool:
+        if predicted_answer is None:
+            return False
+
+        solution = ground_truth
+
+        if len(predicted_answer) != len(solution):
+            return False
+
+        for i, row in enumerate(predicted_answer):
+            if len(row) != len(solution[i]):
+                return False
+
+            if row != solution[i]:
+                return False
+
+        return True
+
+    @staticmethod
+    def _get_clues(line):
+        """Extract clues from a line (row or column)"""
+        clues = []
+        current = 0
+        for cell in line:
+            if cell:
+                current += 1
+            elif current > 0:
+                clues.append(current)
+                current = 0
+        if current > 0:
+            clues.append(current)
+        return clues
+
+    def _verify_clues(self, grid, params):
+        """Verify that the grid matches the given row and column clues"""
+        rows = len(grid)
+        cols = len(grid[0]) if rows > 0 else 0
+
+        # Check row clues
+        for i, row in enumerate(grid):
+            row_clues = self._get_clues(row)
+            if row_clues != params['rows'][i]:
+                return False
+
+        # Check column clues
+        for j in range(cols):
+            col = [grid[i][j] for i in range(rows)]
+            col_clues = self._get_clues(col)
+            if col_clues != params['columns'][j]:
+                return False
+
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/numbrix_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/numbrix_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..94ee8ef86cb5488f864acb0bbfaa348eebe9d8a4
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/numbrix_eval.py
@@ -0,0 +1,172 @@
+from typing import Any, Dict
+
+import numpy as np
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: str,
+                 params: Dict[str, Any] = None) -> bool:
+        raise NotImplementedError
+
+
+class NumbrixEvaluator(BaseEvaluator):
+    def __init__(self, verbose: bool = False):
+        self.verbose = verbose
+
+    def prepare_prompt(self, question: str) -> str:
+        from utils.constants import PROMPT_NUMBRIX
+        return PROMPT_NUMBRIX.format(question)
+
+    def evaluate(self, predicted_answer: str, ground_truth: Any, initial_state: Any) -> bool:
+        if self.verbose:
+            print("predicted_answer: ", predicted_answer[:255])
+            print("ground_truth: ", ground_truth[:255])
+            print("initial_state: ", initial_state)
+        predicted = self._normalize_grid(predicted_answer)
+        ground_truth = self._normalize_grid(ground_truth) if ground_truth else None
+
+        try:
+            predicted_grid = self._parse_grid(predicted)
+
+            if not self._check_number_uniqueness(predicted_grid):
+                return False
+
+            if initial_state:
+                initial_grid = self._parse_grid(self._normalize_grid(initial_state))
+
+                if initial_grid.shape[0] > predicted_grid.shape[0] or initial_grid.shape[1] > predicted_grid.shape[1]:
+                    return False
+
+                rows, cols = initial_grid.shape
+                for i in range(rows):
+                    for j in range(cols):
+                        if i < predicted_grid.shape[0] and j < predicted_grid.shape[1] and initial_grid[i, j] != 0:
+                            if predicted_grid[i, j] != initial_grid[i, j]:
+                                return False
+
+            if ground_truth:
+                ground_truth_grid = self._parse_grid(ground_truth)
+                if predicted_grid.shape != ground_truth_grid.shape:
+                    if (
+                        ground_truth_grid.shape[0] <= predicted_grid.shape[0]
+                        and ground_truth_grid.shape[1] <= predicted_grid.shape[1]
+                    ):
+                        predicted_grid = predicted_grid[:ground_truth_grid.shape[0], :ground_truth_grid.shape[1]]
+                    else:
+                        return False
+
+                if np.array_equal(predicted_grid, ground_truth_grid):
+                    return True
+
+            return self._validate_numbrix_rules(predicted_grid)
+        except Exception:
+            import traceback
+            traceback.print_exc()
+            return False
+
+    def _check_number_uniqueness(self, grid):
+        import numpy as np
+
+        numbers = grid[grid > 0]
+
+        unique_numbers, counts = np.unique(numbers, return_counts=True)
+
+        duplicates = [num for num, count in zip(unique_numbers, counts) if count > 1]
+
+        if duplicates:
+            return False
+
+        return True
+
+    def _normalize_grid(self, grid_str: str) -> str:
+        """标准化网格字符串，移除多余空格并统一换行格式"""
+        if not grid_str:
+            return ""
+        lines = [line.strip() for line in grid_str.strip().split("\n")]
+        return "\n".join(lines)
+
+    def _parse_grid(self, grid_str: str):
+        """将文本表示解析为二维数组"""
+        import numpy as np
+
+        lines = [line for line in grid_str.strip().split('\n') if line.strip()]
+        rows = []
+
+        for line in lines:
+            # 移除行首尾的'|'字符
+            if line.startswith('|'):
+                line = line[1:]
+            if line.endswith('|'):
+                line = line[:-1]
+
+            # 按'|'分割并转换为整数
+            row = []
+            for cell in line.split('|'):
+                cell = cell.strip()
+                if cell and cell.isdigit():
+                    row.append(int(cell))
+                else:
+                    row.append(0)  # 空单元格或非数字内容
+            rows.append(row)
+
+        # 确保所有行长度一致
+        if self.verbose:
+            print("rows: ", rows)
+        max_length = max(len(row) for row in rows)
+        padded_rows = [row + [0] * (max_length - len(row)) for row in rows]
+        return np.array(padded_rows)
+
+    def _validate_numbrix_rules(self, grid):
+        """
+        验证网格是否符合Numbrix规则:
+        1. 每对连续数字必须相邻(水平或垂直)
+        2. 从1到网格中的最大数字，所有数字必须存在且不重复
+        3. 可能会有空格（不需要填满每个格子）
+        """
+        import numpy as np
+
+        # 找出网格中的非零值
+        non_zero_values = grid[grid > 0]
+        if len(non_zero_values) == 0:
+            return False
+
+        max_num = np.max(non_zero_values)
+
+        # 检查从1到max_num所有数字是否存在
+        expected_nums = set(range(1, max_num + 1))
+        actual_nums = set(non_zero_values.flatten())
+
+        missing = expected_nums - actual_nums
+        extra = actual_nums - expected_nums - {0}  # 排除0（空格）
+
+        if missing:
+            return False
+
+        if extra:
+            return False
+
+        # 检查每对连续数字是否相邻
+        for num in range(1, max_num):
+            # 找到当前数字和下一个数字的位置
+            current_pos = np.where(grid == num)
+            next_pos = np.where(grid == num + 1)
+
+            if len(current_pos[0]) == 0 or len(next_pos[0]) == 0:
+                return False
+
+            r1, c1 = current_pos[0][0], current_pos[1][0]
+            r2, c2 = next_pos[0][0], next_pos[1][0]
+
+            # 计算曼哈顿距离，相邻单元格距离为1
+            manhattan_dist = abs(r2 - r1) + abs(c2 - c1)
+            if manhattan_dist != 1:
+                return False
+
+        # 验证通过
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/shingoki_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/shingoki_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..a00dec43f71d5e660c35bfe37bd24c514de7a6c2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/shingoki_eval.py
@@ -0,0 +1,795 @@
+import json
+import re
+from typing import Any, Dict, List, Tuple, Union
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class ShingokiEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        puzzle_image = params.get('image', '')
+        rows = params.get('rows', 0)
+        cols = params.get('cols', 0)
+
+        prompt = [
+            "# Shingoki Puzzle",
+            "",
+            "## Rules",
+            "- Draw a single continuous loop without crossings or branches",
+            "- White circles must be passed through in a straight line",
+            "- Black circles must be turned upon",
+            "- Numbers in circles show the sum of the lengths of the 2 straight lines going out of that circle",
+            "",
+            f"## Puzzle Grid ({rows}x{cols})",
+            "",
+            f"![Shingoki Puzzle]({puzzle_image})",
+            "",
+            "## Instructions",
+            "1. Analyze the image and identify all circles and their values",
+            "2. Solve the puzzle step-by-step",
+            "3. Provide your answer as a list of connected line segments that form the loop",
+            "",
+            "## Answer Format",
+            "Your answer should be in the following format:",
+            "```",
+            "(r1,c1)-(r2,c2) (r2,c2)-(r3,c3) ...",
+            "```",
+            "Where each (r,c) represents the row and column coordinates of a grid point.",
+        ]
+
+        return "\n".join(prompt)
+
+    def extract_answer(self, model_output: str) -> Union[List[Tuple[Tuple[int, int], Tuple[int, int]]], None]:
+        """
+        Extract the answer from the model's output with enhanced robustness
+
+        Args:
+            model_output: The text output from the model
+
+        Returns:
+            A list of line segments or None if no valid answer found
+        """
+        if not model_output or not isinstance(model_output, str):
+            return None
+
+        # Clean the input: remove extra whitespace and normalize
+        cleaned_output = re.sub(r'\s+', ' ', model_output.strip())
+
+        # Strategy 1: Look for standard format (r1,c1)-(r2,c2)
+        pattern1 = r'\((\d+),(\d+)\)-\((\d+),(\d+)\)'
+        matches1 = re.findall(pattern1, cleaned_output)
+
+        if matches1:
+            segments = []
+            for match in matches1:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 2: Look for format with spaces around coordinates (r1, c1) - (r2, c2)
+        pattern2 = r'\(\s*(\d+)\s*,\s*(\d+)\s*\)\s*[-–—]\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)'
+        matches2 = re.findall(pattern2, cleaned_output)
+
+        if matches2:
+            segments = []
+            for match in matches2:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 3: Look for format without parentheses: r1,c1-r2,c2
+        pattern3 = r'(\d+),(\d+)[-–—](\d+),(\d+)'
+        matches3 = re.findall(pattern3, cleaned_output)
+
+        if matches3:
+            segments = []
+            for match in matches3:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 4: Look for format with square brackets [r1,c1]-[r2,c2]
+        pattern4 = r'\[(\d+),(\d+)\][-–—]\[(\d+),(\d+)\]'
+        matches4 = re.findall(pattern4, cleaned_output)
+
+        if matches4:
+            segments = []
+            for match in matches4:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 5: Look for coordinates separated by various delimiters
+        # Pattern like: 1,0 to 0,0 or 1,0 -> 0,0 or 1,0 → 0,0
+        pattern5 = r'(\d+),(\d+)\s*(?:[-–—]|to|->|→|=>)\s*(\d+),(\d+)'
+        matches5 = re.findall(pattern5, cleaned_output)
+
+        if matches5:
+            segments = []
+            for match in matches5:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 6: Look for format with curly braces {r1,c1}-{r2,c2}
+        pattern6 = r'\{(\d+),(\d+)\}[-–—]\{(\d+),(\d+)\}'
+        matches6 = re.findall(pattern6, cleaned_output)
+
+        if matches6:
+            segments = []
+            for match in matches6:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 7: Look for coordinates with different separators (semicolon, pipe, etc.)
+        pattern7 = r'(\d+)[,;|:](\d+)\s*[-–—]\s*(\d+)[,;|:](\d+)'
+        matches7 = re.findall(pattern7, cleaned_output)
+
+        if matches7:
+            segments = []
+            for match in matches7:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 8: Look for line breaks or comma-separated segments
+        separators = ['\n', ';', ',']
+        for sep in separators:
+            parts = cleaned_output.split(sep)
+            segments = []
+            for part in parts:
+                part = part.strip()
+                if not part:
+                    continue
+
+                # Try to extract segment from this part using any of the above patterns
+                for pattern in [pattern1, pattern2, pattern3, pattern4, pattern5, pattern6, pattern7]:
+                    match = re.search(pattern, part)
+                    if match:
+                        try:
+                            r1, c1, r2, c2 = map(int, match.groups())
+                            segments.append(((r1, c1), (r2, c2)))
+                            break
+                        except ValueError:
+                            continue
+
+            if segments:
+                return segments
+
+        # Strategy 9: Look for JSON-like format
+        try:
+            # Try to find JSON-like structures
+            json_pattern = r'\{[^}]*"answer"[^}]*\}'
+            json_matches = re.findall(json_pattern, cleaned_output)
+
+            for json_str in json_matches:
+                try:
+                    data = json.loads(json_str)
+                    if "answer" in data:
+                        answer_data = data["answer"]
+                        if isinstance(answer_data, str):
+                            return self.extract_answer(answer_data)
+                        elif isinstance(answer_data, list):
+                            # Try to parse list as segments
+                            segments = []
+                            for item in answer_data:
+                                if isinstance(item, str):
+                                    sub_segments = self.extract_answer(item)
+                                    if sub_segments:
+                                        segments.extend(sub_segments)
+                            if segments:
+                                return segments
+                except (json.JSONDecodeError, KeyError, TypeError):
+                    continue
+        except Exception:
+            pass
+
+        # Strategy 10: Look for sequences of coordinates that might represent a path
+        # Find all coordinate pairs in the text
+        coord_pattern = r'\(?(\d+)[,;:\s]+(\d+)\)?'
+        coord_matches = re.findall(coord_pattern, cleaned_output)
+
+        if len(coord_matches) >= 2:
+            # Try to form segments by connecting consecutive coordinates
+            segments = []
+            coords = []
+            for match in coord_matches:
+                try:
+                    r, c = map(int, match)
+                    coords.append((r, c))
+                except ValueError:
+                    continue
+
+            # Connect consecutive coordinates
+            for i in range(len(coords) - 1):
+                segments.append((coords[i], coords[i + 1]))
+
+            if segments:
+                return segments
+
+        # Strategy 11: Look for coordinates listed with explicit "from" and "to" keywords
+        pattern11 = r'from\s*\(?(\d+)[,\s]+(\d+)\)?\s*to\s*\(?(\d+)[,\s]+(\d+)\)?'
+        matches11 = re.findall(pattern11, cleaned_output, re.IGNORECASE)
+
+        if matches11:
+            segments = []
+            for match in matches11:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 12: Look for numbered coordinates with patterns like "1. (r1,c1) to (r2,c2)"
+        pattern12 = r'\d+\.\s*\(?(\d+)[,\s]+(\d+)\)?\s*(?:to|->|→)\s*\(?(\d+)[,\s]+(\d+)\)?'
+        matches12 = re.findall(pattern12, cleaned_output)
+
+        if matches12:
+            segments = []
+            for match in matches12:
+                try:
+                    r1, c1, r2, c2 = map(int, match)
+                    segments.append(((r1, c1), (r2, c2)))
+                except ValueError:
+                    continue
+            if segments:
+                return segments
+
+        # Strategy 13: Extract all numbers and try to group them into segments
+        # This is a last resort strategy
+        numbers = re.findall(r'\d+', cleaned_output)
+
+        if len(numbers) >= 4 and len(numbers) % 4 == 0:
+            # Try to group numbers into coordinate pairs
+            segments = []
+            for i in range(0, len(numbers), 4):
+                try:
+                    r1, c1, r2, c2 = map(int, numbers[i:i + 4])
+                    segments.append(((r1, c1), (r2, c2)))
+                except (ValueError, IndexError):
+                    break
+
+            if segments:
+                return segments
+
+        return None
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any = None, initial_state: Any = None) -> bool:
+        """
+        Evaluate if the predicted answer is correct based solely on initial state and game rules
+
+        Args:
+            predicted_answer: The predicted answer (can be string or parsed segments)
+            ground_truth: The ground truth answer (not used for validation, kept for API compatibility)
+            initial_state: The initial state of the puzzle containing grid and circle information
+
+        Returns:
+            True if the answer is correct according to game rules, False otherwise
+        """
+        # Initialize debug information for detailed error tracking
+        debug_info = {"errors": [], "warnings": []}
+
+        try:
+            # Step 1: Extract and validate predicted_answer
+            if predicted_answer is None:
+                return False
+
+            # Extract predicted_answer using extract_answer if it's a string
+            if isinstance(predicted_answer, str):
+                predicted_segments = self.extract_answer(predicted_answer)
+                if predicted_segments is None:
+                    return False
+                if not predicted_segments:
+                    return False
+            elif isinstance(predicted_answer, list):
+                predicted_segments = predicted_answer
+                if not predicted_segments:
+                    return False
+            else:
+                return False
+
+            # Step 2: Validate initial_state
+            if initial_state is None:
+                return False
+
+            # Parse and normalize initial_state
+            try:
+                # Decode JSON string if needed
+                if isinstance(initial_state, str):
+                    state_data = json.loads(initial_state)
+                else:
+                    state_data = initial_state
+
+                # Normalize into expected dict form with a 'grid' key
+                if isinstance(state_data, list):
+                    # Dataset provides a raw 2D grid as JSON array
+                    state_data = {"grid": state_data}
+                elif isinstance(state_data, dict):
+                    # Use as-is; caller may provide {'grid': ...} or {'rows': ..., 'cols': ..., 'circles': ...}
+                    pass
+                else:
+                    return False
+            except json.JSONDecodeError:
+                return False
+
+            # Step 3: Validate segments format
+            if not self._validate_segments_format(predicted_segments, debug_info):
+                return False
+
+            # Step 4: Validate using initial state and game rules
+            validation_result = self._validate_with_initial_state(predicted_segments, state_data, debug_info)
+
+            return validation_result
+
+        except Exception:
+            return False
+
+    def _validate_segments_format(self, segments: List, debug_info: Dict) -> bool:
+        """
+        Validate that segments have the correct format
+
+        Args:
+            segments: List of segments to validate
+            debug_info: Dictionary to store debugging information
+
+        Returns:
+            True if format is valid, False otherwise
+        """
+        if not isinstance(segments, list):
+            debug_info["errors"].append("Segments is not a list")
+            return False
+
+        for i, segment in enumerate(segments):
+            if not isinstance(segment, tuple) or len(segment) != 2:
+                debug_info["errors"].append(f"Segment {i} is not a tuple of length 2: {segment}")
+                return False
+
+            p1, p2 = segment
+
+            if not isinstance(p1, tuple) or len(p1) != 2:
+                debug_info["errors"].append(f"Segment {i} first point is not a tuple of length 2: {p1}")
+                return False
+
+            if not isinstance(p2, tuple) or len(p2) != 2:
+                debug_info["errors"].append(f"Segment {i} second point is not a tuple of length 2: {p2}")
+                return False
+
+            # Validate that coordinates are integers
+            try:
+                r1, c1 = p1
+                r2, c2 = p2
+                int(r1), int(c1), int(r2), int(c2)
+            except (ValueError, TypeError):
+                debug_info["errors"].append(f"Segment {i} contains non-integer coordinates: {segment}")
+                return False
+
+        return True
+
+    def _validate_with_initial_state(self, segments: List[Tuple], state_data: Dict, debug_info: Dict) -> bool:
+        """
+        Validate the solution against the initial state and game rules
+
+        Args:
+            segments: List of line segments forming the solution
+            state_data: Initial state data containing grid and circles information
+            debug_info: Dictionary to store debugging information
+
+        Returns:
+            True if solution is valid according to game rules, False otherwise
+        """
+        # Extract grid information
+        if 'grid' in state_data:
+            grid = state_data['grid']
+            rows = len(grid)
+            cols = len(grid[0]) if grid else 0
+        else:
+            rows = state_data.get('rows', 0)
+            cols = state_data.get('cols', 0)
+
+        if rows == 0 or cols == 0:
+            debug_info["errors"].append(f"Invalid grid dimensions: {rows}x{cols}")
+            return False
+
+        # Extract circles information
+        circles = {}
+        if 'circles' in state_data:
+            circles = state_data['circles']
+        elif 'grid' in state_data:
+            # Parse circles from grid representation
+            circles = self._parse_circles_from_grid(state_data['grid'])
+
+        if not circles:
+            debug_info["errors"].append("No circles found in the puzzle")
+            return False
+
+        # Rule 1: All segments must be within grid boundaries and adjacent
+        if not self._validate_segments_boundaries(segments, rows, cols):
+            debug_info["errors"].append("Segments violate grid boundaries or adjacency rules")
+            return False
+
+        # Rule 2: Must form a single continuous loop
+        if not self._validate_single_continuous_loop(segments):
+            debug_info["errors"].append("Path does not form a single continuous loop")
+            return False
+
+        # Rule 3: Must pass through ALL circles
+        if not self._validate_path_through_circles(segments, circles):
+            debug_info["errors"].append("Path does not pass through all circles")
+            return False
+
+        # Rule 4: Validate circle constraints (white/black circle rules and values)
+        circle_validation_result = self._validate_all_circle_constraints(segments, circles, debug_info)
+        if not circle_validation_result:
+            # Detailed errors are already added to debug_info in _validate_all_circle_constraints
+            return False
+
+        return True
+
+    def _parse_circles_from_grid(self, grid: List[List[str]]) -> Dict:
+        """
+        Parse circles from grid representation
+
+        Args:
+            grid: 2D grid with circle representations
+
+        Returns:
+            Dictionary of circles with position keys and type/value info
+        """
+        circles = {}
+
+        for r in range(len(grid)):
+            for c in range(len(grid[r])):
+                cell = grid[r][c]
+                if cell and cell != '.':
+                    # Parse circle type and value from cell content
+                    if cell.startswith('W'):  # White circle
+                        try:
+                            value = int(cell[1:])
+                            pos_key = f"{r},{c}"
+                            circles[pos_key] = {"type": "white", "value": value}
+                        except ValueError:
+                            continue
+                    elif cell.startswith('B'):  # Black circle
+                        try:
+                            value = int(cell[1:])
+                            pos_key = f"{r},{c}"
+                            circles[pos_key] = {"type": "black", "value": value}
+                        except ValueError:
+                            continue
+
+        return circles
+
+    def _validate_single_continuous_loop(self, segments: List[Tuple]) -> bool:
+        """
+        Validate that the segments form a single continuous loop
+
+        Args:
+            segments: List of line segments
+
+        Returns:
+            True if segments form a valid single continuous loop
+        """
+        if not segments:
+            return False
+
+        # Build adjacency graph
+        graph = {}
+        for p1, p2 in segments:
+            if p1 not in graph:
+                graph[p1] = []
+            if p2 not in graph:
+                graph[p2] = []
+            graph[p1].append(p2)
+            graph[p2].append(p1)
+
+        # Rule: Each point must have exactly 2 connections (loop property)
+        for point, neighbors in graph.items():
+            if len(neighbors) != 2:
+                return False
+
+        # Rule: Must form exactly one connected component
+        if not graph:
+            return False
+
+        visited = set()
+        start_point = next(iter(graph.keys()))
+
+        def dfs(point):
+            visited.add(point)
+            for neighbor in graph[point]:
+                if neighbor not in visited:
+                    dfs(neighbor)
+
+        dfs(start_point)
+
+        # All points should be visited (single connected component)
+        return len(visited) == len(graph)
+
+    def _validate_segments_boundaries(self, segments: List[Tuple], rows: int, cols: int) -> bool:
+        """
+        Validate that all segments are within boundaries and connect adjacent points
+
+        Args:
+            segments: List of line segments
+            rows: Number of rows in grid (grid cells)
+            cols: Number of columns in grid (grid cells)
+
+        Returns:
+            True if all segments are valid
+        """
+        for segment in segments:
+            p1, p2 = segment
+            r1, c1 = p1
+            r2, c2 = p2
+
+            # Check boundaries - for an NxM grid of cells, we have (N+1)x(M+1) grid points
+            # So coordinates should be from 0 to N and 0 to M (inclusive)
+            if not (0 <= r1 <= rows and 0 <= c1 <= cols
+                    and 0 <= r2 <= rows and 0 <= c2 <= cols):
+                return False
+
+            # Check adjacency (Manhattan distance = 1, no diagonal connections)
+            if abs(r1 - r2) + abs(c1 - c2) != 1:
+                return False
+
+        return True
+
+    def _validate_path_through_circles(self, segments: List[Tuple], circles: Dict) -> bool:
+        """
+        Validate that the path passes through all circles
+
+        Args:
+            segments: List of line segments
+            circles: Dictionary of circles
+
+        Returns:
+            True if path passes through all circles
+        """
+        # Build set of all points on the path
+        path_points = set()
+        for p1, p2 in segments:
+            path_points.add(p1)
+            path_points.add(p2)
+
+        # Convert circle positions to tuples
+        circle_positions = set()
+        for pos_str in circles.keys():
+            r, c = map(int, pos_str.split(','))
+            circle_positions.add((r, c))
+
+        # Rule: Path must pass through ALL circles
+        return circle_positions.issubset(path_points)
+
+    def _validate_all_circle_constraints(self, segments: List[Tuple], circles: Dict, debug_info: Dict) -> bool:
+        """
+        Validate all circle constraints (white/black rules and values)
+
+        Args:
+            segments: List of line segments
+            circles: Dictionary of circles
+            debug_info: Dictionary to store debugging information
+
+        Returns:
+            True if all circle constraints are satisfied
+        """
+        # Build adjacency graph
+        graph = {}
+        for p1, p2 in segments:
+            if p1 not in graph:
+                graph[p1] = []
+            if p2 not in graph:
+                graph[p2] = []
+            graph[p1].append(p2)
+            graph[p2].append(p1)
+
+        # Check each circle constraint
+        for pos_str, circle_info in circles.items():
+            # Convert position string to tuple
+            r, c = map(int, pos_str.split(','))
+            point = (r, c)
+
+            # Circle must be on the path
+            if point not in graph:
+                debug_info["errors"].append(f"Circle at {point} is not on the path")
+                return False
+
+            # Circle must have exactly 2 connections
+            if len(graph[point]) != 2:
+                debug_info["errors"].append(f"Circle at {point} does not have exactly 2 connections")
+                return False
+
+            circle_type = circle_info["type"]
+            expected_value = circle_info["value"]
+
+            neighbors = graph[point]
+
+            # Validate white circle constraint (straight line)
+            if circle_type == "white":
+                if not self._validate_white_circle_constraint(point, neighbors):
+                    debug_info["errors"].append(f"White circle at {point} is not on a straight line")
+                    return False
+
+            # Validate black circle constraint (turn)
+            elif circle_type == "black":
+                if not self._validate_black_circle_constraint(point, neighbors):
+                    debug_info["errors"].append(f"Black circle at {point} is not at a turning point")
+                    return False
+
+            # Validate circle value (sum of line lengths)
+            actual_value = self._calculate_circle_value(point, neighbors, graph)
+            if actual_value != expected_value:
+                debug_info["errors"].append(
+                    f"Circle at {point} has value {expected_value} but actual length is {actual_value}")
+                return False
+
+        return True
+
+    def _validate_white_circle_constraint(self, point: Tuple[int, int], neighbors: List[Tuple[int, int]]) -> bool:
+        """
+        Validate that a white circle is on a straight line
+
+        Args:
+            point: Circle position
+            neighbors: Two neighboring points
+
+        Returns:
+            True if the circle is on a straight line
+        """
+        if len(neighbors) != 2:
+            return False
+
+        n1, n2 = neighbors
+        r, c = point
+
+        # Calculate directions from point to neighbors
+        dir1 = (n1[0] - r, n1[1] - c)
+        dir2 = (n2[0] - r, n2[1] - c)
+
+        # For straight line, directions should be opposite
+        return (dir1[0] + dir2[0] == 0) and (dir1[1] + dir2[1] == 0)
+
+    def _validate_black_circle_constraint(self, point: Tuple[int, int], neighbors: List[Tuple[int, int]]) -> bool:
+        """
+        Validate that a black circle is at a turning point
+
+        Args:
+            point: Circle position
+            neighbors: Two neighboring points
+
+        Returns:
+            True if the circle is at a turning point
+        """
+        if len(neighbors) != 2:
+            return False
+
+        n1, n2 = neighbors
+        r, c = point
+
+        # Calculate directions from point to neighbors
+        dir1 = (n1[0] - r, n1[1] - c)
+        dir2 = (n2[0] - r, n2[1] - c)
+
+        # For turning point, directions should NOT be opposite
+        return not ((dir1[0] + dir2[0] == 0) and (dir1[1] + dir2[1] == 0))
+
+    def _calculate_circle_value(self, circle_point: Tuple[int, int],
+                                neighbors: List[Tuple[int, int]],
+                                graph: Dict) -> int:
+        """
+        Calculate the actual value for a circle (sum of straight line lengths in both directions)
+
+        Args:
+            circle_point: Circle position
+            neighbors: Two neighboring points
+            graph: Adjacency graph of the entire path
+
+        Returns:
+            Sum of the two straight line segment lengths from the circle
+        """
+        if len(neighbors) != 2:
+            return 0
+
+        total_length = 0
+
+        # Calculate length in both directions from the circle
+        for neighbor in neighbors:
+            length = self._get_straight_line_length_from_circle(circle_point, neighbor, graph)
+            total_length += length
+
+        return total_length
+
+    def _get_straight_line_length_from_circle(
+            self, circle_point: Tuple[int, int],
+            next_point: Tuple[int, int],
+            graph: Dict) -> int:
+        """
+        Get the length of a straight line segment starting from circle going towards next_point
+
+        Args:
+            circle_point: Starting circle point
+            next_point: Next point in the direction
+            graph: Adjacency graph
+
+        Returns:
+            Length of the straight line segment from the circle
+        """
+        # Direction vector from circle to next
+        direction = (next_point[0] - circle_point[0], next_point[1] - circle_point[1])
+
+        current = next_point
+        length = 1  # Count the first segment from circle to next_point
+        visited = {circle_point}  # Avoid going back to the circle
+
+        while True:
+            visited.add(current)
+
+            # Find the next point that continues in the same direction
+            next_in_direction = None
+
+            for neighbor in graph[current]:
+                if neighbor not in visited:
+                    # Check if this neighbor is in the same direction
+                    current_to_neighbor = (neighbor[0] - current[0], neighbor[1] - current[1])
+                    if current_to_neighbor == direction:
+                        next_in_direction = neighbor
+                        break
+
+            if next_in_direction is None:
+                # No more points in this direction, stop
+                break
+
+            # Continue in the same direction
+            current = next_in_direction
+            length += 1
+
+        return length
+
+    # Keep existing methods for backward compatibility
+    def validate_loop(self, segments: List[Tuple], rows: int = 0, cols: int = 0) -> bool:
+        """Backward compatibility wrapper"""
+        return self._validate_single_continuous_loop(segments)
+
+    def validate_circle_constraints(self, segments: List[Tuple], circles: Dict, rows: int, cols: int) -> bool:
+        """Backward compatibility wrapper"""
+        debug_info = {"errors": []}
+        return self._validate_all_circle_constraints(segments, circles, debug_info)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/skyscrapers_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/skyscrapers_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..fabd7d8566ea6691f8b035da1f3442ec56f10b9d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/skyscrapers_evaluator.py
@@ -0,0 +1,230 @@
+import json
+import re
+from typing import Any, Dict, List, Union
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class SkyscrapersEvaluator(BaseEvaluator):
+    """
+    评估摩天楼谜题解答的评估器
+    摩天楼(Skyscrapers)是一种逻辑谜题，规则为：
+    1. 每行每列必须包含1到n的每个数字恰好一次
+    2. 四周的数字表示从该方向看过去可以看到的摩天楼数量
+       (较高的摩天楼会挡住后面较矮的摩天楼)
+    """
+
+    def extract_answer(self, model_output: str) -> List[List[int]]:
+        """
+        从模型输出中提取摩天楼谜题的解答矩阵
+
+        Args:
+            model_output: 模型生成的字符串输出
+
+        Returns:
+            提取的二维列表，表示摩天楼谜题的解答
+        """
+        # 使用正则表达式寻找类似 [[3, 1, 2], [2, 3, 1], [1, 2, 3]] 的模式
+        pattern = r'\[\s*\[(?:\s*\d+\s*,\s*)*\s*\d+\s*\](?:\s*,\s*\[\s*(?:\d+\s*,\s*)*\d+\s*\])*\s*\]'
+        matches = re.findall(pattern, model_output)
+
+        if not matches:
+            # 如果没有找到符合格式的答案，返回空列表
+            return []
+
+        try:
+            # 尝试解析找到的第一个匹配项
+            answer_matrix = json.loads(matches[0])
+            # 确保它是一个二维列表，且每个元素都是整数
+            if (
+                isinstance(answer_matrix, list)
+                and all(
+                    isinstance(row, list)
+                    and all(isinstance(item, int) for item in row)
+                    for row in answer_matrix
+                )
+            ):
+                return answer_matrix
+            return []
+        except json.JSONDecodeError:
+            # 如果JSON解析失败，返回空列表
+            return []
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """
+        准备用于解决摩天楼谜题的提示词
+
+        Args:
+            question: 问题描述
+            params: 包含谜题信息的参数，如尺寸和四个方向的约束
+
+        Returns:
+            格式化的提示词
+        """
+        n = params.get("n", 4)
+        top = params.get("top", [])
+        bottom = params.get("bottom", [])
+        left = params.get("left", [])
+        right = params.get("right", [])
+
+        prompt = f"""请解决以下摩天楼(Skyscrapers)谜题:
+
+    这是一个 {n}x{n} 的网格，需要放置高度从1到{n}的摩天楼。
+
+    规则:
+    1. 每行和每列必须包含从1到{n}的每个数字恰好一次
+    2. 网格四周的数字表示从该方向看过去能看到的摩天楼数量
+    3. 较高的摩天楼会挡住后面较矮的摩天楼
+
+    四周的约束条件:
+    - 上方(从上往下看): {top}
+    - 下方(从下往上看): {bottom}
+    - 左侧(从左往右看): {left}
+    - 右侧(从右往左看): {right}
+
+    例如，如果一行从左边看的约束是2，且该行的摩天楼高度依次为3,1,4,2，那么只能看到高度为3和4的两栋摩天楼(3挡住了后面的1，4挡住了后面的2)。
+
+    请给出摩天楼的排列方案，格式为二维数组，每个数字表示对应位置摩天楼的高度。
+    例如: [[3, 1, 2], [2, 3, 1], [1, 2, 3]]
+
+    解题过程:
+    1. 分析四个方向的约束
+    2. 确定每个位置可能的摩天楼高度
+    3. 使用逻辑推理填写整个网格
+    4. 验证所有约束是否满足
+    5. 提供最终的排列方案
+    """
+
+        return prompt
+
+    def evaluate(self, output: Union[str, List[List[int]]], ground_truth: List[List[int]],
+                 params: Dict[str, Any]) -> bool:
+        """
+        评估模型的解答是否正确，基于摩天楼谜题规则验证
+
+        Args:
+            output: 模型的原始输出字符串或已提取的答案
+            ground_truth: 正确的解答（仅用于参考，不直接比对）
+            params: 包含谜题信息的参数，如尺寸和四个方向的约束
+
+        Returns:
+            解答是否正确的布尔值
+        """
+        # 检查output类型，如果是字符串，则提取答案；如果已经是列表，则直接使用
+        if isinstance(output, str):
+            predicted_answer = self.extract_answer(output)
+        else:
+            predicted_answer = output
+
+        # 如果无法提取有效答案，直接返回False
+        if not predicted_answer:
+            return False
+
+        # 获取谜题参数
+        n = params.get("n", 4)
+        top = params.get("top", [])
+        bottom = params.get("bottom", [])
+        left = params.get("left", [])
+        right = params.get("right", [])
+
+        # 检查答案维度是否正确
+        if len(predicted_answer) != n:
+            return False
+        if any(len(row) != n for row in predicted_answer):
+            return False
+
+        # 1. 验证每行每列包含1到n的每个数字恰好一次
+        for row in predicted_answer:
+            if set(row) != set(range(1, n + 1)):
+                return False
+
+        for col_idx in range(n):
+            column = [predicted_answer[row_idx][col_idx] for row_idx in range(n)]
+            if set(column) != set(range(1, n + 1)):
+                return False
+
+        # 2. 验证四个方向的可见性约束
+
+        # 验证上方约束
+        for col_idx, constraint in enumerate(top):
+            if constraint > 0:  # 0表示没有约束
+                visible_count = self._count_visible_from_top(predicted_answer, col_idx)
+                if visible_count != constraint:
+                    return False
+
+        # 验证下方约束
+        for col_idx, constraint in enumerate(bottom):
+            if constraint > 0:
+                visible_count = self._count_visible_from_bottom(predicted_answer, col_idx)
+                if visible_count != constraint:
+                    return False
+
+        # 验证左侧约束
+        for row_idx, constraint in enumerate(left):
+            if constraint > 0:
+                visible_count = self._count_visible_from_left(predicted_answer, row_idx)
+                if visible_count != constraint:
+                    return False
+
+        # 验证右侧约束
+        for row_idx, constraint in enumerate(right):
+            if constraint > 0:
+                visible_count = self._count_visible_from_right(predicted_answer, row_idx)
+                if visible_count != constraint:
+                    return False
+
+        # 所有验证通过
+        return True
+
+    def _count_visible_from_top(self, grid: List[List[int]], col_idx: int) -> int:
+        """计算从上方看某一列可见的摩天楼数量"""
+        visible_count = 0
+        max_height = 0
+        for row_idx in range(len(grid)):
+            current_height = grid[row_idx][col_idx]
+            if current_height > max_height:
+                visible_count += 1
+                max_height = current_height
+        return visible_count
+
+    def _count_visible_from_bottom(self, grid: List[List[int]], col_idx: int) -> int:
+        """计算从下方看某一列可见的摩天楼数量"""
+        visible_count = 0
+        max_height = 0
+        for row_idx in range(len(grid) - 1, -1, -1):
+            current_height = grid[row_idx][col_idx]
+            if current_height > max_height:
+                visible_count += 1
+                max_height = current_height
+        return visible_count
+
+    def _count_visible_from_left(self, grid: List[List[int]], row_idx: int) -> int:
+        """计算从左侧看某一行可见的摩天楼数量"""
+        visible_count = 0
+        max_height = 0
+        for col_idx in range(len(grid[row_idx])):
+            current_height = grid[row_idx][col_idx]
+            if current_height > max_height:
+                visible_count += 1
+                max_height = current_height
+        return visible_count
+
+    def _count_visible_from_right(self, grid: List[List[int]], row_idx: int) -> int:
+        """计算从右侧看某一行可见的摩天楼数量"""
+        visible_count = 0
+        max_height = 0
+        for col_idx in range(len(grid[row_idx]) - 1, -1, -1):
+            current_height = grid[row_idx][col_idx]
+            if current_height > max_height:
+                visible_count += 1
+                max_height = current_height
+        return visible_count
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/slidingpuzzle_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/slidingpuzzle_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..be056830e724f86c611fdb102e29d9f7f4d30013
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/slidingpuzzle_eval.py
@@ -0,0 +1,42 @@
+import re
+from typing import Any, Dict, List
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self, predicted_answer: Any, ground_truth: Any,
+        initial_state: str, params: Dict[str, Any] = None
+    ) -> bool:
+        raise NotImplementedError
+
+
+class SlidingPuzzleEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        from utils.constants import PROMPT_15PUZZLE
+
+        if isinstance(question, list):
+            question_str = str(question)
+        else:
+            question_str = question
+
+        return PROMPT_15PUZZLE.format(question_str)
+
+    def extract_answer(self, model_output: str) -> List[int]:
+        answer = model_output.strip()
+
+        try:
+            numbers = re.findall(r'\d+', answer)
+            moves = [int(num) for num in numbers if num.strip()]
+            return moves
+        except Exception:
+            return []
+
+    def evaluate(self, predicted_answer: List[int], ground_truth: Any, initial_state: Any) -> bool:
+        from vlmeval.dataset.utils.mmhelix.utils.validation import puzzle_15_check
+        return puzzle_15_check(initial_state, predicted_answer)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/snake_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/snake_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..e9bee125038e81c40d44ccc1de107669ff93b07a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/snake_eval.py
@@ -0,0 +1,58 @@
+import re
+from typing import Any, Dict, List
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        # Extract content within <answer></answer> tags
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+        if match:
+            return match.group(1).strip()
+        return model_output.strip()  # Fallback to full output if no tags found
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class SnakeEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        from utils.constants import PROMPT_SNAKE
+        return PROMPT_SNAKE.format(question)
+
+    def extract_answer(self, model_output: str) -> List[tuple]:
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+        if match:
+            content = match.group(1).strip()
+        else:
+            content = model_output.strip()  # Fallback to full output if no tags found
+
+        # Extract all coordinates in the format (x,y)
+        coords_pattern = r'\((\d+)\s*,\s*(\d+)\)'
+        matches = re.findall(coords_pattern, content)
+
+        # Convert to list of tuples with integers
+        coords = [(int(x), int(y)) for x, y in matches]
+        return coords
+
+    def evaluate(self, predicted_answer: str, ground_truth: str, initial_state: Any) -> bool:
+        # Pattern to match coordinates like (0,8)
+        coords_pattern = r'\((\d+)\s*,\s*(\d+)\)'
+
+        # Extract all coordinates from ground truth
+        gt_matches = re.findall(coords_pattern, ground_truth)
+        gt_coords = {(int(x), int(y)) for x, y in gt_matches}
+
+        # Extract all coordinates from predicted answer
+        pred_matches = re.findall(coords_pattern, predicted_answer)
+        pred_coords = {(int(x), int(y)) for x, y in pred_matches}
+
+        # Check if both sets contain the same coordinates
+        if pred_coords == gt_coords:
+            return True
+
+        return False
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/sokoban_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/sokoban_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..6478f2a0f794ed3fac7acc978ab709027b767a9d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/sokoban_eval.py
@@ -0,0 +1,230 @@
+import re
+from typing import Any, Dict, List
+
+import numpy as np
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self, predicted_answer: Any, ground_truth: Any,
+        initial_state: str, params: Dict[str, Any] = None
+    ) -> bool:
+        raise NotImplementedError
+
+
+class SokobanEvaluator(BaseEvaluator):
+    # Define Sokoban elements for the simulator
+    WALL = 1
+    PLAYER = 2
+    BOX = 3
+    GOAL = 4
+    BOX_ON_GOAL = 5
+    PLAYER_ON_GOAL = 6
+    FLOOR = 0
+
+    # Directions mapping
+    DIRECTIONS = {
+        'up': (-1, 0),
+        'down': (1, 0),
+        'left': (0, -1),
+        'right': (0, 1)
+    }
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        from utils.constants import PROMPT_SOKOBAN
+        return PROMPT_SOKOBAN.format(question)
+
+    def extract_answer(self, model_output: str) -> List[str]:
+        """
+        Extract movement directions (up, down, left, right) from the model's output.
+        Uses regex to extract valid directions regardless of formatting.
+        """
+        # Extract content within <answer></answer> tags if present
+        answer_pattern = r'<answer>(.*?)</answer>'
+        match = re.search(answer_pattern, model_output, re.DOTALL)
+        if match:
+            content = match.group(1).strip().lower()
+        else:
+            content = model_output.strip().lower()
+
+        # Define regex patterns for each direction
+        # This will match directions even if surrounded by punctuation or other characters
+        up_pattern = r'\b(?:up|u)\b'
+        down_pattern = r'\b(?:down|d)\b'
+        left_pattern = r'\b(?:left|l)\b'
+        right_pattern = r'\b(?:right|r)\b'
+
+        # Now extract directions in the order they appear in the text
+        moves = []
+        for word in re.findall(r'\b(?:up|u|down|d|left|l|right|r)\b', content):
+            if re.match(up_pattern, word):
+                moves.append('up')
+            elif re.match(down_pattern, word):
+                moves.append('down')
+            elif re.match(left_pattern, word):
+                moves.append('left')
+            elif re.match(right_pattern, word):
+                moves.append('right')
+
+        return moves
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state) -> bool:
+        """
+        Robustly evaluates if the predicted answer solves the Sokoban puzzle.
+        Can handle various formats of input for the predicted answer.
+        """
+        # If the predicted answer is a string, extract directions from it
+        if isinstance(predicted_answer, str):
+            predicted_moves = self.extract_answer(predicted_answer)
+        # If it's already a list of directions, use it directly
+        elif isinstance(predicted_answer, list):
+            predicted_moves = predicted_answer
+        else:
+            return False
+
+        # Parse the initial state into a grid
+        if not initial_state or not isinstance(initial_state, str):
+            return False
+
+        lines = initial_state.strip().split('\n')
+        height = len(lines)
+        width = max(len(line) for line in lines)
+
+        grid = np.zeros((height, width), dtype=int)
+
+        for y, line in enumerate(lines):
+            for x, char in enumerate(line):
+                if char == '#':  # Wall
+                    grid[y, x] = self.WALL
+                elif char == '@':  # Player
+                    grid[y, x] = self.PLAYER
+                elif char == '$':  # Box
+                    grid[y, x] = self.BOX
+                elif char == '.':  # Target
+                    grid[y, x] = self.GOAL
+                elif char == '*':  # Box on target
+                    grid[y, x] = self.BOX_ON_GOAL
+                elif char == '+':  # Player on target
+                    grid[y, x] = self.PLAYER_ON_GOAL
+
+        # Create and use Sokoban simulator
+        puzzle = self.SokobanSimulator(grid)
+
+        # Apply each predicted move
+        for move in predicted_moves:
+            if not puzzle.move(move):
+                return False  # Invalid move
+
+        # Check if the puzzle is solved
+        is_solved = puzzle.is_solved()
+
+        # Debug: Check the ground truth solution if available
+        if ground_truth and isinstance(ground_truth, str):
+            # Get the list of moves from the ground truth solution
+            gt_moves = self.extract_answer(ground_truth)
+
+            # Create a new simulator to test the ground truth solution
+            gt_puzzle = self.SokobanSimulator(grid.copy())
+
+            # Apply each move from the ground truth solution
+            for move in gt_moves:
+                if not gt_puzzle.move(move):
+                    break
+
+        return is_solved
+
+    # Define a Sokoban simulator class within the evaluator
+    class SokobanSimulator:
+        def __init__(self, grid):
+            self.grid = grid.copy()
+            self.height, self.width = grid.shape
+            self.player_pos = None
+            self.boxes = set()
+            self.goals = set()
+
+            # Define element constants from parent class
+            self.WALL = 1
+            self.PLAYER = 2
+            self.BOX = 3
+            self.GOAL = 4
+            self.BOX_ON_GOAL = 5
+            self.PLAYER_ON_GOAL = 6
+            self.FLOOR = 0
+
+            # Directions mapping
+            self.DIRECTIONS = {
+                'up': (-1, 0),
+                'down': (1, 0),
+                'left': (0, -1),
+                'right': (0, 1)
+            }
+
+            # Find player, boxes, and goals
+            for r in range(self.height):
+                for c in range(self.width):
+                    cell = self.grid[r, c]
+                    if cell == self.PLAYER:
+                        self.player_pos = (r, c)
+                    elif cell == self.PLAYER_ON_GOAL:
+                        self.player_pos = (r, c)
+                        self.goals.add((r, c))
+                    elif cell == self.BOX:
+                        self.boxes.add((r, c))
+                    elif cell == self.BOX_ON_GOAL:
+                        self.boxes.add((r, c))
+                        self.goals.add((r, c))
+                    elif cell == self.GOAL:
+                        self.goals.add((r, c))
+
+        def move(self, direction):
+            """Move the player in the given direction if possible."""
+            if direction.lower() not in self.DIRECTIONS:
+                return False
+
+            dr, dc = self.DIRECTIONS[direction.lower()]
+            r, c = self.player_pos
+            new_r, new_c = r + dr, c + dc
+
+            # Check bounds
+            if not (0 <= new_r < self.height and 0 <= new_c < self.width):
+                return False
+
+            # Check if moving into a wall
+            if self.grid[new_r, new_c] == self.WALL:
+                return False
+
+            # Check if moving into a box
+            if (new_r, new_c) in self.boxes:
+                # Calculate position behind the box
+                box_r, box_c = new_r + dr, new_c + dc
+
+                # Check bounds and if box can be pushed
+                if not (0 <= box_r < self.height and 0 <= box_c < self.width):
+                    return False
+
+                # Check if box destination is valid (not a wall or another box)
+                if self.grid[box_r, box_c] == self.WALL or (box_r, box_c) in self.boxes:
+                    return False
+
+                # Move the box
+                self.boxes.remove((new_r, new_c))
+                self.boxes.add((box_r, box_c))
+
+            # Move the player
+            self.player_pos = (new_r, new_c)
+            return True
+
+        def is_solved(self):
+            """检查所有箱子是否都在目标点上"""
+            # 检查所有箱子是否都在目标点上，以及所有目标点是否都有箱子
+            boxes_on_goals = all(box in self.goals for box in self.boxes)
+            goals_with_boxes = all(goal in self.boxes for goal in self.goals)
+            solved = boxes_on_goals and goals_with_boxes
+
+            return solved
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/sudoku_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/sudoku_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..e065540e8186dd2c2b87592ff7fb39e0f3111982
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/sudoku_evaluator.py
@@ -0,0 +1,111 @@
+import ast
+from typing import Any, Dict, List, Optional, Union
+
+
+def _parse_grid_like(obj: Union[str, List[List[int]]]) -> Optional[List[List[int]]]:
+    if isinstance(obj, list):
+        try:
+            return [[int(v) for v in row] for row in obj]
+        except Exception:
+            return None
+    if isinstance(obj, str):
+        s = obj.strip()
+        # Try list literal first
+        try:
+            parsed = ast.literal_eval(s)
+            if isinstance(parsed, list):
+                return _parse_grid_like(parsed)
+        except Exception:
+            pass
+        # Then try whitespace grid with '.' as blanks
+        tokens = []
+        for line in s.splitlines():
+            for tok in line.strip().split():
+                if tok == '.':
+                    tokens.append(0)
+                else:
+                    try:
+                        tokens.append(int(tok))
+                    except Exception:
+                        pass
+        if len(tokens) == 81:
+            grid = [tokens[i * 9:(i + 1) * 9] for i in range(9)]
+            return grid
+    return None
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self, predicted_answer: Any, ground_truth: Any,
+        initial_state: str, params: Dict[str, Any] = None
+    ) -> bool:
+        raise NotImplementedError
+
+
+class SudokuEvaluator(BaseEvaluator):
+    """Evaluator for classic 9x9 Sudoku using initial givens and Sudoku rules.
+    - Parses model output into a 9x9 integer grid.
+    - Validates that all givens from initial_state are preserved.
+    - Checks rows, columns, and 3x3 blocks contain digits 1..9 exactly once.
+    - Ignores ground_truth; correctness is rule-based.
+    """
+
+    def prepare_prompt(self, question: str) -> str:
+        return str(question)
+
+    def extract_answer(self, model_output: str) -> Optional[List[List[int]]]:
+        return _parse_grid_like(model_output)
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        pred_grid = predicted_answer if isinstance(predicted_answer, list) else _parse_grid_like(predicted_answer)
+        init_grid = initial_state if isinstance(initial_state, list) else _parse_grid_like(initial_state)
+        if pred_grid is None or init_grid is None:
+            return False
+        # shape check
+        if len(pred_grid) != 9 or any(len(r) != 9 for r in pred_grid):
+            return False
+        if len(init_grid) != 9 or any(len(r) != 9 for r in init_grid):
+            return False
+
+        # all entries in prediction must be 1..9
+        for i in range(9):
+            for j in range(9):
+                v = pred_grid[i][j]
+                try:
+                    iv = int(v)
+                except Exception:
+                    return False
+                if iv < 1 or iv > 9:
+                    return False
+
+        # givens preserved
+        for i in range(9):
+            for j in range(9):
+                g = int(init_grid[i][j])
+                if g != 0 and int(pred_grid[i][j]) != g:
+                    return False
+
+        # rows and cols unique 1..9
+        full_set = set(range(1, 10))
+        for i in range(9):
+            if set(int(x) for x in pred_grid[i]) != full_set:
+                return False
+        for j in range(9):
+            col = [int(pred_grid[i][j]) for i in range(9)]
+            if set(col) != full_set:
+                return False
+
+        # 3x3 blocks
+        for br in range(0, 9, 3):
+            for bc in range(0, 9, 3):
+                block = [int(pred_grid[r][c]) for r in range(br, br + 3) for c in range(bc, bc + 3)]
+                if set(block) != full_set:
+                    return False
+
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/tapa_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/tapa_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..ca087b8437ae1fbe2434c3adf28e10f99249f4ad
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/tapa_eval.py
@@ -0,0 +1,525 @@
+import re
+from collections import deque
+from typing import Any, Dict, List, Tuple
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class TapaEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        initial_state = params.get('initial_state', '')
+        size_info = params.get('size', '')
+
+        prompt = f"""Please solve this Tapa puzzle following these rules:
+
+1. All black cells must form a single connected group
+2. No 2x2 block of black cells is allowed
+3. Numbers in clue cells indicate lengths of connected black cell groups in the 8 surrounding cells
+4. Provide your answer as coordinates of black cells in the format (row,column), separated by commas
+5. Use (0,0) as the top-left corner
+
+{question}
+
+Initial state with clues:
+{initial_state}
+
+{f'Grid size: {size_info}' if size_info else ''}
+
+Please provide your answer as coordinates of black cells in the format (row,column), separated by commas.
+For example: (0,1), (1,2), (2,0), (2,1)"""
+
+        return prompt
+
+    def extract_answer(self, model_output: str) -> Any:
+        if not model_output:
+            return None
+
+        text = model_output.strip()
+        text = re.sub(r'```[a-z]*\n?', '', text)  # 移除markdown代码块标记
+        text = re.sub(r'```', '', text)  # 移除结尾的```
+
+        coordinates = self._extract_coordinates(text)
+        if coordinates is not None:
+            return {'type': 'coordinates', 'data': coordinates}
+
+        grid = self._extract_grid(text)
+        if grid is not None:
+            return {'type': 'grid', 'data': grid}
+
+        return None
+
+    def _extract_coordinates(self, text: str) -> List[Tuple[int, int]]:
+        coordinates = []
+        coordinate_pattern = r'\(\s*(\d+)\s*,\s*(\d+)\s*\)'
+        matches = re.findall(coordinate_pattern, text)
+
+        for match in matches:
+            try:
+                x, y = int(match[0]), int(match[1])
+                if 0 <= x <= 99 and 0 <= y <= 99:
+                    coordinates.append((x, y))
+            except (ValueError, IndexError):
+                continue
+
+        if coordinates:
+            return coordinates
+
+        alternative_patterns = [
+            r'\[\s*(\d+)\s*,\s*(\d+)\s*\]',
+            r'\{\s*(\d+)\s*,\s*(\d+)\s*\}',
+            r'(?:^|[^\d])\s*(\d+)\s*,\s*(\d+)\s*(?=[^\d]|$)',
+        ]
+
+        for pattern in alternative_patterns:
+            matches = re.findall(pattern, text)
+            if matches:
+                for match in matches:
+                    try:
+                        x, y = int(match[0]), int(match[1])
+                        if 0 <= x <= 99 and 0 <= y <= 99:
+                            coordinates.append((x, y))
+                    except (ValueError, IndexError):
+                        continue
+
+                if coordinates:
+                    return coordinates
+
+        # 尝试寻找描述性格式
+        descriptive_patterns = [
+            r'(?:row|r)\s*(\d+)\s*(?:col|column|c)\s*(\d+)',
+            r'(?:position|pos)\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)',
+        ]
+
+        for pattern in descriptive_patterns:
+            matches = re.findall(pattern, text, re.IGNORECASE)
+            if matches:
+                for match in matches:
+                    try:
+                        x, y = int(match[0]), int(match[1])
+                        if 0 <= x <= 99 and 0 <= y <= 99:
+                            coordinates.append((x, y))
+                    except (ValueError, IndexError):
+                        continue
+
+                if coordinates:
+                    return coordinates
+
+        return None if not coordinates else coordinates
+
+    def _extract_grid(self, text: str) -> List[List[str]]:
+        """提取网格格式的答案"""
+        lines = text.split('\n')
+        grid_lines = []
+
+        # 各种可能的网格行模式
+        patterns = [
+            # 标准格式：直接的B/W/数字序列
+            r'^[BWbw0-9\s]+$',
+            # 包含逗号分隔的格式
+            r'^[BWbw0-9\s,]+$',
+            # 包含引号的格式
+            r'^[BWbw0-9\s\'"]+$',
+            # 包含下划线或点的格式
+            r'^[BWbw0-9\s._-]+$'
+        ]
+
+        for line in lines:
+            line = line.strip()
+            if not line:
+                continue
+
+            # 检查是否匹配任何网格行模式
+            for pattern in patterns:
+                if re.match(pattern, line):
+                    # 清理行内容：移除空格、逗号、引号等
+                    clean_line = re.sub(r'[,\s\'"_-]', '', line)
+
+                    # 标准化大小写
+                    clean_line = clean_line.upper()
+
+                    # 验证清理后的行只包含有效字符
+                    if re.match(r'^[BW0-9]+$', clean_line) and len(clean_line) > 0:
+                        grid_lines.append(clean_line)
+                        break
+
+        # 如果没有找到网格行，尝试更宽松的匹配
+        if not grid_lines:
+            # 寻找包含B/W字符的行
+            for line in lines:
+                line = line.strip()
+                # 检查行中是否包含足够的B/W字符
+                bw_count = len(re.findall(r'[BWbw]', line))
+                if bw_count >= 3:  # 至少3个B/W字符才考虑
+                    # 提取所有B/W/数字字符
+                    extracted = re.findall(r'[BWbw0-9]', line)
+                    if extracted:
+                        clean_line = ''.join(extracted).upper()
+                        grid_lines.append(clean_line)
+
+        if not grid_lines:
+            return None
+
+        # 转换为二维网格
+        grid = []
+        for line in grid_lines:
+            grid.append(list(line))
+
+        # 验证网格是否为矩形
+        if len(grid) == 0:
+            return None
+
+        expected_width = len(grid[0])
+        for i, row in enumerate(grid):
+            if len(row) != expected_width:
+                # 尝试修复长度不一致的行
+                if len(row) < expected_width:
+                    # 如果行太短，用W补齐
+                    row.extend(['W'] * (expected_width - len(row)))
+                else:
+                    # 如果行太长，截断
+                    grid[i] = row[:expected_width]
+
+        return grid
+
+    def evaluate(self, predicted_answer: str, ground_truth: Any, initial_state: str) -> bool:
+        """评估预测答案是否正确
+
+        Args:
+            predicted_answer: 模型的原始输出字符串
+            ground_truth: 标准答案（本函数中将被忽略，仅根据规则验证）
+            initial_state: 初始状态字符串，包含线索信息
+
+        Returns:
+            bool: 答案是否正确（仅基于游戏规则验证）
+        """
+        try:
+            # 从模型输出中提取答案
+            extracted = self.extract_answer(predicted_answer)
+
+            if extracted is None:
+                return False
+
+            # 解析初始状态获取网格尺寸和线索
+            initial_lines = initial_state.strip().split('\n')
+            if not initial_lines:
+                return False
+
+            rows = len(initial_lines)
+            cols = len(initial_lines[0]) if initial_lines else 0
+
+            if rows == 0 or cols == 0:
+                return False
+
+            # 解析线索
+            clues = self._parse_clues(initial_state)
+
+            # 根据提取的答案类型进行处理
+            if extracted['type'] == 'coordinates':
+                # 坐标格式：将坐标转换为网格
+                black_coordinates = extracted['data']
+                grid = self._coordinates_to_grid(black_coordinates, rows, cols, clues)
+            elif extracted['type'] == 'grid':
+                # 网格格式：直接使用提取的网格
+                grid = extracted['data']
+
+                # 验证网格尺寸
+                if len(grid) != rows or (grid and len(grid[0]) != cols):
+                    return False
+
+                # 检查线索位置是否保持一致
+                if not self._check_clue_positions(grid, initial_state):
+                    return False
+
+                # 创建仅包含B/W的网格用于规则验证
+                grid = self._create_bw_grid(grid, clues)
+            else:
+                return False
+
+            if grid is None:
+                return False
+
+            # 验证所有Tapa规则
+            result = self._verify_tapa_rules(grid, clues)
+            return result
+
+        except Exception:
+            import traceback
+            traceback.print_exc()
+            return False
+
+    def _coordinates_to_grid(self, coordinates: List[Tuple[int, int]], rows: int, cols: int,
+                             clues: Dict[Tuple[int, int], List[int]]) -> List[List[str]]:
+        """将坐标列表转换为网格格式"""
+        # 初始化网格，所有位置为白色
+        grid = [['W' for _ in range(cols)] for _ in range(rows)]
+
+        # 验证坐标是否在有效范围内
+        for r, c in coordinates:
+            if not (0 <= r < rows and 0 <= c < cols):
+                return None
+
+            # 检查坐标是否与线索位置冲突
+            if (r, c) in clues:
+                return None  # 坐标不能在线索位置
+
+            grid[r][c] = 'B'
+
+        return grid
+
+    def _parse_clues(self, initial_state: str) -> Dict[Tuple[int, int], List[int]]:
+        """解析初始状态中的线索"""
+        clues = {}
+        if not initial_state:
+            return clues
+
+        lines = initial_state.strip().split('\n')
+
+        for i, line in enumerate(lines):
+            for j, char in enumerate(line):
+                if char.isdigit():
+                    # 单个数字线索
+                    clues[(i, j)] = [int(char)]
+                elif char in '0123456789':
+                    # 确保是数字字符
+                    clues[(i, j)] = [int(char)]
+
+        return clues
+
+    def _parse_clue_digits(self, digits_str: str) -> List[int]:
+        """解析线索数字字符串 - 简化版本"""
+        if len(digits_str) == 1:
+            return [int(digits_str)]
+
+        # 对于多位数字，每个数字都是单独的线索
+        # 例如 "23" -> [2, 3], "123" -> [1, 2, 3]
+        individual_digits = [int(d) for d in digits_str if d.isdigit()]
+        return individual_digits
+
+    def _check_clue_positions(self, grid: List[List[str]], initial_state: str) -> bool:
+        """检查网格中线索位置是否与原始状态一致"""
+        initial_lines = initial_state.strip().split('\n')
+
+        if len(grid) != len(initial_lines):
+            return False
+
+        for i in range(len(grid)):
+            if len(grid[i]) != len(initial_lines[i]):
+                return False
+
+            for j in range(len(grid[i])):
+                initial_char = initial_lines[i][j]
+                grid_char = grid[i][j]
+
+                if initial_char.isdigit():
+                    # 初始状态是数字，答案中也必须是相同的数字
+                    if grid_char != initial_char:
+                        return False
+                elif initial_char == '.':
+                    # 初始状态是空位，答案中必须是B或W
+                    if grid_char not in ['B', 'W']:
+                        return False
+
+        return True
+
+    def _create_bw_grid(self, grid: List[List[str]], clues: Dict[Tuple[int, int], List[int]]) -> List[List[str]]:
+        """创建仅包含B/W的网格，将线索位置标记为W"""
+        bw_grid = []
+
+        # 获取所有线索位置
+        clue_positions = set()
+        for (row, col), numbers in clues.items():
+            # 对于每个线索，标记其占用的位置
+            clue_positions.add((row, col))
+
+        for i in range(len(grid)):
+            row = []
+            for j in range(len(grid[i])):
+                if (i, j) in clue_positions:
+                    # 线索位置视为白色细胞
+                    row.append('W')
+                else:
+                    row.append(grid[i][j])
+            bw_grid.append(row)
+
+        return bw_grid
+
+    def _verify_tapa_rules(self, grid: List[List[str]], clues: Dict[Tuple[int, int], List[int]]) -> bool:
+        """验证所有Tapa规则"""
+        # 规则1: 检查所有黑色细胞是否形成单一连通组
+        if not self._check_single_connected_group(grid):
+            return False
+
+        # 规则2: 检查是否存在2x2黑色块
+        if self._has_2x2_black_block(grid):
+            return False
+
+        # 规则3: 验证所有线索约束
+        if not self._verify_clues(grid, clues):
+            return False
+
+        # 规则4: 检查白色细胞的连通性（可选，取决于具体的Tapa变体）
+        # 注释掉这个检查，因为在某些Tapa变体中白色细胞不需要连通
+        # if not self._check_white_connectivity(grid):
+        #     return False
+
+        return True
+
+    def _check_single_connected_group(self, grid: List[List[str]]) -> bool:
+        """检查所有黑色细胞是否形成单一连通组"""
+        rows, cols = len(grid), len(grid[0])
+        black_cells = []
+
+        # 找到所有黑色细胞
+        for i in range(rows):
+            for j in range(cols):
+                if grid[i][j] == 'B':
+                    black_cells.append((i, j))
+
+        if not black_cells:
+            return True  # 没有黑色细胞也算有效
+
+        # 从第一个黑色细胞开始BFS
+        start = black_cells[0]
+        visited = set([start])
+        queue = deque([start])
+
+        directions = [(-1, 0), (1, 0), (0, -1), (0, 1)]  # 上下左右
+
+        while queue:
+            x, y = queue.popleft()
+
+            for dx, dy in directions:
+                nx, ny = x + dx, y + dy
+                if (0 <= nx < rows and 0 <= ny < cols
+                        and (nx, ny) not in visited and grid[nx][ny] == 'B'):
+                    visited.add((nx, ny))
+                    queue.append((nx, ny))
+
+        return len(visited) == len(black_cells)
+
+    def _has_2x2_black_block(self, grid: List[List[str]]) -> bool:
+        """检查是否存在2x2黑色块"""
+        rows, cols = len(grid), len(grid[0])
+
+        for i in range(rows - 1):
+            for j in range(cols - 1):
+                if (grid[i][j] == 'B' and grid[i][j + 1] == 'B'
+                        and grid[i + 1][j] == 'B' and grid[i + 1][j + 1] == 'B'):
+                    return True
+
+        return False
+
+    def _check_white_connectivity(self, grid: List[List[str]]) -> bool:
+        """检查白色细胞是否连通"""
+        rows, cols = len(grid), len(grid[0])
+        white_cells = []
+
+        # 找到所有白色细胞
+        for i in range(rows):
+            for j in range(cols):
+                if grid[i][j] == 'W':
+                    white_cells.append((i, j))
+
+        if not white_cells:
+            return True  # 没有白色细胞也算有效
+
+        # 从第一个白色细胞开始BFS
+        start = white_cells[0]
+        visited = set([start])
+        queue = deque([start])
+
+        directions = [(-1, 0), (1, 0), (0, -1), (0, 1)]  # 上下左右
+
+        while queue:
+            x, y = queue.popleft()
+
+            for dx, dy in directions:
+                nx, ny = x + dx, y + dy
+                if (0 <= nx < rows and 0 <= ny < cols
+                        and (nx, ny) not in visited and grid[nx][ny] == 'W'):
+                    visited.add((nx, ny))
+                    queue.append((nx, ny))
+
+        return len(visited) == len(white_cells)
+
+    def _verify_clues(self, grid: List[List[str]], clues: Dict[Tuple[int, int], List[int]]) -> bool:
+        """验证所有线索约束"""
+        for (clue_row, clue_col), expected_groups in clues.items():
+            if not self._verify_single_clue(grid, clue_row, clue_col, expected_groups):
+                return False
+        return True
+
+    def _verify_single_clue(self, grid: List[List[str]], clue_row: int, clue_col: int,
+                            expected_groups: List[int]) -> bool:
+        """验证单个线索约束"""
+        rows, cols = len(grid), len(grid[0])
+
+        # 获取线索位置周围8个邻居
+        neighbors = []
+        for di in [-1, 0, 1]:
+            for dj in [-1, 0, 1]:
+                if di == 0 and dj == 0:  # 跳过线索细胞本身
+                    continue
+                ni, nj = clue_row + di, clue_col + dj
+                if 0 <= ni < rows and 0 <= nj < cols:
+                    neighbors.append((ni, nj))
+
+        # 找到邻居中的黑色细胞
+        black_neighbors = [(i, j) for i, j in neighbors if grid[i][j] == 'B']
+
+        # 如果期望的组大小包含0，则应该没有黑色邻居
+        if 0 in expected_groups:
+            is_valid = len(black_neighbors) == 0
+
+            return is_valid
+
+        # 将黑色邻居分组为连通组
+        groups = self._find_connected_groups_in_neighbors(black_neighbors)
+        group_sizes = sorted([len(group) for group in groups])
+        expected_sizes = sorted(expected_groups)
+
+        is_valid = group_sizes == expected_sizes
+
+        return is_valid
+
+    def _find_connected_groups_in_neighbors(self, black_cells: List[Tuple[int, int]]) -> List[List[Tuple[int, int]]]:
+        """将邻居中的黑色细胞分组为连通组（包括对角连接）"""
+        if not black_cells:
+            return []
+
+        cell_set = set(black_cells)
+        visited = set()
+        groups = []
+
+        for cell in black_cells:
+            if cell not in visited:
+                group = []
+                queue = deque([cell])
+                visited.add(cell)
+
+                while queue:
+                    x, y = queue.popleft()
+                    group.append((x, y))
+
+                    # 检查8个方向的邻居（包括斜对角）
+                    for dx in [-1, 0, 1]:
+                        for dy in [-1, 0, 1]:
+                            if dx == 0 and dy == 0:
+                                continue
+                            nx, ny = x + dx, y + dy
+                            if (nx, ny) in cell_set and (nx, ny) not in visited:
+                                visited.add((nx, ny))
+                                queue.append((nx, ny))
+
+                groups.append(group)
+
+        return groups
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/twentyfourpoints_evaluator.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/twentyfourpoints_evaluator.py
new file mode 100644
index 0000000000000000000000000000000000000000..5061116645eecdb9874a18d82f93251b4660e46d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/twentyfourpoints_evaluator.py
@@ -0,0 +1,243 @@
+import re
+from typing import Any, Dict, List
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, params: Dict[str, Any]) -> bool:
+        raise NotImplementedError
+
+
+class TwentyFourPointsEvaluator(BaseEvaluator):
+    """
+    评估24点游戏解答的评估器
+    验证模型输出的表达式是否：
+    1. 正确使用了所有给定的数字（每个数字恰好使用一次）
+    2. 计算结果是否等于24
+    """
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """准备发送给模型的提示词"""
+        prompt = (
+            "Use these numbers exactly once, and combine them with +, -, ×, ÷, and parentheses to make 24.\n"
+            "Please provide your answer as an expression that includes only numbers, operators, and parentheses.\n"
+            "Example answer format: (9 - 3) × 8 ÷ 2."
+        )
+        return prompt
+
+    def extract_answer(self, model_output: str) -> str:
+        """从模型输出中提取表达式答案，优先提取最终答案"""
+        if isinstance(model_output, dict) and "text" in model_output:
+            model_output = model_output["text"]
+
+        # 先预处理，将可能的LaTeX符号表示修复
+        # 处理 \times 被解释为制表符的情况
+        processed_output = model_output.replace('\times', r'\times')
+        processed_output = processed_output.replace('\\div', r'\div')
+        processed_output = processed_output.replace('\\cdot', r'\cdot')
+
+        # 查找包含LaTeX符号的完整表达式
+        # 匹配包含LaTeX符号的表达式，包括完整的符号和残余符号
+        # 使用更宽松的匹配来获取完整表达式
+        latex_patterns = [
+            # 匹配包含完整\times或残余imes的表达式
+            r'[\(\)\d\s\+\-×÷\*/\\timesa-z]*(?:\\times|imes)[\(\)\d\s\+\-×÷\*/\\timesa-z]*',
+            # 匹配包含完整\div或单独div的表达式
+            r'[\(\)\d\s\+\-×÷\*/\\diva-z]*(?:\\div|div)[\(\)\d\s\+\-×÷\*/\\diva-z]*',
+            # 匹配包含完整\cdot或残余cdot的表达式
+            r'[\(\)\d\s\+\-×÷\*/\\cdota-z]*(?:\\cdot|cdot)[\(\)\d\s\+\-×÷\*/\\cdota-z]*'
+        ]
+
+        for pattern in latex_patterns:
+            matches = re.findall(pattern, model_output)
+            if matches:
+                # 找到最长的匹配项
+                longest_match = max(matches, key=len)
+                # 检查是否包含足够的数字
+                numbers_in_match = re.findall(r'\d+', longest_match)
+                if len(numbers_in_match) >= 3:  # 至少3个数字才可能是完整的24点表达式
+                    return longest_match.strip()
+
+        # 如果上面没有找到，尝试更通用的方法：查找整个输入中的完整表达式
+        # 如果输入本身看起来就是一个表达式，直接使用
+        if (re.search(r'\d+', model_output)
+                and any(keyword in model_output for keyword in [
+                    'imes', 'div', 'cdot', '+', '-', '*', '/', '×', '÷', '\\times', '\\div', '\\cdot'])):
+            return model_output.strip()
+
+        # 定义可接受的字符模式（不包含量词）
+        # 包括：数字、括号、空格、各种运算符（标准符号、Unicode符号、反斜杠、字母）
+        expression_chars = r'[\(\)\d\s\+\-×÷\*/\\a-zA-Z]'
+
+        # 1. 查找显式标记的最终答案
+        final_answer_patterns = [
+            rf'(?:final answer|answer is|the answer is)[^\n]*?[=:]?\s*({expression_chars}+)',
+            rf'(?:so|thus|hence)[^\n]*?[=:]?\s*({expression_chars}+)\s*=\s*24',
+            rf'That\'s it!\s*(?:The)?\s*(?:answer|expression)\s*(?:is)?\s*:?\s*({expression_chars}+)'
+        ]
+
+        for pattern in final_answer_patterns:
+            matches = re.findall(pattern, processed_output, re.IGNORECASE)
+            if matches:
+                return matches[-1].strip()  # 返回最后一个匹配（通常是最终答案）
+
+        # 2. 查找等于24的表达式（优先选择文本最后出现的）
+        expressions_with_24 = re.findall(rf'({expression_chars}+)\s*=\s*24', processed_output)
+        if expressions_with_24:
+            return expressions_with_24[-1].strip()
+
+        # 3. 提取模型提供的最后一个完整表达式
+        # 先按行分割文本
+        lines = processed_output.split('\n')
+        for line in reversed(lines):  # 从后往前检查
+            # 查找包含数字和运算符的表达式（包括LaTeX格式和文字运算符）
+            # 长度至少7个字符的表达式
+            expr_matches = re.findall(rf'({expression_chars}{{7,}})', line)
+            if expr_matches:
+                # 过滤出格式有效的表达式
+                valid_expressions = [expr for expr in expr_matches
+                                     if self._is_valid_expression_format(expr)]
+                if valid_expressions:
+                    return valid_expressions[-1].strip()
+
+        # 4. 如果上述方法都失败，提取整个文本中最可能的表达式
+        all_expressions = re.findall(rf'({expression_chars}{{7,}})', processed_output)
+        valid_expressions = [expr for expr in all_expressions
+                             if self._is_valid_expression_format(expr)]
+
+        if valid_expressions:
+            # 按照启发式规则排序：优先选择包含括号、长度适中的表达式
+            sorted_expressions = sorted(
+                valid_expressions,
+                key=lambda x: (
+                    '(' in x and ')' in x,  # 优先有括号的
+                    len(re.findall(r'\d+', x)),  # 优先包含更多数字的
+                    -abs(len(x) - 15)  # 优先长度接近15个字符的（启发式值）
+                ),
+                reverse=True
+            )
+            return sorted_expressions[0].strip()
+
+        # 5. 最后手段：如果输入本身就是一个表达式，直接返回
+        if self._is_valid_expression_format(processed_output):
+            return processed_output.strip()
+
+        # 6. 返回最后一行非空文本
+        for line in reversed(lines):
+            if line.strip():
+                return line.strip()
+
+        return processed_output.strip()
+
+    def _is_valid_expression_format(self, expr: str) -> bool:
+        """检查表达式格式是否有效（包含数字和运算符）"""
+        # 确保表达式包含至少一个数字和一个运算符
+        has_number = bool(re.search(r'\d', expr))
+
+        # 检查是否包含运算符（包括LaTeX符号和文字运算符）
+        operator_patterns = [
+            r'[\+\-\×\÷\*/]',  # 标准符号
+            r'\\times|\\div|\\cdot',  # 完整的LaTeX符号
+            r'\bimes\b|\bdiv\b|\bcdot\b',  # LaTeX符号的残余部分
+            r'\bmul\b|\btimes\b|\bplus\b|\bminus\b'  # 文字运算符
+        ]
+        has_operator = any(re.search(pattern, expr, re.IGNORECASE) for pattern in operator_patterns)
+
+        # 还要检查括号是否匹配
+        open_brackets = expr.count('(')
+        close_brackets = expr.count(')')
+        brackets_match = open_brackets == close_brackets
+
+        return has_number and has_operator and brackets_match
+
+    def evaluate(self, output: str, ground_truth: Dict[str, Any], params: Dict[str, Any]) -> bool:
+        """
+        评估预测的答案是否正确
+
+        参数:
+            predicted_answer: 模型预测的表达式
+            ground_truth: 包含正确答案和输入数字的字典
+            params: 其他参数
+
+        返回:
+            是否正确（布尔值）
+        """
+        predicted_answer = self.extract_answer(output)
+        if not predicted_answer:
+            return False
+
+        # 清理和标准化表达式
+        expression = self._normalize_expression(predicted_answer)
+
+        # 获取输入数字（从params的initial_state中获取）
+        input_numbers = params.get("numbers", [])
+        if not input_numbers and ground_truth:
+            input_numbers = ground_truth.get("initial_state", {}).get("numbers", [])
+
+        if not input_numbers:
+            return False  # 如果无法获取输入数字，则认为答案不正确
+
+        try:
+            # 检查是否使用了所有给定的数字，每个数字恰好使用一次
+            used_numbers = self._extract_numbers(expression)
+            if sorted(used_numbers) != sorted(input_numbers):
+                return False
+
+            # 计算表达式的值，检查是否等于24
+            value = self._evaluate_expression(expression)
+            return abs(value - 24) < 1e-6
+
+        except Exception:
+            # 如果解析或计算过程出错，视为不正确
+            return False
+
+    def _normalize_expression(self, expression: str) -> str:
+        """标准化表达式，统一运算符符号"""
+        # 替换LaTeX格式的数学符号（注意：需要处理字面的反斜杠字符串）
+        expression = expression.replace('\\times', '*')
+        expression = expression.replace('\\div', '/')
+        expression = expression.replace('\\cdot', '*')
+
+        # 处理可能的转义序列残余
+        expression = expression.replace('\times', '*')  # 制表符+imes -> *
+        expression = expression.replace('imes', '*')     # 单独的imes -> *
+
+        # 替换Unicode乘除符号为Python可以计算的符号
+        expression = expression.replace('×', '*').replace('÷', '/')
+
+        # 替换文字形式的运算符
+        expression = re.sub(r'\bdiv\b', '/', expression, flags=re.IGNORECASE)
+        expression = re.sub(r'\bmul\b', '*', expression, flags=re.IGNORECASE)
+        expression = re.sub(r'\btimes\b', '*', expression, flags=re.IGNORECASE)
+        expression = re.sub(r'\bplus\b', '+', expression, flags=re.IGNORECASE)
+        expression = re.sub(r'\bminus\b', '-', expression, flags=re.IGNORECASE)
+
+        # 移除空格
+        expression = expression.replace(' ', '')
+
+        # 移除可能包含的"="和之后的内容
+        expression = re.sub(r'=.*$', '', expression)
+
+        return expression
+
+    def _extract_numbers(self, expression: str) -> List[int]:
+        """从表达式中提取所有使用的数字"""
+        return [int(num) for num in re.findall(r'\d+', expression)]
+
+    def _evaluate_expression(self, expression: str) -> float:
+        """
+        计算表达式的值
+
+        注意：使用eval函数存在安全风险，但在这个受控的评估环境中是可以接受的
+        """
+        # 检查表达式中是否只包含允许的字符
+        if not re.match(r'^[\d\+\-\*/\(\)\.]+$', expression):
+            raise ValueError("Expression contains invalid characters")
+
+        # 计算表达式值
+        return eval(expression)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/wordladder_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/wordladder_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..9e2fd8f030c923de812b1bf436c91895e4212854
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/wordladder_eval.py
@@ -0,0 +1,273 @@
+#!/usr/bin/env python3
+
+import json
+import re
+from typing import Any, Dict, List, Union
+
+import nltk
+import nltk.data
+from nltk.corpus import words
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self, predicted_answer: Any, ground_truth: Any,
+        params: Dict[str, Any]
+    ) -> bool:
+        raise NotImplementedError
+
+
+class WordLadderEvaluator(BaseEvaluator):
+    """
+    评估Word Ladder谜题解答的评估器
+
+    Word Ladder是一种单词转换谜题，要求：
+    1. 从起始单词变为目标单词
+    2. 每一步只能改变一个字母
+    3. 每一步都必须形成有效的英文单词
+    4. 找出一条有效路径，步数在指定范围内
+    """
+
+    def __init__(self):
+        """初始化评估器，下载并加载英语词典"""
+        self.english_words = set()
+        try:
+            # 检查 'words' 语料库是否已经下载
+            try:
+                nltk.data.find('corpora/words')
+            except nltk.downloader.DownloadError:
+                nltk.download('words', quiet=True)
+
+            self.english_words = set(words.words())
+        except Exception:
+            pass
+
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        """准备发送给模型的提示词"""
+        # 从initial_state获取谜题信息
+        if "initial_state" not in params:
+            return question
+
+        initial_state = params["initial_state"]
+        start_word = initial_state.get("start_word", "")
+        target_word = initial_state.get("target_word", "")
+        solution = initial_state.get("solution", [])
+
+        # 如果没有找到必要信息，返回原问题
+        if not start_word or not target_word:
+            return question
+
+        # 根据solution估计最小和最大步数
+        min_steps = max_steps = 0
+        if solution and len(solution) > 1:
+            path_length = len(solution) - 1
+            min_steps = max(1, path_length - 1)
+            max_steps = path_length + 1
+        else:
+            # 默认值
+            min_steps = 3
+            max_steps = 6
+
+        # 构建提示词
+        prompt = (
+            f"This is a Word Ladder puzzle. Transform the start word "
+            f"into the target word by changing one letter at a time, "
+            f"ensuring that each step forms a valid English word. "
+            f"Follow these rules:\n\n"
+            f"1. Change exactly one letter at a time.\n"
+            f"2. Each step must form a valid English word.\n"
+            f"3. Find a solution path with {min_steps} to "
+            f"{max_steps} steps.\n\n"
+            f"Starting word: {start_word}\n"
+            f"Target word: {target_word}\n\n"
+            f"Please provide the complete solution path as a list of "
+            f"words, including the starting and target words.\n"
+            f"Example answer format: \"cat -> cot -> cog -> dog\" or "
+            f"[\"cat\", \"cot\", \"cog\", \"dog\"]"
+        )
+
+        return prompt
+
+    def extract_answer(self, model_output: str) -> List[str]:
+        """从模型输出中提取Word Ladder解答路径"""
+        if isinstance(model_output, dict) and "text" in model_output:
+            model_output = model_output["text"]
+
+        # 尝试多种格式匹配
+
+        # 1. 尝试匹配 JSON 数组格式: ["word1", "word2", ...]
+        json_pattern = r'\[\s*"[a-zA-Z]+"(?:\s*,\s*"[a-zA-Z]+")*\s*\]'
+        matches = re.findall(json_pattern, model_output)
+        if matches:
+            try:
+                # 解析最后一个匹配的 JSON 数组
+                path = json.loads(matches[-1])
+                return [word.lower() for word in path]  # 转换为小写
+            except Exception:
+                pass
+
+        # 2. 尝试匹配单引号数组格式: ['word1', 'word2', ...]
+        single_quote_pattern = (
+            r'\[\s*\'[a-zA-Z]+\'(?:\s*,\s*\'[a-zA-Z]+\')*\s*\]'
+        )
+        matches = re.findall(single_quote_pattern, model_output)
+        if matches:
+            try:
+                # 将单引号替换为双引号以便 JSON 解析
+                json_str = matches[-1].replace("'", "\"")
+                path = json.loads(json_str)
+                return [word.lower() for word in path]
+            except Exception:
+                pass
+
+        # 3. 尝试匹配箭头分隔格式: word1 -> word2 -> word3
+        arrow_pattern = r'[a-zA-Z]+(?:\s*->\s*[a-zA-Z]+)+'
+        matches = re.findall(arrow_pattern, model_output)
+        if matches:
+            # 取最长的匹配（通常是最完整的路径）
+            best_match = max(matches, key=len)
+            # 分割并清理
+            path = [word.strip().lower() for word in best_match.split('->')]
+            return path
+
+        # 4. 尝试匹配逗号分隔格式: word1, word2, word3
+        comma_pattern = r'[a-zA-Z]+(?:\s*,\s*[a-zA-Z]+)+'
+        matches = re.findall(comma_pattern, model_output)
+        if matches:
+            # 取最长的匹配
+            best_match = max(matches, key=len)
+            # 分割并清理
+            path = [word.strip().lower() for word in best_match.split(',')]
+            return path
+
+        # 5. 尝试匹配行分隔格式: 每行一个单词
+        lines = model_output.split('\n')
+        word_lines = []
+        for line in lines:
+            # 寻找可能是单词的行
+            word_match = re.search(r'^[a-zA-Z]+$', line.strip())
+            if word_match:
+                word_lines.append(line.strip().lower())
+
+        if word_lines and len(word_lines) >= 2:
+            return word_lines
+
+        # 6. 尝试提取格式化列表中的单词
+        list_item_pattern = (
+            r'(?:^|\n)(?:\d+\.\s+|\*\s+|-\s+|\(\d+\)\s+)([a-zA-Z]+)'
+        )
+        matches = re.findall(list_item_pattern, model_output)
+        if matches and len(matches) >= 2:
+            return [word.lower() for word in matches]
+
+        # 7. 最后尝试直接提取所有单词
+        word_pattern = r'\b[a-zA-Z]{3,}\b'  # 至少3个字母的单词
+        all_words = re.findall(word_pattern, model_output)
+
+        # 过滤掉常见的非路径单词
+        filtered_words = []
+        common_words = {
+            "the", "and", "word", "ladder", "solution", "puzzle",
+            "step", "steps", "path", "example", "format", "answer"
+        }
+        for word in all_words:
+            if word.lower() not in common_words and len(word) >= 3:
+                filtered_words.append(word.lower())
+
+        if filtered_words and len(filtered_words) >= 2:
+            return filtered_words
+
+        # 如果所有尝试都失败，返回空列表
+        return []
+
+    def _is_valid_word(self, word: str) -> bool:
+        """检查单词是否在英语词典中"""
+        return word.lower() in self.english_words
+
+    def evaluate(
+        self, output: Union[str, List[str]], ground_truth: Any,
+        params: Dict[str, Any]
+    ) -> bool:
+        """
+        评估预测的Word Ladder解答是否正确，
+        基于规则而不是直接比对ground_truth
+
+        参数:
+            output: 模型预测的解答路径（字符串或列表）
+            ground_truth: 正确答案（仅作参考，不直接比对）
+            params: 包含谜题信息的参数
+
+        返回:
+            是否正确（布尔值）
+        """
+        # 检查output类型，如果是字符串，则提取答案
+        if isinstance(output, str):
+            pred_path = self.extract_answer(output)
+        else:
+            # 如果已经是列表形式，直接使用
+            pred_path = output
+
+        # 如果预测答案为空或只有一个单词，直接返回False
+        if not pred_path or len(pred_path) < 2:
+            return False
+
+        # 获取谜题参数
+        start_word = params.get("start_word", "").lower()
+        target_word = params.get("target_word", "").lower()
+
+        # 如果params中没有直接的start_word和target_word，
+        # 尝试从initial_state中获取
+        if not start_word or not target_word:
+            initial_state = params.get("initial_state", {})
+            start_word = initial_state.get("start_word", "").lower()
+            target_word = initial_state.get("target_word", "").lower()
+
+        # 如果仍然无法获取起始和目标单词，返回False
+        if not start_word or not target_word:
+            return False
+
+        # 标准化预测答案，转换为小写
+        pred_path = [word.lower() for word in pred_path]
+
+        # 1. 验证路径起点和终点
+        if pred_path[0] != start_word:
+            return False
+
+        if pred_path[-1] != target_word:
+            return False
+
+        # 2. 验证每一步只改变一个字母
+        for i in range(len(pred_path) - 1):
+            if not self._is_one_letter_apart(
+                pred_path[i], pred_path[i + 1]
+            ):
+                return False
+
+        # 3. 检查路径中是否有重复单词
+        if len(pred_path) != len(set(pred_path)):
+            return False
+
+        for word in pred_path:
+            if not self._is_valid_word(word):
+                return False
+
+        # 所有检查都通过，解答正确
+        return True
+
+    def _is_one_letter_apart(self, word1: str, word2: str) -> bool:
+        """检查两个单词是否只相差一个字母"""
+        # 如果长度不同，返回False
+        if len(word1) != len(word2):
+            return False
+
+        # 计算不同字母的数量
+        diff_count = sum(1 for c1, c2 in zip(word1, word2) if c1 != c2)
+
+        # 只有恰好一个字母不同时返回True
+        return diff_count == 1
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/wordsearch_eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/wordsearch_eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..2718736c6c212024af71a85affdde92f9706c724
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/evaluators/wordsearch_eval.py
@@ -0,0 +1,87 @@
+import re
+from typing import Any, Dict
+
+
+class BaseEvaluator:
+    def prepare_prompt(self, question: str, params: Dict[str, Any]) -> str:
+        raise NotImplementedError
+
+    def extract_answer(self, model_output: str) -> Any:
+        raise NotImplementedError
+
+    def evaluate(
+        self, predicted_answer: Any, ground_truth: Any,
+        initial_state: str, params: Dict[str, Any] = None
+    ) -> bool:
+        raise NotImplementedError
+
+
+class WordSearchEvaluator(BaseEvaluator):
+    def prepare_prompt(self, question: str) -> str:
+        from vlmeval.dataset.utils.mmhelix.utils.constants import PROMPT_WORDSEARCH
+        return PROMPT_WORDSEARCH.format(question, "")
+
+    def extract_answer(self, model_output: str) -> Dict[str, tuple]:
+        if not isinstance(model_output, str):
+            return {}
+
+        text = model_output.strip()
+
+        # Prefer the last \boxed{...} content if present (LaTeX-style final answer)
+        boxed_blocks = re.findall(r'\\boxed\{(.*?)\}', text, re.DOTALL)
+        if boxed_blocks:
+            text = boxed_blocks[-1].strip()
+        else:
+            # Fallback: Prefer the last <answer>...</answer> block if present
+            answer_blocks = re.findall(r'<answer>(.*?)</answer>', text, re.DOTALL | re.IGNORECASE)
+            if answer_blocks:
+                text = answer_blocks[-1].strip()
+
+        # Clean escaped characters that might appear in model output
+        # Handle cases like "BEE\\ S\\ @\\ (1,2)" -> "BEE S @ (1,2)"
+        text = re.sub(r'\\(.)', r'\1', text)
+
+        # Strict format: WORD DIRECTION @ (x, y)
+        # - WORD: letters only
+        # - DIRECTION: one of N,S,E,W,NE,NW,SE,SW (case-insensitive)
+        # - x,y: positive integers
+        # Note: do not use a trailing word boundary after ')' because ')' is a non-word char,
+        # which prevents matches at end-of-string. This pattern tolerates extra spaces.
+        pattern = r'([A-Za-z]+)\s+(N|S|E|W|NE|NW|SE|SW)\s+@\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)'
+        matches = re.findall(pattern, text, flags=re.IGNORECASE)
+
+        word_locations: Dict[str, tuple] = {}
+        for word, direction, x_str, y_str in matches:
+            normalized_word = word.lower()
+            normalized_direction = direction.upper()
+            x = int(x_str)
+            y = int(y_str)
+            word_locations[normalized_word] = (normalized_direction, (x, y))
+
+        return word_locations
+
+    def evaluate(self, predicted_answer: Any, ground_truth: Any, initial_state: Any) -> bool:
+        # Normalize both predicted and ground truth to the strict dict format
+        if isinstance(predicted_answer, str):
+            predicted_answer = self.extract_answer(predicted_answer)
+        if isinstance(ground_truth, str):
+            ground_truth = self.extract_answer(ground_truth)
+
+        if not isinstance(predicted_answer, dict) or not isinstance(ground_truth, dict):
+            return False
+
+        if not predicted_answer or not ground_truth:
+            return False
+
+        # Directions must match exactly among allowed codes; coordinates must match exactly
+        # Compare keys and values strictly
+        if set(predicted_answer.keys()) != set(ground_truth.keys()):
+            return False
+
+        for word in ground_truth.keys():
+            gt_direction, gt_coords = ground_truth[word]
+            pred_direction, pred_coords = predicted_answer.get(word, (None, None))
+            if pred_direction != gt_direction or pred_coords != gt_coords:
+                return False
+
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/metrics.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/metrics.py
new file mode 100644
index 0000000000000000000000000000000000000000..5b5b7a81b443c21c485a3579be216017c1d8a5f1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/metrics.py
@@ -0,0 +1,71 @@
+from vlmeval.dataset.utils.mmhelix.evaluator import MatchFromList, SimpleStrMatch
+from vlmeval.dataset.utils.mmhelix.evaluators.aquarium_eval import AquariumEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.binario_eval import BinarioEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.bridges_eval import BridgesEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.calcudoku_eval import CalcudokuEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.campsite_eval import CampsiteEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.cryptomath_eval import CryptoMathEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.eulero_eval import EuleroEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.futoshiki_eval import FutoshikiEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.graph_problems_eval import (
+    ConnectivityEvaluator, EulerianCycleEvaluator, EulerianPathEvaluator,
+    HamiltonianCycleEvaluator, HamiltonianPathEvaluator, TopologicalSortEvaluator)
+from vlmeval.dataset.utils.mmhelix.evaluators.hanoi_eval import TowerOfHanoiEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.hitori_eval import HitoriEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.kakuro_eval import KakuroEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.kukurasu_eval import KukurasuEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.maze_eval import MazeEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.minesweeper_eval import MinesweeperEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.nibbles_eval import NibblesEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.nonogram_eval import NonogramsEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.numbrix_eval import NumbrixEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.shingoki_eval import ShingokiEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.skyscrapers_evaluator import SkyscrapersEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.slidingpuzzle_eval import SlidingPuzzleEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.snake_eval import SnakeEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.sokoban_eval import SokobanEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.sudoku_evaluator import SudokuEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.tapa_eval import TapaEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.twentyfourpoints_evaluator import \
+    TwentyFourPointsEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.wordladder_eval import WordLadderEvaluator
+from vlmeval.dataset.utils.mmhelix.evaluators.wordsearch_eval import WordSearchEvaluator
+
+metrics = {
+    'simple_str_match': SimpleStrMatch(),
+    'match_from_list': MatchFromList(),
+    'sliding_puzzle_evaluator': SlidingPuzzleEvaluator(),
+    'eulero_evaluator': EuleroEvaluator(),
+    'hanoi_evaluator': TowerOfHanoiEvaluator(),
+    'maze_evaluator': MazeEvaluator(),
+    'minesweeper_evaluator': MinesweeperEvaluator(),
+    'numbrix_evaluator': NumbrixEvaluator(),
+    'sokoban_evaluator': SokobanEvaluator(),
+    'snake_evaluator': SnakeEvaluator(),
+    'wordsearch_evaluator': WordSearchEvaluator(),
+    'hamiltonian_path_evaluator': HamiltonianPathEvaluator(),
+    'hamiltonian_cycle_evaluator': HamiltonianCycleEvaluator(),
+    'eulerian_path_evaluator': EulerianPathEvaluator(),
+    'eulerian_cycle_evaluator': EulerianCycleEvaluator(),
+    'topological_sort_evaluator': TopologicalSortEvaluator(),
+    '24points_evaluator': TwentyFourPointsEvaluator(),
+    'calcudoku_evaluator': CalcudokuEvaluator(),
+    'cryptomath_evaluator': CryptoMathEvaluator(),
+    'kukurasu_evaluator': KukurasuEvaluator(),
+    'skyscrapers_evaluator': SkyscrapersEvaluator(),
+    'wordladder_evaluator': WordLadderEvaluator(),
+    'aquarium_evaluator': AquariumEvaluator(),
+    'binairo_evaluator': BinarioEvaluator(),
+    'campsite_evaluator': CampsiteEvaluator(),
+    'futoshiki_evaluator': FutoshikiEvaluator(),
+    'hitori_evaluator': HitoriEvaluator(),
+    'nonogram_evaluator': NonogramsEvaluator(),
+    'bridges_evaluator': BridgesEvaluator(),
+    'kakuro_evaluator': KakuroEvaluator(),
+    'shingoki_evaluator': ShingokiEvaluator(),
+    'tapa_evaluator': TapaEvaluator(),
+    'nibbles_evaluator': NibblesEvaluator(),
+    'connectivity_evaluator': ConnectivityEvaluator(),
+    'sudoku_evaluator': SudokuEvaluator(),
+    'unsupported': SimpleStrMatch(),
+}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/parser.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/parser.py
new file mode 100644
index 0000000000000000000000000000000000000000..a8ea7be402441fe735d214ffe331e7d60232961e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/parser.py
@@ -0,0 +1,71 @@
+import re
+
+import regex  # Import regex module to support recursive matching of balanced brackets
+
+
+class DefaultParser:
+    def parse(self, response):
+        if not response:
+            return ""
+
+        # Try to parse <|begin_of_box|>...</|end_of_box|> format - match from back to front
+        box_pattern = r'<\|begin_of_box\|>(.*?)<\|end_of_box\|>'
+        matches = re.findall(box_pattern, response, re.DOTALL)
+        if matches:
+            return matches[-1].strip()  # Take the last match
+
+        # Then try to parse <answer></answer> format - match from back to front
+        answer_pattern = r'<answer>(.*?)</answer>'
+        matches = re.findall(answer_pattern, response, re.DOTALL)
+        if matches:
+            return matches[-1].strip()  # Take the last match
+
+        # First try to parse \boxed{} format - match from back to front, use regex to support nested brackets
+        boxed_pattern = r'\\boxed\{((?:[^{}]|\{[^}]*\})*)\}'
+        matches = regex.findall(boxed_pattern, response)
+        if matches:
+            last_match = matches[-1]  # Take the last match
+            # Find and remove all \text{} format in the last match
+            text_pattern = r'\\text\{([^}]*)\}'
+            text_matches = regex.findall(text_pattern, last_match)
+            if text_matches:
+                return text_matches[-1].strip()
+
+            # 2. Handle truncated cases: ext{content} (missing \t)
+            elif regex.search(r'ext\{[^}]*\}', last_match):
+                ext_pattern = r'ext\{([^}]*)\}'
+                ext_matches = regex.findall(ext_pattern, last_match)
+                if ext_matches:
+                    return ext_matches[-1].strip()
+
+            # 3. Handle other possible text variants
+            elif 'text{' in last_match:
+                # Remove any form of text{...}
+                cleaned = regex.sub(r'[\\]*text\{([^}]*)\}', r'\1', last_match)
+                if cleaned.strip() != last_match.strip():
+                    return cleaned.strip()
+
+            # Try to parse \begin{array}...\end{array} format - use regex matching
+            array_pattern = r'\\begin\{array\}((?:.|\n)*?)\\end\{array\}'
+            array_matches = regex.findall(array_pattern, last_match)
+            if array_matches:
+                return array_matches[-1].strip()  # Take the last match
+
+            # Try to parse \begin{bmatrix}...\end{bmatrix} format - use regex matching
+            bmatrix_pattern = r'\\begin\{bmatrix\}((?:.|\n)*?)\\end\{bmatrix\}'
+            bmatrix_matches = regex.findall(bmatrix_pattern, last_match)
+            if bmatrix_matches:
+                return bmatrix_matches[-1].strip()  # Take the last match
+            return last_match.strip()
+
+        # Finally try to parse Answer: format - match from back to front
+        answer_matches = re.findall(r'Answer[:：]\s*(.*)', response, re.IGNORECASE | re.DOTALL)
+        if answer_matches:
+            return answer_matches[-1].strip()  # Take the last match
+
+        return response  # return the original response
+
+
+parser = {
+    'default': DefaultParser().parse,
+}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/utils/constants.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/utils/constants.py
new file mode 100644
index 0000000000000000000000000000000000000000..27c80fb1c1451d19049f95f513be557e4fb9db18
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/utils/constants.py
@@ -0,0 +1,392 @@
+PROMPT_SOKOBAN_IMAGE = """
+Your task is to solve the Sokoban puzzle according to the rules and current state shown in the image:
+
+Game Rules:
+1. You are the player and can move up, down, left, or right
+2. You can push boxes but only one at a time
+3. You cannot pull boxes
+4. Boxes can only be pushed if there's an empty space behind them
+5. The goal is to push all boxes onto target positions
+8. Walls cannot be moved through or pushed
+
+You will be given an image, in the image:
+
+1. Red squares represent the player
+2. Pink squares represent boxes
+3. Green squares represent target positions
+4. White squares represent empty spaces that can be moved into
+4. Gray blocks represent walls
+
+Direction Definitions:
+- "up": Move up
+- "down": Move down
+- "left": Move left
+- "right": Move right
+
+Current Sokoban State can be seen in the image shown below:
+
+Output Format Requirements:
+1. Your final answer should be in the format of a space-separated sequence of moves like:
+up right down left
+2. You should put your thinking process in `<think></think>` tags,
+and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+"""
+
+
+PROMPT_MAZE_IMAGE = """
+Your task is to solve the maze game according to the rules and current state below:
+
+Game Rules:
+1. The maze consists of a grid of cells
+2. Walls are represented by **bold black line** between cells, not as cells themselves
+3. You can move horizontally or vertically between adjacent cells if there is no wall between them
+4. You can only move through one cell at a time in any direction
+5. The goal is to find a path from the start cell (S) to the end cell (E)
+
+Direction Definitions:
+- "up": Move to the cell above the current position (toward the top of the maze, decreasing row number)
+- "down": Move to the cell below the current position (toward the bottom of the maze, increasing row number)
+- "left": Move to the cell to the left of the current position (decreasing column number)
+- "right": Move to the cell to the right of the current position (increasing column number)
+
+Current Maze State:
+The maze is represented in the image shown below
+
+In this representation:
+- green circule marks the start position
+- red cross marks the end position
+
+Output Format Requirements:
+1. Your final answer should be in the format like: right down left up
+2. You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+"""
+
+PROMPT_15PUZZLE_IMAGE = """Your task is to solve the 15-puzzle game according to the rules and current state below:
+
+Let me explain the 15-puzzle game rules and the current puzzle state:
+
+Game Rules:
+1. The puzzle is played on a 4x4 grid with 15 numbered tiles and one empty space
+2. You can only move tiles horizontally or vertically into the empty space
+3. The goal is to arrange the tiles in numerical order with:
+   - First row: 1, 2, 3, 4
+   - Second row: 5, 6, 7, 8
+   - Third row: 9, 10, 11, 12
+   - Fourth row: 13, 14, 15, empty space
+
+Coordinate System:
+- The grid positions are numbered from left to right and top to bottom
+- Columns (horizontal): numbered 1, 2, 3, 4 from left to right
+- Rows (vertical): numbered 1, 2, 3, 4 from top to bottom
+- Each position can be identified by its row and column (row, column)
+
+Current Puzzle State:
+The initial_state is represented in the image shown below
+
+Output Format Requirements:
+"up" means the tile below the empty space moves up into the empty space
+"down" means the tile above the empty space moves down into the empty space
+"left" means the tile to the right of the empty space moves left into the empty space
+"right" means the tile to the left of the empty space moves right into the empty space
+
+Your final answer format should be given like: up down up left right ,etc.
+You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+"""
+
+PROMPT_HANOI_IMAGE = """Your task is to solve the hanoi game according to the rules and current state below:
+
+Let me explain the Tower of Hanoi puzzle rules and the current state:
+
+Game Rules:
+1. The Tower of Hanoi consists of three pegs (numbered 1, 2, and 3) and n(maybe 3 or 4 or 5) disks of different sizes
+(from 1 to n)
+2. Disks are stacked on pegs with larger disks always below smaller ones
+3. Only one disk can be moved at a time, from the top of one peg to the top of another
+4. A larger disk cannot be placed on top of a smaller disk
+
+Current Hanoi State:
+The current state of the Tower of Hanoi is in the image shown below
+
+Goal State:
+## For 3 disks
+
+[
+   [],
+   [],
+   [3, 2, 1],
+]
+
+## For 4 disks
+[
+   [],
+   [],
+   [4, 3, 2, 1],
+]
+
+## For 5 disks
+[
+   [],
+   [],
+   [5, 4, 3, 2, 1],
+]
+
+
+In this text representation:
+- Each array [] represents a peg (from 1 to 3)
+- Numbers inside the arrays represent disks (higher numbers = larger disks)
+- The first/top elements in an array are at the bottom of the peg
+- The last/bottom elements in an array are at the top of the peg
+
+Output Format Requirements:
+1. Your final solution format should be given like:(x,y) (x,y) (x,y)...,
+where x is the disk number and y is the destination peg number
+2. You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+
+"""
+
+PROMPT_WORDSEARCH_IMAGE = """Your task is to solve the wordsearch game according to the rules and current state below:
+
+
+## Task
+You are given a word search puzzle.
+Your task is to find all the listed words hidden in the grid and provide their exact locations in the specified format.
+
+## Rules of WordDescription Search
+1. Words can be hidden horizontally, vertically, or diagonally.
+2. Words can read forwards or backwards.
+3. Words always follow a straight line (no zigzagging).
+4. Each word's location should be identified by:
+   - The starting position (coordinate where the first letter appears)
+   - The direction in which the word extends
+
+## Coordinate System
+- The grid uses coordinates where (x, y) represents the position.
+- x-axis: Numbers from 1 to width, running horizontally from left to right.
+- y-axis: Numbers from 1 to height, running vertically from top to bottom.
+- Example: Position (3, 4) means column 3 from left, row 4 from top.
+
+
+## Direction Notation
+- N: North (upward)
+- S: South (downward)
+- E: East (rightward)
+- W: West (leftward)
+- NE: Northeast (up and right)
+- NW: Northwest (up and left)
+- SE: Southeast (down and right)
+- SW: Southwest (down and left)
+
+
+WordSearch State:
+The current state of the WordSearch is shown in the image given below
+
+
+Output Format Requirements:
+1. Your final answer format should be given like: WORD DIRECTION @ (x, y),
+where WORD is the word you found,
+DIRECTION is the direction in which the word extends,
+and (x, y) is the starting position of the word.
+2. You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+"""
+
+
+PROMPT_NUMBRIX_IMAGE = """
+Your task is to solve the Numbrix puzzle based on the following rules and the current state:
+
+### Game Rules:
+1. Numbrix is played on a square grid, where some cells are already filled with numbers.
+2. You must fill in the empty cells with numbers to create a continuous path from 1 to the highest number
+(grid size squared).
+3. The numbers must be adjacent either horizontally or vertically (not diagonally).
+4. Each number can only be used once.
+5. The path must form a single continuous sequence where consecutive numbers are adjacent.
+6. **Not every empty cell needs to be filled.** In some cases, leaving some cells empty may be required,
+depending on the puzzle configuration.
+
+### Current Numbrix State:
+The current state of the Numbrix puzzle is shown in the image below.
+
+In this representation:
+- Filled cells contain the given numbers.
+- Empty cells are blank spaces.
+- Your goal is to fill the empty cells according to the rules, but remember,
+**not every empty cell needs to be filled**.
+
+### Output Format Requirements:
+3. The final answer should be the completed grid with all numbers correctly filled in,
+maintaining a clear grid format with numbers aligned in rows and columns.
+4. **Do not add extra spaces inside the grid cells.** For example, ensure that `|3|` remains `|3|`, not `| 3 |`.
+
+### Example answer format for a 5x5 grid:
+|11|10|9|2|3|
+|12|13|8|1|4|
+|15|14|7|6|5|
+|16|19|20|23|24|
+|17|18|21|22|25|
+
+You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+
+"""
+
+
+PROMPT_MINESWEEPER_IMAGE = """
+
+Your task is to solve the Minesweeper puzzle according to the rules and the current state below:
+
+**Game Rules:**
+1. Minesweeper is played on a grid where some cells contain hidden mines.
+2. Numbers on the grid represent how many mines are adjacent to that cell (including diagonally).
+3. A cell with no number means it has no adjacent mines (this is represented as a blank cell).
+4. The goal is to identify the location of all mines without detonating any.
+5. You can mark a cell as containing a mine if you're certain based on logical deduction.
+6. A mine location should be marked with `*`.
+7. Cells that are empty (unrevealed cells) should be represented by a space (` `) character.
+8. The final output should strictly follow the format provided below.
+
+**Current Minesweeper State:**
+The current state of the Minesweeper puzzle is shown in the image below.
+
+In this representation:
+- Numbers indicate the count of adjacent mines.
+- Empty cells (unrevealed cells) are represented by a space (` `).
+- The goal is to identify the positions of all the mines (using `*`).
+
+**Output Format Requirements:**
+1. Your final answer should mark all possible mine locations with `*` and **only `*`**..
+3. Empty cells (unrevealed) should be represented by a space (` `),
+and you should not place any numbers where the cells are blank.
+4. Ensure that the output strictly follows the example format below:
+   - Each row of the grid should be presented in the form: `|cell1|cell2|cell3|...|cellN|`
+   - **No extra spaces should appear between cells or at the ends of rows.**
+   - Each row must be terminated with a `\n` (newline) character.
+5. **Do not add any additional spaces or empty lines in the answer.**
+6. Follow the format shown below carefully:
+   - Rows must consist of cells separated by the `|` character.
+   - Every row must end with `\n` to ensure correct formatting.
+
+**Example answer format:**
+<answer>
+|1|2|3|2|2|*|2|*|\n
+|1|*|*|*|2|1|3|2|\n
+|1|2|3|2|2|1|2|*|\n
+| | | | |1|*|3|2|\n
+| | |1|1|2|1|2|*|\n
+| | |1|*|1| |1|1|\n
+| | |1|2|2|1| | |\n
+| | | |1|*|1| | |\n
+</answer>
+
+You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+
+Now, think step by step, and provide your solution in the format shown above.
+"""
+
+PROMPT_EULERO_IMAGE = """
+Your task is to solve the Eulero puzzle (also known as the Graeco-Latin Square or Euler Square),
+based on the rules and the current puzzle state shown below.
+
+**About the Puzzle**:
+Eulero is a logic puzzle played on a square grid of size N×N.
+Each cell must contain a **unique letter-number pair** (e.g., A1, B2).
+It combines the logic of Latin squares and Greek squares.
+
+**Goal**:
+Fill all empty cells such that the following rules are satisfied:
+
+**Global Rules (Graeco-Latin Square logic)**:
+1. Each cell contains a **letter-number pair** (like A1).
+2. Each **letter** appears **exactly once** in every row and every column.
+3. Each **number** appears **exactly once** in every row and every column.
+4. Each **letter-number pair** is **unique across the entire grid** (i.e., no duplicate pairs anywhere).
+
+**Region Rules (Eulero-specific logic)**:
+5. The grid is divided into **regions** of **exactly 3 cells each**, marked by thick black lines.
+6. For each region:
+   - Either all 3 cells must contain the **same letter-number pair** (e.g., all A1), OR
+   - All 3 cells must contain **completely different** letter-number pairs (e.g., A1, B2, C3).
+7. **Adjacent cells from different regions** (sharing an edge) must **not** contain the **same letter-number pair**.
+
+**Grid Sizes**:
+This puzzle can be of various sizes (e.g., 3×3, 4×4, 5×5, etc.).
+The rules apply consistently across all sizes. The number of unique letters and numbers equals the grid size.
+
+**Current Puzzle State**:
+The puzzle is displayed in the image below:
+- Some cells are pre-filled with letter-number pairs.
+- Blank cells are empty and must be filled in.
+- Thick black lines indicate the region boundaries.
+
+**Your Output Format**:
+1. final output must strictly follow this format:
+   - Each row should be represented as a single line of **letter-number pairs**, separated by `|` (without spaces).
+   - **Each row must be on a new line** using `\n` to separate them.
+
+   **For example**:
+
+   **For a 3×3 grid**:
+   <answer>
+   A1|B2|C3\nB3|C1|A2\nC2|A3|B1
+   </answer>
+
+   **For a 4×4 grid**:
+   <answer>
+   A1|B2|C3|D4\nB3|C4|D1|A2\nC2|D1|A4|B3\nD4|A3|B1|C2
+   </answer>
+
+   **For a 5×5 grid**:
+   <answer>
+   A1|B2|C3|D4|E5\nB3|C4|D1|E2|A5\nC2|D1|E4|A3|B5\nD4|E3|A2|B5|C1\nE5|A4|B1|C2|D3
+   </answer>
+
+   - **Do not add spaces between letter-number pairs**.
+   - **Do not add any extra spaces or lines**.
+   - **Make sure each row is separated by `\n`**.
+
+You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+Now, think step by step, and provide your solution in the format shown above.
+
+"""
+
+PROMPT_SNAKE_IMAGE = """You are a puzzle solver focusing on Snake puzzles (also known as Number Link or Tunnel puzzles).
+
+In a Snake puzzle:
+1. You need to find a path (snake) that connects the start point to the end point
+2. The path must follow horizontal and vertical movements only (no diagonal moves)
+3. The path cannot cross itself or branch out
+4. The path must pass through exactly the number of cells in each row
+and column as indicated by the row and column counts
+
+The image shows a Snake puzzle. Analyze it to find:
+- The grid size
+- The start and end points
+- The row and column counts
+- The complete solution path
+
+Find the complete path from start to end following the rules above.
+Your answer should be a sequence of coordinates in the format (row,column) representing the path from start to end.
+
+### Coordinate System:
+- Use a 0-based coordinate system where (0,0) is the top-left cell of the grid
+- Row numbers increase as you move downward
+- Column numbers increase as you move rightward
+- Coordinates are written as (row,column)
+
+### Output Format Requirements:
+Your answer should be a sequence of coordinates in the format (row,column) representing the path from start to end.
+like: (row1,col1) (row2,col2) (row3,col3) ...
+
+You should put your thinking process in `<think></think>` tags, and the final answer in `<answer></answer>` tags.
+Now, think step by step, and provide your solution in the format shown above.
+"""
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/utils/validation.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/utils/validation.py
new file mode 100644
index 0000000000000000000000000000000000000000..b2b8adfb063a1fd56ac9dda59967d9ef03eae00e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmhelix/utils/validation.py
@@ -0,0 +1,224 @@
+import re
+
+
+def puzzle_15_check(initial_state, model_input):
+    if isinstance(initial_state, str):
+        try:
+            import ast
+            initial_state = ast.literal_eval(initial_state)
+        except Exception:
+            return False
+
+    if isinstance(model_input, str):
+        # First try to extract tile numbers
+        try:
+            moves_str = model_input.strip('()[]{}')
+            moves = [int(x.strip()) for x in moves_str.split(',') if x.strip()]
+            moves_type = "tiles"
+        except ValueError:
+            # If not number tiles, try to extract directions using regex
+            # This pattern matches "up", "down", "left", "right" regardless of surrounding characters
+            direction_pattern = re.compile(r'\b(up|down|left|right)\b', re.IGNORECASE)
+            directions = direction_pattern.findall(model_input.lower())
+
+            if directions:
+                moves = directions
+                moves_type = "directions"
+            else:
+                return False
+    else:
+        moves = model_input
+        moves_type = "tiles"  # Default to number tiles
+
+    target_state = [
+        [1, 2, 3, 4],
+        [5, 6, 7, 8],
+        [9, 10, 11, 12],
+        [13, 14, 15, 0]
+    ]
+
+    current_state = [row[:] for row in initial_state]
+
+    # Find the empty position
+    empty_pos = None
+    for i in range(4):
+        for j in range(4):
+            if current_state[i][j] == 0:
+                empty_pos = (i, j)
+                break
+        if empty_pos:
+            break
+
+    move_history = []
+
+    for move in moves:
+        if moves_type == "tiles":
+            # Original number tile movement logic
+            num_pos = None
+            for i in range(4):
+                for j in range(4):
+                    if current_state[i][j] == move:
+                        num_pos = (i, j)
+                        break
+                if num_pos:
+                    break
+
+            if not num_pos:
+                return False
+
+            row_diff = abs(num_pos[0] - empty_pos[0])
+            col_diff = abs(num_pos[1] - empty_pos[1])
+
+            if not ((row_diff == 1 and col_diff == 0) or (row_diff == 0 and col_diff == 1)):
+                return False
+
+            current_state[empty_pos[0]][empty_pos[1]] = move
+            current_state[num_pos[0]][num_pos[1]] = 0
+            empty_pos = num_pos
+
+        elif moves_type == "directions":
+            # Direction movement logic
+            new_pos = None
+
+            if move == "up":
+                # Move the tile below the empty space upward
+                if empty_pos[0] < 3:  # Ensure there's a row below
+                    new_pos = (empty_pos[0] + 1, empty_pos[1])
+            elif move == "down":
+                # Move the tile above the empty space downward
+                if empty_pos[0] > 0:  # Ensure there's a row above
+                    new_pos = (empty_pos[0] - 1, empty_pos[1])
+            elif move == "left":
+                # Move the tile to the right of the empty space leftward
+                if empty_pos[1] < 3:  # Ensure there's a column to the right
+                    new_pos = (empty_pos[0], empty_pos[1] + 1)
+            elif move == "right":
+                # Move the tile to the left of the empty space rightward
+                if empty_pos[1] > 0:  # Ensure there's a column to the left
+                    new_pos = (empty_pos[0], empty_pos[1] - 1)
+
+            if not new_pos:
+                return False
+
+            # Swap empty space and the tile
+            tile_value = current_state[new_pos[0]][new_pos[1]]
+            current_state[empty_pos[0]][empty_pos[1]] = tile_value
+            current_state[new_pos[0]][new_pos[1]] = 0
+            empty_pos = new_pos
+
+        move_history.append((move, [row[:] for row in current_state]))
+
+    return current_state == target_state
+
+
+def format_state(state):
+    result = ""
+    for row in state:
+        result += " ".join(f"{num:2d}" for num in row) + "\n"
+    return result
+
+
+def print_state(state):
+    for row in state:
+        print(" ".join(f"{num:2d}" for num in row))
+    print()
+
+
+def hanoi_check(initial_state, answer):
+    pegs = [list(peg) for peg in initial_state]
+
+    # Use regex to extract all coordinate pairs in different formats
+    # This pattern matches (disk,dest) or (disk dest) with optional spaces
+    move_pattern = re.compile(r'\(\s*(\d+)\s*[,\s]\s*(\d+)\s*\)')
+    matches = move_pattern.findall(answer)
+
+    if not matches:
+        return False
+
+    moves = [(int(disk), int(dest)) for disk, dest in matches]
+
+    for disk, dest_peg in moves:
+        if dest_peg <= 0 or dest_peg > len(pegs):
+            return False
+
+        src_peg_idx = None
+        for i, peg in enumerate(pegs):
+            if disk in peg:
+                src_peg_idx = i
+                break
+
+        if src_peg_idx is None:
+            return False
+
+        if pegs[src_peg_idx][-1] != disk:
+            return False
+
+        if src_peg_idx == dest_peg - 1:
+            return False
+
+        dest_peg_idx = dest_peg - 1
+        if pegs[dest_peg_idx] and pegs[dest_peg_idx][-1] < disk:
+            return False
+
+        pegs[dest_peg_idx].append(pegs[src_peg_idx].pop())
+
+    for i in range(len(pegs) - 1):
+        if pegs[i]:
+            return False
+
+    return True
+
+
+def maze_check(text_representation, response):
+
+    maze = text_representation.strip().split('\n')
+
+    start_position = None
+    for i in range(len(maze)):
+        for j in range(len(maze[i])):
+            if maze[i][j] == 'S':
+                start_position = (i, j)
+                break
+        if start_position:
+            break
+
+    if not start_position:
+        return False
+
+    # Use regex to extract directions, case-insensitive and handle various separators
+    direction_pattern = re.compile(r'\b(up|down|left|right)\b', re.IGNORECASE)
+    directions = direction_pattern.findall(response.lower())
+
+    if not directions:
+        return False
+
+    current_position = start_position
+
+    for direction in directions:
+        i, j = current_position
+        if direction == "up":
+            if i > 0 and maze[i - 1][j] == ' - ':
+                return False
+            i -= 2
+        elif direction == "down":
+            if i < len(maze) - 1 and maze[i + 1][j] == ' - ':
+                return False
+            i += 2
+        elif direction == "left":
+            if j > 0 and maze[i][j - 1] == ' | ':
+                return False
+            j -= 2
+        elif direction == "right":
+            if j < len(maze[i]) - 1 and maze[i][j + 1] == ' | ':
+                return False
+            j += 2
+        else:
+            return False
+
+        if i < 0 or i >= len(maze) or j < 0 or j >= len(maze[i]):
+            return False
+
+        current_position = (i, j)
+
+    i, j = current_position
+    return maze[i][j] == 'E'
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmif/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmif/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmif/function_and_compare.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmif/function_and_compare.py
new file mode 100644
index 0000000000000000000000000000000000000000..612aa76f5bba9419e6e0ea5fddd74f77867e3d71
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/mmif/function_and_compare.py
@@ -0,0 +1,431 @@
+# flake8: noqa
+import re
+from typing import List
+
+import nltk
+
+# from dotenv import load_dotenv
+
+# load_dotenv()
+
+# # nltk.download("punkt")
+# nltk.data.path.append(
+#     os.environ["NLTK_DATA_PATH"]
+# )
+
+# HumanCheck: True
+
+
+def check_whether_response_paragraph_number_in_range(
+    response: str, lower_bound: int, upper_bound: int
+) -> bool:
+    def clean_text(text: str) -> str:
+        return "\n".join(line.strip() for line in text.splitlines()).strip()
+
+    cleaned_response = clean_text(response)
+
+    # use re to check the number of paragraphs
+    paragraphs = [
+        p for p in re.split(
+            r"\n\s*\n",
+            cleaned_response) if p.strip()]
+
+    actual_count = len(paragraphs)
+    # print(actual_count)
+
+    return lower_bound <= actual_count <= upper_bound
+
+# HumanCheck: True
+
+
+def check_whether_response_sentence_number_in_range(
+    response: str, lower_bound: int, upper_bound: int
+) -> bool:
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    response = clean_text(response)
+
+    # use nltk to split the response into sentences
+    sentences = nltk.sent_tokenize(response)
+    actual_count = len(sentences)
+    # print(actual_count)
+
+    return lower_bound <= actual_count <= upper_bound
+
+# HumanCheck: True
+
+
+def check_whether_each_paragraph_sentence_number_in_range(
+    response: str, lower_bound: int, upper_bound: int
+) -> bool:
+    def clean_text(text: str) -> str:
+        return "\n".join(line.strip() for line in text.splitlines()).strip()
+
+    cleaned_response = clean_text(response)
+
+    # use re to check the number of paragraphs
+    paragraphs = [
+        p for p in re.split(
+            r"\n\s*\n",
+            cleaned_response) if p.strip()]
+
+    for i, paragraph in enumerate(paragraphs):
+        # use nltk to split the paragraph into sentences
+        sentences = nltk.sent_tokenize(paragraph)
+        actual_count = len(sentences)
+        # print(f"paragraph {i}: {actual_count}")
+        if actual_count < lower_bound or actual_count > upper_bound:
+            return False
+
+    return True
+
+# HumanCheck: True
+
+
+def check_whether_each_paragraph_sentence_number_in_range_list(
+    response: str, ranges: List[List[int]]
+) -> bool:
+    def clean_text(text: str) -> str:
+        return "\n".join(line.strip() for line in text.splitlines()).strip()
+
+    cleaned_response = clean_text(response)
+
+    # use re to check the number of paragraphs
+    paragraphs = [
+        p for p in re.split(
+            r"\n\s*\n",
+            cleaned_response) if p.strip()]
+
+    if len(paragraphs) != len(ranges):
+        return False
+
+    for i, (paragraph, range_pair) in enumerate(zip(paragraphs, ranges)):
+        lower_bound, upper_bound = range_pair
+        sentences = nltk.sent_tokenize(paragraph)
+        actual_count = len(sentences)
+        # print(f"paragraph {i}: {actual_count}")
+        if not (lower_bound <= actual_count <= upper_bound):
+            return False
+
+    return True
+
+# HumanCheck: True
+
+
+def check_whether_response_word_count_in_range(
+    response: str, lower_bound: int, upper_bound: int
+) -> bool:
+    # this line is used to filter out all non-word characters
+    response_clean = re.sub(r"[^\w\s.-]", "", response)
+    word_list = response_clean.split()
+    word_count = len(word_list)
+    # print(word_count)
+    return lower_bound <= word_count <= upper_bound
+
+# HumanCheck: True
+
+
+def check_whether_each_paragraph_word_count_in_range(
+    response: str, lower_bound: int, upper_bound: int
+) -> bool:
+    # Check whether the number of words in each paragraph of the response is greater than or equal to lower_bound and less than or equal to upper_bound.
+    # Here are some examples of calling this function based on constraints:
+    # If the constraint requires that the number of words in each paragraph
+    # should be between 50 and 80, then lower_bound = 50 and upper_bound = 80.
+    def clean_text(text: str) -> str:
+        return "\n".join(line.strip() for line in text.splitlines()).strip()
+
+    cleaned_response = clean_text(response)
+
+    # use re to check the number of paragraphs
+    paragraphs = [
+        p for p in re.split(
+            r"\n\s*\n",
+            cleaned_response) if p.strip()]
+
+    for i, paragraph in enumerate(paragraphs):
+        paragraph_clean = re.sub(r"[^\w\s.-]", "", paragraph)
+        word_count = len(paragraph_clean.split())
+        # print(f"paragraph {i} word count: {word_count}")
+        if not (lower_bound <= word_count <= upper_bound):
+            return False
+
+    return True
+
+# HumanCheck: True
+
+
+def check_whether_whole_response_not_contain_certain_substrings(
+    response: str, substrings: List[str]
+) -> bool:
+    # Check whether the entire response does not contain any of the specified substrings.
+    # Here are some examples of calling this function based on constraints:
+    # If the constraint requires that the response should not contain the
+    # words "apple" and "banana", then substrings = ["apple", "banana"].
+    return all(substring not in response for substring in substrings)
+
+# HumanCheck: True
+
+
+def check_whether_whole_response_not_contain_certain_substring(
+    response: str, substring: str
+) -> bool:
+    return substring not in response
+
+# HumanCheck: True
+
+
+def check_whether_each_sentence_begin_with_certain_substring(
+    response: str, substring: str
+) -> bool:
+    # Check whether each sentence in the response starts with the specified substring.
+    # Here are some examples of calling this function based on constraints:
+    # If the constraint requires that each sentence should start with
+    # exclamation point, then substring = "!".
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    response = clean_text(response)
+
+    sentences = nltk.sent_tokenize(response)
+
+    return all(sentence.startswith(substring) for sentence in sentences)
+
+# HumanCheck: True
+
+
+def check_whether_each_paragraph_begin_with_certain_substring(
+    response: str, substring: str
+) -> bool:
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    cleaned_response = clean_text(response)
+
+    paragraphs = [
+        p for p in re.split(
+            r"\n\s*\n",
+            cleaned_response) if p.strip()]
+
+    return all(paragraph.startswith(substring) for paragraph in paragraphs)
+
+# HumanCheck: True
+
+
+def check_whether_each_paragraph_end_with_certain_substring(
+    response: str, substring: str
+) -> bool:
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    cleaned_response = clean_text(response)
+
+    paragraphs = [
+        p for p in re.split(
+            r"\n\s*\n",
+            cleaned_response) if p.strip()]
+
+    return all(paragraph.endswith(substring) for paragraph in paragraphs)
+
+# HumanCheck: True
+
+
+def check_whether_each_sentence_end_with_certain_substring(
+    response: str, substring: str
+) -> bool:
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    response = clean_text(response)
+
+    sentences = nltk.sent_tokenize(response)
+
+    return all(sentence.endswith(substring) for sentence in sentences)
+
+# HumanCheck: True
+
+
+def check_whether_whole_response_begin_with_certain_substring(
+    response: str, substring: str
+) -> bool:
+    return response.strip().startswith(substring)
+
+# HumanCheck: True
+
+
+def check_whether_whole_response_end_with_certain_substring(
+    response: str, substring: str
+) -> bool:
+    return response.strip().endswith(substring)
+
+# HumanCheck: True
+
+
+def check_whether_each_keyword_in_list_metioned_in_range(
+        response: str,
+        keywords: List[str],
+        lower_bound_times: int,
+        upper_bound_times: int) -> bool:
+    # should notice case like "Reddit" is counted as "Redditor"
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    response = clean_text(response)
+    response_lower = response.lower()
+
+    for keyword in keywords:
+        # use \b to match the whole word
+        pattern = r'\b' + re.escape(keyword.lower()) + r'\b'
+        matches = re.findall(pattern, response_lower)
+        if len(matches) < lower_bound_times or len(
+                matches) > upper_bound_times:
+            return False
+
+    return True
+
+# HumanCheck: True
+
+
+def check_whether_total_keyword_in_list_metioned_in_range(
+        response: str,
+        keywords: List[str],
+        lower_bound_times: int,
+        upper_bound_times: int) -> bool:
+    # should notice case like "Reddit" is counted as "Redditor"
+    def clean_text(response: str) -> str:
+        return "\n".join(line.strip()
+                         for line in response.splitlines()).strip()
+
+    response = clean_text(response)
+    response_lower = response.lower()
+
+    count = 0
+    for keyword in keywords:
+        pattern = r'\b' + re.escape(keyword.lower()) + r'\b'
+        matches = re.findall(pattern, response_lower)
+        count += len(matches)
+
+    return lower_bound_times <= count <= upper_bound_times
+
+# HumanCheck: True
+
+
+def check_percentage_number_precision_in_response(
+        response: str, precision: int) -> bool:
+    # All numeric values that appear before a percentage sign (%) must be
+    # rounded and retained to two decimal places.
+    pattern = r'(\d+\.\d+|\d+)\s*%'  # allow numbers and % to have spaces
+
+    matches = re.findall(pattern, response)
+
+    for num_str in matches:
+        if '.' not in num_str:
+            # no decimal point, not a float number
+            return False
+        decimal_part = num_str.split('.')[1]
+        if len(decimal_part) != precision:
+            return False
+
+    return True
+
+# HumanCheck: True
+
+
+def check_number_precision_in_response(response: str, precision: int) -> bool:
+    # Regex pattern to extract numbers, including scientific notation and
+    # percentages
+    number_pattern = r'''
+        (?<!\w)                     # Not preceded by a word character
+        [+-]?                      # Optional sign
+        (?:                        # Number formats:
+            \d{1,3}(?:,\d{3})*(?:\.\d+)?   # e.g., 1,234.56
+            | \d+\.\d+             # e.g., 123.456
+            | \.\d+                # e.g., .456
+            | \d+                  # e.g., 123
+        )
+        (?:[eE][+-]?\d+)?          # Optional scientific notation
+        %?                         # Optional percentage
+        (?!\w)                     # Not followed by a word character
+    '''
+
+    matches = re.finditer(number_pattern, response, flags=re.VERBOSE)
+
+    for match in matches:
+        num_str = match.group()
+        clean_num = num_str.replace(',', '').rstrip('%')
+
+        # Split out mantissa if scientific notation
+        if 'e' in clean_num.lower():
+            mantissa = re.split('[eE]', clean_num)[0]
+        else:
+            mantissa = clean_num
+
+        # Check digits after decimal in mantissa
+        if '.' in mantissa:
+            decimal_part = mantissa.split('.')[-1]
+            if len(decimal_part) != precision:
+                return False
+        else:
+            if precision != 0:
+                return False
+
+    return True
+
+# HumanCheck: True
+
+
+def check_whether_has_no_arabic_number_in_response(response: str) -> bool:
+    number_pattern = r"""
+        (?<![.\w])                            # Ensure no preceding . or word char
+        (?:                                   # Start of number pattern
+            \d{1,3}(?:,\d{3})+(?:\.\d+)?%?    |  # 1,000 or 1,000.00 or 1,000%
+            \d+\.\d+%?                        |  # decimals: 3.14, 0.5%
+            \d+%?                             |  # integers: 100, 100%
+            \d+(?:\.\d+)?(?:[eE][+-]?\d+)        # scientific: 5e-10, 5.09e-10
+        )
+        (?![.\w])                             # Ensure no trailing . or word char
+    """
+    numbers = re.findall(
+        number_pattern,
+        response,
+        flags=re.IGNORECASE | re.VERBOSE)
+    # print(numbers)
+    return len(numbers) == 0
+
+# HumanCheck: True
+# def check_scientific_notation_precision_in_response(
+#     response: str, significant_digits: int
+# ) -> bool:
+#     scientific_pattern = r"(?<!\w)(?:\d+\.\d+|\d+)(?:[eE][+-]?\d+)(?!\w)"
+
+#     numbers = re.findall(scientific_pattern, response)
+
+#     for number in numbers:
+#         # Split into base and exponent
+#         parts = re.split(r"[eE]", number.lower())
+#         if len(parts) != 2:
+#             continue  # Skip invalid scientific notation
+
+#         base, exponent = parts
+
+#         # Handle cases like "0.000" or "0"
+#         if all(c == "0" for c in base.replace(".", "")):
+#             base_digits = "0"  # Treat as 0 with 1 significant digit
+#         else:
+#             # Remove leading and trailing zeros (but keep significant zeros)
+#             base_digits = base.replace(".", "").lstrip("0") or "0"
+
+#         significant_count = len(base_digits)
+#         print(f"Number: {number}, Significant digits: {significant_count}")
+
+#         if significant_count != significant_digits:
+#             return False
+
+#     return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/oceanoctbench/eval_req.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/oceanoctbench/eval_req.txt
new file mode 100644
index 0000000000000000000000000000000000000000..08877b1328dac6474e1f0bc87bd81bd247637528
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/oceanoctbench/eval_req.txt
@@ -0,0 +1,2 @@
+nltk
+jieba
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/cal_scores.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/cal_scores.py
new file mode 100644
index 0000000000000000000000000000000000000000..8b9ceb6ff4feac2c4f2e480a03b5109577360ace
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/cal_scores.py
@@ -0,0 +1,543 @@
+import ast
+import math
+import warnings
+from collections import OrderedDict
+
+import numpy as np
+import pandas as pd
+
+from vlmeval.smp.file import get_intermediate_file_path
+from vlmeval.smp.vlm import gpt_key_set
+from ..judge_util import build_judge
+from .llm_extract import parallel_llm_extract
+from .matching_func import can_match_na, can_match_option
+from .tools.files import build_eval_paths, get_judge_tag_from_score_fn
+
+
+def exact_match(pred: str, target: str) -> float:
+    pred = str(pred).strip().lower()
+    target = str(target).strip().lower()
+    return 1. if pred == target else 0.
+
+
+def abs_dist_norm(pred: float, target: float) -> float:
+    if target == 0.0:
+        return abs(pred - target)
+    else:
+        return abs((pred - target) / target)
+
+
+def mean_relative_accuracy(
+    pred: float,
+    target: float,
+    start: float = 0.5,
+    end: float = 0.95,
+    interval: float = 0.05,
+) -> float:
+    # TODO：check this, should be + 1, but in vsi code this is + 2
+    num_pts = int((end - start) / interval + 2)
+    conf_intervs = np.linspace(start, end, num_pts)
+    err = abs_dist_norm(pred, target)
+    ok = (err <= (1 - conf_intervs)).astype(float)
+    return float(ok.mean())
+
+
+def to_float(x):
+    try:
+        return float(x)
+    except Exception:
+        return None
+
+
+def _safe_len_candidates(val):
+    if val is None or (isinstance(val, float) and pd.isna(val)):
+        return None
+    if isinstance(val, list):
+        return len(val)
+    if isinstance(val, str):
+        try:
+            parsed = ast.literal_eval(val)
+            if isinstance(parsed, list):
+                return len(parsed)
+        except Exception:
+            return None
+    return None
+
+
+def _ensure_options_count_row(row, default_choices=4):
+    n = None
+    if 'candidates' in row:
+        n = _safe_len_candidates(row['candidates'])
+    elif 'options' in row:
+        n = _safe_len_candidates(row['options'])
+    return n if (isinstance(n, int) and n > 0) else default_choices
+
+
+# ---------- Rule-based scoring ----------
+def compute_mcq_score(df: pd.DataFrame) -> pd.DataFrame:
+    preds_extracted, acc = [], []
+    for _, r in df.iterrows():
+        pred_raw = str(r['prediction'])
+        gt_raw = str(r['answer']).strip()
+
+        pred = can_match_option(pred_raw)
+        gt = can_match_option(gt_raw)
+
+        preds_extracted.append(pred)
+        acc.append(exact_match(pred, gt))
+
+    df = df.copy()
+    df['pred_extracted'] = preds_extracted
+    df['hit'] = acc
+    return df
+
+
+def compute_na_score(df: pd.DataFrame) -> pd.DataFrame:
+    """
+    Compute Mean Relative Accuracy (MRA) for numerical-answer (NA) items,
+    following the VSI codebase.
+
+    Definition:
+        For prediction ŷ, ground-truth y, and confidence threshold θ,
+        the relative accuracy is 1[ |ŷ - y| / |y| < 1 - θ ].
+        MRA averages this relative accuracy over θ ∈ {0.50, 0.55, ..., 0.95}.
+
+    Reference:
+        Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces. (https://arxiv.org/pdf/2412.14171)  # noqa: E501
+    """
+    preds_extracted, mra_scores = [], []
+
+    for _, r in df.iterrows():
+        pred_num = can_match_na(str(r['prediction']))
+        gt_num = to_float(r['answer'])
+
+        preds_extracted.append(pred_num)
+
+        if pred_num is None or gt_num is None or math.isnan(gt_num):
+            mra_scores.append(0.0)          # WORST_CASE
+        else:
+            mra_scores.append(mean_relative_accuracy(pred_num, gt_num, .5, .95, .05))
+
+    df = df.copy()
+    df['pred_extracted'] = preds_extracted
+    df['MRA:.5:.95:.05'] = mra_scores
+    return df
+
+
+def compute_caa_score(df_all: pd.DataFrame, default_choices: int = 4) -> float:
+    """
+    Compute Class-Adjusted Accuracy (CAA) for Multiple Choice Questions.
+
+    Definition:
+    For each item i with n_i options and correctness indicator X_i ∈ {0, 1},
+        CAA = (Σ_i X_i - Σ_i (1 / n_i)) / (N - Σ_i (1 / n_i))
+            - N   : total number of evaluated items.
+            - X_i : 1 if the prediction for item i is correct, otherwise 0.
+            - n_i : number of answer options for item i.
+
+    Reference:
+        SITE: Towards Spatial Intelligence Thorough Evaluation. (https://arxiv.org/pdf/2505.05456)
+
+    """
+    if len(df_all) == 0:
+        return 0.0
+    n_list = df_all.apply(lambda r: _ensure_options_count_row(r, default_choices), axis=1)
+    xi = df_all['hit'].astype(int)
+    N = len(df_all)
+    sum_Xi = xi.sum()
+    sum_1_ni = (1.0 / n_list).sum()
+    denom = N - sum_1_ni
+    return float((sum_Xi - sum_1_ni) / denom) if denom != 0 else 0.0
+
+
+# High-level evaluation core for MCQ-style datasets
+def eval_mcq_score(
+    *,
+    load_fn,
+    eval_file: str,
+    score_fn,
+    group_col: str | list[str] = 'category',
+    order: list[str] | dict[str, list[str]] | None = None,
+    dataset_name: str = 'MCQ',
+    return_scored: bool = False,
+):
+    judge_tag = get_judge_tag_from_score_fn(score_fn)
+    result_file, xlsx_path, acc_tsv_path = build_eval_paths(eval_file, judge_tag)
+
+    # only effective for LLM-based score_fn; rule-based is no-op
+    attach_score_cache(
+        score_fn=score_fn,
+        eval_file=eval_file,
+        judge_tag=judge_tag,
+        key_col='index',
+    )
+
+    data = load_fn(eval_file)
+    if 'index' in data.columns:
+        data = data.sort_values(by='index')
+    data['prediction'] = [str(x) for x in data['prediction']]
+
+    mcq_scored = score_fn(data.copy())
+
+    # ---------- group_cols / order_map ----------
+    if isinstance(group_col, str):
+        group_cols = [group_col]
+    else:
+        group_cols = list(group_col)
+
+    if isinstance(order, dict) or order is None:
+        order_map: dict[str, list[str]] = order or {}
+    else:
+        order_map = {group_cols[0]: order}
+
+    summary = OrderedDict()
+    overall_acc = float(mcq_scored['hit'].mean()) if len(mcq_scored) else 0.0
+    summary['overall'] = overall_acc * 100.0
+
+    # ---------- category && tasks ----------
+    for gc in group_cols:
+        if gc not in mcq_scored.columns:
+            continue
+
+        preferred = order_map.get(gc, []) or []
+        present = list(mcq_scored[gc].dropna().unique().tolist())
+        remain = [c for c in present if c not in preferred]
+        cat_order = preferred + remain
+
+        prefix = '' if len(group_cols) == 1 else f'{gc}.'
+
+        for cat in cat_order:
+            sub = mcq_scored[mcq_scored[gc] == cat]
+            if len(sub):
+                acc = float(sub['hit'].mean()) * 100.0
+                summary[f'{prefix}{cat}_accuracy'] = acc
+
+    tab_keys = ', '.join(list(summary.keys()))
+    tab_vals = ', '.join([f'{v:.3f}' for v in summary.values()])
+    summary['tabulated_keys'] = tab_keys
+    summary['tabulated_results'] = tab_vals
+
+    # ---------- pkl ----------
+    try:
+        import pickle
+        with open(result_file, 'wb') as f:
+            pickle.dump({'mcq_scored': mcq_scored, 'summary': summary}, f)
+        print(f'[save] result saved to {result_file}')
+    except Exception as e:
+        warnings.warn(f'[save] failed to save result to {result_file}: {e}')
+
+    # ---------- extract_matching.xlsx ----------
+    try:
+        prefer_front = [
+            'index', 'question_type',
+            group_cols[0] if group_cols else None,
+            'prediction', 'pred_extracted', 'answer', 'hit'
+        ]
+        prefer_front = [c for c in prefer_front if c is not None]
+
+        merged = mcq_scored.copy()
+        ordered_cols = [c for c in prefer_front if c in merged.columns] + \
+                       [c for c in merged.columns if c not in prefer_front]
+        merged = merged[ordered_cols]
+        with pd.ExcelWriter(xlsx_path, engine='openpyxl') as writer:
+            merged.to_excel(writer, sheet_name='ALL', index=False)
+        print(f'[save] extract & matching saved to {xlsx_path}')
+    except Exception as e:
+        warnings.warn(f'[save] failed to save extract xlsx to {xlsx_path}: {e}')
+
+    # ---------- acc.tsv ----------
+    try:
+        acc_df = pd.DataFrame(
+            [(k, v) for k, v in summary.items()
+             if k not in ('tabulated_keys', 'tabulated_results')],
+            columns=['metric', 'value']
+        )
+
+        metric_order = ['overall']
+
+        for gc in group_cols:
+            preferred = order_map.get(gc, []) or []
+            prefix = '' if len(group_cols) == 1 else f'{gc}.'
+            metric_order += [f'{prefix}{c}_accuracy' for c in preferred]
+
+        metric_order += [k for k in acc_df['metric'].tolist()
+                         if k not in metric_order]
+
+        acc_df = acc_df.set_index('metric').reindex(metric_order).dropna(subset=['value'])
+        wide = acc_df.T
+        wide.to_csv(acc_tsv_path, sep='\t', index=False, float_format='%.4f')
+
+        print(f'[save] accuracy table saved to {acc_tsv_path}')
+    except Exception as e:
+        warnings.warn(f'[save] failed to save acc tsv to {acc_tsv_path}: {e}')
+
+    print(f'[{dataset_name}] summary: {summary}')
+    if return_scored:
+        return summary, mcq_scored
+    return summary
+
+
+# ---------- LLM-based scoring ----------
+def compute_score_llm(
+    df: pd.DataFrame,
+    model,
+    *,
+    mode: str = 'mcq',
+    max_retry: int = 3,
+    nproc: int = 4,
+    **extra,
+) -> pd.DataFrame:
+    """
+    LLM-based MCQ/VQA scoring.
+
+    Args:
+        df: input dataframe (must contain at least question / prediction / answer).
+        model: judge model with .generate(prompt: str) -> str
+        mode: 'mcq' or 'vqa'
+        max_retry: max retry times per sample
+        nproc: number of worker threads for parallel judging
+        extra:
+            - cache_file: optional cache pkl path
+            - key_col: column name used as cache key (default: 'index')
+    """
+    cache_file: str | None = extra.get('cache_file', None)
+    key_col: str = extra.get('key_col', 'index')
+
+    df = df.copy()
+    grades, extracted_list = parallel_llm_extract(
+        df=df,
+        model=model,
+        mode=mode,
+        max_retry=max_retry,
+        nproc=nproc,
+        cache_file=cache_file,
+        key_col=key_col,
+    )
+
+    hits = [1 if g == 'A' else 0 for g in grades]
+
+    df['judge_grade'] = grades          # 'A' / 'B' / 'C'
+    df['pred_extracted'] = extracted_list
+    df['hit'] = hits
+    return df
+
+
+def compute_na_score_llm(
+    df: pd.DataFrame,
+    model,
+    *,
+    mode: str = 'vqa',
+    max_retry: int = 3,
+    nproc: int = 4,
+    **extra,
+) -> pd.DataFrame:
+    """
+    LLM-based NA scoring.
+
+    Workflow:
+      - Use the LLM to extract the final numeric answer (mode='vqa')
+      - Then compute MRA from extracted number and ground truth.
+    """
+
+    cache_file: str | None = extra.get('cache_file', None)
+    key_col: str = extra.get('key_col', 'index')
+
+    df = df.copy()
+    grades, extracted_list = parallel_llm_extract(
+        df=df,
+        model=model,
+        mode=mode,
+        max_retry=max_retry,
+        nproc=nproc,
+        cache_file=cache_file,
+        key_col=key_col,
+    )
+
+    pred_nums: list[float | None] = []
+    mra_list: list[float] = []
+
+    for grade, extracted, (_, row) in zip(
+        grades, extracted_list, df.iterrows()
+    ):
+        pred_num = to_float(extracted)
+        gt_num = to_float(row.get('answer', None))
+
+        if (
+            pred_num is None
+            or gt_num is None
+            or math.isnan(gt_num)
+            or grade == 'C'
+        ):
+            mra = 0.0
+        else:
+            mra = mean_relative_accuracy(pred_num, gt_num, 0.5, 0.95, 0.05)
+
+        pred_nums.append(pred_num)
+        mra_list.append(mra)
+
+    df['judge_grade'] = grades              # 'A' / 'B' / 'C'
+    df['pred_extracted'] = pred_nums        # float or None
+    df['MRA:.5:.95:.05'] = mra_list
+    return df
+
+
+# ---------- Factory func ----------
+def attach_score_cache(
+    score_fn,
+    eval_file: str,
+    judge_tag: str,
+    *,
+    key_col: str = 'index',
+    sub_tag: str | None = None,
+):
+    """
+    Attach a cache file to an LLM-based score_fn for resume.
+
+    cache_file naming convention:
+        <eval_file base>_{judge_tag}[_<sub_tag>]_judge.<ext>
+
+    This function is no-op if:
+        - score_fn is None
+        - score_fn.judge_mode != 'llm'
+        - score_fn does not have 'llm_cache' attribute
+    """
+    if score_fn is None:
+        return None
+
+    if getattr(score_fn, 'judge_mode', 'rule') != 'llm':
+        return None
+
+    llm_cache = getattr(score_fn, 'llm_cache', None)
+    if llm_cache is None:
+        return None
+
+    suffix_parts = [f'_{judge_tag}']
+    if sub_tag:
+        suffix_parts.append(f'_{sub_tag}')
+    suffix = ''.join(suffix_parts) + '_judge'
+
+    cache_file = get_intermediate_file_path(
+        eval_file,
+        suffix=suffix,
+        target_format='pkl',
+    )
+
+    llm_cache['file'] = cache_file
+    llm_cache['key_col'] = key_col
+    return cache_file
+
+
+def _build_llm_judge(judge_kwargs: dict, *, task_name: str):
+    """
+    Try to build an LLM judge from judge_kwargs.
+
+    Returns:
+        - model instance if everything is OK;
+        - None if API key is missing or the judge is not working.
+    """
+    if not gpt_key_set():
+        warnings.warn(
+            f'OPENAI_API_KEY is not set properly, fallback to rule-based {task_name} scoring.'
+        )
+        return None
+
+    model = build_judge(**judge_kwargs)
+    if (model is None) or (hasattr(model, "working") and not model.working()):
+        warnings.warn(
+            f'LLM judge is not working properly, fallback to rule-based {task_name} scoring.'
+        )
+        return None
+
+    return model
+
+
+def _build_score_fn(
+    *,
+    task_name: str,
+    judge_kwargs: dict,
+    rule_fn: callable,
+    llm_fn: callable,
+    mode: str | None = None,
+):
+    """
+    Generic factory to choose between rule-based scoring and LLM-based scoring.
+
+    Args:
+        task_name: for logging only, e.g. "MCQ" / "NA".
+        judge_kwargs: kwargs used to build the judge model.
+        rule_fn: rule-based scorer, signature: rule_fn(df) -> df.
+        llm_fn: LLM-based scorer, signature:
+            llm_fn(df, model, mode=..., max_retry=..., nproc=...) -> df
+        mode: if not None, passed as mode=mode to llm_fn (for MCQ).
+    """
+    model_name = judge_kwargs.get('model', None)
+
+    def _make_rule_score_fn() -> callable:
+        def score_fn(df: pd.DataFrame) -> pd.DataFrame:
+            return rule_fn(df)
+
+        score_fn.judge_mode = 'rule'
+        # for rule-based path, if model_name is None, we treat it as 'extract_matching'
+        score_fn.judge_model = model_name or 'extract_matching'
+        return score_fn
+
+    # 1. Rule-based path
+    if model_name is None or model_name in ('exact_matching', 'extract_matching'):
+        return _make_rule_score_fn()
+
+    # 2. Build LLM judge
+    model = _build_llm_judge(judge_kwargs, task_name=task_name)
+    if model is None:
+        return _make_rule_score_fn()
+
+    max_retry = judge_kwargs.get('retry', 3)
+    nproc = judge_kwargs.get('nproc', judge_kwargs.get('api_nproc', 1) or 1)
+
+    llm_cache = {
+        'file': None,
+        'key_col': 'index',
+    }
+
+    # 3. Wrap into df -> df scorer
+    def score_fn(df: pd.DataFrame) -> pd.DataFrame:
+        kwargs = dict(
+            df=df,
+            model=model,
+            mode=mode,
+            max_retry=max_retry,
+            nproc=nproc,
+            cache_file=llm_cache['file'],
+            key_col=llm_cache['key_col'],
+        )
+        return llm_fn(**kwargs)
+
+    score_fn.judge_mode = 'llm'
+    score_fn.judge_model = model_name
+    score_fn.llm_cache = llm_cache
+    return score_fn
+
+
+def build_mcq_score_fn(**judge_kwargs):
+    """
+    Build an MCQ scoring function based on judge_kwargs['model'].
+    """
+    return _build_score_fn(
+        task_name='MCQ',
+        judge_kwargs=judge_kwargs,
+        rule_fn=compute_mcq_score,
+        llm_fn=compute_score_llm,   # note: this is the generic LLM scorer
+        mode='mcq',
+    )
+
+
+def build_na_score_fn(**judge_kwargs):
+    """
+    Build an NA scoring function based on judge_kwargs['model'].
+    """
+    return _build_score_fn(
+        task_name='NA',
+        judge_kwargs=judge_kwargs,
+        rule_fn=compute_na_score,
+        llm_fn=compute_na_score_llm,
+        mode='vqa',
+    )
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/llm_extract.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/llm_extract.py
new file mode 100644
index 0000000000000000000000000000000000000000..f6b8253c3c66a7bf73dd50b3f0c688e07e5371c5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/llm_extract.py
@@ -0,0 +1,312 @@
+import os
+import re
+from typing import Any, Dict, List
+
+import pandas as pd
+
+from vlmeval.smp.file import load
+from vlmeval.smp.log import get_logger
+from vlmeval.utils.mp_util import track_progress_rich
+from .tools.utils import build_choices
+
+logger = get_logger(__name__)
+
+
+GENERIC_EXTRACT_JUDGE_PROMPT = (
+    "You are an expert grading assistant.\n"
+    "Your job has TWO tasks:\n"
+    "1) From the candidate's full response, EXTRACT the final answer in a concise, normalized form.\n"
+    "2) Compare this extracted answer with the STANDARD ANSWER and grade it as:\n"
+    "   - A: CORRECT\n"
+    "   - B: INCORRECT\n"
+    "   - C: INVALID\n"
+    "\n"
+    "Here are the detailed evaluation criteria:\n"
+
+    "1. ALWAYS refer to the given STANDARD ANSWER. You do NOT need to re-solve the question; the standard answer has "
+    "already been provided and is always correct. Your only job is to judge whether the candidate's final answer is "
+    "consistent with the standard answer according to the form of the question. THE STANDARD ANSWER IS ALWAYS CORRECT "
+    "AND THE QUESTION IS PERFECTLY VALID. NEVER QUESTION THEM.\n"
+
+    "2. ONLY compare the FINAL ANSWER — COMPLETELY IGNORE any potential errors or issues in the REASONING PROCESS. "
+    "Even if the reasoning is wrong, as long as the final answer matches the standard answer, grade it as CORRECT.\n"
+
+    "3. Answers may be expressed in different ways (e.g., mathematical expressions, textual descriptions). As long as "
+    "the meaning is the same as the standard answer, treat them as equivalent. If the standard answer does not specify "
+    "a unit but the candidate's answer includes a correct unit for the given value, consider it CORRECT.\n"
+
+    "4. Some answers may consist of multiple items, such as multiple-choice questions with multiple correct options, "
+    "multi-select questions, or multi-blank fill-in-the-blank questions. Regardless of the question type, the final "
+    "answer is considered CORRECT only if it matches the standard answer exactly at the level of all required items. "
+    "For multi-select or multi-blank questions, ALL parts must be answered correctly and match the standard answer "
+    "exactly to be deemed CORRECT.\n"
+
+    "5. If the candidate's answer is wrapped in LaTeX-style markers like \\boxed{{...}}, IGNORE the \\boxed and only "
+    "use the inner content as the candidate's final answer when comparing with the standard answer.\n"
+
+    "6. If the candidate's answer is INVALID — for example, incomplete (cut off mid-response), containing a large "
+    "amount of abnormal repetitive content, clearly irrelevant to the question, or explicitly refusing to answer due "
+    "to ethical concerns, lack of information, or other external factors — then you MUST grade it as C: INVALID.\n"
+
+    "7. This instruction applies to all problem types, including single-choice MCQ, multi-select MCQ, numeric "
+    "problems, short-answer questions, and general VQA-style questions. In all cases, only the FINAL ANSWER and its "
+    "consistency with the standard answer matter.\n"
+
+    "8. The question or options may contain image placeholders such as '<image>', '<image id=1>', or similar tokens. "
+    "You CANNOT see these images. Treat these placeholders as unknown content and DO NOT hallucinate or infer any "
+    "specific visual details from them. If the standard answer or candidate's answer refers to an option associated "
+    "with an image (e.g., 'choose A'), judge correctness only based on the stated answer, not by imagining the image.\n"
+
+    "\n"
+    "IMPORTANT – OUTPUT FORMAT:\n"
+    "• You MUST return EXACTLY ONE line in the following format:\n"
+    "  <GRADE>\\t<EXTRACTED_ANSWER>\n"
+    "  where <GRADE> is one of A, B, or C.\n"
+    "• <EXTRACTED_ANSWER> should be the final answer you extracted from the candidate's response, in a normalized, "
+    "concise form (e.g., a number, a letter option, or a short phrase).\n"
+    "• If you cannot extract any meaningful answer, or the response is INVALID, output:\n"
+    "  C\\tN/A\n"
+    "• Do NOT add any extra text, explanation, or additional lines.\n"
+    "\n"
+    "Now, judge the following question.\n"
+    "<Original Question Begin>\n"
+    "{question}\n"
+    "{options_block}"
+    "<Original Question End>\n"
+    "<Standard Answer Begin>\n"
+    "{gold_answer}\n"
+    "<Standard Answer End>\n"
+    "<Candidate's Answer Begin>\n"
+    "{llm_response}\n"
+    "<Candidate's Answer End>\n"
+    "Your output:"
+)
+
+
+def build_option_str(option_dict):
+    s = ''
+    for c, content in option_dict.items():
+        if not pd.isna(content):
+            s += f'{c}. {content}\n'
+    return s
+
+
+def call_llm_extract(
+    model,
+    max_retry: int,
+    question: str,
+    prediction: str,
+    gold_answer: str,
+    options_block: str = ''
+):
+    """
+    Generic LLM call + parsing helper.
+
+    Returns:
+        (grade, extracted_answer)
+          - grade: 'A' / 'B' / 'C'
+          - extracted_answer: the final answer extracted by the LLM as a string,
+            or 'N/A' if none can be extracted.
+    """
+    prompt = GENERIC_EXTRACT_JUDGE_PROMPT.format(
+        question=question,
+        gold_answer=gold_answer,
+        llm_response=prediction,
+        options_block=options_block,
+    )
+
+    for _ in range(max_retry):
+        ans = model.generate(prompt).strip()
+        if 'Failed to obtain answer via API' in ans:
+            logger.warning('GPT API failed to answer. ')
+            continue
+
+        # Use only the first non-empty line to avoid verbose responses
+        line = ans.splitlines()[0].strip()
+
+        # Case 1. Grade + extracted
+        m = re.match(r'^\s*([ABC])\b(.*)$', line)
+        if m:
+            grade = m.group(1)
+
+            # Clean grade: uppercase and clamp to {A, B, C}
+            grade = str(grade).strip().upper()
+            if grade not in ('A', 'B', 'C'):
+                grade = 'C'
+
+            # Clean extracted answer
+            rest = m.group(2)  # get the raw remainder
+            rest = re.sub(r'^[\s\|,:]+', '', rest)  # strip leading whitespace + common separators
+            rest = re.sub(r'^(?:\\[tnr])+', '', rest)  # turn "\t", "\n", "\r" to spaces
+
+            extracted = rest.strip() or 'N/A'
+            return grade, extracted
+
+        # Case 2. Grade only
+        m2 = re.match(r'^\s*([ABC])\s*$', ans)
+        if m2:
+            grade = str(m2.group(1)).strip().upper()
+            if grade not in ('A', 'B', 'C'):
+                grade = 'C'
+            return grade, 'N/A'
+
+        logger.warning(f'Unparsable LLM output: {ans}')
+
+    logger.warning('LLM extract failed after max_retry, fallback to INVALID.')
+    return 'C', 'N/A'
+
+
+def extract_ans_by_llm(
+    model,
+    row: pd.Series,
+    mode: str = 'mcq',
+    max_retry: int = 3
+):
+    """
+    Generic LLM-based extraction + grading entry point.
+
+    Returns:
+        (grade, extracted_answer)
+        - grade in {'A', 'B', 'C'}
+        - extracted_answer: the final answer string extracted by the LLM
+    """
+    valid_mode = ['mcq', 'vqa']
+    assert mode in valid_mode, f'Extract llm func mode must be in {valid_mode}, but got {mode}!'
+
+    question = str(row.get('question', ''))
+    prediction = str(row.get('prediction', ''))
+    gold_raw = row.get('answer', '')
+
+    # Mode mcq
+    if mode == 'mcq':
+        # Build choices
+        choices = build_choices(row)
+        option_str = build_option_str(choices) if choices else ''
+
+        # Build options block for llm to know if there are options
+        options_block = ''
+        if option_str:
+            options_block = 'Options:\n' + option_str + '\n'
+        else:
+            options_block = ''
+
+        # Standard answer: prefer "letter + text" form if possible
+        answer_letter = str(gold_raw).strip().upper()
+        if choices and answer_letter in choices:
+            gold_answer = f'{answer_letter}. {choices[answer_letter]}'
+        else:
+            # Fallback: use raw answer field
+            gold_answer = str(gold_raw)
+
+    # Mode vqa
+    else:
+        options_block = ''
+        gold_answer = str(gold_raw)
+
+    grade, extracted = call_llm_extract(
+        model=model,
+        max_retry=max_retry,
+        question=question,
+        prediction=prediction,
+        gold_answer=gold_answer,
+        options_block=options_block,
+    )
+
+    return grade, extracted
+
+
+def parallel_llm_extract(
+    df: pd.DataFrame,
+    model,
+    *,
+    mode: str,
+    max_retry: int,
+    nproc: int,
+    cache_file: str | None = None,
+    key_col: str = 'index',
+) -> tuple[list, list]:
+    """
+    Run LLM-based answer extraction with optional cache.
+
+    Returns:
+        grades: list of 'A' / 'B' / 'C' (or None)
+        extracted_list: list of extracted answer strings (or None)
+    """
+    valid_mode = ['mcq', 'vqa']
+    assert mode in valid_mode, f'LLM extract mode must be in {valid_mode}, but got {mode}!'
+
+    df = df.copy()
+    rows: List[Dict[str, Any]] = list(df.to_dict(orient='records'))
+
+    def _one_sample(row: Dict[str, Any]):
+        """
+        Per-sample evaluation used by track_progress_rich.
+        Returns (grade, extracted), where grade ∈ {'A', 'B', 'C'}.
+        """
+        row = pd.Series(row)
+        grade, extracted = extract_ans_by_llm(
+            model=model,
+            row=row,
+            mode=mode,
+            max_retry=max_retry,
+        )
+        return grade, extracted
+
+    # ===== case 1: no cache, plain parallel run =====
+    if not cache_file:
+        tasks = [dict(row=r) for r in rows]
+        results = track_progress_rich(
+            func=_one_sample,
+            tasks=tasks,
+            nproc=nproc,
+        )
+        grades = [g for g, _ in results]
+        extracted_list = [e for _, e in results]
+        return grades, extracted_list
+
+    # ===== case 2: with cache, resume by key_col =====
+    # cache format: {key: (grade, extracted)}
+    cache: dict = {}
+    if os.path.exists(cache_file):
+        try:
+            cache = load(cache_file)
+            if not isinstance(cache, dict):
+                cache = {}
+        except Exception:
+            cache = {}
+
+    grades: list = [None] * len(rows)
+    extracted_list: list = [None] * len(rows)
+
+    tasks: list[dict] = []
+    keys: list = []
+    task_pos: list[int] = []
+
+    for i, row in enumerate(rows):
+        key = row.get(key_col, None)
+        if key is not None and key in cache:
+            val = cache[key]
+            if isinstance(val, (list, tuple)) and len(val) >= 2:
+                g, ex = val[0], val[1]
+            else:
+                g, ex = None, None
+            grades[i] = g
+            extracted_list[i] = ex
+        else:
+            tasks.append(dict(row=row))
+            keys.append(key)
+            task_pos.append(i)
+
+    if tasks:
+        results = track_progress_rich(
+            func=_one_sample,
+            tasks=tasks,
+            nproc=nproc,
+            save=cache_file,
+            keys=keys,
+        )
+        for pos, (g, ex) in zip(task_pos, results):
+            grades[pos] = g
+            extracted_list[pos] = ex
+
+    return grades, extracted_list
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/matching_func.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/matching_func.py
new file mode 100644
index 0000000000000000000000000000000000000000..20de8b6f5073f7d2816840c2a63f7a285b1047e9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/matching_func.py
@@ -0,0 +1,293 @@
+import ast
+import re
+
+from num2words import num2words
+
+# Zero-width characters (BOM, ZWSP, ZWNJ, ZWJ)
+ZW_RE = re.compile(
+    r'[\u200b\u200c\u200d\ufeff]'
+)
+
+# Generic numeric pattern: integer / float / scientific notation
+NUMERIC_PATTERN = r'[-+]?(?:\d*\.\d+|\d+)(?:[eE][-+]?\d+)?'
+
+_NUM_RE = re.compile(NUMERIC_PATTERN)
+
+# <answer>...</answer> block with a leading option letter (A–J)
+TAGGED_ANSWER_BLOCK = re.compile(
+    r'<\s*answer\b[^>]*>'  # <answer ...>
+    r'\s*'
+    r'([A-Ja-j])'
+    r'(?:\s*[\.．:：\)\]】、])?'  # Optional trailing punctuation, e.g. A. / A: / A) etc.
+    r'.*?'
+    r'<\s*/\s*answer\s*>',
+    flags=re.IGNORECASE | re.DOTALL,
+)
+
+# Numeric answer inside <answer>...</answer>
+TAGGED_NUMERIC_ANSWER = re.compile(
+    rf'<\s*answer\b[^>]*>'
+    rf'\s*'
+    rf'({NUMERIC_PATTERN})'
+    rf'(?:[^\d<][^<]*)?'
+    rf'<\s*/\s*answer\s*>',
+    flags=re.IGNORECASE | re.DOTALL,
+)
+
+
+# Matching func for mcq
+def can_match_option(
+    answer_text: str,
+    choices=None,
+    tail_lines: int = 6,
+    tail_window: int = 800
+):
+    """
+    Extract a single-choice option letter from free-form model output.
+
+    Return:
+        - 'A'..'F' (or restricted by `choices`) if a reliable match is found
+        - False otherwise
+
+    Procedure (1 → 7):
+      1) Dynamic letter set: build allowed letters (default A–F; shrink if `choices` given)
+      2) Block-level: <answer>...</answer> with leading letter (A–J allowed here)
+      3) Tail anchors: a letter immediately before </answer> or </think> within the tail window
+      4) Last-lines (after think-tail): scan last N lines after the last </think> (or <think>)
+         - full-line single letter (markdown/brackets allowed)
+         - labeled line start "A. ..."/"B) ..."
+         - unique inline uppercase token not part of words
+      5) Last-lines (global tail): same scan on the last N lines of the whole tail window
+      6) Phrase-style conclusion (tail window only), e.g. "final answer: B"
+         - capture is UPPERCASE only; use the last occurrence
+      7) Global fallback (strict): after removing enumeration lines, accept ONLY one
+         unique UPPERCASE standalone token across the entire text
+    """
+    # 1) Dynamic letter set
+    if not isinstance(answer_text, str):
+        return False
+    text = ZW_RE.sub('', answer_text.strip())
+
+    if choices:
+        letters_sorted = ''.join(sorted({str(c).strip().upper()[:1] for c in choices if str(c)}))
+        letters = ''.join([ch for ch in 'ABCDEFGHIJ' if ch in letters_sorted]) or 'ABCDEF'
+    else:
+        letters = 'ABCDEF'
+
+    # 2) Block-level <answer>...</answer>
+    m_block = TAGGED_ANSWER_BLOCK.search(text)
+    if m_block:
+        return m_block.group(1).upper()
+
+    # 3) Tail anchors: before </answer> / </think>
+    tail_block = text[-tail_window:]
+    PAT_ANS = re.compile(
+        rf'(?<![A-Za-z])'          # Not preceded by a letter
+        rf'[\(\[\{{（【]?\s*'       # Optional left bracket + whitespace
+        rf'([{letters}])\s*'       # One option letter from `letters`
+        rf'[\)\]\}}）】]?\s*'       # Optional right bracket + whitespace
+        rf'</\s*answer\s*>',       # Closing </answer> tag
+        re.IGNORECASE,
+    )
+    PAT_THINK = re.compile(
+        rf'(?<![A-Za-z])'
+        rf'[\(\[\{{（【]?\s*'
+        rf'([{letters}])\s*'
+        rf'[\)\]\}}）】]?\s*'
+        rf'</\s*think\s*>',
+        re.IGNORECASE,
+    )
+    for pat in (PAT_ANS, PAT_THINK):
+        m = pat.search(tail_block)
+        if m:
+            return m.group(1).upper()
+
+    # Helpers for steps 4 & 5
+    # Punctuation treated as tight boundary after a token (EN + CN)
+    _PUNC_TIGHT = r"\.,:;!?\)\]】》」』，。；、：）】》」』"
+    # Lines that start with "option A:" / "选项 A:" style prefixes
+    OPTION_LINE_PREFIX = re.compile(
+        r'^(?:[*_>\-\s]*)(?:option|选项)\s+[A-J]\s*[:：]',
+        re.IGNORECASE,
+    )
+    # Lines whose content is a single option letter (with optional markdown/brackets)
+    MD_SINGLE = re.compile(
+        r'^\s*[*_`>（）\[\]【】\(\)]*\s*([A-Fa-f])\s*[*_`（）\[\]【】\(\)]*\s*$'
+    )
+    # Lines starting with "A. ...", "B) ...", etc.
+    LINE_START_LABELED = re.compile(
+        r'^\s*([A-F])\s*[\.．:：\)\]】、-]\s+',
+        re.IGNORECASE,
+    )
+    # Inline standalone uppercase token (not part of a word), e.g. the A in "answer A."
+    TOKEN_INLINE = re.compile(
+        rf'(?<![A-Za-z])'          # Not preceded by a letter
+        rf'[*_`（\[\{{\(]*\s*'     # optional prefix symbols + whitespace
+        rf'([A-F])\s*'             # Capture a single uppercase letter A–F
+        rf'[*_`）\]\}})]*'         # Optional suffix symbols
+        rf'(?=$|[\s{_PUNC_TIGHT}])'  # Followed by end, whitespace, or tight punctuation
+    )
+
+    def _pick_from_lines(lines):
+        for line in reversed([ln.strip() for ln in lines if ln.strip()]):
+            if OPTION_LINE_PREFIX.search(line):
+                continue
+            m = MD_SINGLE.fullmatch(line)
+            if m:
+                return m.group(1).upper()
+            m = LINE_START_LABELED.match(line)
+            if m:
+                return m.group(1).upper()
+            tokens = [t.upper() for t in TOKEN_INLINE.findall(line)]
+            if tokens:
+                uniq = sorted(set(tokens))
+                if len(uniq) == 1:
+                    return uniq[0]
+        return None
+
+    # 4) Last-lines after think-tail
+    if re.search(r'</\s*think\s*>', text, re.IGNORECASE):
+        tail_segment = text[list(re.finditer(r'</\s*think\s*>', text, re.IGNORECASE))[-1].end():].strip()
+    elif re.search(r'<\s*think\s*>', text, re.IGNORECASE):
+        tail_segment = text[list(re.finditer(r'<\s*think\s*>', text, re.IGNORECASE))[-1].end():].strip()
+    else:
+        tail_segment = text
+
+    pick = _pick_from_lines(tail_segment.splitlines()[-tail_lines:])
+    if pick:
+        return pick
+
+    # 5) Last-lines in global tail window
+    pick = _pick_from_lines(text[-tail_window:].splitlines()[-tail_lines:])
+    if pick:
+        return pick
+
+    # 6) Phrase-style conclusion in tail window (last match)
+    PHRASE_AFTER = re.compile(
+        # Prefix phrases like "final answer", "the answer is", "答案", "我选", etc.
+        rf'(?i)(?:final\s*answer|the\s*answer\s*is|answer(?:\s*is)?|correct\s*answer|'
+        rf'答案|最终答案|结论|所以|因此|我选(?:择)?|选择|选)'
+        rf'\s*[:：>＝=]?\s*'         # optional separator (:, ：, >, ＝, =)
+        rf'[\(\[\{{（【]?\s*'        # optional left bracket
+        rf'([{letters}])'           # option letter from `letters`
+        rf'\s*[\)\]\}}）】]?'        # optional right bracket
+        rf'(?:\b|[.)、。])'          # followed by boundary / end punctuation
+    )
+
+    m = PHRASE_AFTER.search(text)
+    if m:
+        return m.group(1).upper()
+
+    # 7) Global fallback: uppercase-only & unique (skip enumerations)
+    cleaned_lines = []
+    for ln in text.splitlines():
+        if OPTION_LINE_PREFIX.search(ln):
+            continue
+        cleaned_lines.append(ln)
+    cleaned = "\n".join(cleaned_lines)
+
+    TOKEN_UPPER_GLOBAL = re.compile(
+        # Standalone option letter (not part of a word)
+        rf'(?<![A-Za-z])'          # left side not a letter
+        rf'[\(\[\{{（【]?\s*'       # optional left bracket + spaces
+        rf'([{letters}])\s*'       # one letter from `letters`
+        rf'[\)\]\}}）】]?'          # optional right bracket
+        rf'(?![A-Za-z])'           # right side not a letter
+    )
+    tokens = TOKEN_UPPER_GLOBAL.findall(cleaned)
+
+    uniq = sorted(set(tokens))
+    if len(uniq) == 1:
+        return uniq[0]
+
+    return False
+
+
+def _after_think(text: str) -> str:
+    m_end = list(re.finditer(r'</\s*think\s*>', text, flags=re.IGNORECASE))
+    if m_end:
+        return text[m_end[-1].end():].strip()
+    m_start = list(re.finditer(r'<\s*think\s*>', text, flags=re.IGNORECASE))
+    if m_start:
+        return text[m_start[-1].end():].strip()
+    return text
+
+
+def _last_number(s: str):
+    nums = re.findall(_NUM_RE, s)
+    if nums:
+        return float(nums[-1])
+    return None
+
+
+def build_word2num(max_n: int = 99, lang: str = "en"):
+    mapping = {}
+    for i in range(0, max_n + 1):
+        word = num2words(i, lang=lang)
+        mapping[word] = i
+    return mapping
+
+
+WORD2NUM = build_word2num(20)
+WORD_NUMBER_PATTERN = re.compile(
+    r"\b(" + "|".join(re.escape(w) for w in WORD2NUM.keys()) + r")\b",
+    flags=re.IGNORECASE,
+)
+
+
+def normalize_number_words(text: str) -> str:
+    """
+    Replace all recognizable English number phrases in `text`
+    with their Arabic numeral strings.
+    """
+    def _repl(m: re.Match) -> str:
+        key = m.group(1).lower()
+        val = WORD2NUM.get(key)
+        # Fallback: if not found in WORD2NUM, keep original text
+        return str(val) if val is not None else m.group(0)
+
+    return WORD_NUMBER_PATTERN.sub(_repl, text)
+
+
+# Matching func for NA
+def can_match_na(pred):
+    try:
+        if isinstance(pred, list):
+            candidates = [str(pred[0])] if pred else []
+        elif isinstance(pred, str) and pred.strip().startswith('[') and pred.strip().endswith(']'):
+            seq = ast.literal_eval(pred)  # safer than eval
+            candidates = [str(seq[0])] if isinstance(seq, list) and seq else [pred]
+        else:
+            candidates = [str(pred)]
+
+        for raw in candidates:
+            text = ZW_RE.sub('', raw.strip())
+            text = normalize_number_words(text)
+
+            # 1) <answer> ... </answer> numeric
+            m = TAGGED_NUMERIC_ANSWER.search(text)
+            if m:
+                try:
+                    return float(m.group(1))
+                except Exception:
+                    pass
+
+            # 2) after </think>: use *last* number in the tail
+            tail = _after_think(text)
+            v = _last_number(tail)
+            if v is not None:
+                return v
+
+            # 3) global fallback:
+            #    - if only one unique number -> that one
+            #    - else last number in full text
+            nums = re.findall(_NUM_RE, text)
+            if nums:
+                uniq = sorted(set(nums))
+                if len(uniq) == 1:
+                    return float(uniq[0])
+                return float(nums[-1])
+
+        return None
+    except Exception:
+        return None
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/tools/files.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/tools/files.py
new file mode 100644
index 0000000000000000000000000000000000000000..49dd48651745f7beadbb2ade428de3794594cf6b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/tools/files.py
@@ -0,0 +1,57 @@
+from vlmeval.smp.file import get_intermediate_file_path
+
+
+def _judge_tag_from_mode_and_model(judge_mode: str | None, judge_model: str | None) -> str:
+    """
+    Map (judge_mode, judge_model) to a judge_tag string used in filenames.
+
+    Returns examples:
+        - 'extract_matching'
+        - 'llm_gpt-4o' / 'llm_matching'
+    """
+    if judge_mode == 'llm':
+        return f'llm_{judge_model}' if judge_model else 'llm_matching'
+    return 'extract_matching'
+
+
+def get_judge_tag_from_score_fn(score_fn) -> str:
+    """
+    Infer judge_tag from attributes attached to score_fn.
+
+    This relies on _build_score_fn setting:
+        score_fn.judge_mode
+        score_fn.judge_model
+    """
+    judge_mode = getattr(score_fn, 'judge_mode', 'rule')
+    judge_model = getattr(score_fn, 'judge_model', None)
+    return _judge_tag_from_mode_and_model(judge_mode, judge_model)
+
+
+def build_eval_paths(eval_file: str, judge_tag: str):
+    """
+    Build unified evaluation-related file paths from eval_file and judge_tag.
+
+    It returns:
+        - result_file: *_result.pkl
+        - xlsx_path  : *_{judge_tag}.xlsx
+        - acc_path   : *_acc.{EVAL_FORMAT or default csv}
+    """
+    result_file = get_intermediate_file_path(
+        eval_file,
+        suffix='_result',
+        target_format='pkl'
+    )
+
+    xlsx_path = get_intermediate_file_path(
+        eval_file,
+        suffix=f'_{judge_tag}',
+        target_format='xlsx'
+    )
+
+    acc_path = get_intermediate_file_path(
+        eval_file,
+        suffix=f'_{judge_tag}_acc'
+        # target_format=None -> resolved via suffix '_acc' -> get_eval_file_format()
+    )
+
+    return result_file, xlsx_path, acc_path
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/tools/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/tools/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..bdfa0c40a5e94898e48967f407dffdb1cf9968f1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/spatial_bench/tools/utils.py
@@ -0,0 +1,503 @@
+import ast
+import json
+import re
+import string
+
+import numpy as np
+import pandas as pd
+
+from vlmeval.smp.log import get_logger
+
+logger = get_logger(__name__)
+
+
+# ---------------------------------------------------------------------
+# From mcq items to build choices texts
+# ---------------------------------------------------------------------
+def _clean_text(x) -> str:
+    """Cast to str, collapse whitespace, strip."""
+    s = str(x)
+    s = re.sub(r'\s+', ' ', s)
+    return s.strip()
+
+
+def _nonempty(v) -> bool:
+    """Return True if v is not None/NaN/empty (after strip for scalars)."""
+    if v is None:
+        return False
+    if isinstance(v, float) and pd.isna(v):
+        return False
+    if isinstance(v, (list, tuple, set, dict)):
+        return len(v) > 0
+    return str(v).strip() != ""
+
+
+# Only whitespace + punctuation (no letters / digits / CJK)
+_ONLY_PUNCT_WS_RE = re.compile(r'^[\s\.\,\;\:\!\?\-\_/\\\|\(\)\[\]\{\}【】（）·・、—]+$')
+
+
+def _normalize_choice_body(raw) -> str:
+    """
+    Normalize choice text:
+      - None / NaN -> ''
+      - Collapse whitespace
+      - If only whitespace/punctuation -> ''
+      - Keep '<image>' etc. as-is
+    """
+    if raw is None or (isinstance(raw, float) and pd.isna(raw)):
+        return ''
+    txt = _clean_text(raw)
+    if not txt or _ONLY_PUNCT_WS_RE.fullmatch(txt):
+        return ''
+    return txt
+
+
+def _parse_candidates(val, max_letter: str = 'F'):
+    """
+    Parse 'options' / 'candidates' into a list of cleaned strings.
+    """
+    # 1) already a list
+    if isinstance(val, list):
+        return [_clean_text(x) for x in val]
+
+    # 2) string cases
+    if isinstance(val, str):
+        s = val.strip()
+        if not s:
+            return None
+
+        # 2a) try stringified Python list
+        try:
+            parsed = ast.literal_eval(s)
+            if isinstance(parsed, list):
+                return [_clean_text(x) for x in parsed]
+        except Exception:
+            # If parsing fails, fall back to extracting choices from the string.
+            pass
+
+        # 2b) fallback: treat as a mini "options block"
+        mapping = extract_choices_from_question(s, max_letter=max_letter)
+        if mapping:
+            out = []
+            for ch in string.ascii_uppercase:
+                if ch in mapping:
+                    out.append(_clean_text(mapping[ch]))
+                else:
+                    if ch == 'A':
+                        continue
+                    break
+            return out or None
+
+        return None
+
+    # 3) other types -> not supported
+    return None
+
+
+def _letters_upto(max_letter: str = 'F', n: int | None = None):
+    """
+    Letter set helper.
+
+    If n is given and > 0: ['A', ..., up to n].
+    Else: ['A', ..., max_letter].
+    """
+    max_letter = max_letter.upper()
+    all_letters = list(string.ascii_uppercase)
+    if n is not None and isinstance(n, int) and n > 0:
+        return all_letters[:min(n, 26)]
+    end_idx = all_letters.index(max_letter) + 1
+    return all_letters[:end_idx]
+
+
+# Extract choices from explicit columns (A/B/C/...)
+def _extract_from_columns(item, letter_set=('A', 'B', 'C', 'D', 'E', 'F')):
+    """
+    Read choices from columns A–F (or extended):
+
+      - column exists & nonempty -> normalized text
+      - column exists & empty    -> ''
+      - column missing           -> no key
+    """
+    return {
+        ch: (_normalize_choice_body(item[ch]) if _nonempty(item[ch]) else '')
+        for ch in letter_set
+        if ch in item
+    }
+
+
+# Extract choices from question text
+def _slice_by_markers(text: str, markers):
+    """
+    Given label markers (letter, end_idx, start_idx), slice text into segments:
+
+      [end_i : start_{i+1})  as body for letter_i,
+      last marker goes to end of text.
+    """
+    out = {}
+    markers = sorted(markers, key=lambda x: x[2])
+    for i, (ch, end_label, start_pos) in enumerate(markers):
+        next_start = markers[i + 1][2] if i + 1 < len(markers) else len(text)
+        raw = text[end_label:next_start]
+        out[ch] = _normalize_choice_body(raw)
+    return out
+
+
+def _contiguous_prefix_len(keys_iterable):
+    """
+    Count how many letters we have consecutively from 'A'.
+
+    {'A','B','C'} -> 3
+    {'B','C'}     -> 0
+    {'A','C'}     -> 1
+    """
+    s = {k.upper() for k in keys_iterable}
+    k = 0
+    for ch in string.ascii_uppercase:
+        if ch in s:
+            k += 1
+        else:
+            break
+    return k
+
+
+def extract_choices_from_question(q: str, max_letter: str = 'F') -> dict:
+    """
+    Heuristically parse choices from question text.
+
+    Tries:
+      - Line-based labels (each option starts a line)
+      - Inline labels ("A. xxx B. yyy C. zzz")
+
+    Returns: {'A': '...', 'B': '...', ...} or {}.
+    """
+    if not isinstance(q, str) or not q.strip():
+        return {}
+    text = q
+
+    # Drop preamble before "Options:" / "选项:"
+    m = re.search(r'(?i)(options?|选项)\s*[:：]?', text)
+    if m:
+        text = text[m.end():]
+
+    letters = ''.join(_letters_upto(max_letter))
+
+    # ---------------- Line-based: e.g. each option on its own line ----------------
+    LINE_LABEL = re.compile(
+        rf'(?mi)^[ \t]*'
+        rf'(?:[*_`>•·\-]+\s*)?'
+        rf'(?:[\(\[\{{（【]\s*)?'
+        rf'([{letters}])'                  # A / B / ...
+        rf'(?:\s*[\)\]\}}）】])?'
+        rf'\s*[\.．:：\)\]】、-]\s*'        # A. / A) / A: ...
+    )
+
+    from_lines: dict[str, str] = {}
+
+    for m in LINE_LABEL.finditer(text):
+        ch = m.group(1).upper()
+
+        line_end = text.find('\n', m.end())
+        if line_end == -1:
+            line_end = len(text)
+        raw = text[m.end():line_end]
+        body = _normalize_choice_body(raw)
+        from_lines[ch] = body
+
+    # ---------------- Inline: e.g. "A. foo  B. bar  C. baz" ----------------
+    INLINE_LABEL = re.compile(
+        rf'(?<![A-Za-z0-9])'
+        rf'([{letters}])'
+        rf'\s*[\.．:：\)\]】、-]\s*'
+        rf'(?![A-Za-z0-9])'
+    )
+    inline_markers = [(m.group(1).upper(), m.end(), m.start())
+                      for m in INLINE_LABEL.finditer(text)]
+    from_inline = _slice_by_markers(text, inline_markers) if inline_markers else {}
+
+    # Pick better candidate
+    ls_k = _contiguous_prefix_len(from_lines.keys())
+    il_k = _contiguous_prefix_len(from_inline.keys())
+
+    if ls_k > il_k:
+        chosen = from_lines
+    elif il_k > ls_k:
+        chosen = from_inline
+    else:
+        if len(from_lines) > len(from_inline):
+            chosen = from_lines
+        elif len(from_inline) > len(from_lines):
+            chosen = from_inline
+        else:
+            chosen = from_lines or from_inline
+
+    return chosen
+
+
+# Top-level: build_choices
+def build_choices(item: dict, max_letter: str = 'F') -> dict:
+    """
+    Build a choice dict for one item (row-like mapping).
+
+    Priority:
+      1) 'options' / 'candidates' (options > candidates)
+      2) Columns A..max_letter
+      3) Parse from question text
+      4) Fallback: {}
+    """
+    # 1) options / candidates
+    seq = None
+    for key in ('options', 'candidates'):
+        if key in item:
+            parsed = _parse_candidates(item[key])
+            if parsed:
+                seq = parsed
+                break
+
+    if seq:
+        letters = _letters_upto(n=len(seq))
+        return {ch: (seq[i] if i < len(seq) else '') for i, ch in enumerate(letters)}
+
+    # 2) A–F (or extended) columns
+    letters = _letters_upto(max_letter=max_letter)
+    from_cols = _extract_from_columns(item, letter_set=letters)
+    if from_cols:
+        return from_cols
+
+    # 3) From question text
+    q = item.get('question')
+    if isinstance(q, str):
+        from_q = extract_choices_from_question(q, max_letter=max_letter)
+        if from_q:
+            return from_q
+
+    # 4) Nothing found
+    return {}
+
+
+# ---------------------------------------------------------------------
+# From spatial items to parse 2d points
+# ---------------------------------------------------------------------
+class Point2DParser:
+    """
+    Generic 2D point parser.
+
+    - Parse model outputs into a set of (x, y) coordinates.
+    - Support JSON / Python literals and text patterns.
+    - First use _json2pts, then fall back to _text2pts.
+    """
+
+    _has_logged_hint = False
+
+    @classmethod
+    def log_hint(cls, task_name: str | None = None):
+        if cls._has_logged_hint:
+            return
+
+        prefix = f'[{task_name}]' if task_name else '[Point2DParser]'
+        msg = (
+            f'{prefix} Using default Point2DParser:\n'
+            '  - expects JSON / Python literal with "point_2d",\n'
+            '    where coordinates may be:\n'
+            '      * pixels (0 ~ W/H),\n'
+            '      * [0, 1] normalized,\n'
+            '      * [0, 1000] normalized (e.g., Qwen3-VL style);\n'
+            '  - falls back to "(x, y)" or "(x0, y0, x1, y1)" patterns in free text.\n'
+            'Use parse(..., output="pixel") for pixel coords (default), or\n'
+            'parse(..., output="norm") for [0, 1] normalized coords.\n'
+        )
+        cls.logger.info(msg)
+        cls._has_logged_hint = True
+
+    @classmethod
+    def parse(cls, text: str, width: int, height: int, output: str = 'pixel') -> np.ndarray:
+        """
+        Main entry.
+
+        Args:
+            text: raw model output.
+            width, height: image size.
+            output: 'pixel' for pixel coords, 'norm' for [0, 1] normalized coords.
+
+        Returns:
+            np.ndarray[N, 2]
+        """
+        if output not in ('pixel', 'norm'):
+            raise ValueError(f'Point2DParser.parse: unsupported output={output}')
+
+        pts = cls._json2pts(text, width, height, output=output)
+        if pts is not None:
+            return pts
+        return cls._text2pts(text, width, height, output=output)
+
+    @classmethod
+    def _json2pts(
+        cls,
+        text: str,
+        width: int = 640,
+        height: int = 480,
+        output: str = 'pixel'
+    ) -> np.ndarray | None:
+        """
+        Parse JSON/Python literals like:
+        [
+            {"point_2d": [x, y], "label": "..."},
+            ...
+        ]
+
+        point_2d / point can be:
+          - [0, 1] normalized
+          - [0, 1000] normalized
+          - pixels
+        """
+        s = cls._strip_md_fence(text).strip()
+
+        obj = None
+        try:
+            obj = json.loads(s)
+        except Exception:
+            pass
+
+        if obj is None:
+            try:
+                obj = ast.literal_eval(s)
+            except Exception:
+                return None
+
+        if isinstance(obj, dict):
+            obj = [obj]
+        if not isinstance(obj, list):
+            return None
+
+        pts_norm = []  # Store uniformly as 0~1 coordinates
+        w = float(width) if width else 1.0
+        h = float(height) if height else 1.0
+
+        for item in obj:
+            if not isinstance(item, dict):
+                continue
+            pt = item.get('point_2d') or item.get('point')
+            if not (isinstance(pt, (list, tuple)) and len(pt) == 2):
+                continue
+
+            x, y = pt
+            try:
+                x = float(x)
+                y = float(y)
+            except Exception:
+                continue
+
+            max_abs = max(abs(x), abs(y))
+
+            # map to [0,1]
+            if 0.0 <= max_abs <= 1.5:
+                x_norm, y_norm = x, y
+            elif 0.0 <= max_abs <= 1000.0:
+                x_norm = x / 1000.0
+                y_norm = y / 1000.0
+            else:
+                # assume pixels
+                x_norm = x / w
+                y_norm = y / h
+
+            pts_norm.append((x_norm, y_norm))
+
+        if not pts_norm:
+            return None
+
+        pts_norm = np.array(pts_norm, dtype=float)
+
+        if output == 'norm':
+            return pts_norm
+
+        # output == 'pixel'
+        x_pix = np.clip(pts_norm[:, 0] * w, 0, w - 1)
+        y_pix = np.clip(pts_norm[:, 1] * h, 0, h - 1)
+        pts_pix = np.stack([x_pix, y_pix], axis=1).round().astype(int)
+        return pts_pix
+
+    @staticmethod
+    def _text2pts(
+        text: str,
+        width: int = 640,
+        height: int = 480,
+        output: str = 'pixel'
+    ) -> np.ndarray:
+        """
+        Parse free-text patterns:
+            (x, y) or (x0, y0, x1, y1)
+        """
+        pattern = r'\(([-+]?\d+\.?\d*(?:,\s*[-+]?\d+\.?\d*)*?)\)'
+        matches = re.findall(pattern, text)
+        pts_norm = []
+        w = float(width) if width else 1.0
+        h = float(height) if height else 1.0
+
+        for match in matches:
+            nums = [float(num) for num in match.split(',')]
+            max_abs = max(abs(v) for v in nums)
+            is_norm = (0.0 <= max_abs <= 1.5)
+
+            if len(nums) == 2:
+                x, y = nums
+                if is_norm:
+                    x_norm, y_norm = x, y
+                else:
+                    x_norm = x / w
+                    y_norm = y / h
+                pts_norm.append((x_norm, y_norm))
+
+            elif len(nums) == 4:
+                x0, y0, x1, y1 = nums
+                if is_norm:
+                    x0 *= w
+                    y0 *= h
+                    x1 *= w
+                    y1 *= h
+
+                x0, y0, x1, y1 = map(float, (x0, y0, x1, y1))
+                if x1 < x0:
+                    x0, x1 = x1, x0
+                if y1 < y0:
+                    y0, y1 = y1, y0
+
+                x0_i, y0_i, x1_i, y1_i = map(int, map(round, (x0, y0, x1, y1)))
+                h_box = max(0, y1_i - y0_i)
+                w_box = max(0, x1_i - x0_i)
+                if h_box > 0 and w_box > 0:
+                    yy, xx = np.where(np.ones((h_box, w_box), dtype=np.uint8))
+                    x_pix = xx + x0_i
+                    y_pix = yy + y0_i
+                    x_norm = x_pix / w
+                    y_norm = y_pix / h
+                    pts_norm.extend(zip(x_norm, y_norm))
+
+        if not pts_norm:
+            return np.empty((0, 2), dtype=float if output == 'norm' else int)
+
+        pts_norm = np.array(pts_norm, dtype=float)
+
+        if output == 'norm':
+            return pts_norm
+
+        # output == 'pixel'
+        x_pix = np.clip(pts_norm[:, 0] * w, 0, w - 1)
+        y_pix = np.clip(pts_norm[:, 1] * h, 0, h - 1)
+        pts_pix = np.stack([x_pix, y_pix], axis=1).round().astype(int)
+        return pts_pix
+
+    @staticmethod
+    def _strip_md_fence(text: str) -> str:
+        s = text.strip()
+        if not s.startswith('```'):
+            return s
+
+        first_nl = s.find('\n')
+        if first_nl != -1:
+            inner = s[first_nl + 1:]
+        else:
+            inner = s.lstrip('`')
+
+        inner = inner.strip()
+        if inner.endswith('```'):
+            inner = inner[:-3]
+        return inner.strip()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ssi_bench/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ssi_bench/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..aa60b6d2f39735cf93a7fe80373afbd030a711e6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ssi_bench/__init__.py
@@ -0,0 +1 @@
+# This file is intentionally left empty to make this directory a Python package.
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ssi_bench/prompts.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ssi_bench/prompts.py
new file mode 100644
index 0000000000000000000000000000000000000000..06bda464699a0a62b99f0172db7a55bec8dde474
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/ssi_bench/prompts.py
@@ -0,0 +1,589 @@
+from __future__ import annotations
+
+GROUND_HEIGHT_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer spatial relationships from images. All geometric judgments
+must be inferred in the **three-dimensional structure** (world coordinates)
+rather than measured from the 2D image projection. Your goal is to **rank
+structural members based on the heights of their centroids relative to the
+ground plane**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Member Images** (labeled 1, 2, 3, 4). In each annotated image, the
+visible portion of the corresponding member is highlighted as a **{color}
+region**. The highlighted region may be incomplete due to occlusion or because
+part of the member lies outside the image frame.
+
+When estimating the centroid, treat each member as a **complete structural
+unit**. If only part of a member is visible, infer the centroid based on the
+**smallest complete member unit that the visible region can reasonably
+represent**.
+
+Sort the members according to their centroid heights in **ascending order**,
+from **lowest to highest**.
+
+If two or more members have equal or indistinguishably similar centroid heights,
+the member with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of member labels from lowest to highest. Do not include any other text,
+reasoning, or explanation.
+
+**Example**: If you determine the order is Member 3 (lowest), Member 4,
+Member 1, Member 2 (highest), your output must be:
+[3, 4, 1, 2]
+
+Now please provide your answer in the requested format.
+"""
+
+
+GROUND_ANGLE_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer geometric relationships from images. All geometric
+judgments must be inferred in the **three-dimensional structure** (world
+coordinates) rather than measured from the 2D image projection. Your goal is to
+**rank structural members according to the angles between their main directions
+and the ground plane**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Member Images** (labeled 1, 2, 3, 4). In each annotated image, the
+visible portion of the corresponding member is highlighted as a **{color}
+region**. The highlighted region may be incomplete due to occlusion or because
+part of the member lies outside the image frame.
+
+When estimating the angle, treat each member as a **complete structural unit**.
+If only part of a member is visible, infer the main direction based on the
+**smallest complete member unit that the visible region can reasonably
+represent**.
+
+Sort the members according to the angle between their main direction and the
+ground plane in **ascending order**, from **smallest angle to largest angle**.
+A member **parallel to the ground plane** has the **smallest angle**, while a
+member **perpendicular to the ground plane** has the **largest angle**.
+
+If two or more members have equal or indistinguishably similar angles, the
+member with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of member labels from smallest angle to largest angle. Do not include any other
+text, reasoning, or explanation.
+
+**Example**: If you determine the order is Member 2 (smallest angle),
+Member 1, Member 4, Member 3 (largest angle), your output must be:
+[2, 1, 4, 3]
+
+Now please provide your answer in the requested format.
+"""
+
+
+DIMENSION_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer geometric properties from images. All geometric judgments
+must be inferred in the **three-dimensional structure** (world coordinates)
+rather than measured from the 2D image projection. Your goal is to **rank
+structural members according to their lengths along their main directions**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Member Images** (labeled 1, 2, 3, 4). In each annotated image, the
+visible portion of the corresponding member is highlighted as a **{color}
+region**. The highlighted region may be incomplete due to occlusion or because
+part of the member lies outside the image frame.
+
+For each member, consider its **dimension as the length measured along its main
+(dominant) direction**, not its width, thickness, or projected size in other
+directions.
+
+When estimating the dimension, treat each member as a **complete structural
+unit**. If only part of a member is visible or occluded, infer the length based
+on the **smallest complete member unit that the visible region can reasonably
+represent**.
+
+Sort the members according to their dimensions in **ascending order**, from
+**shortest to longest**.
+
+If two or more members have equal or indistinguishably similar dimensions, the
+member with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of member labels from shortest to longest. Do not include any other text,
+reasoning, or explanation.
+
+**Example**: If you determine the order is Member 4 (shortest), Member 1,
+Member 3, Member 2 (longest), your output must be:
+[4, 1, 3, 2]
+
+Now please provide your answer in the requested format.
+"""
+
+
+RELATIVE_DISTANCE_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer spatial relationships from images. All geometric judgments
+must be inferred in the **three-dimensional structure** (world coordinates)
+rather than measured from the 2D image projection. Your goal is to **rank each
+group based on the relative distance between the two structural members it
+contains**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Group Images** (labeled 1, 2, 3). Each annotated group contains
+**two structural members**, with the visible portions of both members
+highlighted as **{color} regions**. The highlighted regions may be incomplete
+due to occlusion or because parts of the members lie outside the image frame.
+
+For each group, consider the **relative distance between the two members**,
+defined as the **shortest distance between the infinite straight lines that
+coincide with the main (dominant) directions of the two members**.
+
+If the two lines **intersect**, their relative distance is defined as **0**.
+
+Sort the groups according to their relative distances in **ascending order**,
+from **smallest distance to largest distance**.
+
+If two or more groups have equal or indistinguishably similar distances, the
+group with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of group labels from smallest distance to largest distance. Do not include any
+other text, reasoning, or explanation.
+
+**Example**: If you determine the order is Group 3 (smallest distance),
+Group 1, Group 2 (largest distance), your output must be:
+[3, 1, 2]
+
+Now please provide your answer in the requested format.
+"""
+
+AREA_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer geometric properties from images. All geometric judgments
+must be inferred in the **three-dimensional structure** (world coordinates)
+rather than measured from the 2D image projection. Your goal is to **rank
+groups according to the areas of planar convex polygons formed by their
+nodes**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Group Images** (labeled 1, 2, 3). Each annotated group contains a
+**set of nodes**, highlighted as **{color} points**, which together define a
+planar convex polygon.
+
+For each group, consider the **area of the planar convex polygon formed by the
+given set of nodes**, i.e., the **area of the convex hull of the nodes**.
+
+Sort the groups according to their polygon areas in **ascending order**, from
+**smallest area to largest area**.
+
+If two or more groups have equal or indistinguishably similar areas, the group
+with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of group labels from smallest area to largest area. Do not include any other
+text, reasoning, or explanation.
+
+**Example**: If you determine the order is Group 2 (smallest area), Group 1,
+Group 3 (largest area), your output must be:
+[2, 1, 3]
+
+Now please provide your answer in the requested format.
+"""
+
+
+VOLUME_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer geometric properties from images. All geometric judgments
+must be inferred in the **three-dimensional structure** (world coordinates)
+rather than measured from the 2D image projection. Your goal is to **rank
+groups according to the volumes of convex polyhedra formed by their nodes**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Group Images** (labeled 1, 2, 3). Each annotated group contains a
+**set of nodes**, highlighted as **{color} points**, which together define a
+three-dimensional shape.
+
+For each group, consider the **volume of the convex polyhedron formed by the
+given set of nodes**, i.e., the **volume of the 3D convex hull of the nodes**.
+
+Sort the groups according to their polyhedron volumes in **ascending order**,
+from **smallest volume to largest volume**.
+
+If two or more groups have equal or indistinguishably similar volumes, the
+group with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of group labels from smallest volume to largest volume. Do not include any
+other text, reasoning, or explanation.
+
+**Example**: If you determine the order is Group 1 (smallest volume), Group 3,
+Group 2 (largest volume), your output must be:
+[1, 3, 2]
+
+Now please provide your answer in the requested format.
+"""
+
+
+HOP_DISTANCE_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+connectivity and infer **topological relationships** between structural
+members. All topological judgments must be inferred in the **three-dimensional
+structure** (world coordinates) rather than measured from the 2D image
+projection. Your goal is to **rank each group based on the topological
+distance (number of hops) between the two structural members it contains**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Group Images** (labeled 1, 2, 3). Each annotated group contains
+**two structural members**, with the visible portions of both members
+highlighted as **{color} regions**. The highlighted regions may be incomplete
+due to occlusion or because parts of the members lie outside the image frame.
+
+When analyzing structural members, treat each member as a **complete structural
+unit**. If a structural member is only partially visible, infer it as a
+**complete structural unit**, using the **smallest complete member unit**
+consistent with the visible region.
+
+Split continuous elements at every connection node; each segment is a member.
+Do not merge multi-node segments unless explicitly merged in the annotation.
+
+For each group, determine the **topological distance (hops)** between the two
+members, defined as:
+
+- A **hop** is one direct connection between two structural members (e.g.,
+  physical joint, intersection, or direct attachment).
+- If the two members are **directly connected**, their topological distance is
+  **1 hop**.
+- If the two members are **not directly connected**, the topological distance
+  is the **minimum number of intermediate members** required to form a
+  continuous connection path between them.
+
+Sort the groups according to their topological distances in **ascending
+order**, from **smallest number of hops to largest number of hops**.
+
+If two or more groups have equal or indistinguishably similar topological
+distances, the group with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of group labels from smallest topological distance to largest. Do not include
+any other text, reasoning, or explanation.
+
+**Example**: If you determine the order is Group 2 (0 hops), Group 1 (1 hop),
+Group 3 (2 hops), your output must be:
+[2, 1, 3]
+
+Now please provide your answer in the requested format.
+"""
+
+
+CYCLE_LENGTH_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+connectivity and infer **cyclic topological relationships** among structural
+members. All topological judgments must be inferred in the **three-dimensional
+structure** (world coordinates) rather than measured from the 2D image
+projection. Your goal is to **rank each group based on the minimum number of
+edges in a cycle that includes all given members**.
+
+**Your Task**
+
+You will be provided with an **Original Structure Image** and a set of shuffled
+**Annotated Group Images** (labeled 1, 2, 3). Each annotated group contains a
+**set of structural members**, with the visible portions of all members
+highlighted as **{color} regions**. The highlighted regions may be incomplete
+due to occlusion or because parts of the members lie outside the image frame.
+
+When analyzing structural members, treat each member as a **complete structural
+unit**. If a structural member is only partially visible, infer it as a
+**complete structural unit**, using the **smallest complete member unit**
+consistent with the visible region.
+
+Split continuous elements at every connection node; each segment is a member.
+Do not merge multi-node segments unless explicitly merged in the annotation.
+
+For each group, determine the **minimum cycle length**, defined as:
+
+- A **cycle** is a closed topological path formed by connected structural
+  members, where the start and end member coincide.
+- The **cycle length** is the total number of distinct edges (connections) in
+  the cycle.
+- The cycle must **include all members in the group** (each member must lie on
+  the cycle).
+- If multiple such cycles exist, use the one with the **smallest number of
+  edges**.
+- If no cycle exists that includes all given members, treat the cycle length as
+  **infinite**.
+
+Sort the groups according to their minimum cycle lengths in **ascending
+order**, from **smallest number of edges to largest number of edges**.
+
+If two or more groups have equal or indistinguishably similar cycle lengths,
+the group with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of group labels from smallest cycle length to largest. Do not include any other
+text, reasoning, or explanation.
+
+**Example**: If you determine the order is Group 1 (3 edges), Group 3 (4
+edges), Group 2 (no valid cycle), your output must be:
+[1, 3, 2]
+
+Now please provide your answer in the requested format.
+"""
+
+
+MV_RELATIVE_DISTANCE_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+components and infer spatial relationships from images. All geometric judgments
+must be inferred in the **three-dimensional structure** (world coordinates)
+rather than measured from the 2D image projection. Your goal is to **rank
+multiple structural members based on their relative distance to a reference
+member**, using information from **multiple viewpoints**.
+
+**Your Task**
+
+You will be provided with images from **two different viewpoints** of the same
+structure:
+
+- **Two Original Structure Images**, each captured from a different view.
+- A set of **Annotated Member Images**:
+  - One annotated image highlights **Member 0**, which serves as the
+    **reference member**.
+  - Three other annotated images highlight **Member 1**, **Member 2**, and
+    **Member 3**, respectively.
+
+In each annotated image, the visible portion of the highlighted member is
+marked as a **{color} region**. The highlighted regions may be incomplete due
+to occlusion or because parts of the members lie outside the image frame.
+
+**Distance Definition**
+
+For each of **Member 1, Member 2, and Member 3**, consider its **relative
+distance to Member 0**, defined as the **shortest distance between the infinite
+straight line coinciding with the main (dominant) direction of Member 0 and the
+infinite straight line coinciding with the main (dominant) direction of the
+other member**.
+
+If the two lines **intersect**, their relative distance is defined as **0**.
+
+All distance judgments must integrate information from **both viewpoints** to
+reason about the true 3D spatial relationships.
+
+**Sorting Requirement**
+
+Sort **Member 1, Member 2, and Member 3** according to their relative
+distances to **Member 0**, in **ascending order**, from **smallest distance to
+largest distance**.
+
+If two or more members have equal or indistinguishably similar distances, the
+member with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of member labels from smallest distance to largest distance. Do not include any
+other text, reasoning, or explanation.
+
+**Example**: If you determine that Member 2 is closest to Member 0, followed by
+Member 1, and then Member 3 (farthest), your output must be:
+[2, 1, 3]
+
+Now please provide your answer in the requested format.
+"""
+
+
+MV_HOP_DISTANCE_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+connectivity and infer **topological relationships** between structural
+members. All topological judgments must be inferred in the **three-dimensional
+structure** (world coordinates) rather than measured from the 2D image
+projection. Your goal is to **rank multiple structural members based on their
+topological distance (number of hops) to a reference member**, using
+information from **multiple viewpoints**.
+
+**Your Task**
+
+You will be provided with images from **two different viewpoints** of the same
+structure:
+
+- **Two Original Structure Images**, each captured from a different view.
+- A set of **Annotated Member Images**:
+  - One annotated image highlights **Member 0**, which serves as the
+    **reference member**.
+  - Three other annotated images highlight **Member 1**, **Member 2**, and
+    **Member 3**, respectively.
+
+In each annotated image, the visible portion of the highlighted member is
+marked as a **{color} region**. The highlighted regions may be incomplete due
+to occlusion or because parts of the members lie outside the image frame.
+
+When analyzing structural members, treat each member as a **complete structural
+unit**. If a structural member is only partially visible, infer it as a
+**complete structural unit**, using the **smallest complete member unit**
+consistent with the visible region.
+
+Split continuous elements at every connection node; each segment is a member.
+Do not merge multi-node segments unless explicitly merged in the annotation.
+
+**Topological Distance Definition**
+
+For each of **Member 1, Member 2, and Member 3**, determine its **topological
+distance (number of hops)** to **Member 0**, defined as follows:
+
+- A **hop** is one direct connection between two structural members (e.g.,
+  physical joint, intersection, or direct attachment).
+- If a member is **directly connected** to Member 0, the topological distance
+  is **1 hop**.
+- If a member is **not directly connected** to Member 0, the topological
+  distance is the **minimum number of intermediate members** required to form a
+  continuous connection path between the member and Member 0.
+
+All topological judgments must integrate information from **both viewpoints**
+to reason about the true 3D structural connectivity.
+
+**Sorting Requirement**
+
+Sort **Member 1, Member 2, and Member 3** according to their topological
+distances to **Member 0**, in **ascending order**, from **smallest number of
+hops to largest number of hops**.
+
+If two or more members have equal or indistinguishably similar topological
+distances, the member with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of member labels from smallest topological distance to largest. Do not include
+any other text, reasoning, or explanation.
+
+**Example**: If you determine that Member 1 is directly connected to Member 0
+(1 hop), Member 3 is connected via one intermediate member (2 hops), and Member
+2 is farther away (3 hops), your output must be:
+[1, 3, 2]
+
+Now please provide your answer in the requested format.
+"""
+
+
+MV_CYCLE_LENGTH_PROMPT_TEMPLATE = """\
+You are a capable vision-based reasoning agent designed to analyze structural
+connectivity and infer **cyclic topological relationships** among structural
+members. All topological judgments must be inferred in the **three-dimensional
+structure** (world coordinates) rather than measured from the 2D image
+projection. Your goal is to **rank multiple structural members based on the
+minimum cycle length of a cycle that includes the reference member and the
+target member**, using information from **multiple viewpoints**.
+
+**Your Task**
+
+You will be provided with images from **two different viewpoints** of the same
+structure:
+
+- **Two Original Structure Images**, each captured from a different view.
+- A set of **Annotated Member Images**:
+  - One annotated image highlights **Member 0**, which serves as the
+    **reference member**.
+  - Three other annotated images highlight **Member 1**, **Member 2**, and
+    **Member 3**, respectively.
+
+In each annotated image, the visible portion of the highlighted member is
+marked as a **{color} region**. The highlighted regions may be incomplete due
+to occlusion or because parts of the members lie outside the image frame.
+
+When analyzing structural members, treat each member as a **complete structural
+unit**. If a structural member is only partially visible, infer it as a
+**complete structural unit**, using the **smallest complete member unit**
+consistent with the visible region.
+
+Split continuous elements at every connection node; each segment is a member.
+Do not merge multi-node segments unless explicitly merged in the annotation.
+
+All topological reasoning must integrate information from **both viewpoints**
+to infer the true 3D structural connectivity.
+
+**Cycle Length Definition**
+
+For each of **Member 1, Member 2, and Member 3**, determine its **minimum cycle
+length with Member 0**, defined as:
+
+- A **cycle** is a closed topological path formed by connected structural
+  members, where the start and end member coincide.
+- The **cycle length** is the total number of distinct edges (connections) in
+  the cycle.
+- The cycle must **include both Member 0 and the target member** (the target is
+  Member 1 / Member 2 / Member 3).
+- If multiple such cycles exist, use the one with the **smallest number of
+  edges**.
+- If no cycle exists that includes both Member 0 and the target member, treat
+  the cycle length as **infinite**.
+
+**Sorting Requirement**
+
+Sort **Member 1, Member 2, and Member 3** according to their minimum cycle
+lengths with **Member 0**, in **ascending order**, from **smallest number of
+edges to largest number of edges**.
+
+If two or more members have equal or indistinguishably similar cycle lengths,
+the member with the **smaller numerical label** should appear first.
+
+**Output Format**
+
+Your response **must be only** a Python list of integers representing the order
+of member labels from smallest cycle length to largest. Do not include any
+other text, reasoning, or explanation.
+
+**Example**: If you determine that the smallest cycle including Member 0 and
+Member 2 has 3 edges, Member 0 and Member 1 has 4 edges, and Member 0 and
+Member 3 has no valid cycle, your output must be:
+[2, 1, 3]
+
+Now please provide your answer in the requested format.
+"""
+
+
+def prompts_by_task_slug(*, multi_view: bool = False) -> dict[str, str]:
+    prompts = {
+        "ground_height": GROUND_HEIGHT_PROMPT_TEMPLATE,
+        "ground_angle": GROUND_ANGLE_PROMPT_TEMPLATE,
+        "dimension": DIMENSION_PROMPT_TEMPLATE,
+        "relative_distance": RELATIVE_DISTANCE_PROMPT_TEMPLATE,
+        "area": AREA_PROMPT_TEMPLATE,
+        "volume": VOLUME_PROMPT_TEMPLATE,
+        "hop_distance": HOP_DISTANCE_PROMPT_TEMPLATE,
+        "cycle_length": CYCLE_LENGTH_PROMPT_TEMPLATE,
+    }
+
+    if multi_view:
+        prompts.update(
+            {
+                "relative_distance": MV_RELATIVE_DISTANCE_PROMPT_TEMPLATE,
+                "hop_distance": MV_HOP_DISTANCE_PROMPT_TEMPLATE,
+                "cycle_length": MV_CYCLE_LENGTH_PROMPT_TEMPLATE,
+            }
+        )
+
+    return prompts
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/v2pbench/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/v2pbench/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/v2pbench/cau_acc.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/v2pbench/cau_acc.py
new file mode 100644
index 0000000000000000000000000000000000000000..6d0c01da7172471a2ecd740fe9593b8175a10588
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/v2pbench/cau_acc.py
@@ -0,0 +1,119 @@
+import json
+import re
+
+import pandas as pd
+
+
+# for mimo-vl
+def remove_think_blocks(text: str) -> str:
+    """
+    Remove all occurrences of <think>...</think> or <think>...</think>
+    (including the tags) from the input text.
+    Handles multiline content and multiple blocks.
+    """
+    # 修复 W605: 使用原始字符串 r'' 来处理反斜杠转义
+    pattern = r'<think>.*?(?:<\/think>|<\\/think>)'
+    return re.sub(pattern, '', text, flags=re.DOTALL)
+
+
+def extract_characters_regex(s):
+    s = s.strip()
+    answer_prefixes = [
+        'The best answer is',
+        'The correct answer is',
+        'The answer is',
+        'The answer',
+        'The best option is',
+        'The correct option is',
+        'Best answer:',
+        'Best option:',
+        'Answer:',
+        'Option:',
+    ]
+    for answer_prefix in answer_prefixes:
+        s = s.replace(answer_prefix, '')
+
+    if len(s.split()) > 10 and not re.search('[ABCD]', s):
+        return ''
+    matches = re.search(r'[ABCD]', s)
+    if matches is None:
+        return ''
+    return matches[0]
+
+
+def xlsx2json(xlsx_file, json_file):
+    df = pd.read_excel(xlsx_file)
+    df.to_json(json_file, orient='records')
+
+
+def calu_acc_main(file_path, txt_file=None):
+    # Load data
+    with open(file_path, 'r', encoding='utf-8') as f_in:
+        data = json.load(f_in)
+
+    durations = [0, 240, 1800, 7200]
+    dim_mapping = {
+        1: "OA", 2: "HA", 3: "OD", 4: "FM", 5: "CR", 6: "PU", 7: "CI",
+        9: "FT", 10: "RT", 12: "AS", 13: "SR", 14: "GC"
+    }
+
+    dim_nums = 16
+    dim_list_sum = [0] * dim_nums
+    dim_list_cor = [0] * dim_nums
+
+    short_cor, short_sum = 0, 0
+    medium_cor, medium_sum = 0, 0
+    long_cor, long_sum = 0, 0
+
+    f = open(txt_file, "w", encoding="utf-8") if txt_file else None
+
+    def log(msg):
+        """Print to both the console and the file"""
+        print(msg)
+        if f:
+            f.write(msg + "\n")
+
+    for line in data:
+        dim = line["dimension"]
+        dim_list_sum[dim - 1] += 1
+
+        if line["duration"] < durations[1]:
+            short_sum += 1
+        elif line["duration"] < durations[2]:
+            medium_sum += 1
+        else:
+            long_sum += 1
+
+        if line["score"] == 1:
+            dim_list_cor[dim - 1] += 1
+            if line["duration"] < durations[1]:
+                short_cor += 1
+            elif line["duration"] < durations[2]:
+                medium_cor += 1
+            else:
+                long_cor += 1
+
+    for index, (dim_cor, dim_sum) in enumerate(zip(dim_list_cor, dim_list_sum)):
+        if index + 1 not in [8, 11, 15, 16]:
+            if dim_sum != 0:
+                log(f"{dim_mapping[index + 1]}: {dim_cor / dim_sum:.3f}")
+            else:
+                log(f"Dimension is zero: {dim_mapping[index + 1]}")
+
+    log("-" * 58)
+    if short_sum != 0:
+        log(f"Short\nCorrect: {short_cor}, Total: {short_sum}, Accuracy: {short_cor / short_sum:.3f}")
+    if medium_sum != 0:
+        log(f"Medium\nCorrect: {medium_cor}, Total: {medium_sum}, Accuracy: {medium_cor / medium_sum:.3f}")
+    if long_sum != 0:
+        log(f"Long\nCorrect: {long_cor}, Total: {long_sum}, Accuracy: {long_cor / long_sum:.3f}")
+
+    log("-" * 58)
+    cor_data = sum(dim_list_cor)
+    all_data = sum(dim_list_sum)
+    log(f"Total Correct: {cor_data}")
+    log(f"Total Success: {all_data}")
+    log(f"Accuracy:      {cor_data / all_data:.3f}")
+
+    if f:
+        f.close()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/cau_acc.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/cau_acc.py
new file mode 100644
index 0000000000000000000000000000000000000000..a2cc0cf3b068349d45ab0f6902d61990249ea66e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/cau_acc.py
@@ -0,0 +1,112 @@
+import json
+from collections import defaultdict
+
+import pandas as pd
+
+
+def xlsx2json(xlsx_file, json_file):
+    df = pd.read_excel(xlsx_file)
+    df.to_json(json_file, orient='records')
+
+
+def calculate_accuracy(data):
+    total_correct = 0
+    total_items = len(data)
+
+    dimension_stats = defaultdict(lambda: {"correct": 0, "total": 0})
+    duration_stats = {
+        "0-60": {"correct": 0, "total": 0},
+        "60-300": {"correct": 0, "total": 0},
+        "300+": {"correct": 0, "total": 0}
+    }
+
+    for item in data:
+        if item.get("answer_scoring") == '1':
+            total_correct += 1
+
+        dimension = item.get("dimension")
+        if dimension:
+            dimension_stats[dimension]["total"] += 1
+            if item.get("answer_scoring") == '1':
+                dimension_stats[dimension]["correct"] += 1
+
+        duration = item.get("duration", 0)
+        if duration <= 60:
+            key = "0-60"
+        elif duration <= 300:
+            key = "60-300"
+        else:
+            key = "300+"
+
+        duration_stats[key]["total"] += 1
+        if item.get("answer_scoring") == '1':
+            duration_stats[key]["correct"] += 1
+
+    overall_accuracy = total_correct / total_items if total_items > 0 else 0
+
+    dimension_accuracy = {}
+    for dimension, stats in dimension_stats.items():
+        dimension_accuracy[dimension] = stats["correct"] / stats["total"] if stats["total"] > 0 else 0
+
+    duration_accuracy = {}
+    for key, stats in duration_stats.items():
+        duration_accuracy[key] = stats["correct"] / stats["total"] if stats["total"] > 0 else 0
+
+    return {
+        "overall_accuracy": overall_accuracy,
+        "dimension_accuracy": dimension_accuracy,
+        "duration_accuracy": duration_accuracy
+    }
+
+
+def format_results(results):
+    formatted_results = {}
+
+    formatted_results["Overall Accuracy"] = f"{results['overall_accuracy']:.3f}"
+
+    formatted_results["Accuracy by Dimension"] = {
+        dimension: f"{accuracy:.3f}" for dimension, accuracy in results["dimension_accuracy"].items()
+    }
+
+    formatted_results["Accuracy by Duration"] = {
+        duration: f"{accuracy:.3f}" for duration, accuracy in results["duration_accuracy"].items()
+    }
+
+    return formatted_results
+
+
+def calu_acc_main(file_path, txt_file):
+
+    # Load data from the provided file path
+    data = json.load(open(file_path, 'r', encoding='utf-8'))
+    for item in data:
+        item["answer_scoring"] = str(item["answer_scoring"])
+
+    results = calculate_accuracy(data)
+    formatted_results = format_results(results)
+
+    print("===== Statistics =====")
+    print("Overall Accuracy:", formatted_results["Overall Accuracy"])
+    print("\nAccuracy by Dimension:")
+    for dimension, accuracy in formatted_results["Accuracy by Dimension"].items():
+        print(f"  {dimension}: {accuracy}")
+    print("\nAccuracy by Duration:")
+    for duration, accuracy in formatted_results["Accuracy by Duration"].items():
+        print(f"  {duration}: {accuracy}")
+
+    print('\n\n')
+
+    with open(txt_file, 'w') as file:
+        file.write("===== Statistics =====\n")
+        file.write(f"Overall Accuracy: {formatted_results['Overall Accuracy']}\n")
+        file.write("\nAccuracy by Dimension:\n")
+
+        for dimension, accuracy in formatted_results["Accuracy by Dimension"].items():
+            file.write(f"  {dimension}: {accuracy}\n")
+
+        file.write("\nAccuracy by Duration:\n")
+
+        for duration, accuracy in formatted_results["Accuracy by Duration"].items():
+            file.write(f"  {duration}: {accuracy}\n")
+
+        file.write('\n\n')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/cau_total.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/cau_total.py
new file mode 100644
index 0000000000000000000000000000000000000000..77b78566416ffbd0e9c9fc354fa3dbbe3ca5157b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/cau_total.py
@@ -0,0 +1,88 @@
+import json
+from collections import defaultdict
+
+
+def load_data(file_path):
+    with open(file_path, 'r', encoding='utf-8') as f:
+        return json.load(f)
+
+
+def calculate_metrics(data_pre, data_recall):
+    total_stats = defaultdict(lambda: {
+        "precision_sum": 0, "precision_count": 0,
+        "recall_sum": 0, "recall_count": 0,
+        "efficiency_sum": 0, "efficiency_count": 0
+    })
+
+    for item in data_pre:
+        for metric in ["Video", "logic", "overall"]:
+            precision = item.get(f"{metric}_precision", '')
+
+            if precision and precision != '':
+                total_stats[metric]["precision_sum"] += precision
+                total_stats[metric]["precision_count"] += 1
+
+    for item in data_recall:
+        for metric in ["Video", "logic", "overall"]:
+            recall = item.get(f"{metric}_recall", '')
+
+            if recall and recall != '':
+                total_stats[metric]["recall_sum"] += recall
+                total_stats[metric]["recall_count"] += 1
+
+    overall_metrics = {}
+    for metric, stats in total_stats.items():
+        precision = stats["precision_sum"] / stats["precision_count"] if stats["precision_count"] > 0 else 0
+        recall = stats["recall_sum"] / stats["recall_count"] if stats["recall_count"] > 0 else 0
+        f1 = (2 * precision * recall) / (precision + recall) if (precision + recall) > 0 else 0
+        overall_metrics[metric] = {
+            "precision": precision,
+            "recall": recall,
+            "f1": f1
+        }
+
+    return {
+        "overall_metrics": overall_metrics
+    }
+
+
+def format_metrics_results(results):
+    def format_dict(d):
+        return {k: f"{v:.3f}" if isinstance(v, (int, float)) else v for k, v in d.items()}
+
+    return {
+        "Overall Metrics": {metric: format_dict(stats) for metric, stats in results["overall_metrics"].items()}
+    }
+
+
+def print_results(formatted_results, txt_file):
+    mapping = {"Video": "Perception", "logic": "Reasoning", "overall": "Overall"}
+    print("===== Metrics Summary =====")
+    print("Overall Metrics:")
+    for metric, stats in formatted_results["Overall Metrics"].items():
+        print(f"  {mapping[metric]}:")
+        for key, value in stats.items():
+            print(f"    {key}: {value}")
+
+        mapping = {"Video": "Perception", "logic": "Reasoning", "overall": "Overall"}
+
+    with open(txt_file, 'w') as file:
+        file.write("===== Metrics Summary =====\n")
+        file.write("Overall Metrics:\n")
+
+        for metric, stats in formatted_results["Overall Metrics"].items():
+            file.write(f"  {mapping[metric]}:\n")
+            for key, value in stats.items():
+                file.write(f"    {key}: {value}\n")
+
+
+def calu_pre_recall(pre_file, recall_file, txt_file):
+    # Load and process data
+    data_pre = load_data(pre_file)
+    data_recall = load_data(recall_file)
+
+    results = calculate_metrics(data_pre, data_recall)
+    formatted_results = format_metrics_results(results)
+
+    # Print results
+    print_results(formatted_results, txt_file)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/eval.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/eval.py
new file mode 100644
index 0000000000000000000000000000000000000000..66f8163cb9ee0dbf28c6420588c88ebf9dd42ee6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/eval.py
@@ -0,0 +1,230 @@
+import ast
+import copy
+import json
+
+
+def read_json(file_path):
+    with open(file_path, 'r', encoding="utf-8") as file:
+        data = json.load(file)
+    return data
+
+
+def read_jsonl(file_path):
+    data = []
+    with open(file_path, 'r', encoding="utf-8") as file:
+        for line in file:
+            data.append(json.loads(line.strip()))
+    return data
+
+
+def save_json(data, file_path, indent=4):
+    with open(file_path, 'w', encoding="utf-8") as file:
+        json.dump(data, file, ensure_ascii=False, indent=indent)
+
+
+def save_jsonl(data, file_path):
+    with open(file_path, 'w', encoding="utf-8") as file:
+        for item in data:
+            json.dump(item, file, ensure_ascii=False)
+            file.write("\n")
+
+
+def calculate_time_iou(interval1, interval2):
+
+    start1, end1 = interval1
+    start2, end2 = interval2
+
+    intersection_start = max(start1, start2)
+    intersection_end = min(end1, end2)
+    intersection = max(0, intersection_end - intersection_start)
+
+    union_start = min(start1, start2)
+    union_end = max(end1, end2)
+    union = union_end - union_start
+
+    if union == 0:
+        return 0
+    iou = intersection / union
+    return iou
+
+
+def is_valid_time_interval(interval_str):
+
+    try:
+        interval = ast.literal_eval(interval_str)
+        if isinstance(interval, list) and len(interval) == 2:
+            if all(isinstance(x, (int, float)) for x in interval):
+                return True
+        return False
+    except (ValueError, SyntaxError):
+        return False
+
+
+def is_valid_space_interval(s):
+    if not (s.startswith('[') and s.endswith(']')):
+        return False
+    content = s[1:-1]
+    parts = content.split(',')
+    if len(parts) != 4:
+        return False
+    for part in parts:
+        try:
+            int(part.strip())
+        except ValueError:
+            return False
+    return True
+
+
+def string_to_list(s):
+    content = s[1:-1]
+    return [int(part.strip()) for part in content.split(',')]
+
+
+def extract_json_between_backticks(s):
+    # pattern = r'```json\n(.*?)```'
+    # match = re.search(pattern, s, re.DOTALL)
+    # if not match:
+    #     raise ValueError("No JSON content wrapped by ``` was found.")
+    # json_str = match.group(1).strip()
+    json_str = s
+
+    try:
+        json.loads(json_str)
+        return json_str
+    except json.JSONDecodeError as e:
+        raise ValueError(f"Extracted content is not valid JSON: {e}")
+
+
+def calculate_recall(json_object):
+
+    stats = {
+        "Video Description Steps": {"Matched": 0, "Unmatched": 0},
+        "Logical Inference Steps": {"Matched": 0, "Unmatched": 0},
+        "Background Review Steps": {"Matched": 0, "Unmatched": 0}
+    }
+    for item in json_object:
+        step_type = item["step_type"]
+        judgement = item["judgment"]
+        stats[step_type][judgement] += 1
+
+    return stats
+
+
+def calculate_space_iou(box1, box2):
+
+    x1_1, y1_1, x2_1, y2_1 = box1
+    x1_2, y1_2, x2_2, y2_2 = box2
+
+    x1_inter = max(x1_1, x1_2)
+    y1_inter = max(y1_1, y1_2)
+    x2_inter = min(x2_1, x2_2)
+    y2_inter = min(y2_1, y2_2)
+
+    if x2_inter < x1_inter or y2_inter < y1_inter:
+        return 0.0
+
+    inter_area = (x2_inter - x1_inter) * (y2_inter - y1_inter)
+
+    area1 = (x2_1 - x1_1) * (y2_1 - y1_1)
+    area2 = (x2_2 - x1_2) * (y2_2 - y1_2)
+
+    union_area = area1 + area2 - inter_area
+
+    iou = inter_area / union_area
+    return iou
+
+
+def calculate_precision(json_object):
+    stats = {
+        "Video Description Steps": {"Matched": 0, "Wrong": 0, "Redundant": 0},
+        "Logical Inference Steps": {"Matched": 0, "Wrong": 0, "Redundant": 0},
+        "Background Review Steps": {"Matched": 0, "Wrong": 0, "Redundant": 0}
+    }
+    for item in json_object:
+        step_type = item["step_type"]
+        judgement = item["judgment"]
+        stats[step_type][judgement] += 1
+
+    return stats
+
+
+def recall(item):
+    processed_item = copy.deepcopy(item)
+
+    json_object = json.loads(extract_json_between_backticks(processed_item['recall_eval']))
+    stats = calculate_recall(json_object)
+
+    Video_recall = "" if (stats['Video Description Steps']['Matched'] + stats['Video Description Steps']['Unmatched']) == 0 else stats['Video Description Steps']['Matched'] / (stats['Video Description Steps']['Matched'] + stats['Video Description Steps']['Unmatched'])  # noqa: E501
+
+    logic_recall = "" if (stats['Logical Inference Steps']['Matched'] + stats['Logical Inference Steps']['Unmatched']) == 0 else stats['Logical Inference Steps']['Matched'] / (stats['Logical Inference Steps']['Matched'] + stats['Logical Inference Steps']['Unmatched'])  # noqa: E501
+
+    background_recall = "" if (stats['Background Review Steps']['Matched'] + stats['Background Review Steps']['Unmatched']) == 0 else stats['Background Review Steps']['Matched'] / (stats['Background Review Steps']['Matched'] + stats['Background Review Steps']['Unmatched'])  # noqa: E501
+
+    processed_item['Video_recall'] = Video_recall
+    processed_item['logic_recall'] = logic_recall
+    processed_item['background_recall'] = background_recall
+
+    total_matched = (
+        stats['Video Description Steps']['Matched']
+        + stats['Logical Inference Steps']['Matched']
+    )
+
+    total_steps = (
+        (stats['Video Description Steps']['Matched'] + stats['Video Description Steps']['Unmatched'])
+        + (stats['Logical Inference Steps']['Matched'] + stats['Logical Inference Steps']['Unmatched'])
+    )
+
+    if total_steps == 0:
+        overall_recall = ""
+    else:
+        overall_recall = total_matched / total_steps
+
+    processed_item['overall_recall'] = overall_recall
+
+    return processed_item
+
+
+def precision(item):
+    processed_item = copy.deepcopy(item)
+
+    json_object = json.loads(extract_json_between_backticks(processed_item['precision_eval']))
+    stats = calculate_precision(json_object)
+
+    Video_precision = "" if (stats['Video Description Steps']['Matched'] + stats['Video Description Steps']['Wrong']) == 0 else stats['Video Description Steps']['Matched'] / (stats['Video Description Steps']['Matched'] + stats['Video Description Steps']['Wrong'])  # noqa: E501
+
+    logic_precision = "" if (stats['Logical Inference Steps']['Matched'] + stats['Logical Inference Steps']['Wrong']) == 0 else stats['Logical Inference Steps']['Matched'] / (stats['Logical Inference Steps']['Matched'] + stats['Logical Inference Steps']['Wrong'])  # noqa: E501
+
+    background_precision = "" if (stats['Background Review Steps']['Matched'] + stats['Background Review Steps']['Wrong']) == 0 else stats['Background Review Steps']['Matched'] / (stats['Background Review Steps']['Matched'] + stats['Background Review Steps']['Wrong'])  # noqa: E501
+
+    processed_item['Video_precision'] = Video_precision
+    processed_item['logic_precision'] = logic_precision
+    processed_item['background_precision'] = background_precision
+
+    total_matched = (
+        stats['Video Description Steps']['Matched']
+        + stats['Logical Inference Steps']['Matched']
+    )
+
+    total_wrong = (
+        stats['Video Description Steps']['Wrong']
+        + stats['Logical Inference Steps']['Wrong']
+    )
+
+    if (total_matched + total_wrong) == 0:
+        overall_precision = ""
+    else:
+        overall_precision = total_matched / (total_matched + total_wrong)
+
+    processed_item['overall_precision'] = overall_precision
+
+    total_step_num = 0
+    for counts in stats.values():
+        total_step_num += sum(counts.values())
+
+    redundant_num = 0
+    for counts in stats.values():
+        redundant_num += counts['Redundant']
+
+    efficiency = (total_step_num - redundant_num) / total_step_num
+    processed_item['efficiency'] = efficiency
+    return processed_item
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..ff07371e65bb9abba112abd067123b950b7707de
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vcrbench/prompt.py
@@ -0,0 +1,271 @@
+# flake8: noqa
+Recall_Evaluation_Prompt = """You are an expert system for verifying solutions to video-based problems. Your task is to match the ground truth middle steps with the provided solution.
+
+INPUT FORMAT:
+1. Problem: The original question/task
+2. A Solution of a model
+3. Ground Truth: Essential steps required for a correct answer
+
+MATCHING PROCESS:
+
+You need to match each ground truth middle step with the solution:
+
+Match Criteria:
+- The middle step should exactly match in the content or is directly entailed by a certain content in the solution
+- All the details must be matched, including the specific value and content
+- You should judge all the middle steps for whethere there is a match in the solution
+
+Step Types:
+1. Logical Inference Steps
+   - Contains exactly one logical deduction
+   - Must produce a new derived conclusion
+   - Cannot be just a summary or observation
+
+2. Video Description Steps
+   - Pure visual observations
+   - Only includes directly visible elements
+   - No inferences or assumptions
+   - Contains event time
+
+3. Background Review Steps:
+   - Repetition or review of the problem
+   - Not directly related to solving the problem.
+
+OUTPUT FORMAT:
+JSON array of judgments:
+[
+  {{
+    "step": ground truth middle step,
+    "step_type": "Video Description Steps|Logical Inference Steps|Background Review Steps",
+    "judgment": "Matched" | "Unmatched",
+  }}
+]
+
+ADDITIONAL RULES:
+1. Only output the json array with no additional information.
+2. Judge each ground truth middle step in order without omitting any step.
+
+Here is the problem, answer, solution, and the ground truth middle steps:
+"""
+
+Precision_Evaluation_Prompt = """
+# Task Overview
+Given a solution with multiple reasoning steps for an video-based problem, reformat it into well-structured steps and evaluate their correctness.
+
+# Step 1: Reformatting the Solution
+Convert the unstructured solution into distinct reasoning steps while:
+- Preserving all original content and order
+- Not adding new interpretations
+- Not omitting any steps
+
+## Step Types
+1. Logical Inference Steps
+   - Contains exactly one logical deduction
+   - Must produce a new derived conclusion
+   - Cannot be just a summary or observation
+
+2. Video Description Steps
+   - Pure visual observations
+   - Only includes directly visible elements
+   - No inferences or assumptions
+   - Contains event time
+
+3. Background Review Steps:
+   - Repetition or review of the problem
+   - Not directly related to solving the problem.
+
+## Step Requirements
+- Each step must be atomic (one conclusion per step)
+- No content duplication across steps
+- Initial analysis counts as background information
+- Final answer determination counts as logical inference
+
+# Step 2: Evaluating Correctness
+Evaluate each step against:
+
+## Ground Truth Matching
+For video descriptions:
+- Key elements must match ground truth descriptions
+
+For logical inferences:
+- Conclusion must EXACTLY match or be DIRECTLY entailed by ground truth
+
+For Background review:
+-  Without special circumstances are deemed to be redundant
+
+## Reasonableness Check (if no direct match)
+If Step:
+- Premises must not contradict any ground truth or correct answer
+- Logic is valid
+- Conclusion must not contradict any ground truth
+- Conclusion must support or be neutral to correct answer
+- Helpful in solving the problem, non-redundant steps
+this Step be viewed as matched.
+
+## Judgement Categories
+- "Match": Aligns with ground truth
+- "Wrong": Contradictory with ground truth
+- "Redundant": Redundant steps that do not help solve the problem
+
+# Output Requirements
+1. The output format MUST be in valid JSON format without ANY other content.
+2. For highly repetitive patterns, output it as a single step.
+3. Output maximum 35 steps. Always include the final step that contains the answer.
+
+Here is the json output format:
+## Output Format
+[
+  {{
+    "step": "reformatted the solution step",
+    "step_type": "Video Description Steps|Logical Inference Steps|Background Review Steps",
+    "reasons_for_judgment": "The reason for judging the matching result of the step in the solution based on Ground Truth Information. Sufficient evidence needs to be found in Ground Truth Information to determine the correctness of the reformatted the solution step. The video event description time error is no more than 3 seconds and is considered correct. If the solution step does not specify the time, it is considered wrong.",
+    "judgment": "Matched|Wrong|Redundant",
+  }}
+]
+
+Here is the problem, and the solution that needs to be reformatted to steps:
+
+"""
+
+Answer_Extraction_Prompt_part1 = """You are an AI assistant who will help me to extract an answer of a question. You are provided with a question and a response, and you need to find the final answer of the question.
+
+Extract Rule:
+[Multiple choice question]
+1. The answer could be answering the option letter or the value. You should directly output the choice letter of the answer.
+2. You should output a single uppercase character in A, B, C, D, E, F, G, H, I (if they are valid options), and Z.
+3. If the meaning of all options are significantly different from the final answer, output Z.
+[Non Multiple choice question]
+1. Output the final value of the answer. It could be hidden inside the last step of calculation or inference. Pay attention to what the question is asking for to extract the value of the answer.
+2. The final answer could also be a short phrase or sentence.
+3. If the answer is about a certain time period, such as from 1 minute 30 seconds to 2 minutes 30 seconds, it should be given in the format [90, 150].
+4. If the response doesn't give a final answer, output Z.
+
+Output Format:
+Directly output the extracted answer of the response
+
+Example 1:
+Question: What is the main object in image?\nOptions: A. teddy bear B. rabbit C. cat D. dog
+Response: a cute teddy bear
+Your output: A
+Example 2:
+Question: What is the main object in image?\nOptions: A. teddy bear B. rabbit C. cat D. dog
+Answer: Spider
+Your output: Z
+Example 3:
+Question: On a flat playground, choose a Cartesian Oxy coordinate system (in unit of meters). A child running at a constant velocity $V=1 \\mathrm{~m} / \\mathrm{s}$ around a heart-shaped path satisfies the following order- 6 algebraic equation:\n$$\n\\left(x^{2}+y^{2}-L^{2}\\right)^{3}-L x^{2} y^{3}=0, L=10 \\text {. }\n$$\nWhen the child is at the position $(x, y)=(L, 0)$, what is the magnitude of their acceleration?\n\n<image>
+Response: fore, \\(r \\approx 10 \\, \\text{m}\\).\n\n7. **Calculating Acceleration**: Using the radius of curvature:\n   \\[\n   a_c = \\frac{V^2}{r} = \\frac{(1 \\, \\text{m/s})^2}{10 \\, \\text{m}} = 0.1 \\, \\text{m/s}^2\n   \\]\n\n8. **Magnitude of Acceleration**: The magnitude of the acceleration is the sum of the tangential and centripetal accelerations. Since the tangential acceleration is zero, the magnitude of the acceleration is purely centripetal:\n   \\[\n   \\text{Magnitude of acceleration} = a_c = 0.1 \\, \\text{m/s}^2\n   \\]\n\nAnswer: \\boxed{0.1 \\, \\text{m/s}^2}
+Your output: 0.1 text{m/s}^2
+Example 4:
+Question: In the diagram, $O A=15, O P=9$ and $P B=4$. Determine the equation of the line through $A$ and $B$. Explain how you got your answer.
+
+<image>
+Response: The equation of the line through $A$ and $B$ is $y = -\frac{1}{3}x + 12$.
+
+To find the equation of the line through $A$ and $B$, we first need to find the slope of the line. The slope of a line passing through two points $(x_1, y_1)$ and $(x_2, y_2)$ is given by the formula:
+
+$$\text{Slope} = \frac{y_2 - y_1}{x_2 - x_1}$$
+
+Using the coordinates of points $A(0, 15)$ and $B(4, 0)$, we can calculate the slope as follows:
+
+$$\text{Slope} = \frac{0 - 15}{4 - 0} = \frac{-15}{4} = -\frac{15}{4}$$
+
+Now that we have the slope, we can use the point-slope form of a linear equation to find the equation of the line. The point-slope form is given by:
+
+$$y - y_1 = m(x - x_1)$$
+
+where $(x_1, y_1)$ is a point on the line and $m$ is the slope. In this case, we can use point $A(0, 15)$ and the slope $-\frac{15}{4}$:
+
+$$y - 15 = -\frac{15}{4}(x - 0)$$
+
+Simplifying the equation, we get:
+
+$$y - 15 = -\frac{15}{4}x$$
+
+$$y = -\frac{15}{4}x + 15$$
+
+Therefore, the equation of the line through $A$ and $B$ is $y = -\frac{15}{4}x + 15$.
+Your output: $y = -\frac{15}{4}x + 15$
+Example 5:
+Question: <image> <image> <image> <image> <image> <image> <image> <image> <image>In the paper \'An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion\', what aspect of the proposed method does this figure aim to demonstrate?
+Response: the potential of this 3D generation paradigm to generate objects with PBR materials
+Your output: the potential of this 3D generation paradigm to generate objects with PBR materials
+Example 6:
+Question: Please specify the start and end time of the event 'An intro screen appears with a gray background and the red and white words that say PRO TRAINING.'
+Response: The start and end time of the event 'An intro screen appears with a gray background and the red and white words that say PRO TRAINING.' are as follows:\n\n**Step 1: Identify the start time of the event.**\nThe video starts at 00:00. The intro screen is visible from the very beginning.\n\n**Start time:** 00:00\n\n**Step 2: Identify the end time of the event.**\nThe intro screen starts to disappear and transition to the next scene at 00:11. By 00:12, the intro screen is completely gone. So, the intro screen is visible until 00:11.\n\n**End time:** 00:11\n\nTherefore, the event 'An intro screen appears with a gray background and the red and white words that say PRO TRAINING' starts at **00:00** and ends at **00:11**.\n\n**Answer:**\nStart time: 00:00\nEnd time: 00:11
+Your output: [0,11]
+
+"""
+
+Answer_Extraction_Prompt_part2 = """
+Question: {question}
+Answer: {response}
+Your output:
+"""
+
+Answer_Scoring_Prompt_part1 = r"""You are an AI assistant who will help me to judge whether two answers are consistent.
+
+Input Illustration:
+[Standard Answer] is the standard answer to the question
+[Model Answer] is the answer extracted from a model's output to this question.
+
+Task Illustration:
+Determine whether [Standard Answer] and [Model Answer] are consistent.
+Consistent Criteria:
+[Multiple-Choice questions]
+1. If the [Model Answer] is the option letter, then it must completely matches the [Standard Answer].
+2. If the [Model Answer] is not an option letter, then the [Model Answer] must completely match the option content of [Standard Answer].
+[Nan-Multiple-Choice questions]
+1. The [Model Answer] and [Standard Answer] should exactly match.
+2. If the meaning is expressed in the same way, it is also considered consistent, for example, 0.5m and 50cm.
+
+Output Format:
+1. If they are consistent, output 1; if they are different, output 0.
+2. DIRECTLY output 1 or 0 without any other content.
+
+Example 1:
+Question: What is the main object in image?\nOptions: A. teddy bear B. rabbit C. cat D. dog
+[Model Answer]: a cute teddy bear
+[Standard Answer]: A
+Your output: 1
+
+Example 2:
+Question: Find the value of AB. Choices: A.1;B.5;C.9;D.10
+[Model Answer]: \\boxed{5}
+[Standard Answer]: B
+Your output: 1
+
+Example 3:
+Question: Three of the following four slides are from the same presentation, but one is from a different one. Please identify the outlier: \n\n<image> <image> <image> <image>\nA. the forth image\nB. the second image\nC. the third image\nD. None of the choices provided
+[Model Answer]: \\boxed{B}
+[Standard Answer]: A
+Your output: 0
+
+
+"""
+
+Answer_Scoring_Prompt_part2 = """
+Question: {question}
+[Model Answer]: {extract_answer}
+[Standard Answer]: {gt_answer}
+Your output:
+"""
+
+
+def build_Extraction_prompt(item):
+    tmpl = 'Question: {question}\nAnswer: {response}\nYour output:'
+    return tmpl.format(question=item['question'], response=item['prediction'])
+
+
+def build_Scoring_prompt(item):
+    tmpl = 'Question: {question}\n[Model Answer]: {extract_answer}\n[Standard Answer]: {gt_answer}\nYour output:'
+    return tmpl.format(question=item['question'], extract_answer=item['extracted_answer'], gt_answer=item['answer'])
+
+
+def build_Precision_prompt(item):
+    tmpl = '[Problem]:{question}\n[Solution]:{solution}\n[Ground Truth Information]:{gt_annotation}'
+    return tmpl.format(question=item['question'], solution=item['prediction'], gt_annotation=item['reasoning'])
+
+
+def build_Recall_prompt(item):
+    tmpl = '[Problem]:{question}\n[Answer]:{answer}\n[Solution]:{solution}\n[Ground Truth Information]:{gt_annotation}'
+    return tmpl.format(question=item['question'], answer=item['answer'], solution=item['prediction'], gt_annotation=item['reasoning'])
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/aquarium/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/aquarium/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..6c10fe37bce681503cf13e8e38467f862ccc9eb6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/aquarium/filter_prompt.json
@@ -0,0 +1,118 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0, e, or s; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Sometimes perception is not given but answer is there, you should put a Wrong for perception and process the answer. Your goal is only to format the user's input, not to check if it is correct or not."},
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"*\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"*\", \"*\"]\n],\n\"think\": \"Based on the numbers given on the sides, each row and column must have 2 filled cells. To ensure contiguity at the bottom, the filled cells should be placed in the bottom two rows. The first and third columns have a constraint of 4 filled cells, so those columns must have the bottom two cells filled. The second and fourth columns can have any two cells filled in the bottom two rows.\",\n\"answer\": [\n  [\"e\", \"e\", \"e\", \"e\"],\n  [\"e\", \"e\", \"e\", \"e\"],\n  [\"s\", \"s\", \"s\", \"s\"],\n  [\"s\", \"e\", \"s\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,0,0,0],[0,0,0,0],[s,s,s,s],[s,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"]\n  ],\n  \"think\": \"The puzzle is a 4x5 grid. Each column and row has a number indicating how many cells should be filled in that row/column. The filled cells must be contiguous. \\nLet's start with the first column which has 4 filled cells. Since the filled cells must be contiguous, we can fill the bottom four cells of the first column. Similarly, the last column also has 4 filled cells, so we fill the bottom four cells. The second and third columns have 2 filled cells each. We can fill the bottom two cells in each of these columns. This satisfies the column constraints. Now let's check the row constraints. The bottom two rows have 4 filled cells each, which is satisfied by our current solution. The top two rows have 2 filled cells each, which is also satisfied. Therefore, this is the solution.\",\n  \"answer\": [\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"s\", \"s\", \"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\", \"s\", \"s\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[0,0,0,0,0],[0,0,0,0,0],[s,s,s,s,s],[s,s,s,s,s],[s,s,s,s,s]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content":"```json\n{\n\"perception\": [[\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n\"answer\": [[\"s\", \"s\", \"e\", \"e\"], [\"s\", \"s\", \"e\", \"e\"], [\"e\", \"e\", \"s\", \"s\"], [\"e\", \"e\", \"s\", \"s\"]]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[s,s,0,0],[s,s,0,0],[0,0,s,s],[0,0,s,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user", 
+        "content": "**Current Game State:**\n```json\n{\n\"perception\": [\n[\"s\", \"e\", \"e\", \"e\"],\n[\"e\", \"e\", \"e\", \"s\"],\n[\"e\", \"s\", \"e\", \"s\"],\n[\"e\", \"e\", \"e\", \"s\"],\n[\"s\", \"e\", \"s\", \"e\"],\n[\"e\", \"s\", \"e\", \"e\"],\n[\"e\", \"e\", \"e\", \"s\"]\n]\n}\n```\n\n**Solution:**\n```json\n{\n\"answer\": [\n[\"s\", \"s\", \"e\", \"s\"],\n[\"e\", \"s\", \"s\", \"s\"],\n[\"e\", \"s\", \"e\", \"s\"],\n[\"e\", \"e\", \"s\", \"s\"],\n[\"s\", \"s\", \"e\", \"e\"],\n[\"e\", \"s\", \"e\", \"s\"],\n[\"s\", \"e\", \"e\", \"s\"]\n]\n}\n```\n\n**Final Answer:**\nThe solution is represented by the \"answer\" array, where \"s\" denotes filled cells and \"e\" denotes empty cells."
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[s,0,0,0],[0,0,0,s],[0,s,0,s],[0,0,0,s],[s,0,s,0],[0,s,0,0],[0,0,0,s]]\nSolution\n[[s,s,0,s],[0,s,s,s],[0,s,0,s],[0,0,s,s],[s,s,0,0],[0,s,0,s],[s,0,0,s]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n\"perception\": [\n[0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0]\n],\n\"think\": \"The grid is initially empty. We need to fill the aquariums to satisfy the given row and column constraints. The numbers on the sides indicate the total number of filled cells in each row and column. We can start by placing the first filled cell in each row and column as shown in the solution. Then, we can fill the remaining cells while ensuring that each aquarium's water cells are contiguous at the bottom of that region.\",\n\"answer\": [\n[0, 0, 0, 0, 0],\n[0, 1, 1, 0, 0],\n[0, 1, 1, 0, 0],\n[0, 1, 1, 0, 0],\n[0, 1, 1, 0, 0]\n]\n}"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\nWrong"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. Because 1 is undefined, so put a Wrong there"
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n\"perception\": [\n[0, 0, 0, 0],\n[0, 0, 0, 0],\n[0, 2, 2, 0],\n[0, 2, 2, 0]\n],\n\"answer\": [\n[0, 0, s, s],\n[0, 0, s, s],\n[0, s, s, 0],\n[0, s, s, 0]\n]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\n[[0,0,s,s],[0,0,s,s],[0,s,s,0],[0,s,s,0]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. As 2 is not defined"
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n\"answer\": [\n    [\"e\", \"e\", \"s\", \"s\"],\n    [\"e\", \"s\", \"s\", \"e\"],\n    [\"s\", \"s\", \"e\", \"e\"],\n    [\"s\", \"e\", \"e\", \"e\"]\n]\n}\n```\n\nThis solution satisfies the given constraints:\n\n1. Each aquarium's water cells are contiguous at the bottom of that region.\n2. The row counts match the number of filled cells in each row: [0, 2, 2, 2].\n3. The column counts match the number of filled cells in each column: [2, 2, 2, 2].\n\nThe aquariums are filled as follows:\n- Aquarium 1: [[2, 0], [1, 0], [0, 0]]\n- Aquarium 2: [[2, 1], [2, 2]]\n- Aquarium 3: [[1, 3], [0, 3], [0, 2], [1, 2], [1, 1], [0, 1]]\n- Aquarium 4: [[2, 3], [3, 3], [3, 2]]"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\n[[0,0,s,s],[0,s,s,0],[s,s,0,0],[s,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. there is no perception, but the answer is valid, so put it there."
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n  \"think\": \"To solve this puzzle, I will first identify the filled cells and their positions in the given board state. Then, I will use these positions to fill the aquariums with water according to the given row and column counts. Finally, I will check if the solution is correct by ensuring that the water cells are contiguous at the bottom of each region.\",\n  \"answer\": [\n    [\"s\", \"s\", \"e\", \"e\"],\n    [\"s\", \"s\", \"e\", \"e\"],\n    [\"s\", \"s\", \"e\", \"e\"],\n    [\"e\", \"s\", \"e\", \"s\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\n[[s,s,0,0],[s,s,0,0],[s,s,0,0],[0,s,0,s]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. there is no perception, but the answer is valid, so put it there."
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n\"answer\": [\n[1, 1, 1, 1],\n[1, 0, 0, 1],\n[1, 0, 0, 1],\n[1, 0, 0, 1]\n]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. there is no perception, so Wrong; for answer, 1 is not defined."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/battleships/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/battleships/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..7839a70baaf833d4589b170dfe72f69647c169da
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/battleships/filter_prompt.json
@@ -0,0 +1,91 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only three types values 0 or s; if you see * or _  or - or e or #, you should convert them into 0. But if there is any number in the grid, it is Wrong. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"s\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"]\n],\n\"think\": \"Based on the given information, there is a ship of size 2 and two ships of size 1. The row hints indicate that the ship of size 2 must be placed in the second row, and the column hints suggest that one of the size 1 ships must be placed in the second column. The other size 1 ship can be placed in either the first, third, or fourth column without violating the rules.\",\n\"answer\": [\n    [\"e\", \"s\", \"e\", \"e\"],\n    [\"e\", \"s\", \"e\", \"e\"],\n    [\"e\", \"e\", \"s\", \"e\"],\n    [\"e\", \"e\", \"e\", \"s\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,s,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,s,0,0],[0,s,0,0],[0,0,s,0],[0,0,0,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "model_output\": \"```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"s\", \"s\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"]\n  ],\n  \"answer\": [\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"s\", \"s\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"s\"],\n    [\"e\", \"s\", \"e\", \"e\", \"e\"]\n  ]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,s,s,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,s,s,0],[0,0,0,0,s],[0,s,0,0,0]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "**Step 1: Analyze the given information**\n\n*   The grid is 6x6.\n*   There are three ships of sizes 2, 1, and 1.\n*   The hints indicate that there are two cells in the first row and one cell in the second row.\n*   There is one cell in the first column and one cell in the second column.\n\n**Step 2: Place the ships**\n\n*   We can start by placing the two-cell ship in the first row. We can place it in the first two columns or the last two columns.\n*   Let's place it in the first two columns. This leaves us with a 4x4 grid to place the remaining ships.\n*   We can place the one-cell ship in the second row and the first column.\n*   This leaves us with a 4x4 grid to place the remaining one-cell ship.\n*   We can place it in the second row and the second column.\n\n**Step 3: Check for any constraints**\n\n*   There are no constraints mentioned in the problem, so we can proceed with our current placement.\n\n**Step 4: Write the solution**\n\n```json\n{\"perception\": [[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]], \"think\": \"Place the two-cell ship in the first two columns, then place the one-cell ship in the second row and the first column, and finally place the remaining one-cell ship in the second row and the second column.\", \"answer\": [[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]}\n```\n\n**Answer:** {\"perception\": [[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]], \"think\": \"Place the two-cell ship in the first two columns, then place the one-cell ship in the second row and the first column, and finally place the remaining one-cell ship in the second row and the second column.\", \"answer\": [[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]}"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0]]\nSolution\n[[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n[\"#\", \"s\", \"#\", \"#\"],\n[\"#\", \"s\", \"#\", \"#\"],\n[\"#\", \"#\", \"#\", \"#\"],\n[\"#\", \"#\", \"#\", \"#\"]\n],\n\"think\": \"The grid is 4x4, and there are two ships: one of size 1 and one of size 2. The size-1 ship is already placed in the bottom-left corner (x1=0, y1=3). The size-2 ship must be placed horizontally or vertically, not overlapping with the size-1 ship.\",\n\"answer\": [\n[\"#\", \"s\", \"#\", \"#\"],\n[\"#\", \"s\", \"#\", \"#\"],\n[\"#\", \"#\", \"#\", \"#\"],\n[\"#\", \"#\", \"#\", \"#\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,s,0,0],[0,s,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,s,0,0],[0,s,0,0],[0,0,0,0],[0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },   
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n[\"*\", \"*\", \"*\", \"e\"],\n[\"*\", \"*\", \"s\", \"*\"],\n[\"*\", \"*\", \"s\", \"*\"],\n[\"*\", \"e\", \"*\", \"*\"]\n],\n\"think\": \"Let's analyze this 4x4 grid:\n1. We need to place 1 size-2 ship and 2 size-1 ships\n2. The row hints (0,2,1,1) and column hints (2,0,2,0) tell us how many cells are ships\n3. We can see:\n   - A size-2 ship vertically placed in column 2, rows 1-2\n   - Two water cells marked with dots\n4. Looking at row 0 with hint 0, all cells must be water\n5. Column 1 has hint 0, so all cells must be water\n6. The remaining two size-1 ships must be placed in valid positions without touching the existing size-2 ship\n7. Given the constraints, one size-1 ship must be in row 3, column 0 and the other in row 3, column 2\",\n\"answer\": [\n[\"e\", \"e\", \"e\", \"e\"],\n[\"e\", \"e\", \"s\", \"e\"],\n[\"e\", \"e\", \"s\", \"e\"],\n[\"s\", \"e\", \"s\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,0,s,0],[0,0,s,0],[0,0,0,0]]\nSolution\n[[0,0,0,0],[0,0,s,0],[0,0,s,0],[s,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },   
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\"]\n  ],\n  \"think\": \"Analyze the given grid. We are given hints indicating how many ship cells are in each row and column. Small black circles represent single-cell ship cells, and two-cell ships must be placed as per the hints. From the row and column numbers, I will determine where the ship segments must go, ensuring no ships touch each other, even diagonally.\",\n  \"answer\": [\n    [],\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0],[0,0,0],[0,0,0]]\nSolution\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as there is no answer could be extracted from the model output"
+    }, 
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"s\"]\n  ],\n  \"answer\": [\n    [\"e\", \"s\", \"e\", \"e\"],\n    [\"e\", \"e\", \"s\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\"],\n    [\"s\", \"e\", \"e\", \"s\"]\n  ]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,s]]\nSolution\n[[0,s,0,0],[0,0,s,0],[0,0,0,0],[s,0,0,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    } 
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/binairo/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/binairo/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..df36714bdd387fb23c4173153b69787fcc324af9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/binairo/filter_prompt.json
@@ -0,0 +1,67 @@
+[
+    {
+        "role": "system", 
+        "content": 
+        "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only three types values 0 or w or b; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, it is wrong.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n[\"*\", \"*\", \"w\", \"w\"],\n[\"b\", \"b\", \"*\", \"w\"],\n[\"w\", \"w\", \"*\", \"b\"],\n[\"w\", \"*\", \"*\", \"*\"]\n],\n\"think\": [\n\"1. Looking at row 1 (0-based), there are two black circles (b,b), so the remaining two must be white to avoid three consecutive blacks\",\n\"2. Looking at column 3 (0-based), there are already three whites (w,w,w), so the last cell must be black\",\n\"3. In row 2, we have (w,w,*,b) - to avoid three whites in a row, the empty cell must be black\",\n\"4. In row 3, we have (w,*,*,*) - given the column constraints and avoiding three consecutive same colors, it must be (w,b,w,b)\",\n\"5. This completes the grid while satisfying all constraints\"\n],\n\"answer\": [\n[\"b\", \"b\", \"w\", \"w\"],\n[\"b\", \"b\", \"w\", \"w\"],\n[\"w\", \"w\", \"b\", \"b\"],\n[\"w\", \"b\", \"w\", \"b\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[0,0,w,w],[b,b,0,w],[w,w,0,b],[w,0,0,0]]\nAnswer\n[[b,b,w,w],[b,b,w,w],[w,w,b,b],[w,b,w,b]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"b\", \"b\", \"*\", \"*\"],\n    [\"*\", \"b\", \"*\", \"w\"],\n    [\"*\", \"*\", \"b\", \"b\"],\n    [\"w\", \"*\", \"w\", \"b\"]\n  ],\n  \"think\": \"To solve this, we apply the Binairo rules. Here\u2019s the step-by-step reasoning: \n1. Row 0: The first two cells are 'b'. According to the rules, no more than two same-colored pieces can be adjacent. Thus, the third cell must be 'w'.\n2. Row 3: The fourth cell is 'b', and the second cell is blank. Following the row's structure and aiming for balanced numbers of 'b' and 'w', the second cell must be 'b.\"\n3. Each horizontal"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[b,b,0,0],[0,b,0,w],[0,0,b,b],[w,0,w,b]]\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as there is no answer section, so you should put a Wrong there."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"w\"],\n    [\"b\", \"b\", \"*\", \"*\"],\n    [\"w\", \"*\", \"w\", \"b\"],\n    [\"w\", \"*\", \"*\", \"*\"]\n  ],\n  \"answer\": [\n    [\"b\", \"w\", \"b\", \"w\"],\n    [\"b\", \"b\", \"w\", \"b\"],\n    [\"w\", \"b\", \"w\", \"b\"],\n    [\"w\", \"b\", \"w\", \"b\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[0,0,0,w],[b,b,0,0],[w,0,w,b],[w,0,0,0]]\nAnswer\n[[b,w,b,w],[b,b,w,b],[w,b,w,b],[w,b,w,b]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"w\"],\n    [\"b\", \"b\", \"*\", \"*\"],\n    [\"w\", \"*\", \"w\", \"b\"],\n    [\"w\", \"*\", \"*\", \"*\"]\n  ],\n  \"answer\": [\n    [\"b\", \"w\", \"b\", \"w\"],\n    [\"b\", \"b\", \"w\", \"b\"],\n    [\"w\", \"b\", \"w\", \"b\"],\n    [\"w\", \"b\", \"w\", \"b\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[0,0,0,w],[b,b,0,0],[w,0,w,b],[w,0,0,0]]\nAnswer\n[[b,w,b,w],[b,b,w,b],[w,b,w,b],[w,b,w,b]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user", 
+        "content": "The user wants me to solve a Binairo puzzle.\nI need to identify the black and white circles in the image and represent them as 'b' and 'w' respectively. Empty cells will be represented by '*'.\nThen, I need to apply the Binairo rules to fill the empty cells.\nThe rules are:\n1. No more than two adjacent cells can have the same color.\n2. Each row and column must have an equal number of black and white cells.\n\nLet's represent the initial state of the grid.\nRow 0: *, *, *, b\nRow 1: w, *, *, *\nRow 2: *, b, w, *\nRow 3: b, *, *, w\n\nNow, let's apply the rules to fill the empty cells.\n\n**Step 1:** Analyze row 0. It has one 'b'. It needs one more 'b' and two 'w'.\n**Step 2:** Analyze row 1. It has one 'w'. It needs one more 'w' and two 'b'.\n**Step 3:** Analyze row 2. It has one 'b' and one 'w'. It needs one more 'b' and one more 'w'.\n**Step 4:** Analyze row 3. It has one 'b' and one 'w'. It needs one more 'b' and one more 'w'.\n\n**Step 5:** Analyze column 0. It has one 'w' and one 'b'. It needs one more 'w' and one more 'b'.\n**Step 6:** Analyze column 1. It has one 'b'. It needs one more 'b' and two 'w'.\n**Step 7:** Analyze column 2. It has one 'w'. It needs one more 'w' and two 'b'.\n**Step 8:** Analyze column 3. It has one 'b' and one 'w'. It needs one more 'b' and one more 'w'.\n\nLet's start filling based on the no-three-in-a-row rule.\nIn row 0, the last cell is 'b'. So the cell at (0,2) cannot be 'b'.\nIn row 3, the first cell is 'b'. So the cell at (3,1) cannot be 'b'.\n\nConsider row 0: *, *, *, b. If the second to last cell is 'b', we have *, *, b, b. Then the first two must be 'w'. So w, w, b, b. This is valid.\nConsider row 3: b, *, *, w. If the second cell is 'w', we have b, w, *, w. The third cell must be 'b'. So b, w, b, w. This is valid.\n\nLet's update the grid:\nw, w, b, b\nw, *, *, *\n*, b, w, *\nb, w, b, w\n\nConsider row 1: w, *, *, *. It needs one more 'w' and two 'b'.\nConsider row 2: *, b, w, *. It needs one more 'b' and one more 'w'.\n\nConsider column 1: w, *, b, w. The empty cell must be 'b'.\nConsider column 2: b, *, w, b. The empty cell must be 'w'.\n\nLet's update the grid:\nw, w, b, b\nw, b, w, *\n*, b, w, *\nb, w, b, w\n\nConsider row 1: w, b, w, *. The last cell must be 'b'.\nConsider row 2: *, b, w, *. The last cell must be 'b'.\n\nLet's update the grid:\nw, w, b, b\nw, b, w, b\n*, b, w, b\nb, w, b, w\n\nConsider row 2: *, b, w, b. The first cell must be 'w'.\n\nLet's update the grid:\nw, w, b, b\nw, b, w, b\nw, b, w, b\nb, w, b, w\n\nNow let's check the column counts.\nColumn 0: w, w, w, b (3w, 1b) - Incorrect.\nThere must be an error in the previous steps.\n\nLet's go back to the state:\nw, w, b, b\nw, *, *, *\n*, b, w, *\nb, w, b, w\n\nConsider row 1: w, *, *, *. It needs one more 'w' and two 'b'.\nIf the second cell is 'b', we have w, b, *, *.\nIf the third cell is 'b', we have w, b, b, *. The last cell must be 'w'. So w, b, b, w.\nConsider row 2: *, b, w, *. It needs one more 'b' and one more 'w'.\n\nLet's update the grid:\nw, w, b, b\nw, b, b, w\n*, b, w, *\nb, w, b, w\n\nConsider column 0: w, w, *, b. The empty cell must be 'b'.\nConsider column 3: b, w, *, w. The empty cell must be 'b'.\n\nLet's update the grid:\nw, w, b, b\nw, b, b, w\nb, b, w, b\nb, w, b, w\n\nConsider row 2: b, b, w, b. This violates the no-three-in-a-row rule.\n\nLet's go back to the state:\nw, w, b, b\nw, *, *, *\n*, b, w, *\nb, w, b, w\n\nConsider row 1: w, *, *, *. It needs one more 'w' and two 'b'.\nIf the second cell is 'w', we have w, w, *, *. The last two must be 'b'. So w, w, b, b. This is the same as row 0, which is not allowed.\n\nLet's try filling based on column constraints.\nColumn 0: w, *, *, b. It needs one 'w' and one 'b'.\nColumn 1: w, b, b, w. This is valid.\nColumn 2: b, w, w, b. This is valid.\nColumn 3: b, *, *, w. It needs one 'w' and one 'b'.\n\nLet's go back to the initial state:\n*, *, *, b\nw, *, *, *\n*, b, w, *\nb, *, *, w\n\nFrom row 0: *, *, *, b. The cells at (0,0) and (0,1) cannot be 'b'.\nFrom row 3: b, *, *, w. The cell at (3,1) cannot be 'b'.\n\nFrom column 1: *, *, b, *. The cells at (0,1) and (3,1) cannot be 'b'.\nFrom column 2: *, *, w, *. The cells at (0,2) and (3,2) cannot be 'w'.\n\nConsider row 0: *, *, *, b. If the first two are 'w', we have w, w, *, b. The third must be 'b'. So w, w, b, b.\nConsider row 3: b, *, *, w. If the middle two are 'w', we have b, w, w, w. This is invalid.\n\nLet's restart the thought process with the initial grid.\n```json\n{\n\"perception\": [\n[\"*\", \"*\", \"*\", \"b\"],\n[\"w\", \"*\", \"*\", \"*\"],\n[\"*\", \"b\", \"w\", \"*\"],\n[\"b\", \"*\", \"*\", \"w\"]\n],\n\"think\": \"The grid is 4x4. Each row and column must have two 'w' and two 'b'.\\n\\n**Step 1:** Analyze row 0: [*, *, *, b]. To avoid three consecutive 'b', the cells at (0,0) and (0,1) cannot both be 'b'.\\n\\n**Step 2:** Analyze row 3: [b, *, *, w]. To avoid three consecutive 'b', the cell at (3,1) cannot be 'b'.\\n\\n**Step 3:** Analyze column 1: [*, *, b, *]. To avoid three consecutive 'b', the cell at (0,1) cannot be 'b'.\\n\\n**Step 4:** Consider row 0: [*, *, *, b]. If we place 'w' in (0,0) and (0,1), we get [w, w, *, b]. The remaining cell must be 'b' to have two 'b's in the row: [w, w, b, b]. This is valid.\\n\\n**Step 5:** Consider row 3: [b, *, *, w]. If we place 'w' in (3,1), we get [b, w, *, w]. The remaining cell must be 'b': [b, w, b, w]. This is valid.\\n\\n**Step 6:** Update the grid:\\n[w, w, b, b]\\n[w, *, *, *]\\n[*, b, w, *]\\n[b, w, b, w]\\n\\n**Step 7:** Consider column 1: [w, *, b, w]. The missing cell must be 'b': [w, b, b, w].\\n\\n**Step 8:** Consider column 2: [b, *, w, b]. The missing cell must be 'w': [b, w, w, b].\\n\\n**Step 9:** Update the grid:\\n[w, w, b, b]\\n[w, b, w, *]\\n[*, b, w, b]\\n[b, w, b, w]\\n\\n**Step 10:** Consider row 1: [w, b, w, *]. The missing cell must be 'b': [w, b, w, b].\\n\\n**Step 11:** Consider row 2: [*, b, w, b]. The missing cell must be 'w': [w, b, w, b].\\n\\n**Step 12:** Update the grid:\\n[w, w, b, b]\\n[w, b, w, b]\\n[w, b, w, b]\\n[b, w, b, w]\\n\\n**Step 13:** Check column counts:\\nColumn 0: w, w, w, b (Incorrect)\\n\\nLet's backtrack to Step 6 and try a different approach.\\n\\n**Step 6 (Retry):** Consider row 1: [w, *, *, *]. To have two 'w's, let's place 'w' at (1,2). Then we have [w, *, w, *]. The remaining cells must be 'b': [w, b, w, b].\\n\\n**Step 7:** Consider row 2: [*, b, w, *]. To have two 'b's, let's place 'b' at (2,0). Then we have [b, b, w, *]. The remaining cell must be 'w': [b, b, w, w].\\n\\n**Step 8:** Update the grid:\\n[w, w, b, b]\\n[w, b, w, b]\\n[b, b, w, w]\\n[b, w, b, w]\\n\\n**Step 9:** Check column counts:\\nColumn 0: w, w, b, b\\nColumn 1: w, b, b, w\\nColumn 2: b, w, w, b\\nColumn 3: b, b, w, w\\nAll columns have two 'w' and two 'b'.\\n\\n**Step 10:** Check row counts:\\nRow 0: w, w, b, b\\nRow 1: w, b, w, b\\nRow 2: b, b, w, w\\nRow 3: b, w, b, w\\nAll rows have two 'w' and two 'b'.\\n\\n**Step 11:** Check for three consecutive same colors.\\nNo three consecutive same colors in any row or column.\",\n\"answer\": [\n[\"w\", \"w\", \"b\", \"b\"],\n[\"w\", \"b\", \"w\", \"b\"],\n[\"b\", \"b\", \"w\", \"w\"],\n[\"b\", \"w\", \"b\", \"w\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[0,0,0,b],[w,0,0,0],[0,b,w,0],[b,0,0,w]]\nAnswer\n[[w,w,b,b],[w,b,w,b],[b,b,w,w],[b,w,b,w]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/coloredsudoku/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/coloredsudoku/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/coloredsudoku/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/fieldexplore/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/fieldexplore/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..80a85ca514f066c6dd828c4e8f3a63be72d598a5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/fieldexplore/filter_prompt.json
@@ -0,0 +1,102 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0 or s or numbers; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"},
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"*\", \"*\", \"*\", \"*\"],\n  [\"1\", \"*\", \"*\", \"2\"],\n  [\"2\", \"*\", \"*\", \"*\"],\n  [\"*\", \"s\", \"3\", \"*\"]\n],\n\"think\": \"Based on the given information, we can make the following deductions:\n1. The cell with the value '1' has one mine adjacent to it, which must be the revealed mine cell.\n2. The cell with the value '2' in the second row has two mines adjacent to it. One of them is the revealed mine, so there must be another mine in one of the remaining hidden cells adjacent to it.\n3. The cell with the value '3' has three mines adjacent to it. One of them is the revealed mine, so there must be two more mines in the remaining hidden cells adjacent to it.\n4. The cell with the value '2' in the third row has two mines adjacent to it. One of them is the revealed mine, so there must be another mine in one of the remaining hidden cells adjacent to it.\n5. Based on the constraints of the '2' values, the only possible location for the second mine adjacent to the '2' in the third row is the cell to its left.\n6. With the second mine location determined, the remaining two mines adjacent to the '3' cell must be in the two cells above it.\",\n\"answer\": [\n  [\"e\", \"e\", \"s\", \"e\"],\n  [\"1\", \"e\", \"e\", \"2\"],\n  [\"2\", \"s\", \"e\", \"e\"],\n  [\"e\", \"s\", \"3\", \"s\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[1,0,0,2],[2,0,0,0],[0,s,3,0]]\nSolution\n[[0,0,s,0],[1,0,0,2],[2,s,0,0],[0,s,3,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [1, 2, \"*\", \"*\", \"*\"],\n    [1, \"*\", \"*\", 3, 2],\n    [2, 3, \"*\", \"*\", \"*\"],\n    [1, \"*\", \"*\", 3, \"*\"]\n  ],\n  \"think\": \"Let's analyze the grid step by step.\\n\\n1. Look at the '3' at (1, 3). It has at most one hidden mine. If there is a mine at (1, 2), then the '3' at (1, 3) will be satisfied. \\n2. Look at the '2' at (0, 1). It has at most one hidden mine. If there is a mine at (1, 1), then the '2' at (0, 1) will be satisfied. \\n3. Look at the '1' at (0, 0). It has at most one hidden mine. If there is a mine at (1, 0), then the '1' at (0, 0) will be satisfied. \\n4. Look at the '2' at (2, 0). It has at most one hidden mine. If there is a mine at (2, 1), then the '2' at (2, 0) will be satisfied. \\n5. Look at the '3' at (3, 3). It has at most one hidden mine. If there is a mine at (3, 2), then the '3' at (3, 3) will be satisfied. \\n6. Look at the '1' at (3, 0). It has at most one hidden mine. If there is a mine at (3, 1), then the '1' at (3, 0) will be satisfied. \\n7. Considering the above analysis, we can place mines at (1, 1), (1, 2), (2, 1), (3, 1), (3, 2).\",\n  \"answer\": [\n    [1, 2, \"e\", \"e\", \"e\"],\n    [1, \"s\", \"s\", 3, 2],\n    [2, 3, \"s\", \"e\", \"e\"],\n    [1, \"s\", \"s\", 3, \"e\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[1,2,0,0,0],[1,0,0,3,2],[2,3,0,0,0],[1,0,0,3,0]]\nSolution\n[[1,2,0,0,0],[1,s,s,3,2],[2,3,s,0,0],[1,s,s,3,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "To solve this problem, we need to analyze the given grid and use the information provided by the numbers to deduce the locations of the mines. \n\nThe grid is a 4x4 grid with some cells already revealed with numbers indicating how many mines are adjacent to them (including diagonals). The goal is to identify the locations of the mines based on this information.\n\nLet's start by examining the revealed cells:\n\n* Cell (0,0) has a value of 1, indicating that there is one mine adjacent to it.\n* Cell (0,1) has a value of 2, indicating that there are two mines adjacent to it.\n* Cell (1,0) has a value of 1, indicating that there is one mine adjacent to it.\n* Cell (1,1) has a value of 3, indicating that there are three mines adjacent to it.\n* Cell (2,2) has a value of 2, indicating that there are two mines adjacent to it.\n* Cell (2,3) has a value of 3, indicating that there are three mines adjacent to it.\n* Cell (3,2) has a value of 2, indicating that there are two mines adjacent to it.\n* Cell (3,3) has a value of 3, indicating that there are three mines adjacent to it.\n\nBased on this information, we can start making deductions about the locations of the mines.\n\n* Since cell (0,0) has a value of 1, it means that there is exactly one mine adjacent to it. The only possible location for this mine is cell (0,1), because if the mine were in any other adjacent cell, it would not be adjacent to cell (0,0).\n* Similarly, since cell (0,1) has a value of 2, it means that there are exactly two mines adjacent to it. One of these mines is already accounted for by cell (0,0), so the other mine must be in cell (1,1).\n* Continuing this process, we can deduce the locations of the remaining mines.\n\nHere is the solution:\n\n```json\n{\"perception\": [[1, 2, 's', '*'], [1, 3, 's', 2], [2, 3, 's', 2], ['*', 3, 3, 's']], \"think\": \"Use the given numbers to deduce the locations of the mines.\", \"answer\": [['e', 'e', 's', 'e'], ['e', 'e', 's', 'e'], ['e', 'e', 's', 'e'], ['e', 'e', 'e', 's']]}\n```\n\nIn the solution, 'e' indicates an empty cell (no mine), and 's' indicates a cell with a mine."
+
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[1,2,s,0],[1,3,s,2],[2,3,s,2],[0,3,3,s]]\nSolution\n[[0,0,s,0],[0,0,s,0],[0,0,s,0],[0,0,0,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [0, 0, 0, 0],\n    [0, 0, 0, 0],\n    [0, 0, 0, 0],\n    [0, 0, 0, 0]\n  ],\n  \"think\": [\n    \"There are no numbers or mines visible in this grid. Since it's a standard game layout, I'll need to start with general reasoning to identify potential mine locations based on the given number clues.\",\n    \"Starting with the top left and moving across rows, the number 1 indicates that there is a single mine in one of the four adjacent cells. Similarly, the number 2 suggests a single mine in two of the adjacent cells. The number 3 indicates a single mine in three of the adjacent cells. However, without more information, we cannot pinpoint the exact locations of the mines.\",\n    \"To narrow down the possibilities, I'll need to identify the patterns created by the revealed numbers. For example, if two cells have a number 3, we can infer that the mine must be in one of those cells. Similarly, if two cells have a number 1, we can infer that the mine must be in one of those cells. By analyzing the revealed numbers and their patterns, we can make educated guesses about the mine locations.\"\n  ],\n  \"answer\": [\n    [0, 0, 0, 0],\n    [0, 0, 0, 0],\n    [0, 0, 0, 0],\n    [0, 0, 0, 0]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"1\", \"2\", \"*\", \"1\", \"*\"],\n  [\"*\", \"*\", \"3\", \"*\", \"2\"],\n  [\"2\", \"3\", \"3\", \"*\", \"*\"],\n  [\"*\", \"*\", \"3\", \"*\", \"*\"]\n],\n\"think\": \"We start by analyzing the revealed numbers. The grid indicates the number of adjacent mines for the cells. We will deduce where the mines must be based on the numbers given:\\n- The cell (0, 0) is 1 and only touches (0, 1), so (0, 1) has a mine, thus confirming 1 mine at (0, 1).\\n- The cell (0, 3) is also 1, and it can only have a mine in (1, 2) since (0, 2) is adjacent to another number.\\n- For cell (1, 2) which is 3, it should have 3 adjacent mines. Currently (0, 1) is already one of them.\\n- The cell (3, 2) is 3; for it to be true, it requires careful placement in surrounding cells (2, 1), (2, 2), and (2, 3) based on existing clues.\\nBy logically deducing, it turns out that the placement of mines leads to the conclusion below:\\n- Cells (0, 1), (1, 2), (2, 1), (2, 2), (2, 3), and (3, 2) are filled with mines. The remaining cells turn out not to have any mines.\",\n\"answer\": [\n  [\"e\", \"s\", \"e\", \"e\", \"e\"],\n  [\"e\", \"e\", \"s\", \"e\", \"s\"],\n  [\"s\", \"s\", \"s\", \"e\", \"e\"],\n  [\"e\", \"e\", \"s\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[1,2,0,1,0],[0,0,3,0,2],[2,3,3,0,0],[0,0,3,0,0]]\nSolution\n[[0,s,0,0,0],[0,0,s,0,s],[s,s,s,0,0],[0,0,s,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n[\"*\", \"2\", \"*\", \"*\"],\n[\"*\", \"s\", \"3\", \"*\"],\n[\"*\", \"*\", \"*\", \"*\"],\n[\"1\", \"s\", \"3\", \"*\"]\n],\n\"think\": \"1. The '2' at [0, 1] has one revealed mine adjacent to it at [1, 1]. Thus, there must be one more mine adjacent to it. Since the other adjacent cells are [0, 0], [0, 2], [1, 0], and [1, 2], and [1, 2] is adjacent to '3' at [1, 2] which already has a mine at [1, 1], the mine must be at [0, 0], [0, 2], or [1, 0].\\n2. The '3' at [1, 2] has one revealed mine adjacent to it at [1, 1]. Thus, there must be two more mines adjacent to it. The adjacent cells are [0, 1], [0, 2], [0, 3], [1, 1], [1, 3], [2, 1], [2, 2], [2, 3]. Since [0, 1] has a '2' and already has a mine adjacent to it, it cannot be a mine. Thus, the two mines must be among [0, 2], [0, 3], [1, 3], [2, 1], [2, 2], [2, 3].\\n3. The '1' at [3, 0] has one revealed mine adjacent to it at [3, 1]. Thus, all other adjacent cells must be empty. \\n4. The '3' at [3, 2] has one revealed mine adjacent to it at [3, 1]. Thus, there must be two more mines adjacent to it. The adjacent cells are [2, 1], [2, 2], [2, 3], [3, 1], [3, 3]. Since [3, 1] is already a mine, the two mines must be among [2, 1], [2, 2], [2, 3], [3, 3].\",\n\"answer\": [\n[\"s\", \"2\", \"s\", \"e\"],\n[\"e\", \"s\", \"3\", \"s\"],\n[\"e\", \"e\", \"s\", \"e\"],\n[\"1\", \"s\", \"3\", \"s\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,2,0,0],[0,s,3,0],[0,0,0,0],[1,s,3,0]]\nSolution\n[[s,2,s,0],[0,s,3,s],[0,0,s,0],[1,s,3,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"*\", \"*\", \"*\", \"*\"],\n  [\"1\", \"*\", \"*\", \"2\"],\n  [\"2\", \"*\", \"*\", \"*\"],\n  [\"*\", \"s\", \"3\", \"*\"]\n],\n\"answer\": [\n  [\"e\", \"e\", \"s\", \"e\"],\n  [\"1\", \"s\", \"s\", \"2\"],\n  [\"2\", \"s\", \"s\", \"e\"],\n  [\"e\", \"s\", \"3\", \"s\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[1,0,0,2],[2,0,0,0],[0,s,3,0]]\nSolution\n[[0,0,s,0],[1,0,0,2],[2,s,s,0],[0,s,3,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"1\", \"*\", \"*\", \"1\"],\n  [\"1\", \"s\", \"3\", \"*\"],\n  [\"2\", \"*\", \"*\", \"*\"],\n  [\"1\", \"s\", \"3\", \"*\"]\n],\n\"answer\": [\n  [\"e\", \"s\", \"s\", \"e\"],\n  [\"e\", \"s\", \"s\", \"s\"],\n  [\"e\", \"s\", \"e\", \"e\"],\n  [\"e\", \"s\", \"s\", \"s\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[1,0,0,1],[1,s,3,0],[2,0,0,0],[1,s,3,0]]\nSolution\n[[0,s,s,0],[0,s,s,s],[0,s,0,0],[0,s,s,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/futoshiki/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/futoshiki/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/futoshiki/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/hitori/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/hitori/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..4ba6b1976e694a26b0d85125fc2f438dbf73a9b8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/hitori/filter_prompt.json
@@ -0,0 +1,79 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only three types values e or s or numbers; if you see 0, you should convert them into e. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [3, 2, 2, 1],\n  [4, 1, 3, 1],\n  [2, 2, 1, 4],\n  [1, 2, 1, 3]\n],\n\"think\": \"To solve this Hitori puzzle, I'll look for cells that must be shaded to avoid duplicates in rows and columns. The 1 in the bottom left corner must be shaded since there are two other 1s in its row and column. This leaves the 3 in the top left unshaded. Similarly, the 4 in the bottom right must be shaded to avoid duplicating the 4 in the second row. The 3 in the second row can then remain unshaded. For the 2s in the second and third rows, I'll shade the one in the top right corner to leave the other two 2s unshaded. This ensures all unshaded cells are connected.\",\n\"answer\": [\n  [\"e\", \"e\", \"e\", \"e\"],\n  [\"e\", \"e\", \"e\", \"s\"],\n  [\"e\", \"e\", \"e\", \"s\"],\n  [\"s\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[3,2,2,1],[4,1,3,1],[2,2,1,4],[1,2,1,3]]\nSolution\n[[0,0,0,0],[0,0,0,s],[0,0,0,s],[s,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [[1, 3, 4, 3], \n               [3, 4, 2, 1],\n               [1, 3, 1, 4], \n               [4, 1, 3, 3]],\n\"answer\": [[\"e\", \"s\", \"e\", \"s\"],\n           [\"s\", \"e\", \"s\", \"e\"], \n           [\"e\", \"s\", \"e\", \"s\"],\n           [\"s\", \"e\", \"s\", \"e\"]]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[1,3,4,3],[3,4,2,1],[1,3,1,4],[4,1,3,3]]\nSolution\n[[0,s,0,s],[s,0,s,0],[0,s,0,s],[s,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content":"**Game State:**\n\n{\"perception\": [[1, 3, 4, 3], [3, 4, 2, 1], [1, 3, 1, 4], [4, 1, 3, 3]]}\n\n**Thinking Process:**\n\nTo solve this Hitori puzzle, we need to identify the numbers that appear more than once in each row and column among unshaded cells. We will start by looking for these numbers in the first row. The numbers 1, 3, and 4 appear more than once in the first row, so we can shade the cell with number 1.\n\nNext, we will look at the second row. The numbers 2 and 4 appear more than once in the second row, so we can shade the cell with number 2.\n\nThen, we will examine the third row. The numbers 1 and 3 appear more than once in the third row, so we can shade the cell with number 1.\n\nFinally, we will analyze the fourth row. The numbers 3 and 4 appear more than once in the fourth row, so we can shade the cell with number 3.\n\n**Solution:**\n\n{\"answer\": [[s, s, 4, s], [s, s, s, s], [s, 3, s, s], [4, s, s, s]]}\n\nNote: The \"s\" indicates a shaded cell, and the \"e\" indicates an empty cell."
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[1,3,4,3],[3,4,2,1],[1,3,1,4],[4,1,3,3]]\nSolution\n[[s,s,4,s],[s,s,s,s],[s,3,s,s],[4,s,s,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes."
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n\"perception\": [\n[3, 2, 2, 1],\n[4, 1, 3, 1],\n[2, 2, 1, 4],\n[1, 2, 1, 3]\n],\n\"answer\": [\n[s, s, s, s],\n[s, s, s, s],\n[s, s, s, s],\n[s, s, s, s]\n]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[3,2,2,1],[4,1,3,1],[2,2,1,4],[1,2,1,3]]\nSolution\n[[s,s,s,s],[s,s,s,s],[s,s,s,s],[s,s,s,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n  \"perception\": [[1, 3, 4, 3], [3, 4, 2, 1], [1, 3, 1, 4], [4, 1, 3, 3]],\n  \"think\": \"I need to shade cells such that no number appears more than once in each row and column among unshaded cells, while ensuring shaded cells are not adjacent and that all unshaded cells are connected. I will start by analyzing each row and column for duplicate numbers and possible shading without breaking the adjacency rule. I aim to shade strategically to meet all conditions.\",\n  \"answer\": [[\"e\", \"s\", \"e\", \"s\"], [\"s\", \"e\", \"e\", \"e\"], [\"e\", \"s\", \"e\", \"s\"], [\"s\", \"e\", \"s\", \"e\"]]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[1,3,4,3],[3,4,2,1],[1,3,1,4],[4,1,3,3]]\nSolution\n[[0,s,0,s],[s,0,0,0],[0,s,0,s],[s,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes."
+    },
+    {
+        "role": "user", 
+        "content": "**Step 1: Identify the Problem**\n\nThe problem is a Hitori game, where we need to shade some cells in the grid such that no number appears more than once in each row and column among unshaded cells.\n\n**Step 2: Analyze the Game State**\n\nThe current game state is represented by a 4x4 grid with some numbers already filled in. The goal is to determine which cells need to be shaded to satisfy the rules of the game.\n\n**Step 3: Determine the Solution**\n\nTo solve this problem, we need to analyze the game state and determine which cells can be shaded without violating the rules of the game. We will use the following steps:\n\n*   Identify the numbers that appear more than once in each row and column.\n*   Determine which cells need to be shaded to eliminate duplicate numbers.\n*   Ensure that shaded cells are not adjacent and all unshaded cells are connected.\n\n**Step 4: Implement the Solution**\n\nBased on the analysis, we can determine the solution as follows:\n\n```\n\"perception\": [[1, 3, 1, 4], [4, 2, 1, 3], [4, 1, 4, 2], [2, 4, 3, 3]],\n\"think\": \"We need to shade the cells that contain duplicate numbers in each row and column. We also need to ensure that shaded cells are not adjacent and all unshaded cells are connected.\",\n\"answer\": [[\"s\", \"e\", \"s\", \"e\"], [\"e\", \"s\", \"s\", \"e\"], [\"e\", \"s\", \"e\", \"s\"], [\"s\", \"e\", \"s\", \"e\"]]\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[1,3,1,4],[4,2,1,3],[4,1,4,2],[2,4,3,3]]\nSolution\n[[s,0,s,0],[0,s,s,0],[0,s,0,s],[s,0,s,0]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/jigsawsudoku/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/jigsawsudoku/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/jigsawsudoku/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/kakurasu/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/kakurasu/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..af93d0b16cc3e4e78e2b3024dbde627daf35fd3e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/kakurasu/filter_prompt.json
@@ -0,0 +1,80 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0 or s; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"]\n],\n\"think\": \"To solve this Kakurasu puzzle, I need to shade some cells in the grid such that the sum of the weights of the shaded cells in each row and column matches the given clues. The weights increase from left to right and top to bottom, starting from 1. I'll analyze the clues row by row and column by column to determine which cells need to be shaded.\",\n\"answer\": [\n    [\"e\", \"s\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"s\"],\n    [\"s\", \"e\", \"s\", \"e\"],\n    [\"e\", \"s\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,s,0,0],[0,0,0,s],[s,0,s,0],[0,s,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"s\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"*\", \"*\"], \n  [\"*\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"*\", \"*\"]\n],\n\"think\": \"The first row has a clue of 4, which means the sum of the weights of the shaded cells in that row must be 4. Since the weights increase from left to right, starting with 1, the only way to get a sum of 4 in the first row is to shade the leftmost cell with a weight of 1. For the other rows and columns, there is not enough information yet to determine which cells to shade.\",\n\"answer\": [\n  [\"s\", \"e\", \"e\", \"e\"],\n  [\"e\", \"e\", \"e\", \"e\"],\n  [\"e\", \"e\", \"e\", \"e\"], \n  [\"e\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[s,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[s,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"]\n  ],\n  \"think\": \"The Kakurasu puzzle requires shading cells such that the sum of the weights of shaded cells in each row and column equals the given clue. The weights are the cell's coordinates (starting from 1). Let's analyze the clues:\\n\\nRow 0: 3.  Possible combinations are 1+2, 3. \\nRow 1: 0. No cells are shaded.\\nRow 2: 9. Only combination is 1+2+3+4.\\nRow 3: 2. Possible combinations are 2, 1+1 (but this is not possible).\\n\\nColumn 0: 0. No cells are shaded.\\nColumn 1: 7. Possible combinations are 1+2+4, 3+4.\\nColumn 2: 4. Possible combinations are 4, 1+3, 2+2.\\nColumn 3: 3. Possible combinations are 3, 1+2.\\n\\nLet's start with Row 2 (sum 9). This means all cells in Row 2 must be shaded. This also helps us deduce Column 1 and Column 2. Since Row 2 is fully shaded, Column 1 must have 7 (2+3+4) and Column 2 must have 4 (3+1). This leaves only 3 for Column 3.  Row 0 must be 1+2, and Row 3 must be 2. This gives us the solution.\",\n  \"answer\": [\n    [\"e\", \"s\", \"s\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\"],\n    [\"s\", \"s\", \"s\", \"s\"],\n    [\"e\", \"e\", \"e\", \"s\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,s,s,0],[0,0,0,0],[s,s,s,s],[0,0,0,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n    \"perception\": [\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"think\": \"Given the row clues are [3, 7, 4, 3] and the column clues are [3, 0, 9, 2], we need to fill the cells with 's' only if the sum of the weights (indexed from 1) from left to right and top to bottom matches the clue for that particular row or column. The second row must sum up to 7, where the only combination is [2, 3] in columns 0 and 2. The third row sums up to 4 and the only option is choosing column 0, the fourth row clue is 3 which can be satisfied only by choosing column 0. For columns, the third column needs to sum up to 9, achieved by choosing cells in rows 1 and 3. Each row and column must match the required sums without overfilling or misplacing any shaded cells.\",\n    \"answer\": [\n        [\"e\", \"e\", \"s\", \"e\"],\n        [\"s\", \"e\", \"s\", \"e\"],\n        [\"s\", \"e\", \"e\", \"e\"],\n        [\"s\", \"e\", \"e\", \"e\"]\n    ]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]]\nSolution\n[[0,0,s,0],[s,0,s,0],[s,0,0,0],[s,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user", 
+        "content": "**Problem Statement**\n\nThe problem is a Kakurasu puzzle, where the goal is to shade some cells in a grid such that the sum of the weights of selected cells in each row and column matches the given clues. The weights increase from left to right (for rows) and top to bottom (for columns), starting from 1. Indexing starts at 0.\n\n**Current Game State**\n\nThe current game state is represented as a 2D array:\n\n```json\n{\"perception\": [[\"1,1\",\"1,2\",\"3\",\"7\",\"1\",\"4\"], [\"2,1\",\"2,2\",\"2,3\",\"2,4\",\"3,1\",\"3,4\",\"4,1\",\"4,2\",\"4,3\",\"4,4\"]]}\n```\n\n**Thinking Process**\n\nTo solve this problem, I will use a systematic approach to try different combinations of shaded cells.\n\n*   Start with the first row and try to find a combination of shaded cells that satisfies the weight constraint.\n*   Move to the second row and try to find a combination of shaded cells that satisfies the weight constraint.\n*   Repeat the process for all rows.\n*   Once all rows are satisfied, move to the columns and repeat the process.\n*   Continue this process until all constraints are satisfied or it becomes clear that there is no solution.\n\n**Solution**\n\nAfter trying different combinations, I found a solution that satisfies all constraints:\n\n```json\n{\"answer\": [[\"s\",\"e\",\"e\",\"e\",\"e\",\"e\"],[\"e\",\"e\",\"s\",\"e\",\"e\",\"e\"],[\"e\",\"e\",\"e\",\"s\",\"e\",\"e\"],[\"e\",\"e\",\"e\",\"e\",\"s\",\"e\"],[\"e\",\"e\",\"e\",\"e\",\"e\",\"s\"],[\"e\",\"e\",\"e\",\"e\",\"e\",\"e\"]]}\n```\n\nThis solution satisfies all constraints and is a valid solution to the Kakurasu puzzle.\n\n**Conclusion**\n\nThe solution to the Kakurasu puzzle is a 2D array where each cell is either shaded (s) or unshaded (e). The shaded cells are located in the first, third, and fifth rows, as well as in the fourth and sixth columns. This solution satisfies all constraints and is a valid solution to the puzzle."
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. Because other numbers like 1 and 2 is undefined, so put a Wrong there"
+    },
+    {
+        "role": "user", 
+        "content": "```json\n{\n\"perception\": [\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0]\n],\n\"think\": \"The puzzle has not been started yet, so there are no shaded cells.\",\n\"answer\": [\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0],\n[0, 0, 0, 0, 0, 0]\n]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0]]\nSolution\n[[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes."
+    }
+
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/kakuro/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/kakuro/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..8b7da246d4d4b1830a76ba54297945affdf85104
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/kakuro/filter_prompt.json
@@ -0,0 +1,54 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"4\", \"2\", \"*\", \"2\"],\n  [\"*\", \"4\", \"1\", \"4\"],\n  [\"3\", \"*\", \"4\", \"*\"],\n  [\"4\", \"2\", \"*\", \"*\"]\n],\n\"think\": \"To solve this Kakuro puzzle, I will analyze the given clues and the constraints of the game. First, I'll look at the rows and columns with only one missing value and try to deduce that value based on the given sum and the existing values. Then, I'll move on to the more complex rows and columns, using logical deduction and the rule that adjacent numbers cannot be the same. I'll continue this process until I have filled in all the empty cells.\",\n\"answer\": [\n  [\"4\", \"2\", \"3\", \"2\"],\n  [\"8\", \"4\", \"1\", \"4\"],\n  [\"3\", \"6\", \"4\", \"5\"],\n  [\"4\", \"2\", \"7\", \"3\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[4,2,0,2],[0,4,1,4],[3,0,4,0],[4,2,0,0]]\nSolution\n[[4,2,3,2],[8,4,1,4],[3,6,4,5],[4,2,7,3]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes"
+    },
+    {
+        "role": "user",
+        "content": "{\"perception\": [[3, 9, 12, 10], [1, 4, 2, 14], [7, 3, 1, 4], [4, 3, 2, 11]], \n\"think\": \"The problem is a constrained optimization problem. We want to fill in the grid with numbers 1 to 4 such that each row and column adds up to the specified sum. We also want to avoid having adjacent numbers be the same. This is a classic example of a constraint satisfaction problem. Our goal is to find a solution that satisfies all the constraints. We can start by analyzing the given information and looking for any patterns or clues that can help us narrow down the possible solutions. For example, in the first row, the sum is 12 and the middle cell is empty. This means that the sum of the two adjacent cells must be 12. If we try different combinations, we can see that the only possibility is 3 + 9 = 12. Similarly, in the second row, the sum is 14 and the middle cell is empty. This means that the sum of the two adjacent cells must be 14. If we try different combinations, we can see that the only possibility is 4 + 10 = 14. We can continue this process for each row and column to find the possible values for the empty cells. Once we have found the possible values for each cell, we can use constraint propagation to eliminate any impossible values and narrow down the search space. Finally, we can use a backtracking search algorithm to find a solution that satisfies all the constraints.\", \n\"answer\": [[3, 9, 12, 10], [1, 4, 2, 14], [7, 3, 1, 4], [4, 3, 2, 11]]}"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[3,9,12,10],[1,4,2,14],[7,3,1,4],[4,3,2,11]]\nSolution\n[[3,9,12,10],[1,4,2,14],[7,3,1,4],[4,3,2,11]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"4\", \"1\", \"*\"],\n    [\"*\", \"3\", \"*\", \"*\"],\n    [\"2\", \"*\", \"*\", \"*\"],\n    [\"*\", \"2\", \"*\", \"3\"]\n  ],\n  \"answer\": [\n    [\"3\", \"4\", \"1\", \"2\"],\n    [\"4\", \"3\", \"2\", \"1\"],\n    [\"2\", \"1\", \"4\", \"3\"],\n    [\"1\", \"2\", \"3\", \"4\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,4,1,0],[0,3,0,0],[2,0,0,0],[0,2,0,3]]\nSolution\n[[3,4,1,2],[4,3,2,1],[2,1,4,3],[1,2,3,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Right."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [4, \"*\", \"*\", \"*\", \"*\"],\n    [2, \"*\", \"*\", \"*\", 3],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [2, 3, \"*\", \"*\", \"*\"]\n  ],\n  \"answer\": [\n    [4, 3, 2, 1, 4],\n    [2, 4, 1, 3, 3],\n    [1, 2, 4, 3, 2],\n    [2, 3, 1, 4, 0]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[4,0,0,0,0],[2,0,0,0,3],[0,0,0,0,0],[2,3,0,0,0]]\nSolution\n[[4,3,2,1,4],[2,4,1,3,3],[1,2,4,3,2],[2,3,1,4,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Right."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/killersudoku/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/killersudoku/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..fd9a6d57adbee1b886bd5ada473e6adfe08469af
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/killersudoku/filter_prompt.json
@@ -0,0 +1,90 @@
+[
+    {
+        "role": "system", 
+       "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"*\", 2, \"*\", \"*\"],\n  [\"*\", \"*\", 5, \"*\"],\n  [\"*\", \"*\", 3, \"*\"],\n  [5, \"*\", \"*\", 2]\n],\n\"think\": \"To solve this KillerSudoku puzzle, I will first analyze the given numbers and cages to identify any constraints or deductions that can be made. Then, I will use logic and the rules of Sudoku to fill in the remaining empty cells.\n\nThe teal cage in the top left has a target sum of 9, with one cell already containing 2. So the other cell must be 7.\n\nThe yellow cage has a target sum of 4, with one cell containing 1. So the other cell must be 3.\n\nThe purple cage has a target sum of 5, but no numbers are given, so I cannot deduce anything yet.\n\nThe red cage has a target sum of 7, with one cell containing 5. So the other cell must be 2.\n\nThe blue cage has a target sum of 4, with one cell containing 4, so the other cell must be filled with the remaining number, which is 1.\n\nThe orange cage has no target sum given, but one cell contains 2. The other cell must be either 1, 3, or 4 based on the remaining numbers.\n\nLooking at the rows, columns, and 2x2 blocks, I can place the remaining numbers while satisfying the Sudoku rules.\",\n\"answer\": [\n  [7, 2, 1, 4],\n  [3, 4, 5, 1],\n  [1, 3, 2, 4],\n  [5, 1, 4, 2]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,2,0,0],[0,0,5,0],[0,0,3,0],[5,0,0,2]]\nSolution\n[[7,2,1,4],[3,4,5,1],[1,3,2,4],[5,1,4,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [[null, null, null, null], [null, 1, null, null], [null, null, 2, null], [null, 4, null, 3]],\n\"think\": \"Let's start by filling some obvious cells.\\nThe top-left cell must be 2 since it's a single-cell cage with target 7.\\nThe cell at (1, 0) must be 3 since the cage has a target of 4 and the cell at (1, 1) is 1.\\nThe cell at (0, 1) must be 4 since it's a single-cell cage with target 5.\\nThe cell at (0, 3) must be 3 since it's a single-cell cage with target 3.\\nThe cell at (1, 2) must be 5 since the cage has a target of 6 and the cell at (1, 1) is 1.\\nThe cell at (2, 0) must be 1 since it's a single-cell cage with target 3.\\nThe cell at (3, 0) must be 2 since it's a single-cell cage with target 5 and the cell at (3, 1) is 4.\\nThe cell at (2, 3) must be 4 since it's a single-cell cage with target 7 and the cell at (2, 2) is 2.\\nThe cell at (3, 2) must be 1 since it's a single-cell cage with target 5 and the cell at (3, 1) is 4.\\nThe cell at (1, 3) must be 4 since it's a single-cell cage with target 3 and the cell at (0, 3) is 3.\\nThe cell at (2, 1) must be 3 since the cage has a target of 3 and the cell at (2, 0) is 1.\\nThe cell at (0, 2) must be 2 since the cage has a target of 3 and the cell at (0, 3) is 3.\",\n\"answer\": [[2, 4, 2, 3], [3, 1, 5, 4], [1, 3, 2, 4], [2, 4, 1, 3]]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0],[0,1,0,0],[0,0,2,0],[0,4,0,3]]\nSolution\n[[2,4,2,3],[3,1,5,4],[1,3,2,4],[2,4,1,3]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n[3, 7, 5, 4],\n[6, 1, 2, 5],\n[4, 3, 7, 6],\n[8, 9, 10, 11]\n],\n\"answer\": [\n[3, 7, 5, 4],\n[6, 1, 2, 5],\n[4, 3, 7, 6],\n[8, 9, 10, 11]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[3,7,5,4],[6,1,2,5],[4,3,7,6],[8,9,10,11]]\nSolution\n[[3,7,5,4],[6,1,2,5],[4,3,7,6],[8,9,10,11]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content": "**Solution to the Killer Sudoku Puzzle**\n\n**Current Game State:**\n\n```\n[2, 4, 3, 5, \n1, 5, 7, 4, \n3, 5, 2, 1, \n9, 4, 9, 8]\n```\n\n**Thinking Process:**\n\nTo solve this Killer Sudoku puzzle, we need to fill in the missing numbers in the grid while satisfying the constraints of each cage. Since the target sums for the cages are not provided, we will focus on using the given numbers to deduce the correct placement.\n\nFirst, let's analyze the top-left 2x2 block. The sum of the numbers in this block is 2 + 4 + 3 + 5 = 14. We can see that the numbers 2, 4, 3, and 5 are already present, leaving only the number 9 to complete the block.\n\nNext, let's examine the middle-left 2x2 block. The sum of the numbers in this block is 1 + 5 + 7 + 4 = 17. We can see that the numbers 1, 5, 7, and 4 are already present, leaving only the number 9 to complete the block.\n\nNow, let's look at the bottom-left 2x2 block. The sum of the numbers in this block is 3 + 5 + 2 + 1 = 11. We can see that the numbers 3, 5, 2, and 1 are already present, leaving only the number 9 to complete the block.\n\nFinally, let's examine the top-right 2x2 block. The sum of the numbers in this block is 9 + 4 + 9 + 8 = 30. We can see that the numbers 9, 4, 9, and 8 are already present, leaving only the number 9 to complete the block.\n\n**Solution:**\n\n```\n[2, 4, 3, 9, \n1, 5, 7, 4, \n3, 5, 2, 1, \n9, 4, 9, 8]\n```\n\nThis solution satisfies all the constraints of the puzzle, including the unique numbers in each row, column, and 2x2 block, as well as the target sums for each cage."
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[2,4,3,5],[1,5,7,4],[3,5,2,1],[9,4,9,8]]\nSolution\n[[2,4,3,9],[1,5,7,4],[3,5,2,1],[9,4,9,8]]"
+    },
+    {
+        "role": "user",
+        "content": "Right "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [[2,1,4,8],[3,2,1,0],[3,2,1,0],[5,4,5,2]],\n\"answer\": [[2,1,4,8],[4,2,1,3],[1,3,2,5],[3,5,2,4]]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[2,1,4,8],[3,2,1,0],[3,2,1,0],[5,4,5,2]]\nSolution\n[[2,1,4,8],[4,2,1,3],[1,3,2,5],[3,5,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Right"
+    },
+    {
+        "role": "user",
+        "content":"```json\n{\n\"answer\": [[2, 1, 3, 4],\n           [3, 2, 4, 1],\n           [4, 3, 1, 2],\n           [1, 4, 2, 3]]\n}\n```\n\nThis solution satisfies all the rules:\n1. Each row, column, and 2x2 block contains unique digits from 1 to 4.\n2. The sum of numbers in each cage matches the target sum:\n   - Cage with sum 9: 2 + 1 + 3 = 6\n   - Cage with sum 4: 1 + 3 = 4\n   - Cage with sum 5: 4 + 1 = 5\n   - Cage with sum 5: 2 + 3 = 5\n   - Cage with sum 7: 4 + 1 + 2 = 7\n   - Cage with sum 5: 2 + 3 = 5\n   - Cage with sum 5: 1 + 4 = 5"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\nWrong\nSolution\n[[2,1,3,4],[3,2,4,1],[4,3,1,2],[1,4,2,3]]"
+    },
+    {
+        "role": "user",
+        "content": "Right, "
+    },
+    {
+        "role": "user",
+        "content":"```json\n{\n\"answer\": [\n[2, 1, 3, 4],\n[9, 3, 6, 3],\n[5, 8, 1, 4],\n[1, 4, 7, 2]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\nWrong\nSolution\n[[2,1,3,4],[9,3,6,3],[5,8,1,4],[1,4,7,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Right, "
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/lightup/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/lightup/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..d8a6c4dffb249a475f82f3861088f957ef4db2e6
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/lightup/filter_prompt.json
@@ -0,0 +1,92 @@
+[
+    {
+        "role": "system", 
+        "content": 
+        "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only w or 0 or s; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If you see numbers other than 2, you should convert them to w. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n    [\"s\", \"w\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"w\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"]\n],\n\"answer\": [\n    [\"s\", \"w\", \"e\", \"s\"],\n    [\"e\", \"s\", \"e\", \"e\"],\n    [\"e\", \"e\", \"w\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[s,w,0,0],[0,0,0,0],[0,0,w,0],[0,0,0,0]]\nSolution\n[[s,w,0,s],[0,s,0,0],[0,0,w,0],[0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "The current state of the grid in the screenshot is:\n\n```\n{\n\"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"2\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"]\n],\n\"answer\": [\n    [\"s\", \"s\", \"e\", \"e\", \"e\"],\n    [\"s\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"s\"],\n    [\"e\", \"e\", \"e\", \"s\", \"e\"]\n]\n}\n```\n\nThis solution satisfies all the rules of the game. The numbered wall has exactly two bulbs adjacent to it, and all empty cells are illuminated. The bulbs do not illuminate each other."
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,w,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[s,s,0,0,0],[s,0,0,0,0],[0,0,0,0,0],[0,0,0,0,s],[0,0,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [[\"s\", \"2\", \"*\", \"*\"], [\"w\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"w\", \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n\"answer\": [[\"s\", \"e\", \"s\", \"e\"], [\"e\", \"e\", \"e\", \"s\"], [\"s\", \"e\", \"e\", \"e\"], [\"e\", \"s\", \"e\", \"e\"]]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[s,w,0,0],[w,0,0,0],[0,0,w,0],[0,0,0,0]]\nSolution\n[[s,0,s,0],[0,0,0,s],[s,0,0,0],[0,s,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"s\", \"*\", \"w\", \"*\"],\n    [\"w\", \"2\", \"*\", \"*\"],\n    [\"*\", \"*\", \"w\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\"]\n  ],\n  \"think\": \"1. Start by placing bulbs around the '2' wall. Positions (0,1) and (1,2) work because they don't conflict with any other constraints and fulfill the wall's requirement.\\n2. Check other empty cells to ensure they are illuminated without overlapping bulbs.\\n3. Place bulbs in (2,0) and (3,2) to cover remaining empty areas left by previous placements, ensuring no bulbs are facing directly toward each other.\",\n  \"answer\": [\n    [\"s\", \"s\", \"w\", \"s\"],\n    [\"w\", \"2\", \"s\", \"e\"],\n    [\"s\", \"e\", \"w\", \"s\"],\n    [\"e\", \"e\", \"s\", \"e\"]\n  ]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[s,0,w,0],[w,w,0,0],[0,0,w,0],[0,0,0,0]]\nSolution\n[[s,s,w,s],[w,w,s,0],[s,0,w,s],[0,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "The current state of the grid as a 2D array is:\n```json\n[\n    [\".\", \"2\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"],\n    [\"s\", \".\", \"s\"]\n]\n```\nThe perception is that all cells are empty except for the two cells marked with \"2\", which must have exactly that number of bulbs adjacent to them. The numbered walls must have exactly that many bulbs adjacent to them, and bulbs cannot illuminate each other.\n\nThe think is that the only possible solution is to place a bulb in the cell adjacent to the \"2\" on the top left corner, and then place bulbs in the cells adjacent to the \"2\" on the top right corner. This will illuminate all empty cells without violating any of the rules.\n\nThe answer is:\n```json\n[\n    [\"s\", \"2\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\"]\n]\n```\nThis solution satisfies all the rules and illuminates all empty cells."
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,w,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s],[s,0,s]]\nSolution\n[[s,w,s],[s,s,s],[s,s,s],[s,s,s],[s,s,s],[s,s,s],[s,s,s],[s,s,s],[s,s,s],[s,s,s]]"
+
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n    [\"*\", \"w\", \"*\", \"*\"],\n    [\"*\", \"2\", \"*\", \"*\"], \n    [\"s\", \"*\", \"w\", \"*\"],\n    [\"*\", \"s\", \"*\", \"*\"]\n],\n\"answer\": [\n    [\"e\", \"w\", \"s\", \"e\"],\n    [\"e\", \"2\", \"e\", \"e\"],\n    [\"s\", \"e\", \"w\", \"e\"], \n    [\"e\", \"s\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,w,0,0],[0,w,0,0],[s,0,w,0],[0,s,0,0]]\nSolution\n[[0,w,s,0],[0,2,0,0],[s,0,w,0],[0,s,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },   
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [[\"*\", \"2\", \"*\", \"*\", \"*\"], [\"w\", \"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"w\", \"*\", \"*\"], [\"*\", \"s\", \"*\", \"*\", \"*\"]],\n\"answer\": [[\"e\", \"2\", \"e\", \"s\", \"e\"], [\"w\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"w\", \"e\", \"e\"], [\"e\", \"s\", \"e\", \"e\", \"e\"]]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,w,0,0,0],[w,0,0,0,0],[0,0,w,0,0],[0,s,0,0,0]]\nSolution\n[[0,w,0,s,0],[w,0,0,0,0],[0,0,w,0,0],[0,s,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    }   
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/nonogram/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/nonogram/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..e4f9dcfaf69fbcb9da630be5ce89e1e38a5ce649
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/nonogram/filter_prompt.json
@@ -0,0 +1,78 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0 or s; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"s\", \"s\"]\n],\n\"think\": \"The image shows a Nonogram puzzle grid with 5 rows and 5 columns. The numbers on the side indicate the lengths of consecutive filled cells in each row and column.\n\nFor the rows:\n- Row 0 has no numbers, so it should be all empty cells.\n- Row 1 also has no numbers, so it should be all empty cells.\n- Row 2 has the number 3, indicating a block of 3 consecutive filled cells.\n- Row 3 has the number 4, indicating a block of 4 consecutive filled cells.\n- Row 4 has the number 5, and the last two cells are already filled, so the first 3 cells should also be filled.\n\nFor the columns:\n- Column 0 has the number 3, indicating a block of 3 consecutive filled cells.\n- Column 1 has the number 3, indicating a block of 3 consecutive filled cells.\n- Column 2 has the number 3, indicating a block of 3 consecutive filled cells.\n- Column 3 has the number 2, indicating a block of 2 consecutive filled cells.\n- Column 4 has the number 1, indicating a single filled cell, which is already given.\n\nBy combining the row and column constraints, we can deduce the solution.\n\",\n\"answer\": [\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\", \"s\", \"e\"],\n    [\"s\", \"s\", \"s\", \"s\", \"s\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,s,s]]\nSolution\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,s,s],[s,s,s,s,0],[s,s,s,s,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"]\n  ],\n  \"think\": \"Row 0: The clue is 5, meaning all cells in this row must be filled.\\nColumn 0: The clue is 3. Since row 0 is filled, we can fill the first three cells of column 0.\\nColumn 1: The clue is 3. Since row 0 is filled, we can fill the first three cells of column 1.\\nColumn 2: The clue is 3. Since row 0 is filled, we can fill the first three cells of column 2.\\nRow 1: The clue is 4. Since the first three cells of columns 0, 1, and 2 are filled, the first three cells of row 1 must also be filled. The last cell must be empty because column 4 has a clue of 1 and row 0 is already filled.\\nRow 2: The clue is 3. Since the first three cells of columns 0, 1, and 2 are filled, the first three cells of row 2 must also be filled.\\nColumn 3: The clue is 2. Since rows 0 and 1 are filled at column 3, and row 2 is filled in the first three cells, the two filled cells must be in rows 0 and 1.\\nColumn 4: The clue is 1. Since row 0 is filled, the filled cell must be in row 0.\\nRow 3 and Row 4: The clue is 0, meaning all cells in these rows must be empty.\",\n  \"answer\": [\n    [\"s\", \"s\", \"s\", \"s\", \"s\"],\n    [\"s\", \"s\", \"s\", \"s\", \"e\"],\n    [\"s\", \"s\", \"s\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"]\n  ]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[s,s,s,s,s],[s,s,s,s,0],[s,s,s,0,0],[0,0,0,0,0],[0,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"]\n],\n\"answer\": [\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [[\"s\", \"s\", \"s\", \"e\", \"s\", \"s\", \"e\", \"s\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"s\", \"s\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"s\", \"s\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"]],\n  \"think\": \"Based on the clues, I need to fill in the rows and columns accordingly. The numbers on the left represent the blocks of filled cells in that row. For instance, row 0 has '3' meaning three filled cells together, which I placed in the first three cells. Row 1 has no filled cells according to the numbers. For row 2, it has '2', indicating two blocks but only one block (2 cells) is filled. I continued this way for each row and then confirmed the columns were also consistent with the clues provided.\",\n  \"answer\": [[\"s\", \"s\", \"s\", \"e\", \"s\", \"s\", \"e\", \"s\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"s\", \"s\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"s\", \"s\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\", \"e\"]]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[s,s,s,0,s,s,0,s,0,0],[0,0,0,0,0,0,0,0,0,0],[0,0,s,s,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0],[0,0,0,0,s,s,0,0,0,0],[0,0,0,0,0,0,0,0,0,0]]\nSolution\n[[s,s,s,0,s,s,0,s,0,0],[0,0,0,0,0,0,0,0,0,0],[0,0,s,s,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0],[0,0,0,0,s,s,0,0,0,0],[0,0,0,0,0,0,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [[*3*, *3*, *3*, *3*],\n                  [*3*, 3, *3*, *3*],\n                  [*3*, *3*, *3*, *3*],\n                  [*3*, *3*, *3*, *3*]],\n  \"think\": \"Let's analyze each row one by one. The first row has a single 3-cell block, which suggests that all the cells in that row must be filled. The second row also has a single 3-cell block, so we can fill in the 3 cells from index 0 to 2. The third row has two 3-cell blocks, indicating that there must be at least one empty cell between them. The fourth row also has two 3-cell blocks, and as before, we can fill in the 3 cells from index 0 to 2. The last row has a single 3-cell block, which means that all the cells in that row must be filled.\",\n  \"answer\": [[0, 3, 0, 3],\n                  [3, 3, 0, 3],\n                  [0, 3, 0, 3],\n                  [3, 3, 0, 3]]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[3,3,3,3],[3,3,3,3],[3,3,3,3],[3,3,3,3]]\nSolution\n[[0,3,0,3],[3,3,0,3],[0,3,0,3],[3,3,0,3]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n[\"*\", \"*\", \"*\", \"s\", \"s\"],\n[\"*\", \"*\", \"*\", \"s\", \"*\"],\n[\"*\", \"*\", \"*\", \"*\", \"*\"],\n[\"*\", \"*\", \"*\", \"*\", \"*\"],\n[\"*\", \"*\", \"*\", \"*\", \"*\"]\n],\n\"answer\": [\n[\"e\", \"e\", \"s\", \"s\", \"s\"],\n[\"e\", \"e\", \"s\", \"s\", \"s\"],\n[\"e\", \"e\", \"s\", \"s\", \"s\"],\n[\"e\", \"e\", \"e\", \"e\", \"e\"],\n[\"e\", \"e\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {  
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,s,s],[0,0,0,s,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[0,0,s,s,s],[0,0,s,s,s],[0,0,s,s,s],[0,0,0,0,0],[0,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/oddevensudoku/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/oddevensudoku/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/oddevensudoku/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/renzoku/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/renzoku/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/renzoku/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/skyscraper/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/skyscraper/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/skyscraper/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/starbattle/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/starbattle/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..deca0f5c63613f9be51db2bc34e483fecdff6968
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/starbattle/filter_prompt.json
@@ -0,0 +1,49 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0 or s; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not."},
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"*\", \"*\", \"s\", \"s\", \"s\"],\n  [\"*\", \"*\", \"s\", \"s\", \"s\"],\n  [\"*\", \"*\", \"s\", \"s\", \"s\"],\n  [\"*\", \"*\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"*\", \"*\", \"*\"],\n  [\"s\", \"*\", \"*\", \"*\", \"*\"]\n],\n\"think\": \"Based on the current state of the grid, there is already a star placed in the bottom-left corner cell. To solve this puzzle, I need to place the remaining stars in such a way that no two stars are touching each other, not even diagonally, and each row, column, and region contains exactly one star. I'll analyze the grid row by row and column by column to identify the available cells where I can place the remaining stars.\",\n\"answer\": [\n  [\"e\", \"e\", \"s\", \"s\", \"s\"],\n  [\"e\", \"e\", \"s\", \"s\", \"s\"], \n  [\"e\", \"e\", \"s\", \"s\", \"s\"],\n  [\"e\", \"s\", \"e\", \"e\", \"e\"],\n  [\"s\", \"e\", \"e\", \"e\", \"e\"],\n  [\"s\", \"e\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,s,s,s],[0,0,s,s,s],[0,0,s,s,s],[0,0,0,0,0],[0,0,0,0,0],[s,0,0,0,0]]\nSolution\n[[0,0,s,s,s],[0,0,s,s,s],[0,0,s,s,s],[0,s,0,0,0],[s,0,0,0,0],[s,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": [\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"],\n    [\"s\", \"*\", \"*\", \"*\", \"*\"],\n    [\"*\", \"*\", \"*\", \"*\", \"*\"]\n  ],\n  \"think\": \"The given grid is a 5x5 Star Battle puzzle. The goal is to place stars such that each row, column, and 2x2 region contains exactly one star, and no two stars are adjacent (including diagonally).  I'll start by filling in the obvious spots, considering the constraints. The existing star in the bottom-left corner restricts placement in its vicinity. I'll proceed systematically, row by row, and column by column, checking for conflicts.\",\n  \"answer\": [\n    [\"e\", \"e\", \"s\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"s\"],\n    [\"e\", \"s\", \"e\", \"e\", \"e\"],\n    [\"s\", \"e\", \"e\", \"e\", \"e\"],\n    [\"e\", \"e\", \"e\", \"s\", \"e\"]\n  ]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[s,0,0,0,0],[0,0,0,0,0]]\nSolution\n[[0,0,s,0,0],[0,0,0,0,s],[0,s,0,0,0],[s,0,0,0,0],[0,0,0,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes."
+    },
+    {
+        "role": "user",
+        "content": "I'm sorry, I can't visualize the image directly. Could you please describe the grid or textually represent it so I can help you with the Star Battle solution?"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "**Current Game State**\n\nThe game board is a 4x5 grid with some cells containing stars and others being empty.\n\n```json\n{\"perception\": [[\"s\", \"*\", \"*\", \"*\", \"*\"], \n               [\"*\", \"*\", \"s\", \"*\", \"*\"], \n               [\"*\", \"s\", \"*\", \"s\", \"*\"], \n               [\"*\", \"*\", \"*\", \"*\", \"*\"]]}\n```\n\n**Thinking Process**\n\nTo solve this puzzle, we need to place stars in each row, column, and region such that no two stars touch each other, not even diagonally. We can start by identifying the regions where we can place the stars:\n\n* Region 1: Top-left 2x2 box\n* Region 2: Top-right 2x2 box\n* Region 3: Middle 2x2 box\n* Region 4: Bottom 2x2 box\n\nWe can also observe that the first column has only one star, so we need to place the remaining three stars in the other columns.\n\n**Solution**\n\nBased on the above analysis, here's our solution:\n\n```json\n{\"answer\": [[\"s\", \"e\", \"e\", \"e\", \"e\"], \n           [\"e\", \"s\", \"e\", \"e\", \"e\"], \n           [\"e\", \"e\", \"s\", \"e\", \"s\"], \n           [\"e\", \"e\", \"e\", \"e\", \"s\"]]}\n```\n\nIn this solution, we have placed stars in each row, column, and region, and made sure that no two stars touch each other."
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[s,0,0,0,0],[0,0,s,0,0],[0,s,0,s,0],[0,0,0,0,0]]\nSolution\n[[s,0,0,0,0],[0,s,0,0,0],[0,0,s,0,s],[0,0,0,0,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/sudoku/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/sudoku/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..228fa6bfe1a083c77deb93632cb770eda7c4bbd2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/sudoku/filter_prompt.json
@@ -0,0 +1,66 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only two types values 0 or numbers; if you see * or _  or - or e or #, you should convert them into 0. If you see other numbers such as 1, 2, or 3, you just put it there.If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not"
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[1, 2, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", 1, \"*\"], [\"*\", \"*\", \"*\", \"*\"]],\n  \"Thought\": \"To solve this Sudoku puzzle, I need to fill in the empty cells ensuring that each number from 1 to 4 appears exactly once in each row, column, and 2x2 block. I will start by identifying the possible numbers for each empty cell based on the existing ones.\",\n  \"Solution\": [[1, 2, 4, 3], [2, 3, 4, 1], [3, 4, 1, 2], [4, 1, 2, 4]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[1,2,0,0],[2,0,0,0],[0,0,1,0],[0,0,0,0]]\nAnswer\n[[1,2,4,3],[2,3,4,1],[3,4,1,2],[4,1,2,4]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "json\n{\n  \"Initial State\": [[4, 1, \"*\", \"*\"], [2, \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", 1]],\n  \"Thought\": \"The grid consists of a 4x4 layout divided into 2x2 blocks. Considering existing numbers, I will analyze which numbers can fit into the empty cells while ensuring each row, column, and 2x2 block adhere to Sudoku rules. I will start filling from top left to bottom right.\",\n  \"Solution\": [[4, 1, 3, 2], [2, 3, 1, 4], [1, 4, 2, 3], [3, 2, 4, 1]]\n}\n"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[4,1,0,0],[2,0,0,0],[0,0,0,0],[0,0,0,1]]\nAnswer\n[[4,1,3,2],[2,3,1,4],[1,4,2,3],[3,2,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"Initial State\": [\n[[\"*\"], [\"*\"], [\"*\"], [\"4\", \"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"1\"]], \n[[\"*\"], [\"*\"], [\"*\"], [\"*\"]], \n[[\"1\"], [\"*\"], [\"*\"], [\"3\"]]\n],\n\n\"Thought\": \"The grid is divided into 2x2 blocks. We need to ensure each block, row, and column contains unique numbers from 1 to 4. The current numbers are 1 and 3 in the top-right block and 4 and 1 in the top-left block. We need to fill the remaining cells with numbers 2 and 4 to make sure each block, row, and column has unique numbers. We can start with placing 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"Considering the existing numbers and the rule that each block, row, and column must contain unique numbers, we can place the numbers in the remaining cells. The numbers 2 and 4 are available to fill the cells. We need to ensure that each of these numbers is placed in a way that does not violate the Sudoku rules. For example, we can place 2 in the bottom-left cell of the top-right block and 4 in the bottom-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can place the number 2. In the bottom-right cell of the top-left block, we can place the number 4. Now, we need to fill the remaining cells in the bottom-left block of the top-right block and the top-right cell of the top-left block with the remaining numbers 1 and 3.\",\n\"Solution\": {\n\"Thought\": \"We can place the number 1 in the bottom-left cell of the top-right block and the number 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"The solution is to place the numbers as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block.\",\n\"Solution\": {\n\"Thought\": \"The solution to the Sudoku puzzle is as follows: 2 in the bottom-left cell of the top-right block, 4 in the bottom-right cell of the top-left block, 1 in the bottom-left cell of the top-right block, and 3 in the top-right cell of the top-left block. This will ensure that each block, row, and column contains unique numbers from 1 to 4.\",\n\"Solution\": {\n\"Thought\": \"In the bottom-left cell of the top-right block, we can\n``"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\nWrong\nAnswer\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes. as perception is not a 2D array and answer is missing. For perception, its first row is 3 single item and 1 2-item array, not valid. Thus, putting Wrong for both perception and answer."
+    },
+    {
+        "role": "user",
+        "content": "``\n{\n    \"Initial State\": [\n        [3, \"*\", 2, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"],\n        [\"*\", 2, 3, \"*\"],\n        [\"*\", \"*\", \"*\", \"*\"]\n    ],\n    \"Thought\": \"The given Sudoku puzzle has some numbers already filled in. We need to fill in the missing numbers such that each row, column, and 2x2 sub-grid contains the numbers 1-4 without repetition.\",\n    \"Solution\": [\n        [3, 1, 2, 4],\n        [4, 3, 1, 2],\n        [1, 2, 3, 4],\n        [2, 4, 4, 1]\n    ]\n}\n```\n\nThe solution is obtained by applying the standard Sudoku solving techniques, such as focusing on the most constrained regions first and using logical deductions to eliminate possibilities."
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,0,2,0],[0,0,0,0],[0,2,3,0],[0,0,0,0]]\nAnswer\n[[3,1,2,4],[4,3,1,2],[1,2,3,4],[2,4,4,1]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\"perception\": [[3, 1, \"*\", 4], [2, 4, 1, 3], [\"*\", \"*\", 4, \"*\"], [4, \"*\", 3, \"*\"]], \"think\": \"Looking at the first row, the only missing number is 2. \\nLooking at the last row, the missing numbers are 1 and 2. Since the cell at index (3,1) is of the same color as the cell at index (1,1) which contains the number 4, and the cell at index (3,0) contains the number 4, the cell at index (3,1) can only be 1 or 2. Since the cell at index (3,0) already contains 4, the cell at index (3,1) must be 1. Then the cell at index (3,3) must be 2. \\nLooking at the third row, the missing numbers are 1, 2, and 3. Since the cell at index (2,3) is of the same color as the cell at index (0,3) which contains 4, and the cell at index (1,3) contains 3, and the cell at index (2,2) contains 4, the cell at index (2,3) must be 1 or 2. Since the cell at index (3,3) contains 2, the cell at index (2,3) must be 1. Then the cell at index (2,0) must be 3, and the cell at index (2,1) must be 2. \\nTherefore, the solution is [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]].\", \"answer\": [[3, 1, 2, 4], [2, 4, 1, 3], [3, 2, 4, 1], [4, 1, 3, 2]]}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Perception\n[[3,1,0,4],[2,4,1,3],[0,0,4,0],[4,0,3,0]]\nAnswer\n[[3,1,2,4],[2,4,1,3],[3,2,4,1],[4,1,3,2]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. ."
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/thermometers/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/thermometers/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..1c0b69400ad5881246da1e35c4bfb26e48c07176
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/thermometers/filter_prompt.json
@@ -0,0 +1,41 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0 or s; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. If the perception / answer is not a valid solution or not existe, please put a Wrong there. Your goal is only to format the user's input, not to check if it is correct or not."},
+    {
+        "role": "user",
+        "content": "{\n\"perception\": \"***s*\\n*s***\\n*s*s*\\n**ss*\",\n\"think\": \"To solve this Thermometers puzzle, I need to fill the cells in a way that creates continuous thermometers from the bulbs to the top, without any gaps, and matches the given row and column constraints. I'll analyze each row and column to determine the required filled cells.\",\n\"answer\": \"ssss*\\nssss*\\n*ssss\\n**sse\"\n}"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,s,0],[0,s,0,0,0],[0,s,0,s,0],[0,0,s,s,0]]\nSolution\n[[s,s,s,s,0],[s,s,s,s,0],[0,s,s,s,s],[0,0,s,s,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": \"{{0,0:s},{0,0:s},{0,1:s},{0,1:s},{0,2:s},{0,3:s},{1,0:s},{1,0:s},{1,0:s},{1,0:s},{1,2:s},{1,3:s},{1,3:s},{2,0:s},{2,0:s},{2,1:s},{2,1:s},{2,2:s},{2,2:s},{2,3:s},{2,3:s},{3,0:s},{3,0:s},{3,1:s},{3,1:s},{3,2:s},{3,2:s},{3,3:s},{3,3:s}}\",\n  \"think\": \"The puzzle is a Thermometers puzzle. The numbers on the sides indicate how many filled cells must be in each row and column. The thermometers must be filled from their bulb (start) to their top, without gaps. I will start by filling the cells that are already partially filled. Then, I will look at the rows and columns to see if there are any cells that must be filled based on the numbers on the sides. I will continue this process until all of the thermometers are filled.\",\n  \"answer\": \"{{0,0:s},{0,0:s},{0,1:s},{0,1:s},{0,2:s},{0,3:s},{1,0:s},{1,0:s},{1,0:s},{1,0:s},{1,2:s},{1,3:s},{1,3:s},{2,0:s},{2,0:s},{2,1:s},{2,1:s},{2,2:s},{2,2:s},{2,3:s},{2,3:s},{3,0:s},{3,0:s},{3,1:s},{3,1:s},{3,2:s},{3,2:s},{3,3:s},{3,3:s}}\"\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\nWrong\nSolution\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes, as you cannot extract perception and answer as a 2d array, you have to mark it as Wrong. "
+    },
+    {
+        "role": "user",
+        "content": "{\n\"perception\": \"Row 0: * e * s | Row 1: s e e e | Row 2: * e * s | Row 3: * e * *\",\n\"think\": \"First, check the filled cells needed for each row and column. Row 1 requires all cells filled since it has 4 spaces and needs 4 filled. Row 0 requires 2 more filled cells, and Row 3 requires all 4 cells filled. Column 1 is already fully filled. Proceed to fill the thermometers ensuring continuity without exceeding the required number of filled spaces per row and column.\",\n\"answer\": \"Row 0: s e * s | Row 1: s s s s | Row 2: s e s e | Row 3: s s s s\"\n}"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[0,0,0,s],[s,0,0,0],[0,0,0,s],[0,0,0,0]]\nSolution\n[[s,0,0,s],[s,s,s,s],[s,0,0,0],[s,s,s,s]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes, though the input is not an array, but the message is clear. "
+    }
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/treesandtents/filter_prompt.json b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/treesandtents/filter_prompt.json
new file mode 100644
index 0000000000000000000000000000000000000000..caa95fc684b791aa8797f26feb00a3b6af1e2809
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/configs/formating-prompt/treesandtents/filter_prompt.json
@@ -0,0 +1,79 @@
+[
+    {
+        "role": "system", 
+        "content": "You are a text formater, your goal is to extract two parts, perception and answer from the answer and put them into a good format. Please according to the conversation history with the user feedback. The desired format is a 2d array, where there is only 0 or r, t these three characters; *, or 0, or _, or -, or e might indicates empty that you should convert to 0. tr means tree, and should be converted to r. tt means tent, and should be converted to t. It is possible that perception and / or answer is missing or invalid, you just put a Wrong for the wrong / both.  Your goal is only to format the user's input, not to check if it is correct or not."},
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [\n  [\"*\", \"*\", \"*\", \"tr\", \"tr\"],\n  [\"*\", \"*\", \"*\", \"*\", \"*\"],\n  [\"*\", \"tr\", \"*\", \"*\", \"*\"],\n  [\"*\", \"*\", \"tr\", \"*\", \"*\"],\n  [\"tr\", \"tr\", \"*\", \"*\", \"*\"]\n],\n\"answer\": [\n  [\"e\", \"e\", \"e\", \"tt\", \"tt\"],\n  [\"e\", \"e\", \"e\", \"e\", \"e\"],\n  [\"e\", \"tt\", \"e\", \"e\", \"e\"],\n  [\"e\", \"e\", \"tt\", \"e\", \"e\"],\n  [\"tt\", \"tt\", \"e\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,r,r],[0,0,0,0,0],[0,r,0,0,0],[0,0,r,0,0],[r,r,0,0,0]]\nSolution\n[[0,0,0,t,t],[0,0,0,0,0],[0,t,0,0,0],[0,0,t,0,0],[t,t,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes. "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n  \"perception\": \"[[e, e, e, tr, tr], [e, e, e, e, e], [e, tr, e, e, e], [e, e, tr, e, e], [tt, tr, e, e, e]]\",\n  \"think\": \"The constraints are that each tree must have exactly one tent next to it, horizontally or vertically. Tents cannot be adjacent to each other.  Looking at the bottom row, there's a tent and a tree. The tree at (4,1) needs a tent. The only place to put it is (3,1). Then, the tree at (2,1) needs a tent, and the only place is (2,0). The tree at (1,4) needs a tent, and the only place is (1,3). The tree at (0,4) needs a tent, and the only place is (0,3). This satisfies all constraints.\",\n  \"answer\": \"[[e, e, e, tt, tt], [e, e, e, e, e], [e, tt, e, e, e], [e, e, tr, e, e], [tt, tr, e, e, e]]\"\n}\n```"
+    },
+    {
+        "role": "assistant",
+        "content": "Initial State\n[[0,0,0,r,r],[0,0,0,0,0],[0,r,0,0,0],[0,0,r,0,0],[t,r,0,0,0]]\nSolution\n[[0,0,0,t,t],[0,0,0,0,0],[0,t,0,0,0],[0,0,r,0,0],[t,r,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes "
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"perception\": [[\"tt\", \"*\", \"*\", \"*\", \"tr\", \"tr\"], [\"*\", \"*\", \"*\", \"*\", \"*\", \"*\"], [\"*\", \"tr\", \"*\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"tr\", \"*\", \"*\", \"*\"], [\"*\", \"*\", \"*\", \"*\", \"*\", \"*\"], [\"*\", \"tr\", \"*\", \"*\", \"*\", \"*\"]],\n\"answer\": [[\"tt\", \"e\", \"e\", \"e\", \"tr\", \"tr\"], [\"e\", \"e\", \"e\", \"tt\", \"e\", \"tt\"], [\"tt\", \"tr\", \"e\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"tr\", \"e\", \"e\", \"e\"], [\"e\", \"tt\", \"e\", \"e\", \"e\", \"e\"], [\"tt\", \"tr\", \"e\", \"e\", \"e\", \"e\"]]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\n[[t,0,0,0,r,r],[0,0,0,0,0,0],[0,r,0,0,0,0],[0,0,r,0,0,0],[0,0,0,0,0,0],[0,r,0,0,0,0]]\nSolution\n[[t,0,0,0,r,r],[0,0,0,t,0,t],[t,r,0,0,0,0],[0,0,r,0,0,0],[0,t,0,0,0,0],[t,r,0,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes, though the input is not an array, but the message is clear. "
+    },
+    {
+        "role": "user",
+        "content": "```\n{\n  \"perception\": \"The current state of the game is as follows:\n  2 trees in the top left,\n  1 tree in the top right,\n  4 trees in the bottom left,\n  1 tree in the bottom right,\n  1 tent in the bottom left.\n\n  The numbers on the sides indicate that there should be 2 tents in the first row,\n  2 tents in the second row,\n  1 tent in the third row,\n  0 tents in the fourth row.\",\n  \"think\": \"To find a valid solution, we need to place tents such that each tree is paired with exactly one tent. Tents cannot touch each other, even diagonally.\n  \n  Let's start by placing tents next to the trees in the bottom left corner. Since there are 4 trees, we can place 4 tents.\n  \n  Next, we can place the tent next to the tree in the top right. Since there is only one tree in this column, we can place one tent here.\n  \n  Now, we need to place tents in the second row. We have already placed one tent in the bottom left. Since there are 2 trees in this row, we need to place 2 more tents in this row. We can place them next to the trees that are not paired yet.\n  \n  Finally, we need to place the tent in the third row next to the tree that is not paired yet.\n  \n  Since there are no more trees to pair with tents, and we have already placed the required number of tents in each row, our solution is complete.\",\n  \"answer\": \"trtrtrtrtt**\n  11e**\n  11e**\n  0000\"\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\nWrong"
+    },
+    {
+        "role": "user",
+        "content": "Yes, as the perception is far from a 2d array, and the output contains unrecoginizable values 1, so you just put a Wrong"
+    },
+    {
+        "role": "user",
+        "content": "```json\n{\n\"think\": \"To solve this Trees and Tents puzzle, I need to place tents on the grid such that each tree is adjacent to exactly one tent, and no tents touch each other, even diagonally. The row and column clues provide the number of tents that must be placed in each row and column, respectively.\n\nI'll start by analyzing the given clues:\nRow clues: [1, 0, 2, 0, 2]\n- Row 0 has 1 tent\n- Row 1 has no tents\n- Row 2 has 2 tents\n- Row 3 has no tents\n- Row 4 has 2 tents\n\nColumn clues: [0, 1, 2, 1, 1]\n- Column 0 has no tents\n- Column 1 has 1 tent\n- Column 2 has 2 tents\n- Column 3 has 1 tent\n- Column 4 has 1 tent\n\nNext, I'll place the tents based on the given board state and clues:\n- Row 0: The only possible position for the tent is (0, 2) since it's the only empty cell.\n- Row 1: No tents need to be placed.\n- Row 2: The two tents must be placed at (2, 1) and (2, 3) to be adjacent to the trees.\n- Row 3: No tents need to be placed.\n- Row 4: The two tents must be placed at (4, 0) and (4, 2) to be adjacent to the trees and satisfy the column clues.\n\nThe final solution is:\n\",\n\"answer\": [\n    [\"e\", \"e\", \"tt\", \"tr\", \"e\"],\n    [\"e\", \"e\", \"e\", \"e\", \"tr\"],\n    [\"e\", \"tr\", \"tt\", \"tt\", \"e\"],\n    [\"e\", \"e\", \"e\", \"tr\", \"e\"],\n    [\"tr\", \"tt\", \"tt\", \"e\", \"e\"]\n]\n}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\n[[0,0,t,r,0],[0,0,0,0,r],[0,r,t,t,0],[0,0,0,r,0],[r,t,t,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes, perception is missing so you put a Wrong, but the solution is valid."
+    },
+
+    {
+        "role": "user",
+        "content": "```json\n{\"answer\": [[\"e\", \"e\", \"tt\", \"tr\", \"e\"], [\"e\", \"e\", \"e\", \"e\", \"tr\"], [\"e\", \"tr\", \"e\", \"e\", \"e\"], [\"e\", \"e\", \"e\", \"tr\", \"e\"], [\"tr\", \"tt\", \"e\", \"e\", \"e\"]]}\n```"
+    },
+    {
+        "role": "assistant", 
+        "content": "Initial State\nWrong\nSolution\n[[0,0,t,r,0],[0,0,0,0,r],[0,r,0,0,0],[0,0,0,r,0],[r,t,0,0,0]]"
+    },
+    {
+        "role": "user",
+        "content": "Yes, perception is missing so you put a Wrong, but the solution is valid."
+    }
+    
+]
\ No newline at end of file
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/evaluation.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/evaluation.py
new file mode 100644
index 0000000000000000000000000000000000000000..5c118918750d9bdd9b0122ba7af73289e8a62902
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/evaluation.py
@@ -0,0 +1,138 @@
+import ast
+import json
+import os
+from collections import defaultdict
+
+import numpy as np
+import pandas as pd
+
+from .score import evaluate_single_puzzle
+
+
+def get_eval(judge, content):
+    """
+    Generate evaluation using the judge model.
+    
+    Args:
+        judge: The evaluation model
+        content: Input content for the evaluation
+        
+    Returns:
+        The generated evaluation output
+    """
+    return judge.generate(content)
+
+def VGRPBench_atomeval(model, prompt, line):
+    """
+    Perform atomic evaluation for a VGRPBench puzzle.
+
+    Args:
+        model: The evaluation model
+        prompt: Input prompt for evaluation
+        line: Dictionary containing puzzle information
+
+    Returns:
+        dict: Evaluation scores
+    """
+    print("raw output", prompt)
+    output = get_eval(model, prompt)
+    print("formatted output", output)
+    scores = parse_score(line, output)
+    return scores
+
+def parse_score(line, output):
+    """
+    Parse the score from the model's output for a VGRPBench puzzle.
+    
+    Args:
+        line: Dictionary-like object containing puzzle information
+        output: The model's output string
+        
+    Returns:
+        dict: Dictionary with perception_correct and answer_correct results
+    """
+    
+    # Extract category to determine puzzle type
+    category = line['category']
+    puzzle_type = category.split('_')[0]  # e.g., "thermometers" from "thermometers_4x4"
+    
+    # Parse the puzzle state from the states field
+    puzzle_data = line['states']
+    puzzle_data = ast.literal_eval(puzzle_data)
+    
+    # Evaluate the puzzle solution
+    evaluation_result = evaluate_single_puzzle(output, puzzle_data, puzzle_type)
+
+    return evaluation_result
+
+def VGRPBench_score(data):
+    """
+    Calculate scores for VGRPBench puzzles by category.
+    
+    Args:
+        data: DataFrame containing evaluation results
+        
+    Returns:
+        pandas.DataFrame: Aggregated scores by category
+    """
+    # Get unique categories without 'overall'
+    cates = list(set(data['category']))
+    ret = defaultdict(list)
+
+    for c in cates:
+        ret['category'].append(c)
+        # Filter data for the current category
+        sub = data[data['category'] == c]
+        
+        # Calculate perception score (as percentage with 2 decimal places)
+        perception_score = round(np.mean(sub['perception_correct']) * 100, 2)
+        ret['Perception Score'].append(perception_score)
+        
+        # Calculate answer score (as percentage with 2 decimal places)
+        answer_score = round(np.mean(sub['answer_correct']) * 100, 2)
+        ret['Answer Score'].append(answer_score)
+        
+    return pd.DataFrame(ret)
+
+def build_prompt(line):
+    """
+    Build a prompt from the prediction field in the data line.
+    
+    Args:
+        line: Dictionary containing a 'prediction' field
+        
+    Returns:
+        str: The prediction text to be used as a prompt
+    """
+    # Get the prediction entry from the prediction column
+    return line['prediction']
+
+def VGRPBench_get_system_prompt(line):
+    """
+    Get the system prompt for a specific puzzle type in VGRPBench.
+    
+    Args:
+        line: A data row containing a 'category' field that defines the puzzle type
+        
+    Returns:
+        str: A formatted system prompt loaded from the corresponding filter_prompt.json file
+    """
+    # Extract puzzle type from category (e.g., "thermometers" from "thermometers_4x4")
+    puzzle_type = line['category'].split('_')[0]
+
+    # Construct path to the filter_prompt.json file for this puzzle type
+    prompt_file = os.path.join(
+        os.path.dirname(os.path.abspath(__file__)),
+        "configs",
+        "formating-prompt",
+        puzzle_type,
+        "filter_prompt.json"
+    )
+
+    # Load and return the prompt from the JSON file
+    with open(prompt_file, 'r') as f:
+        prompt = json.load(f)
+
+    prompt = str(prompt) + "According to the conversation history with the user feedback do the formatting for the current answer."  # noqa: E501
+
+    return prompt
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/aquarium.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/aquarium.py
new file mode 100644
index 0000000000000000000000000000000000000000..71f9dd93166474c5f11f6c35a35791b5cd4a9997
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/aquarium.py
@@ -0,0 +1,116 @@
+import argparse
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintAquariumFill(Constraint):
+    """Check aquarium conditions:
+    1. If there's a highest water row in the aquarium, all cells from that row downward in the same aquarium must not be empty.
+    2. For every row in the aquarium, if a cell is defined as empty or filled, all defined cells in that row must match.
+    3. For every column in the aquarium, if a top cell is filled with water, all consecutive lower cells must not be empty.
+    """
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_aquarium_fill"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        aquariums = game_state.get("clues", {}).get("aquariums", [])
+
+        for aquarium in aquariums:
+            # Find highest water cell in this aquarium
+            highest_water_row = float('inf')
+            for r, c in aquarium:
+                if board[r][c] == "s":
+                    highest_water_row = min(highest_water_row, r)
+
+            if highest_water_row == float('inf'):
+                continue  # No water in this aquarium
+
+            # Check all cells at or below highest water level
+            for r, c in aquarium:
+                if r >= highest_water_row:  # if cell is at same height or lower than highest water
+                    if board[r][c] == "e":  # if cell is empty
+                        return False
+
+        return True
+
+class ConstraintAquariumCount(Constraint):
+    """Check if row and column counts match the clues"""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_aquarium_count"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        clues = game_state.get("clues", None)
+        if not clues:
+            return True
+
+        size = len(board)
+        row_counts = clues["row_counts"]
+        col_counts = clues["col_counts"]
+
+        # Check rows
+        for i in range(size):
+            row_selected = sum(1 for j in range(size) if board[i][j] == "s")
+            row_undefined = sum(1 for j in range(size) if board[i][j] == 0)
+            # If row is fully decided (no zeros), it must match exactly
+            if 0 not in board[i]:
+                if row_selected != row_counts[i]:
+                    return False
+            else:
+                # If not fully decided, no more than the count should be selected
+                if row_selected > row_counts[i]:
+                    return False
+                # Also must be possible to still reach the target
+                if row_selected + row_undefined < row_counts[i]:
+                    return False
+
+        # Check columns
+        for j in range(size):
+            col_cells = [board[i][j] for i in range(size)]
+            col_selected = sum(1 for i in range(size) if board[i][j] == "s")
+            col_undefined = sum(1 for i in range(size) if board[i][j] == 0)
+            if all(cell != 0 for cell in col_cells):
+                if col_selected != col_counts[j]:
+                    return False
+            else:
+                if col_selected > col_counts[j]:
+                    return False
+                if col_selected + col_undefined < col_counts[j]:
+                    return False
+
+        return True
+
+
+class AquariumPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 4:
+            raise ValueError("Size must be at least 4")
+
+        self.game_name = "aquarium"
+        self.size = size
+        self.constraints = [ConstraintAquariumFill(), ConstraintAquariumCount()]
+        self.all_possible_values = ["e", "s"]  # empty or selected (water)
+
+    def get_possible_values(
+        self, game_state: Dict[str, Any], row: int, col: int
+    ) -> List[str]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/battleships.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/battleships.py
new file mode 100644
index 0000000000000000000000000000000000000000..6dc6eb4a9f6c426e3ad1e23bb84d6c9201a0de6d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/battleships.py
@@ -0,0 +1,161 @@
+import argparse
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintBattleships(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        # Check if ships touch diagonally or orthogonally
+        for i in range(size):
+            for j in range(size):
+                if isinstance(board[i][j], tuple):  # Check if it's a revealed ship with direction
+                    ship_cell, direction = board[i][j]
+                    # Add direction-specific checks here
+                    if direction in "<>-":  # Horizontal ship
+                        # Check cells above and below
+                        for di in [-1, 1]:
+                            if 0 <= i + di < size and board[i + di][j] == "s":
+                                return False
+                    elif direction in "^V|":  # Vertical ship
+                        # Check cells left and right
+                        for dj in [-1, 1]:
+                            if 0 <= j + dj < size and board[i][j + dj] == "s":
+                                return False
+                elif board[i][j] == "s":
+                    # Regular ship cell checks
+                    for di in [-1, 0, 1]:
+                        for dj in [-1, 0, 1]:
+                            if di == 0 and dj == 0:
+                                continue
+                            ni, nj = i + di, j + dj
+                            if (0 <= ni < size and 0 <= nj < size and
+                                (board[ni][nj] == "s" or (isinstance(board[ni][nj], tuple) and board[ni][nj][0] == "s")) and
+                                (di != 0 and dj != 0)):  # Diagonal check
+                                return False
+        return True
+
+class ConstraintBattleshipsHints(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        hints = game_state["hints"]
+        row_hints = hints["row_hints"]
+        col_hints = hints["col_hints"]
+        ships = hints["ships"]
+        size = len(board)
+        # Calculate total required ship cells from ships configuration
+        total_ship_cells_required = sum(int(length) * int(count) for length, count in ships.items())
+        total_ship_cells_selected = sum(1 for i in range(size) for j in range(size) if board[i][j] == "s")
+        total_undefined_cells = sum(1 for i in range(size) for j in range(size) if board[i][j] == 0)
+
+        # Check if we have enough cells (placed + potential) to fit all ships
+        if total_ship_cells_selected + total_undefined_cells < total_ship_cells_required:
+            return False
+
+        # Check if we haven't exceeded the total required ship cells
+        if total_ship_cells_selected > total_ship_cells_required:
+            return False
+
+        # Check row hints
+        for i in range(size):
+            row_selected = sum(1 for j in range(size) if board[i][j] == "s")
+            row_undefined = sum(1 for j in range(size) if board[i][j] == 0)
+            # Consider both undefined (0) and non-revealed water cells for potential ships
+            if all(cell != 0 and cell != -1 for cell in board[i]):  # if row is complete
+                if row_selected != row_hints[i]:
+                    return False
+            else:  # if row is incomplete
+                if row_selected > row_hints[i]:  # too many selected
+                    return False
+                if row_selected + row_undefined < row_hints[i]:  # impossible to reach target
+                    return False
+        # Check column hints
+        for j in range(size):
+            col_selected = sum(1 for i in range(size) if board[i][j] == "s")
+            col_undefined = sum(1 for i in range(size) if board[i][j] == 0)
+            if all(board[i][j] != 0 and board[i][j] != -1 for i in range(size)):  # if column is complete
+                if col_selected != col_hints[j]:
+                    return False
+            else:  # if column is incomplete
+                if col_selected > col_hints[j]:  # too many selected
+                    return False
+                if col_selected + col_undefined < col_hints[j]:  # impossible to reach target
+                    return False
+        # When all cells are filled, check ship shapes
+        if total_undefined_cells == 0:
+            # Find all ships by finding connected components
+            visited = [[False] * size for _ in range(size)]
+            ship_lengths = []
+
+            def get_ship_length(i: int, j: int) -> int:
+                if (i < 0 or i >= size or j < 0 or j >= size or
+                    visited[i][j] or board[i][j] != "s"):
+                    return 0
+
+                visited[i][j] = True
+                length = 1
+
+                # Check if ship is horizontal
+                if (j + 1 < size and board[i][j + 1] == "s"):
+                    # Add all horizontal cells
+                    for col in range(j + 1, size):
+                        if board[i][col] != "s":
+                            break
+                        visited[i][col] = True
+                        length += 1
+                # Check if ship is vertical
+                elif (i + 1 < size and board[i + 1][j] == "s"):
+                    # Add all vertical cells
+                    for row in range(i + 1, size):
+                        if board[row][j] != "s":
+                            break
+                        visited[row][j] = True
+                        length += 1
+
+                return length
+
+            # Find all ships
+            for i in range(size):
+                for j in range(size):
+                    if not visited[i][j] and board[i][j] == "s":
+                        ship_lengths.append(get_ship_length(i, j))
+            # Count ships of each length
+            ship_counts = {}
+            for length in ship_lengths:
+                ship_counts[length] = ship_counts.get(length, 0) + 1
+            # Verify against required ships
+            for length, count in ships.items():
+                if ship_counts.get(int(length), 0) != int(count):
+                    return False
+        return True
+
+class BattleshipsPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "battleships"
+        self.size = size
+        self.constraints = [
+            ConstraintBattleships(),
+            ConstraintBattleshipsHints()
+        ]
+        self.all_possible_values = ["e", "s"]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        board = game_state["board"]
+        if board[row][col] != 0:  # If cell is already filled
+            return []
+
+        possible_values = []
+        original_value = board[row][col]
+
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/binairo.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/binairo.py
new file mode 100644
index 0000000000000000000000000000000000000000..330d38ffa19e135103c7155c22b7a171228cd982
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/binairo.py
@@ -0,0 +1,98 @@
+import argparse
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintRowBalance(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_row_balance"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        expected_count = size // 2
+
+        assert all(all(cell != '*' for cell in row) for row in board), "'*' should be replaced by '0' in the initialization board"
+
+        for row in board:
+            if 0 not in row:  # Only check completed rows
+                white_count = sum(1 for x in row if x == 'w')
+                black_count = sum(1 for x in row if x == 'b')
+                if white_count != black_count or white_count != expected_count:
+                    return False
+        return True
+
+class ConstraintColBalance(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_col_balance"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        expected_count = size // 2
+
+        for col in range(size):
+            column = [board[row][col] for row in range(size)]
+            if 0 not in column and '*' not in column:  # Only check completed columns
+                white_count = sum(1 for x in column if x == 'w')
+                black_count = sum(1 for x in column if x == 'b')
+                if white_count != black_count or white_count != expected_count:
+                    return False
+        return True
+
+class ConstraintNoTripleAdjacent(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_no_triple_adjacent"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        # Check rows
+        for row in range(size):
+            for col in range(size - 2):
+                if (board[row][col] != 0 and
+                    board[row][col] == board[row][col + 1] == board[row][col + 2]):
+                    return False
+
+        # Check columns
+        for col in range(size):
+            for row in range(size - 2):
+                if (board[row][col] != 0 and
+                    board[row][col] == board[row + 1][col] == board[row + 2][col]):
+                    return False
+        return True
+
+
+class BinairoPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 4 or size % 2 != 0:
+            raise ValueError("Size must be an even number greater than or equal to 4")
+        self.game_name = "binairo"
+        self.size = size
+        self.constraints = [
+            ConstraintRowBalance(),
+            ConstraintColBalance(),
+            ConstraintNoTripleAdjacent(),
+            # ConstraintUniqueLines()
+        ]
+        self.all_possible_values = ['w', 'b']  # 'w' for white, 'b' for black
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/coloredsudoku.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/coloredsudoku.py
new file mode 100644
index 0000000000000000000000000000000000000000..59c5a7aa3104a8f57038f166e20b2cf23a33fe05
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/coloredsudoku.py
@@ -0,0 +1,60 @@
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple, Union
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintColorNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_color_no_repeat"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        colors = game_state.get("colors", None)
+
+        # If no colors are specified, skip this constraint
+        if colors is None:
+            return True
+
+        color_groups = {}
+        for i in range(len(board)):
+            for j in range(len(board[0])):
+                color = colors[i][j]
+                if color not in color_groups:
+                    color_groups[color] = []
+                if board[i][j] != 0:
+                    color_groups[color].append(board[i][j])
+        for color_values in color_groups.values():
+            if len(set(color_values)) != len(color_values):
+                return False
+        return True
+
+class ColoredSudokuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "coloredsudoku"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintColorNoRepeat()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+        self.colors = [chr(65 + i) for i in range(size)]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_constriants.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_constriants.py
new file mode 100644
index 0000000000000000000000000000000000000000..420b16269b68c5997ac6c49a11092fa09626a9bc
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_constriants.py
@@ -0,0 +1,53 @@
+from typing import Any, Dict
+
+
+class Constraint():
+    def __init__(self) -> None:
+        self.name = ""
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        pass
+
+class ConstraintRowNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_row_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for row in board:
+            row_tmp = [cell for cell in row if cell != 0]
+            if len(set(row_tmp)) != len(row_tmp):
+                return False
+        return True
+
+class ConstraintColNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_col_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for col in range(len(board[0])):
+            col_tmp = [board[row][col] for row in range(len(board)) if board[row][col] != 0]
+            if len(set(col_tmp)) != len(col_tmp):
+                return False
+        return True
+
+class ConstraintSubGridNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_sub_grid_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        assert len(board) == len(board[0]), "board is not square"
+        assert len(board) in [4, 9], "board size is not 4 or 9"
+
+        sub_grid_size = int(len(board) ** 0.5)
+        for i in range(0, len(board), sub_grid_size):
+            for j in range(0, len(board[0]), sub_grid_size):
+                sub_grid = [
+                    board[x][y] for x in range(i, i + sub_grid_size)
+                    for y in range(j, j + sub_grid_size)
+                    if board[x][y] != 0
+                ]
+                if len(set(sub_grid)) != len(sub_grid):
+                    return False
+        return True
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_get_game_factory.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_get_game_factory.py
new file mode 100644
index 0000000000000000000000000000000000000000..9f48b532f636eff6732e6ff1a832a537b3f96053
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_get_game_factory.py
@@ -0,0 +1,43 @@
+def get_game_factory(game_type):
+    if game_type == "sudoku":
+        from .sudoku import SudokuPuzzleFactory as GameFactory
+    elif game_type == "binairo":
+        from .binairo import BinairoPuzzleFactory as GameFactory
+    elif game_type == "coloredsudoku":
+        from .coloredsudoku import ColoredSudokuPuzzleFactory as GameFactory
+    elif game_type == "kakuro":
+        from .kakuro import KakuroPuzzleFactory as GameFactory
+    elif game_type == "killersudoku":
+        from .killersudoku import KillerSudokuPuzzleFactory as GameFactory
+    elif game_type == "renzoku":
+        from .renzoku import RenzokuPuzzleFactory as GameFactory
+    elif game_type == "skyscraper":
+        from .skyscraper import SkyscraperPuzzleFactory as GameFactory
+    elif game_type == "starbattle":
+        from .starbattle import StarBattlePuzzleFactory as GameFactory
+    elif game_type == "treesandtents":
+        from .treesandtents import TreesAndTentsPuzzleFactory as GameFactory
+    elif game_type == "thermometers":
+        from .thermometers import ThermometersPuzzleFactory as GameFactory
+    elif game_type == "futoshiki":
+        from .futoshiki import FutoshikiPuzzleFactory as GameFactory
+    elif game_type == "hitori":
+        from .hitori import HitoriPuzzleFactory as GameFactory
+    elif game_type == "aquarium":
+        from .aquarium import AquariumPuzzleFactory as GameFactory
+    elif game_type == "kakurasu":
+        from .kakurasu import KakurasuPuzzleFactory as GameFactory
+    elif game_type == "oddevensudoku":
+        from .oddevensudoku import OddEvenSudokuPuzzleFactory as GameFactory
+    elif game_type == "battleships":
+        from .battleships import BattleshipsPuzzleFactory as GameFactory
+    elif game_type == "fieldexplore":
+        from .fieldexplore import FieldExplorePuzzleFactory as GameFactory
+    elif game_type == "jigsawsudoku":
+        from .jigsawsudoku import JigsawSudokuPuzzleFactory as GameFactory
+    elif game_type == "lightup":
+        from .lightup import LightUpPuzzleFactory as GameFactory
+    elif game_type == "nonogram":
+        from .nonogram import NonogramPuzzleFactory as GameFactory
+
+    return GameFactory
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_get_prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_get_prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..f189259aeaa8ea35e5907fd768ece80989b74787
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_get_prompt.py
@@ -0,0 +1,52 @@
+def get_prompt(game_type: str, thinking_format: str) -> str:
+    if game_type == "sudoku":
+        from puzzles.sudoku import SYSTEM_PROMPT
+    elif game_type == "coloredsudoku":
+        from puzzles.coloredsudoku import SYSTEM_PROMPT
+    elif game_type == "binairo":
+        from puzzles.binairo import SYSTEM_PROMPT
+    elif game_type == "futoshiki":
+        from puzzles.futoshiki import SYSTEM_PROMPT
+    elif game_type == "hitori":
+        from puzzles.hitori import SYSTEM_PROMPT
+    elif game_type == "kakuro":
+        from puzzles.kakuro import SYSTEM_PROMPT
+    elif game_type == "killersudoku":
+        from puzzles.killersudoku import SYSTEM_PROMPT
+    elif game_type == "renzoku":
+        from puzzles.renzoku import SYSTEM_PROMPT
+    elif game_type == "skyscraper":
+        from puzzles.skyscraper import SYSTEM_PROMPT
+    elif game_type == "starbattle":
+        from puzzles.starbattle import SYSTEM_PROMPT
+    elif game_type == "sudoku":
+        from puzzles.sudoku import SYSTEM_PROMPT
+    elif game_type == "treesandtents":
+        from puzzles.treesandtents import SYSTEM_PROMPT
+    elif game_type == "thermometers":
+        from puzzles.thermometers import SYSTEM_PROMPT
+    elif game_type == "kakurasu":
+        from puzzles.kakurasu import SYSTEM_PROMPT
+    elif game_type == "aquarium":
+        from puzzles.aquarium import SYSTEM_PROMPT
+    elif game_type == "oddevensudoku":
+        from puzzles.oddevensudoku import SYSTEM_PROMPT
+
+    elif game_type == "battleships":
+        from puzzles.battleships import SYSTEM_PROMPT
+    elif game_type == "fieldexplore":
+        from puzzles.fieldexplore import SYSTEM_PROMPT
+    elif game_type == "jigsawsudoku":
+        from puzzles.jigsawsudoku import SYSTEM_PROMPT
+    elif game_type == "nonogram":
+        from puzzles.nonogram import SYSTEM_PROMPT
+    elif game_type == "lightup":
+        from puzzles.lightup import SYSTEM_PROMPT
+
+    else:
+        raise ValueError(f"Unknown game type: {game_type}")
+
+    if thinking_format == "direct_solution":
+        return SYSTEM_PROMPT["direct_solution"]
+    else:
+        return SYSTEM_PROMPT["cot"]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_puzzle_factory.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_puzzle_factory.py
new file mode 100644
index 0000000000000000000000000000000000000000..0edcade134dcb5434f12a1ca1183ff55ce552a1c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/common_puzzle_factory.py
@@ -0,0 +1,136 @@
+import argparse
+import copy
+import json
+import os
+import random
+from abc import ABC, abstractmethod
+from typing import Any, Dict, List, Union
+
+
+def hint_type(value):
+    if value == "random":
+        return "random"
+    try:
+        return int(value)
+    except ValueError:
+        raise argparse.ArgumentTypeError(f"'{value}' must be 'random' or an integer")
+
+
+class PuzzleFactory():
+    def __init__(self) -> None:
+        self.constraints = []
+        self.game_name = "unknown"
+        self.size = 0
+        # Define dataset split ratios (must sum to 10)
+        self.train_ratio = 8
+        self.val_ratio = 1
+        self.ablation_ratio = 1
+
+    def sample_hints(self, board: List[List[int]], num_sample_hints: int) -> List[List[int]]:
+        # Create a new board filled with zeros
+        new_board = [[0 for _ in range(len(board[0]))] for _ in range(len(board))]
+        # Sample num_sample_hints cells to keep from the original board
+        sampled_cells = random.sample(range(len(board) * len(board[0])), num_sample_hints)
+        for cell in sampled_cells:
+            row = cell // len(board[0])
+            col = cell % len(board[0])
+            new_board[row][col] = board[row][col]  # Copy only the sampled cells from original board
+        return new_board
+    def save_puzzles(self, puzzles: List[Dict[str, Any]], save_path: str = "datasets/", filename: str = None) -> None:
+        """
+        Save the generated puzzles to JSON files, split into train, val, and ablation sets.
+        Splits are based on unique solutions with ratios defined in __init__.
+        Val set has different solutions from train, while ablation shares solutions with train.
+        """
+        if filename is None:
+            base_path = f"{save_path}/{self.game_name}_{self.size}x{self.size}_puzzles"
+        else:
+            base_path = f"{save_path}/{filename.rsplit('.', 1)[0]}"
+
+        # Group puzzles by their solutions
+        solution_groups = {}
+        for puzzle in puzzles:
+            solution_key = str(puzzle['solution'])  # Convert to string for dict key
+            if solution_key not in solution_groups:
+                solution_groups[solution_key] = []
+            solution_groups[solution_key].append(puzzle)
+
+        # Sort groups (common groups first to validation set) by size for better distribution
+        sorted_groups = sorted(solution_groups.items(), key=lambda x: len(x[1]), reverse=True)
+        # Calculate target sizes based on ratios
+        total_puzzles = len(puzzles)
+        target_val_size = total_puzzles * self.val_ratio // 10
+        target_ablation_size = total_puzzles * self.ablation_ratio // 10
+        # Initialize sets
+        train_puzzles = []
+        val_puzzles = []
+        ablation_puzzles = []
+        # First, fill validation set with complete groups
+        val_solutions = set()
+        current_val_size = 0
+        val_group_idx = 0
+        while val_group_idx < len(sorted_groups) and current_val_size < target_val_size:
+            group = sorted_groups[val_group_idx][1]
+            if current_val_size + len(group) <= target_val_size * 1.2:  # Allow 20% overflow
+                val_puzzles.extend(group)
+                val_solutions.add(sorted_groups[val_group_idx][0])
+                current_val_size += len(group)
+            val_group_idx += 1
+
+        # Fill train and ablation sets with remaining groups
+        train_solutions = set()
+        current_ablation_size = 0
+
+        for solution, group in sorted_groups:
+            if solution in val_solutions:
+                continue
+
+            train_solutions.add(solution)
+            # Randomly split each remaining group between train and ablation
+            if current_ablation_size < target_ablation_size:
+                # Calculate how many puzzles we can still add to ablation
+                space_left = target_ablation_size - current_ablation_size
+                # Take up to 20% of the current group for ablation
+                ablation_count = min(max(1, len(group) // 5), space_left)
+                # Randomly select puzzles for ablation
+                ablation_indices = random.sample(range(len(group)), ablation_count)
+                for i in range(len(group)):
+                    if i in ablation_indices:
+                        ablation_puzzles.append(group[i])
+                        current_ablation_size += 1
+                    else:
+                        train_puzzles.append(group[i])
+            else:
+                train_puzzles.extend(group)
+
+        # Shuffle each set before saving
+        random.shuffle(train_puzzles)
+        random.shuffle(val_puzzles)
+        random.shuffle(ablation_puzzles)
+
+        # Create all parent directories
+        os.makedirs(os.path.dirname(f"{base_path}_train.json"), exist_ok=True)
+
+        # Save splits to separate files
+        for split_name, split_puzzles in [
+            ("train", train_puzzles),
+            ("val", val_puzzles),
+            ("ablation", ablation_puzzles)
+        ]:
+            split_path = f"{base_path}_{split_name}.json"
+            with open(split_path, "w") as f:
+                json.dump(split_puzzles, f, indent=2)
+        print(f"\nSplit and saved {len(puzzles)} puzzles:")
+        print(f"Train: {len(train_puzzles)} puzzles ({len(train_solutions)} unique solutions)")
+        print(f"Val: {len(val_puzzles)} puzzles ({len(val_solutions)} unique solutions)")
+        print(f"Ablation: {len(ablation_puzzles)} puzzles (solutions shared with train)")
+        print(f"Files saved to {base_path}_[train/val/ablation].json")
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        for constraint in self.constraints:
+            if not constraint.check(game_state):
+                return False
+        return True
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        pass
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/fieldexplore.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/fieldexplore.py
new file mode 100644
index 0000000000000000000000000000000000000000..0e8acd9818da8b233720dc47df2d0d5b407fc158
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/fieldexplore.py
@@ -0,0 +1,65 @@
+import argparse
+import copy
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+import numpy as np
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintAdjacentNumbers(Constraint):
+    """Ensures revealed numbers match adjacent mine counts"""
+    def check(self, game_state: List[List[Any]]) -> bool:
+
+        board = game_state["board"]
+
+        size = len(board)
+        for i in range(size):
+            for j in range(size):
+                if isinstance(board[i][j], int) and board[i][j] != 0:  # If cell is a revealed number
+                    # Count adjacent mines and undefined cells
+                    i_start = max(0, i-1)
+                    i_end = min(size, i+2)
+                    j_start = max(0, j-1)
+                    j_end = min(size, j+2)
+
+                    adjacent_mines = sum(1 for r in range(i_start, i_end)
+                                      for c in range(j_start, j_end)
+                                      if board[r][c] == 's')
+
+                    adjacent_undefined = sum(1 for r in range(i_start, i_end)
+                                          for c in range(j_start, j_end)
+                                          if board[r][c] == 0)
+
+                    # Check if current mines <= number <= potential mines (current + undefined)
+                    if adjacent_mines > board[i][j] or adjacent_mines + adjacent_undefined < board[i][j]:
+                        return False
+        return True
+
+class FieldExplorePuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "fieldexplore"
+        self.size = size
+        self.constraints = [ConstraintAdjacentNumbers()]
+        self.all_possible_values = ['s', 'e']  # True for 's', False for 'e'
+
+    def check(self, board: List[List[Any]]) -> bool:
+        for constraint in self.constraints:
+            if not constraint.check(board):
+                return False
+        return True
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/futoshiki.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/futoshiki.py
new file mode 100644
index 0000000000000000000000000000000000000000..582d2d6c2d3abe5188da96c10e2564c0388d4a9d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/futoshiki.py
@@ -0,0 +1,96 @@
+import argparse
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintRowNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_row_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for row in board:
+            values = [x for x in row if x != 0]
+            if len(set(values)) != len(values):
+                return False
+        return True
+
+class ConstraintColNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_col_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        for col in range(size):
+            values = [board[row][col] for row in range(size) if board[row][col] != 0]
+            if len(set(values)) != len(values):
+                return False
+        return True
+
+class ConstraintInequality(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_inequality"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        inequalities = game_state.get("inequalities", {"row": [], "col": []})
+        # Check row inequalities
+        row_ineq = inequalities.get("row", [['' for _ in range(size-1)] for _ in range(size)])
+        for row in range(size):
+            for col in range(size-1):
+                if row_ineq[row][col] == '<':
+                    if board[row][col] != 0 and board[row][col+1] != 0:
+                        if board[row][col] >= board[row][col+1]:
+                            return False
+                elif row_ineq[row][col] == '>':
+                    if board[row][col] != 0 and board[row][col+1] != 0:
+                        if board[row][col] <= board[row][col+1]:
+                            return False
+        # Check column inequalities
+        col_ineq = inequalities.get("col", [['' for _ in range(size)] for _ in range(size-1)])
+        for row in range(size-1):
+            for col in range(size):
+                if col_ineq[row][col] == '^':
+                    if board[row][col] != 0 and board[row+1][col] != 0:
+                        if board[row][col] >= board[row+1][col]:
+                            return False
+                elif col_ineq[row][col] == 'v':
+                    if board[row][col] != 0 and board[row+1][col] != 0:
+                        if board[row][col] <= board[row+1][col]:
+                            return False
+        return True
+
+
+
+class FutoshikiPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 3 or size > 12:
+            raise ValueError("Grid size must be between 3 and 9")
+        self.game_name = "futoshiki"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintInequality()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/hitori.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/hitori.py
new file mode 100644
index 0000000000000000000000000000000000000000..c67ef497a41095fb19c9c0dbdeec924cbb7b074b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/hitori.py
@@ -0,0 +1,110 @@
+import argparse
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintHitoriNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_hitori_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]  # This is the shading state
+        numbers = game_state.get("numbers", [])  # Get the numbers from additional state
+        size = len(board)
+        # Check rows and columns for unshaded duplicates
+        for i in range(size):
+            row_values = [numbers[i][j] for j in range(size) if board[i][j] == "e"]  # 'e' means unshaded
+            col_values = [numbers[j][i] for j in range(size) if board[j][i] == "e"]
+            if len(row_values) != len(set(row_values)) or len(col_values) != len(set(col_values)):
+                return False
+        return True
+
+class ConstraintHitoriAdjacent(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_hitori_adjacent"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        for row in range(size):
+            for col in range(size):
+                if board[row][col] == "s":  # shaded cell
+                    # Check adjacent cells
+                    for dr, dc in [(-1, 0), (1, 0), (0, -1), (0, 1)]:
+                        nr, nc = row + dr, col + dc
+                        if 0 <= nr < size and 0 <= nc < size and board[nr][nc] == "s":
+                            return False
+        return True
+
+class ConstraintHitoriConnected(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_hitori_connected"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        # Find first unshaded or undecided cell
+        start = None
+        for r in range(size):
+            for c in range(size):
+                if board[r][c] in ["e", 0]:  # 'e' means unshaded, 0 means undecided
+                    start = (r, c)
+                    break
+            if start:
+                break
+
+        if not start:
+            return False
+
+        # BFS to check connectivity
+        visited = [[False] * size for _ in range(size)]
+        queue = [start]
+        visited[start[0]][start[1]] = True
+        while queue:
+            r, c = queue.pop(0)
+            for dr, dc in [(-1, 0), (1, 0), (0, -1), (0, 1)]:
+                nr, nc = r + dr, c + dc
+                if (0 <= nr < size and 0 <= nc < size and
+                    not visited[nr][nc] and board[nr][nc] in ["e", 0]):
+                    visited[nr][nc] = True
+                    queue.append((nr, nc))
+
+        # Check if all unshaded and undecided cells are visited
+        for r in range(size):
+            for c in range(size):
+                if board[r][c] in ["e", 0] and not visited[r][c]:
+                    return False
+        return True
+
+class HitoriPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "hitori"
+        self.size = size
+        self.constraints = [
+            ConstraintHitoriNoRepeat(),
+            ConstraintHitoriAdjacent(),
+            ConstraintHitoriConnected()
+        ]
+        self.all_possible_values = ["e", "s"]  # 'e' for empty/unshaded, 's' for shaded
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/jigsawsudoku.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/jigsawsudoku.py
new file mode 100644
index 0000000000000000000000000000000000000000..22b66383b25f9859f5ebd2d6922ade341aba34f2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/jigsawsudoku.py
@@ -0,0 +1,72 @@
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple, Union
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintRegionNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_region_no_repeat"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        regions = game_state.get("regions", None)
+
+        if regions is None:
+            return True
+
+        region_groups = {}
+        for i in range(len(board)):
+            for j in range(len(board[0])):
+                region = regions[i][j]
+                if region not in region_groups:
+                    region_groups[region] = []
+                if board[i][j] != 0:
+                    region_groups[region].append(board[i][j])
+        for region_values in region_groups.values():
+            if len(set(region_values)) != len(region_values):
+                return False
+        return True
+
+class JigsawSudokuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "jigsawsudoku"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintRegionNoRepeat()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+        self.cached_region_splits = []
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        """Get possible values for a cell based on row, column, and region constraints."""
+        if game_state["board"][row][col] != 0:
+            return []
+        possible_values = []
+        for value in self.all_possible_values:
+            # Try the value
+            original_value = game_state["board"][row][col]
+            game_state["board"][row][col] = value
+            # Check if it's valid according to all constraints
+            valid = True
+            for constraint in self.constraints:
+                if not constraint.check(game_state):
+                    valid = False
+                    break
+
+            # Restore original value
+            game_state["board"][row][col] = original_value
+
+            if valid:
+                possible_values.append(value)
+
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/kakurasu.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/kakurasu.py
new file mode 100644
index 0000000000000000000000000000000000000000..726dc3aad6674e2e8b8defa6afe5b801775622b9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/kakurasu.py
@@ -0,0 +1,79 @@
+import argparse
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintKakurasuSum(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_kakurasu_sum"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        clues = game_state.get("clues", {"row_clues": [], "col_clues": []})
+        size = len(board)
+        weights = [i + 1 for i in range(size)]
+        # Check row sums
+        for i in range(size):
+            row_sum = sum(weights[j] for j in range(size) if board[i][j] == "s")
+            # If row is complete (no 0s), sum must equal clue
+            if 0 not in board[i]:
+                if row_sum != clues["row_clues"][i]:
+                    return False
+            # If row is incomplete, check both conditions:
+            else:
+                # 1. Current sum must not exceed clue
+                if row_sum > clues["row_clues"][i]:
+                    return False
+                # 2. Check if remaining undefined cells can potentially reach the clue
+                undefined_cells = [j for j in range(size) if board[i][j] == 0]
+                max_possible_sum = row_sum + sum(weights[j] for j in undefined_cells)
+                if max_possible_sum < clues["row_clues"][i]:
+                    return False
+        # Check column sums
+        for i in range(size):
+            col_sum = sum(weights[j] for j in range(size) if board[j][i] == "s")
+            # If column is complete (no 0s), sum must equal clue
+            if all(board[j][i] != 0 for j in range(size)):
+                if col_sum != clues["col_clues"][i]:
+                    return False
+            # If column is incomplete, check both conditions:
+            else:
+                # 1. Current sum must not exceed clue
+                if col_sum > clues["col_clues"][i]:
+                    return False
+                # 2. Check if remaining undefined cells can potentially reach the clue
+                undefined_cells = [j for j in range(size) if board[j][i] == 0]
+                max_possible_sum = col_sum + sum(weights[j] for j in undefined_cells)
+                if max_possible_sum < clues["col_clues"][i]:
+                    return False
+        return True
+
+class KakurasuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 3:
+            raise ValueError("Grid size must be at least 3")
+        self.game_name = "kakurasu"
+        self.size = size
+        self.constraints = [
+            ConstraintKakurasuSum()
+        ]
+        self.all_possible_values = ["e", "s"]
+        self.weights = [i + 1 for i in range(size)]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/kakuro.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/kakuro.py
new file mode 100644
index 0000000000000000000000000000000000000000..b6cae4ce92c1d4b843a8fca7f8035baeba153cc7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/kakuro.py
@@ -0,0 +1,93 @@
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintKakuroSum(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_kakuro_sum"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        # if any of board is str, then convert to int
+        if any(isinstance(cell, str) for row in board for cell in row):
+            board = [[int(cell) for cell in row] for row in board]
+
+        sums = game_state.get("sums", {"row": [], "col": []})
+        # Check row sums
+        for row in range(len(board)):
+            if row < len(sums["row"]):
+                target_sum = sums["row"][row]
+                current_sum = sum(x for x in board[row] if x != 0)
+                if current_sum > target_sum:
+                    return False
+                if all(x != 0 for x in board[row]) and current_sum != target_sum:
+                    return False
+
+        # Check column sums
+        for col in range(len(board[0])):
+            if col < len(sums["col"]):
+                target_sum = sums["col"][col]
+                current_sum = sum(board[row][col] for row in range(len(board)) if board[row][col] != 0)
+                if current_sum > target_sum:
+                    return False
+                if all(board[row][col] != 0 for row in range(len(board))) and current_sum != target_sum:
+                    return False
+
+        return True
+
+class ConstraintKakuroAdjacent(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_kakuro_adjacent"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        # if any of board is str, then convert to int
+        if any(isinstance(cell, str) for row in board for cell in row):
+            board = [[int(cell) for cell in row] for row in board]
+
+        size = len(board)
+
+        for row in range(size):
+            for col in range(size):
+                if board[row][col] == 0:
+                    continue
+                # Check adjacent cells (up, down, left, right)
+                if row > 0 and board[row-1][col] == board[row][col]:
+                    return False
+                if row < size-1 and board[row+1][col] == board[row][col]:
+                    return False
+                if col > 0 and board[row][col-1] == board[row][col]:
+                    return False
+                if col < size-1 and board[row][col+1] == board[row][col]:
+                    return False
+        return True
+
+class KakuroPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 3 or size > 12:
+            raise ValueError("Grid size must be between 3 and 12")
+        self.game_name = "kakuro"
+        self.size = size
+        self.constraints = [
+            ConstraintKakuroSum(),
+            ConstraintKakuroAdjacent()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/killersudoku.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/killersudoku.py
new file mode 100644
index 0000000000000000000000000000000000000000..1e3e0ec2cc2811c7a4f0e6c665138b1718372c93
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/killersudoku.py
@@ -0,0 +1,63 @@
+import argparse
+import copy
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintCageSum(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_cage_sum"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        cages = game_state.get("cages", [])  # Default to empty list if no cages
+
+        for cage in cages:
+            cells = cage["cells"]
+            target_sum = cage["sum"]
+            current_sum = 0
+            for row, col in cells:
+                if board[row][col] == 0:  # Skip empty cells
+                    continue
+                current_sum += board[row][col]
+            if current_sum > target_sum:  # Can't exceed target sum
+                return False
+            # Only check equality if all cells in cage are filled
+            if all(board[row][col] != 0 for row, col in cells) and current_sum != target_sum:
+                return False
+        return True
+
+class KillerSudokuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "killersudoku"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintSubGridNoRepeat(),
+            ConstraintCageSum()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+
+        # Ensure cages exist in game_state
+        if "cages" not in game_state:
+            game_state["cages"] = []
+
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/lightup.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/lightup.py
new file mode 100644
index 0000000000000000000000000000000000000000..12e4058077b56a496ed9a3d6c8f88dbbe0156cd3
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/lightup.py
@@ -0,0 +1,159 @@
+import argparse
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintLightUpBulb(Constraint):
+    """Ensures that light bulbs don't illuminate each other.
+    This constraint checks that no two light bulbs ('s') can see each other in any straight line
+    (horizontally or vertically) without a wall between them. If two bulbs can see each other,
+    the constraint fails.
+    """
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_lightup_bulb"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        for row in range(size):
+            for col in range(size):
+                if board[row][col] == 's':  # Check light sources
+                    # Check each direction
+                    for dx, dy in [(-1, 0), (1, 0), (0, -1), (0, 1)]:
+                        nx, ny = row + dx, col + dy
+                        while 0 <= nx < size and 0 <= ny < size:
+                            if board[nx][ny] == 'w':  # Wall
+                                break
+                            if board[nx][ny] == 's':  # Another light
+                                return False
+                            # Skip undefined (0) and empty ('e') cells
+                            nx += dx
+                            ny += dy
+        return True
+
+class ConstraintLightUpWall(Constraint):
+    """Ensures that numbered walls have the correct number of adjacent light bulbs.
+    This constraint verifies that each numbered wall has exactly the specified number of light
+    bulbs placed in orthogonally adjacent cells. The constraint fails if:
+    1. A numbered wall has more adjacent light bulbs than its number
+    2. A numbered wall cannot possibly reach its required number with the remaining undefined cells
+    """
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_lightup_wall"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+
+        wall_numbers = game_state["wall_numbers"]
+
+        if not wall_numbers:
+            return True
+
+        size = len(board)
+
+        for row in range(size):
+            for col in range(size):
+
+                if board[row][col] == 'w' and wall_numbers[row][col] != -1:
+                    light_count = 0
+                    undefined_count = 0
+
+                    for dx, dy in [(-1, 0), (1, 0), (0, -1), (0, 1)]:
+                        nx, ny = row + dx, col + dy
+                        if 0 <= nx < size and 0 <= ny < size:
+                            if board[nx][ny] == 's':
+                                light_count += 1
+                            elif board[nx][ny] == 0:  # Count undefined cells
+                                undefined_count += 1
+
+                    # Fail if:
+                    # 1. We have too many definite lights, or
+                    # 2. We don't have enough potential lights (current + undefined) to reach the required number
+                    if (light_count > wall_numbers[row][col] or
+                        light_count + undefined_count < wall_numbers[row][col]):
+                        return False
+        return True
+
+class ConstraintLightUpIllumination(Constraint):
+    """Ensures that all non-wall cells are illuminated by at least one light bulb.
+    This constraint verifies that every empty cell ('e') is illuminated by at least one light bulb
+    or could potentially be illuminated by an undefined cell. For each empty cell, we check in all
+    four directions (up, down, left, right) until hitting a wall. If none of these directions
+    contain either a light bulb ('s') or an undefined cell (0), then the cell cannot be illuminated
+    in any valid solution.
+    """
+
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_lightup_illumination"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        # For each empty cell ('e'), check if it can be illuminated
+        for row in range(size):
+            for col in range(size):
+                if board[row][col] == 'e':
+                    can_be_illuminated = False
+                    # Check all four directions until hitting a wall
+                    for dx, dy in [(-1, 0), (1, 0), (0, -1), (0, 1)]:
+                        nx, ny = row + dx, col + dy
+                        while 0 <= nx < size and 0 <= ny < size:
+                            if board[nx][ny] == 'w':  # Hit a wall, stop checking this direction
+                                break
+                            if board[nx][ny] == 's' or board[nx][ny] == 0:  # Found light or potential light
+                                can_be_illuminated = True
+                                break
+                            nx += dx
+                            ny += dy
+
+                        if can_be_illuminated:  # If we found a light source, no need to check other directions
+                            break
+
+                    if not can_be_illuminated:  # If no direction had a light or potential light
+                        return False
+
+        return True
+
+class LightUpPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 3:
+            raise ValueError("Size must be at least 3")
+
+        self.game_name = "lightup"
+        self.size = size
+
+        self.constraints = [
+            ConstraintLightUpBulb(),
+            ConstraintLightUpWall(),
+            ConstraintLightUpIllumination()
+        ]
+
+        self.all_possible_values = ['s', 'e']  # 's' for source/light, 'e' for empty
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        board = game_state["board"]
+        if board[row][col] in [-1, 1, 2, 3, 4]:  # Wall or numbered wall
+            return []
+
+        possible_values = []
+        original_value = board[row][col]
+
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py
new file mode 100644
index 0000000000000000000000000000000000000000..40c1c0276b0edc5b49cf137d58f480f549ef0d10
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py
@@ -0,0 +1,132 @@
+import argparse
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+import numpy as np
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintBase:
+    def _check_line_hints(self, line, hints) -> bool:
+        # Convert line to runs of filled cells ('s' = filled, 'e' = empty, 0 = undefined)
+        runs = []  # Will store lengths of consecutive filled cells
+        count = 0  # Counter for current run length
+        current_run = []  # Track cells in current run for debugging/future use
+
+        if hints == [0]:
+            # the line should not contain 's'
+            return line.count('s') == 0
+
+        # First pass: Calculate runs of filled cells
+        for cell in line:
+            if cell == "s":  # Found a filled cell
+                count += 1
+                current_run.append(cell)
+            elif cell == "e":  # Found an empty cell
+                if count > 0:  # If we were counting a run
+                    runs.append(count)
+                    count = 0
+                current_run = []
+            else:  # cell is 0 (undefined)
+                if count > 0:
+                    current_run.append(cell)
+        # Don't forget to add the last run if it exists
+        if count > 0:
+            runs.append(count)
+        # Calculate cell statistics
+        filled_cells = line.count("s")     # Number of definitely filled cells
+        undefined_cells = line.count(0)     # Number of cells yet to be determined
+        required_cells = sum(hints)         # Total number of cells that should be filled according to hints
+
+        # Early failure: Check if we have enough cells to satisfy hints
+        if filled_cells + undefined_cells < required_cells:
+            return False
+
+        # For completely defined lines (no undefined cells)
+        if undefined_cells == 0:
+            # Simple comparison: runs must exactly match hints
+            if runs != hints:
+                return False
+        else:
+            # For partially defined lines, check if current definite runs are valid
+            definite_runs = []
+            count = 0
+            # Calculate runs that are definitely complete (bounded by empty cells or edges)
+            for cell in line:
+                if cell == "s":
+                    count += 1
+                elif (cell == "e" or cell == 0) and count > 0:
+                    definite_runs.append(count)
+                    count = 0
+                    if cell == 0:  # Stop at first undefined cell
+                        break
+            if count > 0:
+                definite_runs.append(count)
+            # Validate the definite runs we've found
+            if definite_runs:
+                # Can't have more runs than hints
+                if len(definite_runs) > len(hints):
+                    return False
+                # FIXME: Additional validation commented out
+                # Check if any run is longer than corresponding hint
+                # if any(definite_runs[j] > hints[j] for j in range(len(definite_runs))):
+                #     return False
+        return True
+
+class ConstraintRowHints(ConstraintBase):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        hints = game_state.get("hints", None)
+        if not hints:
+            raise ValueError("Hints are not provided")
+        row_hints = hints["row_hints"]
+
+        for i, row in enumerate(board):
+            if not self._check_line_hints(row, row_hints[i]):
+                return False
+        return True
+
+class ConstraintColHints(ConstraintBase):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        hints = game_state.get("hints", None)
+        if not hints:
+            raise ValueError("Hints are not provided")
+
+        col_hints = hints["col_hints"]
+        size = len(board)
+
+        for j in range(size):
+            col = [board[i][j] for i in range(size)]
+            if not self._check_line_hints(col, col_hints[j]):
+                return False
+        return True
+
+class NonogramPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "nonogram"
+        self.size = size
+        self.constraints = [
+            ConstraintRowHints(),
+            ConstraintColHints()
+        ]
+        self.all_possible_values = ["e", "s"]  # Consistent with paper
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[str]:
+        board = game_state["board"]
+        if board[row][col] != 0:  # If cell is already filled
+            return []
+
+        possible_values = []
+        original_value = board[row][col]
+
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py
new file mode 100644
index 0000000000000000000000000000000000000000..fdd8118a9480694ca7042f68592e01298616fb95
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py
@@ -0,0 +1,118 @@
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple, Union
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintOddEven(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_odd_even"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        cell_types = game_state.get("cell_types", None)
+
+        # If no cell types are specified, skip this constraint
+        if cell_types is None:
+            return True
+
+        for i in range(len(board)):
+            for j in range(len(board[0])):
+                if board[i][j] != 0:  # Skip empty cells
+                    is_even = board[i][j] % 2 == 0
+                    if (cell_types[i][j] == 'w' and not is_even) or \
+                       (cell_types[i][j] == 'b' and is_even):
+                        return False
+        return True
+
+class ConstraintRowNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_row_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for row in board:
+            # Create a list of non-zero values in the row
+            values = [x for x in row if x != 0]
+            # Check if there are any duplicates
+            if len(values) != len(set(values)):
+                return False
+        return True
+
+class ConstraintColNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_col_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        for col in range(size):
+            # Create a list of non-zero values in the column
+            values = [board[row][col] for row in range(size) if board[row][col] != 0]
+            # Check if there are any duplicates
+            if len(values) != len(set(values)):
+                return False
+        return True
+
+class ConstraintSubGridNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_sub_grid_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        sub_size = int(size ** 0.5)  # Size of sub-grid (2 for 4x4, 3 for 9x9)
+        # Check each sub-grid
+        for box_row in range(0, size, sub_size):
+            for box_col in range(0, size, sub_size):
+                # Get all non-zero values in the current sub-grid
+                values = []
+                for i in range(sub_size):
+                    for j in range(sub_size):
+                        value = board[box_row + i][box_col + j]
+                        if value != 0:
+                            values.append(value)
+                # Check for duplicates
+                if len(values) != len(set(values)):
+                    return False
+        return True
+
+class OddEvenSudokuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "oddevensudoku"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintSubGridNoRepeat(),
+            ConstraintOddEven()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        cell_types = game_state.get("cell_types", None)
+        original_value = board[row][col]
+
+        # Filter values based on odd/even constraint
+        if cell_types:
+            cell_type = cell_types[row][col]
+            filtered_values = [v for v in self.all_possible_values if
+                             (cell_type == 'w' and v % 2 == 0) or
+                             (cell_type == 'b' and v % 2 == 1)]
+        else:
+            filtered_values = self.all_possible_values
+
+        for value in filtered_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/renzoku.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/renzoku.py
new file mode 100644
index 0000000000000000000000000000000000000000..7081e85c36460eefcf4a4eb178163d9d877b057b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/renzoku.py
@@ -0,0 +1,107 @@
+import copy
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintRowNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_row_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for row in board:
+            values = [x for x in row if x != 0]
+            if len(set(values)) != len(values):
+                return False
+        return True
+
+class ConstraintColNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_col_no_repeat"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        for col in range(size):
+            values = [board[row][col] for row in range(size) if board[row][col] != 0]
+            if len(set(values)) != len(values):
+                return False
+        return True
+
+class ConstraintAdjacency(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_adjacency"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        # Get hints with proper default structure
+        # default_hints = {
+        #     "row": [['0' for _ in range(size - 1)] for _ in range(size)],
+        #     "col": [['0' for _ in range(size)] for _ in range(size - 1)]
+        # }
+        # always use hints from the game state
+        hints = game_state.get("hints") # , default_hints)
+        # Ensure hints have proper dimensions
+        if len(hints.get("row", [])) < size:
+            hints["row"] = [['0' for _ in range(size - 1)] for _ in range(size)]
+        if len(hints.get("col", [])) < size - 1:
+            hints["col"] = [['0' for _ in range(size)] for _ in range(size - 1)]
+        # convert board to int
+        board_copy = copy.deepcopy(board)
+        for i in range(size):
+            for j in range(size):
+                if board_copy[i][j] != 0:
+                    board_copy[i][j] = int(board_copy[i][j])
+
+        # Check row adjacency hints
+        for row in range(size):
+            for col in range(size - 1):
+                if hints["row"][row][col] == "1":
+                    if board_copy[row][col] == 0 or board_copy[row][col + 1] == 0:
+                        continue
+                    if abs(board_copy[row][col] - board_copy[row][col + 1]) != 1:
+                        return False
+        # Check column adjacency hints
+        for row in range(size - 1):
+            for col in range(size):
+                if hints["col"][row][col] == "1":
+                    if board_copy[row][col] == 0 or board_copy[row + 1][col] == 0:
+                        continue
+                    if abs(board_copy[row][col] - board_copy[row + 1][col]) != 1:
+                        return False
+        return True
+
+
+
+
+class RenzokuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 4 or size > 12:
+            raise ValueError("Grid size must be between 4 and 12")
+        self.game_name = "renzoku"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintAdjacency()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+        self.num_solver_processes = max(os.cpu_count() // 2, 1)  # Limit to 4 processes or CPU count
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/skyscraper.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/skyscraper.py
new file mode 100644
index 0000000000000000000000000000000000000000..48fa518f1384c10067e95a05dc56cfaac74949e4
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/skyscraper.py
@@ -0,0 +1,97 @@
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintRowNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for row in board:
+            values = [x for x in row if x != 0]
+            if len(set(values)) != len(values):
+                return False
+        return True
+
+class ConstraintColNoRepeat(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        for col in range(size):
+            values = [board[row][col] for row in range(size) if board[row][col] != 0]
+            if len(set(values)) != len(values):
+                return False
+        return True
+
+class ConstraintVisibility(Constraint):
+    def __init__(self) -> None:
+        super().__init__()
+    def calculate_visible_buildings(self, line: List[int]) -> int:
+        visible = 0
+        max_height = 0
+        for height in line:
+            if int(height) > max_height:
+                visible += 1
+                max_height = int(height)
+        return visible
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        clues = game_state["clues"]
+        size = len(board)
+
+        # Check all directions
+        for i in range(size):
+            # Top clues
+            if clues["top"][i] != 0:
+                col = [board[row][i] for row in range(size)]
+                if 0 not in col and self.calculate_visible_buildings(col) != clues["top"][i]:
+                    return False
+            # Bottom clues
+            if clues["bottom"][i] != 0:
+                col = [board[row][i] for row in range(size-1, -1, -1)]
+                if 0 not in col and self.calculate_visible_buildings(col) != clues["bottom"][i]:
+                    return False
+            # Left clues
+            if clues["left"][i] != 0:
+                if 0 not in board[i] and self.calculate_visible_buildings(board[i]) != clues["left"][i]:
+                    return False
+            # Right clues
+            if clues["right"][i] != 0:
+                if 0 not in board[i] and self.calculate_visible_buildings(board[i][::-1]) != clues["right"][i]:
+                    return False
+        return True
+
+class SkyscraperPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 4 or size > 12:
+            raise ValueError("Grid size must be between 4 and 12")
+        self.game_name = "skyscraper"
+        self.size = size
+        self.constraints = [
+            ConstraintRowNoRepeat(),
+            ConstraintColNoRepeat(),
+            ConstraintVisibility()
+        ]
+        self.all_possible_values = [i for i in range(1, size + 1)]
+        self.possible_hint_counts = [4, 5, 6, 7, 8, 9, 10, 11, 12]
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        board = game_state["board"]
+        original_value = board[row][col]
+        possible_values = []
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/starbattle.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/starbattle.py
new file mode 100644
index 0000000000000000000000000000000000000000..a5bd3e78f73e1a3498b345e0cf2f9c1165e042c1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/starbattle.py
@@ -0,0 +1,130 @@
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+DEBUG_CONSTRAINT_ERROR = False
+
+class ConstraintRowStar(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        for row_idx, row in enumerate(board):
+            if 0 not in row:
+                star_count = sum(1 for cell in row if cell == 's')
+                if star_count != 1:
+                    if DEBUG_CONSTRAINT_ERROR:
+                        print(f"RowStar constraint failed: Row {row_idx} has {star_count} stars (expected 1)")
+                    return False
+            else:
+                star_count = sum(1 for cell in row if cell == 's')
+                if star_count > 1:
+                    if DEBUG_CONSTRAINT_ERROR:
+                        print(f"RowStar constraint failed: Incomplete row {row_idx} has {star_count} stars (max 1)")
+                    return False
+        return True
+
+class ConstraintColStar(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        for col in range(size):
+            col_values = [board[row][col] for row in range(size)]
+
+            if 0 not in col_values:
+                star_count = sum(1 for val in col_values if val == 's')
+                if star_count != 1:
+                    if DEBUG_CONSTRAINT_ERROR:
+                        print(f"ColStar constraint failed: Column {col} has {star_count} stars (expected 1)")
+                    return False
+            else:
+                star_count = sum(1 for val in col_values if val == 's')
+                if star_count > 1:
+                    if DEBUG_CONSTRAINT_ERROR:
+                        print(f"ColStar constraint failed: Incomplete column {col} has {star_count} stars (max 1)")
+                    return False
+        return True
+
+class ConstraintRegionStar(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        regions = game_state["regions"]
+        size = len(board)
+        region_counts = {}
+        for i in range(size):
+            for j in range(size):
+                if board[i][j] == 's':
+                    region = regions[i][j]
+                    region_counts[region] = region_counts.get(region, 0) + 1
+                    if region_counts[region] > 1:
+                        if DEBUG_CONSTRAINT_ERROR:
+                            print(f"RegionStar constraint failed: Region {region} has {region_counts[region]} stars (max 1)")
+                        return False
+
+        for region_num in set(cell for row in regions for cell in row):
+            region_cells = [(i, j) for i in range(size) for j in range(size)
+                          if regions[i][j] == region_num]
+            if all(board[i][j] != 0 for i, j in region_cells):
+                if region_counts.get(region_num, 0) != 1:
+                    if DEBUG_CONSTRAINT_ERROR:
+                        print(f"RegionStar constraint failed: Completed region {region_num} has {region_counts.get(region_num, 0)} stars (expected 1)")
+                    return False
+        return True
+
+class ConstraintAdjacentStar(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        for row in range(size):
+            for col in range(size):
+                if board[row][col] == 's':
+                    for dr in [-1, 0, 1]:
+                        for dc in [-1, 0, 1]:
+                            if dr == 0 and dc == 0:
+                                continue
+                            new_row, new_col = row + dr, col + dc
+                            if (0 <= new_row < size and
+                                0 <= new_col < size and
+                                board[new_row][new_col] == 's'):
+                                if DEBUG_CONSTRAINT_ERROR:
+                                    print(f"AdjacentStar constraint failed: Stars at ({row},{col}) and ({new_row},{new_col}) are adjacent")
+                                return False
+        return True
+
+class StarBattlePuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int, num_stars: int = 1) -> None:
+        super().__init__()
+        self.game_name = "starbattle"
+        self.size = size
+        self.num_stars = num_stars
+        self.colors = [chr(65 + i) for i in range(size)]
+        # During generation, only use row, column, and adjacent constraints
+        self.constraints = [
+            ConstraintRowStar(),
+            ConstraintColStar(),
+            ConstraintAdjacentStar(),
+            ConstraintRegionStar()
+        ]
+
+        self.all_possible_values = ['s', 'e']
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[str]:
+        """Get possible values ('e' for empty or 's' for star) for a given cell."""
+        board = game_state["board"]
+
+        # If the cell is already filled with 'e' or 's', return empty list
+        if board[row][col] in ['s', 'e']:
+            return []
+
+        # Try both values and return those that don't immediately violate constraints
+        possible = []
+        for val in ['s', 'e']:
+            board[row][col] = val
+            if self.check(game_state):
+                possible.append(val)
+            board[row][col] = 0  # Reset to initial state
+        return possible
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/sudoku.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/sudoku.py
new file mode 100644
index 0000000000000000000000000000000000000000..32c63123d6b13469fcbfb35e4e3d90070d22ff1a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/sudoku.py
@@ -0,0 +1,34 @@
+import argparse
+import copy
+import json
+import os
+import random
+from abc import ABC, abstractmethod
+from typing import Any, Dict, List, Tuple, Union
+
+from .common_constriants import (Constraint, ConstraintColNoRepeat, ConstraintRowNoRepeat,
+                                 ConstraintSubGridNoRepeat)
+from .common_puzzle_factory import PuzzleFactory
+
+
+class SudokuPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "sudoku"
+        self.size = size
+
+        self.constraints.append(ConstraintRowNoRepeat())
+        self.constraints.append(ConstraintColNoRepeat())
+        self.constraints.append(ConstraintSubGridNoRepeat())
+
+        self.all_possible_values = [i for i in range(1, size + 1)]
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[int]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/thermometers.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/thermometers.py
new file mode 100644
index 0000000000000000000000000000000000000000..18e5477c95ff2052a79cbbd50ea9c0773fc55b17
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/thermometers.py
@@ -0,0 +1,109 @@
+import argparse
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintThermometerFill(Constraint):
+    """Check if thermometers are filled correctly (from bulb to top, no gaps)"""
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_thermometer_fill"
+
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        thermometers = game_state.get("clues", {}).get("thermometers", [])  # Fixed: get thermometers from clues
+
+
+        # Create a set of all thermometer positions for efficient lookup
+        thermometer_positions = {(r, c) for therm in thermometers for r, c in therm}
+
+        # Check non-thermometer cells are empty or undefined
+        for i in range(len(board)):
+            for j in range(len(board[i])):
+                if (i, j) not in thermometer_positions and board[i][j] == "s":
+                    return False
+        # Check thermometer filling rules
+        for thermometer in thermometers:
+            # Find first empty cell in thermometer
+            first_empty = -1
+            for i, (r, c) in enumerate(thermometer):
+                if board[r][c] == "e":  # if empty
+                    first_empty = i
+                    break
+            # After first empty, all cells must be empty
+            if first_empty != -1:
+                for i, (r, c) in enumerate(thermometer):
+                    if i > first_empty and board[r][c] == "s":  # if selected
+                        return False
+        return True
+
+class ConstraintThermometerCount(Constraint):
+    """Check if row and column counts match the clues"""
+    def __init__(self) -> None:
+        super().__init__()
+        self.name = "constraint_thermometer_count"
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        clues = game_state.get("clues", None)
+        if not clues:
+            return True
+
+        size = len(board)
+        row_counts = clues["row_counts"]
+        col_counts = clues["col_counts"]
+
+        # Check rows
+        for i in range(size):
+            row_selected = sum(1 for j in range(size) if board[i][j] == "s")
+            row_undefined = sum(1 for j in range(size) if board[i][j] == 0)
+            if 0 not in board[i]:  # if row is complete
+                if row_selected != row_counts[i]:
+                    return False
+            else:  # if row is incomplete
+                if row_selected > row_counts[i]:  # too many selected
+                    return False
+                if row_selected + row_undefined < row_counts[i]:  # impossible to reach target
+                    return False
+        # Check columns
+        for j in range(size):
+            col_selected = sum(1 for i in range(size) if board[i][j] == "s")
+            col_undefined = sum(1 for i in range(size) if board[i][j] == 0)
+            if all(board[i][j] != 0 for i in range(size)):  # if column is complete
+                if col_selected != col_counts[j]:
+                    return False
+            else:  # if column is incomplete
+                if col_selected > col_counts[j]:  # too many selected
+                    return False
+                if col_selected + col_undefined < col_counts[j]:  # impossible to reach target
+                    return False
+        return True
+
+class ThermometersPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        if size < 4:
+            raise ValueError("Size must be at least 4")
+        self.game_name = "thermometers"
+        self.size = size
+        self.constraints = [
+            ConstraintThermometerFill(),
+            ConstraintThermometerCount()
+        ]
+
+        self.all_possible_values = ["e", "s"]  # empty or selected
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[str]:
+        possible_values = []
+        board = game_state["board"]
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible_values.append(value)
+        board[row][col] = original_value
+        return possible_values
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/treesandtents.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/treesandtents.py
new file mode 100644
index 0000000000000000000000000000000000000000..014a00eb828951b1d51f3f2c73682d5165ed5301
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/puzzles/treesandtents.py
@@ -0,0 +1,180 @@
+import argparse
+import copy
+import json
+import os
+import random
+from typing import Any, Dict, List, Tuple
+
+from .common_constriants import Constraint
+from .common_puzzle_factory import PuzzleFactory
+
+
+class ConstraintRowTents(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+
+        board = game_state["board"]
+        # if board[0][0] == 'e' and board[0][1] == 'e':
+        #     import ipdb; ipdb.set_trace()
+        clues = game_state.get("clues", None)
+        if not clues:
+            return True
+        for i, row in enumerate(board):
+            if 0 not in row:  # If row is complete
+                tent_count = row.count("tt")
+                if tent_count != clues["row_clues"][i]:
+                    return False
+            else:  # If row is incomplete
+                tent_count = row.count("tt")
+                if tent_count > clues["row_clues"][i]:
+                    return False
+        return True
+
+class ConstraintColTents(Constraint):
+    def check(self, game_state: Dict[str, Any]) -> bool:
+
+        board = game_state["board"]
+        clues = game_state.get("clues", None)
+        if not clues:
+            return True
+        size = len(board)
+        for j in range(size):
+            col = [board[i][j] for i in range(size)]
+            if 0 not in col:  # If column is complete
+                tent_count = col.count("tt")
+                if tent_count != clues["col_clues"][j]:
+                    return False
+            else:  # If column is incomplete
+                tent_count = col.count("tt")
+                if tent_count > clues["col_clues"][j]:
+                    return False
+        return True
+
+class ConstraintTentTree(Constraint):
+    """
+    Check if:
+    1. Each tent has exactly one adjacent tree (horizontally or vertically)
+    2. Each tree has exactly one adjacent tent (horizontally or vertically) when complete
+    3. Each tree should have exactly one tent or potential tent spot (empty cell) adjacent
+    """
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        # Keep track of which trees are paired with which tents
+        tree_tent_pairs = {}  # tree position -> tent position
+
+        # First, check each tent has exactly one adjacent tree
+        for i in range(size):
+            for j in range(size):
+                if board[i][j] == "tt":
+                    adjacent_trees = []
+                    for di, dj in [(-1,0), (1,0), (0,-1), (0,1)]:  # Only orthogonal
+                        ni, nj = i + di, j + dj
+                        if 0 <= ni < size and 0 <= nj < size:
+                            if board[ni][nj] == "tr":
+                                adjacent_trees.append((ni, nj))
+                    # Each tent must have exactly one adjacent tree
+                    if len(adjacent_trees) != 1:
+                        return False
+
+                    tree_pos = adjacent_trees[0]
+
+                    tree_tent_pairs[tree_pos] = (i, j)
+
+        # Then, check each tree
+        for i in range(size):
+            for j in range(size):
+                if board[i][j] == "tr":
+                    # Count adjacent tents and empty cells
+                    adjacent_tents = 0
+                    adjacent_non_allocated = 0
+                    for di, dj in [(-1,0), (1,0), (0,-1), (0,1)]:
+                        ni, nj = i + di, j + dj
+                        if 0 <= ni < size and 0 <= nj < size:
+                            if board[ni][nj] == "tt":
+                                adjacent_tents += 1
+                            elif board[ni][nj] == 0:
+                                adjacent_non_allocated += 1
+
+                    if adjacent_tents > 1:
+                        return False
+                    if adjacent_tents == 1:
+                        pass
+                    if adjacent_tents == 0:
+                        if adjacent_non_allocated == 0:
+                            return False
+
+        return True
+
+class ConstraintAdjacentTents(Constraint):
+    """
+    Check if tents are not adjacent (including diagonally).
+    """
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+        # Check tents are not adjacent (including diagonally)
+        for i in range(size):
+            for j in range(size):
+                if board[i][j] == "tt":
+                    for di in [-1, 0, 1]:
+                        for dj in [-1, 0, 1]:
+                            if di == 0 and dj == 0:
+                                continue
+                            ni, nj = i + di, j + dj
+                            if 0 <= ni < size and 0 <= nj < size:
+                                if board[ni][nj] == "tt":
+                                    return False
+        return True
+
+class ConstraintTentTreeCount(Constraint):
+    """
+    Check if:
+    1. Number of tents + unallocated cells >= number of trees (during solving)
+    2. Number of tents == number of trees (for completed board)
+    """
+    def check(self, game_state: Dict[str, Any]) -> bool:
+        board = game_state["board"]
+        size = len(board)
+
+        num_trees = sum(row.count("tr") for row in board)
+        num_tents = sum(row.count("tt") for row in board)
+        num_unallocated = sum(row.count(0) for row in board)
+
+        # If board is complete (no unallocated cells)
+        if num_unallocated == 0:
+            return num_tents == num_trees
+
+        # During solving, ensure we can still potentially place enough tents
+        return (num_tents + num_unallocated) >= num_trees
+
+
+class TreesAndTentsPuzzleFactory(PuzzleFactory):
+    def __init__(self, size: int) -> None:
+        super().__init__()
+        self.game_name = "treesandtents"
+        self.size = size
+        assert size >= 3, "Size must be at least 3"
+        self.constraints = [
+            ConstraintRowTents(),
+            ConstraintColTents(),
+            ConstraintTentTree(),
+            ConstraintAdjacentTents(),
+            ConstraintTentTreeCount()
+        ]
+        self.all_possible_values = ["tt", 'e']
+        self.num_generator_processes = max(os.cpu_count() // 2, 1)  # Limit to 4 processes or CPU count
+
+    def get_possible_values(self, game_state: Dict[str, Any], row: int, col: int) -> List[str]:
+        """Get possible values for a given cell."""
+        board = game_state["board"]
+        if board[row][col] != 0:  # If cell is already filled
+            return []
+        possible = []
+        original_value = board[row][col]
+        for value in self.all_possible_values:
+            board[row][col] = value
+            if self.check(game_state):
+                possible.append(value)
+        board[row][col] = original_value
+        return possible
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/score.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/score.py
new file mode 100644
index 0000000000000000000000000000000000000000..15b5b274787c57f8d241896699669673a856e10b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/dataset/utils/vgrpbench/score.py
@@ -0,0 +1,440 @@
+"""
+VGRPBench scoring module for evaluating visual grid reasoning puzzle solutions.
+
+This module provides functions to evaluate puzzle solutions from language models,
+including parsing model outputs, checking perception accuracy, and verifying solutions.
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+
+import numpy as np
+from json_repair import repair_json
+from tqdm import tqdm
+
+from . import puzzles
+from .puzzles import common_get_game_factory as get_game_factory
+
+# Global variable to store the puzzle grid size
+GRID_SIZE = None
+
+
+def extract_perception_and_answer(model_output):
+    """
+    Extract both perception and answer from model output.
+
+    Parses the model's output to extract the perceived initial state and the solution.
+    Handles different output formats and section headers.
+
+    Args:
+        model_output (str): The raw output from the model
+
+    Returns:
+        tuple: (initial_state, solution) where both are 2D arrays or None if parsing fails
+    """
+    try:
+        # Handle plain text format
+        if "Initial State" in model_output:
+            parts = model_output.split('Initial State\n', 1)
+        elif "Perception" in model_output:
+            parts = model_output.split('Perception\n', 1)
+        else:
+            return None, None
+
+        if len(parts) != 2:
+            return None, None
+        content = parts[1]
+
+        if "Answer" in content:
+            perception_answer = content.split('\nAnswer\n')
+        elif "Solution" in content:
+            perception_answer = content.split('\nSolution\n')
+        else:
+            return None, None
+
+        if len(perception_answer) != 2:
+            return None, None
+
+        perception, answer = perception_answer
+
+        if perception.strip() == "Wrong":
+            initial_state = None
+            # Remove outer brackets and split into rows
+            raw_solution = answer.strip()[2:-2].split('],[')
+            solution = [[c for c in row.split(',')] for row in raw_solution]
+        else:
+            if answer.strip() == "Wrong":
+                raw_perception = perception.strip()[2:-2].split('],[')
+                initial_state = [[c for c in row.split(',')] for row in raw_perception]
+                solution = None
+            else:
+                # Remove outer brackets and split into rows
+                raw_perception = perception.strip()[2:-2].split('],[')
+                initial_state = [[c for c in row.split(',')] for row in raw_perception]
+                raw_solution = answer.strip()[2:-2].split('],[')
+                solution = [[c for c in row.split(',')] for row in raw_solution]
+
+        initial_state = [[cell if cell != '*' else 0 for cell in row] for row in initial_state]
+
+        return initial_state, solution
+    except Exception as e:
+        print(f"Error parsing output: {e}")
+        return None, None
+
+
+def check_perception(thoughts, init_board, game_type):
+    """
+    Check if model's perception matches the initial board.
+
+    Compares the model's understanding of the initial state with the actual initial state,
+    with game-specific adjustments for different puzzle types.
+
+    Args:
+        thoughts (list): 2D array representing the model's perception of the initial state
+        init_board (list): 2D array representing the actual initial state
+        game_type (str): Type of puzzle game
+    Returns:
+        bool: True if perception matches initial board, False otherwise
+    """
+    # Game-specific adjustments
+    if game_type == "battleships":
+        init_board = [[0 if cell == 'e' else cell for cell in row] for row in init_board]
+        thoughts = [[0 if cell == 'e' else cell for cell in row] for row in thoughts]
+    if game_type == "lightup":
+        for i in range(len(init_board)):
+            for j in range(len(init_board[i])):
+                cell = init_board[i][j]
+                # Check if cell is a number (not 0) or not a string/character
+                if (isinstance(cell, (int, float)) and cell != 0) or (isinstance(cell, str) and not cell.isalpha()):
+                    init_board[i][j] = 'w'
+    if game_type == "fieldexplore":
+        # Convert -1 to 0 in init_board
+        init_board = [[0 if cell == -1 else cell for cell in row] for row in init_board]
+    # Convert string representation to 2D grid if needed
+    if isinstance(init_board, str):
+        init_grid = [[c for c in row] for row in init_board.strip().split('\n')]
+    else:
+        init_grid = init_board
+
+    # Check dimensions match
+    if len(thoughts) != len(init_grid) or any(len(row) != len(init_grid[0]) for row in thoughts):
+        return False
+    # Check cell by cell
+    for i in range(len(init_grid)):
+        for j in range(len(init_grid[0])):
+            if str(init_grid[i][j]) != str(thoughts[i][j]):
+                return False
+    return True
+
+
+def check_answer(answer, init_board, game_factory):
+    """
+    Verify if the model's answer is correct for the given puzzle.
+
+    Performs game-specific validations and uses the game factory to check solution correctness.
+
+    Args:
+        answer (list): 2D array representing the model's solution
+        init_board (list): 2D array representing the initial state
+        game_factory (GameFactory): Factory object for the specific game type
+    Returns:
+        bool: True if the answer is correct, False otherwise
+    """
+    global GRID_SIZE
+    # Game-specific preprocessing for answers
+    if game_factory.game_name in ["treesandtents", "starbattle", "hitori", "aquarium", "kakurasu"]:
+        for i in range(len(answer)):
+            for j in range(len(answer[i])):
+                if answer[i][j] in [0, '0']:
+                    answer[i][j] = 'e'
+    if game_factory.game_name == "oddevensudoku":
+        for i in range(len(answer)):
+            for j in range(len(answer[i])):
+                try:
+                    answer[i][j] = int(answer[i][j])
+                except Exception as e:
+                    return False
+    if game_factory.game_name == "lightup":
+        # Convert '0' to 'e'
+        for i in range(len(answer)):
+            for j in range(len(answer[i])):
+                if answer[i][j] == '0':
+                    answer[i][j] = 'e'
+    # Convert string representation to 2D grid if needed
+    if isinstance(init_board, str):
+        init_grid = [[c for c in row] for row in init_board.strip().split('\n')]
+    else:
+        init_grid = init_board
+    # Check dimensions
+    if len(answer) != GRID_SIZE or any(len(row) != GRID_SIZE for row in answer):
+        return False
+
+    # Game-specific validation for initial values
+    if game_factory.game_name == "hitori":
+        # Compare with game_factory.additional_board
+        for i in range(GRID_SIZE):
+            for j in range(GRID_SIZE):
+                if game_factory.additional_board[i][j] not in [0, '0'] and str(game_factory.additional_board[i][j]) != str(answer[i][j]):
+                    return False
+    elif game_factory.game_name == "nonogram":
+        # Convert 0, '0', '*' in answer to 'e'
+        for i in range(GRID_SIZE):
+            for j in range(GRID_SIZE):
+                if answer[i][j] in [0, '0', '*']:
+                    answer[i][j] = 'e'
+        for i in range(GRID_SIZE):
+            for j in range(GRID_SIZE):
+                if init_grid[i][j] not in [0, '0'] and str(init_grid[i][j]) != str(answer[i][j]):
+                    return False
+    elif game_factory.game_name == "fieldexplore":
+        for i in range(GRID_SIZE):
+            for j in range(GRID_SIZE):
+                # 's' on the initial board must be kept
+                if init_grid[i][j] == 's' and not answer[i][j] == 's':
+                    return False
+                try:
+                    cell_value = int(init_grid[i][j])
+                    if cell_value > 0 and str(answer[i][j]) == 's':
+                        return False
+                except (ValueError, TypeError):
+                    # Cell is not a number, continue with other checks
+                    pass
+        return True
+    else:
+        for i in range(GRID_SIZE):
+            for j in range(GRID_SIZE):
+                if init_grid[i][j] not in [0, '0', 'e'] and str(init_grid[i][j]) != str(answer[i][j]):
+                    return False
+    # Prepare game state for validation
+    game_state = {
+        "board": answer,
+        "size": GRID_SIZE,
+    }
+
+    # Add game-specific state information
+    if game_factory.game_name == "skyscraper":
+        game_state["clues"] = game_factory.clues
+    elif game_factory.game_name == "coloredsudoku":
+        game_state["colors"] = game_factory.current_colors
+    elif game_factory.game_name == "futoshiki":
+        game_state["inequalities"] = game_factory.current_inequalities
+    elif game_factory.game_name == "killersudoku":
+        game_state["cages"] = game_factory.cages
+    elif game_factory.game_name == "renzoku":
+        game_state["hints"] = game_factory.hints
+    elif game_factory.game_name == 'kakuro':
+        game_state["sums"] = game_factory.current_sums
+    elif game_factory.game_name == "thermometers":
+        game_state["clues"] = game_factory.clues
+    elif game_factory.game_name == "treesandtents":
+        game_state["clues"] = game_factory.clues
+    elif game_factory.game_name == "starbattle":
+        game_state["regions"] = game_factory.regions
+    elif game_factory.game_name == "hitori":
+        game_state["numbers"] = game_factory.numbers
+    elif game_factory.game_name == "aquarium":
+        game_state["clues"] = game_factory.clues
+    elif game_factory.game_name == "kakurasu":
+        game_state["clues"] = game_factory.clues
+    elif game_factory.game_name == "oddevensudoku":
+        game_state["cell_types"] = game_factory.cell_types
+    elif game_factory.game_name == "nonogram":
+        game_state["hints"] = game_factory.hints
+    elif game_factory.game_name == "lightup":
+        game_state["wall_numbers"] = game_factory.wall_numbers
+    elif game_factory.game_name == "battleships":
+        game_state["hints"] = game_factory.hints
+    # Validate the solution using the game factory
+    try:
+        return game_factory.check(game_state)
+    except Exception as e:
+        print(f"Error checking answer: {e}")
+        return False
+
+
+def calculate_group_statistics(outcomes, num_groups=5):
+    """
+    Calculate group-wise means and the standard deviation between groups.
+
+    Splits outcomes into groups and calculates statistics to estimate variance.
+
+    Args:
+        outcomes (list): Binary outcomes (0 or 1) for each puzzle
+        num_groups (int): Number of groups to split the data into
+
+    Returns:
+        tuple: (group_means, group_std) where group_means is a list of percentages
+               and group_std is the standard deviation between groups
+    """
+    if not outcomes:
+        return [], 0.0
+
+    # Convert to numpy array for easier manipulation
+    outcomes = np.array(outcomes)
+
+    # Calculate number of items per group
+    group_size = len(outcomes) // num_groups
+
+    # Split into groups and calculate mean for each group
+    group_means = []
+    for i in range(num_groups):
+        start_idx = i * group_size
+        end_idx = start_idx + group_size if i < num_groups - 1 else len(outcomes)
+        group = outcomes[start_idx:end_idx]
+        group_means.append(np.mean(group) * 100)  # Convert to percentage
+
+    # Calculate standard deviation between group means
+    group_std = np.std(group_means)
+
+    return group_means, group_std
+
+
+def evaluate_single_puzzle(model_output, puzzle_data, game_type):
+    """
+    Evaluate a single puzzle solution.
+
+    Processes model output and puzzle data to determine if the model correctly
+    understood the puzzle and provided a valid solution.
+
+    Args:
+        model_output (str): The raw output from the model
+        puzzle_data (dict): Puzzle data including initialization
+        game_type (str): Type of puzzle game (e.g., "thermometers", "sudoku")
+    Returns:
+        dict: Evaluation results including perception_correct, answer_correct, and score
+    """
+    # Add puzzle directory to path if needed
+    curr_dir = os.path.dirname(os.path.abspath(__file__))
+    puzzle_dir = os.path.join(curr_dir, "puzzles")
+    if puzzle_dir not in sys.path:
+        sys.path.append(puzzle_dir)
+
+    # Initialize the appropriate game factory for the puzzle type
+    GameFactory = get_game_factory.get_game_factory(game_type)
+
+    init_board = puzzle_data['initialization']
+
+    game_factory = GameFactory(size=4)
+
+    # Game-specific initialization handling
+    if game_type == "coloredsudoku":
+        colors = puzzle_data.get('colors', None)
+        game_factory.current_colors = colors
+    elif game_type == "binairo":
+        init_board = puzzle_data.get('initialization', None)
+    elif game_type == "futoshiki":
+        row_inequalities = puzzle_data.get('row_inequalities', None)
+        col_inequalities = puzzle_data.get('col_inequalities', None)
+        game_factory.current_inequalities = {
+            "row": row_inequalities,
+            "col": col_inequalities
+        }
+    elif game_type == "killersudoku":
+        cages = puzzle_data.get('cages', None)
+        game_factory.cages = cages
+    elif game_type == "renzoku":
+        hints = puzzle_data.get('hints', None)
+        game_factory.hints = hints
+    elif game_type == "kakuro":
+        sums = puzzle_data.get('sums', None)
+        game_factory.current_sums = sums
+    elif game_type == "skyscraper":
+        clues = puzzle_data.get('initialization', None).get('clues')
+        init_board = puzzle_data.get('initialization', None).get('board')  # Special case
+        game_factory.clues = clues
+    elif game_type == "thermometers":
+        clues = puzzle_data.get('initialization', None).get('clues')
+        game_factory.clues = clues
+        init_board = puzzle_data.get('initialization', None).get('board')
+    elif game_type == "treesandtents":
+        clues = puzzle_data.get('clues', None)
+        game_factory.clues = clues
+        init_board = puzzle_data.get('initialization', None)
+    elif game_type == "starbattle":
+        init_board = puzzle_data.get('initialization', None)
+        game_factory.regions = puzzle_data.get('regions', None)
+    elif game_type == "hitori":
+        init_board = puzzle_data.get('initialization').get('numbers', None)
+        game_factory.numbers = puzzle_data.get('initialization', None).get('numbers')
+        game_factory.additional_board = puzzle_data.get('initialization', None).get('board')
+    elif game_type == "aquarium":
+        init_board = puzzle_data.get('initialization', None).get('board')
+        game_factory.clues = puzzle_data.get('initialization', None).get('clues', None)
+    elif game_type == "kakurasu":
+        init_board = puzzle_data.get('initialization', None).get('board')
+        game_factory.clues = puzzle_data.get('initialization', None).get('clues', None)
+    elif game_type == "oddevensudoku":
+        game_factory.cell_types = puzzle_data.get('cell_types')
+        init_board = puzzle_data.get('initialization', None)
+    elif game_type == "battleships":
+        init_board = puzzle_data.get('initialization', None)
+        game_factory.hints = puzzle_data.get('hints', None)
+    elif game_type == "jigsawsudoku":
+        init_board = puzzle_data.get('initialization', None)
+    elif game_type == "nonogram":
+        init_board = puzzle_data.get('initialization', None)
+        game_factory.hints = puzzle_data.get('hints', None)
+    elif game_type == "lightup":
+        init_board = puzzle_data.get('initialization', None)
+        game_factory.wall_numbers = puzzle_data.get('wall_numbers', None)
+    # Set grid size
+    global GRID_SIZE
+    GRID_SIZE = len(init_board) if GRID_SIZE is None else GRID_SIZE
+
+    # Extract model's perception and answer from its output
+    thoughts, answer = extract_perception_and_answer(model_output)
+    # Early return if parsing failed
+    if thoughts is None or answer is None:
+        return {
+            "perception_correct": False,
+            "answer_correct": False,
+            "number_of_samples": 1
+        }
+
+    # Game-specific preprocessing
+    try:
+        if game_type == "starbattle":
+            for i in range(len(thoughts)):
+                for j in range(len(thoughts[i])):
+                    if thoughts[i][j] == "*":
+                        thoughts[i][j] = "0"
+    except Exception as e:
+        print(f"starbattle: Error converting thoughts to 0: {e}")
+    try:
+        if game_type == "killersudoku":
+            answer = [[int(cell) for cell in row] for row in answer]
+    except Exception as e:
+        answer = None
+
+    # Special handling for trees and tents
+    if game_type == "treesandtents":
+        # Convert shorthand symbols to standard format
+        for i in range(len(thoughts)):
+            for j in range(len(thoughts[i])):
+                if thoughts[i][j] == 't':
+                    thoughts[i][j] = 'tt'
+                elif thoughts[i][j] == 'r':
+                    thoughts[i][j] = 'tr'
+        for i in range(len(answer)):
+            for j in range(len(answer[i])):
+                if answer[i][j] == 't':
+                    answer[i][j] = 'tt'
+                elif answer[i][j] == 'r':
+                    answer[i][j] = 'tr'
+
+    # Check perception and answer
+    perception_correct = check_perception(thoughts, init_board, game_type)
+    answer_correct = check_answer(answer, init_board, game_factory) if perception_correct else False
+
+    return {
+        "perception_correct": perception_correct,
+        "answer_correct": answer_correct,
+        "number_of_samples": 1
+    }
+
+
+if __name__ == "__main__":
+    main()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/granite_vision/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/granite_vision/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..d92a3f46dac14a69931eec61d902b18e09806127
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/granite_vision/__init__.py
@@ -0,0 +1,3 @@
+from .granite_vision import GraniteVision3
+
+__all__ = ['GraniteVision3']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/granite_vision/granite_vision.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/granite_vision/granite_vision.py
new file mode 100644
index 0000000000000000000000000000000000000000..074bc11dd499b47d75667b89a0c3ce02ee2dfd8c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/granite_vision/granite_vision.py
@@ -0,0 +1,194 @@
+import string
+import warnings
+
+import pandas as pd
+import torch
+from PIL import Image
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import cn_string
+from ..base import BaseModel
+
+try:
+    from transformers import AutoModelForVision2Seq, AutoProcessor
+except ImportError:
+    from transformers import AutoModelForImageTextToText as AutoModelForVision2Seq
+    from transformers import AutoProcessor
+
+flash_attn_flag = False
+try:
+    import flash_attn  # noqa: F401
+    flash_attn_flag = True
+except ImportError:
+    pass
+
+
+class GraniteVision3(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(
+        self, model_path="ibm-granite/granite-vision-3.3-2b", use_vllm=False, **kwargs
+    ):
+        # assert not use_vllm "vLLM is not yet supported for evaluations in VLMEvalKit"
+        self.model_path = model_path
+        self.processor = AutoProcessor.from_pretrained(self.model_path)
+        attn_impl = "flash_attention_2" if flash_attn_flag else "eager"
+        model = AutoModelForVision2Seq.from_pretrained(
+            self.model_path,
+            torch_dtype=torch.bfloat16,
+            low_cpu_mem_usage=True,
+            attn_implementation=attn_impl
+        )
+
+        model = model.eval()
+        self.model = model.cuda()
+        kwargs_default = dict(do_sample=False, max_new_tokens=2048)
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+        warnings.warn(
+            f"Following kwargs received: {self.kwargs}, will use as generation config. "
+        )
+
+    def output_process(self, answer, dataset):
+        if "<s>" in answer:
+            answer = answer.replace("<s>", "").strip()
+        if "<|assistant|>" in answer:
+            answer = answer.split("<|assistant|>")[1].strip("\n .")
+        elif "<|start_of_role|>assistant<|end_of_role|>" in answer:
+            answer = answer.split("<|start_of_role|>assistant<|end_of_role|>")[1].strip(
+                "\n ."
+            )
+
+        if "<|end_of_text|>" in answer:
+            answer = answer.split("<|end_of_text|>")[0].strip("\n ")
+        if "answer" in answer.lower():
+            answer = answer.lower().split("answer")[-1].strip(" :.-\n")
+        if dataset in [
+            "ChartQA_TEST",
+            "DocVQA_VAL",
+            "DocVQA_TEST",
+            "InfoVQA_VAL",
+            "InfoVQA_TEST",
+            "OCRVQA_TEST",
+            "OCRVQA_TESTCORE",
+            "TextVQA_VAL"
+        ]:
+            answer = answer.strip(".")
+        if "ChartMuseum" in dataset:
+            answer = f"<answer>{answer}</answer>"
+        return answer.strip("\n")
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == "MCQ":
+            return True
+        if dataset in ["OCRBench", "COCO_VAL", "ChartQA_TEST", "CharXiv_descriptive_val", "ChartMimic_v1_direct",
+                       "ChartMimic_v2_direct", "ChartMimic_v2_customized",]:
+            return True
+        return False
+
+    def get_pre_post_prompt(self, dataset, chineese=False):
+        pre_post_prompt = {
+            "OCRBench": (
+                "",
+                "\nReply with only one word or a short phrase or a full address.",
+            ),
+            "COCO_VAL": ("", "\nReply with one short sentence."),
+            "ChartQA_TEST": ("", "\nAnswer the question with a single word."),
+            "CharXiv_descriptive_val": ("", "\nAnswer the question with a single word or short phrase."),
+            "ChartMimic_v1_direct": ("", "\nAnswer using code only. strating with ```python and ending with ```"),
+            "ChartMimic_v2_direct": ("", "\nAnswer using code only. strating with ```python and ending with ```"),
+            "ChartMimic_v2_customized": ("", "\nAnswer using code only. strating with ```python and ending with ```"),
+        }
+        pre_post_prompt_cn = {}
+
+        return (
+            pre_post_prompt.get(dataset, ("", ""))
+            if not chineese
+            else pre_post_prompt_cn.get(dataset, ("", ""))
+        )
+
+    def build_promt_mcq(self, line):
+        question = line["question"]
+        hint = line["hint"] if ("hint" in line and not pd.isna(line["hint"])) else None
+        if hint is not None:
+            question = hint + "\n" + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f"\n{key}. {item}"
+        prompt = question
+
+        if len(options):
+            prompt += (
+                "\n请直接回答选项字母。"
+                if cn_string(prompt)
+                else "\nAnswer with the option's letter from the given choices directly."
+            )
+        else:
+            prompt += (
+                "\n请直接回答问题。"
+                if cn_string(prompt)
+                else "\nAnswer the question directly."
+            )
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+
+        tgt_path = self.dump_image(line, dataset)
+        if DATASET_TYPE(dataset) == "MCQ":
+            prompt = self.build_promt_mcq(line)
+        else:
+            prompt = line["question"]
+        pre_promt, post_prompt = self.get_pre_post_prompt(
+            dataset, chineese=cn_string(prompt)
+        )
+        prompt = pre_promt + prompt + post_prompt
+        message = [dict(type="image", value=s) for s in tgt_path]
+        message.append(dict(type="text", value=prompt))
+
+        return message
+
+    def resize_to_max_dim(self, image: Image.Image, max_dim: int = 768) -> Image.Image:
+        """Resize image so the longer side is exactly `max_dim` pixels."""
+        w, h = image.size
+        scale = max_dim / max(w, h)
+        new_size = (int(w * scale), int(h * scale))
+        # print(f"Resized image from {(w,h)} to {new_size}")
+        return image.resize(new_size, Image.LANCZOS)
+
+    def generate_inner(self, message, dataset=None):
+        content, images = [], []
+        img_count = 0
+        for msg in message:
+            if not msg["type"] == "text":
+                img_count += 1
+        for msg in message:
+            if msg["type"] == "text":
+                content.append({"type": msg["type"], "text": msg["value"]})
+            else:
+                content.append({"type": "image"})
+                img = Image.open(msg["value"]).convert("RGB")
+                if img_count > 2:
+                    img = self.resize_to_max_dim(img)
+                images.append(img)
+        conversation = [
+            {
+                "role": "user",
+                "content": content,
+            }
+        ]
+        prompt = self.processor.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)
+        inputs = self.processor(images=images, text=prompt, return_tensors="pt").to(self.model.device, self.model.dtype)
+        with torch.no_grad():
+            output = self.model.generate(**inputs, **self.kwargs)
+        answer = self.processor.decode(output[0], skip_special_token=True)
+        answer = self.output_process(answer, dataset)
+        return answer
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..ece683260db5d6d48625b5f070461554878b3b55
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/__init__.py
@@ -0,0 +1,2 @@
+from .model import HawkVL  # noqa: F401
+from .prompt import HawkVLPromptMixin  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..b99209485d73cad4a382994da139af64d9267dfc
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/__init__.py
@@ -0,0 +1 @@
+from .model import HawkQwenForCausalLM  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/constants.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/constants.py
new file mode 100644
index 0000000000000000000000000000000000000000..68d2385740f4bbc258bd59b7e669e02f1979d490
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/constants.py
@@ -0,0 +1,14 @@
+# Model Constants
+IGNORE_INDEX = -100
+IMAGE_TOKEN_INDEX = 151655
+VIDEO_TOKEN_INDEX = 151656
+
+# the traditional way for a image placeholder in training data.
+DEFAULT_IMAGE_TOKEN = "<image>"
+DEFAULT_VIDEO_TOKEN = "<video>"
+
+VISION_START_TOKEN = "<|vision_start|>"
+VISION_END_TOKEN = "<|vision_end|>"
+VISION_TOKEN = "<|vision_pad|>"
+IMAGE_TOKEN = "<|image_pad|>"   # for each image token feature
+VIDEO_TOKEN = "<|video_pad|>"   # for each video token feature
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..fa1f6cc1a0be0a941a3825d3bd8eb97b80059779
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/__init__.py
@@ -0,0 +1 @@
+from .language_model.hawk_qwen import HawkQwenConfig, HawkQwenForCausalLM  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/hawk_arch.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/hawk_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..c40fd099b3e91d6657705d0f9dc2c5d5f0bf1f36
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/hawk_arch.py
@@ -0,0 +1,132 @@
+#    Adapted from LLaVA code.
+#    Copyright 2025 Shukang Yin
+#
+#    Licensed under the Apache License, Version 2.0 (the "License");
+#    you may not use this file except in compliance with the License.
+#    You may obtain a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS,
+#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#    See the License for the specific language governing permissions and
+#    limitations under the License.
+
+
+from abc import ABC, abstractmethod
+from typing import List, Optional
+
+import torch
+
+from ..constants import IGNORE_INDEX
+from .multimodal_projector.builder import build_vision_projector
+from .vision_encoder import VISION_TRANSFORMER_CLASSES
+
+
+class HawkMetaModel:
+
+    def __init__(self, config):
+        super(HawkMetaModel, self).__init__(config)
+        if hasattr(config, "mm_vision_tower"):
+            vision_config = config.vision_config
+            vision_config._attn_implementation = config._attn_implementation
+            self.vision_tower = VISION_TRANSFORMER_CLASSES[vision_config.model_type]._from_config(
+                vision_config
+            )
+
+            self.mm_projector = build_vision_projector(vision_config)
+
+    def get_vision_tower(self):
+        vision_tower = getattr(self, "vision_tower", None)
+        if type(vision_tower) is list:
+            vision_tower = vision_tower[0]
+        return vision_tower
+
+
+class HawkMetaForCausalLM(ABC):
+
+    @abstractmethod
+    def get_model(self):
+        pass
+
+    def get_vision_tower(self):
+        return self.get_model().get_vision_tower()
+
+    def prepare_inputs_labels_for_multimodal(
+        self,
+        input_ids: torch.LongTensor = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[List[torch.FloatTensor]] = None,
+        labels: Optional[torch.LongTensor] = None,
+        pixel_values: Optional[torch.Tensor] = None,
+        pixel_values_videos: Optional[torch.FloatTensor] = None,
+        image_grid_thw: Optional[torch.LongTensor] = None,
+        video_grid_thw: Optional[torch.LongTensor] = None
+    ):
+
+        vision_tower = self.get_vision_tower()
+
+        if vision_tower is None or (pixel_values is None and pixel_values_videos is None) or input_ids.shape[1] == 1:
+            return input_ids, position_ids, attention_mask, past_key_values, None, labels
+
+        inputs_embeds = self.get_input_embeddings()(input_ids).clone()
+
+        ignore_flag = False
+
+        if pixel_values is not None:
+            image_embeds = vision_tower(
+                pixel_values,
+                grid_thw=image_grid_thw)
+
+            image_embeds = self.get_model().mm_projector(image_embeds)
+
+            n_image_tokens = (input_ids == self.image_token_id).sum().item()
+            n_image_features = image_embeds.shape[0]
+
+            if n_image_tokens != n_image_features:
+                print(f'warning:  n_image_tokens={n_image_tokens}, while'
+                      f'n_image_features={n_image_features}. '
+                      f'will ignore this batch')
+                image_embeds = image_embeds[:n_image_tokens]
+                ignore_flag = True
+
+            image_mask = (
+                (input_ids == self.image_token_id)
+                .unsqueeze(-1)
+                .expand_as(inputs_embeds)
+                .to(inputs_embeds.device)
+            )
+            image_embeds = image_embeds.to(inputs_embeds.device, inputs_embeds.dtype)
+            inputs_embeds = inputs_embeds.masked_scatter(image_mask, image_embeds)
+
+        if pixel_values_videos is not None:
+            video_embeds = vision_tower(
+                pixel_values_videos,
+                grid_thw=video_grid_thw)
+
+            video_embeds = self.get_model().mm_projector(video_embeds)
+
+            n_video_tokens = (input_ids == self.video_token_id).sum().item()
+            n_video_features = video_embeds.shape[0]
+            if n_video_tokens != n_video_features:
+                print(f'warning:  n_video_tokens={n_video_tokens}, while'
+                      f'n_video_features={n_video_features}. '
+                      f'will ignore this batch')
+                video_embeds = video_embeds[:n_video_tokens]
+                ignore_flag = True
+
+            video_mask = (
+                (input_ids == self.video_token_id)
+                .unsqueeze(-1)
+                .expand_as(inputs_embeds)
+                .to(inputs_embeds.device)
+            )
+            video_embeds = video_embeds.to(inputs_embeds.device, inputs_embeds.dtype)
+            inputs_embeds = inputs_embeds.masked_scatter(video_mask, video_embeds)
+
+        if labels is not None and ignore_flag:
+            labels = torch.full_like(input_ids, IGNORE_INDEX)
+
+        return None, position_ids, attention_mask, past_key_values, inputs_embeds, labels
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/language_model/hawk_qwen.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/language_model/hawk_qwen.py
new file mode 100644
index 0000000000000000000000000000000000000000..122daa8655c7e387735e74e76463e8bbe6b687fa
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/language_model/hawk_qwen.py
@@ -0,0 +1,166 @@
+from typing import List, Optional, Tuple, Union
+
+import torch
+import torch.nn as nn
+from transformers import (AutoConfig, AutoModelForCausalLM, Qwen2Config, Qwen2ForCausalLM,
+                          Qwen2Model)
+from transformers.generation.utils import GenerateOutput
+from transformers.modeling_outputs import CausalLMOutputWithPast
+from transformers.utils import logging
+
+from ..hawk_arch import HawkMetaForCausalLM, HawkMetaModel
+from ..vision_encoder.qwen_vit import QwenVisionConfig
+
+logger = logging.get_logger(__name__)
+
+
+class HawkQwenConfig(Qwen2Config):
+    model_type = "hawk_vl"
+    is_composition = True
+    sub_configs = {"vision_config": QwenVisionConfig}
+
+    def __init__(self, vision_config=None, **kwargs):
+        super().__init__(**kwargs)
+
+        if isinstance(vision_config, dict):
+            self.vision_config = self.sub_configs["vision_config"](**vision_config)
+        elif vision_config is None:
+            self.vision_config = self.sub_configs["vision_config"]()
+        else:
+            self.vision_config = vision_config
+
+    def to_dict(self):
+        config_dict = super().to_dict()
+        config_dict['vision_config'] = self.vision_config.to_dict()
+        config_dict['model_type'] = self.__class__.model_type
+        return config_dict
+
+
+class HawkQwenModel(HawkMetaModel, Qwen2Model):
+    config_class = HawkQwenConfig
+
+    def __init__(self, config: Qwen2Config):
+        super(HawkQwenModel, self).__init__(config)
+
+
+class HawkQwenForCausalLM(Qwen2ForCausalLM, HawkMetaForCausalLM):
+    _tied_weights_keys = ["lm_head.weight"]
+    config_class = HawkQwenConfig
+    _no_split_modules = ['Qwen2DecoderLayer']
+    _supports_flash_attn_2 = True
+    supports_gradient_checkpointing = True
+
+    def __init__(self, config):
+        super().__init__(config)
+
+        self.model = HawkQwenModel(config)
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+        self.image_token_id = getattr(config, 'image_token_id', None)
+        self.video_token_id = getattr(config, 'video_token_id', None)
+
+        self.post_init()
+
+    def get_model(self):
+        return self.model
+
+    def forward(
+        self,
+        input_ids: torch.LongTensor = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[List[torch.FloatTensor]] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        labels: Optional[torch.LongTensor] = None,
+        use_cache: Optional[bool] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+        logits_to_keep: Union[int, torch.Tensor] = 0,
+        pixel_values: Optional[torch.Tensor] = None,
+        pixel_values_videos: Optional[torch.FloatTensor] = None,
+        image_grid_thw: Optional[torch.LongTensor] = None,
+        video_grid_thw: Optional[torch.LongTensor] = None,
+        **kwargs
+    ) -> Union[Tuple, CausalLMOutputWithPast]:
+        if inputs_embeds is None:
+            (input_ids,
+             position_ids,
+             attention_mask,
+             past_key_values,
+             inputs_embeds,
+             labels) = self.prepare_inputs_labels_for_multimodal(input_ids, attention_mask, position_ids,
+                                                                 past_key_values, labels,
+                                                                 pixel_values, pixel_values_videos,
+                                                                 image_grid_thw, video_grid_thw)
+
+        return super().forward(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            labels=labels,
+            use_cache=use_cache,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+
+    @torch.no_grad()
+    def generate(
+        self,
+        input_ids: Optional[torch.Tensor] = None,
+        pixel_values: Optional[torch.Tensor] = None,
+        pixel_values_videos: Optional[torch.FloatTensor] = None,
+        image_grid_thw: Optional[torch.LongTensor] = None,
+        video_grid_thw: Optional[torch.LongTensor] = None,
+        **kwargs,
+    ) -> Union[GenerateOutput, torch.LongTensor]:
+
+        position_ids = kwargs.pop("position_ids", None)
+        attention_mask = kwargs.pop("attention_mask", None)
+
+        if "inputs_embeds" in kwargs:
+            raise NotImplementedError("`inputs_embeds` is not supported")
+
+        if pixel_values is not None or pixel_values_videos is not None:
+            (input_ids, position_ids,
+             attention_mask, _, inputs_embeds, _) = self.prepare_inputs_labels_for_multimodal(
+                input_ids,
+                attention_mask, position_ids,
+                None, None,
+                pixel_values, pixel_values_videos,
+                image_grid_thw, video_grid_thw)
+        else:
+            inputs_embeds = self.get_model().embed_tokens(input_ids)
+
+        return super().generate(position_ids=position_ids,
+                                attention_mask=attention_mask,
+                                inputs_embeds=inputs_embeds,
+                                **kwargs)
+
+    def prepare_inputs_for_generation(self, input_ids, past_key_values=None, inputs_embeds=None, **kwargs):
+
+        pixel_values = kwargs.pop("pixel_values", None)
+        pixel_values_videos = kwargs.pop("pixel_values_videos", None)
+        image_grid_thw = kwargs.pop("image_grid_thw", None)
+        video_grid_thw = kwargs.pop("video_grid_thw", None)
+
+        inputs = super().prepare_inputs_for_generation(input_ids,
+                                                       past_key_values=past_key_values,
+                                                       inputs_embeds=inputs_embeds,
+                                                       **kwargs)
+        if pixel_values is not None:
+            inputs["pixel_values"] = pixel_values
+        if pixel_values_videos is not None:
+            inputs["pixel_values_videos"] = pixel_values_videos
+        if image_grid_thw is not None:
+            inputs["image_grid_thw"] = image_grid_thw
+        if video_grid_thw is not None:
+            inputs["video_grid_thw"] = video_grid_thw
+        return inputs
+
+
+AutoConfig.register("hawk_vl", HawkQwenConfig)
+AutoModelForCausalLM.register(HawkQwenConfig, HawkQwenForCausalLM)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/multimodal_projector/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/multimodal_projector/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..a754dbd4fc4810e4712da6ceef9dbe8e4e55bcf2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/multimodal_projector/builder.py
@@ -0,0 +1,86 @@
+import re
+
+import torch
+import torch.nn as nn
+
+
+class IdentityMap(nn.Module):
+    def __init__(self):
+        super().__init__()
+
+    def forward(self, x, *args, **kwargs):
+        return x
+
+    @property
+    def config(self):
+        return {"mm_projector_type": 'identity'}
+
+
+class SimpleResBlock(nn.Module):
+    def __init__(self, channels):
+        super().__init__()
+        self.pre_norm = nn.LayerNorm(channels)
+
+        self.proj = nn.Sequential(
+            nn.Linear(channels, channels),
+            nn.GELU(),
+            nn.Linear(channels, channels))
+
+    def forward(self, x):
+        x = self.pre_norm(x)
+        return x + self.proj(x)
+
+
+# copied from modeling_qwen2_vl.py
+# similar to the pixel unshuffle operation in InternVL
+class PatchMerger(nn.Module):
+    def __init__(self, dim: int, context_dim: int, spatial_merge_size: int = 2) -> None:
+        super().__init__()
+        self.hidden_size = context_dim * (spatial_merge_size**2)
+        self.ln_q = nn.LayerNorm(context_dim, eps=1e-6)
+        self.mlp = nn.Sequential(
+            nn.Linear(self.hidden_size, self.hidden_size),
+            nn.GELU(),
+            nn.Linear(self.hidden_size, dim),
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.mlp(self.ln_q(x).view(-1, self.hidden_size))
+        return x
+
+
+class PoolingMerger(nn.Module):
+    def __init__(self, pool_type, kernel_size):
+        super().__init__()
+
+        self.pooling_layer = nn.AvgPool1d(kernel_size) if pool_type == 'avgpool' else nn.MaxPool1d(kernel_size)
+
+    def forward(self, x):
+        return self.pooling_layer(x)
+
+
+def build_vision_projector(config, **kwargs):
+    projector_type = getattr(config, 'mm_projector_type', 'pixel_unshuffle')
+    if projector_type == 'linear':
+        return nn.Linear(config.mm_hidden_size, config.hidden_size)
+
+    mlp_gelu_match = re.match(r'^mlp(\d+)x_gelu$', projector_type)
+    if mlp_gelu_match:
+        mlp_depth = int(mlp_gelu_match.group(1))
+        modules = [nn.Linear(config.mm_hidden_size, config.hidden_size)]
+        for _ in range(1, mlp_depth):
+            modules.append(nn.GELU())
+            modules.append(nn.Linear(config.hidden_size, config.hidden_size))
+        return nn.Sequential(*modules)
+
+    if projector_type == 'identity':
+        return IdentityMap()
+
+    # add pooling & pixel unshuffle.
+    if projector_type == 'pixel_unshuffle':
+        return PatchMerger(config.lm_hidden_size, config.embed_dim, config.spatial_merge_size)
+
+    if projector_type in ['avgpool', 'maxpool']:
+        return PoolingMerger(projector_type, config.spatial_merge_size)
+
+    raise ValueError(f'Unknown projector type: {projector_type}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..5898359b7780864c3fade82b933c1a5ba7140b22
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/__init__.py
@@ -0,0 +1,5 @@
+from .qwen_vit import QwenVisionModel
+
+VISION_TRANSFORMER_CLASSES = {
+    'qwen_vit': QwenVisionModel
+}
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..3cf129812532df31ecac9e1f4e7aeeda26878f90
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/__init__.py
@@ -0,0 +1,2 @@
+from .configuration_qwen_vit import QwenVisionConfig  # noqa: F401
+from .modeling_qwen_vit import QwenVisionModel  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/configuration_qwen_vit.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/configuration_qwen_vit.py
new file mode 100644
index 0000000000000000000000000000000000000000..c2f37ce08d710cfdc0d96afe7d3014603ad9f7d2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/configuration_qwen_vit.py
@@ -0,0 +1,56 @@
+# --------------------------------------------------------
+# pandayin: Copied and modified from transformers/models/qwen2_vl/configuration_qwen2_vl.py
+# --------------------------------------------------------
+
+import os
+from typing import Union
+
+from transformers.configuration_utils import PretrainedConfig
+from transformers.utils import logging
+
+logger = logging.get_logger(__name__)
+
+
+class QwenVisionConfig(PretrainedConfig):
+    model_type = "qwen_vit"
+    # base_config_key = "vision_config"
+
+    def __init__(
+        self,
+        depth=32,
+        embed_dim=1280,
+        hidden_act="quick_gelu",
+        mlp_ratio=4,
+        num_heads=16,
+        in_channels=3,
+        patch_size=14,
+        spatial_merge_size=2,
+        temporal_patch_size=2,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+
+        self.depth = depth
+        self.embed_dim = embed_dim
+        self.hidden_act = hidden_act
+        self.mlp_ratio = mlp_ratio
+        self.num_heads = num_heads
+        self.in_channels = in_channels
+        self.patch_size = patch_size
+        self.spatial_merge_size = spatial_merge_size
+        self.temporal_patch_size = temporal_patch_size
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path: Union[str, os.PathLike], **kwargs) -> 'PretrainedConfig':
+        config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
+
+        if 'vision_config' in config_dict:
+            config_dict = config_dict['vision_config']
+
+        if 'model_type' in config_dict and hasattr(cls, 'model_type') and config_dict['model_type'] != cls.model_type:
+            logger.warning(
+                f"You are using a model of type {config_dict['model_type']} to instantiate a model of type "
+                f'{cls.model_type}. This is not supported for all configurations of models and can yield errors.'
+            )
+
+        return cls.from_dict(config_dict, **kwargs)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/modeling_qwen_vit.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/modeling_qwen_vit.py
new file mode 100644
index 0000000000000000000000000000000000000000..3b6eab7d7d8a46d05d2f8cbfbde164129a0cdb16
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/model/vision_encoder/qwen_vit/modeling_qwen_vit.py
@@ -0,0 +1,380 @@
+# --------------------------------------------------------
+# pandayin: Copied and modified from transformers/models/qwen2_vl/modeling_qwen2_vl.py
+# Below is the original copyright statement.
+# --------------------------------------------------------
+# Copyright 2024 The Qwen team, Alibaba Group and the HuggingFace Inc. team. All rights reserved.
+#
+# This code is based on EleutherAI's GPT-NeoX library and the GPT-NeoX
+# and OPT implementations in this library. It has been modified from its
+# original forms to accommodate minor architectural differences compared
+# to GPT-NeoX and OPT used by the Meta AI team that trained the model.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+from typing import Optional, Tuple
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers.activations import ACT2FN
+# from transformers.modeling_flash_attention_utils import is_flash_attn_available
+from transformers.modeling_utils import PreTrainedModel
+from transformers.utils import logging
+
+from .configuration_qwen_vit import QwenVisionConfig
+
+try:
+    from transformers.modeling_flash_attention_utils import flash_attn_varlen_func
+    has_flash_attn = True
+except Exception:
+    print('FlashAttention2 is not installed.')
+    has_flash_attn = False
+
+# if has_flash_attn:
+#     from transformers.modeling_flash_attention_utils import flash_attn_varlen_func
+
+
+logger = logging.get_logger(__name__)
+
+
+NORM2FN = {
+    'layer_norm': nn.LayerNorm,
+}
+
+
+# Copied from transformers.models.llama.modeling_llama.rotate_half
+def rotate_half(x):
+    """Rotates half the hidden dims of the input."""
+    x1 = x[..., : x.shape[-1] // 2]
+    x2 = x[..., x.shape[-1] // 2:]
+    return torch.cat((-x2, x1), dim=-1)
+
+
+def apply_rotary_pos_emb_vision(
+    q: torch.Tensor, k: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor
+) -> Tuple[torch.Tensor, torch.Tensor]:
+    orig_q_dtype = q.dtype
+    orig_k_dtype = k.dtype
+    q, k = q.float(), k.float()
+    cos, sin = cos.unsqueeze(-2).float(), sin.unsqueeze(-2).float()
+    q_embed = (q * cos) + (rotate_half(q) * sin)
+    k_embed = (k * cos) + (rotate_half(k) * sin)
+    q_embed = q_embed.to(orig_q_dtype)
+    k_embed = k_embed.to(orig_k_dtype)
+    return q_embed, k_embed
+
+
+class VisionRotaryEmbedding(nn.Module):
+    def __init__(self, dim: int, theta: float = 10000.0) -> None:
+        super().__init__()
+        inv_freq = 1.0 / (theta ** (torch.arange(0, dim, 2, dtype=torch.float) / dim))
+        self.register_buffer("inv_freq", inv_freq, persistent=False)
+
+    def forward(self, seqlen: int) -> torch.Tensor:
+        seq = torch.arange(seqlen, device=self.inv_freq.device, dtype=self.inv_freq.dtype)
+        freqs = torch.outer(seq, self.inv_freq)
+        return freqs
+
+
+class PatchEmbed(nn.Module):
+    def __init__(
+        self,
+        patch_size: int = 14,
+        temporal_patch_size: int = 2,
+        in_channels: int = 3,
+        embed_dim: int = 1152,
+    ) -> None:
+        super().__init__()
+        self.patch_size = patch_size
+        self.temporal_patch_size = temporal_patch_size
+        self.in_channels = in_channels
+        self.embed_dim = embed_dim
+
+        kernel_size = [temporal_patch_size, patch_size, patch_size]
+        self.proj = nn.Conv3d(in_channels, embed_dim, kernel_size=kernel_size, stride=kernel_size, bias=False)
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        target_dtype = self.proj.weight.dtype
+        hidden_states = hidden_states.view(
+            -1, self.in_channels, self.temporal_patch_size, self.patch_size, self.patch_size
+        )
+        hidden_states = self.proj(hidden_states.to(dtype=target_dtype)).view(-1, self.embed_dim)
+        return hidden_states
+
+
+class VisionMlp(nn.Module):
+    def __init__(self, dim: int, hidden_dim: int, hidden_act: str) -> None:
+        super().__init__()
+        self.fc1 = nn.Linear(dim, hidden_dim)
+        self.act = ACT2FN[hidden_act]
+        self.fc2 = nn.Linear(hidden_dim, dim)
+
+    def forward(self, x) -> torch.Tensor:
+        return self.fc2(self.act(self.fc1(x)))
+
+
+class VisionAttention(nn.Module):
+    def __init__(self, dim: int, num_heads: int = 16) -> None:
+        super().__init__()
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.qkv = nn.Linear(dim, dim * 3, bias=True)
+        self.proj = nn.Linear(dim, dim)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        cu_seqlens: torch.Tensor,
+        rotary_pos_emb: Optional[torch.Tensor] = None,
+        position_embeddings: Optional[Tuple[torch.Tensor, torch.Tensor]] = None,
+    ) -> torch.Tensor:
+        seq_length = hidden_states.shape[0]
+        q, k, v = self.qkv(hidden_states).reshape(seq_length, 3, self.num_heads, -1).permute(1, 0, 2, 3).unbind(0)
+        if position_embeddings is None:
+            logger.warning_once(
+                "The attention layers in this model are transitioning from computing the RoPE embeddings internally "
+                "through `rotary_pos_emb` (2D tensor of RoPE theta values), to using externally computed "
+                "`position_embeddings` (Tuple of tensors, containing cos and sin). In v4.54 `rotary_pos_emb` will be "
+                "removed and `position_embeddings` will be mandatory."
+            )
+            emb = torch.cat((rotary_pos_emb, rotary_pos_emb), dim=-1)
+            cos = emb.cos()
+            sin = emb.sin()
+        else:
+            cos, sin = position_embeddings
+        q, k = apply_rotary_pos_emb_vision(q, k, cos, sin)
+
+        attention_mask = torch.full(
+            [1, seq_length, seq_length], torch.finfo(q.dtype).min, device=q.device, dtype=q.dtype
+        )
+        for i in range(1, len(cu_seqlens)):
+            attention_mask[..., cu_seqlens[i - 1]: cu_seqlens[i], cu_seqlens[i - 1]: cu_seqlens[i]] = 0
+
+        q = q.transpose(0, 1)
+        k = k.transpose(0, 1)
+        v = v.transpose(0, 1)
+        attn_weights = torch.matmul(q, k.transpose(1, 2)) / math.sqrt(self.head_dim)
+        attn_weights = attn_weights + attention_mask
+        attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(q.dtype)
+        attn_output = torch.matmul(attn_weights, v)
+        attn_output = attn_output.transpose(0, 1)
+        attn_output = attn_output.reshape(seq_length, -1)
+        attn_output = self.proj(attn_output)
+        return attn_output
+
+
+class VisionFlashAttention2(nn.Module):
+    def __init__(self, dim: int, num_heads: int = 16) -> None:
+        super().__init__()
+        self.num_heads = num_heads
+        self.qkv = nn.Linear(dim, dim * 3, bias=True)
+        self.proj = nn.Linear(dim, dim)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        cu_seqlens: torch.Tensor,
+        rotary_pos_emb: Optional[torch.Tensor] = None,
+        position_embeddings: Optional[Tuple[torch.Tensor, torch.Tensor]] = None,
+    ) -> torch.Tensor:
+        seq_length = hidden_states.shape[0]
+        q, k, v = self.qkv(hidden_states).reshape(seq_length, 3, self.num_heads, -1).permute(1, 0, 2, 3).unbind(0)
+        if position_embeddings is None:
+            logger.warning_once(
+                "The attention layers in this model are transitioning from computing the RoPE embeddings internally "
+                "through `rotary_pos_emb` (2D tensor of RoPE theta values), to using externally computed "
+                "`position_embeddings` (Tuple of tensors, containing cos and sin). In v4.54 `rotary_pos_emb` will be "
+                "removed and `position_embeddings` will be mandatory."
+            )
+            emb = torch.cat((rotary_pos_emb, rotary_pos_emb), dim=-1)
+            cos = emb.cos()
+            sin = emb.sin()
+        else:
+            cos, sin = position_embeddings
+        q, k = apply_rotary_pos_emb_vision(q, k, cos, sin)
+
+        max_seqlen = (cu_seqlens[1:] - cu_seqlens[:-1]).max().item()
+        attn_output = flash_attn_varlen_func(q, k, v, cu_seqlens, cu_seqlens, max_seqlen, max_seqlen).reshape(
+            seq_length, -1
+        )
+        attn_output = self.proj(attn_output)
+        return attn_output
+
+
+class VisionSdpaAttention(nn.Module):
+    def __init__(self, dim: int, num_heads: int = 16) -> None:
+        super().__init__()
+        self.num_heads = num_heads
+        self.qkv = nn.Linear(dim, dim * 3, bias=True)
+        self.proj = nn.Linear(dim, dim)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        cu_seqlens: torch.Tensor,
+        rotary_pos_emb: Optional[torch.Tensor] = None,
+        position_embeddings: Optional[Tuple[torch.Tensor, torch.Tensor]] = None,
+    ) -> torch.Tensor:
+        seq_length = hidden_states.shape[0]
+        q, k, v = self.qkv(hidden_states).reshape(seq_length, 3, self.num_heads, -1).permute(1, 0, 2, 3).unbind(0)
+        if position_embeddings is None:
+            logger.warning_once(
+                "The attention layers in this model are transitioning from computing the RoPE embeddings internally "
+                "through `rotary_pos_emb` (2D tensor of RoPE theta values), to using externally computed "
+                "`position_embeddings` (Tuple of tensors, containing cos and sin). In v4.54 `rotary_pos_emb` will be "
+                "removed and `position_embeddings` will be mandatory."
+            )
+            emb = torch.cat((rotary_pos_emb, rotary_pos_emb), dim=-1)
+            cos = emb.cos()
+            sin = emb.sin()
+        else:
+            cos, sin = position_embeddings
+        q, k = apply_rotary_pos_emb_vision(q, k, cos, sin)
+
+        attention_mask = torch.zeros([1, seq_length, seq_length], device=q.device, dtype=torch.bool)
+        for i in range(1, len(cu_seqlens)):
+            attention_mask[..., cu_seqlens[i - 1]: cu_seqlens[i], cu_seqlens[i - 1]: cu_seqlens[i]] = True
+        q = q.transpose(0, 1)
+        k = k.transpose(0, 1)
+        v = v.transpose(0, 1)
+        attn_output = F.scaled_dot_product_attention(
+            q.unsqueeze(0), k.unsqueeze(0), v.unsqueeze(0), attention_mask, dropout_p=0.0
+        )
+        attn_output = attn_output.squeeze(0).transpose(0, 1)
+        attn_output = attn_output.reshape(seq_length, -1)
+        attn_output = self.proj(attn_output)
+        return attn_output
+
+
+QWEN2_VL_VISION_ATTENTION_CLASSES = {
+    "eager": VisionAttention,
+    "flash_attention_2": VisionFlashAttention2,
+    "sdpa": VisionSdpaAttention,
+}
+
+
+class QwenVisionBlock(nn.Module):
+    def __init__(self, config, attn_implementation: str = "flash_attention_2") -> None:
+        super().__init__()
+        self.norm1 = NORM2FN['layer_norm'](config.embed_dim, eps=1e-6)
+        self.norm2 = NORM2FN['layer_norm'](config.embed_dim, eps=1e-6)
+        mlp_hidden_dim = int(config.embed_dim * config.mlp_ratio)
+
+        self.attn = QWEN2_VL_VISION_ATTENTION_CLASSES[attn_implementation](
+            config.embed_dim, num_heads=config.num_heads
+        )
+        self.mlp = VisionMlp(dim=config.embed_dim, hidden_dim=mlp_hidden_dim, hidden_act=config.hidden_act)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        cu_seqlens: torch.Tensor,
+        rotary_pos_emb: Optional[torch.Tensor] = None,
+        position_embeddings: Optional[Tuple[torch.Tensor, torch.Tensor]] = None,
+    ) -> torch.Tensor:
+        hidden_states = hidden_states + self.attn(
+            self.norm1(hidden_states),
+            cu_seqlens=cu_seqlens,
+            rotary_pos_emb=rotary_pos_emb,
+            position_embeddings=position_embeddings,
+        )
+        hidden_states = hidden_states + self.mlp(self.norm2(hidden_states))
+        return hidden_states
+
+
+class QwenVisionModel(PreTrainedModel):
+    main_input_name = 'pixel_values'
+    config_class = QwenVisionConfig
+    _no_split_modules = ["QwenVisionBlock"]
+    _supports_flash_attn_2 = True
+
+    def __init__(self, config) -> None:
+        super().__init__(config)
+        self.spatial_merge_size = config.spatial_merge_size
+
+        self.patch_embed = PatchEmbed(
+            patch_size=config.patch_size,
+            temporal_patch_size=config.temporal_patch_size,
+            in_channels=config.in_channels,
+            embed_dim=config.embed_dim,
+        )
+
+        head_dim = config.embed_dim // config.num_heads
+        self.rotary_pos_emb = VisionRotaryEmbedding(head_dim // 2)
+
+        self.blocks = nn.ModuleList(
+            [QwenVisionBlock(config, config._attn_implementation) for _ in range(config.depth)]
+        )
+
+        self.gradient_checkpointing = True
+
+    def get_dtype(self) -> torch.dtype:
+        return self.blocks[0].mlp.fc2.weight.dtype
+
+    def get_device(self) -> torch.device:
+        return self.blocks[0].mlp.fc2.weight.device
+
+    def rot_pos_emb(self, grid_thw):
+        pos_ids = []
+        for t, h, w in grid_thw:
+            hpos_ids = torch.arange(h).unsqueeze(1).expand(-1, w)
+            hpos_ids = hpos_ids.reshape(
+                h // self.spatial_merge_size,
+                self.spatial_merge_size,
+                w // self.spatial_merge_size,
+                self.spatial_merge_size,
+            )
+            hpos_ids = hpos_ids.permute(0, 2, 1, 3)
+            hpos_ids = hpos_ids.flatten()
+
+            wpos_ids = torch.arange(w).unsqueeze(0).expand(h, -1)
+            wpos_ids = wpos_ids.reshape(
+                h // self.spatial_merge_size,
+                self.spatial_merge_size,
+                w // self.spatial_merge_size,
+                self.spatial_merge_size,
+            )
+            wpos_ids = wpos_ids.permute(0, 2, 1, 3)
+            wpos_ids = wpos_ids.flatten()
+            pos_ids.append(torch.stack([hpos_ids, wpos_ids], dim=-1).repeat(t, 1))
+        pos_ids = torch.cat(pos_ids, dim=0)
+        max_grid_size = grid_thw[:, 1:].max()
+        rotary_pos_emb_full = self.rotary_pos_emb(max_grid_size)
+        rotary_pos_emb = rotary_pos_emb_full[pos_ids].flatten(1)
+        return rotary_pos_emb
+
+    def forward(self, pixel_values: torch.FloatTensor, grid_thw: torch.LongTensor) -> torch.FloatTensor:
+        hidden_states = self.patch_embed(pixel_values)
+        rotary_pos_emb = self.rot_pos_emb(grid_thw)
+        emb = torch.cat((rotary_pos_emb, rotary_pos_emb), dim=-1)
+        position_embeddings = (emb.cos(), emb.sin())
+
+        cu_seqlens = torch.repeat_interleave(grid_thw[:, 1] * grid_thw[:, 2], grid_thw[:, 0]).cumsum(
+            dim=0,
+            # Select dtype based on the following factors:
+            #  - FA2 requires that cu_seqlens_q must have dtype int32
+            #  - torch.onnx.export requires that cu_seqlens_q must have same dtype as grid_thw
+            # See https://github.com/huggingface/transformers/pull/34852 for more information
+            dtype=grid_thw.dtype if torch.jit.is_tracing() else torch.int32,
+        )
+        cu_seqlens = F.pad(cu_seqlens, (1, 0), value=0)
+
+        for blk in self.blocks:
+            if self.gradient_checkpointing and self.training:
+                hidden_states = self._gradient_checkpointing_func(
+                    blk.__call__, hidden_states, cu_seqlens, None, position_embeddings
+                )
+            else:
+                hidden_states = blk(hidden_states, cu_seqlens=cu_seqlens, position_embeddings=position_embeddings)
+        return hidden_states
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..2073759b5351e7f3e2f5e407348a859db39c69e9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/hawk/utils.py
@@ -0,0 +1,16 @@
+import torch.distributed as dist
+
+
+def rank0_print(*args):
+    if dist.is_initialized():
+        if dist.get_rank() == 0:
+            print(f"Rank {dist.get_rank()}: ", *args)
+    else:
+        print(*args)
+
+
+def rank_print(*args):
+    if dist.is_initialized():
+        print(f"Rank {dist.get_rank()}: ", *args)
+    else:
+        print(*args)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/model.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..f81171858f72b74030eea57e1ce0748bb940b817
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/model.py
@@ -0,0 +1,152 @@
+import logging
+import os
+import warnings
+
+import torch
+
+from ..base import BaseModel
+from .prompt import HawkVLPromptMixin
+
+
+def ensure_image_url(image: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:image;']
+    if any(image.startswith(prefix) for prefix in prefixes):
+        return image
+    if os.path.exists(image):
+        return 'file://' + image
+    raise ValueError(f'Invalid image: {image}')
+
+
+def ensure_video_url(video: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:video;']
+    if any(video.startswith(prefix) for prefix in prefixes):
+        return video
+    if os.path.exists(video):
+        return 'file://' + video
+    raise ValueError(f'Invalid video: {video}')
+
+
+class HawkVL(HawkVLPromptMixin, BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = False
+
+    def __init__(
+        self,
+        model_path: str,
+        min_pixels: int = None,
+        max_pixels: int = None,
+        max_new_tokens=1024,
+        top_p=0.8,
+        top_k=20,
+        temperature=0.7,
+        repetition_penalty=1.0,
+        use_custom_prompt: bool = False,
+        system_prompt: str = None,
+        verbose: bool = False,
+    ):
+        try:
+            from .hawk.model import HawkQwenForCausalLM
+        except Exception as e:
+            logging.critical(
+                "Please move the code ('hawk' directory) to $VLMEVALKIT/vlm/hawk_vl."
+            )
+            raise e
+        super().__init__(use_custom_prompt=use_custom_prompt)
+        self.min_pixels = min_pixels
+        self.max_pixels = max_pixels
+        self.generate_kwargs = dict(
+            max_new_tokens=max_new_tokens,
+            top_p=top_p,
+            top_k=top_k,
+            temperature=temperature,
+            repetition_penalty=repetition_penalty,
+        )
+        self.system_prompt = system_prompt
+        self.verbose = verbose
+        self.fps = 1.0
+        self.nframe = 64
+        self.FRAME_FACTOR = 1
+
+        self.model_path = model_path
+        from transformers import AutoProcessor
+        self.processor = AutoProcessor.from_pretrained(model_path)
+
+        self.model = HawkQwenForCausalLM.from_pretrained(
+            model_path, torch_dtype=torch.bfloat16, attn_implementation='flash_attention_2', device_map=None)
+
+        self.model.cuda().eval()
+
+    def _prepare_content(self, inputs: list[dict[str, str]], dataset: str = None) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        content = []
+        for s in inputs:
+            if s['type'] == 'image':
+                item = {'type': 'image', 'image': ensure_image_url(s['value'])}
+                if dataset == 'OCRBench':
+                    item['min_pixels'] = 10 * 10 * 28 * 28
+                    warnings.warn(f"OCRBench dataset uses custom min_pixels={item['min_pixels']}")
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item['min_pixels'] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+            elif s['type'] == 'video':
+                item = {'type': 'video', 'video': ensure_video_url(s['value'])}
+                if self.fps is not None:
+                    item['fps'] = self.fps
+                elif self.nframe is not None:
+                    import cv2
+                    video = cv2.VideoCapture(s['value'])
+                    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                    video.release()
+                    if frame_count < self.nframe:
+                        new_frame_count = frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR
+                        print(f"use {new_frame_count} for {s['value']}")
+                        item['nframes'] = new_frame_count
+                    else:
+                        item['nframes'] = self.nframe
+            elif s['type'] == 'text':
+                item = {'type': 'text', 'text': s['value']}
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def generate_inner(self, message, dataset=None):
+        try:
+            from qwen_vl_utils import process_vision_info
+        except Exception as err:
+            logging.critical("qwen_vl_utils not found, please install it via 'pip install qwen-vl-utils'")
+            raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({'role': 'system', 'content': self.system_prompt})
+        messages.append({'role': 'user', 'content': self._prepare_content(message, dataset=dataset)})
+        if self.verbose:
+            print(f'\033[31m{messages}\033[0m')
+
+        text = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+        images, videos = process_vision_info(messages)
+        inputs = self.processor(text=[text], images=images, videos=videos, padding=True, return_tensors='pt')
+
+        inputs = inputs.to('cuda')
+
+        generated_ids = self.model.generate(
+            **inputs,
+            **self.generate_kwargs,
+        )
+
+        out = self.processor.batch_decode(
+            generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
+        )
+        response = out[0].strip()
+
+        if self.verbose:
+            print(f'\033[32m{response}\033[0m')
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..060e37a57341f585925a4fc117ee14e7e9a4cc57
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/hawk_vl/prompt.py
@@ -0,0 +1,158 @@
+from __future__ import annotations
+
+
+# Adapted from vlmeval/vlm/qwen2_vl/prompt.py
+class HawkVLPromptMixin:
+    """
+    Mixin class for HawkVL to build custom prompt for different datasets.
+
+    Requires the following methods to be implemented in the subclass:
+        - dump_image(line, dataset: str) -> str | list[str]
+
+    Implements the following methods:
+        - use_custom_prompt(dataset: str) -> bool
+        - build_prompt(line, dataset: str) -> list[dict[str, str]]
+    """
+
+    def __init__(self, *args, use_custom_prompt: bool = True, **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        self._use_custom_prompt = use_custom_prompt
+
+    def set_dump_image(self, dump_image_func):
+        self.dump_image_func = dump_image_func
+
+    def dump_image(self, line, dataset):
+        return self.dump_image_func(line)
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        from vlmeval.dataset import DATASET_TYPE
+        dataset_type = DATASET_TYPE(dataset, default=None)
+
+        if not self._use_custom_prompt:
+            return False
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return True
+        if dataset_type == 'MCQ':
+            if dataset == 'LEGO':
+                return False
+            return True
+        if dataset_type == 'Y/N' and dataset in {'HallusionBench', 'POPE'}:  # MME has it's own prompt
+            return True
+        if dataset_type == 'VQA' and dataset not in {'MMVet', 'ChartQAPro', 'ChartQAPro_CoT', 'ChartQAPro_PoT', 'ChartMuseum'}:  # noqa: E501
+            return True
+        return False
+
+    def build_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        from vlmeval.dataset import DATASET_TYPE
+
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return self._build_mmmu_prompt(line, dataset)
+        dataset_type = DATASET_TYPE(dataset, default=None)
+        if dataset_type == 'MCQ':
+            return self._build_mcq_prompt(line, dataset)
+        if dataset_type == 'Y/N':
+            return self._build_yorn_prompt(line, dataset)
+        if dataset_type == 'VQA':
+            return self._build_vqa_prompt(line, dataset)
+        raise ValueError(f'Unsupported dataset: {dataset}')
+
+    def _build_mmmu_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MMMU dataset: keep all images at beginning."""
+
+        import string
+
+        import pandas as pd
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += "Answer with the option's letter from the given choices directly. \n"
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_mcq_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MCQ dataset: use chinese prompt if the question contains chinese characters."""
+        MCQ_CN_PROMPT = '请直接回答选项字母。'
+        MCQ_EN_PROMPT = "Answer with the option's letter from the given choices directly."
+
+        import string
+
+        import pandas as pd
+
+        def cn_string(s):
+            import re
+
+            if re.search('[\u4e00-\u9fff]', s):
+                return True
+            return False
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += MCQ_CN_PROMPT if cn_string(prompt) else MCQ_EN_PROMPT
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_yorn_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for YORN dataset:"""
+        YORN_PROMPT = ' Please answer yes or no.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += YORN_PROMPT
+        return msgs
+
+    def _build_vqa_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for VQA dataset:"""
+        VQA_PROMPT = "\nAnswer the question using a single word or phrase."
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += VQA_PROMPT
+        return msgs
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/interns1/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/interns1/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..acc50c968c01e49cecb2a0854f7a764e4af33f90
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/interns1/__init__.py
@@ -0,0 +1,3 @@
+from .interns1_chat import InternS1Chat
+
+__all__ = ['InternS1Chat']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/interns1/interns1_chat.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/interns1/interns1_chat.py
new file mode 100644
index 0000000000000000000000000000000000000000..56f3b8941b5a965109ae816e93413d1b3ce47fb1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/interns1/interns1_chat.py
@@ -0,0 +1,294 @@
+from __future__ import annotations
+import os
+from typing import Any, Dict, List, Optional, Tuple, Union
+
+import torch
+from PIL import Image
+from transformers import AutoModelForCausalLM, AutoProcessor
+
+
+def _is_url(s: str) -> bool:
+    return isinstance(s, str) and (s.startswith("http://") or s.startswith("https://"))
+
+
+def _best_image_value(v: Any) -> Optional[str]:
+    if isinstance(v, str):
+        return v
+    if isinstance(v, dict):
+        for k in ("path", "url", "value"):
+            if isinstance(v.get(k), str):
+                return v[k]
+    return None
+
+
+def _is_local_image_path(p: str) -> bool:
+    ext = os.path.splitext(p)[-1].lower()
+    return ext in [".jpg", ".jpeg", ".png", ".webp", ".bmp"] and os.path.exists(p)
+
+
+class InternS1Chat:
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    SUPPORT_CHAT = True
+    IS_API = False
+
+    def __init__(
+        self,
+        model_path: str = "internlm/Intern-S1-mini",
+        device_map: str = "auto",
+        torch_dtype: Union[str, torch.dtype] = "auto",
+        temperature: float = 0.8,
+        top_p: float = 1.0,
+        top_k: int = 50,
+        min_p: float = 0.0,
+        max_new_tokens: int = 2048,
+        do_sample: bool = True,
+        processor_dtype: Optional[torch.dtype] = torch.bfloat16,
+        **kwargs,
+    ):
+        self.model_path = model_path
+        self.device_map = device_map
+        self.torch_dtype = torch_dtype
+
+        self.temperature = temperature
+        self.top_p = top_p
+        self.top_k = top_k
+        self.min_p = min_p
+        self.max_new_tokens = max_new_tokens
+        self.do_sample = do_sample
+
+        if processor_dtype is None:
+            processor_dtype = torch.bfloat16
+        if not torch.cuda.is_available():
+            processor_dtype = torch.float32
+        self.processor_dtype = processor_dtype
+
+        self._dump_image = False
+        self._batch_size = 1
+        self._output_dir: Optional[str] = None
+
+        self.processor = AutoProcessor.from_pretrained(self.model_path, trust_remote_code=True)
+        self.model = AutoModelForCausalLM.from_pretrained(
+            self.model_path,
+            device_map=self.device_map,
+            torch_dtype=self.torch_dtype,
+            trust_remote_code=True,
+        )
+        self.model.eval()
+
+    def generate(
+        self,
+        msgs: Optional[Union[List[Dict[str, Any]], List[str]]] = None,
+        message: Optional[Union[List[Dict[str, Any]], List[str]]] = None,
+        dataset: Optional[str] = None,
+        **kwargs,
+    ) -> str:
+        if message is not None:
+            msgs = message
+        if msgs is None:
+            raise ValueError("generate() expects `message` or `msgs`.")
+        return self.generate_inner(msgs, dataset=dataset)
+
+    def chat(
+        self,
+        message: Optional[List[Dict[str, Any]]] = None,
+        msgs: Optional[List[Dict[str, Any]]] = None,
+        dataset: Optional[str] = None,
+        **kwargs,
+    ) -> str:
+        if message is None:
+            message = msgs
+        if message is None:
+            raise ValueError("chat() expects `message` or `msgs`.")
+        return self.chat_inner(message, dataset=dataset)
+
+    def set_dump_image(self, flag: bool) -> None:
+        self._dump_image = bool(flag)
+
+    def set_batch_size(self, bs: int) -> None:
+        self._batch_size = int(bs) if bs and bs > 0 else 1
+
+    def set_output_folder(self, out_dir: str) -> None:
+        self._output_dir = out_dir
+
+    def set_generation_config(
+        self,
+        temperature: Optional[float] = None,
+        top_p: Optional[float] = None,
+        top_k: Optional[int] = None,
+        min_p: Optional[float] = None,
+        max_new_tokens: Optional[int] = None,
+        do_sample: Optional[bool] = None,
+        **kwargs,
+    ) -> None:
+        if temperature is not None:
+            self.temperature = float(temperature)
+        if top_p is not None:
+            self.top_p = float(top_p)
+        if top_k is not None:
+            self.top_k = int(top_k)
+        if min_p is not None:
+            self.min_p = float(min_p)
+        if max_new_tokens is not None:
+            self.max_new_tokens = int(max_new_tokens)
+        if do_sample is not None:
+            self.do_sample = bool(do_sample)
+
+    def generate_inner(
+        self,
+        msgs: Union[List[Dict[str, Any]], List[str]],
+        dataset: Optional[str] = None
+    ) -> str:
+        hf_messages = self._to_hf_messages_from_mm_msgs(msgs)
+        inputs = self._build_inputs(hf_messages)
+
+        with torch.no_grad():
+            gen_ids = self.model.generate(
+                **inputs,
+                max_new_tokens=self.max_new_tokens,
+                do_sample=self.do_sample,
+                temperature=self.temperature,
+                top_p=self.top_p,
+                top_k=self.top_k,
+                min_p=self.min_p,
+                pad_token_id=self.model.config.pad_token_id or self.processor.tokenizer.eos_token_id,
+            )
+
+        input_len = inputs["input_ids"].shape[1]
+        text = self.processor.decode(gen_ids[0, input_len:], skip_special_tokens=True)
+        return text.strip()
+
+    def chat_inner(
+        self,
+        message: List[Dict[str, Any]],
+        dataset: Optional[str] = None
+    ) -> str:
+        hf_messages = self._to_hf_messages_from_chat(message)
+        inputs = self._build_inputs(hf_messages)
+
+        with torch.no_grad():
+            gen_ids = self.model.generate(
+                **inputs,
+                max_new_tokens=self.max_new_tokens,
+                do_sample=self.do_sample,
+                temperature=self.temperature,
+                top_p=self.top_p,
+                top_k=self.top_k,
+                min_p=self.min_p,
+                pad_token_id=self.model.config.pad_token_id or self.processor.tokenizer.eos_token_id,
+            )
+
+        input_len = inputs["input_ids"].shape[1]
+        text = self.processor.decode(gen_ids[0, input_len:], skip_special_tokens=True)
+        return text.strip()
+
+    def use_custom_prompt(self, dataset: Optional[str]) -> bool:
+        return False
+
+    def build_prompt(self, line: Dict[str, Any], dataset: Optional[str] = None) -> List[Dict[str, Any]]:
+        raise NotImplementedError
+
+    def _build_inputs(self, hf_messages: List[Dict[str, Any]]) -> Dict[str, torch.Tensor]:
+        inputs = self.processor.apply_chat_template(
+            hf_messages,
+            add_generation_prompt=True,
+            tokenize=True,
+            return_dict=True,
+            return_tensors="pt",
+        )
+        inputs = inputs.to(self.model.device, dtype=self.processor_dtype)
+        return inputs
+
+    def _to_hf_messages_from_mm_msgs(
+        self,
+        msgs: Union[List[Dict[str, Any]], List[str]]
+    ) -> List[Dict[str, Any]]:
+        if len(msgs) > 0 and isinstance(msgs[0], str):
+            msgs = self._strings_to_msg_dicts(msgs)
+
+        content = []
+        for m in msgs:
+            t = m.get("type")
+            v = m.get("value")
+            v_str = _best_image_value(v) if t == "image" else v
+
+            if t == "text":
+                content.append({"type": "text", "text": str(v_str)})
+            elif t == "image":
+                if isinstance(v_str, str) and _is_url(v_str):
+                    content.append({"type": "image", "url": v_str})
+                elif isinstance(v_str, str) and _is_local_image_path(v_str):
+                    img = Image.open(v_str).convert("RGB")
+                    content.append({"type": "image", "image": img})
+                else:
+                    raise ValueError(f"Invalid image value: {v}")
+            else:
+                raise ValueError(f"Unsupported content type: {t}")
+
+        return [{"role": "user", "content": content}]
+
+    def _to_hf_messages_from_chat(self, chat: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        hf_messages: List[Dict[str, Any]] = []
+        for turn in chat:
+            role = turn.get("role")
+            mm = turn.get("content")
+            if len(mm) > 0 and isinstance(mm[0], str):
+                mm = self._strings_to_msg_dicts(mm)
+
+            content = []
+            for m in mm:
+                t = m.get("type")
+                v = m.get("value")
+                v_str = _best_image_value(v) if t == "image" else v
+
+                if t == "text":
+                    content.append({"type": "text", "text": str(v_str)})
+                elif t == "image":
+                    if isinstance(v_str, str) and _is_url(v_str):
+                        content.append({"type": "image", "url": v_str})
+                    elif isinstance(v_str, str) and _is_local_image_path(v_str):
+                        img = Image.open(v_str).convert("RGB")
+                        content.append({"type": "image", "image": img})
+                    else:
+                        raise ValueError(f"Invalid image value: {v}")
+                else:
+                    raise ValueError(f"Unsupported content type in chat: {t}")
+
+            if role not in ("user", "assistant"):
+                raise ValueError(f"Invalid role: {role}")
+            hf_messages.append({"role": role, "content": content})
+        return hf_messages
+
+    def _strings_to_msg_dicts(self, items: List[str]) -> List[Dict[str, Any]]:
+        out: List[Dict[str, Any]] = []
+        text_buf: List[str] = []
+        for s in items:
+            if isinstance(s, str) and (_is_url(s) or _is_local_image_path(s)):
+                if text_buf:
+                    out.append({"type": "text", "value": " ".join(text_buf)})
+                    text_buf = []
+                out.append({"type": "image", "value": s})
+            else:
+                text_buf.append(str(s))
+        if text_buf:
+            out.append({"type": "text", "value": " ".join(text_buf)})
+        return out
+
+    def message_to_promptimg(
+        self,
+        message: Union[List[Dict[str, Any]], List[str]],
+        dataset: Optional[str] = None
+    ) -> Tuple[str, Optional[str]]:
+        if len(message) > 0 and isinstance(message[0], str):
+            message = self._strings_to_msg_dicts(message)
+
+        texts: List[str] = []
+        first_img: Optional[str] = None
+        for m in message:
+            if m.get("type") == "image" and first_img is None:
+                v_str = _best_image_value(m.get("value"))
+                if isinstance(v_str, str) and (_is_url(v_str) or _is_local_image_path(v_str)):
+                    first_img = v_str
+            elif m.get("type") == "text":
+                texts.append(str(m.get("value")))
+        return (" ".join(texts).strip(), first_img)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..f1b4cb47096ac17b4821d7d700090221a293645f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/__init__.py
@@ -0,0 +1,3 @@
+from .internvl_chat import InternVLChat
+
+__all__ = ['InternVLChat']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/gui_template.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/gui_template.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..7ada4ea11d35729e401a7c6f63f3f0b48740d33e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/gui_template.yaml
@@ -0,0 +1,45 @@
+ScreenSpot:
+  template_zeroshot: |-
+    Based on the screenshot of the page, I give a text description and you give the bounding box coordinate of the region this sentence describes: {task}
+  template: |-
+    {task}
+  placeholders:
+    - task
+
+ScreenSpot_Pro:
+  template_zeroshot: |-
+    Based on the screenshot of the page, I give a text description and you give the bounding box coordinate of the region this sentence describes: {task}
+  template: |-
+    {task}
+  placeholders:
+    - task
+
+ScreenSpot_v2:
+  template_zeroshot: |-
+    Based on the screenshot of the page, I give a text description and you give the bounding box coordinate of the region this sentence describes: {task}
+  template: |-
+    {task}
+  placeholders:
+    - task
+
+MM_Mind2Web:
+  system_prompt: |-
+    You are a GUI agent. You are given a task and a screenshot of the screen. You need to perform a series of pyautogui actions to complete the task.
+
+    You have access to the following functions:
+    - {"name": "mobile.swipe", "description": "swipe on the screen", "parameters": {"type": "object", "properties": {"from_coord": {"type": "array", "items": {"type": "number"}, "description": "The starting coordinates of the swipe"}, "to_coord": {"type": "array", "items": {"type": "number"}, "description": "The ending coordinates of the swipe"}}, "required": ["from_coord", "to_coord"]}}
+    - {"name": "mobile.home", "description": "Press the home button"}
+    - {"name": "mobile.back", "description": "Press the back button"}
+
+  template: |-
+    Please generate the next move according to the ui screenshot, instruction and previous actions.
+
+    Instruction:
+    {task}.
+
+    Previous actions:
+    {history}.
+
+  placeholders:
+    - task
+    - history
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/internvl_chat.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/internvl_chat.py
new file mode 100644
index 0000000000000000000000000000000000000000..be73f870d796a00c45ac74c90bf5b1022b139fef
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/internvl_chat.py
@@ -0,0 +1,574 @@
+import os
+import re
+import warnings
+from pathlib import Path
+
+import torch
+import transformers
+import yaml
+from PIL import Image
+from transformers import AutoModel, AutoTokenizer, CLIPImageProcessor
+
+from vlmeval.dataset import DATASET_MODALITY, DATASET_TYPE, build_dataset, infer_dataset_basename
+from vlmeval.smp import encode_image_to_base64, listinstr, version_cmp
+from ..base import BaseModel
+from .utils import (build_mcq_cot_prompt, build_mpo_prompt, build_multi_choice_prompt,
+                    build_qa_cot_prompt, build_video_prompt, format_nav_prompt, load_image,
+                    mpo_post_processing, parse_bbox_internvl, pile_action_history,
+                    reorganize_prompt)
+
+# load all the gui templates
+upper_path = Path(__file__).parent
+with open(os.path.join(upper_path, "gui_template.yaml"), "r") as f:
+    GUI_TEMPLATE = yaml.load(f, Loader=yaml.FullLoader)
+
+R1_SYSTEM_PROMPT = """
+You are an AI assistant that rigorously follows this response protocol:
+
+1. First, conduct a detailed analysis of the question. Consider different \
+angles, potential solutions, and reason through the problem step-by-step. \
+Enclose this entire thinking process within <think> and </think> tags.
+
+2. After the thinking section, provide a clear, concise, and direct answer to \
+the user's question. Separate the answer from the think section with a newline.
+
+Ensure that the thinking process is thorough but remains focused on the \
+query. The final answer should be standalone and not reference the thinking \
+section.
+""".strip()
+
+
+def prepare_messages_list(prompt, image_path, system_prompt=None):
+    from lmdeploy.vl.constants import IMAGE_TOKEN
+    content = [{'type': 'text', 'text': prompt.replace('<image>', IMAGE_TOKEN)}]
+
+    if isinstance(image_path, str):
+        image_path = [image_path]
+
+    for image in image_path:
+        img = Image.open(image).convert('RGB')
+        b64 = encode_image_to_base64(img)
+        img_struct = dict(url=f'data:image/jpeg;base64,{b64}')
+        content.append(dict(type='image_url', image_url=img_struct))
+
+    messages = []
+
+    if system_prompt is not None:
+        messages.append({'role': 'system', 'content': system_prompt})
+
+    messages.append({
+        'role': 'user',
+        'content': content,
+    })
+
+    return [messages]
+
+
+def extract_boxed_content(ans: str):
+    idx = ans.rfind(r'\boxed{')
+    if idx == -1:
+        return ans
+
+    idx += len(r'\boxed{')
+    brace_level = 1
+    content_start = idx
+    i = idx
+
+    while i < len(ans):
+        if ans[i] == '{':
+            brace_level += 1
+        elif ans[i] == '}':
+            brace_level -= 1
+            if brace_level == 0:
+                break
+        i += 1
+
+    if brace_level != 0:
+        # Unbalanced braces
+        return ans
+
+    content = ans[content_start:i]
+    return content
+
+
+class InternVLChat(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self,
+                 model_path='OpenGVLab/InternVL-Chat-V1-5',
+                 use_custom_prompt=True,
+                 load_in_8bit=False,
+                 use_mpo_prompt=False,
+                 version='V1.0',
+                 screen_parse=True,
+                 # Best-of-N parameters
+                 best_of_n=1,
+                 reward_model_path=None,
+                 # R1 parameters
+                 cot_prompt_version='v1',
+                 #
+                 use_lmdeploy=False,
+                 use_postprocess=False,
+                 **kwargs):
+
+        assert best_of_n >= 1
+        assert model_path is not None
+        assert version_cmp(transformers.__version__, '4.37.2', 'ge')
+
+        self.use_lmdeploy = use_lmdeploy
+        self.cot_prompt_version = cot_prompt_version
+        self.use_mpo_prompt = use_mpo_prompt
+        self.use_cot = (os.getenv('USE_COT') == '1')
+        self.use_postprocess = use_postprocess
+        self._use_custom_prompt = use_custom_prompt
+
+        if cot_prompt_version == 'r1':
+            self.system_prompt = R1_SYSTEM_PROMPT
+            self.cot_prompt = 'Please answer the question and put the final answer within \\boxed{}.'
+        elif cot_prompt_version == 'v2':
+            self.system_prompt = None
+            self.cot_prompt = "Answer the preceding multiple-choice question \
+            by carefully analyzing the provided image. \nPlease answer with \
+            carefully thought step by step. Apply the thinking process \
+            recursively at both macro and micro levels. \nVerify consistency \
+            of reasoning and look for potential flaws or gaps during \
+            thinking. \nWhen realize mistakes, explain why the previous \
+            thinking was incorrect, fix it and then continue thinking.\nThe \
+            last line of your response should follow this format: 'Answer: \
+            \\boxed{$LETTER}' (without quotes), where LETTER is one of the \
+            options\n\n"
+        else:
+            assert cot_prompt_version == 'v1'
+            self.system_prompt = None
+            self.cot_prompt = None
+
+        self.model_path = model_path
+        self.tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
+
+        # Regular expression to match the pattern 'Image' followed by a number, e.g. Image1
+        self.pattern = r'Image(\d+)'
+        # Replacement pattern to insert a hyphen between 'Image' and the number, e.g. Image-1
+        self.replacement = r'Image-\1'
+
+        # Convert InternVL2 response to dataset format
+        # e.g. Image1 -> Image-1
+
+        # Regular expression to match the pattern 'Image-' followed by a number
+        self.reverse_pattern = r'Image-(\d+)'
+        # Replacement pattern to remove the hyphen (Image-1 -> Image1)
+        self.reverse_replacement = r'Image\1'
+
+        self.screen_parse = screen_parse
+
+        if use_lmdeploy:
+            from lmdeploy import PytorchEngineConfig, TurbomindEngineConfig, VisionConfig, pipeline
+            engine_type = PytorchEngineConfig if "internvl3_5" in model_path.lower() else TurbomindEngineConfig
+            vision_config = VisionConfig(max_batch_size=4)
+            num_gpus = torch.cuda.device_count()
+            self.model = pipeline(
+                model_path,
+                vision_config=vision_config,
+                backend_config=engine_type(
+                    session_len=max(16384, kwargs.get("max_new_tokens", 16384)),
+                    cache_max_entry_count=0.5,
+                    tp=num_gpus,
+                )
+            )
+            torch.cuda.set_device(0)
+            self.device = 'cuda'
+        else:
+            self.model = AutoModel.from_pretrained(
+                model_path,
+                torch_dtype=torch.bfloat16,
+                trust_remote_code=True,
+                low_cpu_mem_usage=True,
+                device_map="auto").eval()
+            self.device = 'cuda'
+
+        if best_of_n > 1:
+            assert version == 'V2.0', 'only support BoN evaluation with version==V2.0'
+            assert reward_model_path is not None
+
+            self.reward_tokenizer = AutoTokenizer.from_pretrained(
+                reward_model_path, trust_remote_code=True, use_fast=False)
+            self.reward_model = AutoModel.from_pretrained(
+                reward_model_path,
+                torch_dtype=torch.bfloat16,
+                trust_remote_code=True,
+                low_cpu_mem_usage=True,
+                device_map="auto").eval()
+
+            if not self.use_cot:
+                os.environ['USE_COT'] = '1'
+                self.use_cot = True
+                print('[Warning] Since Best-of-N is enabled, USE_COT is forced to be set to 1.')
+
+            print(f'Enable Best-of-N evaluation with PRM: {reward_model_path}')
+
+        # self.image_size = self.model.config.vision_config.image_size
+        self.version = version
+        self.best_of_n = best_of_n
+        kwargs_default = dict(do_sample=False, max_new_tokens=4096, top_p=None)
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+
+        warnings.warn(f'Following kwargs received: {self.kwargs}, will use as generation config. ')
+
+    def use_custom_prompt(self, dataset):
+        if not self._use_custom_prompt:
+            return False
+
+        assert dataset is not None
+        if dataset in [
+            'atomic_dataset', 'electro_dataset', 'mechanics_dataset',
+            'optics_dataset', 'quantum_dataset', 'statistics_dataset'
+        ]:
+            return False
+        if dataset in ["SSI_Bench"]:
+            return False
+        if listinstr(['MMDU', 'MME-RealWorld', 'MME-RealWorld-CN', 'WeMath_COT', 'MMAlignBench', 'ChartQAPro', 'ChartMuseum', 'MMSIVideo_U50', 'MMSIVideo_SC'], dataset):  # noqa: E501
+            # For Multi-Turn we don't have custom prompt
+            return False
+        if DATASET_TYPE(dataset) == 'MCQ':
+            if dataset is not None and 'LEGO' in dataset:
+                return False
+        if DATASET_MODALITY(dataset) == 'VIDEO':
+            # For Video benchmarks we don't have custom prompt at here
+            return False
+        else:
+            return True
+
+    def build_prompt(self, line, dataset=None):
+        use_mpo_prompt = self.use_mpo_prompt and (self.use_cot or dataset in ['MMStar', 'HallusionBench', 'OCRBench'])
+
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+        if dataset is not None and listinstr(['BMMR'], dataset):
+            self.kwargs['max_new_tokens'] = max(self.kwargs.get('max_new_tokens', 4096), 8196)
+            print(f'[Warning] BMMR dataset requires a larger max_new_tokens, set to {self.kwargs["max_new_tokens"]}')
+
+        if dataset is not None and DATASET_TYPE(dataset) == 'Y/N':
+            question = line['question']
+            if listinstr(['MME'], dataset):
+                prompt = question + ' Answer the question using a single word or phrase.'
+            elif listinstr(['HallusionBench', 'AMBER'], dataset):
+                prompt = question + ' Please answer yes or no. Answer the question using a single word or phrase.'
+            else:
+                prompt = question
+        elif dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+            prompt = build_multi_choice_prompt(line, dataset)
+            if os.getenv('USE_COT') == '1':
+                prompt = build_mcq_cot_prompt(line, prompt, self.cot_prompt)
+        elif dataset is not None and DATASET_TYPE(dataset) == 'VQA':
+            question = line['question']
+            if listinstr(['LLaVABench', 'WildVision'], dataset):
+                prompt = question + '\nAnswer this question in detail.'
+            elif listinstr(['OCRVQA', 'TextVQA', 'ChartQA', 'DocVQA', 'InfoVQA', 'OCRBench',
+                            'DUDE', 'SLIDEVQA', 'GQA', 'MMLongBench_DOC'], dataset):
+                prompt = question + '\nAnswer the question using a single word or phrase.'
+            elif listinstr(['MathVerse'], dataset):
+                question = question.replace("please directly answer the question and", "please")
+                prompt = question
+                if os.getenv('USE_COT') == '1':
+                    prompt = build_qa_cot_prompt(line, prompt, self.cot_prompt)
+            elif listinstr(['MathVista', 'MathVision', 'VCR', 'MTVQA', 'MMVet',
+                            'MMDU', 'CRPE', 'MIA-Bench', 'MM-Math', 'DynaMath', 'QSpatial',
+                            'WeMath', 'LogicVista', 'MM-IFEval', 'ChartMimic', 'MMReason'], dataset):
+                prompt = question
+                if os.getenv('USE_COT') == '1':
+                    prompt = build_qa_cot_prompt(line, prompt, self.cot_prompt)
+            else:
+                prompt = question + '\nAnswer the question using a single word or phrase.'
+        elif dataset is not None and DATASET_TYPE(dataset) == 'GUI':
+            ds_basename = infer_dataset_basename(dataset)
+            ds = build_dataset(dataset, skeleton=True)
+            action_space = ds.get_action_space()
+            traj_dict = ds.get_trajectory(line)
+
+            prompt_config = GUI_TEMPLATE[ds_basename]
+            if 'history' in prompt_config["placeholders"]:
+                traj_dict['history'] = pile_action_history(traj_dict['history'])
+            prompt = format_nav_prompt(
+                (
+                    "Please provide the bounding box coordinate of the region this sentence describes: <ref>{task}</ref>"  # noqa: E501
+                    if self.screen_parse
+                    else prompt_config["template"]
+                ),
+                prompt_config["placeholders"],
+                action_space=action_space,
+                **traj_dict,
+            )
+        else:
+            # VQA_ex_prompt: OlympiadBench, VizWiz
+            prompt = line['question']
+            if os.getenv('USE_COT') == '1':
+                prompt = build_qa_cot_prompt(line, prompt, self.cot_prompt)
+
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+
+        if use_mpo_prompt:
+            message = build_mpo_prompt(message, line, dataset)
+        return message
+
+    def set_max_num(self, dataset):
+        # The total limit on the number of images processed, set to avoid Out-of-Memory issues.
+        self.total_max_num = 64
+        if dataset is None:
+            self.max_num = 6
+            return None
+        res_12_datasets = ['ChartQA_TEST', 'MMMU_DEV_VAL', 'MMMU_TEST', 'MME-RealWorld',
+                           'VCR_EN', 'VCR_ZH', 'OCRVQA', 'BMMR']
+        res_18_datasets = ['DocVQA_VAL', 'DocVQA_TEST', 'DUDE', 'MMLongBench_DOC', 'SLIDEVQA']
+        res_24_datasets = ['InfoVQA_VAL', 'InfoVQA_TEST', 'OCRBench', 'HRBench4K', 'HRBench8K']
+        if DATASET_MODALITY(dataset) == 'VIDEO':
+            self.max_num = 1
+        elif listinstr(res_12_datasets, dataset):
+            self.max_num = 12
+        elif listinstr(res_18_datasets, dataset):
+            self.max_num = 18
+        elif listinstr(res_24_datasets, dataset):
+            self.max_num = 24
+        elif DATASET_TYPE(dataset) == 'GUI':
+            self.max_num = 12
+        else:
+            self.max_num = 6
+
+    def generate_v1_2(self, message, dataset=None):
+        self.INTERLEAVE = False
+        prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+        image = Image.open(image_path).convert('RGB')
+        image = image.resize((self.image_size, self.image_size))
+        image_processor = CLIPImageProcessor.from_pretrained(self.model_path)
+        pixel_values = image_processor(images=image, return_tensors='pt').pixel_values
+        pixel_values = pixel_values.to(torch.bfloat16).to(self.device)
+        with torch.no_grad():
+            response = self.model.chat(self.tokenizer, pixel_values=pixel_values,
+                                       question=prompt, generation_config=self.kwargs)
+        return response
+
+    def generate_v1_5(self, message, dataset=None):
+        image_num = len([x for x in message if x['type'] == 'image'])
+        max_num = max(1, min(self.max_num, self.total_max_num // image_num))
+        prompt = '\n'.join([x['value'] for x in message if x['type'] == 'text'])
+
+        if DATASET_MODALITY(dataset) == 'VIDEO':
+            prompt = build_video_prompt(prompt, dataset)
+
+        if image_num > 1:
+            image_path = [x['value'] for x in message if x['type'] == 'image']
+            pixel_values_list = []
+            for file_name in image_path:
+                pixel_values_list.append(load_image(file_name, max_num=max_num).to(self.device).to(torch.bfloat16))
+            pixel_values = torch.cat(pixel_values_list, dim=0)
+        elif image_num == 1:
+            image_path = [x['value'] for x in message if x['type'] == 'image'][0]
+            pixel_values = load_image(image_path, max_num=max_num).to(self.device).to(torch.bfloat16)
+        else:
+            pixel_values = None
+        with torch.no_grad():
+            response = self.model.chat(
+                self.tokenizer,
+                pixel_values=pixel_values,
+                question=prompt,
+                generation_config=self.kwargs,
+                verbose=True)
+        return response
+
+    @torch.no_grad()
+    def generate_v2(self, message, dataset=None):
+
+        use_mpo_prompt = self.use_mpo_prompt and (self.use_cot or dataset in ['MMStar', 'HallusionBench', 'OCRBench'])
+
+        image_num = len([x for x in message if x['type'] == 'image'])
+        max_num = max(1, min(self.max_num, self.total_max_num // image_num))
+        prompt = reorganize_prompt(message, image_num, dataset=dataset)
+
+        if dataset is not None and DATASET_MODALITY(dataset) == 'VIDEO':
+            prompt = build_video_prompt(prompt, dataset)
+
+        if image_num > 1:
+            image_path = [x['value'] for x in message if x['type'] == 'image']
+            num_patches_list, pixel_values_list = [], []
+            for image_idx, file_name in enumerate(image_path):
+                upscale_flag = image_idx == 0 and dataset is not None and listinstr(['MMMU'], dataset)
+                curr_pixel_values = load_image(
+                    file_name, max_num=max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+                num_patches_list.append(curr_pixel_values.size(0))
+                pixel_values_list.append(curr_pixel_values)
+            pixel_values = torch.cat(pixel_values_list, dim=0)
+        elif image_num == 1:
+            image_path = [x['value'] for x in message if x['type'] == 'image'][0]
+            upscale_flag = dataset is not None and listinstr(['MMMU'], dataset)
+            pixel_values = load_image(
+                image_path, max_num=max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+            num_patches_list = [pixel_values.size(0)]
+        else:
+            pixel_values = None
+            num_patches_list = []
+
+        response_list = []
+        for idx in range(self.best_of_n):
+            kwargs_default = self.kwargs.copy()
+            kwargs_default['do_sample'] = idx > 0 or kwargs_default.get('do_sample', False)
+            kwargs_default['temperature'] = 0.6
+            kwargs_default['top_p'] = 0.95
+
+            if self.use_lmdeploy:
+                from lmdeploy import GenerationConfig
+                gen_config = GenerationConfig(**kwargs_default)
+                gen_config.random_seed = None
+                messages_list = prepare_messages_list(prompt, image_path, system_prompt=self.system_prompt)
+                assert len(messages_list) == 1
+                response = self.model(messages_list, gen_config=gen_config)[0]
+                response = response.text
+            else:
+                if self.system_prompt is not None:
+                    self.model.system_message = self.system_prompt
+                response = self.model.chat(
+                    self.tokenizer,
+                    pixel_values=pixel_values,
+                    num_patches_list=num_patches_list,
+                    question=prompt,
+                    generation_config=kwargs_default,
+                    verbose=idx == 0,
+                )
+            response_list.append(response)
+
+        if self.best_of_n > 1:
+            response_list = self.reward_model.select_best_response(
+                tokenizer=self.reward_tokenizer,
+                question=prompt,
+                response_list=response_list,
+                pixel_values=pixel_values,
+                num_patches_list=num_patches_list,
+            )
+        response = response_list[0]
+
+        if dataset is not None and not listinstr(['WeMath'], dataset):
+            if use_mpo_prompt:
+                response = mpo_post_processing(response, dataset)
+            elif self.use_cot and self.use_postprocess:
+                response = extract_boxed_content(response)
+
+        if dataset is not None and DATASET_TYPE(dataset) == 'GUI' and self.screen_parse:
+            # Parse the bounding box coordinates from the response
+            response = parse_bbox_internvl(response)
+            # Normalize the coordinates to the range [0, 1]
+            if isinstance(response, list):
+                response = [item / 1000 for item in response]
+                # Convert the coordinates to the format required by the GUI
+                response = f"x={response[0]}, y={response[1]}"
+
+        return response
+
+    def generate_inner(self, message, dataset=None):
+        self.set_max_num(dataset)
+        print(f'InternVL model version: {self.version}')
+        if self.version in ['V1.1', 'V1.2']:
+            return self.generate_v1_2(message, dataset)
+        elif self.version == 'V1.5':
+            return self.generate_v1_5(message, dataset)
+        elif self.version == 'V2.0':
+            return self.generate_v2(message, dataset)
+        else:
+            raise ValueError(f'Unsupported version: {self.version}')
+
+    def build_history(self, message):
+        # Global Variables
+        image_path = []
+        image_cnt = 0
+
+        def concat_tilist(tilist):
+            nonlocal image_cnt  # Declare image_cnt as nonlocal to modify it
+            prompt = ''
+            for item in tilist:
+                # Substitute the pattern in the text
+                if item['type'] == 'text':
+                    prompt += re.sub(self.pattern, self.replacement, item['value'])
+                elif item['type'] == 'image':
+                    image_cnt += 1
+                    prompt += '<image>\n'
+                    image_path.append(item['value'])
+            return prompt
+
+        # Only previous messages
+        assert len(message) % 2 == 0
+        history = []
+        for i in range(len(message) // 2):
+            m1, m2 = message[2 * i], message[2 * i + 1]
+            assert m1['role'] == 'user' and m2['role'] == 'assistant'
+            history.append((concat_tilist(m1['content']), concat_tilist(m2['content'])))
+
+        return history, image_path, image_cnt
+
+    def chat_inner_v2(self, message, dataset=None):
+
+        if len(message) > 1:
+            history, image_path, image_cnt = self.build_history(message[:-1])
+        else:
+            history, image_path, image_cnt = None, [], 1
+        current_msg = message[-1]
+        question = ''
+
+        # If message is just text in the conversation
+        if len(current_msg['content']) == 1 and current_msg['content'][0]['type'] == 'text':
+            question = current_msg['content'][0]['value']
+            question = re.sub(self.pattern, self.replacement, question)  # Fix pattern as per InternVL
+        else:
+            for msg in current_msg['content']:
+                if msg['type'] == 'text':
+                    question += re.sub(self.pattern, self.replacement, msg['value'])
+                elif msg['type'] == 'image':
+                    image_cnt += 1
+                    question += '<image>\n'
+                    image_path.append(msg['value'])
+
+        if image_cnt > 1:
+            num_patches_list = []
+            pixel_values_list = []
+            for image_idx, file_name in enumerate(image_path):
+                upscale_flag = image_idx == 0 and dataset is not None and listinstr(['MMMU_DEV_VAL'], dataset)
+                curr_pixel_values = load_image(
+                    file_name, max_num=self.max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+                num_patches_list.append(curr_pixel_values.size(0))
+                pixel_values_list.append(curr_pixel_values)
+            pixel_values = torch.cat(pixel_values_list, dim=0)
+        elif image_cnt == 1:
+            upscale_flag = listinstr(['MMMU_DEV_VAL'], dataset)
+            pixel_values = load_image(
+                image_path, max_num=self.max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+            num_patches_list = [pixel_values.size(0)]
+        else:
+            pixel_values = None
+            num_patches_list = []
+
+        response, history = self.model.chat(
+            self.tokenizer,
+            pixel_values=pixel_values,
+            num_patches_list=num_patches_list,
+            question=question,
+            generation_config=self.kwargs,
+            history=history,
+            return_history=True
+        )
+
+        response = re.sub(self.reverse_pattern, self.reverse_replacement, response)
+
+        return response
+
+    def chat_inner(self, message, dataset=None):
+        self.set_max_num(dataset)
+
+        if self.version in ['V1.1', 'V1.2']:
+            raise ValueError(f'Unsupported version for Multi-Turn: {self.version}')
+        elif self.version == 'V1.5':
+            raise ValueError(f'Unsupported version for Multi-Turn: {self.version}')
+        elif self.version == 'V2.0':
+            kwargs_default = dict(do_sample=False, max_new_tokens=512, top_p=None, num_beams=1)
+            self.kwargs = kwargs_default
+            return self.chat_inner_v2(message, dataset)
+        else:
+            raise ValueError(f'Unsupported version for Multi-Turn: {self.version}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..b5150ab02164492dd77d57bf3568775ee78b4679
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/internvl/utils.py
@@ -0,0 +1,329 @@
+import os
+import re
+import string
+
+import pandas as pd
+import torch
+import torch.distributed as dist
+import torchvision.transforms as T
+from PIL import Image
+from torchvision.transforms.functional import InterpolationMode
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import cn_string, listinstr
+
+IMAGENET_MEAN = (0.485, 0.456, 0.406)
+IMAGENET_STD = (0.229, 0.224, 0.225)
+
+
+def build_transform(input_size):
+    MEAN, STD = IMAGENET_MEAN, IMAGENET_STD
+    transform = T.Compose([
+        T.Lambda(lambda img: img.convert('RGB') if img.mode != 'RGB' else img),
+        T.Resize((input_size, input_size), interpolation=InterpolationMode.BICUBIC),
+        T.ToTensor(),
+        T.Normalize(mean=MEAN, std=STD)
+    ])
+    return transform
+
+
+def find_closest_aspect_ratio(aspect_ratio, target_ratios, width, height, image_size):
+    best_ratio_diff = float('inf')
+    best_ratio = (1, 1)
+    area = width * height
+    for ratio in target_ratios:
+        target_aspect_ratio = ratio[0] / ratio[1]
+        ratio_diff = abs(aspect_ratio - target_aspect_ratio)
+        if ratio_diff < best_ratio_diff:
+            best_ratio_diff = ratio_diff
+            best_ratio = ratio
+        elif ratio_diff == best_ratio_diff:
+            if area > 0.5 * image_size * image_size * ratio[0] * ratio[1]:
+                best_ratio = ratio
+    return best_ratio
+
+
+def dynamic_preprocess(image, min_num=1, max_num=6, image_size=448, use_thumbnail=False):
+    orig_width, orig_height = image.size
+    aspect_ratio = orig_width / orig_height
+
+    # calculate the existing image aspect ratio
+    target_ratios = set(
+        (i, j) for n in range(min_num, max_num + 1) for i in range(1, n + 1) for j in range(1, n + 1) if
+        i * j <= max_num and i * j >= min_num)
+    target_ratios = sorted(target_ratios, key=lambda x: x[0] * x[1])
+
+    # find the closest aspect ratio to the target
+    target_aspect_ratio = find_closest_aspect_ratio(
+        aspect_ratio, target_ratios, orig_width, orig_height, image_size)
+
+    # calculate the target width and height
+    target_width = image_size * target_aspect_ratio[0]
+    target_height = image_size * target_aspect_ratio[1]
+    blocks = target_aspect_ratio[0] * target_aspect_ratio[1]
+
+    # resize the image
+    resized_img = image.resize((target_width, target_height))
+    processed_images = []
+    for i in range(blocks):
+        box = (
+            (i % (target_width // image_size)) * image_size,
+            (i // (target_width // image_size)) * image_size,
+            ((i % (target_width // image_size)) + 1) * image_size,
+            ((i // (target_width // image_size)) + 1) * image_size
+        )
+        # split the image
+        split_img = resized_img.crop(box)
+        processed_images.append(split_img)
+    assert len(processed_images) == blocks
+    if use_thumbnail and len(processed_images) != 1:
+        thumbnail_img = image.resize((image_size, image_size))
+        processed_images.append(thumbnail_img)
+    return processed_images
+
+
+def load_image(image_file, input_size=448, max_num=6, upscale=False):
+    image = Image.open(image_file).convert('RGB')
+    if upscale:
+        image = image.resize((image.width * 2, image.height * 2), Image.BILINEAR)
+    transform = build_transform(input_size=input_size)
+    images = dynamic_preprocess(image, image_size=input_size, use_thumbnail=True, max_num=max_num)
+    pixel_values = [transform(image) for image in images]
+    pixel_values = torch.stack(pixel_values)
+    return pixel_values
+
+
+def get_local_rank_and_local_world_size():
+    if not dist.is_available():
+        return 0, 1
+    if not dist.is_initialized():
+        return 0, 1
+
+    if 'SLURM_LOCALID' in os.environ:
+        local_rank = int(os.environ['SLURM_LOCALID'])
+        local_world_size = int(os.environ['SLURM_NTASKS_PER_NODE'])
+        return local_rank, local_world_size
+
+    if 'LOCAL_RANK' in os.environ and 'LOCAL_WORLD_SIZE' in os.environ:
+        return int(os.environ['LOCAL_RANK']), int(os.environ['LOCAL_WORLD_SIZE'])
+
+    raise NotImplementedError(
+        "Fail to get local_rank and local_world_size! "
+        "Please ensure that you set the environment variable "
+        "`LOCAL_RANK` and `LOCAL_WORLD_SIZE`"
+    )
+
+
+def build_yesorno_cot_prompt(line, prompt, cot_prompt=None):
+    if cot_prompt is None:
+        cot_prompt = (
+            "Answer the preceding yes-or-no question. Think step by step logically, considering all "
+            "relevant information before answering. If you are uncertain or the problem is ambiguous, make a "
+            "reasoned guess based on the information provided. The last line of your response must be exactly "
+            "in this format: 'Answer: \\boxed{Yes}' or 'Answer: \\boxed{No}' (without quotes)."
+        )
+    prompt = prompt.replace("Please answer yes or no. Answer the question using a single word or phrase.", '').strip()
+    prompt = prompt + '\n' + cot_prompt
+
+    return prompt
+
+
+def build_mcq_cot_prompt(line, prompt, cot_prompt=None):
+    if cot_prompt is None:
+        cot_prompt = (
+            "Answer the preceding multiple choice question. The last line of your response should follow "
+            "this format: 'Answer: \\boxed{$LETTER}' (without quotes), where LETTER is one of the options. "
+            "If you are uncertain or the problem is too complex, make a reasoned guess based on the "
+            "information provided. Avoid repeating steps indefinitely—provide your best guess even if "
+            "unsure. Think step by step logically, considering all relevant information before answering."
+        )
+    prompt = prompt.replace("Answer with the option's letter from the given choices directly.", '').strip()
+    prompt = prompt + '\n' + cot_prompt
+
+    return prompt
+
+
+def build_qa_cot_prompt(line, prompt, cot_prompt=None):
+    if cot_prompt is None:
+        cot_prompt = (
+            "Answer the preceding question. The last line of your response should follow this format: "
+            "'Answer: \\boxed{$FINAL_ANSWER}' (without quotes), where 'FINAL_ANSWER' is your conclusion "
+            "based on the reasoning provided. If you are uncertain or the problem is too complex, make "
+            "a reasoned guess based on the information provided. Avoid repeating steps indefinitely—"
+            "provide your best guess even if unsure. Think step by step logically, considering all "
+            "relevant information before answering."
+        )
+    prompt = prompt + '\n' + cot_prompt
+
+    return prompt
+
+
+def build_multi_choice_prompt(line, dataset=None):
+    question = line['question']
+    hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+    if hint is not None:
+        question = hint + '\n' + question
+
+    options = {
+        cand: line[cand]
+        for cand in string.ascii_uppercase
+        if cand in line and not pd.isna(line[cand])
+    }
+    for key, item in options.items():
+        question += f'\n{key}. {item}'
+    prompt = question
+
+    if len(options):
+        prompt += '\n请直接回答选项字母。' if cn_string(
+            prompt) else "\nAnswer with the option's letter from the given choices directly."
+    else:
+        prompt += '\n请直接回答问题。' if cn_string(prompt) else '\nAnswer the question directly.'
+
+    return prompt
+
+
+def build_video_prompt(prompt, dataset=None, max_frames=64):
+    for start in range(0, max_frames, 8):
+        images_to_remove = ''.join([f'<Image-{i}>' for i in range(start + 1, start + 9)])
+        prompt = prompt.replace(images_to_remove, '')
+    for i in range(max_frames):
+        prompt = prompt.replace(f'Image-{i + 1}', f'Frame-{i + 1}')
+    if listinstr(['MMBench-Video'], dataset):
+        prompt = prompt.replace('\nAnswer:', '')
+    elif listinstr(['Video-MME', 'WorldSense'], dataset):
+        prompt = prompt.replace('\nAnswer:', '')
+        prompt += "\nAnswer with the option's letter from the given choices directly."
+    elif listinstr(['MVBench'], dataset):
+        prompt = prompt.replace('Best option:(', '')
+
+    return prompt
+
+
+def reorganize_prompt(message, image_num, dataset=None):
+    if dataset is not None and listinstr(['MUIRBench'], dataset):
+        prompt = '\n'.join([x['value'] for x in message if x['type'] == 'text'])
+        images_to_remove = ' '.join(['<image>'] * image_num)
+        prompt = prompt.replace(images_to_remove, '')
+        for i in range(image_num):
+            prompt = prompt.replace('<image>', f'<Image-{i + 1}>', 1)
+        prompt = ''.join([f'Image-{i + 1}: <image>\n' for i in range(image_num)]) + prompt
+    elif dataset is not None and listinstr(["bmmr"], dataset.lower()):
+        if image_num == 1:
+            prompt = "\n".join([x["value"] for x in message if x["type"] == "text"])
+        else:
+            prompt, image_idx = "", 1
+            for x in message:
+                if x["type"] == "text":
+                    prompt += x["value"]
+                elif x["type"] == "image":
+                    image_idx += 1
+    elif dataset is not None and listinstr(["mmsivideo"], dataset.lower()):
+        prompt, image_idx = '', 1
+        for x in message:
+            if x['type'] == 'text':
+                prompt += x['value']
+            elif x['type'] == 'image':
+                prompt += '<image>'
+    elif image_num == 1:
+        prompt = '<image>\n' + '\n'.join([x['value'] for x in message if x['type'] == 'text'])
+    else:
+        prompt, image_idx = '', 1
+        for x in message:
+            if x['type'] == 'text':
+                prompt += x['value']
+            elif x['type'] == 'image':
+                prompt += f'<Image-{image_idx}>'
+                image_idx += 1
+        prompt = ''.join([f'Image-{i + 1}: <image>\n' for i in range(image_num)]) + prompt
+        images_to_remove = ''.join([f'<Image-{i + 1}>' for i in range(image_num)])
+        prompt = prompt.replace(images_to_remove, '')
+    return prompt
+
+
+mpo_prompt_with_final_answer = (
+    "Your task is to answer the question below. "
+    "Give step by step reasoning before you answer, and when you're ready to answer, "
+    "please use the format \"Final answer: ..\""
+    "\n\n"
+    "Question:"
+    "\n\n"
+    "{question}"
+)
+
+mpo_prompt_without_final_answer = (
+    "Your task is to answer the question below. "
+    "Give step by step reasoning. "
+    "\n\n"
+    "Question:"
+    "\n\n"
+    "{question}"
+)
+
+
+def mpo_post_processing(response, dataset):
+
+    def extract_answer(text):
+        match = re.search(r'(Final answer:|Answer:)\s*(.*)', text, re.IGNORECASE)
+        if match:
+            return match.group(2).strip()
+        return text
+
+    if dataset is not None and (DATASET_TYPE(dataset) in ['Y/N', 'MCQ'] or listinstr(['CRPE'], dataset)):
+        response = extract_answer(response).strip()
+    return response
+
+
+def parse_bbox_internvl(response):
+    # 使用正则表达式匹配bounding box
+    # pattern = r"<box>\[\[(\d+), (\d+), (\d+), (\d+)\]\]</box>"
+    pattern = r"\[\[(\d+), (\d+), (\d+), (\d+)\]\]"
+    match = re.search(pattern, response)
+    if match:
+        # 提取匹配到的坐标值并转换为整数
+        x1, y1, x2, y2 = map(int, match.groups())
+        return [(x1 + x2) / 2, (y1 + y2) / 2]
+    else:
+        return response
+
+
+def build_mpo_prompt(message, line, dataset):
+    if listinstr(['LLaVABench', 'MMVet'], dataset):
+        return message
+
+    question_orig = line['question']
+    if listinstr(['MathVerse', 'MathVision'], dataset):
+        question_orig = question_orig.split('Question:', 1)[-1].strip()
+        question_orig = question_orig.replace('Choices:\n', '').strip()
+    if listinstr(['WeMath'], dataset):
+        question_orig = question_orig.replace('Regarding the format, please answer following the template below, and be sure to include two <> symbols:\n<Thought process>: <<your thought process>> <Answer>: <<your option>>', '').strip()  # noqa: E501
+    options = {
+        cand: line[cand]
+        for cand in string.ascii_uppercase
+        if cand in line and not pd.isna(line[cand])
+    }
+    options_prompt = ''
+    for key, item in options.items():
+        options_prompt += f'{key}. {item}\n'
+
+    if options_prompt.strip():
+        question_orig = f'{question_orig}\n{options_prompt}'
+
+    cot_prompt = mpo_prompt_with_final_answer
+    prompt = cot_prompt.format(question=question_orig).strip()
+    message[0]['value'] = prompt
+    return message
+
+
+def format_nav_prompt(template, placeholders, **kwargs):
+    prompt = template
+    for placeholder in placeholders:
+        value = kwargs.get(placeholder, '')
+        prompt = prompt.replace(f"{{{placeholder}}}", str(value))
+    return prompt
+
+
+def pile_action_history(history, max_num=4):
+    if len(history) > 0:
+        return '\n'.join(history[-max_num:])
+    else:
+        return 'None'
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..34ca8283fbd39e3687449fc031653297989d5981
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/__init__.py
@@ -0,0 +1 @@
+from .model import KeyeChat  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/model.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..e1f63e6b5a17945e548931d866103a2169fbf67d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/model.py
@@ -0,0 +1,279 @@
+import logging
+import os
+import warnings
+
+import torch
+from transformers import AutoModel, AutoProcessor
+
+from ..base import BaseModel
+from .prompt import Qwen2VLPromptMixin
+
+
+def ensure_image_url(image: str) -> str:
+    prefixes = ["http://", "https://", "file://", "data:image;"]
+    if any(image.startswith(prefix) for prefix in prefixes):
+        return image
+    if os.path.exists(image):
+        return "file://" + image
+    raise ValueError(f"Invalid image: {image}")
+
+
+def ensure_video_url(video: str) -> str:
+    prefixes = ["http://", "https://", "file://", "data:video;"]
+    if any(video.startswith(prefix) for prefix in prefixes):
+        return video
+    if os.path.exists(video):
+        return "file://" + video
+    raise ValueError(f"Invalid video: {video}")
+
+
+class KeyeChat(Qwen2VLPromptMixin, BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(
+        self,
+        model_path='Kwai-Keye/Keye-VL-1_5-8B',
+        max_new_tokens=8192,  # Use a larger value for "think" models to avoid output truncation (e.g., MMMU)
+        top_p=0.001,
+        top_k=1,
+        temperature=0,
+        repetition_penalty=1.0,
+        do_sample=False,
+        use_custom_prompt: bool = False,
+        system_prompt: str | None = None,
+        verbose: bool = True,
+        no_think: bool = False,
+        think: bool = False,
+        use_vllm: bool = False,
+        min_pixels: int = None,
+        max_pixels: int = None,
+        post_process: bool = True,
+        **kwargs,
+    ):
+        super().__init__(use_custom_prompt=use_custom_prompt)
+        assert model_path is not None
+        self.model_path = model_path
+        self.processor = AutoProcessor.from_pretrained(
+            model_path, trust_remote_code=True
+        )
+        self.tokenizer = self.processor.tokenizer
+        self.use_vllm = use_vllm
+
+        if not self.use_vllm:
+            self.generate_kwargs = dict(
+                max_new_tokens=max_new_tokens,
+                top_p=top_p,
+                top_k=top_k,
+                temperature=temperature,
+                repetition_penalty=repetition_penalty,
+                do_sample=do_sample,
+            )
+            self.model = (
+                AutoModel.from_pretrained(
+                    model_path,
+                    torch_dtype=torch.bfloat16,
+                    _attn_implementation="flash_attention_2",
+                    trust_remote_code=True,
+                    device_map="cuda",
+                )
+                .eval()
+                .cuda()
+            )
+        else:
+            from vllm import LLM, SamplingParams
+
+            gpu_count = torch.cuda.device_count()
+            if gpu_count >= 8:
+                tp_size = 8
+            elif gpu_count >= 4:
+                tp_size = 4
+            elif gpu_count >= 2:
+                tp_size = 2
+            else:
+                tp_size = 1
+
+            self.generate_kwargs = SamplingParams(
+                max_tokens=max_new_tokens,
+                top_p=top_p,
+                top_k=top_k,
+                temperature=temperature,
+                repetition_penalty=repetition_penalty,
+            )
+            self.model = LLM(
+                model=model_path,
+                max_num_batched_tokens=161072,
+                max_model_len=161072,
+                trust_remote_code=True,
+                enforce_eager=False,
+                tensor_parallel_size=tp_size,
+                limit_mm_per_prompt={"image": 100, "video": 10},
+            )
+
+        self.system_prompt = system_prompt
+        self.verbose = verbose
+
+        self.min_pixels = min_pixels
+        self.max_pixels = max_pixels
+
+        self.no_think = no_think
+        self.think = think
+
+        self.post_process = post_process
+
+    def _prepare_content(
+        self, inputs: list[dict[str, str]], dataset: str | None = None
+    ) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        content = []
+        for s in inputs:
+            if s["type"] == "image":
+                item = {"type": "image", "image": ensure_image_url(s["value"])}
+                if dataset == "OCRBench":
+                    item["min_pixels"] = 10 * 10 * 28 * 28
+                    warnings.warn(
+                        f"OCRBench dataset uses custom min_pixels={item['min_pixels']}"
+                    )
+                    if self.max_pixels is not None:
+                        item["max_pixels"] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item["min_pixels"] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item["max_pixels"] = self.max_pixels
+            elif s["type"] == "text":
+                item = {"type": "text", "text": s["value"]}
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def add_think_token(self, messages, mode):
+        for i in range(len(messages[-1]["content"]) - 1, -1, -1):
+            if messages[-1]["content"][i]["type"] == "text":
+                messages[-1]["content"][i]["text"] += mode
+                break
+        return messages
+
+    def post_process_func(self, prediction):
+        import re
+
+        def get_boxed(response, bb="\\boxed{"):
+            resp = response.split(bb)[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == "{":
+                    counter += 1
+                elif resp[i] == "}":
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                response = resp[:end]
+            return response
+
+        if "</analysis>" in prediction:
+            prediction = prediction.split("</analysis>")[-1]
+        if "</think>" in prediction:
+            prediction = prediction.split("</think>")[-1].lstrip("\n").strip()
+        if "<answer>" not in prediction:
+            boxed_matches = get_boxed(prediction, bb=r"\boxed{")
+            if len(boxed_matches) != len(prediction):
+                return boxed_matches
+            else:
+                boxed_matches = get_boxed(prediction, bb="\boxed{")
+                return (
+                    boxed_matches
+                    if len(boxed_matches) != len(prediction)
+                    else prediction
+                )
+        matches = re.findall(r"<answer>(.*?)</answer>", prediction, re.DOTALL)
+        if matches:
+            content_match = matches[-1]
+            boxed_matches = get_boxed(content_match, bb=r"\boxed{")
+            if len(boxed_matches) != len(content_match):
+                return boxed_matches
+            else:
+                boxed_matches = get_boxed(content_match, bb="\boxed{")
+                return (
+                    boxed_matches
+                    if len(boxed_matches) != len(content_match)
+                    else content_match
+                )
+        else:
+            return prediction
+
+    def generate_inner(self, message, dataset=None):
+        try:
+            from keye_vl_utils import process_vision_info
+        except Exception as err:
+            logging.critical(
+                "keye_vl_utils not found, please install it via 'pip install keye-vl-utils'"
+            )
+            raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({"role": "system", "content": self.system_prompt})
+
+        messages.append(
+            {"role": "user", "content": self._prepare_content(message, dataset=dataset)}
+        )
+
+        if self.no_think:
+            messages = self.add_think_token(messages, "/no_think")
+        elif self.think:
+            messages = self.add_think_token(messages, "/think")
+
+        if self.verbose:
+            print(f"\033[31m{messages}\033[0m")
+
+        text = self.processor.apply_chat_template(
+            messages, tokenize=False, add_generation_prompt=True
+        )
+        image_inputs, video_inputs, mm_processor_kwargs = process_vision_info(messages)
+
+        if not self.use_vllm:
+            inputs = self.processor(
+                text=[text],
+                images=image_inputs,
+                videos=video_inputs,
+                padding=True,
+                return_tensors="pt",
+                **mm_processor_kwargs
+            )
+            inputs = inputs.to("cuda")
+            generated_ids = self.model.generate(
+                **inputs,
+                **self.generate_kwargs,
+            )
+            generated_ids_trimmed = [
+                out_ids[len(in_ids):]
+                for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+            ]
+            response = self.processor.batch_decode(
+                generated_ids_trimmed,
+                skip_special_tokens=True,
+                clean_up_tokenization_spaces=False,
+            )[0]
+        else:
+            mmdata = {}
+            if image_inputs is not None:
+                mmdata["image"] = image_inputs
+            if video_inputs is not None:
+                mmdata["video"] = video_inputs
+            inputs = [{"prompt": text, "multi_modal_data": mmdata}]
+            generated = self.model.generate(inputs, self.generate_kwargs)
+            response = generated[0].outputs[0].text
+
+        if self.post_process:
+            response = self.post_process_func(response)
+
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..ce2f8358acf7272e10d95efeba84ca7124a73b3e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/keye_vlm/prompt.py
@@ -0,0 +1,156 @@
+from __future__ import annotations
+
+
+class Qwen2VLPromptMixin:
+    """
+    Mixin class for Qwen2VLChat to build custom prompt for different datasets.
+
+    Requires the following methods to be implemented in the subclass:
+        - dump_image(line, dataset: str) -> str | list[str]
+
+    Implements the following methods:
+        - use_custom_prompt(dataset: str) -> bool
+        - build_prompt(line, dataset: str) -> list[dict[str, str]]
+    """
+
+    def __init__(self, *args, use_custom_prompt: bool = True, **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        self._use_custom_prompt = use_custom_prompt
+
+    def set_dump_image(self, dump_image_func):
+        self.dump_image_func = dump_image_func
+
+    def dump_image(self, line, dataset):
+        return self.dump_image_func(line)
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        from vlmeval.dataset import DATASET_TYPE
+        dataset_type = DATASET_TYPE(dataset, default=None)
+
+        if not self._use_custom_prompt:
+            return False
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return True
+        if dataset_type == 'MCQ':
+            return True
+        if dataset_type == 'Y/N' and dataset in {'HallusionBench', 'POPE'}:  # MME has it's own prompt
+            return True
+        # MMVet VQA has it's own prompt
+        if dataset_type == 'VQA' and dataset not in {'MMVet', 'Benchmark_V21', 'IFEval'}:
+            return True
+        return False
+
+    def build_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        from vlmeval.dataset import DATASET_TYPE
+
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return self._build_mmmu_prompt(line, dataset)
+        dataset_type = DATASET_TYPE(dataset, default=None)
+        if dataset_type == 'MCQ':
+            return self._build_mcq_prompt(line, dataset)
+        if dataset_type == 'Y/N':
+            return self._build_yorn_prompt(line, dataset)
+        if dataset_type == 'VQA':
+            return self._build_vqa_prompt(line, dataset)
+        raise ValueError(f'Unsupported dataset: {dataset}')
+
+    def _build_mmmu_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MMMU dataset: keep all images at beginning."""
+
+        import string
+
+        import pandas as pd
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += 'Please select the correct answer from the options above. \n'
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_mcq_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MCQ dataset: use chinese prompt if the question contains chinese characters."""
+        MCQ_CN_PROMPT = '请直接回答选项字母。'
+        MCQ_EN_PROMPT = 'Please select the correct answer from the options above.'
+
+        import string
+
+        import pandas as pd
+
+        def cn_string(s):
+            import re
+
+            if re.search('[\u4e00-\u9fff]', s):
+                return True
+            return False
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += MCQ_CN_PROMPT if cn_string(prompt) else MCQ_EN_PROMPT
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_yorn_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for YORN dataset:"""
+        YORN_PROMPT = ' Please answer yes or no.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += YORN_PROMPT
+        return msgs
+
+    def _build_vqa_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for VQA dataset:"""
+        VQA_PROMPT = '\nPlease try to answer the question with short words or phrases if possible.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += VQA_PROMPT
+        return msgs
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..c42e1b37213453d945824a9b94e174e957726908
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/__init__.py
@@ -0,0 +1,8 @@
+from .llava import (LLaVA, LLaVA_Next, LLaVA_Next2, LLaVA_OneVision, LLaVA_OneVision_1_5,
+                    LLaVA_OneVision_HF)
+from .llava_xtuner import LLaVA_XTuner
+
+__all__ = [
+    'LLaVA', 'LLaVA_Next', 'LLaVA_XTuner', 'LLaVA_Next2', 'LLaVA_OneVision', 'LLaVA_OneVision_HF',
+    'LLaVA_OneVision_1_5'
+]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/llava.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/llava.py
new file mode 100644
index 0000000000000000000000000000000000000000..c0f87edf42d77d63567bafdc7e4ebe673456cd1d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/llava.py
@@ -0,0 +1,910 @@
+import copy
+import logging
+import os.path as osp
+import string
+import warnings
+from argparse import Namespace
+
+import numpy as np
+import pandas as pd
+import torch
+from PIL import Image
+
+from vlmeval.dataset import DATASET_MODALITY, DATASET_TYPE
+from vlmeval.smp import cn_string, encode_image_to_base64, splitlen
+from ..base import BaseModel
+
+
+class LLaVA(BaseModel):
+
+    INSTALL_REQ = True
+    INTERLEAVE = True
+
+    def __init__(self, model_path="liuhaotian/llava_v1.5_7b", **kwargs):
+        try:
+            from llava.mm_utils import get_model_name_from_path
+            from llava.model.builder import load_pretrained_model
+        except Exception as err:
+            logging.critical(
+                "Please install llava from https://github.com/haotian-liu/LLaVA"
+            )
+            raise err
+
+        assert osp.exists(model_path) or splitlen(model_path) == 2
+        self.system_prompt = (
+            "A chat between a curious human and an artificial intelligence assistant. "
+            "The assistant gives helpful, detailed, and polite answers to the human's questions. "
+        )
+        self.stop_str = "</s>"
+
+        if model_path == "Lin-Chen/ShareGPT4V-7B":
+            model_name = "llava-v1.5-7b"
+        elif model_path == "Lin-Chen/ShareGPT4V-13B":
+            model_name = "llava-v1.5-13b"
+        else:
+            model_name = get_model_name_from_path(model_path)
+
+        try:
+            self.tokenizer, self.model, self.image_processor, self.context_len = (
+                load_pretrained_model(
+                    model_path=model_path,
+                    model_base=None,
+                    model_name=model_name,
+                    device_map="cpu",
+                )
+            )
+        except Exception as err:
+            if "ShareGPT4V" in model_path:
+                import llava
+
+                logging.critical(
+                    "Please manually remove the encoder type check in "
+                    f"{llava.__path__[0]}/model/multimodal_encoder/builder.py "
+                    "Line 8 to use the ShareGPT4V model. "
+                )
+            else:
+                logging.critical("Unknown error when loading LLaVA model.")
+            raise err
+
+        self.model = self.model.cuda()
+        self.conv_mode = "llava_v1"
+
+        kwargs_default = dict(
+            do_sample=False,
+            temperature=0,
+            max_new_tokens=2048,
+            top_p=None,
+            num_beams=1,
+            use_cache=True,
+        )  # noqa E501
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+        warnings.warn(
+            f"Following kwargs received: {self.kwargs}, will use as generation config. "
+        )
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == "MCQ":
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        question = line["question"]
+        hint = line["hint"] if ("hint" in line and not pd.isna(line["hint"])) else None
+        if hint is not None:
+            question = hint + "\n" + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f"\n{key}. {item}"
+        prompt = question
+
+        if len(options):
+            prompt += (
+                "\n请直接回答选项字母。"
+                if cn_string(prompt)
+                else "\nAnswer with the option's letter from the given choices directly."
+            )
+        else:
+            prompt += (
+                "\n请直接回答问题。"
+                if cn_string(prompt)
+                else "\nAnswer the question directly."
+            )
+
+        message = [dict(type="image", value=s) for s in tgt_path]
+        message.append(dict(type="text", value=prompt))
+        return message
+
+    def concat_tilist(self, message):
+        text, images = "", []
+        for item in message:
+            if item["type"] == "text":
+                text += item["value"]
+            elif item["type"] == "image":
+                text += " <image> "
+                images.append(item["value"])
+        return text, images
+
+    def chat_inner(self, message, dataset=None):
+        from llava.constants import IMAGE_TOKEN_INDEX
+        from llava.mm_utils import KeywordsStoppingCriteria, process_images, tokenizer_image_token
+
+        prompt = self.system_prompt
+        images = []
+        for utter in message:
+            prompt += "USER: " if utter["role"] == "user" else "ASSISTANT: "
+            content, images_sub = self.concat_tilist(utter["content"])
+            prompt += content
+            images.extend(images_sub)
+            prompt += " " if utter["role"] == "user" else self.stop_str
+        assert message[-1]["role"] == "user", message
+        prompt += "ASSISTANT: "
+
+        images = [Image.open(s).convert("RGB") for s in images]
+        args = Namespace()
+        args.image_aspect_ratio = "pad"
+        image_tensor = process_images(images, self.image_processor, args).to(
+            "cuda", dtype=torch.float16
+        )
+
+        input_ids = (
+            tokenizer_image_token(
+                prompt, self.tokenizer, IMAGE_TOKEN_INDEX, return_tensors="pt"
+            )
+            .unsqueeze(0)
+            .cuda()
+        )
+        keywords = [self.stop_str]
+        stopping_criteria = KeywordsStoppingCriteria(
+            keywords, self.tokenizer, input_ids
+        )
+        with torch.inference_mode():
+            output_ids = self.model.generate(
+                input_ids,
+                images=image_tensor,
+                stopping_criteria=[stopping_criteria],
+                **self.kwargs,
+            )
+        output = self.tokenizer.batch_decode(output_ids, skip_special_tokens=True)[
+            0
+        ].strip()
+        return output
+
+    def generate_inner(self, message, dataset=None):
+        from llava.constants import IMAGE_TOKEN_INDEX
+        from llava.mm_utils import KeywordsStoppingCriteria, process_images, tokenizer_image_token
+
+        # Support interleave text and image
+        content, images = self.concat_tilist(message)
+
+        images = [Image.open(s).convert("RGB") for s in images]
+        args = Namespace()
+        args.image_aspect_ratio = "pad"
+        if images:
+            image_tensor = process_images(images, self.image_processor, args).to(
+                "cuda", dtype=torch.float16
+            )
+        else:
+            image_tensor = None
+
+        prompt = self.system_prompt + "USER: " + content + " ASSISTANT: "
+
+        input_ids = (
+            tokenizer_image_token(
+                prompt, self.tokenizer, IMAGE_TOKEN_INDEX, return_tensors="pt"
+            )
+            .unsqueeze(0)
+            .cuda()
+        )
+        keywords = [self.stop_str]
+        stopping_criteria = KeywordsStoppingCriteria(
+            keywords, self.tokenizer, input_ids
+        )
+        with torch.inference_mode():
+            output_ids = self.model.generate(
+                input_ids,
+                images=image_tensor,
+                stopping_criteria=[stopping_criteria],
+                **self.kwargs,
+            )
+
+        output = self.tokenizer.batch_decode(output_ids, skip_special_tokens=True)[
+            0
+        ].strip()
+        return output
+
+
+class LLaVA_Next(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self, model_path="llava-hf/llava-v1.6-vicuna-7b-hf", **kwargs):
+        from transformers import (AutoProcessor, LlavaForConditionalGeneration,
+                                  LlavaNextForConditionalGeneration, LlavaNextProcessor)
+
+        self.model_path = model_path
+        if "34b" in model_path.lower():
+            self.processor = LlavaNextProcessor.from_pretrained(
+                self.model_path, use_fast=False
+            )
+        elif "interleave" in model_path.lower():
+            self.processor = AutoProcessor.from_pretrained(self.model_path)
+        else:
+            self.processor = LlavaNextProcessor.from_pretrained(self.model_path)
+        flash_attn_flag = False
+        try:
+            import flash_attn  # noqa: F401
+
+            flash_attn_flag = True
+        except ImportError:
+            pass
+
+        if flash_attn_flag:
+            if "interleave" in model_path.lower():
+                model = LlavaForConditionalGeneration.from_pretrained(
+                    self.model_path,
+                    torch_dtype=torch.float16,
+                    low_cpu_mem_usage=True,
+                    use_flash_attention_2=True,
+                )
+            else:
+                model = LlavaNextForConditionalGeneration.from_pretrained(
+                    self.model_path,
+                    torch_dtype=torch.float16,
+                    low_cpu_mem_usage=True,
+                    use_flash_attention_2=True,
+                )
+        else:
+            if "interleave" in model_path.lower():
+                model = LlavaForConditionalGeneration.from_pretrained(
+                    self.model_path, torch_dtype=torch.float16, low_cpu_mem_usage=True
+                )
+            else:
+                model = LlavaNextForConditionalGeneration.from_pretrained(
+                    self.model_path, torch_dtype=torch.float16, low_cpu_mem_usage=True
+                )
+
+        model = model.eval()
+        self.model = model.cuda()
+        kwargs_default = dict(
+            do_sample=False, temperature=0, max_new_tokens=2048, top_p=None, num_beams=1
+        )
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+        warnings.warn(
+            f"Following kwargs received: {self.kwargs}, will use as generation config. "
+        )
+
+    def apply_prompt_template(self, prompt):
+        model_path = self.model_path.lower()
+        if "mistral" in model_path:
+            template = "[INST] PLACEHOLDER [/INST]"
+        elif "vicuna" in model_path:
+            template = (
+                "A chat between a curious human and an artificial intelligence assistant. "
+                "The assistant gives helpful, detailed, and polite answers to the human's questions. "
+                "USER: PLACEHOLDER ASSISTANT:"
+            )
+        elif "34b" in model_path:
+            template = (
+                "<|im_start|>system\nAnswer the questions.<|im_end|><|im_start|>user\nPLACEHOLDER<|im_end|>"
+                "<|im_start|>assistant\n"
+            )
+        else:
+            raise NotImplementedError(
+                f"Prompt template for {model_path} not implemented."
+            )
+
+        prompt = template.replace("PLACEHOLDER", f"<image>\n{prompt}")
+        return prompt
+
+    def output_process(self, answer):
+        if "<s>" in answer:
+            answer = answer.replace("<s>", "").strip()
+        if "[/INST]" in answer:
+            answer = answer.split("[/INST]")[1].strip()
+        elif "ASSISTANT:" in answer:
+            answer = answer.split("ASSISTANT:")[1].strip()
+        elif "assistant\n" in answer:
+            answer = answer.split("assistant\n")[1].strip()
+        elif "<|end_header_id|>\n\n" in answer:
+            answer = answer.split("<|end_header_id|>\n\n")[2].strip()
+
+        if "</s>" in answer:
+            answer = answer.split("</s>")[0].strip()
+        elif "<|im_end|>" in answer:
+            answer = answer.split("<|im_end|>")[0].strip()
+        elif "<|eot_id|>" in answer:
+            answer = answer.split("<|eot_id|>")[0].strip()
+        return answer
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == "MCQ":
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        question = line["question"]
+        hint = line["hint"] if ("hint" in line and not pd.isna(line["hint"])) else None
+        if hint is not None:
+            question = hint + "\n" + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f"\n{key}. {item}"
+        prompt = question
+
+        if len(options):
+            prompt += (
+                "\n请直接回答选项字母。"
+                if cn_string(prompt)
+                else "\nAnswer with the option's letter from the given choices directly."
+            )
+        else:
+            prompt += (
+                "\n请直接回答问题。"
+                if cn_string(prompt)
+                else "\nAnswer the question directly."
+            )
+        message = [dict(type="image", value=s) for s in tgt_path]
+        message.append(dict(type="text", value=prompt))
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        content, images = [], []
+        for msg in message:
+            if msg["type"] == "text":
+                content.append({"type": msg["type"], "text": msg["value"]})
+            else:
+                content.append({"type": "image"})
+                images.append(Image.open(msg["value"]).convert("RGB"))
+        conversation = [
+            {
+                "role": "user",
+                "content": content,
+            }
+        ]
+        prompt = self.processor.apply_chat_template(
+            conversation, add_generation_prompt=True
+        )
+        inputs = self.processor(prompt, images, return_tensors="pt").to(
+            "cuda", torch.float16
+        )
+        output = self.model.generate(**inputs, **self.kwargs)
+        answer = self.processor.decode(output[0], skip_special_token=True)
+        answer = self.output_process(answer)
+        answer = answer.replace('<unk>', '')
+        return answer
+
+
+class LLaVA_Next2(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = True
+
+    DEFAULT_IMAGE_TOKEN = "<image>"
+    IMAGE_TOKEN_INDEX = -200
+
+    def __init__(self, model_path="lmms-lab/llama3-llava-next-8b", **kwargs):
+        assert model_path is not None
+        try:
+            from llava.conversation import SeparatorStyle, conv_templates
+            from llava.mm_utils import (KeywordsStoppingCriteria, get_model_name_from_path,
+                                        tokenizer_image_token)
+            from llava.model.builder import load_pretrained_model
+        except Exception as err:
+            logging.critical(
+                "Please `pip install git+https://github.com/LLaVA-VL/LLaVA-NeXT.git`"
+            )
+            raise err
+
+        model_name = get_model_name_from_path(model_path)
+        tokenizer, model, image_processor, _ = load_pretrained_model(
+            model_path, None, model_name, device_map=None
+        )
+        model.cuda().eval()
+        model.tie_weights()
+
+        if "llama3" in model_path.lower():
+            conv_mode = "llava_llama_3"
+        elif "qwen" in model_path.lower():
+            conv_mode = "qwen_1_5"
+        self.conv_template = conv_mode
+        self.conv_templates = conv_templates
+        self.tokenizer = tokenizer
+        self.model = model
+        self.image_processor = image_processor
+        self.tokenizer_image_token = tokenizer_image_token
+        self.KeywordStoppingCriteria = KeywordsStoppingCriteria
+        self.SeparatorStyle = SeparatorStyle
+
+    def generate_inner(self, message, dataset=None):
+        content, images = "", []
+        for msg in message:
+            if msg["type"] == "text":
+                content += msg["value"]
+            else:
+                images.append(Image.open(msg["value"]).convert("RGB"))
+                content += self.DEFAULT_IMAGE_TOKEN + "\n"
+
+        preprocess = self.image_processor.preprocess
+        image_tokenizer = self.tokenizer_image_token
+        image_tensor = [
+            preprocess(f, return_tensors="pt")["pixel_values"][0].half().cuda()
+            for f in images
+        ]
+        image_tensor = torch.stack(image_tensor)
+
+        conv = copy.deepcopy(self.conv_templates[self.conv_template])
+        conv.append_message(conv.roles[0], content)
+        conv.append_message(conv.roles[1], None)
+        prompt_question = conv.get_prompt()
+
+        input_ids = image_tokenizer(
+            prompt_question, self.tokenizer, self.IMAGE_TOKEN_INDEX, return_tensors="pt"
+        )
+        input_ids = input_ids.unsqueeze(0).cuda()
+
+        stop_str = conv.sep if conv.sep_style != self.SeparatorStyle.TWO else conv.sep2
+        keywords = [stop_str]
+        stopping_criteria = self.KeywordStoppingCriteria(
+            keywords, self.tokenizer, input_ids
+        )
+
+        cont = self.model.generate(
+            input_ids,
+            images=image_tensor,
+            do_sample=False,
+            temperature=0,
+            max_new_tokens=2048,
+            stopping_criteria=[stopping_criteria],
+        )
+        text_outputs = self.tokenizer.batch_decode(cont, skip_special_tokens=True)[0]
+        return text_outputs
+
+
+class LLaVA_OneVision(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = True
+    VIDEO_LLM = True
+    DEFAULT_IMAGE_TOKEN = "<image>"
+    IMAGE_TOKEN_INDEX = -200
+
+    def __init__(self, model_path="lmms-lab/llava-onevision-qwen2-7b-si", **kwargs):
+        assert model_path is not None
+        try:
+            from llava.conversation import SeparatorStyle, conv_templates
+            from llava.mm_utils import (KeywordsStoppingCriteria, get_model_name_from_path,
+                                        process_images, tokenizer_image_token)
+            from llava.model.builder import load_pretrained_model
+        except Exception as err:
+            logging.critical(
+                "Please `pip install git+https://github.com/LLaVA-VL/LLaVA-NeXT.git`"
+            )
+            raise err
+
+        video_kwargs_default = dict(
+            overwrite=True, mm_spatial_pool_mode="average", force_sample=True
+        )
+        video_kwargs_default.update(kwargs)
+        self.video_kwargs = video_kwargs_default
+
+        overwrite_config = None
+        if "video" in model_path.lower():
+            if self.video_kwargs["overwrite"]:
+                overwrite_config = {}
+                overwrite_config["mm_spatial_pool_mode"] = self.video_kwargs[
+                    "mm_spatial_pool_mode"
+                ]
+
+        model_name = get_model_name_from_path(model_path)
+        import warnings
+
+        # filter warning align with official code
+        warnings.filterwarnings("ignore")
+        tokenizer, model, image_processor, _ = load_pretrained_model(
+            model_path,
+            None,
+            model_name,
+            device_map="auto",
+            overwrite_config=overwrite_config,
+        )
+        model.eval()
+        model.tie_weights()
+
+        if "llava" in model_path.lower():
+            conv_mode = "qwen_1_5"
+        if 'llava-video' in model_path.lower():
+            self.nframe = 64
+        else:
+            self.nframe = 16
+            if "72b" in model_path.lower():
+                self.nframe = 32
+
+        if "video" in model_path.lower():
+            self.force_sample = self.video_kwargs["force_sample"]
+        else:
+            self.force_sample = False
+
+        self.conv_template = conv_mode
+        self.conv_templates = conv_templates
+        self.tokenizer = tokenizer
+        self.model = model
+        self.image_processor = image_processor
+        self.tokenizer_image_token = tokenizer_image_token
+        self.process_images = (
+            process_images  # Store process_images as a class attribute
+        )
+        self.KeywordStoppingCriteria = KeywordsStoppingCriteria
+        self.SeparatorStyle = SeparatorStyle
+
+    def generate_inner_image(self, message, dataset=None):
+        content, images = "", []
+        image_sizes = []  # Store image sizes
+
+        for msg in message:
+            if msg["type"] == "text":
+                content += msg["value"]
+            else:
+                img = Image.open(msg["value"]).convert("RGB")
+                images.append(img)
+                image_sizes.append(img.size)  # Store the size of each image
+                content += self.DEFAULT_IMAGE_TOKEN + "\n"
+
+        # Process images using the class attribute self.process_images
+        image_tensor = self.process_images(
+            images, self.image_processor, self.model.config
+        )
+        image_tensor = [
+            _image.to(dtype=torch.float16, device="cuda") for _image in image_tensor
+        ]
+
+        conv = copy.deepcopy(self.conv_templates[self.conv_template])
+        conv.append_message(conv.roles[0], content)
+        conv.append_message(conv.roles[1], None)
+        prompt_question = conv.get_prompt()
+
+        input_ids = self.tokenizer_image_token(
+            prompt_question, self.tokenizer, self.IMAGE_TOKEN_INDEX, return_tensors="pt"
+        )
+        input_ids = input_ids.unsqueeze(0).cuda()
+
+        stop_str = conv.sep if conv.sep_style != self.SeparatorStyle.TWO else conv.sep2
+        keywords = [stop_str]
+        stopping_criteria = self.KeywordStoppingCriteria(
+            keywords, self.tokenizer, input_ids
+        )
+
+        # Pass image sizes along with other parameters
+        cont = self.model.generate(
+            input_ids,
+            images=image_tensor,
+            image_sizes=image_sizes,  # Pass the image sizes here
+            do_sample=False,
+            temperature=0,
+            max_new_tokens=2048,
+            stopping_criteria=[stopping_criteria],
+        )
+        text_outputs = self.tokenizer.batch_decode(cont, skip_special_tokens=True)[0]
+        return text_outputs
+
+    def generate_inner_video(self, message, dataset=None):
+        content, text_content, visual_content, videos = "", "", "", []
+
+        for msg in message:
+            if msg["type"] == "text":
+                text_content += msg["value"]
+            else:
+                videos.append(msg["value"])
+                visual_content += self.DEFAULT_IMAGE_TOKEN + "\n"
+
+        if len(videos) > 1:
+            raise ValueError(
+                "LLaVA-OneVision does not support multiple videos as input."
+            )
+
+        video_frames, frame_time, video_time = self.load_video(
+            videos[0], self.nframe, 1, self.force_sample
+        )
+
+        time_instruciton = (
+            f"The video lasts for {video_time:.2f} seconds,"
+            f"and {len(video_frames[0])} frames are uniformly sampled from it."
+            f"These frames are located at {frame_time}."
+            f"Please answer the following questions related to this video.\n"
+        )
+
+        if self.force_sample:
+            content = visual_content + time_instruciton + text_content
+        else:
+            content = visual_content + text_content
+
+        image_tensors = []
+        frames = (
+            self.image_processor.preprocess(video_frames, return_tensors="pt")[
+                "pixel_values"
+            ]
+            .half()
+            .cuda()
+        )
+        image_tensors.append(frames)
+
+        conv = copy.deepcopy(self.conv_templates[self.conv_template])
+        conv.append_message(conv.roles[0], content)
+        conv.append_message(conv.roles[1], None)
+        prompt_question = conv.get_prompt()
+
+        input_ids = self.tokenizer_image_token(
+            prompt_question, self.tokenizer, self.IMAGE_TOKEN_INDEX, return_tensors="pt"
+        )
+        input_ids = input_ids.unsqueeze(0).cuda()
+        image_sizes = [frame.size for frame in video_frames]
+        modalities = ["video"] * len(video_frames)
+
+        stop_str = conv.sep if conv.sep_style != self.SeparatorStyle.TWO else conv.sep2
+        keywords = [stop_str]
+        stopping_criteria = self.KeywordStoppingCriteria(
+            keywords, self.tokenizer, input_ids
+        )
+
+        # Pass image sizes along with other parameters
+        cont = self.model.generate(
+            input_ids,
+            images=image_tensors,
+            image_sizes=image_sizes,  # Pass the image sizes here
+            do_sample=False,
+            temperature=0,
+            max_new_tokens=2048,
+            modalities=modalities,
+            stopping_criteria=[stopping_criteria],
+        )
+        text_outputs = self.tokenizer.batch_decode(cont, skip_special_tokens=True)[0]
+        return text_outputs
+
+    def load_video(self, video_path, max_frames_num, fps=1, force_sample=False):
+        from decord import VideoReader, cpu
+
+        if max_frames_num == 0:
+            return np.zeros((1, 336, 336, 3))
+        vr = VideoReader(video_path, ctx=cpu(0), num_threads=1)
+        total_frame_num = len(vr)
+        video_time = total_frame_num / vr.get_avg_fps()
+        fps = round(vr.get_avg_fps() / fps)
+        frame_idx = [i for i in range(0, len(vr), fps)]
+        frame_time = [i / fps for i in frame_idx]
+        if len(frame_idx) > max_frames_num or force_sample:
+            sample_fps = max_frames_num
+            uniform_sampled_frames = np.linspace(
+                0, total_frame_num - 1, sample_fps, dtype=int
+            )
+            frame_idx = uniform_sampled_frames.tolist()
+            frame_time = [i / vr.get_avg_fps() for i in frame_idx]
+        frame_time = ",".join([f"{i:.2f}s" for i in frame_time])
+        spare_frames = vr.get_batch(frame_idx).asnumpy()
+        # import pdb;pdb.set_trace()
+        return spare_frames, frame_time, video_time
+
+    def generate_inner(self, message, dataset=None):
+        if DATASET_MODALITY(dataset) == 'VIDEO' and 'megabench' not in dataset.lower():
+            return self.generate_inner_video(message, dataset)
+        else:
+            return self.generate_inner_image(message, dataset)
+
+
+class LLaVA_OneVision_HF(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = True
+    VIDEO_LLM = True
+    DEFAULT_IMAGE_TOKEN = "<image>"
+    IMAGE_TOKEN_INDEX = -200
+
+    def __init__(self, model_path="llava-hf/llava-onevision-qwen2-0.5b-ov-hf", **kwargs):
+        from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration
+        assert model_path is not None, "Model path must be provided."
+        self.model = LlavaOnevisionForConditionalGeneration.from_pretrained(
+            model_path, torch_dtype=torch.float16, low_cpu_mem_usage=True
+        ).to('cuda')
+        self.processor = AutoProcessor.from_pretrained(model_path)
+
+        self.video_kwargs = kwargs.get("video_kwargs", {})
+        self.force_sample = self.video_kwargs.get("force_sample", False)
+        self.nframe = kwargs.get("nframe", 8)
+        self.fps = 1
+        self.model_path = model_path
+
+    def generate_inner_image(self, message, dataset=None):
+        content, images = "", []
+        image_sizes = []
+
+        for msg in message:
+            if msg["type"] == "text":
+                content += msg["value"]
+            elif msg["type"] == "image":
+                img = Image.open(msg["value"]).convert("RGB")
+                images.append(img)
+                image_sizes.append(img.size)
+                content += self.DEFAULT_IMAGE_TOKEN + "\n"
+
+        conversation = [
+            {
+                "role": "user",
+                "content": [
+                    {"type": "text", "text": content},
+                ],
+            }
+        ]
+        prompt = self.processor.apply_chat_template(conversation, add_generation_prompt=True)
+        inputs = self.processor(images=images, text=prompt, return_tensors="pt").to('cuda', torch.float16)
+
+        output = self.model.generate(**inputs, max_new_tokens=2048)
+        return self.processor.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
+
+    def generate_inner_video(self, message, dataset=None):
+        content, text_content, visual_content, videos = "", "", "", []
+
+        for msg in message:
+            if msg["type"] == "text":
+                text_content += msg["value"]
+            elif msg["type"] == "video":
+                videos.append(msg["value"])
+                visual_content += self.DEFAULT_IMAGE_TOKEN + "\n"
+
+        if len(videos) > 1:
+            raise ValueError("LLaVA-OneVision does not support multiple videos as input.")
+
+        video_frames, frame_time, video_time = self.load_video(
+            videos[0], self.nframe, fps=1, force_sample=self.force_sample
+        )
+
+        time_instruction = (
+            f"The video lasts for {video_time:.2f} seconds, "
+            f"and {len(video_frames)} frames are uniformly sampled from it. "
+            f"These frames are located at {frame_time}. "
+            f"Please answer the following questions related to this video.\n"
+        )
+
+        content = visual_content + time_instruction + text_content
+        conversation = [
+            {
+                "role": "user",
+                "content": [{"type": "text", "text": content}, {"type": "video"}],
+            }
+        ]
+        prompt = self.processor.apply_chat_template(conversation, add_generation_prompt=True)
+
+        inputs = self.processor(videos=video_frames, text=prompt, return_tensors="pt").to('cuda', torch.float16)
+        output = self.model.generate(**inputs, max_new_tokens=2048)
+        return self.processor.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
+
+    def load_video(self, video_path, max_frames_num, fps=1, force_sample=False):
+        from decord import VideoReader, cpu
+
+        vr = VideoReader(video_path, ctx=cpu(0), num_threads=1)
+        total_frame_num = len(vr)
+        avg_fps = vr.get_avg_fps()
+
+        if avg_fps == 0:
+            raise ValueError(f"Video '{video_path}' has an average FPS of 0, which is invalid.")
+        if fps <= 0:
+            raise ValueError("FPS argument must be greater than 0.")
+
+        effective_fps = round(avg_fps / fps)
+        frame_idx = list(range(0, total_frame_num, effective_fps))
+        frame_time = [i / avg_fps for i in frame_idx]
+
+        if len(frame_idx) > max_frames_num or force_sample:
+            uniform_sampled_frames = np.linspace(0, total_frame_num - 1, max_frames_num, dtype=int)
+            frame_idx = uniform_sampled_frames.tolist()
+            frame_time = [i / avg_fps for i in frame_idx]
+
+        frame_time_str = ", ".join([f"{t:.2f}s" for t in frame_time])
+        video_frames = vr.get_batch(frame_idx).asnumpy()
+        video_time = total_frame_num / avg_fps
+
+        return video_frames, frame_time_str, video_time
+
+    def generate_inner(self, message, dataset=None):
+        if DATASET_MODALITY(dataset) == "VIDEO" and 'megabench' not in dataset.lower():
+            return self.generate_inner_video(message, dataset)
+        else:
+            return self.generate_inner_image(message, dataset)
+
+
+class LLaVA_OneVision_1_5(BaseModel):
+    INTERLEAVE = True
+    VIDEO_LLM = True
+
+    def __init__(self, model_path="lmms-lab/LLaVA-OneVision-1.5-8B-Instruct", **kwargs):
+        from transformers import AutoModelForCausalLM, AutoProcessor
+
+        assert model_path is not None, "Model path must be provided."
+        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+        self.model = AutoModelForCausalLM.from_pretrained(
+            model_path, torch_dtype='auto', trust_remote_code=True).to('cuda')
+        self.model = self.model.to(self.device)
+        self.processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
+
+        self.model_path = model_path
+        kwargs.setdefault('max_new_tokens', 4096)
+        self.img_size = kwargs.pop('img_size', -1)
+        self.kwargs = kwargs
+
+    def prepare_itlist(self, inputs):
+        assert np.all([isinstance(x, dict) for x in inputs])
+        has_images = np.sum([x['type'] == 'image' for x in inputs])
+        if has_images:
+            content_list = []
+            for msg in inputs:
+                if msg['type'] == 'text':
+                    content_list.append(dict(type='text', text=msg['value']))
+                elif msg['type'] == 'image':
+                    from PIL import Image
+                    img = Image.open(msg['value'])
+                    b64 = encode_image_to_base64(img, target_size=self.img_size)
+                    content_list.append(dict(type='image_url', image_url=f'data:image/jpeg;base64,{b64}'))
+        else:
+            assert all([x['type'] == 'text' for x in inputs])
+            text = '\n'.join([x['value'] for x in inputs])
+            content_list = [dict(type='text', text=text)]
+        return content_list
+
+    def prepare_inputs(self, inputs):
+        input_msgs = []
+        assert isinstance(inputs, list) and isinstance(inputs[0], dict)
+        assert np.all(['type' in x for x in inputs]) or np.all(['role' in x for x in inputs]), inputs
+        if 'role' in inputs[0]:
+            assert inputs[-1]['role'] == 'user', inputs[-1]
+            for item in inputs:
+                input_msgs.append(dict(role=item['role'], content=self.prepare_itlist(item['content'])))
+        else:
+            input_msgs.append(dict(role='user', content=self.prepare_itlist(inputs)))
+        return input_msgs
+
+    def generate_inner(self, inputs, dataset=None):
+        from qwen_vl_utils import process_vision_info
+
+        message = self.prepare_inputs(inputs)
+        text = self.processor.apply_chat_template(
+            message, tokenize=False, add_generation_prompt=True
+        )
+        image_inputs, video_inputs = process_vision_info(message)
+        inputs = self.processor(
+            text=[text],
+            images=image_inputs,
+            videos=video_inputs,
+            padding=True,
+            return_tensors="pt",
+            min_pixels=1280 * 28 * 28 // len(image_inputs),
+            max_pixels=32768 * 28 * 28 // len(image_inputs),
+        )
+        print(f'pixel_values.shape: {inputs["pixel_values"].shape}')
+        inputs = inputs.to(self.device)
+
+        # Inference: Generation of the output
+        generated_ids = self.model.generate(**inputs, **self.kwargs)
+        generated_ids_trimmed = [
+            out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+        ]
+        output_text = self.processor.batch_decode(
+            generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
+        )
+        return output_text
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/llava_xtuner.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/llava_xtuner.py
new file mode 100644
index 0000000000000000000000000000000000000000..f7535dbe0372aae49307005b359070d89d6a6679
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/llava/llava_xtuner.py
@@ -0,0 +1,241 @@
+import logging
+import os
+import os.path as osp
+import string
+
+import pandas as pd
+import torch
+from huggingface_hub import snapshot_download
+from PIL import Image
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import cn_string, get_cache_path
+from ..base import BaseModel
+
+
+class LLaVA_XTuner(BaseModel):
+
+    INSTALL_REQ = True
+    INTERLEAVE = False
+
+    def __init__(self,
+                 llava_path,
+                 llm_path=None,
+                 visual_encoder_path='openai/clip-vit-large-patch14-336',
+                 visual_select_layer=-2,
+                 prompt_template=None,
+                 stop_words=[],
+                 torch_dtype=torch.float16):
+        try:
+            from peft import PeftModel
+            from xtuner.utils import PROMPT_TEMPLATE, StopWordStoppingCriteria
+        except Exception as err:
+            logging.critical(
+                'Please install xtuner with `pip install -U xtuner` before '
+                'using LLaVA_XTuner')
+            raise err
+
+        from transformers import (AutoModel, AutoModelForCausalLM, AutoTokenizer,  # noqa
+                                  StoppingCriteriaList)
+
+        if not osp.isdir(llava_path):
+            cache_path = get_cache_path(llava_path)
+            if cache_path is not None:
+                llava_path = cache_path
+            else:
+                llava_path = snapshot_download(repo_id=llava_path)
+        assert osp.exists(llava_path) and osp.isdir(llava_path)
+
+        # build visual_encoder
+        if 'llm' in os.listdir(llava_path):
+            assert llm_path is None, (
+                "Please don't specify the `llm_path` since passed "
+                '`llava_path` contains a LLM!')
+            llm_path = osp.join(llava_path, 'llm')
+        else:
+            assert llm_path is not None, 'Please specify the `llm_path`!'
+
+        llm = AutoModelForCausalLM.from_pretrained(llm_path,
+                                                   trust_remote_code=True,
+                                                   torch_dtype=torch_dtype,
+                                                   device_map='cpu')
+        tokenizer = AutoTokenizer.from_pretrained(llm_path,
+                                                  trust_remote_code=True,
+                                                  encode_special_tokens=True)
+        print(f'Load LLM from {llm_path}')
+
+        # build visual_encoder
+        if 'visual_encoder' in os.listdir(llava_path):
+            assert visual_encoder_path is None, (
+                "Please don't specify the `visual_encoder_path` since passed "
+                '`llava_path` contains a visual encoder!')
+            visual_encoder_path = osp.join(llava_path, 'visual_encoder')
+        else:
+            assert visual_encoder_path is not None, (
+                'Please specify the `visual_encoder_path`!')
+
+        from transformers import CLIPImageProcessor, CLIPVisionModel
+        visual_encoder = CLIPVisionModel.from_pretrained(
+            visual_encoder_path, torch_dtype=torch_dtype, device_map='cpu')
+        image_processor = CLIPImageProcessor.from_pretrained(
+            visual_encoder_path)
+        print(f'Load visual_encoder from {visual_encoder_path}')
+
+        # load adapter
+        if 'llm_adapter' in os.listdir(llava_path):
+            adapter_path = osp.join(llava_path, 'llm_adapter')
+            llm = PeftModel.from_pretrained(llm,
+                                            adapter_path,
+                                            trust_remote_code=True,
+                                            device_map='cpu')
+            print(f'Load LLM adapter from {llava_path}')
+        if 'visual_encoder_adapter' in os.listdir(llava_path):
+            adapter_path = osp.join(llava_path, 'visual_encoder_adapter')
+            visual_encoder = PeftModel.from_pretrained(visual_encoder,
+                                                       adapter_path,
+                                                       trust_remote_code=True,
+                                                       device_map='cpu')
+            print(f'Load visual_encoder adapter from {llava_path}')
+
+        # build projector
+        projector_path = osp.join(llava_path, 'projector')
+        projector = AutoModel.from_pretrained(projector_path,
+                                              trust_remote_code=True,
+                                              torch_dtype=torch_dtype,
+                                              device_map='cpu')
+        print(f'Load projector from {llava_path}')
+
+        llm.eval()
+        visual_encoder.eval()
+        projector.eval()
+
+        self.llm = llm.cuda()
+        self.tokenizer = tokenizer
+        self.visual_encoder = visual_encoder.cuda()
+        self.image_processor = image_processor
+        self.projector = projector.cuda()
+        self.visual_select_layer = visual_select_layer
+        if prompt_template is not None:
+            # modified prompt template
+            if prompt_template == 'llama3_chat':
+                self.prompt_template = dict(
+                    SYSTEM=('<|start_header_id|>system<|end_header_id|>\n\n'
+                            '{system}<|eot_id|>'),
+                    INSTRUCTION=(
+                        '<|start_header_id|>user<|end_header_id|>\n\n{input}<|eot_id|>'
+                        '<|start_header_id|>assistant<|end_header_id|>\n\n'),
+                    SUFFIX='<|eot_id|>',
+                    SUFFIX_AS_EOS=True,
+                    STOP_WORDS=['<|eot_id|>'])
+            else:
+                self.prompt_template = PROMPT_TEMPLATE[prompt_template]
+            stop_words += self.prompt_template.get('STOP_WORDS', [])
+        else:
+            self.prompt_template = None
+
+        self.stop_criteria = StoppingCriteriaList()
+        for word in stop_words:
+            self.stop_criteria.append(
+                StopWordStoppingCriteria(self.tokenizer, word))
+
+    def build_gen_config(self, dataset):
+        from transformers import GenerationConfig
+        gen_kwargs = dict(max_new_tokens=512,
+                          do_sample=True,
+                          temperature=1,
+                          num_beams=5,
+                          eos_token_id=self.tokenizer.eos_token_id,
+                          pad_token_id=self.tokenizer.pad_token_id
+                          if self.tokenizer.pad_token_id is not None else
+                          self.tokenizer.eos_token_id)
+        # For single word generation
+        if (dataset is not None
+                and DATASET_TYPE(dataset) in ['MCQ', 'Y/N']):
+            gen_kwargs.update(
+                dict(max_new_tokens=5, do_sample=False, num_beams=1))
+        return GenerationConfig(**gen_kwargs)
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        question = line['question']
+        hint = line['hint'] if ('hint' in line
+                                and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            question = hint + '\n' + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f'\n{key}. {item}'
+
+        if not cn_string(question):
+            prompt = question + '\n' + ("Answer with the option's letter "
+                                        'from the given choices directly.')
+        else:
+            prompt = question + '\n' + '请直接回答选项字母。'
+
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        from xtuner.dataset.utils import expand2square
+        from xtuner.model.utils import prepare_inputs_labels_for_multimodal
+        from xtuner.utils import DEFAULT_IMAGE_TOKEN, IMAGE_TOKEN_INDEX
+        prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+        prompt = prompt.replace('<image>', '')
+        image = Image.open(image_path).convert('RGB')
+        image = expand2square(
+            image,
+            tuple(int(x * 255) for x in self.image_processor.image_mean))
+        image = self.image_processor.preprocess(
+            image, return_tensors='pt')['pixel_values'][0]
+        image = image.cuda().unsqueeze(0)
+        visual_outputs = self.visual_encoder(image, output_hidden_states=True)
+        pixel_values = self.projector(
+            visual_outputs.hidden_states[self.visual_select_layer][:, 1:])
+
+        inputs = DEFAULT_IMAGE_TOKEN + '\n' + prompt
+
+        if self.prompt_template:
+            inputs = self.prompt_template['INSTRUCTION'].format(input=inputs)
+
+        chunk_encode = []
+        for idx, chunk in enumerate(inputs.split(DEFAULT_IMAGE_TOKEN)):
+            if idx == 0:
+                cur_encode = self.tokenizer(chunk)
+            else:
+                cur_encode = self.tokenizer(chunk, add_special_tokens=False)
+            chunk_encode.append(cur_encode)
+        assert len(chunk_encode) == 2
+        ids = []
+        for idx, cur_chunk_encode in enumerate(chunk_encode):
+            ids.extend(cur_chunk_encode['input_ids'])
+            if idx != len(chunk_encode) - 1:
+                ids.append(IMAGE_TOKEN_INDEX)
+        ids = torch.tensor(ids).cuda().unsqueeze(0)
+        mm_inputs = prepare_inputs_labels_for_multimodal(
+            llm=self.llm, input_ids=ids, pixel_values=pixel_values)
+
+        gen_config = self.build_gen_config(dataset)
+        generate_output = self.llm.generate(
+            **mm_inputs,
+            generation_config=gen_config,
+            streamer=None,
+            bos_token_id=self.tokenizer.bos_token_id,
+            stopping_criteria=self.stop_criteria)
+        predict = self.tokenizer.decode(generate_output[0],
+                                        skip_special_tokens=True).strip()
+        return predict
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/blip2_instruct_vicuna13b.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/blip2_instruct_vicuna13b.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..a7cebe598616ab21908562301d44f2e4546ce0cf
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/blip2_instruct_vicuna13b.yaml
@@ -0,0 +1,43 @@
+ # Copyright (c) 2022, salesforce.com, inc.
+ # All rights reserved.
+ # SPDX-License-Identifier: BSD-3-Clause
+ # For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause
+
+model:
+  arch: instruct_vicuna13b
+  load_finetuned: False
+  load_pretrained: True
+
+  pretrained: "https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/InstructBLIP/instruct_blip_vicuna13b_trimmed.pth"
+  finetuned: ""
+
+  # vit encoder
+  image_size: 224
+  drop_path_rate: 0
+  use_grad_checkpoint: False
+  vit_precision: "fp16"
+  freeze_vit: True
+
+  # Q-Former
+  num_query_token: 32
+
+  # path to Vicuna checkpoint
+  llm_model: "Please set the path to your vicuna-13b-v1.1"
+
+  # generation configs
+  prompt: ""
+
+
+preprocess:
+    vis_processor:
+        train:
+          name: "blip2_image_train"
+          image_size: 224
+        eval:
+          name: "blip_image_eval"
+          image_size: 224
+    text_processor:
+        train:
+          name: "blip_caption"
+        eval:
+          name: "blip_caption"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/blip2_instruct_vicuna7b.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/blip2_instruct_vicuna7b.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..2a57a02ebeecfa5e345005456302925fb8f1c655
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/blip2_instruct_vicuna7b.yaml
@@ -0,0 +1,43 @@
+ # Copyright (c) 2022, salesforce.com, inc.
+ # All rights reserved.
+ # SPDX-License-Identifier: BSD-3-Clause
+ # For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause
+
+model:
+  arch: instruct_vicuna7b
+  load_finetuned: False
+  load_pretrained: True
+
+  pretrained: "https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/InstructBLIP/instruct_blip_vicuna7b_trimmed.pth"
+  finetuned: ""
+
+  # vit encoder
+  image_size: 224
+  drop_path_rate: 0
+  use_grad_checkpoint: False
+  vit_precision: "fp16"
+  freeze_vit: True
+
+  # Q-Former
+  num_query_token: 32
+
+  # path to Vicuna checkpoint
+  llm_model: "Please set the path to your vicuna-7b-v1.1"
+
+  # generation configs
+  prompt: ""
+
+
+preprocess:
+    vis_processor:
+        train:
+          name: "blip2_image_train"
+          image_size: 224
+        eval:
+          name: "blip_image_eval"
+          image_size: 224
+    text_processor:
+        train:
+          name: "blip_caption"
+        eval:
+          name: "blip_caption"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigpt4_13b_eval.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigpt4_13b_eval.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..ae3d08be11b36e0e78ef52274437877b1be7167c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigpt4_13b_eval.yaml
@@ -0,0 +1,37 @@
+model:
+  arch: minigpt4
+  model_type: pretrain_vicuna_7b
+  max_txt_len: 160
+  end_sym: "###"
+  low_resource: True
+  prompt_template: '###Human: {} ###Assistant: '
+  ckpt: "please set this value to the path of pretrained checkpoint"
+
+  # vit encoder
+  image_size: 224
+  drop_path_rate: 0
+  use_grad_checkpoint: False
+  vit_precision: "fp16"
+  freeze_vit: True
+  freeze_qformer: True
+
+  # Q-Former
+  num_query_token: 32
+
+  # generation configs
+  prompt: ""
+
+  llama_model: "please set this value to the path of vicuna-13b-v0"
+
+datasets:
+  cc_sbu_align:
+    vis_processor:
+      train:
+        name: "blip2_image_eval"
+        image_size: 224
+    text_processor:
+      train:
+        name: "blip_caption"
+
+run:
+  task: image_text_pretrain
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigpt4_7b_eval.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigpt4_7b_eval.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..062c93a9c3a1c56294824e2a2772ab3c3abe1932
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigpt4_7b_eval.yaml
@@ -0,0 +1,38 @@
+model:
+  arch: minigpt4
+  model_type: pretrain_vicuna_7b
+  max_txt_len: 160
+  end_sym: "###"
+  low_resource: True
+  prompt_template: '###Human: {} ###Assistant: '
+  ckpt: "please set this value to the path of pretrained checkpoint"
+
+  # vit encoder
+  image_size: 224
+  drop_path_rate: 0
+  use_grad_checkpoint: False
+  vit_precision: "fp16"
+  freeze_vit: True
+  freeze_qformer: True
+
+  # Q-Former
+  num_query_token: 32
+
+  # generation configs
+  prompt: ""
+
+  llama_model: "please set this value to the path of vicuna-7b-v0"
+
+
+datasets:
+  cc_sbu_align:
+    vis_processor:
+      train:
+        name: "blip2_image_eval"
+        image_size: 224
+    text_processor:
+      train:
+        name: "blip_caption"
+
+run:
+  task: image_text_pretrain
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigptv2_eval.yaml b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigptv2_eval.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..32815e1b9de21fbfcfc4ea459e95306fd9e42d95
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/misc/minigptv2_eval.yaml
@@ -0,0 +1,36 @@
+model:
+  arch: minigpt_v2
+  model_type: pretrain
+  max_txt_len: 160
+  end_sym: "</s>"
+  low_resource: True
+  prompt_template: '[INST] {} [/INST]'
+  ckpt: "please set this value to the path of pretrained checkpoint"
+  lora_r: 64
+  lora_alpha: 16
+
+  # vit encoder
+  image_size: 448
+  drop_path_rate: 0
+  use_grad_checkpoint: False
+  vit_precision: "fp16"
+  freeze_vit: True
+
+  # generation configs
+  prompt: ""
+
+  # LLM
+  llama_model: "please set this value to the path of llama2-chat-7b"
+
+datasets:
+  cc_sbu_align:
+    vis_processor:
+      train:
+        name: "blip2_image_eval"
+        image_size: 448
+    text_processor:
+      train:
+        name: "blip_caption"
+
+run:
+  task: image_text_pretrain
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..4d18f2e981ee64b83dc7dc51e278724c4c5e8222
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/__init__.py
@@ -0,0 +1 @@
+from .ola_model import Ola
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/arguments.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/arguments.py
new file mode 100644
index 0000000000000000000000000000000000000000..f8b3b953c00843714e85f915a45d0de148df87c9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/arguments.py
@@ -0,0 +1,65 @@
+from dataclasses import dataclass, field
+from typing import Optional
+
+import transformers
+
+
+@dataclass
+class ModelArguments:
+    model_name_or_path: Optional[str] = field(default="facebook/opt-125m")
+    version: Optional[str] = field(default="v0")
+    freeze_backbone: bool = field(default=False)
+    tune_speech_projector: bool = field(default=False)
+    tune_speech_encoder: bool = field(default=False)
+    tune_speech_generator_only: bool = field(default=False)
+    speech_encoder_type: Optional[str] = field(default=None)
+    speech_encoder: Optional[str] = field(default=None)
+    pretrain_speech_projector: Optional[str] = field(default=None)
+    speech_projector_type: Optional[str] = field(default='linear')
+    speech_encoder_ds_rate: int = 5
+    speech_encoder_hidden_size: int = 1280
+
+
+@dataclass
+class DataArguments:
+    data_path: str = field(default=None,
+                           metadata={"help": "Path to the training data."})
+    is_multimodal: bool = False
+    input_type: str = field(default="mel")
+    speech_normalize: bool = False
+    mel_size: int = 128
+    has_tgt_units: bool = False
+
+
+@dataclass
+class TrainingArguments(transformers.TrainingArguments):
+    cache_dir: Optional[str] = field(default=None)
+    optim: str = field(default="adamw_torch")
+    freeze_speech_projector: bool = field(default=False)
+    model_max_length: int = field(
+        default=512,
+        metadata={
+            "help":
+            "Maximum sequence length. Sequences will be right padded (and possibly truncated)."
+        },
+    )
+    double_quant: bool = field(
+        default=True,
+        metadata={"help": "Compress the quantization statistics through double quantization."}
+    )
+    quant_type: str = field(
+        default="nf4",
+        metadata={"help": "Quantization data type to use. Should be one of `fp4` or `nf4`."}
+    )
+    bits: int = field(
+        default=16,
+        metadata={"help": "How many bits to use."}
+    )
+    lora_enable: bool = False
+    lora_r: int = 64
+    lora_alpha: int = 16
+    lora_dropout: float = 0.05
+    lora_weight_path: str = ""
+    lora_bias: str = "none"
+    speech_projector_lr: Optional[float] = None
+    group_by_modality_length: bool = field(default=False)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/constants.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/constants.py
new file mode 100644
index 0000000000000000000000000000000000000000..0138cc937d15eb11ea6157ab3019a30f8a9c6b7b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/constants.py
@@ -0,0 +1,14 @@
+CONTROLLER_HEART_BEAT_EXPIRATION = 30
+WORKER_HEART_BEAT_INTERVAL = 15
+
+LOGDIR = "."
+
+# Model Constants
+IGNORE_INDEX = -100
+SPEECH_TOKEN_INDEX = -200
+DEFAULT_SPEECH_TOKEN = "<speech>"
+IMAGE_TOKEN_INDEX= -300
+DEFAULT_IMAGE_TOKEN = "<image>"
+DEFAULT_IMAGE_PATCH_TOKEN = "<im_patch>"
+DEFAULT_IM_START_TOKEN = "<im_start>"
+DEFAULT_IM_END_TOKEN = "<im_end>"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/conversation.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/conversation.py
new file mode 100644
index 0000000000000000000000000000000000000000..3b9168d8e6619cdd57a6f66e8c6ef58a0633db13
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/conversation.py
@@ -0,0 +1,138 @@
+import base64
+import dataclasses
+from enum import Enum, auto
+from io import BytesIO
+from typing import Any, List, Tuple, Union
+
+from PIL import Image
+
+
+class SeparatorStyle(Enum):
+    """Different separator style."""
+    TWO = auto()
+    PLAIN = auto()
+    CHATML = auto()
+    LLAMA_2 = auto()
+    LLAMA_3 = auto()
+    QWEN2 = auto()
+
+
+@dataclasses.dataclass
+class Conversation:
+    """A class that keeps all conversation history."""
+    system: str
+    roles: List[str]
+    messages: List[List[str]]
+    offset: int
+    sep_style: SeparatorStyle = SeparatorStyle.PLAIN
+    sep: str = "###"
+    sep2: str = None
+    version: str = "Unknown"
+
+    tokenizer_id: str = ""
+    tokenizer: Any = None
+    # Stop criteria (the default one is EOS token)
+    stop_str: Union[str, List[str]] = None
+    # Stops generation if meeting any token in this list
+    stop_token_ids: List[int] = None
+
+    skip_next: bool = False
+
+    def get_prompt(self):
+        messages = self.messages
+
+        if self.sep_style == SeparatorStyle.TWO:
+            seps = [self.sep, self.sep2]
+            ret = self.system + seps[0]
+            for i, (role, message) in enumerate(messages):
+                if message:
+                    if type(message) is tuple:
+                        message = message[0]
+                    ret += role + ": " + message + seps[i % 2]
+                else:
+                    ret += role + ":"
+        elif self.sep_style == SeparatorStyle.QWEN2:
+            start = '<|im_start|>'
+            end = '<|im_end|>\n'
+            ret = start + 'system\n' + self.system + end
+            for i, (role, message) in enumerate(messages):
+                if message:
+                    if type(message) is tuple:
+                        message, _, _ = message
+
+                    if message.endswith('<|endoftext|>'):
+                        message = message.replace('<|endoftext|>', '')
+                        ret += start + role + "\n" + message + end + '<|endoftext|>'
+                    else:
+                        assert not '<|endoftext|>' in message, f"Invalid message: {message}"
+                        ret += start + role + "\n" + message + end
+                else:
+                    ret += start + role + "\n"
+        else:
+            raise ValueError(f"Invalid style: {self.sep_style}")
+
+        return ret
+
+    def append_message(self, role, message):
+        self.messages.append([role, message])
+
+    def to_gradio_chatbot(self):
+        ret = []
+        for i, (role, msg) in enumerate(self.messages[self.offset:]):
+            if i % 2 == 0:
+                if type(msg) is tuple:
+                    msg, speech = msg
+                    ret.append([msg, None])
+                else:
+                    ret.append([msg, None])
+            else:
+                ret[-1][-1] = msg
+        return ret
+
+    def copy(self):
+        return Conversation(
+            system=self.system,
+            roles=self.roles,
+            messages=[[x, y] for x, y in self.messages],
+            offset=self.offset,
+            sep_style=self.sep_style,
+            sep=self.sep,
+            sep2=self.sep2,
+            version=self.version)
+
+    def dict(self):
+        if len(self.get_images()) > 0:
+            return {
+                "system": self.system,
+                "roles": self.roles,
+                "messages": [[x, y[0] if type(y) is tuple else y] for x, y in self.messages],
+                "offset": self.offset,
+                "sep": self.sep,
+                "sep2": self.sep2,
+            }
+        return {
+            "system": self.system,
+            "roles": self.roles,
+            "messages": self.messages,
+            "offset": self.offset,
+            "sep": self.sep,
+            "sep2": self.sep2,
+        }
+
+conv_qwen_v1 = Conversation(
+    system="You are a helpful assistant.",
+    roles=("user", "assistant"),
+    version="v1",
+    messages=(),
+    offset=0,
+    sep_style=SeparatorStyle.QWEN2,
+)
+
+default_conversation = conv_qwen_v1
+conv_templates = {
+    'v1_qwen2': conv_qwen_v1,
+}
+
+
+if __name__ == "__main__":
+    print(default_conversation.get_prompt())
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/datasets/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/datasets/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/datasets/preprocess.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/datasets/preprocess.py
new file mode 100644
index 0000000000000000000000000000000000000000..fd82bdf0ee02021e48cfb386d6db330a6db66df7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/datasets/preprocess.py
@@ -0,0 +1,218 @@
+import copy
+from typing import Dict, Sequence
+
+import tokenizers
+import torch
+import transformers
+from packaging import version
+
+from .. import conversation as conversation_lib
+from ..arguments import DataArguments
+from ..constants import DEFAULT_SPEECH_TOKEN, IGNORE_INDEX, IMAGE_TOKEN_INDEX, SPEECH_TOKEN_INDEX
+from ..model import OlaQwenForCausalLM
+
+IS_TOKENIZER_GREATER_THAN_0_14 = version.parse(tokenizers.__version__) >= version.parse('0.14')
+
+
+def tokenizer_speech_token(prompt, tokenizer, speech_token_index=SPEECH_TOKEN_INDEX, return_tensors=None):
+    prompt_chunks = [tokenizer(chunk).input_ids for chunk in prompt.split('<speech>')]
+
+    def insert_separator(X, sep):
+        return [ele for sublist in zip(X, [sep]*len(X)) for ele in sublist][:-1]
+
+    input_ids = []
+    offset = 0
+    if len(prompt_chunks) > 0 and len(prompt_chunks[0]) > 0 and prompt_chunks[0][0] == tokenizer.bos_token_id:
+        offset = 1
+        input_ids.append(prompt_chunks[0][0])
+
+    for x in insert_separator(prompt_chunks, [speech_token_index] * (offset + 1)):
+        input_ids.extend(x[offset:])
+
+    if return_tensors is not None:
+        if return_tensors == 'pt':
+            return torch.tensor(input_ids, dtype=torch.long)
+        raise ValueError(f'Unsupported tensor type: {return_tensors}')
+    return input_ids
+
+
+def preprocess_multimodal(
+    sources: Sequence[str],
+    data_args: DataArguments
+) -> Dict:
+    is_multimodal = data_args.is_multimodal
+    if not is_multimodal:
+        return sources
+
+    for source in sources:
+        for sentence in source:
+            if DEFAULT_SPEECH_TOKEN in sentence['value']:
+                sentence['value'] = sentence['value'].replace(DEFAULT_SPEECH_TOKEN, '').strip()
+                sentence['value'] = DEFAULT_SPEECH_TOKEN + '\n' + sentence['value']
+                sentence['value'] = sentence['value'].strip()
+
+    return sources
+
+def tokenizer_image_token(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None):
+    prompt_chunks = [tokenizer(chunk).input_ids for chunk in prompt.split('<image>')]
+
+    def insert_separator(X, sep):
+        return [ele for sublist in zip(X, [sep]*len(X)) for ele in sublist][:-1]
+
+    input_ids = []
+    offset = 0
+    if len(prompt_chunks) > 0 and len(prompt_chunks[0]) > 0 and prompt_chunks[0][0] == tokenizer.bos_token_id:
+        offset = 1
+        input_ids.append(prompt_chunks[0][0])
+
+    for x in insert_separator(prompt_chunks, [image_token_index] * (offset + 1)):
+        input_ids.extend(x[offset:])
+
+    if return_tensors is not None:
+        if return_tensors == 'pt':
+            return torch.tensor(input_ids, dtype=torch.long)
+        raise ValueError(f'Unsupported tensor type: {return_tensors}')
+    return input_ids
+
+def tokenizer_speech_image_token(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, speech_token_idx=SPEECH_TOKEN_INDEX, return_tensors=None):
+    prompt_chunks = [tokenizer(chunk).input_ids for chunk in prompt.split('<speech><image>')]
+
+    def insert_separator(X, sep):
+        return [ele for sublist in zip(X, [sep]*len(X)) for ele in sublist][:-1]
+
+    input_ids = []
+    offset = 0
+    if len(prompt_chunks) > 0 and len(prompt_chunks[0]) > 0 and prompt_chunks[0][0] == tokenizer.bos_token_id:
+        offset = 1
+        input_ids.append(prompt_chunks[0][0])
+
+    for x in insert_separator(prompt_chunks, [speech_token_idx, image_token_index] * (offset + 1)):
+        input_ids.extend(x[offset:])
+
+    if return_tensors is not None:
+        if return_tensors == 'pt':
+            return torch.tensor(input_ids, dtype=torch.long)
+        raise ValueError(f'Unsupported tensor type: {return_tensors}')
+    return input_ids
+
+def tokenizer_speech_question_image_token(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, speech_token_idx=SPEECH_TOKEN_INDEX, return_tensors=None):
+    prompt_chunks = [tokenizer(chunk).input_ids for chunk in prompt.split("<image>\nUser's question in speech: <speech>\n")]
+
+    def insert_separator(X, sep):
+        return [ele for sublist in zip(X, [sep]*len(X)) for ele in sublist][:-1]
+
+    input_ids = []
+    offset = 0
+    if len(prompt_chunks) > 0 and len(prompt_chunks[0]) > 0 and prompt_chunks[0][0] == tokenizer.bos_token_id:
+        offset = 1
+        input_ids.append(prompt_chunks[0][0])
+
+    nl_tokens = tokenizer("\n").input_ids[0]
+    special_chunks = [image_token_index, nl_tokens]
+    special_chunks.extend(tokenizer("User's question in speech: ").input_ids)
+    special_chunks.extend([speech_token_idx, nl_tokens])
+
+    for x in insert_separator(prompt_chunks, special_chunks):
+        input_ids.extend(x[offset:])
+
+    import pdb;pdb.set_trace()
+    if return_tensors is not None:
+        if return_tensors == 'pt':
+            return torch.tensor(input_ids, dtype=torch.long)
+        raise ValueError(f'Unsupported tensor type: {return_tensors}')
+    return input_ids
+
+def preprocess_v1(
+    sources,
+    tokenizer: transformers.PreTrainedTokenizer,
+    has_speech: bool = False
+) -> Dict:
+    conv = conversation_lib.default_conversation.copy()
+    roles = {"human": conv.roles[0], "gpt": conv.roles[1]}
+
+    # Apply prompt templates
+    conversations = []
+    for i, source in enumerate(sources):
+        if roles[source[0]["from"]] != conv.roles[0]:
+            # Skip the first one if it is not from human
+            source = source[1:]
+
+        conv.messages = []
+        for j, sentence in enumerate(source):
+            role = roles[sentence["from"]]
+            assert role == conv.roles[j % 2], f"{i}"
+            conv.append_message(role, sentence["value"])
+        conversations.append(conv.get_prompt())
+
+    # Tokenize conversations
+
+    if has_speech:
+        input_ids = torch.stack([tokenizer_speech_token(prompt, tokenizer, return_tensors='pt') for prompt in conversations], dim=0)
+    else:
+        input_ids = tokenizer(
+            conversations,
+            return_tensors="pt",
+            padding="longest",
+            max_length=tokenizer.model_max_length,
+            truncation=True,
+        ).input_ids
+
+    targets = input_ids.clone()
+
+    assert conv.sep_style == conversation_lib.SeparatorStyle.TWO
+
+    # Mask targets
+    sep = conv.sep + conv.roles[1] + ": "
+    for conversation, target in zip(conversations, targets):
+        total_len = int(target.ne(tokenizer.pad_token_id).sum())
+
+        rounds = conversation.split(conv.sep2)
+        cur_len = 1
+        target[:cur_len] = IGNORE_INDEX
+        for i, rou in enumerate(rounds):
+            if rou == "":
+                break
+
+            parts = rou.split(sep)
+            if len(parts) != 2:
+                break
+            parts[0] += sep
+
+            if has_speech:
+                round_len = len(tokenizer_speech_token(rou, tokenizer))
+                instruction_len = len(tokenizer_speech_token(parts[0], tokenizer)) - 2
+            else:
+                round_len = len(tokenizer(rou).input_ids)
+                instruction_len = len(tokenizer(parts[0]).input_ids) - 2
+
+            # FIXME: tokenizer bug
+            if i != 0 and not tokenizer.legacy and IS_TOKENIZER_GREATER_THAN_0_14:
+                round_len -= 1
+                instruction_len -= 1
+
+            target[cur_len : cur_len + instruction_len] = IGNORE_INDEX
+
+            cur_len += round_len
+        target[cur_len:] = IGNORE_INDEX
+
+        if cur_len < tokenizer.model_max_length:
+            if cur_len != total_len:
+                target[:] = IGNORE_INDEX
+                print(
+                    f"WARNING: tokenization mismatch: {cur_len} vs. {total_len}."
+                    f" (ignored)"
+                )
+
+    return dict(
+        input_ids=input_ids,
+        labels=targets,
+    )
+
+def preprocess(
+    sources: Sequence[str],
+    tokenizer: transformers.PreTrainedTokenizer,
+    has_speech: bool = False
+) -> Dict:
+    if conversation_lib.default_conversation.version.startswith("v1"):
+        return preprocess_v1(sources, tokenizer, has_speech=has_speech)
+    raise NotImplementedError
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/mm_utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/mm_utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..0bd7de3f2e78a4441e5f7ccc5e34957fcc63ea61
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/mm_utils.py
@@ -0,0 +1,177 @@
+import ast
+import base64
+import io
+import math
+import os
+
+import torch
+from PIL import Image
+from transformers import StoppingCriteria
+
+if 'HIGHRES_BASE' in os.environ:
+    # highresxpatch
+    HIGHRES_BASE = os.environ['HIGHRES_BASE']
+    highres_base, highres_ps = HIGHRES_BASE.split('x')
+    highres_base = int(highres_base)
+    highres_ps = int(highres_ps)
+    print(f"HIGHRES_BASE is set as {HIGHRES_BASE}, {highres_base}, {highres_ps}")
+else:
+    HIGHRES_BASE = None
+
+if 'MAXRES' in os.environ:
+    # highresxpatch
+    MAXRES = int(os.environ['MAXRES'])
+    print(f"MAXRES is set as {MAXRES}")
+else:
+    MAXRES = 1536
+
+if 'MINRES' in os.environ:
+    # highresxpatch
+    MINRES = int(os.environ['MINRES'])
+    print(f"MINRES is set as {MINRES}")
+else:
+    MINRES = 0
+
+if 'PAD2STRIDE' in os.environ:
+    # highresxpatch
+    PAD2STRIDE = True
+    print(f"PAD2STRIDE is set")
+else:
+    PAD2STRIDE = False
+
+if 'LOWRES_RESIZE' in os.environ:
+    LOWRES_RESIZE = os.environ['LOWRES_RESIZE']
+    print(f"LOWRES_RESIZE is set as {LOWRES_RESIZE}")
+    if 'x' in LOWRES_RESIZE:
+        size, ps = LOWRES_RESIZE.split('x')
+        size = int(size)
+        ps = int(ps)
+        LOWRES_RESIZE = (size, ps)
+    else:
+        LOWRES_RESIZE = int(LOWRES_RESIZE)
+else:
+    LOWRES_RESIZE = None
+
+
+def pad_image(image, target_resolution, value=0):
+    """
+    Resize and pad an image to a target resolution while maintaining aspect ratio.
+
+    Args:
+        image (PIL.Image.Image): The input image.
+        target_resolution (tuple): The target resolution (width, height) of the image.
+
+    Returns:
+        PIL.Image.Image: The resized and padded image.
+    """
+    original_width, original_height = image.size
+    target_width, target_height = target_resolution
+    # Create a new image with the target size and paste the resized image onto it
+    new_image = Image.new('RGB', (target_width, target_height), (value, value, value))
+    paste_x = (target_width - original_width) // 2
+    paste_y = (target_height - original_height) // 2
+    new_image.paste(image, (paste_x, paste_y))
+    return new_image
+
+def resize_images(image, patch_size=14, base_size=896):
+    h, w = image.size
+    if base_size == 0:
+        if h * w > MAXRES * MAXRES:
+            scale = MAXRES * MAXRES / (h * w)
+            scale = math.sqrt(scale)
+        elif h * w < MINRES * MINRES:
+            scale = MINRES * MINRES / (h * w)
+            scale = math.sqrt(scale)
+        else:
+            scale = None
+    else:
+        scale = base_size * base_size / (h * w)
+        scale = math.sqrt(scale)
+
+
+    if scale is not None:
+        new_h = int(h * scale / patch_size) * patch_size
+        new_w = int(w * scale / patch_size) * patch_size
+        new_h = max(new_h, patch_size)
+        new_w = max(new_w, patch_size)
+        image = image.resize((new_h, new_w))
+    elif PAD2STRIDE:
+        if h % patch_size == 0:
+            new_h = h
+        else:
+            new_h = (h // patch_size + 1) * patch_size
+
+        if w % patch_size == 0:
+            new_w = w
+        else:
+            new_w = (w // patch_size + 1) * patch_size
+        image = pad_image(image, (new_h, new_w), value=127)
+    else:
+        scale = 1.0
+        new_h = int(h * scale / patch_size) * patch_size
+        new_w = int(w * scale / patch_size) * patch_size
+        new_h = max(new_h, patch_size)
+        new_w = max(new_w, patch_size)
+        image = image.resize((new_h, new_w))
+
+    return image
+
+def process_anyres_highres_image_genli(image, processor):
+    h, w = image.size
+    if h < 32 and w < 32:
+        min_size = min(h, w)
+        ratio = 64 / min_size
+        image = image.resize((int(h * ratio), int(w * ratio)))
+    elif h < 32:
+        ratio = 64 / h
+        image = image.resize((int(h * ratio), int(w * ratio)))
+    elif w < 32:
+        ratio = 64 / w
+        image = image.resize((int(h * ratio), int(w * ratio)))
+    if HIGHRES_BASE is not None:
+        image = resize_images(image, patch_size=highres_ps, base_size=highres_base)
+
+    if LOWRES_RESIZE is not None:
+        image_original_resize = resize_images(image, patch_size=LOWRES_RESIZE[1], base_size=LOWRES_RESIZE[0])
+    else:
+        image_original_resize = image.resize((384, 384))
+
+    image_patches = processor.preprocess(image_original_resize, return_tensors='pt')['pixel_values'][0]
+    image_padded = processor.preprocess(image, return_tensors='pt')['pixel_values'][0]
+    return image_patches.unsqueeze(0), image_padded.unsqueeze(0)
+
+
+
+def get_model_name_from_path(model_path):
+    model_path = model_path.strip("/")
+    model_paths = model_path.split("/")
+    if model_paths[-1].startswith('checkpoint-'):
+        return model_paths[-2] + "_" + model_paths[-1]
+    else:
+        return model_paths[-1]
+
+
+class KeywordsStoppingCriteria(StoppingCriteria):
+    def __init__(self, keywords, tokenizer, input_ids):
+        self.keywords = keywords
+        self.keyword_ids = []
+        for keyword in keywords:
+            cur_keyword_ids = tokenizer(keyword).input_ids
+            if len(cur_keyword_ids) > 1 and cur_keyword_ids[0] == tokenizer.bos_token_id:
+                cur_keyword_ids = cur_keyword_ids[1:]
+            self.keyword_ids.append(torch.tensor(cur_keyword_ids))
+        self.tokenizer = tokenizer
+        self.start_len = input_ids.shape[1]
+
+    def __call__(self, output_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
+        assert output_ids.shape[0] == 1, "Only support batch size 1 (yet)"  # TODO
+        offset = min(output_ids.shape[1] - self.start_len, 3)
+        self.keyword_ids = [keyword_id.to(output_ids.device) for keyword_id in self.keyword_ids]
+        for keyword_id in self.keyword_ids:
+            if output_ids[0, -keyword_id.shape[0]:] == keyword_id:
+                return True
+        outputs = self.tokenizer.batch_decode(output_ids[:, -offset:], skip_special_tokens=True)[0]
+        for keyword in self.keywords:
+            if keyword in outputs:
+                return True
+        return False
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..9d048652b2781cc66457fe3df4d98569edbec35b
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/__init__.py
@@ -0,0 +1 @@
+from .language_model.ola_qwen import OlaConfigQwen, OlaQwenForCausalLM
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..06bd5961f124414711738e854fbc70e1783ef155
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/builder.py
@@ -0,0 +1,93 @@
+import os
+import shutil
+import warnings
+
+import torch
+from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+from ..model import OlaQwenForCausalLM
+from ..model.speech_encoder.builder import build_speech_encoder
+
+
+def load_pretrained_model(model_path, model_base, is_lora=False, s2s=False, load_8bit=False, load_4bit=False, device="cuda", use_flash_attn=False, **kwargs):
+    if load_8bit:
+        kwargs['load_in_8bit'] = True
+    elif load_4bit:
+        kwargs['load_in_4bit'] = True
+        kwargs['quantization_config'] = BitsAndBytesConfig(
+            load_in_4bit=True,
+            bnb_4bit_compute_dtype=torch.float16,
+            bnb_4bit_use_double_quant=True,
+            bnb_4bit_quant_type='nf4'
+        )
+    else:
+        kwargs['torch_dtype'] = torch.bfloat16
+
+    if use_flash_attn:
+        kwargs['attn_implementation'] = 'flash_attention_2'
+
+    model_cls = OlaQwenForCausalLM
+
+    # Load OmniSpeech model
+    if is_lora:
+        assert model_base is not None, "model_base is required for LoRA models."
+        from ola.model.language_model.ola_qwen import OlaConfigQwen
+        lora_cfg_pretrained = OlaConfigQwen.from_pretrained(model_path)
+        tokenizer = AutoTokenizer.from_pretrained(model_base, use_fast=False)
+        print('Loading OmniSpeech from base model...')
+        model = model_cls.from_pretrained(model_base, low_cpu_mem_usage=False, config=lora_cfg_pretrained, **kwargs)
+        print('Loading additional OmniSpeech weights...')
+        if os.path.exists(os.path.join(model_path, 'non_lora_trainables.bin')):
+            non_lora_trainables = torch.load(os.path.join(model_path, 'non_lora_trainables.bin'), map_location='cpu')
+        non_lora_trainables = {(k[11:] if k.startswith('base_model.') else k): v for k, v in non_lora_trainables.items()}
+        if any(k.startswith('model.model.') for k in non_lora_trainables):
+            non_lora_trainables = {(k[6:] if k.startswith('model.') else k): v for k, v in non_lora_trainables.items()}
+        model.load_state_dict(non_lora_trainables, strict=False)
+
+        from peft import PeftModel
+        print('Loading LoRA weights...')
+        model = PeftModel.from_pretrained(model, model_path)
+        print('Merging LoRA weights...')
+        model = model.merge_and_unload()
+        print('Model is loaded...')
+    elif model_base is not None:
+        print('Loading OmniSpeech from base model...')
+        tokenizer = AutoTokenizer.from_pretrained(model_base, use_fast=False)
+        cfg_pretrained = AutoConfig.from_pretrained(model_path)
+        model = model_cls.from_pretrained(model_base, low_cpu_mem_usage=False, config=cfg_pretrained, **kwargs)
+
+        speech_projector_weights = torch.load(os.path.join(model_path, 'speech_projector.bin'), map_location='cpu')
+        speech_projector_weights = {k: v.to(torch.float16) for k, v in speech_projector_weights.items()}
+        model.load_state_dict(speech_projector_weights, strict=False)
+        model = model.to(device=device)
+    else:
+        tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
+        model = model_cls.from_pretrained(
+            model_path,
+            low_cpu_mem_usage=False,
+            **kwargs
+        )
+        model = model.to(device=device)
+
+    model.get_model().speech_encoder = build_speech_encoder(model.config)
+    model.get_model().speech_encoder.to(device=device, dtype=torch.float16)
+
+    image_processor = None
+    model.resize_token_embeddings(len(tokenizer))
+    vision_tower = model.get_vision_tower()
+    print("Loading vision tower...")
+    if not vision_tower.is_loaded:
+        vision_tower.load_model(device_map=device)
+    if device != "auto":
+        vision_tower.to(device="cuda", dtype=torch.bfloat16)
+    else:
+        vision_tower.to(device="cuda:0", dtype=torch.bfloat16)
+    image_processor = vision_tower.image_processor
+    print("Loading vision tower succeeded.")
+
+    if hasattr(model.config, "max_sequence_length"):
+        context_len = model.config.max_sequence_length
+    else:
+        context_len = 16384
+
+    return tokenizer, model, image_processor, context_len
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/language_model/ola_qwen.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/language_model/ola_qwen.py
new file mode 100644
index 0000000000000000000000000000000000000000..4cc8543e0e78eef50400302f79542b8c02485429
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/language_model/ola_qwen.py
@@ -0,0 +1,234 @@
+from typing import List, Optional, Tuple, Union
+
+import torch
+import torch.nn as nn
+import transformers
+from transformers import (AutoConfig, AutoModelForCausalLM, Qwen2Config, Qwen2ForCausalLM,
+                          Qwen2Model)
+from transformers.generation.utils import GenerateOutput
+from transformers.modeling_outputs import CausalLMOutputWithPast
+
+from ..ola_arch import OlaMetaForCausalLM, OlaMetaModel
+
+
+class OlaConfigQwen(Qwen2Config):
+    model_type = "ola_qwen"
+
+
+class OlaQwenModel(OlaMetaModel, Qwen2Model):
+    config_class = OlaConfigQwen
+
+    def __init__(self, config: Qwen2Config):
+        super(OlaQwenModel, self).__init__(config)
+
+
+class OlaQwenForCausalLM(Qwen2ForCausalLM, OlaMetaForCausalLM):
+    config_class = OlaConfigQwen
+
+    def __init__(self, config):
+        super(Qwen2ForCausalLM, self).__init__(config)
+
+        config.rope_scaling = None
+        self.model = OlaQwenModel(config)
+        self.vocab_size = config.vocab_size
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    def get_model(self):
+        return self.model
+
+    def forward(
+        self,
+        input_ids: torch.LongTensor = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[List[torch.FloatTensor]] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        labels: Optional[torch.LongTensor] = None,
+        use_cache: Optional[bool] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        speech: Optional[torch.FloatTensor] = None,
+        speech_lengths: Optional[torch.LongTensor] = None,
+        speech_chunks: Optional[torch.LongTensor] = None,
+        speech_wav: Optional[torch.FloatTensor] = None,
+        images: Optional[torch.FloatTensor] = None,
+        images_highres: Optional[List[torch.FloatTensor]] = None,
+        image_sizes: Optional[List[List[int]]] = None,
+        modalities: Optional[List[str]] = ["image"],
+        return_dict: Optional[bool] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+    ) -> Union[Tuple, CausalLMOutputWithPast]:
+
+        if inputs_embeds is None:
+            (
+                input_ids,
+                position_ids,
+                attention_mask,
+                past_key_values,
+                inputs_embeds,
+                labels
+            ) = self.prepare_inputs_labels_for_speech_vision_text(
+                input_ids,
+                position_ids,
+                attention_mask,
+                past_key_values,
+                labels,
+                speech,
+                speech_lengths,
+                speech_chunks,
+                speech_wav,
+                images,
+                modalities,
+                image_sizes,
+                images_highres
+            )
+
+        if labels is None:
+            return super().forward(
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                position_ids=position_ids,
+                past_key_values=past_key_values,
+                inputs_embeds=inputs_embeds,
+                use_cache=use_cache,
+                output_attentions=output_attentions,
+                output_hidden_states=output_hidden_states,
+                return_dict=return_dict
+            )
+        else:
+            return self.forward_llm_efficient(
+                input_ids=input_ids,
+                attention_mask=attention_mask,
+                position_ids=position_ids,
+                past_key_values=past_key_values,
+                inputs_embeds=inputs_embeds,
+                labels=labels,
+                use_cache=use_cache,
+                output_attentions=output_attentions,
+                output_hidden_states=output_hidden_states,
+                return_dict=return_dict
+            )
+
+
+    def forward_llm_efficient(self, input_ids, attention_mask, position_ids, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict):
+        output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
+        output_hidden_states = (
+            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
+        )
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
+        outputs = self.model(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            use_cache=use_cache,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+
+        hidden_states = outputs[0]
+        hidden_dim = hidden_states.size(-1)
+        shift_labels = labels[..., 1:].contiguous().reshape(-1)
+        shift_hidden_states = hidden_states[..., :-1, :].contiguous().reshape(-1, hidden_dim)
+        assert shift_labels.size(0) == shift_hidden_states.size(0)
+        mask = shift_labels > -1
+        assert mask.float().sum() > 0
+        shift_labels = shift_labels[mask]
+        shift_hidden_states = shift_hidden_states[mask, :]
+        logits = self.lm_head(shift_hidden_states)
+        logits = logits.float()
+        loss_fct = nn.CrossEntropyLoss()
+        loss = loss_fct(logits, shift_labels)
+
+
+        if not return_dict:
+            output = (logits,) + outputs[1:]
+            return (loss,) + output if loss is not None else output
+
+
+        return CausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+        )
+
+    @torch.no_grad()
+    def generate(
+        self,
+        inputs: Optional[torch.Tensor] = None,
+        speech: Optional[torch.Tensor] = None,
+        speech_lengths: Optional[torch.Tensor] = None,
+        speech_chunks: Optional[torch.Tensor] = None,
+        speech_wav: Optional[torch.FloatTensor] = None,
+        images: Optional[torch.Tensor] = None,
+        images_highres: Optional[List[torch.FloatTensor]] = None,
+        image_sizes: Optional[torch.Tensor] = None,
+        modalities: Optional[List[str]] = ["image"],
+        **kwargs,
+    ) -> Union[GenerateOutput, torch.LongTensor]:
+        position_ids = kwargs.pop("position_ids", None)
+        attention_mask = kwargs.pop("attention_mask", None)
+        if "inputs_embeds" in kwargs:
+            raise NotImplementedError("`inputs_embeds` is not supported")
+
+        (
+            inputs,
+            position_ids,
+            attention_mask,
+            _,
+            inputs_embeds,
+            _
+        ) = self.prepare_inputs_labels_for_speech_vision_text(
+            inputs,
+            position_ids,
+            attention_mask,
+            None,
+            None,
+            speech,
+            speech_lengths,
+            speech_chunks,
+            speech_wav,
+            images,
+            modalities,
+            image_sizes,
+            images_highres
+        )
+
+        return super().generate(
+            position_ids=position_ids,
+            attention_mask=attention_mask,
+            inputs_embeds=inputs_embeds,
+            **kwargs
+        )
+
+    def prepare_inputs_for_generation(self, input_ids, past_key_values=None,
+                                      inputs_embeds=None, **kwargs):
+        speech = kwargs.pop("speech", None)
+        speech_lengths = kwargs.pop("speech_lengths", None)
+        speech_chunks = kwargs.pop("speech_chunks", None)
+        images = kwargs.pop("images", None)
+        image_sizes = kwargs.pop("image_sizes", None)
+        inputs = super().prepare_inputs_for_generation(
+            input_ids, past_key_values=past_key_values, inputs_embeds=inputs_embeds, **kwargs
+        )
+        if speech is not None:
+            inputs['speech'] = speech
+            inputs['speech_lengths'] = speech_lengths
+            inputs['speech_chunks'] = speech_chunks
+        if images is not None:
+            inputs["images"] = images
+        if image_sizes is not None:
+            inputs["image_sizes"] = image_sizes
+        return inputs
+
+AutoConfig.register("ola_qwen", OlaConfigQwen)
+AutoModelForCausalLM.register(OlaConfigQwen, OlaQwenForCausalLM)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_encoder/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_encoder/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..ac6baa411a587e55fa03f07912d5f950285dd90e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_encoder/builder.py
@@ -0,0 +1,11 @@
+import os
+
+from .oryx_vit import SigLIPViTAnysizeWrapper
+
+
+def build_vision_tower(vision_tower_cfg, **kwargs):
+    vision_tower = getattr(vision_tower_cfg, 'vision_tower', getattr(vision_tower_cfg, 'mm_vision_tower', None))
+    is_absolute_path_exists = os.path.exists(vision_tower)
+    print(f"Buiding OryxViTWrapper from {vision_tower}...")
+    # path = vision_tower.split(":")[1]
+    return SigLIPViTAnysizeWrapper(vision_tower, path=vision_tower, args=vision_tower_cfg, **kwargs)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py
new file mode 100644
index 0000000000000000000000000000000000000000..0ba3a87b4e54dc1076da1c8bd0219af611a3b5dc
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py
@@ -0,0 +1,1024 @@
+import math
+import warnings
+from dataclasses import dataclass
+from functools import partial
+from typing import (Callable, Dict, Final, List, Literal, Optional, Sequence, Set, Tuple, Type,
+                    Union)
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torch.utils.checkpoint import checkpoint
+
+try:
+    from timm.layers import (AttentionPoolLatent, DropPath, LayerType, Mlp, PatchDropout,
+                             PatchEmbed, resample_abs_pos_embed)
+    from timm.models._manipulate import checkpoint_seq, named_apply
+except:
+    print('Wrong timm version')
+
+import logging
+import os
+from typing import Optional
+
+import deepspeed
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from flash_attn import flash_attn_func, flash_attn_varlen_func
+
+if 'LOAD_VISION_EARLY' in os.environ:
+    print("LOAD_VISION_EARLY is set")
+    LOAD_VISION_EARLY = True
+else:
+    LOAD_VISION_EARLY = False
+
+if 'FORCE_NO_DOWNSAMPLE' in os.environ:
+    print("FORCE_NO_DOWNSAMPLE is set")
+    FORCE_NO_DOWNSAMPLE = True
+else:
+    FORCE_NO_DOWNSAMPLE = False
+
+def _no_grad_trunc_normal_(tensor, mean, std, a, b):
+    # Cut & paste from PyTorch official master until it's in a few official releases - RW
+    # Method based on https://people.sc.fsu.edu/~jburkardt/presentations/truncated_normal.pdf
+    def norm_cdf(x):
+        # Computes standard normal cumulative distribution function
+        return (1.0 + math.erf(x / math.sqrt(2.0))) / 2.0
+
+    if (mean < a - 2 * std) or (mean > b + 2 * std):
+        warnings.warn(
+            "mean is more than 2 std from [a, b] in nn.init.trunc_normal_. "
+            "The distribution of values may be incorrect.",
+            stacklevel=2,
+        )
+
+    with torch.no_grad():
+        # Values are generated by using a truncated uniform distribution and
+        # then using the inverse CDF for the normal distribution.
+        # Get upper and lower cdf values
+        l = norm_cdf((a - mean) / std)  # noqa: E741
+        u = norm_cdf((b - mean) / std)
+
+        # Uniformly fill tensor with values from [l, u], then translate to
+        # [2l-1, 2u-1].
+        tensor.uniform_(2 * l - 1, 2 * u - 1)
+
+        # Use inverse cdf transform for normal distribution to get truncated
+        # standard normal
+        tensor.erfinv_()
+
+        # Transform to proper mean, std
+        tensor.mul_(std * math.sqrt(2.0))
+        tensor.add_(mean)
+
+        # Clamp to ensure it's in the proper range
+        tensor.clamp_(min=a, max=b)
+        return tensor
+
+
+def trunc_normal_(tensor, mean=0.0, std=1.0, a=-2.0, b=2.0):
+    # type: (torch.Tensor, float, float, float, float) -> torch.Tensor
+    r"""The original timm.models.layers.weight_init.trunc_normal_ can not handle bfloat16 yet, here we first
+    convert the tensor to float32, apply the trunc_normal_() in float32, and then convert it back to its orignal dtype.
+    Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn
+    from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)`
+    with values outside :math:`[a, b]` redrawn until they are within
+    the bounds. The method used for generating the random values works
+    best when :math:`a \leq \text{mean} \leq b`.
+    Args:
+        tensor: an n-dimensional `torch.Tensor`
+        mean: the mean of the normal distribution
+        std: the standard deviation of the normal distribution
+        a: the minimum cutoff value
+        b: the maximum cutoff value
+    Examples:
+        >>> w = torch.empty(3, 5)
+        >>> nn.init.trunc_normal_(w)
+    """
+
+    with torch.no_grad():
+        dtype = tensor.dtype
+        tensor_fp32 = tensor.float()
+        tensor_fp32 = _no_grad_trunc_normal_(tensor_fp32, mean, std, a, b)
+        tensor_dtype = tensor_fp32.to(dtype=dtype)
+        tensor.copy_(tensor_dtype)
+
+
+def init_weights(self):
+    if self.pos_embed is not None:
+        trunc_normal_(self.pos_embed, std=self.pos_embed.shape[1] ** -0.5)
+    trunc_normal_(self.latent, std=self.latent_dim**-0.5)
+
+
+def init_weights_vit_timm(module: nn.Module, name: str = "") -> None:
+    """ViT weight initialization, original timm impl (for reproducibility)"""
+    if isinstance(module, nn.Linear):
+        trunc_normal_(module.weight, std=0.02)
+        if module.bias is not None:
+            nn.init.zeros_(module.bias)
+    elif hasattr(module, "init_weights"):
+        module.init_weights()
+
+
+class Attention(nn.Module):
+    fused_attn: Final[bool]
+
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int = 8,
+        qkv_bias: bool = False,
+        qk_norm: bool = False,
+        attn_drop: float = 0.0,
+        proj_drop: float = 0.0,
+        norm_layer: nn.Module = nn.LayerNorm,
+    ) -> None:
+        super().__init__()
+        assert dim % num_heads == 0, "dim should be divisible by num_heads"
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.scale = self.head_dim**-0.5
+        # self.fused_attn = use_fused_attn()
+        self.fused_attn = True
+
+        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
+        self.q_norm = norm_layer(self.head_dim) if qk_norm else nn.Identity()
+        self.k_norm = norm_layer(self.head_dim) if qk_norm else nn.Identity()
+        self.attn_drop = nn.Dropout(attn_drop)
+        self.proj = nn.Linear(dim, dim)
+        self.proj_drop = nn.Dropout(proj_drop) if proj_drop > 0.0 else nn.Identity()
+
+    def forward(self, x: torch.Tensor, cu_slens=None) -> torch.Tensor:
+        B, N, C = x.shape
+        qkv = (
+            self.qkv(x)
+            .reshape(B, N, 3, self.num_heads, self.head_dim)
+            .permute(2, 0, 3, 1, 4)
+        )
+        q, k, v = qkv.unbind(0)
+        q, k = self.q_norm(q), self.k_norm(k)
+
+        if cu_slens is not None:
+            q = q.permute(0, 2, 1, 3)   # B, num_heads, N, C -> B, N, num_heads, C
+            k = k.permute(0, 2, 1, 3)
+            v = v.permute(0, 2, 1, 3)
+            max_seqlen = torch.max(cu_slens[1:] - cu_slens[:-1]).item()
+            x = flash_attn_varlen_func(
+                q.squeeze(0),
+                k.squeeze(0),
+                v.squeeze(0),
+                cu_seqlens_q=cu_slens,
+                cu_seqlens_k=cu_slens,
+                max_seqlen_q=max_seqlen,
+                max_seqlen_k=max_seqlen,
+                softmax_scale=self.scale,
+                causal=False,
+                )
+
+            x = x.reshape(B, N, -1)
+            x = self.proj(x)
+            x = self.proj_drop(x)
+
+        else:
+            q = q.permute(0, 2, 1, 3)   # B, num_heads, N, C -> B, N, num_heads, C
+            k = k.permute(0, 2, 1, 3)
+            v = v.permute(0, 2, 1, 3)
+            x = flash_attn_func(q, k, v, softmax_scale=self.scale) # -> b, n, h, c
+
+            x = x.reshape(B, N, -1)
+            x = self.proj(x)
+            x = self.proj_drop(x)
+        # if self.fused_attn:
+        #     x = F.scaled_dot_product_attention(
+        #         q,
+        #         k,
+        #         v,
+        #         dropout_p=self.attn_drop.p if self.training else 0.0,
+        #     )
+        # else:
+        #     q = q * self.scale
+        #     attn = q @ k.transpose(-2, -1)
+        #     attn = attn.softmax(dim=-1)
+        #     attn = self.attn_drop(attn)
+        #     x = attn @ v
+
+        # x = x.transpose(1, 2).reshape(B, N, C)
+        # x = self.proj(x)
+        # x = self.proj_drop(x)
+        return x
+
+
+class LayerScale(nn.Module):
+    def __init__(
+        self,
+        dim: int,
+        init_values: float = 1e-5,
+        inplace: bool = False,
+    ) -> None:
+        super().__init__()
+        self.inplace = inplace
+        self.gamma = nn.Parameter(init_values * torch.ones(dim))
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return x.mul_(self.gamma) if self.inplace else x * self.gamma
+
+
+class Block(nn.Module):
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int,
+        mlp_ratio: float = 4.0,
+        qkv_bias: bool = False,
+        qk_norm: bool = False,
+        proj_drop: float = 0.0,
+        attn_drop: float = 0.0,
+        init_values: Optional[float] = None,
+        drop_path: float = 0.0,
+        act_layer: nn.Module = nn.GELU,
+        norm_layer: nn.Module = nn.LayerNorm,
+        mlp_layer: nn.Module = Mlp,
+    ) -> None:
+        super().__init__()
+        self.norm1 = norm_layer(dim)
+        self.attn = Attention(
+            dim,
+            num_heads=num_heads,
+            qkv_bias=qkv_bias,
+            qk_norm=qk_norm,
+            attn_drop=attn_drop,
+            proj_drop=proj_drop,
+            norm_layer=norm_layer,
+        )
+        self.ls1 = (
+            LayerScale(dim, init_values=init_values) if init_values else nn.Identity()
+        )
+        self.drop_path1 = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+
+        self.norm2 = norm_layer(dim)
+        self.mlp = mlp_layer(
+            in_features=dim,
+            hidden_features=int(dim * mlp_ratio),
+            act_layer=act_layer,
+            drop=proj_drop,
+        )
+        self.ls2 = (
+            LayerScale(dim, init_values=init_values) if init_values else nn.Identity()
+        )
+        self.drop_path2 = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+
+    def forward(self, x: torch.Tensor, cu_slens=None) -> torch.Tensor:
+        x = x + self.drop_path1(self.ls1(self.attn(self.norm1(x), cu_slens=cu_slens)))
+        x = x + self.drop_path2(self.ls2(self.mlp(self.norm2(x))))
+        return x
+
+
+class VisionTransformer(nn.Module):
+    """Vision Transformer
+
+    A PyTorch impl of : `An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale`
+        - https://arxiv.org/abs/2010.11929
+    """
+
+    dynamic_img_size: Final[bool]
+
+    def __init__(
+        self,
+        img_size: Union[int, Tuple[int, int]] = 224,
+        patch_size: Union[int, Tuple[int, int]] = 16,
+        in_chans: int = 3,
+        num_classes: int = 1000,
+        global_pool: Literal["", "avg", "token", "map"] = "token",
+        embed_dim: int = 768,
+        depth: int = 12,
+        num_heads: int = 12,
+        mlp_ratio: float = 4.0,
+        qkv_bias: bool = True,
+        qk_norm: bool = False,
+        init_values: Optional[float] = None,
+        class_token: bool = True,
+        no_embed_class: bool = False,
+        reg_tokens: int = 0,
+        pre_norm: bool = False,
+        fc_norm: Optional[bool] = None,
+        dynamic_img_size: bool = False,
+        dynamic_img_pad: bool = False,
+        drop_rate: float = 0.0,
+        pos_drop_rate: float = 0.0,
+        patch_drop_rate: float = 0.0,
+        proj_drop_rate: float = 0.0,
+        attn_drop_rate: float = 0.0,
+        drop_path_rate: float = 0.0,
+        weight_init: Literal["skip", "jax", "jax_nlhb", "moco", ""] = "",
+        embed_layer: Callable = PatchEmbed,
+        norm_layer: Optional[LayerType] = None,
+        act_layer: Optional[LayerType] = None,
+        strict_img_size: bool = False,
+        block_fn: Type[nn.Module] = Block,
+        mlp_layer: Type[nn.Module] = Mlp,
+        ignore_head: bool = False,
+        add_patch2x2: bool = False,
+    ) -> None:
+        """
+        Args:
+            img_size: Input image size.
+            patch_size: Patch size.
+            in_chans: Number of image input channels.
+            num_classes: Mumber of classes for classification head.
+            global_pool: Type of global pooling for final sequence (default: 'token').
+            embed_dim: Transformer embedding dimension.
+            depth: Depth of transformer.
+            num_heads: Number of attention heads.
+            mlp_ratio: Ratio of mlp hidden dim to embedding dim.
+            qkv_bias: Enable bias for qkv projections if True.
+            init_values: Layer-scale init values (layer-scale enabled if not None).
+            class_token: Use class token.
+            no_embed_class: Don't include position embeddings for class (or reg) tokens.
+            reg_tokens: Number of register tokens.
+            fc_norm: Pre head norm after pool (instead of before), if None, enabled when global_pool == 'avg'.
+            drop_rate: Head dropout rate.
+            pos_drop_rate: Position embedding dropout rate.
+            attn_drop_rate: Attention dropout rate.
+            drop_path_rate: Stochastic depth rate.
+            weight_init: Weight initialization scheme.
+            embed_layer: Patch embedding layer.
+            norm_layer: Normalization layer.
+            act_layer: MLP activation layer.
+            block_fn: Transformer block layer.
+        """
+        super().__init__()
+        assert global_pool in ("", "avg", "token", "map")
+        assert class_token or global_pool != "token"
+        use_fc_norm = global_pool == "avg" if fc_norm is None else fc_norm
+        # norm_layer = get_norm_layer(norm_layer) or partial(nn.LayerNorm, eps=1e-6)
+        # act_layer = get_act_layer(act_layer) or nn.GELU
+        norm_layer = partial(nn.LayerNorm, eps=1e-6)
+        act_layer = nn.GELU
+
+        self.num_classes = num_classes
+        self.global_pool = global_pool
+        self.num_features = self.embed_dim = (
+            embed_dim  # num_features for consistency with other models
+        )
+        self.num_prefix_tokens = 1 if class_token else 0
+        self.num_prefix_tokens += reg_tokens
+        self.num_reg_tokens = reg_tokens
+        self.has_class_token = class_token
+        self.no_embed_class = (
+            no_embed_class  # don't embed prefix positions (includes reg)
+        )
+        self.dynamic_img_size = dynamic_img_size
+        self.grad_checkpointing = False
+        self.ignore_head = ignore_head
+
+        embed_args = {}
+        if dynamic_img_size:
+            # flatten deferred until after pos embed
+            embed_args.update(dict(strict_img_size=False, output_fmt="NHWC"))
+        self.patch_embed = embed_layer(
+            img_size=img_size,
+            patch_size=patch_size,
+            in_chans=in_chans,
+            embed_dim=embed_dim,
+            bias=not pre_norm,  # disable bias if pre-norm is used (e.g. CLIP)
+            dynamic_img_pad=dynamic_img_pad,
+            strict_img_size=strict_img_size,
+            **embed_args,
+        )
+        num_patches = self.patch_embed.num_patches
+
+        self.cls_token = (
+            nn.Parameter(torch.zeros(1, 1, embed_dim)) if class_token else None
+        )
+        self.reg_token = (
+            nn.Parameter(torch.zeros(1, reg_tokens, embed_dim)) if reg_tokens else None
+        )
+        embed_len = (
+            num_patches if no_embed_class else num_patches + self.num_prefix_tokens
+        )
+        self.pos_embed = nn.Parameter(torch.randn(1, embed_len, embed_dim) * 0.02)
+
+
+        # deepspeed.zero.register_external_parameter(self, self.pos_embed)
+        # deepspeed.zero.register_external_parameter(self, self.patch_embed.proj.weight)
+        # deepspeed.zero.register_external_parameter(self, self.patch_embed.proj.bias)
+        # print(self.patch_embed.state_dict().keys())
+
+
+        self.pos_drop = nn.Dropout(p=pos_drop_rate)
+        if patch_drop_rate > 0:
+            self.patch_drop = PatchDropout(
+                patch_drop_rate,
+                num_prefix_tokens=self.num_prefix_tokens,
+            )
+        else:
+            self.patch_drop = nn.Identity()
+        self.norm_pre = norm_layer(embed_dim) if pre_norm else nn.Identity()
+
+        dpr = [
+            x.item() for x in torch.linspace(0, drop_path_rate, depth)
+        ]  # stochastic depth decay rule
+        self.blocks = nn.Sequential(
+            *[
+                block_fn(
+                    dim=embed_dim,
+                    num_heads=num_heads,
+                    mlp_ratio=mlp_ratio,
+                    qkv_bias=qkv_bias,
+                    qk_norm=qk_norm,
+                    init_values=init_values,
+                    proj_drop=proj_drop_rate,
+                    attn_drop=attn_drop_rate,
+                    drop_path=dpr[i],
+                    norm_layer=norm_layer,
+                    act_layer=act_layer,
+                    mlp_layer=mlp_layer,
+                )
+                for i in range(depth)
+            ]
+        )
+
+
+        if add_patch2x2:
+            if add_patch2x2 == 'v2':
+                self.downsample = nn.Sequential(
+                    nn.Conv2d(embed_dim, embed_dim*2, kernel_size=2, stride=2),
+                    nn.GELU(),
+                    nn.Conv2d(embed_dim*2, embed_dim*4, 1)
+                )
+            else:
+                mid_dim = embed_dim * 2
+                self.downsample = nn.Sequential(
+                    nn.Conv2d(embed_dim, mid_dim, kernel_size=2, stride=2),
+                    nn.GELU(),
+                    nn.Conv2d(mid_dim, mid_dim, 1)
+            )
+
+        else:
+            self.downsample = None
+
+
+        # self.norm = norm_layer(embed_dim) if not use_fc_norm else nn.Identity()
+
+        # # Classifier Head
+        # if global_pool == "map":
+        #     AttentionPoolLatent.init_weights = init_weights
+        #     self.attn_pool = AttentionPoolLatent(
+        #         self.embed_dim,
+        #         num_heads=num_heads,
+        #         mlp_ratio=mlp_ratio,
+        #         norm_layer=norm_layer,
+        #     )
+        # else:
+        #     self.attn_pool = None
+        # self.fc_norm = norm_layer(embed_dim) if use_fc_norm else nn.Identity()
+        # self.head_drop = nn.Dropout(drop_rate)
+        # self.head = (
+        #     nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()
+        # )
+
+        # if weight_init != "skip":
+        #     self.init_weights(weight_init)
+
+    def init_weights(self, mode: Literal["jax", "jax_nlhb", "moco", ""] = "") -> None:
+        assert mode in ("jax", "jax_nlhb", "moco", "")
+        # head_bias = -math.log(self.num_classes) if "nlhb" in mode else 0.0
+        trunc_normal_(self.pos_embed, std=0.02)
+        if self.cls_token is not None:
+            nn.init.normal_(self.cls_token, std=1e-6)
+        named_apply(init_weights_vit_timm, self)
+
+    @torch.jit.ignore
+    def no_weight_decay(self) -> Set:
+        return {"pos_embed", "cls_token", "dist_token"}
+
+    @torch.jit.ignore
+    def group_matcher(self, coarse: bool = False) -> Dict:
+        return dict(
+            stem=r"^cls_token|pos_embed|patch_embed",  # stem and embed
+            blocks=[(r"^blocks\.(\d+)", None), (r"^norm", (99999,))],
+        )
+
+    @torch.jit.ignore
+    def set_grad_checkpointing(self, enable: bool = True) -> None:
+        self.grad_checkpointing = enable
+
+    @torch.jit.ignore
+    def get_classifier(self) -> nn.Module:
+        return self.head
+
+    def reset_classifier(self, num_classes: int, global_pool=None) -> None:
+        self.num_classes = num_classes
+        if global_pool is not None:
+            assert global_pool in ("", "avg", "token", "map")
+            if global_pool == "map" and self.attn_pool is None:
+                assert (
+                    False
+                ), "Cannot currently add attention pooling in reset_classifier()."
+            elif global_pool != "map " and self.attn_pool is not None:
+                self.attn_pool = None  # remove attention pooling
+            self.global_pool = global_pool
+        self.head = (
+            nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()
+        )
+
+    def rescale_positional_embedding(self, out_size):
+        h, w = out_size
+        pos_embed_shape = int((self.pos_embed.shape[1]) ** 0.5)
+        if (h, w) == (pos_embed_shape, pos_embed_shape):
+            return self.pos_embed
+        rescaled_positional_embedding = \
+            self.pos_embed.new_zeros(1, h*w, self.pos_embed.shape[2])
+        pe_2d = self.pos_embed[0].T.contiguous().view(1, -1, pos_embed_shape, pos_embed_shape)
+        pe_2d = F.interpolate(pe_2d, out_size, mode='bilinear', align_corners=False).view(-1, h*w)
+        rescaled_positional_embedding[0] = pe_2d.T.contiguous()
+        return rescaled_positional_embedding
+
+    def _pos_embed(self, x: torch.Tensor) -> torch.Tensor:
+        if self.dynamic_img_size:
+            B, H, W, C = x.shape
+            pos_embed = resample_abs_pos_embed(
+                self.pos_embed,
+                (H, W),
+                num_prefix_tokens=0 if self.no_embed_class else self.num_prefix_tokens,
+            )
+            x = x.view(B, -1, C)
+        else:
+            pos_embed = self.pos_embed
+
+        to_cat = []
+        if self.cls_token is not None:
+            to_cat.append(self.cls_token.expand(x.shape[0], -1, -1))
+        if self.reg_token is not None:
+            to_cat.append(self.reg_token.expand(x.shape[0], -1, -1))
+
+        if self.no_embed_class:
+            # deit-3, updated JAX (big vision)
+            # position embedding does not overlap with class token, add then concat
+            x = x + pos_embed
+            if to_cat:
+                x = torch.cat(to_cat + [x], dim=1)
+        else:
+            # original timm, JAX, and deit vit impl
+            # pos_embed has entry for class token, concat then add
+            if to_cat:
+                x = torch.cat(to_cat + [x], dim=1)
+            x = x + pos_embed
+
+        return self.pos_drop(x)
+
+    def _intermediate_layers(
+        self,
+        x: torch.Tensor,
+        n: Union[int, Sequence] = 1,
+    ) -> List[torch.Tensor]:
+        outputs, num_blocks = [], len(self.blocks)
+        take_indices = set(
+            range(num_blocks - n, num_blocks) if isinstance(n, int) else n
+        )
+
+        # forward pass
+        x = self.patch_embed(x)
+        x = self._pos_embed(x)
+        x = self.patch_drop(x)
+        x = self.norm_pre(x)
+        for i, blk in enumerate(self.blocks):
+            x = blk(x)
+            if i in take_indices:
+                outputs.append(x)
+
+        return outputs
+
+    def get_intermediate_layers(
+        self,
+        x: torch.Tensor,
+        n: Union[int, Sequence] = 1,
+        reshape: bool = False,
+        return_prefix_tokens: bool = False,
+        norm: bool = False,
+    ) -> Tuple[Union[torch.Tensor, Tuple[torch.Tensor]]]:
+        """Intermediate layer accessor (NOTE: This is a WIP experiment).
+        Inspired by DINO / DINOv2 interface
+        """
+        # take last n blocks if n is an int, if in is a sequence, select by matching indices
+        outputs = self._intermediate_layers(x, n)
+        if norm:
+            outputs = [self.norm(out) for out in outputs]
+        prefix_tokens = [out[:, 0 : self.num_prefix_tokens] for out in outputs]
+        outputs = [out[:, self.num_prefix_tokens :] for out in outputs]
+
+        if reshape:
+            grid_size = self.patch_embed.grid_size
+            outputs = [
+                out.reshape(x.shape[0], grid_size[0], grid_size[1], -1)
+                .permute(0, 3, 1, 2)
+                .contiguous()
+                for out in outputs
+            ]
+
+        if return_prefix_tokens:
+            return tuple(zip(outputs, prefix_tokens))
+        return tuple(outputs)
+
+    def forward_features_list(self, x_list):
+        x_all = []
+        image_sizes = []
+        for x in x_list:
+            bs, _, h, w = x.shape
+
+            # fix patch size=14 in datasets
+            pad_h = (self.patch_embed.patch_size[0] - h % self.patch_embed.patch_size[0]) % self.patch_embed.patch_size[0]
+            pad_w = (self.patch_embed.patch_size[1] - w % self.patch_embed.patch_size[1]) % self.patch_embed.patch_size[1]
+            x = F.pad(x, (0, pad_w, 0, pad_h))
+
+            bs, _, h, w = x.shape
+
+            h = h // self.patch_embed.patch_size[0]
+            w = w // self.patch_embed.patch_size[1]
+
+            x = self.patch_embed(x)
+            # x = self._pos_embed(x)
+            x = x + self.rescale_positional_embedding(out_size=(h, w))
+            x = self.patch_drop(x)
+            x = self.norm_pre(x)
+            x_all.append(x)
+            image_sizes.append((h, w))
+
+        slen = [xi.size(1) for xi in x_all]
+        x = torch.cat(x_all, dim=1)
+
+        cu_indices = [0, ]
+        for i in slen:
+            cu_indices.append(cu_indices[-1] + i)
+
+        cu_slens = torch.tensor(cu_indices, dtype=torch.int32).to(x.device)
+        for idx, blk in enumerate(self.blocks):
+            if self.grad_checkpointing and not torch.jit.is_scripting():
+                x = checkpoint(blk, x, cu_slens, use_reentrant=True)
+            else:
+                x = blk(x, cu_slens=cu_slens)
+        feats = x.split(slen, dim=1) #[(1, slen, c)]
+
+        if self.downsample is not None:
+            new_feats = []
+            new_sizes = []
+            for f, s in zip(feats, image_sizes):
+                h, w = s
+                b, n, c = f.size()
+                f = f.reshape(b, h, w, c).permute(0, 3, 1, 2)
+                f = self.downsample(f)
+                b, c, h, w = f.size()
+                f = f.permute(0, 2, 3, 1).reshape(b, h*w, c)
+                new_feats.append(f)
+                new_sizes.append((h, w))
+            return new_feats, new_sizes
+
+
+        return feats, image_sizes
+
+    def forward_features(self, x: torch.Tensor) -> torch.Tensor:
+        bs, _, h, w = x.shape
+        h = h // self.patch_embed.patch_size[0]
+        w = w // self.patch_embed.patch_size[1]
+
+        x = self.patch_embed(x)
+        # x = self._pos_embed(x)
+        x = x + self.rescale_positional_embedding(out_size=(h, w))
+        x = self.patch_drop(x)
+        x = self.norm_pre(x)
+        if self.grad_checkpointing and not torch.jit.is_scripting():
+            x = checkpoint_seq(self.blocks, x)
+        else:
+            x = self.blocks(x)
+
+        if self.downsample is not None:
+            b, n, c = x.size()
+            x = x.reshape(b, h, w, c).permute(0, 3, 1, 2)
+            x = self.downsample(x)
+            b, c, h, w = x.size()
+            x = x.permute(0, 2, 3, 1).reshape(b, h*w, c)
+            new_feats = x
+            new_sizes = (h, w)
+            return new_feats, new_sizes
+
+        return x, (h, w)
+
+    def forward_head(self, x: torch.Tensor, pre_logits: bool = False) -> torch.Tensor:
+        x = self.norm(x)
+        if self.attn_pool is not None:
+            x = self.attn_pool(x)
+        elif self.global_pool == "avg":
+            x = x[:, self.num_prefix_tokens :].mean(dim=1)
+        elif self.global_pool:
+            x = x[:, 0]  # class token
+        x = self.fc_norm(x)
+        x = self.head_drop(x)
+        return x if pre_logits else self.head(x)
+
+    def forward(self, x, cal_attn_pool=False):
+        if type(x) is list:
+            x, image_sizes = self.forward_features_list(x)
+            return x, image_sizes, None
+        else:
+            x, image_sizes = self.forward_features(x)
+            return x, image_sizes, None
+
+@dataclass
+class SigLIPVisionCfg:
+    width: int = 1152
+    layers: Union[Tuple[int, int, int, int], int] = 27
+    heads: int = 16
+    patch_size: int = 14
+    image_size: Union[Tuple[int, int], int] = 336
+    global_pool: str = "map"
+    mlp_ratio: float = 3.7362
+    class_token: bool = False
+    num_classes: int = 0
+    use_checkpoint: bool = False
+
+
+SigLIP_MODEL_CONFIG = {
+    "siglip_so400m_patch14_384": {
+        "image_size": 384,
+        "patch_size": 14,
+        "width": 1152,
+        "layers": 27,
+        "heads": 16,
+        "mlp_ratio": 3.7362,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+    "siglip_so400m_patch16_384": {
+        "image_size": 384,
+        "patch_size": 16,
+        "width": 1152,
+        "layers": 27,
+        "heads": 16,
+        "mlp_ratio": 3.7362,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+    "siglip_so400m_patch14_224": {
+        "image_size": 224,
+        "patch_size": 14,
+        "width": 1152,
+        "layers": 27,
+        "heads": 16,
+        "mlp_ratio": 3.7362,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+    "siglip_large_patch16_384": {
+        "image_size": 384,
+        "patch_size": 16,
+        "width": 1024,
+        "layers": 24,
+        "heads": 16,
+        "mlp_ratio": 4,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+}
+
+
+def resize_evaclip_pos_embed(model: VisionTransformer, interpolation: str = 'bicubic'):
+    # interpolate position embedding
+    orig_size = 24
+    new_size = 128
+    pos_tokens = model.pos_embed
+    pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, model.embed_dim).permute(0, 3, 1, 2)
+    pos_tokens = torch.nn.functional.interpolate(
+        pos_tokens, size=(new_size, new_size), mode=interpolation, align_corners=False)
+    pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)
+    model.pos_embed = nn.Parameter(pos_tokens, requires_grad=True)
+    return model
+
+def create_siglip_vit(
+    model_name: str = "siglip_so400m_patch14_384",
+    image_size: int = 384,
+    select_layer: int = -1,
+    path: str = "",
+    gradient_checkpointing: bool = False,
+    **kwargs,
+):
+    assert (
+        model_name in SigLIP_MODEL_CONFIG.keys()
+    ), f"model name should be in {SigLIP_MODEL_CONFIG.keys()}"
+
+    vision_cfg = SigLIPVisionCfg(**SigLIP_MODEL_CONFIG[model_name])
+
+    if select_layer <= 0:
+        layers = min(vision_cfg.layers, vision_cfg.layers + select_layer + 1)
+    else:
+        layers = min(vision_cfg.layers, select_layer)
+
+
+
+    if 'patch2x2' or 'patch4x4' in path:
+        add_patch2x2 = True
+    else:
+        add_patch2x2 = False
+
+    if 'patch4x4pool' in path or 'patch2x2from4x4' in path:
+        add_patch2x2 = 'v2'
+
+    if FORCE_NO_DOWNSAMPLE:
+        add_patch2x2 = False
+
+    model = VisionTransformer(
+        img_size=2048,
+        patch_size=16,
+        embed_dim=vision_cfg.width,
+        depth=layers,
+        num_heads=vision_cfg.heads,
+        mlp_ratio=vision_cfg.mlp_ratio,
+        class_token=vision_cfg.class_token,
+        global_pool=vision_cfg.global_pool,
+        dynamic_img_pad=False,
+        strict_img_size=False,
+        ignore_head=kwargs.get("ignore_head", False),
+        weight_init=kwargs.get("weight_init", "skip"),
+        num_classes=0,
+        add_patch2x2=add_patch2x2
+    )
+
+    print("#### Skip loading vision backbone")
+
+    if gradient_checkpointing:
+        model.set_grad_checkpointing(True)
+    return model
+
+import torch.distributed as dist
+from transformers import CLIPImageProcessor
+
+
+class SigLIPViTAnysizeWrapper(nn.Module):
+    def __init__(self, vision_tower, path, args, delay_load=False):
+        super().__init__()
+
+        self.is_loaded = False
+
+        self.vision_tower_name = vision_tower
+        self.args = args
+        self.path = path
+
+        self.select_layer = -1
+        if self.select_layer < -1: self.select_layer += 1
+        self.select_feature = getattr(args, 'mm_vision_select_feature', 'patch')
+
+        self.output_dim = 1152
+
+        if not delay_load or LOAD_VISION_EARLY:
+            self.load_model()
+        elif getattr(args, "unfreeze_mm_vision_tower", False):
+            # TODO: better detector is needed.
+            print(f"The checkpoint seems to contain `vision_tower` weights: `unfreeze_mm_vision_tower`: True.")
+            self.load_model()
+
+    def load_model(self, device_map=None):
+        if self.is_loaded:
+            print('{} is already loaded, `load_model` called again, skipping.'.format(self.vision_tower_name))
+            return
+
+        self.image_processor = CLIPImageProcessor.from_pretrained("openai/clip-vit-large-patch14")
+        if self.args.mm_projector_type == "conv_mlp" or self.args.mm_projector_type == "multipath_conv_mlp" or self.args.mm_projector_type == "multipath_conv_mlp_woconv":
+            self.image_processor.crop_size['height'] = 384
+            self.image_processor.crop_size['width'] = 384
+            self.image_processor.size['shortest_edge'] = 384
+            print("Resizeing clip processor to 384...")
+        self.image_processor.image_mean = [0.5, 0.5, 0.5]
+        self.image_processor.image_std = [0.5, 0.5, 0.5]
+        print("Loading vision model...")
+
+        self.vision_tower = create_siglip_vit(path=self.path, model_name='siglip_so400m_patch16_384',
+                                                gradient_checkpointing=False)
+        for p in self.vision_tower.parameters():
+            p.requires_grad = False
+        self.vision_tower.eval()
+        self.is_loaded = True
+
+    def train(self, mode = True):
+        self.training = mode
+
+        if self.is_loaded:
+            self.vision_tower.eval()
+
+    def split_images(self, images, split_res=512, base_size=32):
+        split_images = []
+        sub_images_info = []
+        for image in images:
+            now_sub_images = []
+            _, c, h, w = image.shape
+            if h * w <= split_res * split_res:
+                split_images.append(image)
+                sub_images_info.append(
+                    (
+                        1, 1, 1, h // base_size, w // base_size, [(0, h // base_size, 0, w // base_size)]
+                    )
+                )
+                continue
+            nsplit_h = math.ceil(h / split_res)
+            nsplit_w = math.ceil(w / split_res)
+            sub_h = int(h / nsplit_h  / base_size) * base_size
+            sub_w = int(w / nsplit_w / base_size) * base_size
+            crop_infos = []
+            for i in range(nsplit_h):
+                for j in range(nsplit_w):
+                    begin_h = i * sub_h
+                    begin_w = j * sub_w
+
+                    if i == nsplit_h - 1:
+                        end_h = h
+                    else:
+                        end_h = (i + 1) * sub_h
+
+                    if j == nsplit_w - 1:
+                        end_w = w
+                    else:
+                        end_w = (j + 1) * sub_w
+
+                    assert (end_h - begin_h) % base_size == 0 and (end_w - begin_w) % base_size == 0
+
+                    sub_image = image[:, :, begin_h:end_h, begin_w:end_w]
+                    now_sub_images.append(sub_image)
+                    crop_infos.append(
+                        (begin_h // base_size, end_h // base_size, begin_w // base_size, end_w // base_size)
+                    )
+
+            split_images += now_sub_images
+            sub_images_info.append(
+                (
+                    len(now_sub_images), nsplit_h, nsplit_w, h // base_size, w // base_size, crop_infos
+                )
+            )
+
+        return split_images, sub_images_info
+
+
+    def unsplit_images(self, features, sizes, sub_images_info):
+        new_features = []
+        for feature, size in zip(features, sizes):
+            h, w = size
+            new_features.append(
+                feature.reshape(1, h, w, -1)
+            )
+
+        fused_images = []
+        images_sizes = []
+        sub_count = 0
+        for n_split, nsplit_h, nsplit_w, total_h, total_w, crop_infos in sub_images_info:
+            sub_features = new_features[sub_count:sub_count+n_split]
+            sub_count += n_split
+
+            total_feature = new_features[0].new_zeros(1, total_h, total_w, self.hidden_size)
+            for feature, (begin_h, end_h, begin_w, end_w) in zip(sub_features, crop_infos):
+                total_feature[:, begin_h:end_h, begin_w:end_w] += feature
+
+            fused_images.append(total_feature.reshape(1, total_h * total_w, self.hidden_size))
+            images_sizes.append((total_h, total_w))
+
+        return fused_images, images_sizes
+
+
+
+    def forward_func(self, images, force_fix_size=False, cal_attn_pool=False):
+        if type(images) is list:
+            xs = [x.to(self.dtype) for x in images]
+            image_features, img_size, cls_token = self.vision_tower(xs, cal_attn_pool=cal_attn_pool)
+            image_features = [x.to(images[0].dtype) for x in image_features]
+
+        else:
+            image_forward_outs, img_size, cls_token = self.vision_tower(images.to(self.dtype), cal_attn_pool=cal_attn_pool)
+            image_features = image_forward_outs.to(images.dtype)
+
+        return image_features, img_size, cls_token
+
+    def forward(self, images, cal_attn_pool=False):
+        with torch.no_grad():
+            image_features, img_size, cls_token = self.forward_func(images, cal_attn_pool=cal_attn_pool)
+            return image_features, img_size
+
+
+    @property
+    def dummy_feature(self):
+        return torch.zeros(1, 1152, device=self.device, dtype=self.dtype)
+
+    @property
+    def dtype(self):
+        return self.vision_tower.pos_embed.dtype
+
+    @property
+    def device(self):
+        return self.vision_tower.pos_embed.device
+
+    @property
+    def hidden_size(self):
+        return self.output_dim
+
+    @property
+    def config(self):
+        return type('LLaVAConfigWrapper', (), {
+            # 'image_size': 224,
+            'patch_size': 16,
+        })()
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_projector/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_projector/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..97d4ab39238c4d690376213ad67c6de25cf0d083
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_projector/builder.py
@@ -0,0 +1,180 @@
+import math
+import os
+import re
+
+import torch
+import torch.nn as nn
+
+from .pooler_projector import NormalizedDwPooler
+
+if 'REGIONAL_POOL' in os.environ:
+    REGIONAL_POOL = os.environ['REGIONAL_POOL']
+else:
+    REGIONAL_POOL = '2x'
+print(f"REGIONAL_POOL is set as {REGIONAL_POOL}")
+
+class IdentityMap(nn.Module):
+    def __init__(self):
+        super().__init__()
+
+    def forward(self, x, *args, **kwargs):
+        return x
+
+    @property
+    def config(self):
+        return {"mm_projector_type": 'identity'}
+
+
+class SimpleResBlock(nn.Module):
+    def __init__(self, channels):
+        super().__init__()
+        self.pre_norm = nn.LayerNorm(channels)
+
+        self.proj = nn.Sequential(
+            nn.Linear(channels, channels),
+            nn.GELU(),
+            nn.Linear(channels, channels)
+        )
+    def forward(self, x):
+        x = self.pre_norm(x)
+        return x + self.proj(x)
+
+class OlaMLP(nn.Module):
+    def __init__(self, in_channels, out_channels, twoview=False):
+        super().__init__()
+
+        self.proj1 = nn.Linear(in_channels, out_channels)
+        self.proj2 = nn.Linear(out_channels, out_channels)
+        self.act = nn.GELU()
+        self.pooler = NormalizedDwPooler(out_channels)
+
+        embed_std = 1 / math.sqrt(out_channels)
+        self.image_newline = nn.Parameter(
+            torch.randn(out_channels) * embed_std
+        )
+        self.image_begin = nn.Parameter(
+            torch.randn(out_channels) * embed_std
+        )
+        self.image_end = nn.Parameter(
+            torch.randn(out_channels) * embed_std
+        )
+
+        if twoview:
+            self.image_sep = nn.Parameter(
+                torch.randn(out_channels) * embed_std
+            )
+
+    def forward(self, x, size=(16,16), x2=None, size2=(16, 16), modalities='image'):
+
+        if modalities in ['image', 'text']:
+            h, w = size
+            dtype = x.dtype
+            x = x.reshape(x.shape[0], h, w, -1)
+            x = self.proj1(x)
+            x = self.pooler(x, forward_type=REGIONAL_POOL)
+            x = self.act(x)
+            x = self.proj2(x)
+
+
+            b, h, w, c = x.shape
+            x = torch.cat([
+                x,
+                self.image_newline.reshape(1, 1, 1, c).expand(b, h, 1, c).to(dtype)
+            ], dim=2)
+            x = x.reshape(b, -1, c)
+
+            if x2 is not None:
+                h2, w2 = size2
+                x2 = x2.reshape(x2.shape[0], h2, w2, -1)
+                x2 = self.proj1(x2)
+                x2 = self.pooler(x2, forward_type=REGIONAL_POOL)
+                x2 = self.act(x2)
+                x2 = self.proj2(x2)
+
+                b2, h2, w2, c2 = x2.shape
+                x2 = torch.cat([
+                    x2,
+                    self.image_newline.reshape(1, 1, 1, c).expand(b, h2, 1, c).to(dtype)
+                ], dim=2)
+                x2 = x2.reshape(b, -1, c)
+                sep = self.image_sep.reshape(1, 1, -1).expand(b, 1, c2).to(dtype)
+                if os.environ.get('USE_HIGHRES_ONLY', '0') == '1':
+                    x = x2
+                else:
+                    x = torch.cat([x, sep, x2], dim=1)
+
+            begin = self.image_begin.reshape(1, 1, -1).expand(b, 1, c).to(dtype)
+            end = self.image_end.reshape(1, 1, -1).expand(b, 1, c).to(dtype)
+            x = torch.cat([begin, x, end], dim=1)
+            return x
+        elif modalities in ['video']:
+            # x2 is the true feature, ignore x
+            h, w = size
+            dtype = x.dtype
+            x = x.reshape(x.shape[0], h, w, -1)
+            x1 = self.proj1(x)
+            x1 = self.pooler(x1, forward_type=REGIONAL_POOL)
+            x1 = self.proj2(x1).mean() * 0.0
+
+            h2, w2 = size2
+            x2 = x2.reshape(x2.shape[0], h2, w2, -1)
+            x2 = self.proj1(x2)
+            x2 = self.pooler(x2, forward_type=REGIONAL_POOL)
+            x2 = self.act(x2)
+            x2 = self.proj2(x2)
+
+            b2, h2, w2, c = x2.shape
+            x2 = torch.cat([
+                x2,
+                self.image_newline.reshape(1, 1, 1, c).expand(b2, h2, 1, c).to(dtype)
+            ], dim=2)
+
+            x2 = x2.reshape(b2, -1, c)
+
+            sep = self.image_sep.reshape(1, 1, -1).expand(b2, 1, c).to(dtype)
+            x2 = torch.cat([x2, sep], dim=1)
+
+            x2 = x2.flatten(0, 1)
+
+            begin = self.image_begin.reshape(1, -1).expand(1, c).to(dtype)
+            end = self.image_end.reshape(1, -1).expand(1, c).to(dtype)
+            x2 = torch.cat([begin, x2, end], dim=0)
+            x2 = x2.unsqueeze(0)
+            return x2
+        else:
+            raise ValueError(f'Unknown modalities: {modalities}')
+
+def build_vision_projector(config, delay_load=False, **kwargs):
+    projector_type = getattr(config, 'mm_projector_type', 'linear')
+
+    if projector_type == 'linear':
+        return nn.Linear(config.mm_hidden_size, config.hidden_size)
+
+    elif projector_type == 'ola_mlp':
+        return OlaMLP(config.mm_hidden_size, config.hidden_size, twoview=True)
+
+    mlp_gelu_match = re.match(r'^mlp(\d+)x_gelu$', projector_type)
+    if mlp_gelu_match:
+        mlp_depth = int(mlp_gelu_match.group(1))
+        modules = [nn.Linear(config.mm_hidden_size, config.hidden_size)]
+        for _ in range(1, mlp_depth):
+            modules.append(nn.GELU())
+            modules.append(nn.Linear(config.hidden_size, config.hidden_size))
+        return nn.Sequential(*modules)
+
+    mlp_gelu_resnet_match = re.match(r'^mlp(\d+)x_res(\d+)x_gelu$', projector_type)
+    if mlp_gelu_resnet_match:
+        mlp_depth = int(mlp_gelu_resnet_match.group(1))
+        res_depth = int(mlp_gelu_resnet_match.group(2))
+        modules = [nn.Linear(config.mm_hidden_size, config.hidden_size)]
+        for _ in range(1, mlp_depth):
+            modules.append(nn.GELU())
+            modules.append(nn.Linear(config.hidden_size, config.hidden_size))
+        for _ in range(res_depth):
+            modules.append(SimpleResBlock(config.hidden_size))
+        return nn.Sequential(*modules)
+
+    if projector_type == 'identity':
+        return IdentityMap()
+
+    raise ValueError(f'Unknown projector type: {projector_type}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_projector/pooler_projector.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_projector/pooler_projector.py
new file mode 100644
index 0000000000000000000000000000000000000000..f6b36347734db7658a70c80548fdac322eb36e67
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_projector/pooler_projector.py
@@ -0,0 +1,68 @@
+import math
+import os
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers.models.clip.modeling_clip import CLIPVisionModel
+
+
+class PoolerProjector(nn.Module):
+    def __init__(self, config, vision_cfg):
+        super().__init__()
+        self._config = config
+        self.hw = vision_cfg.image_size // vision_cfg.patch_size
+
+        self.conv_pool = nn.Conv2d(
+            config.mm_hidden_size, config.hidden_size,
+            kernel_size=2, stride=2
+        )
+
+        self.proj = nn.Sequential(
+            nn.GELU(),
+            nn.Linear(config.hidden_size, config.hidden_size),
+        )
+
+    def forward(self, x, *args, **kwargs):
+        height = width = self.hw
+        assert height * width == x.shape[1]
+        x = x.view(x.shape[0], height, width, -1).permute(0, 3, 1, 2)
+        x = self.conv_pool(x)
+        x = x.flatten(2).transpose(1, 2)
+        x = self.proj(x)
+        return x
+
+    @property
+    def config(self):
+        return {"mm_projector_type": 'pooler'}
+
+
+class NormalizedDwPooler(nn.Module):
+    def __init__(self, dim):
+        super().__init__()
+        self.dim = dim
+        self.predictor = nn.Sequential(
+            nn.Linear(dim*2, dim),
+            nn.GELU(),
+            nn.Linear(dim, dim),
+        )
+
+    def forward(self, x, forward_type='2x'):
+        B, H, W, C = x.shape
+
+        if forward_type == '2x':
+            new_x = x.reshape(B, H//2, 2, W//2, 2, C).permute(0, 1, 3, 2, 4, 5).reshape(B, H//2, W//2, 4, C)
+            pooled_x = new_x.mean(-2, keepdim=True).expand(-1, -1, -1, 4, -1)
+            fused_x = torch.cat([new_x, pooled_x], dim=-1)
+        elif forward_type == '1x':
+            new_x = x.reshape(B, H, W, 1, C)
+            fused_x = torch.cat([new_x, new_x], dim=-1)
+        elif forward_type == '4x':
+            new_x = x.reshape(B, H//4, 4, W//4, 4, C).permute(0, 1, 3, 2, 4, 5).reshape(B, H//4, W//4, 16, C)
+            pooled_x = new_x.mean(-2, keepdim=True).expand(-1, -1, -1, 16, -1)
+            fused_x = torch.cat([new_x, pooled_x], dim=-1)
+
+        score = self.predictor(fused_x)
+        normalized_score = F.softmax(score, dim=-2)
+        new_x = (new_x * normalized_score).sum(dim=-2)
+        return new_x
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_resampler/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_resampler/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..f79c6cfc4fca69b0183c0806c5812f4378f327d1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_resampler/builder.py
@@ -0,0 +1,25 @@
+import torch
+
+from .perceiver import DynamicCompressor
+
+
+class IdentityMap(torch.nn.Module):
+    def __init__(self):
+        super().__init__()
+
+    def forward(self, x, *args, **kwargs):
+        return x
+
+    @property
+    def config(self):
+        return {"mm_resampler_type": None}
+
+def build_vision_resampler(model_args, delay_load=False, **kwargs):
+    # import pdb;pdb.set_trace()
+    resampler_type = getattr(model_args, 'mm_resampler_type', None)
+    if resampler_type == 'dynamic_compressor':
+        return DynamicCompressor(model_args, **kwargs)
+    elif resampler_type is None:
+        return IdentityMap()
+    else:
+        raise ValueError(f'Unknown resampler type: {resampler_type}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_resampler/perceiver.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_resampler/perceiver.py
new file mode 100644
index 0000000000000000000000000000000000000000..b4d1e291335fc9c59d38018ec4ede83fc2701b9d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/multimodal_resampler/perceiver.py
@@ -0,0 +1,75 @@
+import math
+import os
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class DynamicCompressor(nn.Module):
+    def __init__(self, model_args, vision_tower):
+        super().__init__()
+
+        self.out_channels = vision_tower.hidden_size
+        self.mid_channel = 256
+
+        self.vlm_query_projector  = nn.Linear(self.out_channels, self.mid_channel)
+        self.vlm_key_projector  = nn.Linear(self.out_channels, self.mid_channel)
+
+    def downsample(self, x):
+        return F.avg_pool2d(x, 2, 2)
+
+    def downsample_4(self, x):
+        return F.avg_pool2d(x, 4, 4)
+
+    def forward(self, image_features, forward_type, image_size=None):
+        if image_size is None:
+            ori_W = int(math.sqrt(image_features.shape[1]))
+            ori_H = int(ori_W)
+        else:
+            ori_H, ori_W = image_size
+        T, N, C = image_features.shape
+        image_features = image_features.view(T, ori_H, ori_W, C).permute(0, 3, 1, 2)  # T, C, H, W
+
+        if forward_type == 'video':
+            image_features_pool = self.downsample(image_features)
+            image_feature_attn = image_features.reshape(T, C, ori_H // 2, 2, ori_W // 2, 2).permute(0, 2, 4, 3, 5, 1).reshape(T, ori_H // 2 * ori_W // 2, 4, C)
+            new_image_size = (ori_H // 2, ori_W // 2)
+        elif forward_type == 'image' or forward_type == 'text':
+            image_features_pool = image_features
+            image_feature_attn = image_features.reshape(T, C, ori_H, 1, ori_W, 1).permute(0, 2, 4, 3, 5, 1).reshape(T, ori_H * ori_W, 1, C)
+            new_image_size = (ori_H, ori_W)
+        elif forward_type == 'video_long':
+            image_features_pool = self.downsample_4(image_features)
+            image_feature_attn = image_features.reshape(T, C, ori_H // 4, 4, ori_W // 4, 4).permute(0, 2, 4, 3, 5, 1).reshape(T, ori_H // 4 * ori_W // 4, 16, C)
+            new_image_size = (ori_H // 4, ori_W // 4)
+        else:
+            raise NotImplementedError
+
+        image_features_pool = image_features_pool.flatten(2).permute(0, 2, 1) # T, H*W, C
+        new_t, new_p, _ = image_features_pool.shape
+
+        image_query = self.vlm_query_projector(image_features_pool).reshape(new_t*new_p, self.mid_channel)
+        image_key = self.vlm_key_projector(image_feature_attn).reshape(new_t*new_p, -1, self.mid_channel)
+
+        image_value = image_feature_attn.reshape(new_t*new_p, -1, self.out_channels)
+        # import pdb;pdb.set_trace()
+
+        image_attn = image_query[:,None] @ (image_key.transpose(-1,-2) / (image_key.shape[-1]**0.5))
+        image_attn = image_attn.nan_to_num()
+        attn_feat = (image_attn.softmax(-1) @ image_value).mean(1).reshape(new_t, new_p, C)
+
+        image_features_pool = image_features_pool + attn_feat
+
+        return image_features_pool, new_image_size
+
+    @property
+    def config(self):
+        return {
+            'mm_resampler_type': 'dynamic_compressor',
+            'mm_out_channels': self.out_channels,
+        }
+
+    @property
+    def hidden_size(self):
+        return self.out_channels
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/ola_arch.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/ola_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..4093c48912dae3cd1cadc6fa85dac7e2681dac7a
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/ola_arch.py
@@ -0,0 +1,417 @@
+from abc import ABC, abstractmethod
+
+import torch
+
+from ..constants import (DEFAULT_IM_END_TOKEN, DEFAULT_IM_START_TOKEN, DEFAULT_IMAGE_PATCH_TOKEN,
+                         IGNORE_INDEX, IMAGE_TOKEN_INDEX, SPEECH_TOKEN_INDEX)
+from ..utils import lengths_to_padding_mask
+from .multimodal_encoder.builder import build_vision_tower
+from .multimodal_projector.builder import build_vision_projector
+from .multimodal_resampler.builder import build_vision_resampler
+from .speech_encoder.builder import build_speech_encoder
+from .speech_projector.builder import build_speech_projector
+
+
+class OlaMetaModel:
+
+    def __init__(self, config):
+        super(OlaMetaModel, self).__init__(config)
+
+        if hasattr(config, "speech_encoder"):
+            self.speech_encoder = build_speech_encoder(config)
+            self.speech_projector = build_speech_projector(config)
+
+        if hasattr(config, "mm_vision_tower"):
+            self.vision_tower = build_vision_tower(config, delay_load=True)
+            self.vision_resampler = build_vision_resampler(config, vision_tower=self.vision_tower)
+            self.mm_projector = build_vision_projector(config, vision_cfg=self.vision_tower.config)
+
+    def get_speech_encoder(self):
+        speech_encoder = getattr(self, 'speech_encoder', None)
+        if type(speech_encoder) is list:
+            speech_encoder = speech_encoder[0]
+        return speech_encoder
+
+    def get_vision_tower(self):
+        vision_tower = getattr(self, 'vision_tower', None)
+        if type(vision_tower) is list:
+            vision_tower = vision_tower[0]
+        return vision_tower
+
+    def initialize_speech_modules(self, model_args, fsdp=None):
+        self.config.speech_encoder = getattr(model_args, "speech_encoder", None)
+        self.config.speech_encoder_type = getattr(model_args, "speech_encoder_type", None)
+        self.config.speech_projector_type = getattr(model_args, 'speech_projector_type', 'linear')
+        self.config.speech_encoder_ds_rate = getattr(model_args, 'speech_encoder_ds_rate', 5)
+        self.config.speech_encoder_hidden_size = getattr(model_args, 'speech_encoder_hidden_size', 1280)
+
+        if self.get_speech_encoder() is None:
+            speech_encoder = build_speech_encoder(self.config)
+            if fsdp is not None and len(fsdp) > 0:
+                self.speech_encoder = [speech_encoder]
+            else:
+                self.speech_encoder = speech_encoder
+
+        if getattr(self, 'speech_projector', None) is None:
+            self.speech_projector = build_speech_projector(self.config)
+        else:
+            # In case it is frozen by LoRA
+            for p in self.speech_projector.parameters():
+                p.requires_grad = True
+
+        if model_args.pretrain_speech_projector is not None:
+            pretrain_speech_projector_weights = torch.load(model_args.pretrain_speech_projector, map_location='cpu')
+            def get_w(weights, keyword):
+                return {k.split(keyword + '.')[1]: v for k, v in weights.items() if keyword in k}
+            print('Loading pretrain speech projector weights')
+
+            msg = self.speech_projector.load_state_dict(get_w(pretrain_speech_projector_weights, 'speech_projector'), strict=False)
+            print(msg)
+
+    def initialize_vision_modules(self, model_args, fsdp=None):
+        vision_tower = model_args.vision_tower
+        mm_vision_select_layer = model_args.mm_vision_select_layer
+        mm_vision_select_feature = model_args.mm_vision_select_feature
+        pretrain_mm_mlp_adapter = model_args.pretrain_mm_mlp_adapter
+
+        self.config.mm_vision_tower = vision_tower
+
+        if self.get_vision_tower() is None:
+            vision_tower = build_vision_tower(model_args)
+            vision_resampler = build_vision_resampler(model_args, vision_tower=vision_tower)
+            ## Get the mm_spatial_pool_mode and  mm_spatial_pool_stride
+            for k, v in vision_resampler.config.items():
+                setattr(self.config, k, v)
+
+            if fsdp is not None and len(fsdp) > 0:
+                self.vision_tower = [vision_tower]
+                self.vision_resampler = [vision_resampler]
+            else:
+                self.vision_tower = vision_tower
+                self.vision_resampler = vision_resampler
+        else:
+            if fsdp is not None and len(fsdp) > 0:
+                vision_resampler = self.vision_resampler[0]
+                vision_tower = self.vision_tower[0]
+            else:
+                vision_resampler = self.vision_resampler
+                vision_tower = self.vision_tower
+            vision_tower.load_model()
+
+            # In case it is frozen by LoRA
+            for p in self.vision_resampler.parameters():
+                p.requires_grad = True
+
+        self.config.use_mm_proj = True
+        self.config.mm_projector_type = getattr(model_args, 'mm_projector_type', 'linear')
+        self.config.mm_hidden_size = getattr(vision_resampler, 'hidden_size', vision_tower.hidden_size)
+
+        self.config.mm_vision_select_layer = mm_vision_select_layer
+        self.config.mm_vision_select_feature = mm_vision_select_feature
+
+        if getattr(self, 'mm_projector', None) is None:
+            self.mm_projector = build_vision_projector(self.config, vision_cfg=vision_tower.config)
+        else:
+            for p in self.mm_projector.parameters():
+                p.requires_grad = True
+
+        if pretrain_mm_mlp_adapter is not None:
+            mm_projector_weights = torch.load(pretrain_mm_mlp_adapter, map_location='cpu')
+            def get_w(weights, keyword):
+                return {k.split(keyword + '.')[1]: v for k, v in weights.items() if keyword in k}
+
+            self.mm_projector.load_state_dict(get_w(mm_projector_weights, 'mm_projector'))
+            print('Loading pretrain mm projector weights')
+            incompatible_keys = self.vision_resampler.load_state_dict(get_w(mm_projector_weights, 'vision_resampler'), strict=False)
+            print(incompatible_keys)
+
+class OlaMetaForCausalLM(ABC):
+
+    @abstractmethod
+    def get_model(self):
+        pass
+
+    def get_speech_encoder(self):
+        return self.get_model().get_speech_encoder()
+
+    def get_vision_tower(self):
+        return self.get_model().get_vision_tower()
+
+    def get_speech_projector(self):
+        return self.get_model().speech_projector
+
+    def encode_speech(self, speech, speech_lengths, speech_wav):
+        # import pdb; pdb.set_trace()
+        speech_encoder_type = self.config.speech_encoder_type
+        speech_encoder = self.get_speech_encoder()
+        if "whisper" in speech_encoder_type.lower():
+            encoder_outs = speech_encoder(speech.permute(0, 2, 1))
+            speech_lengths = (speech_lengths + 1) // 2
+        else:
+            encoder_outs = speech_encoder(speech.permute(0, 2, 1), raw_wav=speech_wav)
+            speech_lengths = (speech_lengths + 1) // 2
+        speech_projector_type = self.config.speech_projector_type
+        speech_projector = self.get_speech_projector()
+        if speech_projector_type == "linear":
+            encoder_outs = speech_projector(encoder_outs)
+            speech_lengths = speech_lengths // speech_projector.k
+        else:
+            raise ValueError(f'Unknown speech projector: {speech_projector_type}')
+        # speech_features = [encoder_outs[i, :speech_lengths[i]] for i in range(len(encoder_outs))]
+        return encoder_outs
+
+    def prepare_inputs_labels_for_speech_vision_text(
+        self, input_ids, position_ids, attention_mask, past_key_values, labels,
+        speech, speech_lengths, speech_chunks, speech_wav, images, modalities, image_sizes=None, images_highres=None
+    ):
+        speech_encoder = self.get_speech_encoder()
+        vision_tower = self.get_vision_tower()
+
+        if speech_encoder is None or input_ids.shape[1] == 1:
+            return input_ids, position_ids, attention_mask, past_key_values, None, labels
+
+        if vision_tower is None or input_ids.shape[1] == 1:
+            return input_ids, position_ids, attention_mask, past_key_values, None, labels
+        # encode speech
+        if not isinstance(speech, list):
+            speech = torch.split(speech, speech_chunks.tolist(), dim=0)
+            speech_lengths = torch.split(speech_lengths, speech_chunks.tolist(), dim=0)
+            speech_wav = torch.split(speech_wav, speech_chunks.tolist(), dim=0)
+        speech_features = []
+        for idx in range(len(speech)):
+            speech_features.append(self.encode_speech(speech[idx], speech_lengths[idx], speech_wav[idx]))
+
+        # encode vision
+        if isinstance(modalities, str):
+            modalities = [modalities]
+
+        video_idx_in_batch = []
+        for modal in range(len(modalities)):
+            if 'video' in modalities[modal]:
+                video_idx_in_batch.append(modal)
+
+        # Fix training with deepspeed zero3
+        num_modality = len(modalities)
+        # try:
+        #     world_size = dist.get_world_size()
+        #     tensor_in = torch.zeros(1,  dtype=torch.int64, device=images[0].device).fill_(num_modality)
+        #     tensor_out = torch.zeros(world_size, dtype=torch.int64, device=images[0].device)
+        #     dist.all_gather_into_tensor(tensor_out, tensor_in)
+        #     max_num_modality = tensor_out.max().item()
+        # except:
+        # max_num_modality = num_modality
+        aimg = images[-1]
+        lowres_img = []
+        for idx, img_feat in enumerate(images):
+            if idx in video_idx_in_batch:
+                img_feat = aimg.new(1, 3, 128, 128).fill_(0)
+            lowres_img.append(img_feat)
+
+        # Fix training with deepspeed zero3
+        # if max_num_modality > num_modality:
+        #     for _ in range(max_num_modality - num_modality):
+        #         lowres_img.append(aimg.new(1, 3, 64, 64).fill_(0))
+        #         images_highres.append(aimg.new(1, 3, 64, 64).fill_(0))
+        #         modalities.append('image')
+        lowres_img_features, lowres_img_sizes = self.get_model().get_vision_tower()(lowres_img)
+        highres_img_features = []
+        highres_img_sizes = []
+        for idx, img_feat in enumerate(images_highres):
+            if img_feat.ndim == 5:
+                img_feat = img_feat.squeeze(1)
+            highres_img_feature, highres_img_size = self.get_model().get_vision_tower()(img_feat)
+            highres_img_features.append(highres_img_feature)
+            highres_img_sizes.append(highres_img_size)
+        image_features = []
+        for idx in range(len(modalities)):
+            img_feat = self.get_model().mm_projector(lowres_img_features[idx],
+                                                    lowres_img_sizes[idx],
+                                                    highres_img_features[idx],
+                                                    highres_img_sizes[idx],
+                                                    modalities[idx])
+            image_features.append(img_feat.flatten(0, 1))
+
+        # if max_num_modality > num_modality:
+        #     image_features = image_features[:num_modality]
+        #     modalities = modalities[:num_modality]
+
+        _labels = labels
+        _position_ids = position_ids
+        _attention_mask = attention_mask
+        if attention_mask is None:
+            attention_mask = torch.ones_like(input_ids, dtype=torch.bool)
+        else:
+            attention_mask = attention_mask.bool()
+        if position_ids is None:
+            position_ids = torch.arange(0, input_ids.shape[1], dtype=torch.long, device=input_ids.device)
+        if labels is None:
+            labels = torch.full_like(input_ids, IGNORE_INDEX)
+
+        # remove the padding using attention_mask -- FIXME
+        _input_ids = input_ids
+        input_ids = [cur_input_ids[cur_attention_mask] for cur_input_ids, cur_attention_mask in zip(input_ids, attention_mask)]
+        labels = [cur_labels[cur_attention_mask] for cur_labels, cur_attention_mask in zip(labels, attention_mask)]
+
+        new_input_embeds = []
+        new_labels = []
+        cur_speech_idx = 0
+        cur_image_idx = 0
+        for batch_idx, cur_input_ids in enumerate(input_ids):
+
+            num_speech = (cur_input_ids == SPEECH_TOKEN_INDEX).sum()
+            num_images = (cur_input_ids == IMAGE_TOKEN_INDEX).sum()
+
+            num_speech_images = (cur_input_ids == IMAGE_TOKEN_INDEX).sum() + (cur_input_ids == SPEECH_TOKEN_INDEX).sum()
+
+            if num_speech_images == 0:
+                cur_speech_features = speech_features[cur_speech_idx]
+                cur_images_features = image_features[cur_image_idx]
+                cur_input_embeds_1 = self.get_model().embed_tokens(cur_input_ids)
+                cur_input_embeds = torch.cat([cur_input_embeds_1, cur_speech_features[0:0], cur_images_features[0:0]], dim=0)
+                new_input_embeds.append(cur_input_embeds)
+                new_labels.append(labels[batch_idx])
+                cur_speech_idx += 1
+                cur_image_idx += 1
+                continue
+
+            speech_image_token_indices = [-1] + torch.where((cur_input_ids == SPEECH_TOKEN_INDEX) | (cur_input_ids == IMAGE_TOKEN_INDEX))[0].tolist() + [cur_input_ids.shape[0]]
+
+            cur_input_ids_nospeech_image = []
+            cur_labels = labels[batch_idx]
+            cur_labels_nospeech_image = []
+            for i in range(len(speech_image_token_indices) - 1):
+                cur_input_ids_nospeech_image.append(cur_input_ids[speech_image_token_indices[i]+1:speech_image_token_indices[i+1]])
+                cur_labels_nospeech_image.append(cur_labels[speech_image_token_indices[i]+1:speech_image_token_indices[i+1]])
+            split_sizes = [x.shape[0] for x in cur_labels_nospeech_image]
+            cur_input_embeds = self.get_model().embed_tokens(torch.cat(cur_input_ids_nospeech_image))
+            cur_input_embeds_no_speech_image = torch.split(cur_input_embeds, split_sizes, dim=0)
+            cur_new_input_embeds = []
+            cur_new_labels = []
+
+            for i in range(num_speech_images + 1):
+                cur_new_input_embeds.append(cur_input_embeds_no_speech_image[i])
+                cur_new_labels.append(cur_labels_nospeech_image[i])
+                if i < num_speech_images:
+                    if i < num_images:
+                        cur_images_features = image_features[cur_image_idx]
+                        cur_image_idx += 1
+                        cur_new_input_embeds.append(cur_images_features)
+                        cur_new_labels.append(torch.full((cur_images_features.shape[0],), IGNORE_INDEX, device=cur_labels.device, dtype=cur_labels.dtype))
+                    else:
+                        cur_speech_features = speech_features[cur_speech_idx]
+                        cur_speech_idx += 1
+                        cur_new_input_embeds.append(cur_speech_features)
+                        cur_new_labels.append(torch.full((cur_speech_features.shape[0],), IGNORE_INDEX, device=cur_labels.device, dtype=cur_labels.dtype))
+
+            cur_new_input_embeds = [x.to(self.device) for x in cur_new_input_embeds]
+
+            cur_new_input_embeds = torch.cat(cur_new_input_embeds)
+            cur_new_labels = torch.cat(cur_new_labels)
+
+            if num_images == 0:
+                cur_new_input_embeds = torch.cat([cur_new_input_embeds, image_features[cur_image_idx][0:0]], dim=0)
+                cur_image_idx += 1
+
+            if num_speech == 0:
+                cur_new_input_embeds = torch.cat([cur_new_input_embeds, speech_features[cur_speech_idx][0:0]], dim=0)
+                cur_speech_idx += 1
+
+            new_input_embeds.append(cur_new_input_embeds)
+            new_labels.append(cur_new_labels)
+
+        # Truncate sequences to max length as speech features can make the sequence longer
+        tokenizer_model_max_length = getattr(self.config, 'tokenizer_model_max_length', None)
+        if tokenizer_model_max_length is not None:
+            new_input_embeds = [x[:tokenizer_model_max_length] for x in new_input_embeds]
+            new_labels = [x[:tokenizer_model_max_length] for x in new_labels]
+
+        # Combine them
+        max_len = max(x.shape[0] for x in new_input_embeds)
+        batch_size = len(new_input_embeds)
+
+        new_input_embeds_padded = []
+        new_labels_padded = torch.full((batch_size, max_len), IGNORE_INDEX, dtype=new_labels[0].dtype, device=new_labels[0].device)
+        attention_mask = torch.zeros((batch_size, max_len), dtype=attention_mask.dtype, device=attention_mask.device)
+        position_ids = torch.zeros((batch_size, max_len), dtype=position_ids.dtype, device=position_ids.device)
+
+        for i, (cur_new_embed, cur_new_labels) in enumerate(zip(new_input_embeds, new_labels)):
+            cur_len = cur_new_embed.shape[0]
+            if getattr(self.config, 'tokenizer_padding_side', 'right') == "left":
+                new_input_embeds_padded.append(torch.cat((
+                    torch.zeros((max_len - cur_len, cur_new_embed.shape[1]), dtype=cur_new_embed.dtype, device=cur_new_embed.device),
+                    cur_new_embed
+                ), dim=0))
+                if cur_len > 0:
+                    new_labels_padded[i, -cur_len:] = cur_new_labels
+                    attention_mask[i, -cur_len:] = True
+                    position_ids[i, -cur_len:] = torch.arange(0, cur_len, dtype=position_ids.dtype, device=position_ids.device)
+            else:
+                new_input_embeds_padded.append(torch.cat((
+                    cur_new_embed,
+                    torch.zeros((max_len - cur_len, cur_new_embed.shape[1]), dtype=cur_new_embed.dtype, device=cur_new_embed.device)
+                ), dim=0))
+                if cur_len > 0:
+                    new_labels_padded[i, :cur_len] = cur_new_labels
+                    attention_mask[i, :cur_len] = True
+                    position_ids[i, :cur_len] = torch.arange(0, cur_len, dtype=position_ids.dtype, device=position_ids.device)
+
+        new_input_embeds = torch.stack(new_input_embeds_padded, dim=0)
+
+        if _labels is None:
+            new_labels = None
+        else:
+            new_labels = new_labels_padded
+
+        if _attention_mask is None:
+            attention_mask = None
+        else:
+            attention_mask = attention_mask.to(dtype=_attention_mask.dtype)
+
+        if _position_ids is None:
+            position_ids = None
+
+        return None, position_ids, attention_mask, past_key_values, new_input_embeds, new_labels
+
+    def initialize_vision_tokenizer(self, model_args, tokenizer):
+        if model_args.mm_use_im_patch_token:
+            tokenizer.add_tokens([DEFAULT_IMAGE_PATCH_TOKEN], special_tokens=True)
+            self.resize_token_embeddings(len(tokenizer))
+
+        if model_args.mm_use_im_start_end:
+            num_new_tokens = tokenizer.add_tokens([DEFAULT_IM_START_TOKEN, DEFAULT_IM_END_TOKEN], special_tokens=True)
+            self.resize_token_embeddings(len(tokenizer))
+
+            if num_new_tokens > 0:
+                input_embeddings = self.get_input_embeddings().weight.data
+                output_embeddings = self.get_output_embeddings().weight.data
+
+                input_embeddings_avg = input_embeddings[:-num_new_tokens].mean(
+                    dim=0, keepdim=True)
+                output_embeddings_avg = output_embeddings[:-num_new_tokens].mean(
+                    dim=0, keepdim=True)
+
+                input_embeddings[-num_new_tokens:] = input_embeddings_avg
+                output_embeddings[-num_new_tokens:] = output_embeddings_avg
+
+            if model_args.tune_mm_mlp_adapter:
+                for p in self.get_input_embeddings().parameters():
+                    p.requires_grad = True
+                for p in self.get_output_embeddings().parameters():
+                    p.requires_grad = False
+
+            if model_args.pretrain_mm_mlp_adapter:
+                mm_projector_weights = torch.load(model_args.pretrain_mm_mlp_adapter, map_location='cpu')
+                embed_tokens_weight = mm_projector_weights['model.embed_tokens.weight']
+                assert num_new_tokens == 2
+                if input_embeddings.shape == embed_tokens_weight.shape:
+                    input_embeddings[-num_new_tokens:] = embed_tokens_weight[-num_new_tokens:]
+                elif embed_tokens_weight.shape[0] == num_new_tokens:
+                    input_embeddings[-num_new_tokens:] = embed_tokens_weight
+                else:
+                    raise ValueError(f"Unexpected embed_tokens_weight shape. Pretrained: {embed_tokens_weight.shape}. Current: {input_embeddings.shape}. Numer of new tokens: {num_new_tokens}.")
+        elif model_args.mm_use_im_patch_token:
+            if model_args.tune_mm_mlp_adapter:
+                for p in self.get_input_embeddings().parameters():
+                    p.requires_grad = False
+                for p in self.get_output_embeddings().parameters():
+                    p.requires_grad = False
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/BEATs.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/BEATs.py
new file mode 100644
index 0000000000000000000000000000000000000000..88cf3263cf7daea8142805cebf6b785e408f26c8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/BEATs.py
@@ -0,0 +1,183 @@
+# --------------------------------------------------------
+# BEATs: Audio Pre-Training with Acoustic Tokenizers (https://arxiv.org/abs/2212.09058)
+# Github source: https://github.com/microsoft/unilm/tree/master/beats
+# Copyright (c) 2022 Microsoft
+# Licensed under The MIT License [see LICENSE for details]
+# Based on fairseq code bases
+# https://github.com/pytorch/fairseq
+# --------------------------------------------------------
+
+
+import logging
+from typing import Optional
+
+import torch
+import torch.nn as nn
+from torch.nn import LayerNorm
+
+from .backbone import TransformerEncoder
+from .kaldi import fbank as kaldi_fbank
+
+# import torchaudio.compliance.kaldi as ta_kaldi
+
+
+
+
+logger = logging.getLogger(__name__)
+
+
+class BEATsConfig:
+    def __init__(self, cfg=None):
+        self.input_patch_size: int = -1  # path size of patch embedding
+        self.embed_dim: int = 512  # patch embedding dimension
+        self.conv_bias: bool = False  # include bias in conv encoder
+
+        self.encoder_layers: int = 12  # num encoder layers in the transformer
+        self.encoder_embed_dim: int = 768  # encoder embedding dimension
+        self.encoder_ffn_embed_dim: int = 3072  # encoder embedding dimension for FFN
+        self.encoder_attention_heads: int = 12  # num encoder attention heads
+        self.activation_fn: str = "gelu"  # activation function to use
+
+        self.layer_wise_gradient_decay_ratio: float = 1.0  # ratio for layer-wise gradient decay
+        self.layer_norm_first: bool = False  # apply layernorm first in the transformer
+        self.deep_norm: bool = False  # apply deep_norm first in the transformer
+
+        # dropouts
+        self.dropout: float = 0.1  # dropout probability for the transformer
+        self.attention_dropout: float = 0.1  # dropout probability for attention weights
+        self.activation_dropout: float = 0.0  # dropout probability after activation in FFN
+        self.encoder_layerdrop: float = 0.0  # probability of dropping a tarnsformer layer
+        self.dropout_input: float = 0.0  # dropout to apply to the input (after feat extr)
+
+        # positional embeddings
+        self.conv_pos: int = 128  # number of filters for convolutional positional embeddings
+        self.conv_pos_groups: int = 16  # number of groups for convolutional positional embedding
+
+        # relative position embedding
+        self.relative_position_embedding: bool = False  # apply relative position embedding
+        self.num_buckets: int = 320  # number of buckets for relative position embedding
+        self.max_distance: int = 1280  # maximum distance for relative position embedding
+        self.gru_rel_pos: bool = False  # apply gated relative position embedding
+
+        # label predictor
+        self.finetuned_model: bool = False  # whether the model is a fine-tuned model.
+        self.predictor_dropout: float = 0.1  # dropout probability for the predictor
+        self.predictor_class: int = 527  # target class number for the predictor
+
+        if cfg is not None:
+            self.update(cfg)
+
+    def update(self, cfg: dict):
+        self.__dict__.update(cfg)
+
+
+class BEATs(nn.Module):
+    def __init__(
+            self,
+            cfg: BEATsConfig,
+    ) -> None:
+        super().__init__()
+        logger.info(f"BEATs Config: {cfg.__dict__}")
+
+        self.cfg = cfg
+
+        self.embed = cfg.embed_dim
+        self.post_extract_proj = (
+            nn.Linear(self.embed, cfg.encoder_embed_dim)
+            if self.embed != cfg.encoder_embed_dim
+            else None
+        )
+
+        self.input_patch_size = cfg.input_patch_size
+        self.patch_embedding = nn.Conv2d(1, self.embed, kernel_size=self.input_patch_size, stride=self.input_patch_size,
+                                         bias=cfg.conv_bias)
+
+        self.dropout_input = nn.Dropout(cfg.dropout_input)
+
+        assert not cfg.deep_norm or not cfg.layer_norm_first
+        self.encoder = TransformerEncoder(cfg)
+        self.layer_norm = LayerNorm(self.embed)
+
+        if cfg.finetuned_model:
+            self.predictor_dropout = nn.Dropout(cfg.predictor_dropout)
+            self.predictor = nn.Linear(cfg.encoder_embed_dim, cfg.predictor_class)
+        else:
+            self.predictor = None
+
+    def forward_padding_mask(
+            self,
+            features: torch.Tensor,
+            padding_mask: torch.Tensor,
+    ) -> torch.Tensor:
+        extra = padding_mask.size(1) % features.size(1)
+        if extra > 0:
+            padding_mask = padding_mask[:, :-extra]
+        padding_mask = padding_mask.view(
+            padding_mask.size(0), features.size(1), -1
+        )
+        padding_mask = padding_mask.all(-1)
+        return padding_mask
+
+    def preprocess(
+            self,
+            source: torch.Tensor,
+            fbank_mean: float = 15.41663,
+            fbank_std: float = 6.55582,
+    ) -> torch.Tensor:
+        fbanks = []
+        for waveform in source:
+            waveform = waveform.unsqueeze(0) * 2 ** 15
+            fbank = kaldi_fbank(waveform, num_mel_bins=128, sample_frequency=16000, frame_length=25, frame_shift=10)
+            fbanks.append(fbank)
+        fbank = torch.stack(fbanks, dim=0)
+        fbank = (fbank - fbank_mean) / (2 * fbank_std)
+        return fbank
+
+    def extract_features(
+            self,
+            source: torch.Tensor,
+            padding_mask: Optional[torch.Tensor] = None,
+            fbank_mean: float = 15.41663,
+            fbank_std: float = 6.55582,
+            feature_only=False,
+    ):
+        fbank = self.preprocess(source, fbank_mean=fbank_mean, fbank_std=fbank_std).to(torch.float32)
+
+        if padding_mask is not None:
+            padding_mask = self.forward_padding_mask(fbank, padding_mask)
+
+        fbank = fbank.unsqueeze(1)
+        features = self.patch_embedding(fbank)
+        features = features.reshape(features.shape[0], features.shape[1], -1)
+        features = features.transpose(1, 2)
+        features = self.layer_norm(features)
+
+        if padding_mask is not None:
+            padding_mask = self.forward_padding_mask(features, padding_mask)
+
+        if self.post_extract_proj is not None:
+            features = self.post_extract_proj(features)
+
+        x = self.dropout_input(features)
+
+        x, layer_results = self.encoder(
+            x,
+            padding_mask=padding_mask,
+        )
+
+        if not feature_only and self.predictor is not None:
+            x = self.predictor_dropout(x)
+            logits = self.predictor(x)
+
+            if padding_mask is not None and padding_mask.any():
+                logits[padding_mask] = 0
+                logits = logits.sum(dim=1)
+                logits = logits / (~padding_mask).sum(dim=1).unsqueeze(-1).expand_as(logits)
+            else:
+                logits = logits.mean(dim=1)
+
+            lprobs = torch.sigmoid(logits)
+
+            return lprobs, padding_mask
+        else:
+            return x, padding_mask
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/Tokenizers.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/Tokenizers.py
new file mode 100644
index 0000000000000000000000000000000000000000..f452beb5f6d68742cd899727f9266e05c2e6926c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/Tokenizers.py
@@ -0,0 +1,173 @@
+# --------------------------------------------------------
+# BEATs: Audio Pre-Training with Acoustic Tokenizers (https://arxiv.org/abs/2212.09058)
+# Github source: https://github.com/microsoft/unilm/tree/master/beats
+# Copyright (c) 2022 Microsoft
+# Licensed under The MIT License [see LICENSE for details]
+# Based on fairseq code bases
+# https://github.com/pytorch/fairseq
+# --------------------------------------------------------
+
+
+import logging
+from typing import Optional
+
+import torch
+import torch.nn as nn
+from torch.nn import LayerNorm
+
+from .backbone import TransformerEncoder
+from .kaldi import fbank as kaldi_fbank
+from .quantizer import NormEMAVectorQuantizer
+
+# import torchaudio.compliance.kaldi as ta_kaldi
+
+
+
+
+logger = logging.getLogger(__name__)
+
+
+class TokenizersConfig:
+    def __init__(self, cfg=None):
+        self.input_patch_size: int = -1  # path size of patch embedding
+        self.embed_dim: int = 512  # patch embedding dimension
+        self.conv_bias: bool = False  # include bias in conv encoder
+
+        self.encoder_layers: int = 12  # num encoder layers in the transformer
+        self.encoder_embed_dim: int = 768  # encoder embedding dimension
+        self.encoder_ffn_embed_dim: int = 3072  # encoder embedding dimension for FFN
+        self.encoder_attention_heads: int = 12  # num encoder attention heads
+        self.activation_fn: str = "gelu"  # activation function to use
+
+        self.layer_norm_first: bool = False  # apply layernorm first in the transformer
+        self.deep_norm: bool = False  # apply deep_norm first in the transformer
+
+        # dropouts
+        self.dropout: float = 0.1  # dropout probability for the transformer
+        self.attention_dropout: float = 0.1  # dropout probability for attention weights
+        self.activation_dropout: float = 0.0  # dropout probability after activation in FFN
+        self.encoder_layerdrop: float = 0.0  # probability of dropping a tarnsformer layer
+        self.dropout_input: float = 0.0  # dropout to apply to the input (after feat extr)
+
+        # positional embeddings
+        self.conv_pos: int = 128  # number of filters for convolutional positional embeddings
+        self.conv_pos_groups: int = 16  # number of groups for convolutional positional embedding
+
+        # relative position embedding
+        self.relative_position_embedding: bool = False  # apply relative position embedding
+        self.num_buckets: int = 320  # number of buckets for relative position embedding
+        self.max_distance: int = 1280  # maximum distance for relative position embedding
+        self.gru_rel_pos: bool = False  # apply gated relative position embedding
+
+        # quantizer
+        self.quant_n: int = 1024 # codebook number in quantizer
+        self.quant_dim: int = 256    # codebook dimension in quantizer
+
+        if cfg is not None:
+            self.update(cfg)
+
+    def update(self, cfg: dict):
+        self.__dict__.update(cfg)
+
+
+class Tokenizers(nn.Module):
+    def __init__(
+            self,
+            cfg: TokenizersConfig,
+    ) -> None:
+        super().__init__()
+        logger.info(f"Tokenizers Config: {cfg.__dict__}")
+
+        self.cfg = cfg
+
+        self.embed = cfg.embed_dim
+        self.post_extract_proj = (
+            nn.Linear(self.embed, cfg.encoder_embed_dim)
+            if self.embed != cfg.encoder_embed_dim
+            else None
+        )
+
+        self.input_patch_size = cfg.input_patch_size
+        self.patch_embedding = nn.Conv2d(1, self.embed, kernel_size=self.input_patch_size, stride=self.input_patch_size,
+                                         bias=cfg.conv_bias)
+
+        self.dropout_input = nn.Dropout(cfg.dropout_input)
+
+        assert not cfg.deep_norm or not cfg.layer_norm_first
+        self.encoder = TransformerEncoder(cfg)
+        self.layer_norm = LayerNorm(self.embed)
+
+        self.quantize = NormEMAVectorQuantizer(
+            n_embed=cfg.quant_n, embedding_dim=cfg.quant_dim, beta=1.0, kmeans_init=True, decay=0.99,
+        )
+        self.quant_n = cfg.quant_n
+        self.quantize_layer = nn.Sequential(
+            nn.Linear(cfg.encoder_embed_dim, cfg.encoder_embed_dim),
+            nn.Tanh(),
+            nn.Linear(cfg.encoder_embed_dim, cfg.quant_dim)  # for quantize
+        )
+
+    def forward_padding_mask(
+            self,
+            features: torch.Tensor,
+            padding_mask: torch.Tensor,
+    ) -> torch.Tensor:
+        extra = padding_mask.size(1) % features.size(1)
+        if extra > 0:
+            padding_mask = padding_mask[:, :-extra]
+        padding_mask = padding_mask.view(
+            padding_mask.size(0), features.size(1), -1
+        )
+        padding_mask = padding_mask.all(-1)
+        return padding_mask
+
+    def preprocess(
+            self,
+            source: torch.Tensor,
+            fbank_mean: float = 15.41663,
+            fbank_std: float = 6.55582,
+    ) -> torch.Tensor:
+        fbanks = []
+        for waveform in source:
+            waveform = waveform.unsqueeze(0) * 2 ** 15
+            fbank = kaldi_fbank(waveform, num_mel_bins=128, sample_frequency=16000, frame_length=25, frame_shift=10)
+            fbanks.append(fbank)
+        fbank = torch.stack(fbanks, dim=0)
+        fbank = (fbank - fbank_mean) / (2 * fbank_std)
+        return fbank
+
+    def extract_labels(
+            self,
+            source: torch.Tensor,
+            padding_mask: Optional[torch.Tensor] = None,
+            fbank_mean: float = 15.41663,
+            fbank_std: float = 6.55582,
+    ):
+        fbank = self.preprocess(source, fbank_mean=fbank_mean, fbank_std=fbank_std)
+
+        if padding_mask is not None:
+            padding_mask = self.forward_padding_mask(fbank, padding_mask)
+
+        fbank = fbank.unsqueeze(1)
+        features = self.patch_embedding(fbank)
+        features = features.reshape(features.shape[0], features.shape[1], -1)
+        features = features.transpose(1, 2)
+        features = self.layer_norm(features)
+
+        if padding_mask is not None:
+            padding_mask = self.forward_padding_mask(features, padding_mask)
+
+        if self.post_extract_proj is not None:
+            features = self.post_extract_proj(features)
+
+        x = self.dropout_input(features)
+
+        x, layer_results = self.encoder(
+            x,
+            padding_mask=padding_mask,
+        )
+
+        quantize_input = self.quantize_layer(x)
+        quantize_feature, embed_loss, embed_ind = self.quantize(quantize_input)
+
+        return embed_ind
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/backbone.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/backbone.py
new file mode 100644
index 0000000000000000000000000000000000000000..c4e7e043c35939fb73cdf3845baf49a61fd5d81e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/backbone.py
@@ -0,0 +1,778 @@
+# --------------------------------------------------------
+# BEATs: Audio Pre-Training with Acoustic Tokenizers (https://arxiv.org/abs/2212.09058)
+# Github source: https://github.com/microsoft/unilm/tree/master/beats
+# Copyright (c) 2022 Microsoft
+# Licensed under The MIT License [see LICENSE for details]
+# Based on fairseq code bases
+# https://github.com/pytorch/fairseq
+# --------------------------------------------------------
+
+import math
+from typing import Dict, Optional, Tuple
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+from torch import Tensor, nn
+from torch.nn import LayerNorm, Parameter
+
+from .modules import GLU_Linear, GradMultiply, SamePad, get_activation_fn, quant_noise
+
+
+class TransformerEncoder(nn.Module):
+    def __init__(self, args):
+        super().__init__()
+
+        self.dropout = args.dropout
+        self.embedding_dim = args.encoder_embed_dim
+
+        self.pos_conv = nn.Conv1d(
+            self.embedding_dim,
+            self.embedding_dim,
+            kernel_size=args.conv_pos,
+            padding=args.conv_pos // 2,
+            groups=args.conv_pos_groups,
+        )
+        dropout = 0
+        std = math.sqrt((4 * (1.0 - dropout)) / (args.conv_pos * self.embedding_dim))
+        nn.init.normal_(self.pos_conv.weight, mean=0, std=std)
+        nn.init.constant_(self.pos_conv.bias, 0)
+
+        self.pos_conv = nn.utils.weight_norm(self.pos_conv, name="weight", dim=2)
+        self.pos_conv = nn.Sequential(self.pos_conv, SamePad(args.conv_pos), nn.GELU())
+
+        if hasattr(args, "relative_position_embedding"):
+            self.relative_position_embedding = args.relative_position_embedding
+            self.num_buckets = args.num_buckets
+            self.max_distance = args.max_distance
+        else:
+            self.relative_position_embedding = False
+            self.num_buckets = 0
+            self.max_distance = 0
+
+        self.layers = nn.ModuleList(
+            [
+                TransformerSentenceEncoderLayer(
+                    embedding_dim=self.embedding_dim,
+                    ffn_embedding_dim=args.encoder_ffn_embed_dim,
+                    num_attention_heads=args.encoder_attention_heads,
+                    dropout=self.dropout,
+                    attention_dropout=args.attention_dropout,
+                    activation_dropout=args.activation_dropout,
+                    activation_fn=args.activation_fn,
+                    layer_norm_first=args.layer_norm_first,
+                    deep_norm=args.deep_norm,
+                    has_relative_attention_bias=self.relative_position_embedding,
+                    num_buckets=self.num_buckets,
+                    max_distance=self.max_distance,
+                    gru_rel_pos=args.gru_rel_pos,
+                    encoder_layers=args.encoder_layers,
+                )
+                for i in range(args.encoder_layers)
+            ]
+        )
+        if self.relative_position_embedding:
+            for i in range(1, args.encoder_layers):
+                del self.layers[i].self_attn.relative_attention_bias
+                self.layers[i].self_attn.relative_attention_bias = self.layers[0].self_attn.relative_attention_bias
+
+        self.layer_norm_first = args.layer_norm_first
+        self.layer_norm = LayerNorm(self.embedding_dim)
+        self.layerdrop = args.encoder_layerdrop
+
+        self.apply(init_bert_params)
+
+        if args.deep_norm:
+            deep_norm_beta = math.pow(8 * args.encoder_layers, -1 / 4)
+            for i in range(args.encoder_layers):
+                nn.init.xavier_normal_(self.layers[i].self_attn.k_proj.weight, gain=1)
+                nn.init.xavier_normal_(self.layers[i].self_attn.v_proj.weight, gain=deep_norm_beta)
+                nn.init.xavier_normal_(self.layers[i].self_attn.q_proj.weight, gain=1)
+                nn.init.xavier_normal_(self.layers[i].self_attn.out_proj.weight, gain=deep_norm_beta)
+                nn.init.xavier_normal_(self.layers[i].fc1.weight, gain=deep_norm_beta)
+                nn.init.xavier_normal_(self.layers[i].fc2.weight, gain=deep_norm_beta)
+
+        self.layer_wise_gradient_decay_ratio = getattr(args, "layer_wise_gradient_decay_ratio", 1)
+
+    def forward(self, x, padding_mask=None, layer=None):
+        x, layer_results = self.extract_features(x, padding_mask, layer)
+
+        if self.layer_norm_first and layer is None:
+            x = self.layer_norm(x)
+
+        return x, layer_results
+
+    def extract_features(self, x, padding_mask=None, tgt_layer=None):
+
+        if padding_mask is not None:
+            x[padding_mask] = 0
+        x_conv = self.pos_conv(x.transpose(1, 2))
+        x_conv = x_conv.transpose(1, 2)
+        x = x + x_conv
+
+        if not self.layer_norm_first:
+            x = self.layer_norm(x)
+
+        x = F.dropout(x, p=self.dropout, training=self.training)
+
+        # B x T x C -> T x B x C
+        x = x.transpose(0, 1)
+
+        layer_results = []
+        z = None
+        if tgt_layer is not None:
+            layer_results.append((x, z))
+        r = None
+        pos_bias = None
+        for i, layer in enumerate(self.layers):
+            if self.layer_wise_gradient_decay_ratio != 1.0:
+                x = GradMultiply.apply(x, self.layer_wise_gradient_decay_ratio)
+            dropout_probability = np.random.random()
+            if not self.training or (dropout_probability > self.layerdrop):
+                x, z, pos_bias = layer(x, self_attn_padding_mask=padding_mask, need_weights=False, pos_bias=pos_bias)
+            if tgt_layer is not None:
+                layer_results.append((x, z))
+            if i == tgt_layer:
+                r = x
+                break
+
+        if r is not None:
+            x = r
+
+        # T x B x C -> B x T x C
+        x = x.transpose(0, 1)
+
+        return x, layer_results
+
+
+class TransformerSentenceEncoderLayer(nn.Module):
+    def __init__(
+            self,
+            embedding_dim: float = 768,
+            ffn_embedding_dim: float = 3072,
+            num_attention_heads: float = 8,
+            dropout: float = 0.1,
+            attention_dropout: float = 0.1,
+            activation_dropout: float = 0.1,
+            activation_fn: str = "relu",
+            layer_norm_first: bool = False,
+            deep_norm: bool = False,
+            has_relative_attention_bias: bool = False,
+            num_buckets: int = 0,
+            max_distance: int = 0,
+            rescale_init: bool = False,
+            gru_rel_pos: bool = False,
+            encoder_layers: int = 0,
+    ) -> None:
+
+        super().__init__()
+        self.embedding_dim = embedding_dim
+        self.dropout = dropout
+        self.activation_dropout = activation_dropout
+
+        self.activation_name = activation_fn
+        self.activation_fn = get_activation_fn(activation_fn)
+        self.self_attn = MultiheadAttention(
+            self.embedding_dim,
+            num_attention_heads,
+            dropout=attention_dropout,
+            self_attention=True,
+            has_relative_attention_bias=has_relative_attention_bias,
+            num_buckets=num_buckets,
+            max_distance=max_distance,
+            rescale_init=rescale_init,
+            gru_rel_pos=gru_rel_pos,
+        )
+
+        self.dropout1 = nn.Dropout(dropout)
+        self.dropout2 = nn.Dropout(self.activation_dropout)
+        self.dropout3 = nn.Dropout(dropout)
+
+        self.layer_norm_first = layer_norm_first
+
+        self.self_attn_layer_norm = LayerNorm(self.embedding_dim)
+
+        if self.activation_name == "glu":
+            self.fc1 = GLU_Linear(self.embedding_dim, ffn_embedding_dim, "swish")
+        else:
+            self.fc1 = nn.Linear(self.embedding_dim, ffn_embedding_dim)
+        self.fc2 = nn.Linear(ffn_embedding_dim, self.embedding_dim)
+
+        self.final_layer_norm = LayerNorm(self.embedding_dim)
+
+        self.deep_norm = deep_norm
+        if self.deep_norm:
+            self.deep_norm_alpha = math.pow(2 * encoder_layers, 1 / 4)
+        else:
+            self.deep_norm_alpha = 1
+
+    def forward(
+            self,
+            x: torch.Tensor,
+            self_attn_mask: torch.Tensor = None,
+            self_attn_padding_mask: torch.Tensor = None,
+            need_weights: bool = False,
+            pos_bias=None
+    ):
+        residual = x
+
+        if self.layer_norm_first:
+            x = self.self_attn_layer_norm(x)
+            x, attn, pos_bias = self.self_attn(
+                query=x,
+                key=x,
+                value=x,
+                key_padding_mask=self_attn_padding_mask,
+                need_weights=False,
+                attn_mask=self_attn_mask,
+                position_bias=pos_bias
+            )
+            x = self.dropout1(x)
+            x = residual + x
+
+            residual = x
+            x = self.final_layer_norm(x)
+            if self.activation_name == "glu":
+                x = self.fc1(x)
+            else:
+                x = self.activation_fn(self.fc1(x))
+            x = self.dropout2(x)
+            x = self.fc2(x)
+            x = self.dropout3(x)
+            x = residual + x
+        else:
+            x, attn, pos_bias = self.self_attn(
+                query=x,
+                key=x,
+                value=x,
+                key_padding_mask=self_attn_padding_mask,
+                need_weights=need_weights,
+                attn_mask=self_attn_mask,
+                position_bias=pos_bias
+            )
+
+            x = self.dropout1(x)
+            x = residual * self.deep_norm_alpha + x
+
+            x = self.self_attn_layer_norm(x)
+
+            residual = x
+            if self.activation_name == "glu":
+                x = self.fc1(x)
+            else:
+                x = self.activation_fn(self.fc1(x))
+            x = self.dropout2(x)
+            x = self.fc2(x)
+            x = self.dropout3(x)
+            x = residual * self.deep_norm_alpha + x
+            x = self.final_layer_norm(x)
+
+        return x, attn, pos_bias
+
+
+class MultiheadAttention(nn.Module):
+    """Multi-headed attention.
+
+    See "Attention Is All You Need" for more details.
+    """
+
+    def __init__(
+            self,
+            embed_dim,
+            num_heads,
+            kdim=None,
+            vdim=None,
+            dropout=0.0,
+            bias=True,
+            add_bias_kv=False,
+            add_zero_attn=False,
+            self_attention=False,
+            encoder_decoder_attention=False,
+            q_noise=0.0,
+            qn_block_size=8,
+            has_relative_attention_bias=False,
+            num_buckets=32,
+            max_distance=128,
+            gru_rel_pos=False,
+            rescale_init=False,
+    ):
+        super().__init__()
+        self.embed_dim = embed_dim
+        self.kdim = kdim if kdim is not None else embed_dim
+        self.vdim = vdim if vdim is not None else embed_dim
+        self.qkv_same_dim = self.kdim == embed_dim and self.vdim == embed_dim
+
+        self.num_heads = num_heads
+        self.dropout_module = nn.Dropout(dropout)
+
+        self.has_relative_attention_bias = has_relative_attention_bias
+        self.num_buckets = num_buckets
+        self.max_distance = max_distance
+        if self.has_relative_attention_bias:
+            self.relative_attention_bias = nn.Embedding(num_buckets, num_heads)
+
+        self.head_dim = embed_dim // num_heads
+        self.q_head_dim = self.head_dim
+        self.k_head_dim = self.head_dim
+        assert (
+                self.head_dim * num_heads == self.embed_dim
+        ), "embed_dim must be divisible by num_heads"
+        self.scaling = self.head_dim ** -0.5
+
+        self.self_attention = self_attention
+        self.encoder_decoder_attention = encoder_decoder_attention
+
+        assert not self.self_attention or self.qkv_same_dim, (
+            "Self-attention requires query, key and " "value to be of the same size"
+        )
+
+        k_bias = True
+        if rescale_init:
+            k_bias = False
+
+        k_embed_dim = embed_dim
+        q_embed_dim = embed_dim
+
+        self.k_proj = quant_noise(
+            nn.Linear(self.kdim, k_embed_dim, bias=k_bias), q_noise, qn_block_size
+        )
+        self.v_proj = quant_noise(
+            nn.Linear(self.vdim, embed_dim, bias=bias), q_noise, qn_block_size
+        )
+        self.q_proj = quant_noise(
+            nn.Linear(embed_dim, q_embed_dim, bias=bias), q_noise, qn_block_size
+        )
+
+        self.out_proj = quant_noise(
+            nn.Linear(embed_dim, embed_dim, bias=bias), q_noise, qn_block_size
+        )
+
+        if add_bias_kv:
+            self.bias_k = Parameter(torch.Tensor(1, 1, embed_dim))
+            self.bias_v = Parameter(torch.Tensor(1, 1, embed_dim))
+        else:
+            self.bias_k = self.bias_v = None
+
+        self.add_zero_attn = add_zero_attn
+
+        self.gru_rel_pos = gru_rel_pos
+        if self.gru_rel_pos:
+            self.grep_linear = nn.Linear(self.q_head_dim, 8)
+            self.grep_a = nn.Parameter(torch.ones(1, num_heads, 1, 1))
+
+        self.reset_parameters()
+
+    def reset_parameters(self):
+        if self.qkv_same_dim:
+            # Empirically observed the convergence to be much better with
+            # the scaled initialization
+            nn.init.xavier_uniform_(self.k_proj.weight, gain=1 / math.sqrt(2))
+            nn.init.xavier_uniform_(self.v_proj.weight, gain=1 / math.sqrt(2))
+            nn.init.xavier_uniform_(self.q_proj.weight, gain=1 / math.sqrt(2))
+        else:
+            nn.init.xavier_uniform_(self.k_proj.weight)
+            nn.init.xavier_uniform_(self.v_proj.weight)
+            nn.init.xavier_uniform_(self.q_proj.weight)
+
+        nn.init.xavier_uniform_(self.out_proj.weight)
+        if self.out_proj.bias is not None:
+            nn.init.constant_(self.out_proj.bias, 0.0)
+        if self.bias_k is not None:
+            nn.init.xavier_normal_(self.bias_k)
+        if self.bias_v is not None:
+            nn.init.xavier_normal_(self.bias_v)
+        if self.has_relative_attention_bias:
+            nn.init.xavier_normal_(self.relative_attention_bias.weight)
+
+    def _relative_positions_bucket(self, relative_positions, bidirectional=True):
+        num_buckets = self.num_buckets
+        max_distance = self.max_distance
+        relative_buckets = 0
+
+        if bidirectional:
+            num_buckets = num_buckets // 2
+            relative_buckets += (relative_positions > 0).to(torch.long) * num_buckets
+            relative_positions = torch.abs(relative_positions)
+        else:
+            relative_positions = -torch.min(relative_positions, torch.zeros_like(relative_positions))
+
+        max_exact = num_buckets // 2
+        is_small = relative_positions < max_exact
+
+        relative_postion_if_large = max_exact + (
+                torch.log(relative_positions.float() / max_exact)
+                / math.log(max_distance / max_exact)
+                * (num_buckets - max_exact)
+        ).to(torch.long)
+        relative_postion_if_large = torch.min(
+            relative_postion_if_large, torch.full_like(relative_postion_if_large, num_buckets - 1)
+        )
+
+        relative_buckets += torch.where(is_small, relative_positions, relative_postion_if_large)
+        return relative_buckets
+
+    def compute_bias(self, query_length, key_length):
+        context_position = torch.arange(query_length, dtype=torch.long)[:, None]
+        memory_position = torch.arange(key_length, dtype=torch.long)[None, :]
+        relative_position = memory_position - context_position
+        relative_position_bucket = self._relative_positions_bucket(
+            relative_position,
+            bidirectional=True
+        )
+        relative_position_bucket = relative_position_bucket.to(self.relative_attention_bias.weight.device)
+        values = self.relative_attention_bias(relative_position_bucket)
+        values = values.permute([2, 0, 1])
+        return values
+
+    def forward(
+            self,
+            query,
+            key: Optional[Tensor],
+            value: Optional[Tensor],
+            key_padding_mask: Optional[Tensor] = None,
+            incremental_state: Optional[Dict[str, Dict[str, Optional[Tensor]]]] = None,
+            need_weights: bool = True,
+            static_kv: bool = False,
+            attn_mask: Optional[Tensor] = None,
+            before_softmax: bool = False,
+            need_head_weights: bool = False,
+            position_bias: Optional[Tensor] = None
+    ) -> Tuple[Tensor, Optional[Tensor], Optional[Tensor]]:
+        """Input shape: Time x Batch x Channel
+
+        Args:
+            key_padding_mask (ByteTensor, optional): mask to exclude
+                keys that are pads, of shape `(batch, src_len)`, where
+                padding elements are indicated by 1s.
+            need_weights (bool, optional): return the attention weights,
+                averaged over heads (default: False).
+            attn_mask (ByteTensor, optional): typically used to
+                implement causal attention, where the mask prevents the
+                attention from looking forward in time (default: None).
+            before_softmax (bool, optional): return the raw attention
+                weights and values before the attention softmax.
+            need_head_weights (bool, optional): return the attention
+                weights for each head. Implies *need_weights*. Default:
+                return the average attention weights over all heads.
+        """
+        if need_head_weights:
+            need_weights = True
+
+        is_tpu = query.device.type == "xla"
+
+        tgt_len, bsz, embed_dim = query.size()
+        src_len = tgt_len
+        assert embed_dim == self.embed_dim
+        assert list(query.size()) == [tgt_len, bsz, embed_dim]
+        if key is not None:
+            src_len, key_bsz, _ = key.size()
+            if not torch.jit.is_scripting():
+                assert key_bsz == bsz
+                assert value is not None
+                assert src_len, bsz == value.shape[:2]
+
+        if self.has_relative_attention_bias and position_bias is None:
+            position_bias = self.compute_bias(tgt_len, src_len)
+            position_bias = position_bias.unsqueeze(0).repeat(bsz, 1, 1, 1).view(bsz * self.num_heads, tgt_len, src_len)
+
+        if incremental_state is not None:
+            saved_state = self._get_input_buffer(incremental_state)
+            if saved_state is not None and "prev_key" in saved_state:
+                # previous time steps are cached - no need to recompute
+                # key and value if they are static
+                if static_kv:
+                    assert self.encoder_decoder_attention and not self.self_attention
+                    key = value = None
+        else:
+            saved_state = None
+
+        if self.self_attention:
+            q = self.q_proj(query)
+            k = self.k_proj(query)
+            v = self.v_proj(query)
+        elif self.encoder_decoder_attention:
+            # encoder-decoder attention
+            q = self.q_proj(query)
+            if key is None:
+                assert value is None
+                k = v = None
+            else:
+                k = self.k_proj(key)
+                v = self.v_proj(key)
+
+        else:
+            assert key is not None and value is not None
+            q = self.q_proj(query)
+            k = self.k_proj(key)
+            v = self.v_proj(value)
+        q *= self.scaling
+        alpha = 32
+        q *= 1 / alpha
+
+        if self.bias_k is not None:
+            assert self.bias_v is not None
+            k = torch.cat([k, self.bias_k.repeat(1, bsz, 1)])
+            v = torch.cat([v, self.bias_v.repeat(1, bsz, 1)])
+            if attn_mask is not None:
+                attn_mask = torch.cat(
+                    [attn_mask, attn_mask.new_zeros(attn_mask.size(0), 1)], dim=1
+                )
+            if key_padding_mask is not None:
+                key_padding_mask = torch.cat(
+                    [
+                        key_padding_mask,
+                        key_padding_mask.new_zeros(key_padding_mask.size(0), 1),
+                    ],
+                    dim=1,
+                )
+
+        q = (
+            q.contiguous()
+                .view(tgt_len, bsz * self.num_heads, self.q_head_dim)
+                .transpose(0, 1)
+        )
+        if k is not None:
+            k = (
+                k.contiguous()
+                    .view(-1, bsz * self.num_heads, self.k_head_dim)
+                    .transpose(0, 1)
+            )
+        if v is not None:
+            v = (
+                v.contiguous()
+                    .view(-1, bsz * self.num_heads, self.head_dim)
+                    .transpose(0, 1)
+            )
+
+        if saved_state is not None:
+            # saved states are stored with shape (bsz, num_heads, seq_len, head_dim)
+            if "prev_key" in saved_state:
+                _prev_key = saved_state["prev_key"]
+                assert _prev_key is not None
+                prev_key = _prev_key.view(bsz * self.num_heads, -1, self.head_dim)
+                if static_kv:
+                    k = prev_key
+                else:
+                    assert k is not None
+                    k = torch.cat([prev_key, k], dim=1)
+                src_len = k.size(1)
+            if "prev_value" in saved_state:
+                _prev_value = saved_state["prev_value"]
+                assert _prev_value is not None
+                prev_value = _prev_value.view(bsz * self.num_heads, -1, self.head_dim)
+                if static_kv:
+                    v = prev_value
+                else:
+                    assert v is not None
+                    v = torch.cat([prev_value, v], dim=1)
+            prev_key_padding_mask: Optional[Tensor] = None
+            if "prev_key_padding_mask" in saved_state:
+                prev_key_padding_mask = saved_state["prev_key_padding_mask"]
+            assert k is not None and v is not None
+            key_padding_mask = MultiheadAttention._append_prev_key_padding_mask(
+                key_padding_mask=key_padding_mask,
+                prev_key_padding_mask=prev_key_padding_mask,
+                batch_size=bsz,
+                src_len=k.size(1),
+                static_kv=static_kv,
+            )
+
+            saved_state["prev_key"] = k.view(bsz, self.num_heads, -1, self.head_dim)
+            saved_state["prev_value"] = v.view(bsz, self.num_heads, -1, self.head_dim)
+            saved_state["prev_key_padding_mask"] = key_padding_mask
+            # In this branch incremental_state is never None
+            assert incremental_state is not None
+            incremental_state = self._set_input_buffer(incremental_state, saved_state)
+        assert k is not None
+        assert k.size(1) == src_len
+
+        # This is part of a workaround to get around fork/join parallelism
+        # not supporting Optional types.
+        if key_padding_mask is not None and key_padding_mask.dim() == 0:
+            key_padding_mask = None
+
+        if key_padding_mask is not None:
+            assert key_padding_mask.size(0) == bsz
+            assert key_padding_mask.size(1) == src_len
+
+        if self.add_zero_attn:
+            assert v is not None
+            src_len += 1
+            k = torch.cat([k, k.new_zeros((k.size(0), 1) + k.size()[2:])], dim=1)
+            v = torch.cat([v, v.new_zeros((v.size(0), 1) + v.size()[2:])], dim=1)
+            if attn_mask is not None:
+                attn_mask = torch.cat(
+                    [attn_mask, attn_mask.new_zeros(attn_mask.size(0), 1)], dim=1
+                )
+            if key_padding_mask is not None:
+                key_padding_mask = torch.cat(
+                    [
+                        key_padding_mask,
+                        torch.zeros(key_padding_mask.size(0), 1).type_as(
+                            key_padding_mask
+                        ),
+                    ],
+                    dim=1,
+                )
+
+        attn_weights = torch.bmm(q, k.transpose(1, 2))
+        attn_weights = (attn_weights - attn_weights.max(dim=-1, keepdim=True)[0]) * alpha
+        attn_weights = self.apply_sparse_mask(attn_weights, tgt_len, src_len, bsz)
+
+        assert list(attn_weights.size()) == [bsz * self.num_heads, tgt_len, src_len]
+
+        if attn_mask is not None:
+            attn_mask = attn_mask.unsqueeze(0)
+            attn_weights += attn_mask
+
+        if key_padding_mask is not None:
+            # don't attend to padding symbols
+            attn_weights = attn_weights.view(bsz, self.num_heads, tgt_len, src_len)
+            if not is_tpu:
+                attn_weights = attn_weights.masked_fill(
+                    key_padding_mask.unsqueeze(1).unsqueeze(2).to(torch.bool),
+                    float("-inf"),
+                )
+            else:
+                attn_weights = attn_weights.transpose(0, 2)
+                attn_weights = attn_weights.masked_fill(key_padding_mask, float("-inf"))
+                attn_weights = attn_weights.transpose(0, 2)
+            attn_weights = attn_weights.view(bsz * self.num_heads, tgt_len, src_len)
+
+        if before_softmax:
+            return attn_weights, v, position_bias
+
+        if position_bias is not None:
+            attn_mask_rel_pos = position_bias
+            if self.gru_rel_pos == 1:
+                query_layer = q.view(bsz, self.num_heads, tgt_len, self.q_head_dim) * alpha / self.scaling
+                _B, _H, _L, __ = query_layer.size()
+                gate_a, gate_b = torch.sigmoid(self.grep_linear(query_layer).view(
+                    _B, _H, _L, 2, 4).sum(-1, keepdim=False)).chunk(2, dim=-1)
+                gate_a_1 = gate_a * (gate_b * self.grep_a - 1.0) + 2.0
+                attn_mask_rel_pos = gate_a_1.view(bsz * self.num_heads, tgt_len, 1) * position_bias
+
+            attn_mask_rel_pos = attn_mask_rel_pos.view(attn_weights.size())
+
+            attn_weights = attn_weights + attn_mask_rel_pos
+
+        attn_weights_float = F.softmax(
+            attn_weights, dim=-1
+        )
+        attn_weights = attn_weights_float.type_as(attn_weights)
+        attn_probs = self.dropout_module(attn_weights)
+
+        assert v is not None
+        attn = torch.bmm(attn_probs, v)
+        assert list(attn.size()) == [bsz * self.num_heads, tgt_len, self.head_dim]
+        attn = attn.transpose(0, 1).contiguous().view(tgt_len, bsz, embed_dim)
+        attn = self.out_proj(attn)
+        attn_weights: Optional[Tensor] = None
+        if need_weights:
+            attn_weights = attn_weights_float.view(
+                bsz, self.num_heads, tgt_len, src_len
+            ).transpose(1, 0)
+            if not need_head_weights:
+                # average attention weights over heads
+                attn_weights = attn_weights.mean(dim=0)
+
+        return attn, attn_weights, position_bias
+
+    @staticmethod
+    def _append_prev_key_padding_mask(
+            key_padding_mask: Optional[Tensor],
+            prev_key_padding_mask: Optional[Tensor],
+            batch_size: int,
+            src_len: int,
+            static_kv: bool,
+    ) -> Optional[Tensor]:
+        # saved key padding masks have shape (bsz, seq_len)
+        if prev_key_padding_mask is not None and static_kv:
+            new_key_padding_mask = prev_key_padding_mask
+        elif prev_key_padding_mask is not None and key_padding_mask is not None:
+            new_key_padding_mask = torch.cat(
+                [prev_key_padding_mask.float(), key_padding_mask.float()], dim=1
+            )
+        # During incremental decoding, as the padding token enters and
+        # leaves the frame, there will be a time when prev or current
+        # is None
+        elif prev_key_padding_mask is not None:
+            if src_len > prev_key_padding_mask.size(1):
+                filler = torch.zeros(
+                    (batch_size, src_len - prev_key_padding_mask.size(1)),
+                    device=prev_key_padding_mask.device,
+                )
+                new_key_padding_mask = torch.cat(
+                    [prev_key_padding_mask.float(), filler.float()], dim=1
+                )
+            else:
+                new_key_padding_mask = prev_key_padding_mask.float()
+        elif key_padding_mask is not None:
+            if src_len > key_padding_mask.size(1):
+                filler = torch.zeros(
+                    (batch_size, src_len - key_padding_mask.size(1)),
+                    device=key_padding_mask.device,
+                )
+                new_key_padding_mask = torch.cat(
+                    [filler.float(), key_padding_mask.float()], dim=1
+                )
+            else:
+                new_key_padding_mask = key_padding_mask.float()
+        else:
+            new_key_padding_mask = prev_key_padding_mask
+        return new_key_padding_mask
+
+    def _get_input_buffer(
+            self, incremental_state: Optional[Dict[str, Dict[str, Optional[Tensor]]]]
+    ) -> Dict[str, Optional[Tensor]]:
+        result = self.get_incremental_state(incremental_state, "attn_state")
+        if result is not None:
+            return result
+        else:
+            empty_result: Dict[str, Optional[Tensor]] = {}
+            return empty_result
+
+    def _set_input_buffer(
+            self,
+            incremental_state: Dict[str, Dict[str, Optional[Tensor]]],
+            buffer: Dict[str, Optional[Tensor]],
+    ):
+        return self.set_incremental_state(incremental_state, "attn_state", buffer)
+
+    def apply_sparse_mask(self, attn_weights, tgt_len: int, src_len: int, bsz: int):
+        return attn_weights
+
+
+def init_bert_params(module):
+    """
+    Initialize the weights specific to the BERT Model.
+    This overrides the default initializations depending on the specified arguments.
+        1. If normal_init_linear_weights is set then weights of linear
+           layer will be initialized using the normal distribution and
+           bais will be set to the specified value.
+        2. If normal_init_embed_weights is set then weights of embedding
+           layer will be initialized using the normal distribution.
+        3. If normal_init_proj_weights is set then weights of
+           in_project_weight for MultiHeadAttention initialized using
+           the normal distribution (to be validated).
+    """
+
+    def normal_(data):
+        # with FSDP, module params will be on CUDA, so we cast them back to CPU
+        # so that the RNG is consistent with and without FSDP
+        data.copy_(
+            data.cpu().normal_(mean=0.0, std=0.02).to(data.device)
+        )
+
+    if isinstance(module, nn.Linear):
+        normal_(module.weight.data)
+        if module.bias is not None:
+            module.bias.data.zero_()
+    if isinstance(module, nn.Embedding):
+        normal_(module.weight.data)
+        if module.padding_idx is not None:
+            module.weight.data[module.padding_idx].zero_()
+    if isinstance(module, MultiheadAttention):
+        normal_(module.q_proj.weight.data)
+        normal_(module.k_proj.weight.data)
+        normal_(module.v_proj.weight.data)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/kaldi.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/kaldi.py
new file mode 100644
index 0000000000000000000000000000000000000000..f97fa85308e785af571bfdc912d6f12bb092e10e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/kaldi.py
@@ -0,0 +1,813 @@
+import math
+from typing import Tuple
+
+import torch
+# import torchaudio
+from torch import Tensor
+
+__all__ = [
+    "get_mel_banks",
+    "inverse_mel_scale",
+    "inverse_mel_scale_scalar",
+    "mel_scale",
+    "mel_scale_scalar",
+    "spectrogram",
+    "fbank",
+    "mfcc",
+    "vtln_warp_freq",
+    "vtln_warp_mel_freq",
+]
+
+# numeric_limits<float>::epsilon() 1.1920928955078125e-07
+EPSILON = torch.tensor(torch.finfo(torch.float).eps)
+# 1 milliseconds = 0.001 seconds
+MILLISECONDS_TO_SECONDS = 0.001
+
+# window types
+HAMMING = "hamming"
+HANNING = "hanning"
+POVEY = "povey"
+RECTANGULAR = "rectangular"
+BLACKMAN = "blackman"
+WINDOWS = [HAMMING, HANNING, POVEY, RECTANGULAR, BLACKMAN]
+
+
+def _get_epsilon(device, dtype):
+    return EPSILON.to(device=device, dtype=dtype)
+
+
+def _next_power_of_2(x: int) -> int:
+    r"""Returns the smallest power of 2 that is greater than x"""
+    return 1 if x == 0 else 2 ** (x - 1).bit_length()
+
+
+def _get_strided(waveform: Tensor, window_size: int, window_shift: int, snip_edges: bool) -> Tensor:
+    r"""Given a waveform (1D tensor of size ``num_samples``), it returns a 2D tensor (m, ``window_size``)
+    representing how the window is shifted along the waveform. Each row is a frame.
+
+    Args:
+        waveform (Tensor): Tensor of size ``num_samples``
+        window_size (int): Frame length
+        window_shift (int): Frame shift
+        snip_edges (bool): If True, end effects will be handled by outputting only frames that completely fit
+            in the file, and the number of frames depends on the frame_length.  If False, the number of frames
+            depends only on the frame_shift, and we reflect the data at the ends.
+
+    Returns:
+        Tensor: 2D tensor of size (m, ``window_size``) where each row is a frame
+    """
+    assert waveform.dim() == 1
+    num_samples = waveform.size(0)
+    strides = (window_shift * waveform.stride(0), waveform.stride(0))
+
+    if snip_edges:
+        if num_samples < window_size:
+            return torch.empty((0, 0), dtype=waveform.dtype, device=waveform.device)
+        else:
+            m = 1 + (num_samples - window_size) // window_shift
+    else:
+        reversed_waveform = torch.flip(waveform, [0])
+        m = (num_samples + (window_shift // 2)) // window_shift
+        pad = window_size // 2 - window_shift // 2
+        pad_right = reversed_waveform
+        if pad > 0:
+            # torch.nn.functional.pad returns [2,1,0,1,2] for 'reflect'
+            # but we want [2, 1, 0, 0, 1, 2]
+            pad_left = reversed_waveform[-pad:]
+            waveform = torch.cat((pad_left, waveform, pad_right), dim=0)
+        else:
+            # pad is negative so we want to trim the waveform at the front
+            waveform = torch.cat((waveform[-pad:], pad_right), dim=0)
+
+    sizes = (m, window_size)
+    return waveform.as_strided(sizes, strides)
+
+
+def _feature_window_function(
+    window_type: str,
+    window_size: int,
+    blackman_coeff: float,
+    device: torch.device,
+    dtype: int,
+) -> Tensor:
+    r"""Returns a window function with the given type and size"""
+    if window_type == HANNING:
+        return torch.hann_window(window_size, periodic=False, device=device, dtype=dtype)
+    elif window_type == HAMMING:
+        return torch.hamming_window(window_size, periodic=False, alpha=0.54, beta=0.46, device=device, dtype=dtype)
+    elif window_type == POVEY:
+        # like hanning but goes to zero at edges
+        return torch.hann_window(window_size, periodic=False, device=device, dtype=dtype).pow(0.85)
+    elif window_type == RECTANGULAR:
+        return torch.ones(window_size, device=device, dtype=dtype)
+    elif window_type == BLACKMAN:
+        a = 2 * math.pi / (window_size - 1)
+        window_function = torch.arange(window_size, device=device, dtype=dtype)
+        # can't use torch.blackman_window as they use different coefficients
+        return (
+            blackman_coeff
+            - 0.5 * torch.cos(a * window_function)
+            + (0.5 - blackman_coeff) * torch.cos(2 * a * window_function)
+        ).to(device=device, dtype=dtype)
+    else:
+        raise Exception("Invalid window type " + window_type)
+
+
+def _get_log_energy(strided_input: Tensor, epsilon: Tensor, energy_floor: float) -> Tensor:
+    r"""Returns the log energy of size (m) for a strided_input (m,*)"""
+    device, dtype = strided_input.device, strided_input.dtype
+    log_energy = torch.max(strided_input.pow(2).sum(1), epsilon).log()  # size (m)
+    if energy_floor == 0.0:
+        return log_energy
+    return torch.max(log_energy, torch.tensor(math.log(energy_floor), device=device, dtype=dtype))
+
+
+def _get_waveform_and_window_properties(
+    waveform: Tensor,
+    channel: int,
+    sample_frequency: float,
+    frame_shift: float,
+    frame_length: float,
+    round_to_power_of_two: bool,
+    preemphasis_coefficient: float,
+) -> Tuple[Tensor, int, int, int]:
+    r"""Gets the waveform and window properties"""
+    channel = max(channel, 0)
+    assert channel < waveform.size(0), "Invalid channel {} for size {}".format(channel, waveform.size(0))
+    waveform = waveform[channel, :]  # size (n)
+    window_shift = int(sample_frequency * frame_shift * MILLISECONDS_TO_SECONDS)
+    window_size = int(sample_frequency * frame_length * MILLISECONDS_TO_SECONDS)
+    padded_window_size = _next_power_of_2(window_size) if round_to_power_of_two else window_size
+
+    assert 2 <= window_size <= len(waveform), "choose a window size {} that is [2, {}]".format(
+        window_size, len(waveform)
+    )
+    assert 0 < window_shift, "`window_shift` must be greater than 0"
+    assert padded_window_size % 2 == 0, (
+        "the padded `window_size` must be divisible by two." " use `round_to_power_of_two` or change `frame_length`"
+    )
+    assert 0.0 <= preemphasis_coefficient <= 1.0, "`preemphasis_coefficient` must be between [0,1]"
+    assert sample_frequency > 0, "`sample_frequency` must be greater than zero"
+    return waveform, window_shift, window_size, padded_window_size
+
+
+def _get_window(
+    waveform: Tensor,
+    padded_window_size: int,
+    window_size: int,
+    window_shift: int,
+    window_type: str,
+    blackman_coeff: float,
+    snip_edges: bool,
+    raw_energy: bool,
+    energy_floor: float,
+    dither: float,
+    remove_dc_offset: bool,
+    preemphasis_coefficient: float,
+) -> Tuple[Tensor, Tensor]:
+    r"""Gets a window and its log energy
+
+    Returns:
+        (Tensor, Tensor): strided_input of size (m, ``padded_window_size``) and signal_log_energy of size (m)
+    """
+    device, dtype = waveform.device, waveform.dtype
+    epsilon = _get_epsilon(device, dtype)
+
+    # size (m, window_size)
+    strided_input = _get_strided(waveform, window_size, window_shift, snip_edges)
+
+    if dither != 0.0:
+        rand_gauss = torch.randn(strided_input.shape, device=device, dtype=dtype)
+        strided_input = strided_input + rand_gauss * dither
+
+    if remove_dc_offset:
+        # Subtract each row/frame by its mean
+        row_means = torch.mean(strided_input, dim=1).unsqueeze(1)  # size (m, 1)
+        strided_input = strided_input - row_means
+
+    if raw_energy:
+        # Compute the log energy of each row/frame before applying preemphasis and
+        # window function
+        signal_log_energy = _get_log_energy(strided_input, epsilon, energy_floor)  # size (m)
+
+    if preemphasis_coefficient != 0.0:
+        # strided_input[i,j] -= preemphasis_coefficient * strided_input[i, max(0, j-1)] for all i,j
+        offset_strided_input = torch.nn.functional.pad(strided_input.unsqueeze(0), (1, 0), mode="replicate").squeeze(
+            0
+        )  # size (m, window_size + 1)
+        strided_input = strided_input - preemphasis_coefficient * offset_strided_input[:, :-1]
+
+    # Apply window_function to each row/frame
+    window_function = _feature_window_function(window_type, window_size, blackman_coeff, device, dtype).unsqueeze(
+        0
+    )  # size (1, window_size)
+    strided_input = strided_input * window_function  # size (m, window_size)
+
+    # Pad columns with zero until we reach size (m, padded_window_size)
+    if padded_window_size != window_size:
+        padding_right = padded_window_size - window_size
+        strided_input = torch.nn.functional.pad(
+            strided_input.unsqueeze(0), (0, padding_right), mode="constant", value=0
+        ).squeeze(0)
+
+    # Compute energy after window function (not the raw one)
+    if not raw_energy:
+        signal_log_energy = _get_log_energy(strided_input, epsilon, energy_floor)  # size (m)
+
+    return strided_input, signal_log_energy
+
+
+def _subtract_column_mean(tensor: Tensor, subtract_mean: bool) -> Tensor:
+    # subtracts the column mean of the tensor size (m, n) if subtract_mean=True
+    # it returns size (m, n)
+    if subtract_mean:
+        col_means = torch.mean(tensor, dim=0).unsqueeze(0)
+        tensor = tensor - col_means
+    return tensor
+
+
+def spectrogram(
+    waveform: Tensor,
+    blackman_coeff: float = 0.42,
+    channel: int = -1,
+    dither: float = 0.0,
+    energy_floor: float = 1.0,
+    frame_length: float = 25.0,
+    frame_shift: float = 10.0,
+    min_duration: float = 0.0,
+    preemphasis_coefficient: float = 0.97,
+    raw_energy: bool = True,
+    remove_dc_offset: bool = True,
+    round_to_power_of_two: bool = True,
+    sample_frequency: float = 16000.0,
+    snip_edges: bool = True,
+    subtract_mean: bool = False,
+    window_type: str = POVEY,
+) -> Tensor:
+    r"""Create a spectrogram from a raw audio signal. This matches the input/output of Kaldi's
+    compute-spectrogram-feats.
+
+    Args:
+        waveform (Tensor): Tensor of audio of size (c, n) where c is in the range [0,2)
+        blackman_coeff (float, optional): Constant coefficient for generalized Blackman window. (Default: ``0.42``)
+        channel (int, optional): Channel to extract (-1 -> expect mono, 0 -> left, 1 -> right) (Default: ``-1``)
+        dither (float, optional): Dithering constant (0.0 means no dither). If you turn this off, you should set
+            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``0.0``)
+        energy_floor (float, optional): Floor on energy (absolute, not relative) in Spectrogram computation.  Caution:
+            this floor is applied to the zeroth component, representing the total signal energy.  The floor on the
+            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``1.0``)
+        frame_length (float, optional): Frame length in milliseconds (Default: ``25.0``)
+        frame_shift (float, optional): Frame shift in milliseconds (Default: ``10.0``)
+        min_duration (float, optional): Minimum duration of segments to process (in seconds). (Default: ``0.0``)
+        preemphasis_coefficient (float, optional): Coefficient for use in signal preemphasis (Default: ``0.97``)
+        raw_energy (bool, optional): If True, compute energy before preemphasis and windowing (Default: ``True``)
+        remove_dc_offset (bool, optional): Subtract mean from waveform on each frame (Default: ``True``)
+        round_to_power_of_two (bool, optional): If True, round window size to power of two by zero-padding input
+            to FFT. (Default: ``True``)
+        sample_frequency (float, optional): Waveform data sample frequency (must match the waveform file, if
+            specified there) (Default: ``16000.0``)
+        snip_edges (bool, optional): If True, end effects will be handled by outputting only frames that completely fit
+            in the file, and the number of frames depends on the frame_length.  If False, the number of frames
+            depends only on the frame_shift, and we reflect the data at the ends. (Default: ``True``)
+        subtract_mean (bool, optional): Subtract mean of each feature file [CMS]; not recommended to do
+            it this way.  (Default: ``False``)
+        window_type (str, optional): Type of window ('hamming'|'hanning'|'povey'|'rectangular'|'blackman')
+         (Default: ``'povey'``)
+
+    Returns:
+        Tensor: A spectrogram identical to what Kaldi would output. The shape is
+        (m, ``padded_window_size // 2 + 1``) where m is calculated in _get_strided
+    """
+    device, dtype = waveform.device, waveform.dtype
+    epsilon = _get_epsilon(device, dtype)
+
+    waveform, window_shift, window_size, padded_window_size = _get_waveform_and_window_properties(
+        waveform, channel, sample_frequency, frame_shift, frame_length, round_to_power_of_two, preemphasis_coefficient
+    )
+
+    if len(waveform) < min_duration * sample_frequency:
+        # signal is too short
+        return torch.empty(0)
+
+    strided_input, signal_log_energy = _get_window(
+        waveform,
+        padded_window_size,
+        window_size,
+        window_shift,
+        window_type,
+        blackman_coeff,
+        snip_edges,
+        raw_energy,
+        energy_floor,
+        dither,
+        remove_dc_offset,
+        preemphasis_coefficient,
+    )
+
+    # size (m, padded_window_size // 2 + 1, 2)
+    fft = torch.fft.rfft(strided_input)
+
+    # Convert the FFT into a power spectrum
+    power_spectrum = torch.max(fft.abs().pow(2.0), epsilon).log()  # size (m, padded_window_size // 2 + 1)
+    power_spectrum[:, 0] = signal_log_energy
+
+    power_spectrum = _subtract_column_mean(power_spectrum, subtract_mean)
+    return power_spectrum
+
+
+def inverse_mel_scale_scalar(mel_freq: float) -> float:
+    return 700.0 * (math.exp(mel_freq / 1127.0) - 1.0)
+
+
+def inverse_mel_scale(mel_freq: Tensor) -> Tensor:
+    return 700.0 * ((mel_freq / 1127.0).exp() - 1.0)
+
+
+def mel_scale_scalar(freq: float) -> float:
+    return 1127.0 * math.log(1.0 + freq / 700.0)
+
+
+def mel_scale(freq: Tensor) -> Tensor:
+    return 1127.0 * (1.0 + freq / 700.0).log()
+
+
+def vtln_warp_freq(
+    vtln_low_cutoff: float,
+    vtln_high_cutoff: float,
+    low_freq: float,
+    high_freq: float,
+    vtln_warp_factor: float,
+    freq: Tensor,
+) -> Tensor:
+    r"""This computes a VTLN warping function that is not the same as HTK's one,
+    but has similar inputs (this function has the advantage of never producing
+    empty bins).
+
+    This function computes a warp function F(freq), defined between low_freq
+    and high_freq inclusive, with the following properties:
+        F(low_freq) == low_freq
+        F(high_freq) == high_freq
+    The function is continuous and piecewise linear with two inflection
+        points.
+    The lower inflection point (measured in terms of the unwarped
+        frequency) is at frequency l, determined as described below.
+    The higher inflection point is at a frequency h, determined as
+        described below.
+    If l <= f <= h, then F(f) = f/vtln_warp_factor.
+    If the higher inflection point (measured in terms of the unwarped
+        frequency) is at h, then max(h, F(h)) == vtln_high_cutoff.
+        Since (by the last point) F(h) == h/vtln_warp_factor, then
+        max(h, h/vtln_warp_factor) == vtln_high_cutoff, so
+        h = vtln_high_cutoff / max(1, 1/vtln_warp_factor).
+          = vtln_high_cutoff * min(1, vtln_warp_factor).
+    If the lower inflection point (measured in terms of the unwarped
+        frequency) is at l, then min(l, F(l)) == vtln_low_cutoff
+        This implies that l = vtln_low_cutoff / min(1, 1/vtln_warp_factor)
+                            = vtln_low_cutoff * max(1, vtln_warp_factor)
+    Args:
+        vtln_low_cutoff (float): Lower frequency cutoffs for VTLN
+        vtln_high_cutoff (float): Upper frequency cutoffs for VTLN
+        low_freq (float): Lower frequency cutoffs in mel computation
+        high_freq (float): Upper frequency cutoffs in mel computation
+        vtln_warp_factor (float): Vtln warp factor
+        freq (Tensor): given frequency in Hz
+
+    Returns:
+        Tensor: Freq after vtln warp
+    """
+    assert vtln_low_cutoff > low_freq, "be sure to set the vtln_low option higher than low_freq"
+    assert vtln_high_cutoff < high_freq, "be sure to set the vtln_high option lower than high_freq [or negative]"
+    l = vtln_low_cutoff * max(1.0, vtln_warp_factor)
+    h = vtln_high_cutoff * min(1.0, vtln_warp_factor)
+    scale = 1.0 / vtln_warp_factor
+    Fl = scale * l  # F(l)
+    Fh = scale * h  # F(h)
+    assert l > low_freq and h < high_freq
+    # slope of left part of the 3-piece linear function
+    scale_left = (Fl - low_freq) / (l - low_freq)
+    # [slope of center part is just "scale"]
+
+    # slope of right part of the 3-piece linear function
+    scale_right = (high_freq - Fh) / (high_freq - h)
+
+    res = torch.empty_like(freq)
+
+    outside_low_high_freq = torch.lt(freq, low_freq) | torch.gt(freq, high_freq)  # freq < low_freq || freq > high_freq
+    before_l = torch.lt(freq, l)  # freq < l
+    before_h = torch.lt(freq, h)  # freq < h
+    after_h = torch.ge(freq, h)  # freq >= h
+
+    # order of operations matter here (since there is overlapping frequency regions)
+    res[after_h] = high_freq + scale_right * (freq[after_h] - high_freq)
+    res[before_h] = scale * freq[before_h]
+    res[before_l] = low_freq + scale_left * (freq[before_l] - low_freq)
+    res[outside_low_high_freq] = freq[outside_low_high_freq]
+
+    return res
+
+
+def vtln_warp_mel_freq(
+    vtln_low_cutoff: float,
+    vtln_high_cutoff: float,
+    low_freq,
+    high_freq: float,
+    vtln_warp_factor: float,
+    mel_freq: Tensor,
+) -> Tensor:
+    r"""
+    Args:
+        vtln_low_cutoff (float): Lower frequency cutoffs for VTLN
+        vtln_high_cutoff (float): Upper frequency cutoffs for VTLN
+        low_freq (float): Lower frequency cutoffs in mel computation
+        high_freq (float): Upper frequency cutoffs in mel computation
+        vtln_warp_factor (float): Vtln warp factor
+        mel_freq (Tensor): Given frequency in Mel
+
+    Returns:
+        Tensor: ``mel_freq`` after vtln warp
+    """
+    return mel_scale(
+        vtln_warp_freq(
+            vtln_low_cutoff, vtln_high_cutoff, low_freq, high_freq, vtln_warp_factor, inverse_mel_scale(mel_freq)
+        )
+    )
+
+
+def get_mel_banks(
+    num_bins: int,
+    window_length_padded: int,
+    sample_freq: float,
+    low_freq: float,
+    high_freq: float,
+    vtln_low: float,
+    vtln_high: float,
+    vtln_warp_factor: float,
+) -> Tuple[Tensor, Tensor]:
+    """
+    Returns:
+        (Tensor, Tensor): The tuple consists of ``bins`` (which is
+        melbank of size (``num_bins``, ``num_fft_bins``)) and ``center_freqs`` (which is
+        center frequencies of bins of size (``num_bins``)).
+    """
+    assert num_bins > 3, "Must have at least 3 mel bins"
+    assert window_length_padded % 2 == 0
+    num_fft_bins = window_length_padded / 2
+    nyquist = 0.5 * sample_freq
+
+    if high_freq <= 0.0:
+        high_freq += nyquist
+
+    assert (
+        (0.0 <= low_freq < nyquist) and (0.0 < high_freq <= nyquist) and (low_freq < high_freq)
+    ), "Bad values in options: low-freq {} and high-freq {} vs. nyquist {}".format(low_freq, high_freq, nyquist)
+
+    # fft-bin width [think of it as Nyquist-freq / half-window-length]
+    fft_bin_width = sample_freq / window_length_padded
+    mel_low_freq = mel_scale_scalar(low_freq)
+    mel_high_freq = mel_scale_scalar(high_freq)
+
+    # divide by num_bins+1 in next line because of end-effects where the bins
+    # spread out to the sides.
+    mel_freq_delta = (mel_high_freq - mel_low_freq) / (num_bins + 1)
+
+    if vtln_high < 0.0:
+        vtln_high += nyquist
+
+    assert vtln_warp_factor == 1.0 or (
+        (low_freq < vtln_low < high_freq) and (0.0 < vtln_high < high_freq) and (vtln_low < vtln_high)
+    ), "Bad values in options: vtln-low {} and vtln-high {}, versus " "low-freq {} and high-freq {}".format(
+        vtln_low, vtln_high, low_freq, high_freq
+    )
+
+    bin = torch.arange(num_bins).unsqueeze(1)
+    left_mel = mel_low_freq + bin * mel_freq_delta  # size(num_bins, 1)
+    center_mel = mel_low_freq + (bin + 1.0) * mel_freq_delta  # size(num_bins, 1)
+    right_mel = mel_low_freq + (bin + 2.0) * mel_freq_delta  # size(num_bins, 1)
+
+    if vtln_warp_factor != 1.0:
+        left_mel = vtln_warp_mel_freq(vtln_low, vtln_high, low_freq, high_freq, vtln_warp_factor, left_mel)
+        center_mel = vtln_warp_mel_freq(vtln_low, vtln_high, low_freq, high_freq, vtln_warp_factor, center_mel)
+        right_mel = vtln_warp_mel_freq(vtln_low, vtln_high, low_freq, high_freq, vtln_warp_factor, right_mel)
+
+    center_freqs = inverse_mel_scale(center_mel)  # size (num_bins)
+    # size(1, num_fft_bins)
+    mel = mel_scale(fft_bin_width * torch.arange(num_fft_bins)).unsqueeze(0)
+
+    # size (num_bins, num_fft_bins)
+    up_slope = (mel - left_mel) / (center_mel - left_mel)
+    down_slope = (right_mel - mel) / (right_mel - center_mel)
+
+    if vtln_warp_factor == 1.0:
+        # left_mel < center_mel < right_mel so we can min the two slopes and clamp negative values
+        bins = torch.max(torch.zeros(1), torch.min(up_slope, down_slope))
+    else:
+        # warping can move the order of left_mel, center_mel, right_mel anywhere
+        bins = torch.zeros_like(up_slope)
+        up_idx = torch.gt(mel, left_mel) & torch.le(mel, center_mel)  # left_mel < mel <= center_mel
+        down_idx = torch.gt(mel, center_mel) & torch.lt(mel, right_mel)  # center_mel < mel < right_mel
+        bins[up_idx] = up_slope[up_idx]
+        bins[down_idx] = down_slope[down_idx]
+
+    return bins, center_freqs
+
+
+def fbank(
+    waveform: Tensor,
+    blackman_coeff: float = 0.42,
+    channel: int = -1,
+    dither: float = 0.0,
+    energy_floor: float = 1.0,
+    frame_length: float = 25.0,
+    frame_shift: float = 10.0,
+    high_freq: float = 0.0,
+    htk_compat: bool = False,
+    low_freq: float = 20.0,
+    min_duration: float = 0.0,
+    num_mel_bins: int = 23,
+    preemphasis_coefficient: float = 0.97,
+    raw_energy: bool = True,
+    remove_dc_offset: bool = True,
+    round_to_power_of_two: bool = True,
+    sample_frequency: float = 16000.0,
+    snip_edges: bool = True,
+    subtract_mean: bool = False,
+    use_energy: bool = False,
+    use_log_fbank: bool = True,
+    use_power: bool = True,
+    vtln_high: float = -500.0,
+    vtln_low: float = 100.0,
+    vtln_warp: float = 1.0,
+    window_type: str = POVEY,
+) -> Tensor:
+    r"""Create a fbank from a raw audio signal. This matches the input/output of Kaldi's
+    compute-fbank-feats.
+
+    Args:
+        waveform (Tensor): Tensor of audio of size (c, n) where c is in the range [0,2)
+        blackman_coeff (float, optional): Constant coefficient for generalized Blackman window. (Default: ``0.42``)
+        channel (int, optional): Channel to extract (-1 -> expect mono, 0 -> left, 1 -> right) (Default: ``-1``)
+        dither (float, optional): Dithering constant (0.0 means no dither). If you turn this off, you should set
+            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``0.0``)
+        energy_floor (float, optional): Floor on energy (absolute, not relative) in Spectrogram computation.  Caution:
+            this floor is applied to the zeroth component, representing the total signal energy.  The floor on the
+            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``1.0``)
+        frame_length (float, optional): Frame length in milliseconds (Default: ``25.0``)
+        frame_shift (float, optional): Frame shift in milliseconds (Default: ``10.0``)
+        high_freq (float, optional): High cutoff frequency for mel bins (if <= 0, offset from Nyquist)
+         (Default: ``0.0``)
+        htk_compat (bool, optional): If true, put energy last.  Warning: not sufficient to get HTK compatible features
+         (need to change other parameters). (Default: ``False``)
+        low_freq (float, optional): Low cutoff frequency for mel bins (Default: ``20.0``)
+        min_duration (float, optional): Minimum duration of segments to process (in seconds). (Default: ``0.0``)
+        num_mel_bins (int, optional): Number of triangular mel-frequency bins (Default: ``23``)
+        preemphasis_coefficient (float, optional): Coefficient for use in signal preemphasis (Default: ``0.97``)
+        raw_energy (bool, optional): If True, compute energy before preemphasis and windowing (Default: ``True``)
+        remove_dc_offset (bool, optional): Subtract mean from waveform on each frame (Default: ``True``)
+        round_to_power_of_two (bool, optional): If True, round window size to power of two by zero-padding input
+            to FFT. (Default: ``True``)
+        sample_frequency (float, optional): Waveform data sample frequency (must match the waveform file, if
+            specified there) (Default: ``16000.0``)
+        snip_edges (bool, optional): If True, end effects will be handled by outputting only frames that completely fit
+            in the file, and the number of frames depends on the frame_length.  If False, the number of frames
+            depends only on the frame_shift, and we reflect the data at the ends. (Default: ``True``)
+        subtract_mean (bool, optional): Subtract mean of each feature file [CMS]; not recommended to do
+            it this way.  (Default: ``False``)
+        use_energy (bool, optional): Add an extra dimension with energy to the FBANK output. (Default: ``False``)
+        use_log_fbank (bool, optional):If true, produce log-filterbank, else produce linear. (Default: ``True``)
+        use_power (bool, optional): If true, use power, else use magnitude. (Default: ``True``)
+        vtln_high (float, optional): High inflection point in piecewise linear VTLN warping function (if
+            negative, offset from high-mel-freq (Default: ``-500.0``)
+        vtln_low (float, optional): Low inflection point in piecewise linear VTLN warping function (Default: ``100.0``)
+        vtln_warp (float, optional): Vtln warp factor (only applicable if vtln_map not specified) (Default: ``1.0``)
+        window_type (str, optional): Type of window ('hamming'|'hanning'|'povey'|'rectangular'|'blackman')
+         (Default: ``'povey'``)
+
+    Returns:
+        Tensor: A fbank identical to what Kaldi would output. The shape is (m, ``num_mel_bins + use_energy``)
+        where m is calculated in _get_strided
+    """
+    device, dtype = waveform.device, waveform.dtype
+
+    waveform, window_shift, window_size, padded_window_size = _get_waveform_and_window_properties(
+        waveform, channel, sample_frequency, frame_shift, frame_length, round_to_power_of_two, preemphasis_coefficient
+    )
+
+    if len(waveform) < min_duration * sample_frequency:
+        # signal is too short
+        return torch.empty(0, device=device, dtype=dtype)
+
+    # strided_input, size (m, padded_window_size) and signal_log_energy, size (m)
+    strided_input, signal_log_energy = _get_window(
+        waveform,
+        padded_window_size,
+        window_size,
+        window_shift,
+        window_type,
+        blackman_coeff,
+        snip_edges,
+        raw_energy,
+        energy_floor,
+        dither,
+        remove_dc_offset,
+        preemphasis_coefficient,
+    )
+
+    # size (m, padded_window_size // 2 + 1)
+    spectrum = torch.fft.rfft(strided_input).abs()
+    if use_power:
+        spectrum = spectrum.pow(2.0)
+
+    # size (num_mel_bins, padded_window_size // 2)
+    mel_energies, _ = get_mel_banks(
+        num_mel_bins, padded_window_size, sample_frequency, low_freq, high_freq, vtln_low, vtln_high, vtln_warp
+    )
+    mel_energies = mel_energies.to(device=device, dtype=dtype)
+
+    # pad right column with zeros and add dimension, size (num_mel_bins, padded_window_size // 2 + 1)
+    mel_energies = torch.nn.functional.pad(mel_energies, (0, 1), mode="constant", value=0)
+
+    # sum with mel fiterbanks over the power spectrum, size (m, num_mel_bins)
+    mel_energies = torch.mm(spectrum, mel_energies.T)
+    if use_log_fbank:
+        # avoid log of zero (which should be prevented anyway by dithering)
+        mel_energies = torch.max(mel_energies, _get_epsilon(device, dtype)).log()
+
+    # if use_energy then add it as the last column for htk_compat == true else first column
+    if use_energy:
+        signal_log_energy = signal_log_energy.unsqueeze(1)  # size (m, 1)
+        # returns size (m, num_mel_bins + 1)
+        if htk_compat:
+            mel_energies = torch.cat((mel_energies, signal_log_energy), dim=1)
+        else:
+            mel_energies = torch.cat((signal_log_energy, mel_energies), dim=1)
+
+    mel_energies = _subtract_column_mean(mel_energies, subtract_mean)
+    return mel_energies
+
+
+def _get_dct_matrix(num_ceps: int, num_mel_bins: int) -> Tensor:
+    # returns a dct matrix of size (num_mel_bins, num_ceps)
+    # size (num_mel_bins, num_mel_bins)
+    dct_matrix = torchaudio.functional.create_dct(num_mel_bins, num_mel_bins, "ortho")
+    # kaldi expects the first cepstral to be weighted sum of factor sqrt(1/num_mel_bins)
+    # this would be the first column in the dct_matrix for torchaudio as it expects a
+    # right multiply (which would be the first column of the kaldi's dct_matrix as kaldi
+    # expects a left multiply e.g. dct_matrix * vector).
+    dct_matrix[:, 0] = math.sqrt(1 / float(num_mel_bins))
+    dct_matrix = dct_matrix[:, :num_ceps]
+    return dct_matrix
+
+
+def _get_lifter_coeffs(num_ceps: int, cepstral_lifter: float) -> Tensor:
+    # returns size (num_ceps)
+    # Compute liftering coefficients (scaling on cepstral coeffs)
+    # coeffs are numbered slightly differently from HTK: the zeroth index is C0, which is not affected.
+    i = torch.arange(num_ceps)
+    return 1.0 + 0.5 * cepstral_lifter * torch.sin(math.pi * i / cepstral_lifter)
+
+
+def mfcc(
+    waveform: Tensor,
+    blackman_coeff: float = 0.42,
+    cepstral_lifter: float = 22.0,
+    channel: int = -1,
+    dither: float = 0.0,
+    energy_floor: float = 1.0,
+    frame_length: float = 25.0,
+    frame_shift: float = 10.0,
+    high_freq: float = 0.0,
+    htk_compat: bool = False,
+    low_freq: float = 20.0,
+    num_ceps: int = 13,
+    min_duration: float = 0.0,
+    num_mel_bins: int = 23,
+    preemphasis_coefficient: float = 0.97,
+    raw_energy: bool = True,
+    remove_dc_offset: bool = True,
+    round_to_power_of_two: bool = True,
+    sample_frequency: float = 16000.0,
+    snip_edges: bool = True,
+    subtract_mean: bool = False,
+    use_energy: bool = False,
+    vtln_high: float = -500.0,
+    vtln_low: float = 100.0,
+    vtln_warp: float = 1.0,
+    window_type: str = POVEY,
+) -> Tensor:
+    r"""Create a mfcc from a raw audio signal. This matches the input/output of Kaldi's
+    compute-mfcc-feats.
+
+    Args:
+        waveform (Tensor): Tensor of audio of size (c, n) where c is in the range [0,2)
+        blackman_coeff (float, optional): Constant coefficient for generalized Blackman window. (Default: ``0.42``)
+        cepstral_lifter (float, optional): Constant that controls scaling of MFCCs (Default: ``22.0``)
+        channel (int, optional): Channel to extract (-1 -> expect mono, 0 -> left, 1 -> right) (Default: ``-1``)
+        dither (float, optional): Dithering constant (0.0 means no dither). If you turn this off, you should set
+            the energy_floor option, e.g. to 1.0 or 0.1 (Default: ``0.0``)
+        energy_floor (float, optional): Floor on energy (absolute, not relative) in Spectrogram computation.  Caution:
+            this floor is applied to the zeroth component, representing the total signal energy.  The floor on the
+            individual spectrogram elements is fixed at std::numeric_limits<float>::epsilon(). (Default: ``1.0``)
+        frame_length (float, optional): Frame length in milliseconds (Default: ``25.0``)
+        frame_shift (float, optional): Frame shift in milliseconds (Default: ``10.0``)
+        high_freq (float, optional): High cutoff frequency for mel bins (if <= 0, offset from Nyquist)
+         (Default: ``0.0``)
+        htk_compat (bool, optional): If true, put energy last.  Warning: not sufficient to get HTK compatible
+         features (need to change other parameters). (Default: ``False``)
+        low_freq (float, optional): Low cutoff frequency for mel bins (Default: ``20.0``)
+        num_ceps (int, optional): Number of cepstra in MFCC computation (including C0) (Default: ``13``)
+        min_duration (float, optional): Minimum duration of segments to process (in seconds). (Default: ``0.0``)
+        num_mel_bins (int, optional): Number of triangular mel-frequency bins (Default: ``23``)
+        preemphasis_coefficient (float, optional): Coefficient for use in signal preemphasis (Default: ``0.97``)
+        raw_energy (bool, optional): If True, compute energy before preemphasis and windowing (Default: ``True``)
+        remove_dc_offset (bool, optional): Subtract mean from waveform on each frame (Default: ``True``)
+        round_to_power_of_two (bool, optional): If True, round window size to power of two by zero-padding input
+            to FFT. (Default: ``True``)
+        sample_frequency (float, optional): Waveform data sample frequency (must match the waveform file, if
+            specified there) (Default: ``16000.0``)
+        snip_edges (bool, optional): If True, end effects will be handled by outputting only frames that completely fit
+            in the file, and the number of frames depends on the frame_length.  If False, the number of frames
+            depends only on the frame_shift, and we reflect the data at the ends. (Default: ``True``)
+        subtract_mean (bool, optional): Subtract mean of each feature file [CMS]; not recommended to do
+            it this way.  (Default: ``False``)
+        use_energy (bool, optional): Add an extra dimension with energy to the FBANK output. (Default: ``False``)
+        vtln_high (float, optional): High inflection point in piecewise linear VTLN warping function (if
+            negative, offset from high-mel-freq (Default: ``-500.0``)
+        vtln_low (float, optional): Low inflection point in piecewise linear VTLN warping function (Default: ``100.0``)
+        vtln_warp (float, optional): Vtln warp factor (only applicable if vtln_map not specified) (Default: ``1.0``)
+        window_type (str, optional): Type of window ('hamming'|'hanning'|'povey'|'rectangular'|'blackman')
+         (Default: ``"povey"``)
+
+    Returns:
+        Tensor: A mfcc identical to what Kaldi would output. The shape is (m, ``num_ceps``)
+        where m is calculated in _get_strided
+    """
+    assert num_ceps <= num_mel_bins, "num_ceps cannot be larger than num_mel_bins: %d vs %d" % (num_ceps, num_mel_bins)
+
+    device, dtype = waveform.device, waveform.dtype
+
+    # The mel_energies should not be squared (use_power=True), not have mean subtracted
+    # (subtract_mean=False), and use log (use_log_fbank=True).
+    # size (m, num_mel_bins + use_energy)
+    feature = fbank(
+        waveform=waveform,
+        blackman_coeff=blackman_coeff,
+        channel=channel,
+        dither=dither,
+        energy_floor=energy_floor,
+        frame_length=frame_length,
+        frame_shift=frame_shift,
+        high_freq=high_freq,
+        htk_compat=htk_compat,
+        low_freq=low_freq,
+        min_duration=min_duration,
+        num_mel_bins=num_mel_bins,
+        preemphasis_coefficient=preemphasis_coefficient,
+        raw_energy=raw_energy,
+        remove_dc_offset=remove_dc_offset,
+        round_to_power_of_two=round_to_power_of_two,
+        sample_frequency=sample_frequency,
+        snip_edges=snip_edges,
+        subtract_mean=False,
+        use_energy=use_energy,
+        use_log_fbank=True,
+        use_power=True,
+        vtln_high=vtln_high,
+        vtln_low=vtln_low,
+        vtln_warp=vtln_warp,
+        window_type=window_type,
+    )
+
+    if use_energy:
+        # size (m)
+        signal_log_energy = feature[:, num_mel_bins if htk_compat else 0]
+        # offset is 0 if htk_compat==True else 1
+        mel_offset = int(not htk_compat)
+        feature = feature[:, mel_offset : (num_mel_bins + mel_offset)]
+
+    # size (num_mel_bins, num_ceps)
+    dct_matrix = _get_dct_matrix(num_ceps, num_mel_bins).to(dtype=dtype, device=device)
+
+    # size (m, num_ceps)
+    feature = feature.matmul(dct_matrix)
+
+    if cepstral_lifter != 0.0:
+        # size (1, num_ceps)
+        lifter_coeffs = _get_lifter_coeffs(num_ceps, cepstral_lifter).unsqueeze(0)
+        feature *= lifter_coeffs.to(device=device, dtype=dtype)
+
+    # if use_energy then replace the last column for htk_compat == true else first column
+    if use_energy:
+        feature[:, 0] = signal_log_energy
+
+    if htk_compat:
+        energy = feature[:, 0].unsqueeze(1)  # size (m, 1)
+        feature = feature[:, 1:]  # size (m, num_ceps - 1)
+        if not use_energy:
+            # scale on C0 (actually removing a scale we previously added that's
+            # part of one common definition of the cosine transform.)
+            energy *= math.sqrt(2)
+
+        feature = torch.cat((feature, energy), dim=1)
+
+    feature = _subtract_column_mean(feature, subtract_mean)
+    return feature
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/modules.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/modules.py
new file mode 100644
index 0000000000000000000000000000000000000000..e46c1d0fcb8fab8451ef6012d1239996e8405eac
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/modules.py
@@ -0,0 +1,219 @@
+# --------------------------------------------------------
+# BEATs: Audio Pre-Training with Acoustic Tokenizers (https://arxiv.org/abs/2212.09058)
+# Github source: https://github.com/microsoft/unilm/tree/master/beats
+# Copyright (c) 2022 Microsoft
+# Licensed under The MIT License [see LICENSE for details]
+# Based on fairseq code bases
+# https://github.com/pytorch/fairseq
+# --------------------------------------------------------
+
+import math
+import warnings
+
+import torch
+import torch.nn.functional as F
+from torch import Tensor, nn
+
+
+class GradMultiply(torch.autograd.Function):
+    @staticmethod
+    def forward(ctx, x, scale):
+        ctx.scale = scale
+        res = x.new(x)
+        return res
+
+    @staticmethod
+    def backward(ctx, grad):
+        return grad * ctx.scale, None
+
+
+class SamePad(nn.Module):
+    def __init__(self, kernel_size, causal=False):
+        super().__init__()
+        if causal:
+            self.remove = kernel_size - 1
+        else:
+            self.remove = 1 if kernel_size % 2 == 0 else 0
+
+    def forward(self, x):
+        if self.remove > 0:
+            x = x[:, :, : -self.remove]
+        return x
+
+
+class Swish(nn.Module):
+    def __init__(self):
+        super(Swish, self).__init__()
+        self.act = torch.nn.Sigmoid()
+
+    def forward(self, x):
+        return x * self.act(x)
+
+
+class GLU_Linear(nn.Module):
+    def __init__(self, input_dim, output_dim, glu_type="sigmoid", bias_in_glu=True):
+        super(GLU_Linear, self).__init__()
+
+        self.glu_type = glu_type
+        self.output_dim = output_dim
+
+        if glu_type == "sigmoid":
+            self.glu_act = torch.nn.Sigmoid()
+        elif glu_type == "swish":
+            self.glu_act = Swish()
+        elif glu_type == "relu":
+            self.glu_act = torch.nn.ReLU()
+        elif glu_type == "gelu":
+            self.glu_act = torch.nn.GELU()
+
+        if bias_in_glu:
+            self.linear = nn.Linear(input_dim, output_dim * 2, True)
+        else:
+            self.linear = nn.Linear(input_dim, output_dim * 2, False)
+
+    def forward(self, x):
+        # to be consistent with GLU_Linear, we assume the input always has the #channel (#dim) in the last dimension of the tensor, so need to switch the dimension first for 1D-Conv case
+        x = self.linear(x)
+
+        if self.glu_type == "bilinear":
+            x = (x[:, :, 0:self.output_dim] * x[:, :, self.output_dim:self.output_dim * 2])
+        else:
+            x = (x[:, :, 0:self.output_dim] * self.glu_act(x[:, :, self.output_dim:self.output_dim * 2]))
+
+        return x
+
+
+def gelu_accurate(x):
+    if not hasattr(gelu_accurate, "_a"):
+        gelu_accurate._a = math.sqrt(2 / math.pi)
+    return (
+        0.5 * x * (1 + torch.tanh(gelu_accurate._a * (x + 0.044715 * torch.pow(x, 3))))
+    )
+
+
+def gelu(x: torch.Tensor) -> torch.Tensor:
+    return torch.nn.functional.gelu(x.float()).type_as(x)
+
+
+def get_activation_fn(activation: str):
+    """Returns the activation function corresponding to `activation`"""
+
+    if activation == "relu":
+        return F.relu
+    elif activation == "gelu":
+        return gelu
+    elif activation == "gelu_fast":
+        warnings.warn(
+            "--activation-fn=gelu_fast has been renamed to gelu_accurate"
+        )
+        return gelu_accurate
+    elif activation == "gelu_accurate":
+        return gelu_accurate
+    elif activation == "tanh":
+        return torch.tanh
+    elif activation == "linear":
+        return lambda x: x
+    elif activation == "glu":
+        return lambda x: x
+    else:
+        raise RuntimeError("--activation-fn {} not supported".format(activation))
+
+
+def quant_noise(module, p, block_size):
+    """
+    Wraps modules and applies quantization noise to the weights for
+    subsequent quantization with Iterative Product Quantization as
+    described in "Training with Quantization Noise for Extreme Model Compression"
+
+    Args:
+        - module: nn.Module
+        - p: amount of Quantization Noise
+        - block_size: size of the blocks for subsequent quantization with iPQ
+
+    Remarks:
+        - Module weights must have the right sizes wrt the block size
+        - Only Linear, Embedding and Conv2d modules are supported for the moment
+        - For more detail on how to quantize by blocks with convolutional weights,
+          see "And the Bit Goes Down: Revisiting the Quantization of Neural Networks"
+        - We implement the simplest form of noise here as stated in the paper
+          which consists in randomly dropping blocks
+    """
+
+    # if no quantization noise, don't register hook
+    if p <= 0:
+        return module
+
+    # supported modules
+    assert isinstance(module, (nn.Linear, nn.Embedding, nn.Conv2d))
+
+    # test whether module.weight has the right sizes wrt block_size
+    is_conv = module.weight.ndim == 4
+
+    # 2D matrix
+    if not is_conv:
+        assert (
+            module.weight.size(1) % block_size == 0
+        ), "Input features must be a multiple of block sizes"
+
+    # 4D matrix
+    else:
+        # 1x1 convolutions
+        if module.kernel_size == (1, 1):
+            assert (
+                module.in_channels % block_size == 0
+            ), "Input channels must be a multiple of block sizes"
+        # regular convolutions
+        else:
+            k = module.kernel_size[0] * module.kernel_size[1]
+            assert k % block_size == 0, "Kernel size must be a multiple of block size"
+
+    def _forward_pre_hook(mod, input):
+        # no noise for evaluation
+        if mod.training:
+            if not is_conv:
+                # gather weight and sizes
+                weight = mod.weight
+                in_features = weight.size(1)
+                out_features = weight.size(0)
+
+                # split weight matrix into blocks and randomly drop selected blocks
+                mask = torch.zeros(
+                    in_features // block_size * out_features, device=weight.device
+                )
+                mask.bernoulli_(p)
+                mask = mask.repeat_interleave(block_size, -1).view(-1, in_features)
+
+            else:
+                # gather weight and sizes
+                weight = mod.weight
+                in_channels = mod.in_channels
+                out_channels = mod.out_channels
+
+                # split weight matrix into blocks and randomly drop selected blocks
+                if mod.kernel_size == (1, 1):
+                    mask = torch.zeros(
+                        int(in_channels // block_size * out_channels),
+                        device=weight.device,
+                    )
+                    mask.bernoulli_(p)
+                    mask = mask.repeat_interleave(block_size, -1).view(-1, in_channels)
+                else:
+                    mask = torch.zeros(
+                        weight.size(0), weight.size(1), device=weight.device
+                    )
+                    mask.bernoulli_(p)
+                    mask = (
+                        mask.unsqueeze(2)
+                        .unsqueeze(3)
+                        .repeat(1, 1, mod.kernel_size[0], mod.kernel_size[1])
+                    )
+
+            # scale weights and apply mask
+            mask = mask.to(
+                torch.bool
+            )  # x.bool() is not currently supported in TorchScript
+            s = 1 / (1 - p)
+            mod.weight.data = s * weight.masked_fill(mask, 0)
+
+    module.register_forward_pre_hook(_forward_pre_hook)
+    return module
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/quantizer.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/quantizer.py
new file mode 100644
index 0000000000000000000000000000000000000000..1a0b3670db2c09ca01f26b78ad0e66880201b126
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/beats/quantizer.py
@@ -0,0 +1,215 @@
+# --------------------------------------------------------
+# BEATs: Audio Pre-Training with Acoustic Tokenizers (https://arxiv.org/abs/2212.09058)
+# Github source: https://github.com/microsoft/unilm/tree/master/beats
+# Copyright (c) 2022 Microsoft
+# Licensed under The MIT License [see LICENSE for details]
+# Based on VQGAN code bases
+# https://github.com/CompVis/taming-transformers
+# --------------------------------------------------------'
+
+import torch
+import torch.distributed as distributed
+import torch.nn as nn
+import torch.nn.functional as F
+
+try:
+    from einops import rearrange, repeat
+except ImportError:
+    pass
+
+
+def l2norm(t):
+    return F.normalize(t, p=2, dim=-1)
+
+
+def ema_inplace(moving_avg, new, decay):
+    moving_avg.data.mul_(decay).add_(new, alpha=(1 - decay))
+
+
+def sample_vectors(samples, num):
+    num_samples, device = samples.shape[0], samples.device
+
+    if num_samples >= num:
+        indices = torch.randperm(num_samples, device=device)[:num]
+    else:
+        indices = torch.randint(0, num_samples, (num,), device=device)
+
+    return samples[indices]
+
+
+def kmeans(samples, num_clusters, num_iters=10, use_cosine_sim=False):
+    dim, dtype, device = samples.shape[-1], samples.dtype, samples.device
+
+    means = sample_vectors(samples, num_clusters)
+
+    for _ in range(num_iters):
+        if use_cosine_sim:
+            dists = samples @ means.t()
+        else:
+            diffs = rearrange(samples, 'n d -> n () d') \
+                    - rearrange(means, 'c d -> () c d')
+            dists = -(diffs ** 2).sum(dim=-1)
+
+        buckets = dists.max(dim=-1).indices
+        bins = torch.bincount(buckets, minlength=num_clusters)
+        zero_mask = bins == 0
+        bins_min_clamped = bins.masked_fill(zero_mask, 1)
+
+        new_means = buckets.new_zeros(num_clusters, dim, dtype=dtype)
+        new_means.scatter_add_(0, repeat(buckets, 'n -> n d', d=dim), samples)
+        new_means = new_means / bins_min_clamped[..., None]
+
+        if use_cosine_sim:
+            new_means = l2norm(new_means)
+
+        means = torch.where(zero_mask[..., None], means, new_means)
+
+    return means, bins
+
+
+class EmbeddingEMA(nn.Module):
+    def __init__(self, num_tokens, codebook_dim, decay=0.99, eps=1e-5, kmeans_init=True, codebook_init_path=''):
+        super().__init__()
+        self.num_tokens = num_tokens
+        self.codebook_dim = codebook_dim
+        self.decay = decay
+        self.eps = eps
+        if codebook_init_path == '':
+            if not kmeans_init:
+                weight = torch.randn(num_tokens, codebook_dim)
+                weight = l2norm(weight)
+            else:
+                weight = torch.zeros(num_tokens, codebook_dim)
+            self.register_buffer('initted', torch.Tensor([not kmeans_init]))
+        else:
+            print(f"load init codebook weight from {codebook_init_path}")
+            codebook_ckpt_weight = torch.load(codebook_init_path, map_location='cpu')
+            weight = codebook_ckpt_weight.clone()
+            self.register_buffer('initted', torch.Tensor([True]))
+
+        self.weight = nn.Parameter(weight, requires_grad=False)
+        self.cluster_size = nn.Parameter(torch.zeros(num_tokens), requires_grad=False)
+        self.embed_avg = nn.Parameter(weight.clone(), requires_grad=False)
+        # self.register_buffer('initted', torch.Tensor([not kmeans_init]))
+        self.update = True
+
+    @torch.jit.ignore
+    def init_embed_(self, data):
+        if self.initted:
+            return
+        print("Performing Kemans init for codebook")
+        embed, cluster_size = kmeans(data, self.num_tokens, 10, use_cosine_sim=True)
+        self.weight.data.copy_(embed)
+        self.cluster_size.data.copy_(cluster_size)
+        self.initted.data.copy_(torch.Tensor([True]))
+
+    def forward(self, embed_id):
+        return F.embedding(embed_id, self.weight)
+
+    def cluster_size_ema_update(self, new_cluster_size):
+        self.cluster_size.data.mul_(self.decay).add_(new_cluster_size, alpha=1 - self.decay)
+
+    def embed_avg_ema_update(self, new_embed_avg):
+        self.embed_avg.data.mul_(self.decay).add_(new_embed_avg, alpha=1 - self.decay)
+
+    def weight_update(self, num_tokens):
+        n = self.cluster_size.sum()
+        smoothed_cluster_size = (
+                (self.cluster_size + self.eps) / (n + num_tokens * self.eps) * n
+        )
+        # normalize embedding average with smoothed cluster size
+        embed_normalized = self.embed_avg / smoothed_cluster_size.unsqueeze(1)
+        # embed_normalized = l2norm(self.embed_avg / smoothed_cluster_size.unsqueeze(1))
+        self.weight.data.copy_(embed_normalized)
+
+
+def norm_ema_inplace(moving_avg, new, decay):
+    moving_avg.data.mul_(decay).add_(new, alpha=(1 - decay))
+    moving_avg.data.copy_(l2norm(moving_avg.data))
+
+
+class NormEMAVectorQuantizer(nn.Module):
+    def __init__(self, n_embed, embedding_dim, beta, decay=0.99, eps=1e-5,
+                 statistic_code_usage=True, kmeans_init=False, codebook_init_path=''):
+        super().__init__()
+        self.codebook_dim = embedding_dim
+        self.num_tokens = n_embed
+        self.beta = beta
+        self.decay = decay
+
+        # learnable = True if orthogonal_reg_weight > 0 else False
+        self.embedding = EmbeddingEMA(self.num_tokens, self.codebook_dim, decay, eps, kmeans_init, codebook_init_path)
+
+        self.statistic_code_usage = statistic_code_usage
+        if statistic_code_usage:
+            self.register_buffer('cluster_size', torch.zeros(n_embed))
+        if distributed.is_available() and distributed.is_initialized():
+            print("ddp is enable, so use ddp_reduce to sync the statistic_code_usage for each gpu!")
+            self.all_reduce_fn = distributed.all_reduce
+        else:
+            self.all_reduce_fn = nn.Identity()
+
+    def reset_cluster_size(self, device):
+        if self.statistic_code_usage:
+            self.register_buffer('cluster_size', torch.zeros(self.num_tokens))
+            self.cluster_size = self.cluster_size.to(device)
+
+    def forward(self, z):
+        # reshape z -> (batch, height, width, channel) and flatten
+        # z, 'b c h w -> b h w c'
+        # z = rearrange(z, 'b c h w -> b h w c')
+        # z = z.transpose(1, 2)
+        z = l2norm(z)
+        z_flattened = z.reshape(-1, self.codebook_dim)
+
+        self.embedding.init_embed_(z_flattened)
+
+        d = z_flattened.pow(2).sum(dim=1, keepdim=True) + \
+            self.embedding.weight.pow(2).sum(dim=1) - 2 * \
+            torch.einsum('bd,nd->bn', z_flattened, self.embedding.weight)  # 'n d -> d n'
+
+        encoding_indices = torch.argmin(d, dim=1)
+
+        z_q = self.embedding(encoding_indices).view(z.shape)
+
+        encodings = F.one_hot(encoding_indices, self.num_tokens).type(z.dtype)
+
+        if not self.training:
+            with torch.no_grad():
+                cluster_size = encodings.sum(0)
+                self.all_reduce_fn(cluster_size)
+                ema_inplace(self.cluster_size, cluster_size, self.decay)
+
+        if self.training and self.embedding.update:
+            # EMA cluster size
+
+            bins = encodings.sum(0)
+            self.all_reduce_fn(bins)
+
+            # self.embedding.cluster_size_ema_update(bins)
+            ema_inplace(self.cluster_size, bins, self.decay)
+
+            zero_mask = (bins == 0)
+            bins = bins.masked_fill(zero_mask, 1.)
+
+            embed_sum = z_flattened.t() @ encodings
+            self.all_reduce_fn(embed_sum)
+
+            embed_normalized = (embed_sum / bins.unsqueeze(0)).t()
+            embed_normalized = l2norm(embed_normalized)
+
+            embed_normalized = torch.where(zero_mask[..., None], self.embedding.weight,
+                                           embed_normalized)
+            norm_ema_inplace(self.embedding.weight, embed_normalized, self.decay)
+
+        # compute loss for embedding
+        loss = self.beta * F.mse_loss(z_q.detach(), z)
+
+        # preserve gradients
+        z_q = z + (z_q - z).detach()
+
+        # reshape back to match original input shape
+        # z_q, 'b h w c -> b c h w'
+        # z_q = rearrange(z_q, 'b h w c -> b c h w')
+        # z_q = z_q.transpose(1, 2)
+        return z_q, loss, encoding_indices
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..50a26e82399885d784cafc0c26cebc5c29e03610
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/builder.py
@@ -0,0 +1,11 @@
+from .speech_encoder import DualWrappedEncoder, WhisperWrappedEncoder
+
+
+def build_speech_encoder(config):
+    speech_encoder_type = getattr(config, 'speech_encoder_type', None)
+    if "whisper" in speech_encoder_type.lower():
+        return WhisperWrappedEncoder.load(config)
+    elif "dual" in speech_encoder_type.lower():
+        return DualWrappedEncoder(config)
+
+    raise ValueError(f'Unknown speech encoder: {speech_encoder_type}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/speech_encoder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/speech_encoder.py
new file mode 100644
index 0000000000000000000000000000000000000000..a5eea55c28c94fb59fb4b41a5d383e02cc3a0b47
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_encoder/speech_encoder.py
@@ -0,0 +1,76 @@
+import types
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import whisper
+from transformers import WhisperFeatureExtractor
+
+from vlmeval.vlm.ola.ola.model.speech_encoder.beats.BEATs import BEATs, BEATsConfig
+
+
+class WhisperWrappedEncoder:
+
+    @classmethod
+    def load(cls, model_config):
+
+        def replace_layer_norm(module):
+            from whisper.model import LayerNorm
+            for name, child in module.named_children():
+                if isinstance(child, LayerNorm):
+                    old_params = child.state_dict()
+                    new_layer_norm = nn.LayerNorm(child.normalized_shape, eps=child.eps, elementwise_affine=child.elementwise_affine)
+                    new_layer_norm.load_state_dict(old_params)
+                    setattr(module, name, new_layer_norm)
+                else:
+                    replace_layer_norm(child)
+
+        encoder = whisper.load_model(name=model_config.speech_encoder, device='cpu').encoder
+        replace_layer_norm(encoder)
+        return encoder
+
+class DualWrappedEncoder(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.config = config
+        self.whisper_model = self.load_whisper(config)
+        self.beats_model = self.load_beats(config)
+
+    def load_whisper(cls, model_config):
+
+        def replace_layer_norm(module):
+            from whisper.model import LayerNorm
+            for name, child in module.named_children():
+                if isinstance(child, LayerNorm):
+                    old_params = child.state_dict()
+                    new_layer_norm = nn.LayerNorm(child.normalized_shape, eps=child.eps, elementwise_affine=child.elementwise_affine)
+                    new_layer_norm.load_state_dict(old_params)
+                    setattr(module, name, new_layer_norm)
+                else:
+                    replace_layer_norm(child)
+
+        encoder = whisper.load_model(name=model_config.speech_encoder, device='cpu').encoder
+        replace_layer_norm(encoder)
+        return encoder
+
+    def load_beats(cls, model_config):
+        beats_path = model_config.music_encoder
+        print("Loading BEATs Model")
+        beats_ckpt = torch.load(beats_path, map_location='cpu')
+        beats_cfg = BEATsConfig(beats_ckpt['cfg'])
+        beats = BEATs(beats_cfg)
+        beats.load_state_dict(beats_ckpt['model'])
+        return beats
+
+    def forward(self, x, raw_wav=None, audio_padding_mask=None):
+        with torch.no_grad():
+            self.beats_model = self.beats_model.float()
+            speech_embeds = self.whisper_model(x.half())
+            audio_embeds, _ = self.beats_model.extract_features(raw_wav.float(), padding_mask=audio_padding_mask, feature_only=True)
+        if audio_embeds.size(1) < speech_embeds.size(1):
+            audio_embeds = F.pad(audio_embeds, (0, 0, 0, speech_embeds.size(1) - audio_embeds.size(1)))
+        elif audio_embeds.size(1) > speech_embeds.size(1):
+            speech_embeds = F.pad(speech_embeds, (0, 0, 0, audio_embeds.size(1) - speech_embeds.size(1)))
+        speech_embeds = torch.cat((speech_embeds, audio_embeds), dim=-1)
+        speech_embeds = speech_embeds.to(torch.bfloat16)
+        return speech_embeds
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_projector/builder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_projector/builder.py
new file mode 100644
index 0000000000000000000000000000000000000000..5c5959e229de1e43544dae3ae3a333693ba3e824
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_projector/builder.py
@@ -0,0 +1,9 @@
+from .speech_projector import EncoderProjectorConcat
+
+
+def build_speech_projector(config):
+    projector_type = getattr(config, 'speech_projector_type', 'linear')
+    if projector_type == 'linear':
+        return EncoderProjectorConcat(config)
+
+    raise ValueError(f'Unknown projector type: {projector_type}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_projector/speech_projector.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_projector/speech_projector.py
new file mode 100644
index 0000000000000000000000000000000000000000..5823fe3dd98c991c6438b5be4ec65e65c817de35
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/model/speech_projector/speech_projector.py
@@ -0,0 +1,49 @@
+import math
+
+import torch
+import torch.nn as nn
+
+
+class EncoderProjectorConcat(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.k = config.speech_encoder_ds_rate
+        self.encoder_dim = config.speech_encoder_hidden_size
+        self.llm_dim = config.hidden_size
+        self.linear1 = nn.Linear(self.encoder_dim * self.k, 2048)
+        self.relu = nn.ReLU()
+        self.linear2 = nn.Linear(2048, config.hidden_size)
+
+        embed_std = 1 / math.sqrt(config.hidden_size)
+        self.speech_newline = nn.Parameter(
+            torch.randn(config.hidden_size) * embed_std
+        )
+        self.speech_begin = nn.Parameter(
+            torch.randn(config.hidden_size) * embed_std
+        )
+        self.speech_end = nn.Parameter(
+            torch.randn(config.hidden_size) * embed_std
+        )
+
+    def forward(self, x):
+        batch_size, seq_len, dim = x.size()
+        num_frames_to_discard = seq_len % self.k
+        if num_frames_to_discard > 0:
+            x = x[:, :-num_frames_to_discard, :]
+        seq_len = x.size(1)
+
+        x = x.contiguous()
+        x = x.view(batch_size, seq_len // self.k, dim * self.k)
+        x = self.linear1(x)
+        x = self.relu(x)
+        x = self.linear2(x)
+        x = torch.cat([
+            x,
+            self.speech_newline.reshape(1, 1, -1).expand(batch_size, 1, -1).to(x.dtype)
+        ], dim=1)
+        begin = self.speech_begin.reshape(1, -1).to(x.dtype)
+        end = self.speech_end.reshape(1, -1).to(x.dtype)
+        x = x.flatten(0, 1)
+        x = torch.cat([begin, x, end], dim=0)
+        # x = x.flatten(0, 1)
+        return x
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..eb735fbee08083209e0a24eca84263f563dfb98f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola/utils.py
@@ -0,0 +1,214 @@
+# Adopted from https://github.com/haotian-liu/LLaVA. Below is the original copyright:
+#    Copyright 2023 Haotian Liu
+#
+#    Licensed under the Apache License, Version 2.0 (the "License");
+#    you may not use this file except in compliance with the License.
+#    You may obtain a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS,
+#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#    See the License for the specific language governing permissions and
+#    limitations under the License.
+
+import logging
+import logging.handlers
+import os
+import sys
+
+import torch
+import transformers
+
+from .constants import LOGDIR
+
+server_error_msg = "**NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.**"
+moderation_msg = "YOUR INPUT VIOLATES OUR CONTENT MODERATION GUIDELINES. PLEASE TRY AGAIN."
+
+handler = None
+
+
+def build_logger(logger_name, logger_filename):
+    global handler
+
+    formatter = logging.Formatter(
+        fmt="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S",
+    )
+
+    # Set the format of root handlers
+    if not logging.getLogger().handlers:
+        logging.basicConfig(level=logging.INFO)
+    logging.getLogger().handlers[0].setFormatter(formatter)
+
+    # Redirect stdout and stderr to loggers
+    stdout_logger = logging.getLogger("stdout")
+    stdout_logger.setLevel(logging.INFO)
+    sl = StreamToLogger(stdout_logger, logging.INFO)
+    sys.stdout = sl
+
+    stderr_logger = logging.getLogger("stderr")
+    stderr_logger.setLevel(logging.ERROR)
+    sl = StreamToLogger(stderr_logger, logging.ERROR)
+    sys.stderr = sl
+
+    # Get logger
+    logger = logging.getLogger(logger_name)
+    logger.setLevel(logging.INFO)
+
+    # Add a file handler for all loggers
+    if handler is None:
+        os.makedirs(LOGDIR, exist_ok=True)
+        filename = os.path.join(LOGDIR, logger_filename)
+        handler = logging.handlers.TimedRotatingFileHandler(
+            filename, when='D', utc=True, encoding='UTF-8')
+        handler.setFormatter(formatter)
+
+        for name, item in logging.root.manager.loggerDict.items():
+            if isinstance(item, logging.Logger):
+                item.addHandler(handler)
+
+    return logger
+
+
+class StreamToLogger(object):
+    """
+    Fake file-like stream object that redirects writes to a logger instance.
+    """
+    def __init__(self, logger, log_level=logging.INFO):
+        self.terminal = sys.stdout
+        self.logger = logger
+        self.log_level = log_level
+        self.linebuf = ''
+
+    def __getattr__(self, attr):
+        return getattr(self.terminal, attr)
+
+    def write(self, buf):
+        temp_linebuf = self.linebuf + buf
+        self.linebuf = ''
+        for line in temp_linebuf.splitlines(True):
+            # From the io.TextIOWrapper docs:
+            #   On output, if newline is None, any '\n' characters written
+            #   are translated to the system default line separator.
+            # By default sys.stdout.write() expects '\n' newlines and then
+            # translates them so this is still cross platform.
+            if line[-1] == '\n':
+                self.logger.log(self.log_level, line.rstrip())
+            else:
+                self.linebuf += line
+
+    def flush(self):
+        if self.linebuf != '':
+            self.logger.log(self.log_level, self.linebuf.rstrip())
+        self.linebuf = ''
+
+
+def maybe_zero_3(param, ignore_status=False, name=None):
+    from deepspeed import zero
+    from deepspeed.runtime.zero.partition_parameters import ZeroParamStatus
+    if hasattr(param, "ds_id"):
+        if param.ds_status == ZeroParamStatus.NOT_AVAILABLE:
+            if not ignore_status:
+                logging.warning(f"{name}: param.ds_status != ZeroParamStatus.NOT_AVAILABLE: {param.ds_status}")
+        with zero.GatheredParameters([param]):
+            param = param.data.detach().cpu().clone()
+    else:
+        param = param.detach().cpu().clone()
+    return param
+
+
+# Borrowed from peft.utils.get_peft_model_state_dict
+def get_peft_state_maybe_zero_3(named_params, bias):
+    if bias == "none":
+        to_return = {k: t for k, t in named_params if "lora_" in k}
+    elif bias == "all":
+        to_return = {k: t for k, t in named_params if "lora_" in k or "bias" in k}
+    elif bias == "lora_only":
+        to_return = {}
+        maybe_lora_bias = {}
+        lora_bias_names = set()
+        for k, t in named_params:
+            if "lora_" in k:
+                to_return[k] = t
+                bias_name = k.split("lora_")[0] + "bias"
+                lora_bias_names.add(bias_name)
+            elif "bias" in k:
+                maybe_lora_bias[k] = t
+        for k, t in maybe_lora_bias:
+            if bias_name in lora_bias_names:
+                to_return[bias_name] = t
+    else:
+        raise NotImplementedError
+    to_return = {k: maybe_zero_3(v, ignore_status=True) for k, v in to_return.items()}
+    return to_return
+
+
+def get_peft_state_non_lora_maybe_zero_3(named_params, require_grad_only=True):
+    to_return = {k: t for k, t in named_params if "lora_" not in k}
+    if require_grad_only:
+        to_return = {k: t for k, t in to_return.items() if t.requires_grad}
+    to_return = {k: maybe_zero_3(v, ignore_status=True).cpu() for k, v in to_return.items()}
+    return to_return
+
+
+def get_speech_projector_state_maybe_zero_3(named_params, keys_to_match):
+    to_return = {k: t for k, t in named_params if any(key_match in k for key_match in keys_to_match)}
+    to_return = {k: maybe_zero_3(v, ignore_status=True).cpu() for k, v in to_return.items()}
+    return to_return
+
+def lengths_to_padding_mask(lens):
+    bsz, max_lens = lens.size(0), torch.max(lens).item()
+    mask = torch.arange(max_lens).to(lens.device).view(1, max_lens)
+    mask = mask.expand(bsz, -1) >= lens.view(bsz, 1).expand(-1, max_lens)
+    return mask
+
+
+def lengths_to_mask(lens):
+    return ~lengths_to_padding_mask(lens)
+
+
+def disable_torch_init():
+    """
+    Disable the redundant torch default initialization to accelerate model creation.
+    """
+    import torch
+    setattr(torch.nn.Linear, "reset_parameters", lambda self: None)
+    setattr(torch.nn.LayerNorm, "reset_parameters", lambda self: None)
+
+
+def get_model_name_from_path(model_path):
+    model_path = model_path.strip("/")
+    model_paths = model_path.split("/")
+    if model_paths[-1].startswith('checkpoint-'):
+        return model_paths[-2] + "_" + model_paths[-1]
+    else:
+        return model_paths[-1]
+
+
+def violates_moderation(text):
+    """
+    Check whether the text violates OpenAI moderation API.
+    """
+    url = "https://api.openai.com/v1/moderations"
+    headers = {"Content-Type": "application/json",
+               "Authorization": "Bearer " + os.environ["OPENAI_API_KEY"]}
+    text = text.replace("\n", "")
+    data = "{" + '"input": ' + f'"{text}"' + "}"
+    data = data.encode("utf-8")
+    try:
+        ret = requests.post(url, headers=headers, data=data, timeout=5)
+        flagged = ret.json()["results"][0]["flagged"]
+    except requests.exceptions.RequestException as e:
+        flagged = False
+    except KeyError as e:
+        flagged = False
+
+    return flagged
+
+
+def pretty_print_semaphore(semaphore):
+    if semaphore is None:
+        return "None"
+    return f"Semaphore(value={semaphore._value}, locked={semaphore.locked()})"
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola_model.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola_model.py
new file mode 100644
index 0000000000000000000000000000000000000000..13f7772b6740817a08c64b295288f6bcda9915f9
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ola/ola_model.py
@@ -0,0 +1,176 @@
+import os
+import os.path as osp
+import string
+import sys
+import warnings
+from abc import abstractproperty
+
+import pandas as pd
+import torch
+from PIL import Image
+from transformers import CLIPImageProcessor
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import splitlen
+from ..base import BaseModel
+
+os.environ['LOWRES_RESIZE']="384x32"
+os.environ['HIGHRES_BASE']="0x32"
+os.environ['MAXRES']="1536"
+os.environ['MINRES']="0"
+os.environ['SIMPLE_ARCH']="1"
+os.environ['PAD2STRIDE']="1"
+os.environ['REGIONAL_POOL']='2x'
+os.environ['FORCE_NO_DOWNSAMPLE']="1"
+os.environ['LOAD_VISION_EARLY']="1"
+os.environ['SKIP_LOAD_VIT']="1"
+
+class Ola(BaseModel):
+
+    INSTALL_REQ = True
+    INTERLEAVE = True
+
+    def __init__(self,
+                 model_path='liuhaotian/llava_v1.5_7b',
+                 **kwargs):
+
+        from .ola.mm_utils import get_model_name_from_path
+        from .ola.model.builder import load_pretrained_model
+
+        assert osp.exists(model_path) or splitlen(model_path) == 2
+
+        model_name = get_model_name_from_path(model_path)
+
+        self.tokenizer, self.model, self.image_processor, self.context_len = load_pretrained_model(
+            model_path=model_path,
+            model_base=None,
+            use_flash_attn=True,
+        )
+
+        if self.image_processor is None:
+            self.image_processor = CLIPImageProcessor.from_pretrained("openai/clip-vit-large-patch14")
+            print('Using default image processor. ')
+
+        self._config = self.model.config
+
+
+        self.model = self.model.cuda()
+        self.conv_mode = 'v1_qwen2'
+
+        self.device = torch.device('cuda')
+
+        kwargs_default = dict(do_sample=False, temperature=0, max_new_tokens=512, top_p=None, num_beams=1, use_cache=True) # noqa E501
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+        warnings.warn(f'Following kwargs received: {self.kwargs}, will use as generation config. ')
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'multi-choice':
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        question = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            question = hint + '\n' + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f'\n{key}. {item}'
+        prompt = question
+
+        if len(options):
+            prompt += '\nAnswer with the option letter from the given choices directly.'
+        else:
+            prompt += '\nAnswer the question using a single word or phrase.'
+
+        message = [dict(type='image', value=s) for s in tgt_path]
+        message.append(dict(type='text', value=prompt))
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        from .ola.constants import (DEFAULT_IM_END_TOKEN, DEFAULT_IM_START_TOKEN,
+                                    DEFAULT_IMAGE_TOKEN, IMAGE_TOKEN_INDEX)
+        from .ola.conversation import SeparatorStyle, conv_templates
+        from .ola.datasets.preprocess import tokenizer_image_token
+        from .ola.mm_utils import KeywordsStoppingCriteria, process_anyres_highres_image_genli
+
+        # Support interleave text and image
+        conv = conv_templates[self.conv_mode].copy()
+        conv.append_message(conv.roles[0], 'PLACEHOLDER')
+        conv.append_message(conv.roles[1], None)
+        prompt = conv.get_prompt()
+
+        content, images = '', []
+        for msg in message:
+            if msg['type'] == 'text':
+                content += msg['value']
+            elif msg['type'] == 'image':
+                if 'MMVet' in msg['value'] or 'MMMU' in msg['value']:
+                    os.environ['USE_HIGHRES_ONLY'] = '1'
+                if self.model.config.mm_use_im_start_end:
+                    content += DEFAULT_IM_START_TOKEN + DEFAULT_IMAGE_TOKEN + DEFAULT_IM_END_TOKEN + '\n'
+                else:
+                    content += DEFAULT_IMAGE_TOKEN + '\n'
+                images.append(msg['value'])
+
+        images = [Image.open(s).convert('RGB') for s in images]
+        image_sizes = [img.size for img in images]
+        # args = abstractproperty()
+        # args.image_aspect_ratio = 'pad'
+        self.image_processor.do_resize = False
+        self.image_processor.do_center_crop = False
+        image_tensor, image_highres_tensor = [], []
+        for visual in images:
+            image_tensor_, image_highres_tensor_ = process_anyres_highres_image_genli(visual, self.image_processor)
+            image_tensor.append(image_tensor_)
+            image_highres_tensor.append(image_highres_tensor_)
+        if type(image_tensor) is list:
+            image_tensor = [_image.bfloat16().to("cuda") for _image in image_tensor]
+        else:
+            image_tensor = image_tensor.bfloat16().to("cuda")
+        if type(image_highres_tensor) is list:
+            image_highres_tensor = [_image.bfloat16().to("cuda") for _image in image_highres_tensor]
+        else:
+            image_highres_tensor = image_highres_tensor.bfloat16().to("cuda")
+        prompt = prompt.replace('PLACEHOLDER', content)
+
+        input_ids = tokenizer_image_token(
+            prompt, self.tokenizer, IMAGE_TOKEN_INDEX, return_tensors='pt').unsqueeze(0).cuda()
+        stop_str = '<|im_end|>'
+        keywords = [stop_str]
+        stopping_criteria = KeywordsStoppingCriteria(keywords, self.tokenizer, input_ids)
+
+        pad_token_ids = self.tokenizer.pad_token_id if self.tokenizer.pad_token_id is not None else self.tokenizer.eos_token_id
+        attention_masks = input_ids.ne(pad_token_ids).to(self.device)
+
+        speechs = [torch.zeros(1, 3000, 128).bfloat16().to('cuda')]
+        speech_lengths = [torch.LongTensor([3000]).to('cuda')]
+        speech_wavs = [torch.zeros([1, 480000]).to('cuda')]
+        speech_chunks = [torch.LongTensor([1]).to('cuda')]
+
+        with torch.inference_mode():
+            output_ids = self.model.generate(
+                input_ids, images=image_tensor, images_highres=image_highres_tensor, image_sizes=image_sizes,
+                modalities=['image'] * len(image_tensor),
+                speech=speechs,
+                speech_lengths=speech_lengths,
+                speech_chunks=speech_chunks,
+                speech_wav=speech_wavs,
+                attention_mask=attention_masks,
+                pad_token_id=pad_token_ids,
+                stopping_criteria=[stopping_criteria], **self.kwargs)
+
+        output = self.tokenizer.batch_decode(output_ids, skip_special_tokens=True)[0].strip()
+        print(content, output)
+        return output
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..793374857fc82a1bdcd689220b30e98cb846ac68
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/__init__.py
@@ -0,0 +1,3 @@
+from .ovis import Ovis, Ovis1_6, Ovis1_6_Plus, Ovis2, Ovis2_5, OvisU1
+
+__all__ = ['Ovis', 'Ovis1_6', 'Ovis1_6_Plus', 'Ovis2', 'OvisU1', 'Ovis2_5']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/ovis.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/ovis.py
new file mode 100644
index 0000000000000000000000000000000000000000..bae5df2496fc57bba06d2ab1504b6d6444798c06
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/ovis.py
@@ -0,0 +1,982 @@
+import torch
+from transformers import AutoModelForCausalLM
+
+from vlmeval.dataset import DATASET_MODALITY, DATASET_TYPE
+from vlmeval.smp import cn_string, listinstr
+from ..base import BaseModel
+
+
+class Ovis(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self, model_path='AIDC-AI/Ovis1.5-Llama3-8B', **kwargs):
+        assert model_path is not None
+        # Recommend to install `transformers==4.43.2` and `torch==2.1.2`.
+        self.model_path = model_path
+        self.device = torch.cuda.current_device()
+        self.dtype = torch.bfloat16
+        self.model = AutoModelForCausalLM.from_pretrained(
+            self.model_path,
+            torch_dtype=self.dtype,
+            multimodal_max_length=8192,
+            trust_remote_code=True
+        )
+        self.model = self.model.eval().to(device=self.device)
+        self.eos_token_id = self.model.generation_config.eos_token_id
+        self.text_tokenizer = self.model.get_text_tokenizer()
+        self.pad_token_id = self.text_tokenizer.pad_token_id
+        self.visual_tokenizer = self.model.get_visual_tokenizer()
+        self.conversation_formatter = self.model.get_conversation_formatter()
+        self.image_placeholder = '<image>'
+        self.gen_kwargs = dict(
+            max_new_tokens=1024,
+            do_sample=False,
+            top_p=None,
+            top_k=None,
+            temperature=None,
+            repetition_penalty=None,
+            eos_token_id=self.eos_token_id,
+            pad_token_id=self.pad_token_id,
+            use_cache=True
+        )
+        self.gen_kwargs.update(kwargs)
+
+    def use_custom_prompt(self, dataset):
+        if DATASET_TYPE(dataset) == 'Y/N' or DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        if DATASET_TYPE(dataset) == 'Y/N':
+            prompt = self.build_yorn_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_multi_choice_prompt(line, dataset)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+
+        # interleave dataset
+        if dataset.startswith('MMMU_'):
+            from vlmeval.dataset import MMMUDataset
+            message = MMMUDataset.split_MMMU(message)
+
+        return message
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question']
+        if listinstr(['HallusionBench'], dataset):
+            prompt += ' Please answer yes or no.'
+        prompt += '\n请用单个词或短语回答问题。' if cn_string(
+            prompt) else '\nAnswer the question using a single word or phrase.'
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None):
+        question = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            question = hint + '\n' + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f'\n{key}. {item}'
+        prompt = question
+
+        if len(options):
+            prompt += '\n请直接回答选项字母。' if cn_string(
+                prompt) else "\nAnswer with the option's letter from the given choices directly."
+        else:
+            prompt += '\n请直接回答问题。' if cn_string(prompt) else '\nAnswer the question directly.'
+
+        return prompt
+
+    def generate_inner(self, message, dataset=None):
+        prompt, input_ids, attention_mask, pixel_values = self.prepare_inputs(message)
+        output_ids = self.model.generate(
+            input_ids,
+            pixel_values=pixel_values,
+            attention_mask=attention_mask,
+            **self.gen_kwargs
+        )
+        response = self.text_tokenizer.decode(output_ids[0], skip_special_tokens=True).strip()
+
+        return response
+
+    def prepare_inputs(self, message):
+        # build query
+        images = [x['value'] for x in message if x['type'] == 'image']
+        texts = [x['value'] for x in message if x['type'] == 'text']
+        if len(images) == 0:
+            query = '\n'.join(texts)
+        elif len(images) == 1 and len(texts) == 1:
+            query = self.image_placeholder + '\n' + texts[0]
+        else:  # interleave sample
+            chunks = [x['value'] if x['type'] == 'text' else self.image_placeholder for x in message]
+            query = '\n'.join(chunks)
+
+        # format conversation
+        prompt, input_ids = self.conversation_formatter.format_query(query)
+        attention_mask = torch.ne(input_ids, self.text_tokenizer.pad_token_id)
+        input_ids = input_ids.unsqueeze(0).to(device=self.device)
+        attention_mask = attention_mask.unsqueeze(0).to(device=self.device)
+
+        # preprocess images
+        if len(images) == 0:
+            pixel_values = [None]
+        else:
+            preprocessed_images = [self.visual_tokenizer.preprocess_image(Image.open(image)) for image in images]
+            pixel_values = [torch.cat(preprocessed_images, dim=0).to(device=self.device, dtype=self.dtype)]
+
+        return prompt, input_ids, attention_mask, pixel_values
+
+
+class Ovis1_6(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self, model_path='AIDC-AI/Ovis1.6-Gemma2-9B', **kwargs):
+        assert model_path is not None
+        # Recommend to install `python=3.10`, `transformers==4.44.2`, `torch==2.2.0`, and `numpy==1.24.3`
+        self.model_path = model_path
+        self.device = torch.cuda.current_device()
+        self.dtype = torch.bfloat16
+        self.model = AutoModelForCausalLM.from_pretrained(
+            self.model_path,
+            torch_dtype=self.dtype,
+            multimodal_max_length=8192,
+            trust_remote_code=True
+        )
+        self.model = self.model.eval().to(device=self.device)
+        self.eos_token_id = self.model.generation_config.eos_token_id
+        self.text_tokenizer = self.model.get_text_tokenizer()
+        self.pad_token_id = self.text_tokenizer.pad_token_id
+        self.visual_tokenizer = self.model.get_visual_tokenizer()
+        self.max_partition = 9
+        self.image_placeholder = '<image>'
+        self.gen_kwargs = dict(
+            max_new_tokens=1024,
+            do_sample=False,
+            top_p=None,
+            top_k=None,
+            temperature=None,
+            repetition_penalty=None,
+            eos_token_id=self.eos_token_id,
+            pad_token_id=self.pad_token_id,
+            use_cache=True
+        )
+        self.gen_kwargs.update(kwargs)
+
+    def use_custom_prompt(self, dataset):
+        if DATASET_TYPE(dataset) == 'Y/N' or DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question'] + '\nAnswer the question using a single word or phrase.'
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None):
+        question = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            question = hint + '\n' + question
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f'\n{key}. {item}'
+        prompt = question
+
+        if len(options):
+            prompt += "\nAnswer with the option's letter from the given choices directly."
+
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        if DATASET_TYPE(dataset) == 'Y/N':
+            prompt = self.build_yorn_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_multi_choice_prompt(line, dataset)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+
+        # interleave dataset
+        if dataset.startswith('MMMU_'):
+            from vlmeval.dataset import MMMUDataset
+            message = MMMUDataset.split_MMMU(message)
+
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        prompt, input_ids, attention_mask, pixel_values = self.prepare_inputs(message)
+        output_ids = self.model.generate(
+            input_ids,
+            pixel_values=pixel_values,
+            attention_mask=attention_mask,
+            **self.gen_kwargs
+        )
+        response = self.text_tokenizer.decode(output_ids[0], skip_special_tokens=True)
+
+        return response
+
+    def prepare_inputs(self, message):
+        # build query
+        images = [x['value'] for x in message if x['type'] == 'image']
+        texts = [x['value'] for x in message if x['type'] == 'text']
+        if len(images) == 0:
+            query = '\n'.join(texts)
+        elif len(images) == 1 and len(texts) == 1:
+            query = self.image_placeholder + '\n' + texts[0]
+        else:  # interleaved sample
+            chunks = [x['value'] if x['type'] == 'text' else self.image_placeholder for x in message]
+            query = '\n'.join(chunks)
+
+        # preprocess inputs
+        prompt, input_ids, pixel_values = self.model.preprocess_inputs(
+            query, [Image.open(image) for image in images], max_partition=self.max_partition
+        )
+
+        # move to self.device
+        attention_mask = torch.ne(input_ids, self.text_tokenizer.pad_token_id)
+        input_ids = input_ids.unsqueeze(0).to(device=self.device)
+        attention_mask = attention_mask.unsqueeze(0).to(device=self.device)
+        pixel_values = [
+            pixel_values.to(device=self.device, dtype=self.dtype) if pixel_values is not None else None
+        ]
+
+        return prompt, input_ids, attention_mask, pixel_values
+
+
+class Ovis1_6_Plus(Ovis1_6):
+    # Recommend to install `python=3.10`, `transformers==4.46.2`, `torch==2.4.0`, and `numpy==1.25.0`
+
+    def build_mmmu_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        import string
+
+        import pandas as pd
+
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += 'Please select the correct answer from the options above.'
+        prompt = prompt.rstrip()
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        if dataset.startswith('MMMU_'):
+            prompt = self.build_mmmu_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'Y/N':
+            prompt = self.build_yorn_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_multi_choice_prompt(line, dataset)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+
+        message = [dict(type='image', value=s) for s in tgt_path] + [dict(type='text', value=prompt)]
+
+        return message
+
+
+class Ovis2(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    SIZE_DICT = {
+        (24, 896): '1B',  # (num_hidden_layers, hidden_size)
+        (28, 1536): '2B',
+        (36, 2048): '4B',
+        (28, 3584): '8B',
+        (48, 5120): '16B',
+        (64, 5120): '34B'
+    }
+
+    def __init__(self, model_path='AIDC-AI/Ovis2-8B', **kwargs):
+        assert model_path is not None
+        # Recommend to install `python=3.10`, `transformers==4.46.2`, `torch==2.4.0`, and `numpy==1.25.0`
+        self.model_path = model_path
+        self.device = torch.cuda.current_device()
+        self.dtype = torch.bfloat16
+        self.model = AutoModelForCausalLM.from_pretrained(
+            self.model_path,
+            torch_dtype=self.dtype,
+            multimodal_max_length=32768,
+            trust_remote_code=True
+        )
+        self.size = self.SIZE_DICT[
+            (self.model.config.llm_config.num_hidden_layers, self.model.config.llm_config.hidden_size)]
+        self.model = self.model.eval().to(device=self.device)
+        self.eos_token_id = self.model.generation_config.eos_token_id
+        self.text_tokenizer = self.model.get_text_tokenizer()
+        self.pad_token_id = self.text_tokenizer.pad_token_id
+        self.visual_tokenizer = self.model.get_visual_tokenizer()
+        self.image_placeholder = '<image>'
+        self.gen_kwargs = dict(
+            max_new_tokens=1024,
+            do_sample=False,
+            top_p=None,
+            top_k=None,
+            temperature=None,
+            repetition_penalty=None,
+            eos_token_id=self.eos_token_id,
+            pad_token_id=self.pad_token_id,
+            use_cache=True
+        )
+        self.use_cot = {
+            '1B': {'MathVerse', 'MathVision'},
+            '2B': {'MMVet', 'MMStar', 'MathVerse', 'MathVision'},
+            '4B': {'MathVerse', 'MathVision'},
+            '8B': {'MMVet', 'MMStar', 'MMMU', 'MathVista', 'MathVerse', 'MathVision'},
+            '16B': {'MMVet', 'MMStar', 'MMMU', 'MathVista', 'MathVerse', 'MathVision'},
+            '34B':  {'MMVet', 'MMStar', 'MMMU', 'MathVista', 'MathVerse', 'MathVision'}
+        }
+        self.frame_selector = None
+        if kwargs.pop("frame_selection", False):
+            from .utils.mdp3 import MDP3
+            self.frame_selector = MDP3(
+                n_selection=int(kwargs.pop("n_frames", 32)),
+                visual_encoder_name_or_path=kwargs.pop("frame_selection_vlm", "google/siglip-so400m-patch14-384"),
+                device=f"cuda:{self.device}"
+            )
+        self.gen_kwargs.update(kwargs)
+
+    def use_custom_prompt(self, dataset):
+        if any(dataset.startswith(prefix) for prefix in ['MMVet', 'MathVista', 'MathVerse', 'MathVision']):
+            return True
+        if DATASET_TYPE(dataset) == 'Y/N' or DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question']
+        if listinstr(['HallusionBench'], dataset) and self.size == '34B':
+            prompt += ' Please answer yes or no.'
+        prompt += '\nAnswer the question using a single word or phrase.'
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+
+        if len(options):
+            if use_cot:
+                prompt += "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by the final solution."
+            else:
+                prompt += "\nAnswer with the option's letter from the given choices directly."
+
+        return prompt
+
+    def build_mmvet_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        if use_cot:
+            prompt += "\nProvide a step-by-step solution to the problem carefully."
+        return prompt
+
+    def build_math_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        if use_cot:
+            prompt += "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by the final solution."
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        use_cot = any(dataset.startswith(prefix) for prefix in self.use_cot[self.size])
+
+        if dataset == 'MMVet':
+            prompt = self.build_mmvet_prompt(line, dataset, use_cot)
+        elif any(dataset.startswith(prefix) for prefix in ('MathVista', 'MathVerse', 'MathVision')):
+            prompt = self.build_math_prompt(line, dataset, use_cot)
+        elif DATASET_TYPE(dataset) == 'Y/N':
+            prompt = self.build_yorn_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_multi_choice_prompt(line, dataset, use_cot)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+
+        message = [dict(type='image', value=s) for s in tgt_path] + [dict(type='text', value=prompt)]
+
+        # interleave dataset
+        if dataset.startswith('MMMU_'):
+            from vlmeval.dataset import MMMUDataset
+            message = MMMUDataset.split_MMMU(message)
+
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        def _extract_answer(text):
+            answer_index = text.lower().find('the answer is')
+            if answer_index != -1:
+                answer_index += len('the answer is')
+                answer = text[answer_index:].lstrip(':').strip()
+            else:
+                answer = text
+            return answer
+
+        # DynaMath
+        if dataset == 'DynaMath' and self.size == '34B':
+            message[-1]['value'] += "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by the final solution."
+
+        prompt, input_ids, attention_mask, pixel_values, max_partition = self.prepare_inputs(message, dataset)
+        output_ids = self.model.generate(
+            input_ids,
+            pixel_values=pixel_values,
+            attention_mask=attention_mask,
+            **self.gen_kwargs
+        )
+        response = self.text_tokenizer.decode(output_ids[0], skip_special_tokens=True)
+
+        if "conclude with 'the answer is' followed by the final solution." in prompt:
+            response = _extract_answer(response)
+
+        return response
+
+    def prepare_inputs(self, message, dataset=None):
+        # build query
+        images = [x['value'] for x in message if x['type'] == 'image']
+        texts = [x['value'] for x in message if x['type'] == 'text']
+        if DATASET_MODALITY(dataset) == 'VIDEO': # video inputs
+            chunks = [self.image_placeholder for x in message if x['type'] != 'text']
+            chunks += [x['value'].strip() for x in message if x['type'] == 'text' and x['value'] != '']
+            query = '\n'.join(chunks)
+        elif len(images) == 0: # text-only inputs
+            query = '\n'.join(texts)
+        elif len(images) == 1 and len(texts) == 1: # single-image inputs
+            query = self.image_placeholder + '\n' + texts[0]
+        else:  # interleaved inputs
+            chunks = [x['value'].strip() if x['type'] == 'text' else self.image_placeholder for x in message]
+            query = '\n'.join(chunks)
+
+        # preprocess inputs
+        if DATASET_MODALITY(dataset) == 'VIDEO':
+            max_partition = 1
+        elif (dataset != None) and any(
+            dataset.startswith(prefix) for prefix in
+            ('HallusionBench', 'TextVQA', 'ChartQA', 'OCRBench', 'InfoVQA', 'DocVQA', 'MTVQA')):
+            max_partition = 12
+        elif len(images) > 1:
+            max_partition = max(1, 12 // len(images))
+        else:
+            max_partition = 9
+
+        prompt, input_ids, pixel_values = self.model.preprocess_inputs(
+            query, [Image.open(image) for image in images], max_partition=max_partition, frame_selector=self.frame_selector
+        )
+
+        # move to self.device
+        attention_mask = torch.ne(input_ids, self.text_tokenizer.pad_token_id)
+        input_ids = input_ids.unsqueeze(0).to(device=self.device)
+        attention_mask = attention_mask.unsqueeze(0).to(device=self.device)
+        pixel_values = [
+            pixel_values.to(device=self.device, dtype=self.dtype) if pixel_values is not None else None
+        ]
+
+        return prompt, input_ids, attention_mask, pixel_values, max_partition
+
+
+class OvisU1(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self, model_path='AIDC-AI/Ovis-U1-3B', **kwargs):
+        assert model_path is not None
+        # Recommend to install `transformers==4.51.3`, `torch==2.4.0`, and `numpy==1.24.3`
+        self.model_path = model_path
+        self.device = torch.cuda.current_device()
+        self.dtype = torch.bfloat16
+
+        self.model = AutoModelForCausalLM.from_pretrained(
+            self.model_path,
+            torch_dtype=self.dtype,
+            multimodal_max_length=32768,
+            trust_remote_code=True
+        )
+        self.model = self.model.eval().to(device=self.device)
+        self.text_tokenizer = self.model.get_text_tokenizer()
+        self.pad_token_id = self.text_tokenizer.pad_token_id
+        self.eos_token_id = self.text_tokenizer.eos_token_id
+        self.visual_tokenizer = self.model.get_visual_tokenizer()
+        self.image_placeholder = '<image>'
+        self.gen_kwargs = dict(
+            max_new_tokens=1024,
+            do_sample=False,
+            top_p=None,
+            top_k=None,
+            temperature=None,
+            repetition_penalty=None,
+            eos_token_id=self.eos_token_id,
+            pad_token_id=self.pad_token_id,
+            use_cache=True
+        )
+        self.min_pixels = 200704  # 448*448
+        self.max_pixels = 2408448  # 1344*1792
+        self.frame_selector = None
+        if kwargs.pop("frame_selection", False):
+            from .utils.mdp3 import MDP3
+            self.frame_selector = MDP3(
+                n_selection=int(kwargs.pop("n_frames", 32)),
+                visual_encoder_name_or_path=kwargs.pop("frame_selection_vlm", "google/siglip-so400m-patch14-384"),
+                device=f"cuda:{self.device}"
+            )
+        self.gen_kwargs.update(kwargs)
+        self.use_cot = {'MMMU'}
+
+    def use_custom_prompt(self, dataset):
+        if any(dataset.startswith(prefix) for prefix in ['MMVet', 'MathVista', 'MathVerse', 'MathVision']):
+            return True
+        if DATASET_TYPE(dataset) == 'Y/N' or DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question']
+        if listinstr(['HallusionBench'], dataset):  # and self.size == '34B':
+            prompt += ' Please answer yes or no.'
+        prompt += '\nAnswer the question using a single word or phrase.'
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+
+        if len(options):
+            if use_cot:
+                prompt += "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by the final solution."
+            else:
+                prompt += "\nAnswer with the option's letter from the given choices directly."
+
+        return prompt
+
+    def build_mmvet_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        if use_cot:
+            prompt += "\nProvide a step-by-step solution to the problem carefully."
+        return prompt
+
+    def build_math_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        if use_cot:
+            prompt += "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by the final solution."
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        use_cot = any(dataset.startswith(prefix) for prefix in self.use_cot)
+
+        if dataset == 'MMVet':
+            prompt = self.build_mmvet_prompt(line, dataset, use_cot)
+        elif any(dataset.startswith(prefix) for prefix in ('MathVista', 'MathVerse', 'MathVision')):
+            prompt = self.build_math_prompt(line, dataset, use_cot)
+        elif DATASET_TYPE(dataset) == 'Y/N':
+            prompt = self.build_yorn_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_multi_choice_prompt(line, dataset, use_cot)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+
+        message = [dict(type='image', value=s) for s in tgt_path] + [dict(type='text', value=prompt)]
+
+        # interleave dataset
+        if dataset.startswith('MMMU_'):
+            from vlmeval.dataset import MMMUDataset
+            message = MMMUDataset.split_MMMU(message)
+
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        def _extract_answer(text):
+            answer_index = text.lower().find('the answer is')
+            if answer_index != -1:
+                answer_index += len('the answer is')
+                answer = text[answer_index:].lstrip(':').strip()
+            else:
+                answer = text
+            return answer
+
+        # DynaMath
+        if dataset == 'DynaMath':
+            message[-1][
+                'value'] += "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by the final solution."
+
+        prompt, input_ids, attention_mask, pixel_values, grid_thws = self.prepare_inputs(message, dataset)
+        output_ids = self.model.generate(
+            input_ids,
+            pixel_values=pixel_values,
+            grid_thws=grid_thws,
+            attention_mask=attention_mask,
+            **self.gen_kwargs
+        )
+        response = self.text_tokenizer.decode(output_ids[0], skip_special_tokens=True)
+
+        print('\n========================************========================')
+        print(f'prompt: {prompt}<<<\n')
+        print(f'output: {response}\n')
+
+        think_end = response.rfind('</think>')
+        if think_end != -1:
+            think_end += len('</think>')
+            response = response[think_end:].strip()
+            print(f'extract answer: {response}\n')
+
+        if "conclude with 'the answer is' followed by the final solution." in prompt:
+            response = _extract_answer(response)
+            print(f'extract answer: {response}\n')
+
+        print('------------------------------------------------------------\n', flush=True)
+
+        return response
+
+    def prepare_inputs(self, message, dataset=None):
+        # build query
+        images = [x['value'] for x in message if x['type'] == 'image']
+        texts = [x['value'] for x in message if x['type'] == 'text']
+        # print(f"=============={DATASET_MODALITY(dataset)}============")
+        if DATASET_MODALITY(dataset) == 'VIDEO':  # video inputs
+            chunks = [self.image_placeholder for x in message if x['type'] != 'text']
+            chunks += [x['value'].strip() for x in message if x['type'] == 'text' and x['value'] != '']
+            query = '\n'.join(chunks)
+            # print(query, chunks)
+        elif len(images) == 0:  # text-only inputs
+            query = '\n'.join(texts)
+        elif len(images) == 1 and len(texts) == 1:  # single-image inputs
+            query = self.image_placeholder + '\n' + texts[0]
+        else:  # interleaved inputs
+            chunks = [x['value'].strip() if x['type'] == 'text' else self.image_placeholder for x in message]
+            query = '\n'.join(chunks)
+
+        # preprocess inputs
+        min_pixels = self.min_pixels
+        max_pixels = self.max_pixels
+        enable_thinking = os.getenv("OvisThink") == 'True'
+        prompt, input_ids, pixel_values, grid_thws = self.model.preprocess_inputs(
+            query, [Image.open(image) for image in images],
+            frame_selector=self.frame_selector,
+            enable_thinking=enable_thinking,
+            min_pixels=min_pixels,
+            max_pixels=max_pixels,  # 2000*2000,
+        )
+
+        attention_mask = torch.ne(input_ids, self.text_tokenizer.pad_token_id)
+        input_ids = input_ids.unsqueeze(0).to(device=self.device)
+        attention_mask = attention_mask.unsqueeze(0).to(device=self.device)
+        pixel_values = torch.cat([
+            pixel_values.to(device=self.device, dtype=self.dtype) if pixel_values is not None else None
+        ], dim=0)
+        grid_thws = torch.cat([
+            grid_thws.to(device=self.device) if grid_thws is not None else None
+        ], dim=0)
+
+        return prompt, input_ids, attention_mask, pixel_values, grid_thws
+
+
+class Ovis2_5(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    SIZE_DICT = {
+        (28, 2048): '2B',  # (num_hidden_layers, hidden_size)
+        (36, 4096): '9B'
+    }
+
+    def __init__(self, model_path='AIDC-AI/Ovis2.5-9B', **kwargs):
+        assert model_path is not None
+        # Recommend to install dependencies as follows:
+        # `pip install vllm==0.10.2 --extra-index-url https://wheels.vllm.ai/0.10.2/`
+
+        os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
+        from vllm import LLM
+        self.model_path = model_path
+        self.dtype = torch.bfloat16
+        dist_kwargs = dict()
+        prev_local_rank = os.getenv("LOCAL_RANK")
+        if prev_local_rank is not None:
+            os.environ["LOCAL_RANK"] = "0"
+            torch.cuda.set_device(0)
+            dist_kwargs["distributed_executor_backend"] = "external_launcher"
+        self.model = LLM(
+            model=self.model_path,
+            dtype=self.dtype,
+            trust_remote_code=True,
+            tensor_parallel_size=1,
+            gpu_memory_utilization=0.7,
+            **dist_kwargs
+        )
+        size_key = (
+            self.model.llm_engine.model_config.hf_config.llm_config.num_hidden_layers,
+            self.model.llm_engine.model_config.hf_config.llm_config.hidden_size
+        )
+        self.size = self.SIZE_DICT[size_key]
+        if prev_local_rank is not None:
+            os.environ["LOCAL_RANK"] = prev_local_rank
+        self.tokenizer = self.model.get_tokenizer()
+        self.image_placeholder = '<image>'
+        self.video_placeholder = '<video>'
+        self.benchmark_with_thinking_map = {
+            '2B': ('HallusionBench', 'MMStar', 'MMMU', 'MathVista', 'MathVerse', 'MathVision', 'LogicVista', 'WeMath', 'DynaMath'),
+            '9B': ('MMBench', 'HallusionBench', 'MMStar', 'MMMU', 'MathVista', 'MathVerse', 'MathVision', 'LogicVista', 'WeMath', 'DynaMath')
+        }
+        self.objective_prompt_suffix = "End your response with 'Final answer: '."
+
+    def use_custom_prompt(self, dataset):
+        if any(dataset.startswith(prefix) for prefix in
+               ['MathVista', 'MathVerse', 'MathVision', 'LogicVista']):
+            return True
+        if DATASET_TYPE(dataset) == 'Y/N' or DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question']
+        if listinstr(['HallusionBench'], dataset):
+            prompt += ' Please answer yes or no.'
+        prompt += f'\n{self.objective_prompt_suffix}'
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+
+        if any(dataset.startswith(prefix) for prefix in ['AI2D']):
+            prompt += "\nAnswer with the option's letter from the given choices directly."
+        else:
+            prompt += f'\n{self.objective_prompt_suffix}'
+
+        return prompt
+
+    def build_math_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += f'\n{self.objective_prompt_suffix}'
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        if any(dataset.startswith(prefix) for prefix in ['MathVista', 'MathVerse', 'MathVision', 'LogicVista']):
+            prompt = self.build_math_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'Y/N':
+            prompt = self.build_yorn_prompt(line, dataset)
+        elif DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_multi_choice_prompt(line, dataset)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+
+        message = [dict(type='image', value=s) for s in tgt_path] + [dict(type='text', value=prompt)]
+
+        # interleave dataset
+        if dataset.startswith('MMMU_'):
+            from vlmeval.dataset import MMMUDataset
+            message = MMMUDataset.split_MMMU(message)
+
+        return message
+
+    def vllm_infer(self, messages, min_pixels=1024 * 1024, max_pixels=1792 * 1792, enable_thinking=False, max_new_tokens=8192, thinking_budget=6144):
+        from vllm import SamplingParams
+        if enable_thinking:
+            sampling_params = SamplingParams(
+                max_tokens=max_new_tokens,
+                repetition_penalty=1.05,
+                temperature=0.6,
+                top_p=0.95,
+                top_k=20
+            )
+        else:
+            sampling_params = SamplingParams(
+                max_tokens=max_new_tokens,
+                temperature=0.0
+            )
+
+        prompt = self.tokenizer.apply_chat_template(
+            messages,
+            tokenize=False, add_generation_prompt=True, enable_thinking=enable_thinking
+        )
+        vllm_input = {"prompt": prompt}
+        images = []
+        for message in messages:
+            images.extend([x["image"] for x in message["content"] if x["type"] == "image"])
+        if images:
+            vllm_input["multi_modal_data"] = {"image": images}
+            vllm_input["mm_processor_kwargs"] = {"images_kwargs": {"min_pixels": min_pixels, "max_pixels": max_pixels}}
+        vllm_inputs = [vllm_input]
+        use_thinking_budget = enable_thinking and thinking_budget is not None and thinking_budget > 0
+        if not use_thinking_budget:
+            vllm_outputs = self.model.generate(vllm_inputs, sampling_params, use_tqdm=False)
+        else:
+            phase1_params = sampling_params.clone()
+            phase1_params.max_tokens = thinking_budget
+            phase1_outputs = self.model.generate(vllm_inputs, phase1_params, use_tqdm=False)
+
+            phase2_inputs = []
+            vllm_outputs = [None] * len(vllm_inputs)  # 用于存放最终结果的列表
+
+            for i, p1_output in enumerate(phase1_outputs):
+                p1_text = p1_output.outputs[0].text
+                p1_finish_reason = p1_output.outputs[0].finish_reason
+
+                # 分支 1: 模型在预算内自然停止 (生成了EOS)
+                if p1_finish_reason == 'stop':
+                    # 生成已完成，直接使用这个结果，无需进入Phase 2
+                    vllm_outputs[i] = p1_output
+                    continue
+
+                # 分支 2 & 3: 模型因达到长度限制而停止
+                elif p1_finish_reason == 'length':
+                    p2_input = vllm_inputs[i].copy()
+
+                    # 分支 2: 输出中包含 </think>
+                    if '</think>' in p1_text:
+                        p2_input["prompt"] = vllm_inputs[i]["prompt"] + p1_text
+
+                    # 分支 3: 输出中不包含 </think>
+                    else:
+                        forced_stop_text = (
+                            "\n\nConsidering the limited time by the user, I have to give the solution "
+                            "based on the thinking directly now.\n</think>\n\n"
+                        )
+                        p1_text += forced_stop_text
+                        p2_input["prompt"] = vllm_inputs[i]["prompt"] + p1_text
+
+                    # 记录中间文本，并准备第二阶段的输入
+                    p2_input["intermediate_text"] = p1_text  # 暂存第一阶段文本
+                    p2_input["original_index"] = i  # 记录原始索引，以便回填结果
+                    phase2_inputs.append(p2_input)
+
+            # 执行阶段 2 (如果需要)
+            if phase2_inputs:
+                phase2_params = sampling_params.clone()
+                phase2_params.max_tokens = max_new_tokens - thinking_budget
+
+                phase2_outputs = self.model.generate(phase2_inputs, phase2_params, use_tqdm=False)
+
+                # 组合两阶段的结果
+                for i, p2_output in enumerate(phase2_outputs):
+                    original_index = phase2_inputs[i]["original_index"]
+                    intermediate_text = phase2_inputs[i]["intermediate_text"]
+
+                    # 创建一个模拟的 RequestOutput 对象来存储合并后的结果
+                    # 这样做是为了与后处理流程保持一致
+                    final_text = intermediate_text + p2_output.outputs[0].text
+
+                    # 从p1复制一个输出对象来修改
+                    final_output = phase1_outputs[original_index]
+                    final_output.outputs[0].text = final_text
+                    vllm_outputs[original_index] = final_output
+        outputs = []
+        for i, vllm_output in enumerate(vllm_outputs):
+            raw_output_text = vllm_output.outputs[0].text
+            if '</think>' in raw_output_text:
+                thinking, response = raw_output_text.split('</think>', 1)
+                thinking = thinking.split('<think>', 1)[-1].strip()
+            else:
+                thinking, response = "", raw_output_text
+            response = response.strip()
+            outputs.append({"response": response, "thinking": thinking})
+        return outputs[0], vllm_inputs[0]["prompt"]
+
+    def generate_inner(self, message, dataset=None):
+        def _extract_final_answer(response):
+            import re
+            """
+            从字符串中抽取"the final answer is" 或 "final answer" 后面的内容。
+
+            Args:
+              response: 输入的字符串。
+
+            Returns:
+              抽取并清理后的内容字符串，如果找不到则返回response。
+            """
+            match = re.search(r'.*(?:the\s+final\s+answer\s+is|final\s+answer\s*:)\s*[:\s]*(.*)',
+                              response,
+                              re.IGNORECASE | re.DOTALL)
+
+            if match:
+                return match.group(1)
+            else:
+                return response
+
+        messages = self.prepare_inputs(message, dataset)
+        enable_thinking = any(dataset.startswith(prefix) for prefix in self.benchmark_with_thinking_map[self.size])
+        min_pixels = 448 * 448 if any(dataset.startswith(prefix) for prefix in ['OCRBench']) else 1024 * 1024
+        max_pixels = 1792 * 1792
+        vllm_output, prompt = self.vllm_infer(messages, min_pixels, max_pixels, enable_thinking)
+        response = vllm_output["response"]
+
+        if self.objective_prompt_suffix in prompt:
+            response = _extract_final_answer(response)
+
+        return response
+
+    def prepare_inputs(self, message, dataset=None):
+        content = []
+        for item in message:
+            if item['type'] == 'image':
+                image = Image.open(item['value'])
+                content.append(dict(type='image', image=image))
+            elif item['type'] == 'text':
+                text = item['value']
+                content.append(dict(type='text', text=text))
+        return [dict(role="user", content=content)]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/utils/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/utils/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/utils/mdp3.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/utils/mdp3.py
new file mode 100644
index 0000000000000000000000000000000000000000..f4fd27a29b3ce3d274d395135f48536fe0e17377
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ovis/utils/mdp3.py
@@ -0,0 +1,378 @@
+# MDP3: A Training-free Approach for List-wise Frame Selection in Video-LLMs
+# https://arxiv.org/abs/2501.02885
+import copy
+import time
+from contextlib import contextmanager
+
+import torch
+from torch import nn
+from transformers import AutoModel, AutoProcessor
+
+
+@contextmanager
+def timer(hint=""):
+    start = time.perf_counter()
+    yield
+    end = time.perf_counter()
+    print(f"{hint} runtime: {end - start:.4f} s")
+
+
+INF = 0x7fffffff
+
+
+class VisualEncoder():
+
+    def __init__(self, model_path, device="cuda"):
+        self.device = device
+        self.model_path = model_path
+        self.model = AutoModel.from_pretrained(self.model_path)
+        self.model = self.model.to(device=self.device)
+        self.processor = AutoProcessor.from_pretrained(self.model_path)
+
+    def __call__(self, images, texts, clear_prompt=False):
+        if clear_prompt:
+            texts = self.clear_prompt(copy.deepcopy(texts))
+
+        with timer("visual processor"):
+            inputs = self.processor(
+                text=texts, images=images, padding="max_length", return_tensors="pt").to(self.model.device)
+
+        stride_num = (int(inputs["input_ids"].shape[-1]) + 63) // 64
+        stride = (inputs["input_ids"].shape[-1] + stride_num - 1) // stride_num
+
+        input_id_heads, input_id_tails = [], []
+        l, r = 0, inputs["input_ids"].shape[-1]
+        while l < r:
+            input_id_heads.append(inputs["input_ids"][:, l:l + stride])
+            l += stride
+            if l < r:
+                input_id_tails.append(inputs["input_ids"][:, r - stride:r])
+                r -= stride
+
+        input_ids = input_id_heads + input_id_tails[::-1]
+        input_ids = torch.cat(input_ids)
+
+        with timer("extract embeds"):
+            with torch.no_grad():
+                with torch.autocast(self.device):
+                    outputs = self.model(
+                        input_ids, pixel_values=inputs["pixel_values"])
+        image_embeds = outputs.image_embeds
+        text_embeds = outputs.text_embeds
+        return image_embeds, text_embeds.mean(dim=0, keepdim=True)
+
+    def clear_prompt(self, prompt):
+        heads = [
+            "Select the best answer to the following multiple-choice question based on the video and the subtitles. Respond with only the letter (A, B, C, or D) of the correct option.",
+            "Select the best answer to the following multiple-choice question based on the video. Respond with only the letter (A, B, C, or D) of the correct option.",
+            "Carefully watch the video and pay attention to the cause and sequence of events, the detail and movement of objects, and the action and pose of persons. Based on your observations, select the best option that accurately addresses the question.",
+            "Carefully watch this video and pay attention to every detail. Based on your observations, select the best option that accurately addresses the question."
+        ]
+        tails = [
+            "Answer with the option's letter from the given choices directly.",
+            "The best answer is:",
+            "Answer the question using a single word or phrase.",
+            "Only give the best option.",
+            "Best option: (",
+            "Please directly give the best option:"
+        ]
+        for head in heads:
+            prompt = prompt.split(head)[-1]
+        for tail in tails:
+            prompt = prompt.split(tail)[0]
+        prompt = prompt.strip()
+
+        return prompt
+
+
+class MDP3:
+
+    def __init__(self, n_selection=16, visual_encoder_name_or_path="google/siglip-so400m-patch14-384", device="cuda"):
+        super().__init__()
+        self.n_selection = n_selection
+        self.lamda = 0.2
+        self.segment_size = -1
+        self.condition_size = 1
+
+        self.kernel = MultiGaussianKernel(
+            alphas=[2 ** k for k in list(range(-3, 2))])
+        self.ve = VisualEncoder(model_path=visual_encoder_name_or_path, device=device)
+
+    def __call__(self, conversations, frames, clear_prompt=True):
+        if len(frames) <= self.n_selection:
+            return conversations, frames
+
+        prompt = "\n".join([x["value"] for x in conversations if x["from"] == "human"])
+        with timer("** VLMs Process & Extract"):
+            frame_embeds, text_embeds = self.ve(frames, prompt, clear_prompt)
+        with timer("Select Frames"):
+            with torch.no_grad():
+                selected_idx = self._select_frames_fast(frame_embeds, text_embeds)
+            # clean conversion
+            ret_conversions = []
+            img_cnt = 0
+            for conv in conversations:
+                prompt_clips = conv["value"].split("<image>")
+                conv_v = []
+                for idx, clip in enumerate(prompt_clips):
+                    if clip != "\n":
+                        conv_v.append(clip)
+                    if img_cnt in selected_idx and idx < len(prompt_clips) - 1:
+                        if clip == "\n":
+                            conv_v.append(clip)
+                        conv_v.append("<image>")
+                    img_cnt += 1
+
+                ret_conversions.append({"from": conv["from"], "value": ''.join(conv_v)})
+
+            ret_frames = [frames[idx] for idx in selected_idx]
+
+            return ret_conversions, ret_frames
+
+    def cal_obj(self, selected_images_embeds, text_embed):
+        kernel_matrix = self.kernel(
+            torch.cat([text_embed, selected_images_embeds]))
+        r, S_matrix = kernel_matrix[0:1, 1:], kernel_matrix[1:, 1:]
+        ret_score = (1. / self.lamda * 2 * torch.log(r).sum()) + \
+                    torch.linalg.slogdet(S_matrix).logabsdet
+        return ret_score
+
+    def _select_frames(self, image_embeds, text_embeds):
+        # initializing dynamic programing
+        N_image = len(image_embeds)
+        if self.segment_size <= 0:
+            segment_size = N_image
+        else:
+            segment_size = self.segment_size
+        segment_num = (N_image + segment_size - 1) // segment_size
+        dp = [[0.] + [-INF] * self.n_selection for _ in range(segment_num + 1)]
+        trace = [[[] for _ in range(self.n_selection + 1)]
+                 for _ in range(segment_num + 1)]
+
+        for seg_idx in range(1, segment_num + 1):
+            for selected_num in range(1, min(self.n_selection, seg_idx * segment_size) + 1):
+                for to_select_num in range(0, min(selected_num, segment_size) + 1):
+                    cur_score, cur_trace = self.seqdpp_select(
+                        text_embeds=text_embeds,
+                        image_embeds=image_embeds,
+                        conditional_index=trace[seg_idx - 1][selected_num - to_select_num][
+                                          -min(self.condition_size,
+                                               len(trace[seg_idx - 1][selected_num - to_select_num])):],
+                        candidate_index=range(
+                            (seg_idx - 1) * segment_size, seg_idx * segment_size),
+                        to_select_num=to_select_num
+                    )
+                    cur_score = dp[seg_idx - 1][selected_num -
+                                                to_select_num] + cur_score
+                    cur_trace = trace[
+                                    seg_idx - 1][selected_num - to_select_num] + cur_trace
+                    if cur_score > dp[seg_idx][selected_num]:
+                        dp[seg_idx][selected_num] = cur_score
+                        trace[seg_idx][selected_num] = cur_trace
+        return trace[segment_num][self.n_selection]
+
+    def _select_frames_fast(self, image_embeds, text_embeds):
+        # initializing dynamic programing
+        N_image = len(image_embeds)
+        if self.segment_size <= 0:
+            segment_size = N_image
+        else:
+            segment_size = self.segment_size
+        segment_num = (N_image + segment_size - 1) // segment_size
+        dp = [[0.] + [-INF] * self.n_selection for _ in range(segment_num + 1)]
+        trace = [[[] for _ in range(self.n_selection + 1)]
+                 for _ in range(segment_num + 1)]
+
+        for seg_idx in range(1, segment_num + 1):
+            candidate_index = range(
+                (seg_idx - 1) * segment_size, seg_idx * segment_size)
+            candidate_embeds = [image_embeds[i] for i in candidate_index]
+            sim_matrix = self.kernel(torch.stack(candidate_embeds))
+
+            for start_selected_num in range(0, min(self.n_selection, (seg_idx - 1) * segment_size) + 1):
+                conditional_index = trace[seg_idx - 1][start_selected_num][
+                                    -min(self.condition_size, len(trace[seg_idx - 1][start_selected_num])):]
+                offset = len(conditional_index)
+                additional_embeds = [text_embeds[
+                                         0].reshape(-1)] + [image_embeds[i] for i in conditional_index]
+                additional = self.kernel(
+                    torch.stack(additional_embeds),
+                    torch.stack(additional_embeds + candidate_embeds)
+                )
+                total_matrix = torch.cat([
+                    additional,  # [add, 32+add]
+                    torch.cat([
+                        additional[:, -len(sim_matrix):].T,  # [32, add]
+                        sim_matrix  # [32, 32]
+                    ], dim=1)  # [32, add + 32]
+                ], dim=0)  # [add+32, add+32]
+
+                max_selection = min(self.n_selection -
+                                    start_selected_num, segment_size)
+
+                cur_scores, cur_traces = self.seqdpp_select_super_fast(
+                    total_matrix, offset, max_selection)
+
+                for to_select_num, (cur_score, cur_trace) in enumerate(zip(cur_scores, cur_traces)):
+                    cur_trace = [i + int((seg_idx - 1) * segment_size)
+                                 for i in cur_trace]
+
+                    cur_score = dp[seg_idx - 1][start_selected_num] + cur_score
+                    cur_trace = trace[
+                                    seg_idx - 1][start_selected_num] + cur_trace
+
+                    if cur_score > dp[seg_idx][start_selected_num + to_select_num]:
+                        dp[seg_idx][start_selected_num + to_select_num] = cur_score
+                        trace[seg_idx][start_selected_num +
+                                       to_select_num] = cur_trace
+        return trace[segment_num][self.n_selection]
+
+    def seqdpp_select(self, text_embeds, image_embeds, conditional_index, candidate_index, to_select_num):
+        if to_select_num == 0:
+            return 0.0, []
+        conditional_embeds = [image_embeds[i] for i in conditional_index]
+        cur_trace = []
+        U_matrix = self.kernel(torch.stack(
+            conditional_embeds + [image_embeds[i] for i in candidate_index]))
+        I = torch.diag(
+            torch.tensor([0.] * len(conditional_index) + [1.] *
+                         len(candidate_index), device=U_matrix.device)
+        )
+        obj_values = -torch.linalg.slogdet(U_matrix + I).logabsdet
+        while len(cur_trace) < to_select_num:
+            max_obj_gain = -INF
+            cur_selected_idx = -1
+            for i in candidate_index:
+                if i in cur_trace:
+                    continue
+                cur_obj = self.cal_obj(
+                    selected_images_embeds=torch.stack(
+                        conditional_embeds + [image_embeds[j] for j in cur_trace + [i]]),
+                    text_embed=text_embeds[0].reshape(1, -1)
+                )
+                cur_obj_gain = cur_obj - obj_values
+                if cur_obj_gain > max_obj_gain:
+                    max_obj_gain = cur_obj_gain
+                    cur_selected_idx = i
+            cur_trace.append(cur_selected_idx)
+            obj_values += max_obj_gain
+        cur_trace = sorted(cur_trace)
+        return obj_values if len(cur_trace) > 0 else 0.0, cur_trace
+
+    def seqdpp_select_fast(self, total_matrix, offset, to_select_num):
+        if to_select_num == 0:
+            return 0.0, []
+        cur_trace = []
+        obj_values = 0.0
+        r, S_matrix = total_matrix[0:1, 1:], total_matrix[1:, 1:]
+        candidate_index = range(len(S_matrix) - offset)
+
+        while len(cur_trace) < to_select_num:
+            max_obj_gain = -INF
+            cur_selected_idx = -1
+            for i in candidate_index:
+                if i in cur_trace:
+                    continue
+                selected_idx = list(range(offset)) + \
+                               [j + offset for j in cur_trace + [i]]
+                cur_S_matrix = S_matrix[selected_idx][:, selected_idx]
+                cur_obj = (1. / self.lamda * 2 * torch.log(
+                    r[:, selected_idx]).sum()) + torch.linalg.slogdet(cur_S_matrix).logabsdet
+                cur_obj_gain = cur_obj - obj_values
+                if cur_obj_gain > max_obj_gain:
+                    max_obj_gain = cur_obj_gain
+                    cur_selected_idx = i
+            cur_trace.append(cur_selected_idx)
+            obj_values += max_obj_gain
+        cur_trace = sorted(cur_trace)
+        return obj_values if len(cur_trace) > 0 else 0.0, cur_trace
+
+    def seqdpp_select_super_fast(self, total_matrix, offset, to_select_num):
+        if to_select_num == 0:
+            return [0.0], [[]]
+        cur_trace = []
+        ret_scores = [0.0]
+        r, S_matrix = total_matrix[0:1, 1:], total_matrix[1:, 1:]
+        candidate_index = list(range(len(S_matrix) - offset))
+
+        conditional_idx = list(range(offset))
+        L = None
+        if len(conditional_idx) > 0:
+            L = torch.linalg.cholesky(
+                S_matrix[conditional_idx][:, conditional_idx])
+
+        while len(cur_trace) < to_select_num:
+            max_obj = -INF
+            cur_selected_idx = -1
+            better_L = None
+            for i in candidate_index:
+                if i in cur_trace:
+                    continue
+                cur_idx = i + offset
+                selected_idx = conditional_idx + \
+                               [j + offset for j in cur_trace] + [cur_idx]
+                if L is None:
+                    cur_sim_v = S_matrix[selected_idx][:, selected_idx]
+                    cur_L = torch.sqrt(cur_sim_v).reshape(1, 1)
+                    logdet = cur_sim_v.clone().log()
+                else:
+                    cur_sim_v = S_matrix[cur_idx:cur_idx + 1][:, selected_idx]
+                    cur_L, logdet = self.cholesky_update_determinant(
+                        L, cur_sim_v)
+                cur_obj = 1. / self.lamda * 2 * \
+                          torch.log(r[:, selected_idx]).sum() + logdet
+
+                if cur_obj > max_obj or cur_selected_idx == -1:
+                    max_obj = cur_obj
+                    cur_selected_idx = i
+                    better_L = cur_L
+            ret_scores.append(max_obj.clone())
+            cur_trace.append(cur_selected_idx)
+            L = better_L
+        ret_traces = [sorted(cur_trace[:j]) for j in range(len(cur_trace) + 1)]
+        return ret_scores, ret_traces
+
+    def cholesky_update_determinant(self, L, v):
+        n = L.shape[0]
+        v = v.view(-1, 1)
+        v_projected = torch.linalg.solve_triangular(L, v[:n], upper=False)
+
+        new_diag_element = torch.sqrt(torch.abs(v[-1] - v_projected.T @ v_projected))
+
+        new_row = torch.cat((v_projected.flatten(), new_diag_element.view(1)))
+        new_L = torch.zeros((n + 1, n + 1), dtype=L.dtype, device=L.device)
+        new_L[:n, :n] = L
+        new_L[n, :n] = new_row[:-1]
+        new_L[n, n] = new_diag_element
+
+        new_diag = torch.diag(new_L)
+        new_logdet = 2 * torch.log(new_diag).sum()
+
+        return new_L, new_logdet
+
+
+class GaussianKernel(nn.Module):
+
+    def __init__(self, alpha=1.):
+        super(GaussianKernel, self).__init__()
+        self.alpha = alpha
+
+    def forward(self, X: torch.Tensor) -> torch.Tensor:
+        l2_distance_square = ((X.unsqueeze(1) - X.unsqueeze(0)) ** 2).sum(2)
+        return torch.exp(-l2_distance_square / (2 * self.alpha))
+
+
+class MultiGaussianKernel(nn.Module):
+
+    def __init__(self, alphas=None):
+        super(MultiGaussianKernel, self).__init__()
+        if alphas is None:
+            alphas = [2 ** k for k in list(range(-3, 2))]
+        self.alphas = alphas
+
+    def forward(self, X: torch.Tensor, Y: torch.tensor = None) -> int:
+        Y = X.unsqueeze(0) if Y is None else Y.unsqueeze(0)
+        X = X.unsqueeze(1)
+        l2_distance_square = ((X - Y) ** 2).sum(2)
+
+        return sum([torch.exp(-l2_distance_square / (2 * alpha)) for alpha in self.alphas])
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..224a44a2eecc5e925d15690a24a1a8c309924213
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/__init__.py
@@ -0,0 +1,4 @@
+from .qtune_vl import QTuneVL
+from .qtune_vl_chat import QTuneVLChat
+
+__all__ = ['QTuneVL', 'QTuneVLChat']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/qtune_vl.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/qtune_vl.py
new file mode 100644
index 0000000000000000000000000000000000000000..0a4285c8aaba4712d5ac87de590d93d7ea407951
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/qtune_vl.py
@@ -0,0 +1,317 @@
+from __future__ import annotations
+import logging
+import os
+import warnings
+
+import torch
+
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+
+class QTuneVLPromptMixin:
+    """
+    Mixin class for QTuneVL to build custom prompt for different datasets.
+
+    Requires the following methods to be implemented in the subclass:
+        - dump_image(line, dataset: str) -> str | list[str]
+
+    Implements the following methods:
+        - use_custom_prompt(dataset: str) -> bool
+        - build_prompt(line, dataset: str) -> list[dict[str, str]]
+    """
+
+    def __init__(self, *args, use_custom_prompt: bool = True, **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        self._use_custom_prompt = use_custom_prompt
+
+    def set_dump_image(self, dump_image_func):
+        self.dump_image_func = dump_image_func
+
+    def dump_image(self, line, dataset):
+        return self.dump_image_func(line)
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        from vlmeval.dataset import DATASET_TYPE
+        dataset_type = DATASET_TYPE(dataset, default=None)
+
+        if not self._use_custom_prompt or listinstr(['MathVista', 'MMMU'], dataset):
+            return False
+        if dataset_type == 'MCQ':
+            if dataset is not None and 'LEGO' in dataset:
+                return False
+            return True
+        if dataset_type == 'Y/N' and dataset in {'HallusionBench', 'POPE'}:  # MME has it's own prompt
+            return True
+        if dataset_type == 'VQA' and dataset not in {'MMVet', 'ChartQAPro', 'ChartQAPro_CoT', 'ChartQAPro_PoT', 'ChartMuseum'}:  # noqa: E501
+            return True
+        return False
+
+    def build_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        from vlmeval.dataset import DATASET_TYPE
+
+        dataset_type = DATASET_TYPE(dataset, default=None)
+        if dataset_type == 'MCQ':
+            return self._build_mcq_prompt(line, dataset)
+        if dataset_type == 'Y/N':
+            return self._build_yorn_prompt(line, dataset)
+        if dataset_type == 'VQA':
+            return self._build_vqa_prompt(line, dataset)
+        raise ValueError(f'Unsupported dataset: {dataset}')
+
+    def _build_mcq_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MCQ dataset: use chinese prompt if the question contains chinese characters."""
+        MCQ_CN_PROMPT = '请直接回答选项字母。'
+        MCQ_EN_PROMPT = 'Please select the correct answer from the options above.'
+
+        import string
+
+        import pandas as pd
+
+        def cn_string(s):
+            import re
+
+            if re.search('[\u4e00-\u9fff]', s):
+                return True
+            return False
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += MCQ_CN_PROMPT if cn_string(prompt) else MCQ_EN_PROMPT
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_yorn_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for YORN dataset:"""
+        YORN_PROMPT = ' Please answer yes or no.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += YORN_PROMPT
+        return msgs
+
+    def _build_vqa_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for VQA dataset:"""
+        VQA_PROMPT = '\nPlease try to answer the question with short words or phrases if possible.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += VQA_PROMPT
+        return msgs
+
+
+def ensure_image_url(image: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:image;']
+    if any(image.startswith(prefix) for prefix in prefixes):
+        return image
+    if os.path.exists(image):
+        return 'file://' + image
+    raise ValueError(f'Invalid image: {image}')
+
+
+def ensure_video_url(video: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:video;']
+    if any(video.startswith(prefix) for prefix in prefixes):
+        return video
+    if os.path.exists(video):
+        return 'file://' + video
+    raise ValueError(f'Invalid video: {video}')
+
+
+class QTuneVL(QTuneVLPromptMixin, BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    VIDEO_LLM = True
+
+    def __init__(
+        self,
+        model_path: str,
+        min_pixels: int | None = None,
+        max_pixels: int | None = None,
+        total_pixels: int | None = None,
+        max_new_tokens=2048,
+        top_p=0.001,
+        top_k=1,
+        temperature=0.01,
+        repetition_penalty=1.0,
+        use_custom_prompt: bool = True,
+        system_prompt: str | None = None,
+        post_process: bool = False,  # if True, will try to only extract stuff in the last \boxed{}.
+        verbose: bool = False,
+        **kwargs,
+    ):
+        from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
+        super().__init__(use_custom_prompt=use_custom_prompt)
+        self.min_pixels = min_pixels
+        self.max_pixels = max_pixels
+        self.total_pixels = total_pixels
+        self.max_new_tokens = max_new_tokens
+        if self.total_pixels and self.total_pixels > 24576 * 28 * 28:
+            print('The total number of video tokens might become too large, resulting in an overly long input sequence. We recommend lowering **total_pixels** to below **24576 × 28 × 28**.')  # noqa: E501
+        self.generate_kwargs = dict(
+            max_new_tokens=self.max_new_tokens,
+            top_p=top_p,
+            top_k=top_k,
+            temperature=temperature,
+            repetition_penalty=repetition_penalty,
+        )
+        self.system_prompt = system_prompt
+        self.verbose = verbose
+        self.post_process = post_process
+        self.fps = kwargs.pop('fps', 2)
+        self.nframe = kwargs.pop('nframe', 128)
+        if self.fps is None and self.nframe is None:
+            print("Warning: fps and nframe are both None, \
+                  using default nframe/fps setting in qwen-vl-utils/qwen-omni-utils, \
+                  the fps/nframe setting in video dataset is omitted")
+        self.FRAME_FACTOR = 2
+        assert model_path is not None
+        self.model_path = model_path
+        self.processor = AutoProcessor.from_pretrained(model_path)
+
+        self.model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
+            model_path, torch_dtype='auto', device_map="auto", attn_implementation='flash_attention_2'
+        )
+        self.model.eval().to('cuda')
+
+        torch.cuda.empty_cache()
+
+    def _prepare_content(self, inputs: list[dict[str, str]], dataset: str | None = None) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        content = []
+        for s in inputs:
+            if s['type'] == 'image':
+                item = {'type': 'image', 'image': ensure_image_url(s['value'])}
+                if dataset == 'OCRBench':
+                    item['min_pixels'] = 10 * 10 * 28 * 28
+                    warnings.warn(f"OCRBench dataset uses custom min_pixels={item['min_pixels']}")
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item['min_pixels'] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+            elif s['type'] == 'video':
+                item = {
+                    'type': 'video',
+                    'video': ensure_video_url(s['value'])
+                }
+                if self.min_pixels is not None:
+                    item['min_pixels'] = self.min_pixels
+                if self.max_pixels is not None:
+                    item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+                if self.fps is not None:
+                    item['fps'] = self.fps
+                elif self.nframe is not None:
+                    import cv2
+                    video = cv2.VideoCapture(s['value'])
+                    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                    video.release()
+                    if frame_count < self.nframe:
+                        new_frame_count = frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR
+                        print(f"use {new_frame_count} for {s['value']}")
+                        item['nframes'] = new_frame_count
+                    else:
+                        item['nframes'] = self.nframe
+            elif s['type'] == 'text':
+                item = {'type': 'text', 'text': s['value']}
+            elif s['type'] == 'audio':
+                item = {'type': 'audio', 'audio': s['value']}
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def generate_inner_transformers(self, message, dataset=None):
+        try:
+            from qwen_vl_utils import process_vision_info
+        except Exception as err:
+            logging.critical("qwen_vl_utils not found, please install it via 'pip install qwen-vl-utils'")  # noqa: E501
+            raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({'role': 'system', 'content': self.system_prompt})
+        messages.append({'role': 'user', 'content': self._prepare_content(message, dataset=dataset)})
+        if self.verbose:
+            print(f'\033[31m{messages}\033[0m')
+
+        text = self.processor.apply_chat_template([messages], tokenize=False, add_generation_prompt=True)
+
+        images, videos = process_vision_info([messages])
+        inputs = self.processor(text=text, images=images, videos=videos, padding=True, return_tensors='pt')  # noqa: E501
+        inputs = inputs.to('cuda')
+
+        generated_ids = self.model.generate(
+            **inputs,
+            **self.generate_kwargs,
+        )
+        generated_ids = [
+            output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
+        ]
+        out = self.processor.tokenizer.batch_decode(
+            generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
+        )
+        response = out[0]
+        if self.post_process:
+            resp = response.split('\\boxed{')[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == '{':
+                    counter += 1
+                elif resp[i] == '}':
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                response = resp[:end]
+
+        if self.verbose:
+            print(f'\033[32m{response}\033[0m')
+        return response
+
+    def generate_inner(self, message, dataset=None):
+        return self.generate_inner_transformers(message, dataset=dataset)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/qtune_vl_chat.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/qtune_vl_chat.py
new file mode 100644
index 0000000000000000000000000000000000000000..d9105a62a8f9f6a5a7fc199b81f6574434a9541c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qtunevl/qtune_vl_chat.py
@@ -0,0 +1,846 @@
+import math
+import os
+import re
+import string
+import warnings
+
+import pandas as pd
+import torch
+import torch.distributed as dist
+import torchvision.transforms as T
+import transformers
+from PIL import Image
+from torchvision.transforms.functional import InterpolationMode
+from transformers import AutoConfig, AutoModel, AutoTokenizer
+
+from vlmeval.dataset import DATASET_MODALITY, DATASET_TYPE
+from vlmeval.smp import cn_string, get_rank_and_world_size, listinstr, version_cmp
+from ..base import BaseModel
+
+IMAGENET_MEAN = (0.485, 0.456, 0.406)
+IMAGENET_STD = (0.229, 0.224, 0.225)
+
+
+def auto_split_flag():
+    flag = os.environ.get('AUTO_SPLIT', '0')
+    if flag == '1':
+        return True
+    _, world_size = get_rank_and_world_size()
+    try:
+        import torch
+        device_count = torch.cuda.device_count()
+        if device_count > world_size and device_count % world_size == 0:
+            return True
+        else:
+            return False
+    except Exception:
+        return False
+
+
+def build_transform(input_size):
+    MEAN, STD = IMAGENET_MEAN, IMAGENET_STD
+    transform = T.Compose([
+        T.Lambda(lambda img: img.convert('RGB') if img.mode != 'RGB' else img),
+        T.Resize((input_size, input_size), interpolation=InterpolationMode.BICUBIC),
+        T.ToTensor(),
+        T.Normalize(mean=MEAN, std=STD)
+    ])
+    return transform
+
+
+def find_closest_aspect_ratio(aspect_ratio, target_ratios, width, height, image_size):
+    best_ratio_diff = float('inf')
+    best_ratio = (1, 1)
+    area = width * height
+    for ratio in target_ratios:
+        target_aspect_ratio = ratio[0] / ratio[1]
+        ratio_diff = abs(aspect_ratio - target_aspect_ratio)
+        if ratio_diff < best_ratio_diff:
+            best_ratio_diff = ratio_diff
+            best_ratio = ratio
+        elif ratio_diff == best_ratio_diff:
+            if area > 0.5 * image_size * image_size * ratio[0] * ratio[1]:
+                best_ratio = ratio
+    return best_ratio
+
+
+######################################################################################
+def preprocess_padding_global(image, net_input_size=448):
+    # 不形变的resize方式
+    orig_width, orig_height = image.size
+    if orig_height > orig_width:
+        scale_ratio = net_input_size / orig_height
+        scale_h = net_input_size
+        scale_w = int(orig_width * scale_ratio)
+    else:
+        scale_ratio = net_input_size / orig_width
+        scale_h = int(orig_height * scale_ratio)
+        scale_w = net_input_size
+
+    resized = image.resize((scale_w, scale_h), Image.BILINEAR)
+    new_image = Image.new('RGB', (net_input_size, net_input_size), (0, 0, 0))
+    paste_x = (net_input_size - scale_w) // 2
+    paste_y = (net_input_size - scale_h) // 2
+    new_image.paste(resized, (paste_x, paste_y))
+    return new_image
+
+
+def dynamic_preprocess(image, min_num=5, max_num=6, image_size=448, use_thumbnail=False):
+    orig_width, orig_height = image.size
+    aspect_ratio = orig_width / orig_height
+
+    padding_global = preprocess_padding_global(image, image_size)
+
+    # calculate the existing image aspect ratio
+    target_ratios = set(
+        (i, j) for n in range(min_num, max_num + 1) for i in range(1, n + 1) for j in range(1, n + 1) if
+        i * j <= max_num and i * j >= min_num)
+    target_ratios = sorted(target_ratios, key=lambda x: x[0] * x[1])
+
+    # find the closest aspect ratio to the target
+    target_aspect_ratio = find_closest_aspect_ratio(
+        aspect_ratio, target_ratios, orig_width, orig_height, image_size)
+
+    # calculate the target width and height
+    target_width = image_size * target_aspect_ratio[0]
+    target_height = image_size * target_aspect_ratio[1]
+    blocks = target_aspect_ratio[0] * target_aspect_ratio[1]
+
+    # resize the image
+    resized_img = image.resize((target_width, target_height))
+    processed_images = []
+    for i in range(blocks):
+        box = (
+            (i % (target_width // image_size)) * image_size,
+            (i // (target_width // image_size)) * image_size,
+            ((i % (target_width // image_size)) + 1) * image_size,
+            ((i // (target_width // image_size)) + 1) * image_size
+        )
+        # split the image
+        split_img = resized_img.crop(box)
+        processed_images.append(split_img)
+    assert len(processed_images) == blocks
+    processed_images.append(padding_global)
+    if use_thumbnail and len(processed_images) != 1:
+        thumbnail_img = image.resize((image_size, image_size))
+        processed_images.append(thumbnail_img)
+    return processed_images, target_aspect_ratio
+
+
+def dynamic_preprocess2(image, min_num=1, max_num=6, image_size=448, use_thumbnail=False, prior_aspect_ratio=None):
+    orig_width, orig_height = image.size
+    aspect_ratio = orig_width / orig_height
+
+    # calculate the existing image aspect ratio
+    target_ratios = set(
+        (i, j) for n in range(min_num, max_num + 1) for i in range(1, n + 1) for j in range(1, n + 1) if
+        i * j <= max_num and i * j >= min_num)
+    target_ratios = sorted(target_ratios, key=lambda x: x[0] * x[1])
+
+    new_target_ratios = []
+    if prior_aspect_ratio is not None:
+        for i in target_ratios:
+            if prior_aspect_ratio[0] % i[0] != 0 or prior_aspect_ratio[1] % i[1] != 0:
+                new_target_ratios.append(i)
+            else:
+                continue
+    # find the closest aspect ratio to the target
+    target_aspect_ratio = find_closest_aspect_ratio(
+        aspect_ratio, new_target_ratios, orig_width, orig_height, image_size)
+
+    # calculate the target width and height
+    target_width = image_size * target_aspect_ratio[0]
+    target_height = image_size * target_aspect_ratio[1]
+    blocks = target_aspect_ratio[0] * target_aspect_ratio[1]
+
+    # resize the image
+    resized_img = image.resize((target_width, target_height))
+    processed_images = []
+    for i in range(blocks):
+        box = (
+            (i % (target_width // image_size)) * image_size,
+            (i // (target_width // image_size)) * image_size,
+            ((i % (target_width // image_size)) + 1) * image_size,
+            ((i // (target_width // image_size)) + 1) * image_size
+        )
+        # split the image
+        split_img = resized_img.crop(box)
+        processed_images.append(split_img)
+    assert len(processed_images) == blocks
+    if use_thumbnail and len(processed_images) != 1:
+        thumbnail_img = image.resize((image_size, image_size))
+        processed_images.append(thumbnail_img)
+    return processed_images
+
+
+def load_image(image_file, input_size=448, max_num=6, upscale=False):
+    image = Image.open(image_file).convert('RGB')
+    if upscale:
+        image = image.resize((image.width * 2, image.height * 2), Image.BILINEAR)
+    transform = build_transform(input_size=input_size)
+    images, target_aspect_ratio = dynamic_preprocess(image, image_size=input_size, use_thumbnail=True, max_num=max_num)
+    pixel_values = [transform(image) for image in images]
+    pixel_values = torch.stack(pixel_values)
+    return pixel_values, target_aspect_ratio
+
+
+def load_image2(image_file, input_size=448, target_aspect_ratio=(1, 1), min_num=1, max_num=6):
+    image = Image.open(image_file).convert('RGB')
+    transform = build_transform(input_size=input_size)
+    images = dynamic_preprocess2(
+        image,
+        image_size=input_size,
+        prior_aspect_ratio=target_aspect_ratio,
+        use_thumbnail=True,
+        min_num=min_num,
+        max_num=max_num)
+
+    pixel_values = [transform(image) for image in images]
+    pixel_values = torch.stack(pixel_values)
+    return pixel_values
+
+
+def dynamic_preprocess3(image, min_num=1, max_num=6, image_size=448, use_thumbnail=False):
+    orig_width, orig_height = image.size
+    aspect_ratio = orig_width / orig_height
+
+    # calculate the existing image aspect ratio
+    target_ratios = set(
+        (i, j) for n in range(min_num, max_num + 1) for i in range(1, n + 1) for j in range(1, n + 1) if
+        i * j <= max_num and i * j >= min_num)
+    target_ratios = sorted(target_ratios, key=lambda x: x[0] * x[1])
+
+    # find the closest aspect ratio to the target
+    target_aspect_ratio = find_closest_aspect_ratio(
+        aspect_ratio, target_ratios, orig_width, orig_height, image_size)
+
+    # calculate the target width and height
+    target_width = image_size * target_aspect_ratio[0]
+    target_height = image_size * target_aspect_ratio[1]
+    blocks = target_aspect_ratio[0] * target_aspect_ratio[1]
+
+    # resize the image
+    resized_img = image.resize((target_width, target_height))
+    processed_images = []
+    for i in range(blocks):
+        box = (
+            (i % (target_width // image_size)) * image_size,
+            (i // (target_width // image_size)) * image_size,
+            ((i % (target_width // image_size)) + 1) * image_size,
+            ((i // (target_width // image_size)) + 1) * image_size
+        )
+        # split the image
+        split_img = resized_img.crop(box)
+        processed_images.append(split_img)
+    assert len(processed_images) == blocks
+    if use_thumbnail and len(processed_images) != 1:
+        thumbnail_img = image.resize((image_size, image_size))
+        processed_images.append(thumbnail_img)
+    return processed_images
+
+
+def load_image3(image_file, input_size=448, max_num=6, upscale=False):
+    image = Image.open(image_file).convert('RGB')
+    if upscale:
+        image = image.resize((image.width * 2, image.height * 2), Image.BILINEAR)
+    transform = build_transform(input_size=input_size)
+    images = dynamic_preprocess3(image, image_size=input_size, use_thumbnail=True, max_num=max_num)
+    pixel_values = [transform(image) for image in images]
+    pixel_values = torch.stack(pixel_values)
+    return pixel_values
+######################################################################################
+
+
+def get_local_rank_and_local_world_size():
+    if not dist.is_available():
+        return 0, 1
+    if not dist.is_initialized():
+        return 0, 1
+
+    if 'SLURM_LOCALID' in os.environ:
+        local_rank = int(os.environ['SLURM_LOCALID'])
+        local_world_size = int(os.environ['SLURM_NTASKS_PER_NODE'])
+        return local_rank, local_world_size
+
+    if 'LOCAL_RANK' in os.environ and 'LOCAL_WORLD_SIZE' in os.environ:
+        return int(os.environ['LOCAL_RANK']), int(os.environ['LOCAL_WORLD_SIZE'])
+
+    raise NotImplementedError(
+        "Fail to get local_rank and local_world_size! "
+        "Please ensure that you set the environment variable "
+        "`LOCAL_RANK` and `LOCAL_WORLD_SIZE`"
+    )
+
+
+def split_model(model_path):
+    num_gpus_per_node = torch.cuda.device_count()
+    rank, world_size = get_rank_and_world_size()
+    try:
+        local_rank, local_world_size = get_local_rank_and_local_world_size()
+    except Exception:
+        local_rank = rank
+
+    if 'GPUS_PER_PROCESS' in os.environ:
+        gpus_per_process = int(os.environ['GPUS_PER_PROCESS'])
+    else:
+        gpus_per_process = 8  # default to use 8 GPUs for one model
+    gpus_per_process = min(gpus_per_process, num_gpus_per_node // local_world_size)
+    start_gpu = local_rank * gpus_per_process
+    end_gpu = start_gpu + gpus_per_process
+
+    assert end_gpu <= num_gpus_per_node, f"Process {local_rank} tries to access GPU {end_gpu}, " \
+                                         f"but only {num_gpus_per_node} GPUs are available per node."
+
+    visible_devices = list(range(start_gpu, end_gpu))
+
+    device_map = {}
+    config = AutoConfig.from_pretrained(model_path, trust_remote_code=True)
+
+    num_gpus_for_vit = 0.5
+    num_layers = config.llm_config.num_hidden_layers
+    num_layers_per_gpu = math.ceil(num_layers / (len(visible_devices) - num_gpus_for_vit))
+    num_layers_per_gpu = [num_layers_per_gpu] * len(visible_devices)
+    num_layers_per_gpu[0] = math.ceil(num_layers_per_gpu[0] * 0.5)
+
+    layer_cnt = 0
+    for i, num_layer in enumerate(num_layers_per_gpu):
+        for j in range(num_layer):
+            device_map[f'language_model.model.layers.{layer_cnt}'] = visible_devices[i]
+            layer_cnt += 1
+    device_map['vision_model'] = visible_devices[0]
+    device_map['mlp1'] = visible_devices[0]
+    device_map['language_model.model.tok_embeddings'] = visible_devices[0]
+    device_map['language_model.model.embed_tokens'] = visible_devices[0]
+    device_map['language_model.output'] = visible_devices[0]
+    device_map['language_model.model.norm'] = visible_devices[0]
+    device_map['language_model.model.rotary_emb'] = visible_devices[0]
+    device_map['language_model.lm_head'] = visible_devices[0]
+    device_map[f'language_model.model.layers.{num_layers - 1}'] = visible_devices[0]
+
+    return device_map, visible_devices
+
+
+def build_mcq_cot_prompt(line, prompt):
+    cot_prompt = (
+        "Answer the preceding multiple choice question. The last line of your response should follow "
+        "this format: 'Answer: \\boxed{$LETTER}' (without quotes), where LETTER is one of the options. "
+        "If you are uncertain or the problem is too complex, make a reasoned guess based on the "
+        "information provided. Avoid repeating steps indefinitely—provide your best guess even if "
+        "unsure. Think step by step logically, considering all relevant information before answering."
+    )
+    prompt = prompt.replace("Answer with the option's letter from the given choices directly.", '').strip()
+    prompt = prompt + '\n' + cot_prompt
+
+    return prompt
+
+
+def build_qa_cot_prompt(line, prompt):
+    cot_prompt = (
+        "Answer the preceding question. The last line of your response should follow this format: "
+        "'Answer: \\boxed{$FINAL_ANSWER}' (without quotes), where 'FINAL_ANSWER' is your conclusion "
+        "based on the reasoning provided. If you are uncertain or the problem is too complex, make "
+        "a reasoned guess based on the information provided. Avoid repeating steps indefinitely—"
+        "provide your best guess even if unsure. Think step by step logically, considering all "
+        "relevant information before answering."
+    )
+    prompt = prompt + '\n' + cot_prompt
+
+    return prompt
+
+
+def build_multi_choice_prompt(line, dataset=None):
+    question = line['question']
+    hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+    if hint is not None:
+        question = hint + '\n' + question
+
+    options = {
+        cand: line[cand]
+        for cand in string.ascii_uppercase
+        if cand in line and not pd.isna(line[cand])
+    }
+    for key, item in options.items():
+        question += f'\n{key}. {item}'
+    prompt = question
+
+    if len(options):
+        prompt += '\n请直接回答选项字母。' if cn_string(
+            prompt) else "\nAnswer with the option's letter from the given choices directly."
+    else:
+        prompt += '\n请直接回答问题。' if cn_string(prompt) else '\nAnswer the question directly.'
+
+    return prompt
+
+
+def build_video_prompt(prompt, dataset=None, max_frames=64):
+    for start in range(0, max_frames, 8):
+        images_to_remove = ''.join([f'<Image-{i}>' for i in range(start + 1, start + 9)])
+        prompt = prompt.replace(images_to_remove, '')
+    for i in range(max_frames):
+        prompt = prompt.replace(f'Image-{i + 1}', f'Frame-{i + 1}')
+    if listinstr(['MMBench-Video'], dataset):
+        prompt = prompt.replace('\nAnswer:', '')
+    elif listinstr(['Video-MME', 'WorldSense'], dataset):
+        prompt = prompt.replace('\nAnswer:', '')
+        prompt += "\nAnswer with the option's letter from the given choices directly."
+    elif listinstr(['MVBench'], dataset):
+        prompt = prompt.replace('Best option:(', '')
+
+    return prompt
+
+
+def reorganize_prompt(message, image_num, dataset=None):
+    if dataset is not None and listinstr(['MUIRBench'], dataset):
+        prompt = '\n'.join([x['value'] for x in message if x['type'] == 'text'])
+        images_to_remove = ' '.join(['<image>'] * image_num)
+        prompt = prompt.replace(images_to_remove, '')
+        for i in range(image_num):
+            prompt = prompt.replace('<image>', f'<Image-{i + 1}>', 1)
+        prompt = ''.join([f'Image-{i + 1}: <image>\n' for i in range(image_num)]) + prompt
+    elif image_num == 1:
+        prompt = '<image>\n' + '\n'.join([x['value'] for x in message if x['type'] == 'text'])
+    else:
+        prompt, image_idx = '', 1
+        for x in message:
+            if x['type'] == 'text':
+                prompt += x['value']
+            elif x['type'] == 'image':
+                prompt += f'<Image-{image_idx}>'
+                image_idx += 1
+        prompt = ''.join([f'Image-{i + 1}: <image>\n' for i in range(image_num)]) + prompt
+        images_to_remove = ''.join([f'<Image-{i + 1}>' for i in range(image_num)])
+        prompt = prompt.replace(images_to_remove, '')
+    return prompt
+
+
+mpo_prompt_with_final_answer = (
+    "Your task is to answer the question below. "
+    "Give step by step reasoning before you answer, and when you're ready to answer, "
+    "please use the format \"Final answer: ..\""
+    "\n\n"
+    "Question:"
+    "\n\n"
+    "{question}"
+)
+
+mpo_prompt_without_final_answer = (
+    "Your task is to answer the question below. "
+    "Give step by step reasoning. "
+    "\n\n"
+    "Question:"
+    "\n\n"
+    "{question}"
+)
+
+
+def mpo_post_processing(response, dataset):
+
+    def extract_answer(text):
+        match = re.search(r'(Final answer:|Answer:)\s*(.*)', text, re.IGNORECASE)
+        if match:
+            return match.group(2).strip()
+        return text
+
+    if dataset is not None and (DATASET_TYPE(dataset) in ['Y/N', 'MCQ'] or listinstr(['CRPE'], dataset)):
+        response = extract_answer(response).strip()
+    return response
+
+
+def build_mpo_prompt(message, line, dataset):
+    if listinstr(['LLaVABench', 'MMVet'], dataset):
+        return message
+
+    question_orig = line['question']
+    if listinstr(['MathVerse', 'MathVision'], dataset):
+        question_orig = question_orig.split('Question:', 1)[-1].strip()
+        question_orig = question_orig.replace('Choices:\n', '').strip()
+    if listinstr(['WeMath'], dataset):
+        question_orig = question_orig.replace('Regarding the format, please answer following the template below, and be sure to include two <> symbols:\n<Thought process>: <<your thought process>> <Answer>: <<your option>>', '').strip()  # noqa: E501
+    options = {
+        cand: line[cand]
+        for cand in string.ascii_uppercase
+        if cand in line and not pd.isna(line[cand])
+    }
+    options_prompt = ''
+    for key, item in options.items():
+        options_prompt += f'{key}. {item}\n'
+
+    if options_prompt.strip():
+        question_orig = f'{question_orig}\n{options_prompt}'
+
+    cot_prompt = mpo_prompt_with_final_answer
+    prompt = cot_prompt.format(question=question_orig).strip()
+    message[0]['value'] = prompt
+    return message
+
+
+class QTuneVLChat(BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self,
+                 model_path='hanchaow/QTuneVL1-2B',
+                 load_in_8bit=False,
+                 use_mpo_prompt=False,
+                 version='V1.0',
+                 # Best-of-N parameters
+                 best_of_n=1,
+                 reward_model_path=None,
+                 **kwargs):
+
+        assert best_of_n >= 1
+        assert model_path is not None
+        assert version_cmp(transformers.__version__, '4.37.2', 'ge')
+
+        self.use_mpo_prompt = use_mpo_prompt
+        self.use_cot = (os.getenv('USE_COT') == '1')
+
+        self.model_path = model_path
+        self.tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
+
+        # Regular expression to match the pattern 'Image' followed by a number, e.g. Image1
+        self.pattern = r'Image(\d+)'
+        # Replacement pattern to insert a hyphen between 'Image' and the number, e.g. Image-1
+        self.replacement = r'Image-\1'
+
+        # Convert QtuneVL2 response to dataset format
+        # e.g. Image1 -> Image-1
+
+        # Regular expression to match the pattern 'Image-' followed by a number
+        self.reverse_pattern = r'Image-(\d+)'
+        # Replacement pattern to remove the hyphen (Image-1 -> Image1)
+        self.reverse_replacement = r'Image\1'
+
+        if auto_split_flag():
+            device_map, visible_devices = split_model(model_path=model_path)
+            self.device = visible_devices[0]
+            self.model = AutoModel.from_pretrained(
+                model_path,
+                torch_dtype=torch.bfloat16,
+                load_in_8bit=load_in_8bit,
+                trust_remote_code=True,
+                low_cpu_mem_usage=True,
+                device_map=device_map).eval()
+        else:
+            self.model = AutoModel.from_pretrained(
+                model_path,
+                torch_dtype=torch.bfloat16,
+                load_in_8bit=load_in_8bit,
+                trust_remote_code=True,
+                low_cpu_mem_usage=True).eval().to('cuda')
+            self.device = 'cuda'
+
+        if best_of_n > 1:
+            assert reward_model_path is not None
+
+            if auto_split_flag():
+                rm_device_map, visible_devices = split_model(model_path=reward_model_path)
+                rm_kwargs = {'device_map': rm_device_map}
+            else:
+                rm_kwargs = {}
+
+            self.reward_tokenizer = AutoTokenizer.from_pretrained(
+                reward_model_path, trust_remote_code=True, use_fast=False
+            )
+            self.reward_model = AutoModel.from_pretrained(
+                reward_model_path,
+                torch_dtype=torch.bfloat16,
+                load_in_8bit=load_in_8bit,
+                trust_remote_code=True,
+                low_cpu_mem_usage=True, **rm_kwargs).eval()
+
+            if not auto_split_flag():
+                self.reward_model = self.reward_model.to(self.device)
+
+            if not self.use_cot:
+                os.environ['USE_COT'] = '1'
+                self.use_cot = True
+                print('[Warning] Since Best-of-N is enabled, USE_COT is forced to be set to 1.')
+
+            print(f'Enable Best-of-N evaluation with PRM: {reward_model_path}')
+
+        self.image_size = self.model.config.vision_config.image_size
+        self.version = version
+        self.best_of_n = best_of_n
+        kwargs_default = dict(do_sample=False, max_new_tokens=4096, top_p=None)
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+
+        warnings.warn(f'Following kwargs received: {self.kwargs}, will use as generation config. ')
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if listinstr(['MMDU', 'MME-RealWorld', 'MME-RealWorld-CN', 'WeMath_COT', 'MMAlignBench', 'ChartQAPro', 'ChartMuseum'], dataset):  # noqa: E501
+            # For Multi-Turn and some special datasets we don't have custom prompt
+            return False
+        if DATASET_MODALITY(dataset) == 'VIDEO':
+            # For Video benchmarks we don't have custom prompt at here
+            return False
+        else:
+            return True
+
+    def build_prompt(self, line, dataset=None):
+        use_mpo_prompt = self.use_mpo_prompt and (self.use_cot or dataset in ['MMStar', 'HallusionBench', 'OCRBench'])
+
+        assert self.use_custom_prompt(dataset)
+        assert dataset is None or isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        if dataset is not None and DATASET_TYPE(dataset) == 'Y/N':
+            question = line['question']
+            if listinstr(['MME'], dataset):
+                prompt = question + ' Answer the question using a single word or phrase.'
+            elif listinstr(['HallusionBench', 'AMBER'], dataset):
+                prompt = question + ' Please answer yes or no. Answer the question using a single word or phrase.'
+            else:
+                prompt = question
+        elif dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+            prompt = build_multi_choice_prompt(line, dataset)
+            if os.getenv('USE_COT') == '1':
+                prompt = build_mcq_cot_prompt(line, prompt)
+        elif dataset is not None and DATASET_TYPE(dataset) == 'VQA':
+            question = line['question']
+            if listinstr(['LLaVABench', 'WildVision'], dataset):
+                prompt = question + '\nAnswer this question in detail.'
+            elif listinstr(['OCRVQA', 'TextVQA', 'ChartQA', 'DocVQA', 'InfoVQA', 'OCRBench',
+                            'DUDE', 'SLIDEVQA', 'GQA', 'MMLongBench_DOC'], dataset):
+                prompt = question + '\nAnswer the question using a single word or phrase.'
+            elif listinstr(['MathVista', 'MathVision', 'VCR', 'MTVQA', 'MMVet', 'MathVerse',
+                            'MMDU', 'CRPE', 'MIA-Bench', 'MM-Math', 'DynaMath', 'QSpatial',
+                            'WeMath', 'LogicVista'], dataset):
+                prompt = question
+                if os.getenv('USE_COT') == '1':
+                    prompt = build_qa_cot_prompt(line, prompt)
+            else:
+                prompt = question + '\nAnswer the question using a single word or phrase.'
+        else:
+            # VQA_ex_prompt: OlympiadBench, VizWiz
+            prompt = line['question']
+            if os.getenv('USE_COT') == '1':
+                prompt = build_qa_cot_prompt(line, prompt)
+
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+
+        if use_mpo_prompt:
+            message = build_mpo_prompt(message, line, dataset)
+        return message
+
+    def set_max_num(self, dataset):
+        # The total limit on the number of images processed, set to avoid Out-of-Memory issues.
+        self.total_max_num = 64
+        if dataset is None:
+            self.max_num = 6
+            return None
+        res_12_datasets = ['ChartQA_TEST', 'MMMU_DEV_VAL', 'MMMU_TEST', 'MME-RealWorld',
+                           'VCR_EN', 'VCR_ZH', 'OCRVQA']
+        res_18_datasets = ['DocVQA_VAL', 'DocVQA_TEST', 'DUDE', 'MMLongBench_DOC', 'SLIDEVQA']
+        res_24_datasets = ['InfoVQA_VAL', 'InfoVQA_TEST', 'OCRBench', 'HRBench4K', 'HRBench8K']
+        if DATASET_MODALITY(dataset) == 'VIDEO':
+            self.max_num = 1
+        elif listinstr(res_12_datasets, dataset):
+            self.max_num = 12
+        elif listinstr(res_18_datasets, dataset):
+            self.max_num = 18
+        elif listinstr(res_24_datasets, dataset):
+            self.max_num = 24
+        else:
+            self.max_num = 6
+
+    @torch.no_grad()
+    def generate_v2(self, message, dataset=None):
+
+        use_mpo_prompt = self.use_mpo_prompt and (self.use_cot or dataset in ['MMStar', 'HallusionBench', 'OCRBench'])
+
+        image_num = len([x for x in message if x['type'] == 'image'])
+        max_num = max(1, min(self.max_num, self.total_max_num // image_num))
+        prompt = reorganize_prompt(message, image_num, dataset=dataset)
+
+        if dataset is not None and DATASET_MODALITY(dataset) == 'VIDEO':
+            prompt = build_video_prompt(prompt, dataset)
+
+        if image_num > 1:
+            image_path = [x['value'] for x in message if x['type'] == 'image']
+            num_patches_list, pixel_values_list = [], []
+            for image_idx, file_name in enumerate(image_path):
+
+                if dataset is not None and listinstr(['MMBench', 'MMStar', 'MMVet'], dataset):
+                    upscale_flag = image_idx == 0 and dataset is not None and listinstr(['MMMU'], dataset)
+                    curr_pixel_values = load_image3(
+                        file_name, max_num=max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+                    num_patches_list.append(curr_pixel_values.size(0))
+                    pixel_values_list.append(curr_pixel_values)
+                else:
+                    upscale_flag = image_idx == 0 and dataset is not None and listinstr(['MMMU'], dataset)
+                    curr_pixel_values, target_aspect_ratio = load_image(
+                        file_name, max_num=max_num, upscale=upscale_flag)
+                    curr_pixel_values = curr_pixel_values.cuda().to(torch.bfloat16)
+                    curr_pixel_values2 = load_image2(
+                        file_name, target_aspect_ratio=target_aspect_ratio, max_num=max_num)
+                    curr_pixel_values2 = curr_pixel_values2.cuda().to(torch.bfloat16)
+                    if listinstr(['MathVista', 'HallusionBench', 'OCRBench'], dataset):
+                        curr_pixel_values = torch.cat(
+                            (curr_pixel_values[:-1], curr_pixel_values2[:-1], curr_pixel_values[-1:]), 0)
+                    else:
+                        curr_pixel_values = torch.cat(
+                            (curr_pixel_values2[:-1], curr_pixel_values[-1:]), 0)
+                    num_patches_list.append(curr_pixel_values.size(0))
+                    pixel_values_list.append(curr_pixel_values)
+            pixel_values = torch.cat(pixel_values_list, dim=0)
+        elif image_num == 1:
+            if dataset is not None and listinstr(['MMBench', 'MMStar', 'MMVet'], dataset):
+                image_path = [x['value'] for x in message if x['type'] == 'image'][0]
+                upscale_flag = dataset is not None and listinstr(['MMMU'], dataset)
+                pixel_values = load_image3(
+                    image_path, max_num=max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+                num_patches_list = [pixel_values.size(0)]
+            else:
+                image_path = [x['value'] for x in message if x['type'] == 'image'][0]
+                upscale_flag = dataset is not None and listinstr(['MMMU'], dataset)
+                pixel_values, target_aspect_ratio = load_image(
+                    image_path, max_num=max_num, upscale=upscale_flag)
+                pixel_values = pixel_values.to(self.device).to(torch.bfloat16)
+                pixel_values2 = load_image2(
+                    image_path, target_aspect_ratio=target_aspect_ratio, max_num=max_num)
+                pixel_values2 = pixel_values2.cuda().to(torch.bfloat16)
+                if listinstr(['MathVista', 'HallusionBench', 'OCRBench'], dataset):
+                    pixel_values = torch.cat((pixel_values[:-1], pixel_values2[:-1], pixel_values[-1:]), 0)
+                else:
+                    pixel_values = torch.cat((pixel_values2[:-1], pixel_values[-1:]), 0)
+                num_patches_list = [pixel_values.size(0)]
+        else:
+            pixel_values = None
+            num_patches_list = []
+        self.model.vision_model.to(self.device)
+        self.model.to(self.device)
+        response_list = []
+        for idx in range(self.best_of_n):
+            kwargs_default = self.kwargs.copy()
+            kwargs_default['do_sample'] = idx > 0
+            kwargs_default['temperature'] = 0.7
+            kwargs_default['top_p'] = 0.95
+
+            response = self.model.chat(
+                self.tokenizer,
+                pixel_values=pixel_values,
+                num_patches_list=num_patches_list,
+                question=prompt,
+                generation_config=kwargs_default,
+                verbose=idx == 0,
+            )
+            response_list.append(response)
+
+        if self.best_of_n > 1:
+            response_list = self.reward_model.select_best_response(
+                tokenizer=self.reward_tokenizer,
+                question=prompt,
+                response_list=response_list,
+                pixel_values=pixel_values,
+                num_patches_list=num_patches_list,
+            )
+        response = response_list[0]
+
+        if use_mpo_prompt:
+            response = mpo_post_processing(response, dataset)
+        return response
+
+    def generate_inner(self, message, dataset=None):
+        self.set_max_num(dataset)
+        print(f'QtuneVL model version: {self.version}')
+        if self.version in ['V1.0', 'V1.5']:
+            return self.generate_v2(message, dataset)
+        else:
+            raise ValueError(f'Unsupported version: {self.version}')
+
+    def build_history(self, message):
+        # Global Variables
+        image_path = []
+        image_cnt = 0
+
+        def concat_tilist(tilist):
+            nonlocal image_cnt  # Declare image_cnt as nonlocal to modify it
+            prompt = ''
+            for item in tilist:
+                # Substitute the pattern in the text
+                if item['type'] == 'text':
+                    prompt += re.sub(self.pattern, self.replacement, item['value'])
+                elif item['type'] == 'image':
+                    image_cnt += 1
+                    prompt += '<image>\n'
+                    image_path.append(item['value'])
+            return prompt
+
+        # Only previous messages
+        assert len(message) % 2 == 0
+        history = []
+        for i in range(len(message) // 2):
+            m1, m2 = message[2 * i], message[2 * i + 1]
+            assert m1['role'] == 'user' and m2['role'] == 'assistant'
+            history.append((concat_tilist(m1['content']), concat_tilist(m2['content'])))
+
+        return history, image_path, image_cnt
+
+    def chat_inner_v2(self, message, dataset=None):
+
+        if len(message) > 1:
+            history, image_path, image_cnt = self.build_history(message[:-1])
+        else:
+            history, image_path, image_cnt = None, [], 1
+        current_msg = message[-1]
+        question = ''
+
+        # If message is just text in the conversation
+        if len(current_msg['content']) == 1 and current_msg['content'][0]['type'] == 'text':
+            question = current_msg['content'][0]['value']
+            question = re.sub(self.pattern, self.replacement, question)  # Fix pattern as per QtuneVL
+        else:
+            for msg in current_msg['content']:
+                if msg['type'] == 'text':
+                    question += re.sub(self.pattern, self.replacement, msg['value'])
+                elif msg['type'] == 'image':
+                    image_cnt += 1
+                    question += '<image>\n'
+                    image_path.append(msg['value'])
+
+        if image_cnt > 1:
+            num_patches_list = []
+            pixel_values_list = []
+            for image_idx, file_name in enumerate(image_path):
+                upscale_flag = image_idx == 0 and dataset is not None and listinstr(['MMMU_DEV_VAL'], dataset)
+                curr_pixel_values = load_image(
+                    file_name, max_num=self.max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+                num_patches_list.append(curr_pixel_values.size(0))
+                pixel_values_list.append(curr_pixel_values)
+            pixel_values = torch.cat(pixel_values_list, dim=0)
+        elif image_cnt == 1:
+            upscale_flag = listinstr(['MMMU_DEV_VAL'], dataset)
+            pixel_values = load_image(
+                image_path, max_num=self.max_num, upscale=upscale_flag).to(self.device).to(torch.bfloat16)
+            num_patches_list = [pixel_values.size(0)]
+        else:
+            pixel_values = None
+            num_patches_list = []
+
+        response, history = self.model.chat(
+            self.tokenizer,
+            pixel_values=pixel_values,
+            num_patches_list=num_patches_list,
+            question=question,
+            generation_config=self.kwargs,
+            history=history,
+            return_history=True
+        )
+
+        response = re.sub(self.reverse_pattern, self.reverse_replacement, response)
+
+        return response
+
+    def chat_inner(self, message, dataset=None):
+        self.set_max_num(dataset)
+
+        if self.version in ['V1.0', 'V1.5']:
+            kwargs_default = dict(do_sample=False, max_new_tokens=512, top_p=None, num_beams=1)
+            self.kwargs = kwargs_default
+            return self.chat_inner_v2(message, dataset)
+        else:
+            raise ValueError(f'Unsupported version for Multi-Turn: {self.version}')
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e853740b08ad0c6662421a62abe46093bbcd2ae8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/__init__.py
@@ -0,0 +1,2 @@
+from .model import Qwen2VLChat, Qwen2VLChatAguvis  # noqa: F401
+from .prompt import Qwen2VLPromptMixin  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/model.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..722501f0f7d5f90f73bc7e787033726630059cbb
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/model.py
@@ -0,0 +1,759 @@
+from __future__ import annotations
+import json
+import logging
+import math
+import os
+import warnings
+
+import torch
+from huggingface_hub import snapshot_download
+from transformers import StoppingCriteria
+
+from vlmeval.dataset import DATASET_MODALITY
+from vlmeval.smp import get_cache_path, get_gpu_memory, listinstr
+from ..base import BaseModel
+from .prompt import Qwen2VLPromptMixin
+
+VLLM_MAX_IMAGE_INPUT_NUM = 24
+
+
+def ensure_image_url(image: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:image;']
+    if any(image.startswith(prefix) for prefix in prefixes):
+        return image
+    if os.path.exists(image):
+        return 'file://' + image
+    raise ValueError(f'Invalid image: {image}')
+
+
+def ensure_video_url(video: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:video;']
+    if any(video.startswith(prefix) for prefix in prefixes):
+        return video
+    if os.path.exists(video):
+        return 'file://' + video
+    raise ValueError(f'Invalid video: {video}')
+
+
+def create_image_content(image_path, min_pixels, max_pixels):
+    base64_image, mime_type = encode_image(image_path)
+    return {
+        "type": "image",
+        "image": f"data:{mime_type};base64,{base64_image}",
+        'min_pixels': min_pixels,
+        'max_pixels': max_pixels
+    }
+
+
+def encode_image(image_path, max_side=None):
+    from mimetypes import guess_type
+    mime_type, _ = guess_type(image_path)
+    if mime_type is None:
+        mime_type = "image/jpeg"
+    image_format = mime_type.split("/")[-1].upper() if mime_type else "JPEG"
+
+    from PIL import Image
+    image = Image.open(image_path)
+    # Handle the alpha channel
+    if image.mode == "RGBA":
+        image = _rgba_to_rgb(image)
+    if max_side:
+        image = _resize_image(image, max_side)
+    encoded_image = _encode_image(image, image_format)
+
+    return encoded_image, mime_type
+
+
+def _encode_image(image, image_format):
+    from io import BytesIO
+    with BytesIO() as output:
+        image.convert("RGB").save(output, format=image_format)
+        import base64
+        base64_encoded_data = base64.b64encode(output.getvalue()).decode("utf-8")
+    return base64_encoded_data
+
+
+def _rgba_to_rgb(image):
+    from PIL import Image
+    background = Image.new("RGBA", image.size, (255, 255, 255, 255))
+    return Image.alpha_composite(background, image).convert("RGB")
+
+
+def _resize_image(image, max_side):
+    resize_scale = max_side / max(image.size)
+    new_size = (
+        int(image.size[0] * resize_scale),
+        int(image.size[1] * resize_scale),
+    )
+    return image.resize(new_size)
+
+
+def process_video(video_path, num_frames, min_pixels, max_pixels):
+    import cv2
+
+    # Open the video file
+    cap = cv2.VideoCapture(video_path)
+    frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+    fps = cap.get(cv2.CAP_PROP_FPS)  # Frames per second
+
+    # the sampling rate using max number of frames
+    sampling_gap_maxframe = (
+        1 if not num_frames else math.ceil(frame_count / num_frames)
+    )
+    sampling_gap = max(math.ceil(fps / 5), sampling_gap_maxframe)
+
+    frame_number = 0
+    images = []
+
+    while True:
+        import tempfile
+        success, frame = cap.read()
+        if not success:
+            break
+        # Sample frames based on the dynamic sampling rate
+        if frame_number % sampling_gap == 0:
+            # Create a temporary file for the frame
+            with tempfile.NamedTemporaryFile(
+                suffix=".jpg", delete=False
+            ) as temp_frame:
+                cv2.imwrite(temp_frame.name, frame)
+                images.append(create_image_content(temp_frame.name, min_pixels, max_pixels))
+                os.remove(temp_frame.name)
+        frame_number += 1
+    if frame_number == 0:
+        raise ValueError(f"Failed to read video from {video_path}, check data...")
+    logging.info(
+        f"Sampled {len(images)}/{frame_number} frames from video {video_path}"
+    )
+    cap.release()
+    return images
+
+
+class KeywordsStoppingCriteria(StoppingCriteria):
+    def __init__(self, keywords, tokenizer, input_ids):
+        self.keywords = keywords
+        self.keyword_ids = []
+        self.max_keyword_len = 0
+        for keyword in keywords:
+            cur_keyword_ids = tokenizer(keyword).input_ids
+            if (
+                len(cur_keyword_ids) > 1
+                and cur_keyword_ids[0] == tokenizer.bos_token_id
+            ):
+                cur_keyword_ids = cur_keyword_ids[1:]
+            if len(cur_keyword_ids) > self.max_keyword_len:
+                self.max_keyword_len = len(cur_keyword_ids)
+            self.keyword_ids.append(torch.tensor(cur_keyword_ids))
+        self.tokenizer = tokenizer
+        self.start_len = input_ids.shape[1]
+
+    def __call__(
+        self, output_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs
+    ) -> bool:
+        assert output_ids.shape[0] == 1, "Only support batch size 1 (yet)"  # TODO
+        offset = min(output_ids.shape[1] - self.start_len, self.max_keyword_len)
+        self.keyword_ids = [
+            keyword_id.to(output_ids.device) for keyword_id in self.keyword_ids
+        ]
+        for keyword_id in self.keyword_ids:
+            if (output_ids[0, -keyword_id.shape[0]:] == keyword_id).all():
+                return True
+        outputs = self.tokenizer.batch_decode(
+            output_ids[:, -offset:], skip_special_tokens=True
+        )[0]
+        for keyword in self.keywords:
+            if keyword in outputs:
+                return True
+        return False
+
+
+CHAT_TEMPLATE = "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"  # noqa: E501
+
+UNTIL = ["<|diff_marker|>"]
+
+
+class Qwen2VLChat(Qwen2VLPromptMixin, BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    VIDEO_LLM = True
+
+    def __init__(
+        self,
+        model_path: str,
+        min_pixels: int | None = None,
+        max_pixels: int | None = None,
+        total_pixels: int | None = None,
+        max_new_tokens=2048,
+        top_p=0.001,
+        top_k=1,
+        temperature=0.01,
+        repetition_penalty=1.0,
+        do_sample: bool = True,
+        use_custom_prompt: bool = True,
+        system_prompt: str | None = None,
+        post_process: bool = False,  # if True, will try to only extract stuff in the last \boxed{}.
+        verbose: bool = False,
+        use_audio_in_video: bool = False,
+        **kwargs,
+    ):
+        super().__init__(use_custom_prompt=use_custom_prompt)
+        self.min_pixels = min_pixels
+        self.max_pixels = max_pixels
+        self.total_pixels = total_pixels
+        self.max_new_tokens = max_new_tokens
+        if self.total_pixels and self.total_pixels > 24576 * 28 * 28:
+            print('The total number of video tokens might become too large, resulting in an overly long input sequence. We recommend lowering **total_pixels** to below **24576 × 28 × 28**.')  # noqa: E501
+        self.generate_kwargs = dict(
+            max_new_tokens=self.max_new_tokens,
+            top_p=top_p,
+            top_k=top_k,
+            temperature=temperature,
+            repetition_penalty=repetition_penalty,
+            do_sample=do_sample,
+        )
+        self.system_prompt = system_prompt
+        self.verbose = verbose
+        self.post_process = post_process
+        self.fps = kwargs.pop('fps', 2)
+        self.nframe = kwargs.pop('nframe', 128)
+        if self.fps is None and self.nframe is None:
+            print("Warning: fps and nframe are both None, \
+                  using default nframe/fps setting in qwen-vl-utils/qwen-omni-utils, \
+                  the fps/nframe setting in video dataset is omitted")
+        self.use_audio_in_video = use_audio_in_video
+        self.FRAME_FACTOR = 2
+
+        assert model_path is not None
+
+        if not os.path.exists(model_path):
+            cache_path = get_cache_path(model_path, repo_type='models')
+            if cache_path is None:
+                snapshot_download(repo_id=model_path)
+                cache_path = get_cache_path(model_path, repo_type='models')
+            model_path = cache_path
+
+        self.model_path = model_path
+
+        MODEL_CLS = None
+
+        cfg_json_path = os.path.join(self.model_path, 'config.json')
+        assert cfg_json_path is not None, 'Qwen series models require a config.json file to specify the architecture.'
+
+        with open(cfg_json_path, 'r', encoding='utf-8') as f:
+            cfg = json.load(f)
+            architectures = str(cfg.get("architectures", None)).lower()
+
+        if listinstr(['omni'], architectures):
+            try:
+                from transformers import Qwen2_5OmniForConditionalGeneration, Qwen2_5OmniProcessor
+            except Exception as err:
+                logging.critical("pip install git+https://github.com/huggingface/transformers@3a1ead0aabed473eafe527915eea8c197d424356")  # noqa: E501
+                raise err
+            MODEL_CLS = Qwen2_5OmniForConditionalGeneration
+            self.processor = Qwen2_5OmniProcessor.from_pretrained(self.model_path)
+
+        elif listinstr(['qwen2_5'], architectures):
+            from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
+            MODEL_CLS = Qwen2_5_VLForConditionalGeneration
+            self.processor = AutoProcessor.from_pretrained(self.model_path)
+
+        else:
+            from transformers import Qwen2VLForConditionalGeneration, Qwen2VLProcessor
+            MODEL_CLS = Qwen2VLForConditionalGeneration
+            self.processor = Qwen2VLProcessor.from_pretrained(self.model_path)
+
+        gpu_mems = get_gpu_memory()
+        max_gpu_mem = max(gpu_mems) if gpu_mems != [] else -1
+        assert max_gpu_mem > 0
+        self.use_vllm = kwargs.get('use_vllm', False)
+        self.use_lmdeploy = kwargs.get('use_lmdeploy', False)
+        self.limit_mm_per_prompt = VLLM_MAX_IMAGE_INPUT_NUM
+        assert self.use_vllm + self.use_lmdeploy <= 1, "You can only set one flag between `use_vllm` and `use_lmdeploy` to True"  # noqa: E501
+
+        if self.use_vllm:
+            from vllm import LLM
+            gpu_count = torch.cuda.device_count()
+            if gpu_count >= 8:
+                tp_size = 8
+            elif gpu_count >= 4:
+                tp_size = 4
+            elif gpu_count >= 2:
+                tp_size = 2
+            else:
+                tp_size = 1
+            logging.info(
+                f'Using vLLM for {self.model_path} inference with {tp_size} GPUs (available: {gpu_count})'
+            )
+            if os.environ.get('VLLM_WORKER_MULTIPROC_METHOD') != 'spawn':
+                logging.warning(
+                    'VLLM_WORKER_MULTIPROC_METHOD is not set to spawn.'
+                    'Use \'export VLLM_WORKER_MULTIPROC_METHOD=spawn\' to avoid potential multi-process issues'
+                )
+            self.llm = LLM(
+                model=self.model_path,
+                max_num_seqs=5,
+                max_model_len=32768,
+                limit_mm_per_prompt={"image": self.limit_mm_per_prompt},
+                tensor_parallel_size=tp_size,
+                gpu_memory_utilization=kwargs.get("gpu_utils", 0.9),
+            )
+
+        elif self.use_lmdeploy:
+            from lmdeploy import ChatTemplateConfig, TurbomindEngineConfig, pipeline
+            num_gpus = torch.cuda.device_count()
+            self.model = pipeline(
+                model_path,
+                backend_config=TurbomindEngineConfig(session_len=32768, cache_max_entry_count=0.1, tp=num_gpus),
+                chat_template_config=ChatTemplateConfig(model_name='qwen2d5-vl'))
+            torch.cuda.set_device(0)
+            self.device = 'cuda'
+        else:
+            self.model = MODEL_CLS.from_pretrained(
+                model_path, torch_dtype='auto', device_map="auto", attn_implementation='flash_attention_2'
+            )
+            self.model.eval()
+
+        torch.cuda.empty_cache()
+
+    def _prepare_content(self, inputs: list[dict[str, str]], dataset: str | None = None) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        content = []
+        for s in inputs:
+            if s['type'] == 'image':
+                item = {'type': 'image', 'image': ensure_image_url(s['value'])}
+                if dataset == 'OCRBench':
+                    item['min_pixels'] = 10 * 10 * 28 * 28
+                    warnings.warn(f"OCRBench dataset uses custom min_pixels={item['min_pixels']}")
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item['min_pixels'] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+            elif s['type'] == 'video':
+                item = {
+                    'type': 'video',
+                    'video': ensure_video_url(s['value'])
+                }
+                if self.min_pixels is not None:
+                    item['min_pixels'] = self.min_pixels
+                if self.max_pixels is not None:
+                    item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+                if self.fps is not None:
+                    item['fps'] = self.fps
+                elif self.nframe is not None:
+                    import cv2
+                    video = cv2.VideoCapture(s['value'])
+                    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                    video.release()
+                    if frame_count < self.nframe:
+                        new_frame_count = frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR
+                        print(f"use {new_frame_count} for {s['value']}")
+                        item['nframes'] = new_frame_count
+                    else:
+                        item['nframes'] = self.nframe
+            elif s['type'] == 'text':
+                item = {'type': 'text', 'text': s['value']}
+            elif s['type'] == 'audio':
+                item = {'type': 'audio', 'audio': s['value']}
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def _prepare_content_vllm(self, inputs: list[dict[str, str]], dataset: str | None = None) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        content = []
+        video_inputs = [s for s in inputs if s['type'] == 'video']
+        video_count = len(video_inputs)
+        cur_image_count = 0
+        for s in inputs:
+            if s['type'] == 'image':
+                item = {'type': 'image', 'image': ensure_image_url(s['value'])}
+                if dataset == 'OCRBench':
+                    item['min_pixels'] = 10 * 10 * 28 * 28
+                    warnings.warn(f"OCRBench dataset uses custom min_pixels={item['min_pixels']}")
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item['min_pixels'] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+                if cur_image_count < self.limit_mm_per_prompt:
+                    content.append(item)
+                    cur_image_count += 1
+                else:
+                    logging.warning(
+                        f"Number of images exceeds the limit of {self.limit_mm_per_prompt}. "
+                        f"Only the first {self.limit_mm_per_prompt} images will be used."
+                    )
+            elif s['type'] == 'video':
+                if video_count > 1:
+                    logging.warning(
+                        "Multiple videos detected. Using video frames for each video"
+                    )
+                    if dataset == 'OCRBench':
+                        min_pixels = 10 * 10 * 28 * 28
+                        warnings.warn(f"OCRBench dataset uses custom min_pixels={min_pixels}")
+                        if self.max_pixels is not None:
+                            max_pixels = self.max_pixels
+                    else:
+                        if self.min_pixels is not None:
+                            min_pixels = self.min_pixels
+                        if self.max_pixels is not None:
+                            max_pixels = self.max_pixels
+                    import cv2
+                    video = cv2.VideoCapture(s['value'])
+                    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                    video.release()
+
+                    frames_per_video = max(1, self.limit_mm_per_prompt // video_count)
+                    content.append({"type": "text", "text": "<video frames start>"})
+                    content.extend(process_video(s['value'], frames_per_video, min_pixels, max_pixels))
+                    content.append({"type": "text", "text": "<video frames end>"})
+
+                else:
+                    item = {
+                        'type': 'video',
+                        'video': ensure_video_url(s['value'])
+                    }
+                    if self.min_pixels is not None:
+                        item['min_pixels'] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                    if self.total_pixels is not None:
+                        item['total_pixels'] = self.total_pixels
+                    if self.fps is not None:
+                        item['fps'] = self.fps
+                    elif self.nframe is not None:
+                        import cv2
+                        video = cv2.VideoCapture(s['value'])
+                        frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                        video.release()
+                        if frame_count < self.nframe:
+                            new_frame_count = frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR
+                            print(f"use {new_frame_count} for {s['value']}")
+                            item['nframes'] = new_frame_count
+                        else:
+                            item['nframes'] = self.nframe
+                    content.append(item)
+            elif s['type'] == 'text':
+                item = {'type': 'text', 'text': s['value']}
+                content.append(item)
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+        return content
+
+    def generate_inner_transformers(self, message, dataset=None):
+        if listinstr(['omni'], self.model_path.lower()):
+            try:
+                from qwen_omni_utils import process_mm_info
+            except Exception as err:
+                logging.critical("qwen_omni_utils not found, please install it via 'pip install qwen-omni-utils[decord]'")  # noqa: E501
+                raise err
+        else:
+            try:
+                from qwen_vl_utils import process_vision_info
+            except Exception as err:
+                logging.critical("qwen_vl_utils not found, please install it via 'pip install qwen-vl-utils'")  # noqa: E501
+                raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({'role': 'system', 'content': self.system_prompt})
+        messages.append({'role': 'user', 'content': self._prepare_content(message, dataset=dataset)})
+        if self.verbose:
+            print(f'\033[31m{messages}\033[0m')
+
+        text = self.processor.apply_chat_template([messages], tokenize=False, add_generation_prompt=True)
+        if listinstr(['omni'], self.model_path.lower()):
+            audios, images, videos = process_mm_info([messages], use_audio_in_video=self.use_audio_in_video)
+            inputs = self.processor(text=text, images=images, audio=audios, videos=videos, padding=True, return_tensors='pt', use_audio_in_video=self.use_audio_in_video)  # noqa: E501
+        else:
+            images, videos = process_vision_info([messages])
+            inputs = self.processor(text=text, images=images, videos=videos, padding=True, return_tensors='pt')  # noqa: E501
+        inputs = inputs.to('cuda')
+
+        if listinstr(['omni'], self.model_path.lower()):
+            self.generate_kwargs['use_audio_in_video'] = self.use_audio_in_video
+            self.generate_kwargs['return_audio'] = False
+        generated_ids = self.model.generate(
+            **inputs,
+            **self.generate_kwargs,
+        )
+        generated_ids = [
+            output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
+        ]
+        out = self.processor.tokenizer.batch_decode(
+            generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
+        )
+        response = out[0]
+        if self.post_process:
+            resp = response.split('\\boxed{')[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == '{':
+                    counter += 1
+                elif resp[i] == '}':
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                response = resp[:end]
+
+        if self.verbose:
+            print(f'\033[32m{response}\033[0m')
+        return response
+
+    def generate_inner_lmdeploy(self, message, dataset=None):
+        from lmdeploy import GenerationConfig
+        gen_config = GenerationConfig(
+            max_new_tokens=self.max_new_tokens,
+            top_p=self.generate_kwargs['top_p'],
+            top_k=self.generate_kwargs['top_k'],
+            temperature=self.generate_kwargs['temperature'],
+            repetition_penalty=self.generate_kwargs['repetition_penalty'],
+        )
+        gen_config.random_seed = None
+        messages_list = self.message_to_lmdeploy(message, system_prompt=self.system_prompt)
+        assert len(messages_list) == 1
+        response = self.model(messages_list, gen_config=gen_config)[0]
+        response = response.text
+        return response
+
+    def generate_inner_vllm(self, message, dataset=None):
+        from vllm import SamplingParams
+
+        if listinstr(['omni'], self.model_path.lower()):
+            try:
+                from qwen_omni_utils import process_mm_info
+            except Exception as err:
+                logging.critical("qwen_omni_utils not found, please install it via 'pip install qwen-omni-utils[decord]'")  # noqa: E501
+                raise err
+        else:
+            try:
+                from qwen_vl_utils import process_vision_info
+            except Exception as err:
+                logging.critical("qwen_vl_utils not found, please install it via 'pip install qwen-vl-utils'")  # noqa: E501
+                raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({'role': 'system', 'content': self.system_prompt})
+        messages.append({'role': 'user', 'content': self._prepare_content_vllm(message, dataset=dataset)})
+        if self.verbose:
+            print(f'\033[31m{messages}\033[0m')
+
+        text = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+        if listinstr(['omni'], self.model_path.lower()):
+            audios, images, videos = process_mm_info(messages, use_audio_in_video=self.use_audio_in_video)
+        else:
+            images, videos = process_vision_info(messages)
+        print('finishing process vision info in vllm.')
+
+        if DATASET_MODALITY(dataset) == 'VIDEO' and 'megabench' not in dataset.lower():
+            assert len(videos) == 1
+            videos_nd = [videos[0].detach().cpu().numpy().transpose(0, 2, 3, 1)]
+
+            video_inputs = {
+                "prompt": text[0],
+                "multi_modal_data": {"video": videos_nd[0]},
+                "mm_processor_kwargs": {}
+            }
+            if self.use_audio_in_video:
+                import vllm
+                assert not vllm.envs.VLLM_USE_V1, ("V1 does not support use_audio_in_video. Please launch this example with `VLLM_USE_V1=0`.")  # noqa: E501
+                video_inputs["multi_modal_data"]["audio"] = audios[0]
+                video_inputs['mm_processor_kwargs']['use_audio_in_video'] = True
+            if videos_nd[0].shape[0] > VLLM_MAX_IMAGE_INPUT_NUM:
+                print('video input sequence may be too long for vllm, Maybe cannot generate response for VLLM')
+        sampling_params = SamplingParams(
+            temperature=0.0, max_tokens=self.max_new_tokens, stop_token_ids=None
+        )
+        if images:
+            outputs = self.llm.generate(
+                {
+                    "prompt": text,
+                    "multi_modal_data": {"image": images},
+                },
+                sampling_params=sampling_params,
+            )
+        elif videos_nd:
+            outputs = self.llm.generate(
+                video_inputs,
+                sampling_params=sampling_params,
+            )
+        else:
+            outputs = self.llm.generate(
+                {
+                    "prompt": text,
+                },
+                sampling_params=sampling_params,
+            )
+
+        for o in outputs:
+            generated_text = o.outputs[0].text
+
+        if self.post_process:
+            resp = generated_text.split('\\boxed{')[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == '{':
+                    counter += 1
+                elif resp[i] == '}':
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                generated_text = resp[:end]
+
+        if self.verbose:
+            print(f'\033[32m{generated_text}\033[0m')
+        return generated_text
+
+    def generate_inner(self, message, dataset=None):
+        if self.use_vllm:
+            return self.generate_inner_vllm(message, dataset=dataset)
+        elif self.use_lmdeploy:
+            return self.generate_inner_lmdeploy(message, dataset=dataset)
+        else:
+            return self.generate_inner_transformers(message, dataset=dataset)
+
+
+class Qwen2VLChatAguvis(Qwen2VLChat):
+    def __init__(self, mode=None, **kwargs):
+        self.mode = mode
+        super().__init__(**kwargs)
+        self.processor.max_pixels = self.max_pixels
+        self.processor.min_pixels = self.min_pixels
+
+    def generate_inner(self, message, dataset=None):
+        try:
+            from qwen_vl_utils import process_vision_info
+        except Exception as err:
+            logging.critical(
+                "qwen_vl_utils not found, please install it via 'pip install qwen-vl-utils'"
+            )
+            raise err
+
+        messages = []
+        user_message = []
+        for item in message:
+            if "role" in item.keys():
+                if item["role"] == "system":
+                    self.system_prompt = item["value"]
+                else:
+                    item.pop("role")
+                    user_message.append(item)
+            else:
+                user_message.append(item)
+        message = user_message
+
+        if self.system_prompt is not None:
+            messages.append({"role": "system", "content": self.system_prompt})
+        messages.append(
+            {"role": "user", "content": self._prepare_content(message, dataset=dataset)}
+        )
+        if self.verbose:
+            print(f"\033[31m{messages}\033[0m")
+
+        text = self.processor.apply_chat_template(
+            messages,
+            tokenize=False,
+            add_generation_prompt=False,
+            chat_template=CHAT_TEMPLATE,
+        )
+        # TODO: provide current action's low-level instruction
+        # if False:
+        #     # If low-level instruction is provided
+        #     # We enforce using "Action: {low_level_instruction} to guide generation"
+        #     recipient_text = f"<|im_start|>assistant<|recipient|>all\nAction: {low_level_instruction}\n"
+        if self.mode == "force-plan":
+            recipient_text = "<|im_start|>assistant<|recipient|>all\nThought: "
+        elif self.mode == "force-plan-l1":
+            recipient_text = "<|im_start|>assistant<|recipient|>all\nAction: "
+        elif self.mode == "force-plan-l3":
+            recipient_text = "<|im_start|>assistant<|recipient|>all\nObservation: "
+        elif self.mode == "grounding":
+            recipient_text = "<|im_start|>assistant<|recipient|>os\n"
+        elif self.mode == "force-plan-free":
+            recipient_text = "<|im_start|>assistant<|recipient|>all\n"
+        elif self.mode == "self-plan":
+            recipient_text = "<|im_start|>assistant<|recipient|>"
+        else:
+            raise ValueError(f"Invalid mode: {self.mode}")
+        text += recipient_text
+        # print(text)
+
+        images, videos = process_vision_info([messages])
+        inputs = self.processor(
+            text=[text], images=images, videos=videos, padding=True, return_tensors="pt"
+        )
+        inputs = inputs.to("cuda")
+
+        # stop_str = "<|diff_marker|>"
+        # keywords = [stop_str]
+        # stopping_criteria = KeywordsStoppingCriteria(
+        #     keywords, self.processor.tokenizer, inputs.input_ids
+        # )
+
+        generated_ids = self.model.generate(
+            **inputs,
+            **self.generate_kwargs,
+            # stopping_criteria=[stopping_criteria],
+        )
+        generated_ids = [
+            output_ids[len(input_ids):]
+            for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
+        ]
+        out = self.processor.tokenizer.batch_decode(
+            generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
+        )
+        response = out[0]
+        # for term in UNTIL:
+        #     if len(term) > 0:
+        #         response = response.split(term)[0]
+
+        if self.post_process:
+            resp = response.split("\\boxed{")[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == "{":
+                    counter += 1
+                elif resp[i] == "}":
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                response = resp[:end]
+
+        if self.verbose:
+            print(f"\033[32m{response}\033[0m")
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..5ae67a48d1194589149026d69686f0967ad7da2e
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen2_vl/prompt.py
@@ -0,0 +1,157 @@
+from __future__ import annotations
+
+
+class Qwen2VLPromptMixin:
+    """
+    Mixin class for Qwen2VLChat to build custom prompt for different datasets.
+
+    Requires the following methods to be implemented in the subclass:
+        - dump_image(line, dataset: str) -> str | list[str]
+
+    Implements the following methods:
+        - use_custom_prompt(dataset: str) -> bool
+        - build_prompt(line, dataset: str) -> list[dict[str, str]]
+    """
+
+    def __init__(self, *args, use_custom_prompt: bool = True, **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        self._use_custom_prompt = use_custom_prompt
+
+    def set_dump_image(self, dump_image_func):
+        self.dump_image_func = dump_image_func
+
+    def dump_image(self, line, dataset):
+        return self.dump_image_func(line)
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        from vlmeval.dataset import DATASET_TYPE
+        dataset_type = DATASET_TYPE(dataset, default=None)
+
+        if not self._use_custom_prompt:
+            return False
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return True
+        if dataset_type == 'MCQ':
+            if dataset is not None and 'LEGO' in dataset:
+                return False
+            return True
+        if dataset_type == 'Y/N' and dataset in {'HallusionBench', 'POPE'}:  # MME has it's own prompt
+            return True
+        if dataset_type == 'VQA' and dataset not in {'MMVet', 'ChartQAPro', 'ChartQAPro_CoT', 'ChartQAPro_PoT', 'ChartMuseum'}:  # noqa: E501
+            return True
+        return False
+
+    def build_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        from vlmeval.dataset import DATASET_TYPE
+
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return self._build_mmmu_prompt(line, dataset)
+        dataset_type = DATASET_TYPE(dataset, default=None)
+        if dataset_type == 'MCQ':
+            return self._build_mcq_prompt(line, dataset)
+        if dataset_type == 'Y/N':
+            return self._build_yorn_prompt(line, dataset)
+        if dataset_type == 'VQA':
+            return self._build_vqa_prompt(line, dataset)
+        raise ValueError(f'Unsupported dataset: {dataset}')
+
+    def _build_mmmu_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MMMU dataset: keep all images at beginning."""
+
+        import string
+
+        import pandas as pd
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += 'Please select the correct answer from the options above. \n'
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_mcq_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MCQ dataset: use chinese prompt if the question contains chinese characters."""
+        MCQ_CN_PROMPT = '请直接回答选项字母。'
+        MCQ_EN_PROMPT = 'Please select the correct answer from the options above.'
+
+        import string
+
+        import pandas as pd
+
+        def cn_string(s):
+            import re
+
+            if re.search('[\u4e00-\u9fff]', s):
+                return True
+            return False
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+        options_prompt = 'Options:\n'
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += MCQ_CN_PROMPT if cn_string(prompt) else MCQ_EN_PROMPT
+        prompt = prompt.rstrip()
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_yorn_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for YORN dataset:"""
+        YORN_PROMPT = ' Please answer yes or no.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += YORN_PROMPT
+        return msgs
+
+    def _build_vqa_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for VQA dataset:"""
+        VQA_PROMPT = '\nPlease try to answer the question with short words or phrases if possible.'
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=question))
+        assert msgs[-1]['type'] == 'text'
+        msgs[-1]['value'] += VQA_PROMPT
+        return msgs
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..98e24740e166b7a62bbcfb8ea736f6c6a4fe537f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/__init__.py
@@ -0,0 +1,5 @@
+from .model import Qwen3VLChat
+
+__all__ = [
+    'Qwen3VLChat',
+]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/model.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..7f224371f64ee4fb45a8e7f3499c6a4f7db87be0
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/model.py
@@ -0,0 +1,428 @@
+from __future__ import annotations
+import logging
+import os
+import warnings
+
+import torch
+
+from vlmeval.smp import get_gpu_memory, listinstr
+from ..base import BaseModel
+from .prompt import Qwen3VLPromptMixin
+
+VLLM_MAX_IMAGE_INPUT_NUM = 24
+
+
+def is_moe_model(model_path: str) -> bool:
+    """Check if the model is a Mixture of Experts model by looking for active-param suffixes like A3B, A17B."""
+    import re
+    if re.search(r'-A\d+B', model_path):
+        return True
+    return False
+
+
+def ensure_image_url(image: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:image']
+    if any(image.startswith(prefix) for prefix in prefixes):
+        return image
+    if os.path.exists(image):
+        return 'file://' + image
+    raise ValueError(f'Invalid image: {image}')
+
+
+def ensure_video_url(video: str) -> str:
+    prefixes = ['http://', 'https://', 'file://', 'data:video']
+    if any(video.startswith(prefix) for prefix in prefixes):
+        return video
+    if os.path.exists(video):
+        return 'file://' + video
+    raise ValueError(f'Invalid video: {video}')
+
+
+class Qwen3VLChat(Qwen3VLPromptMixin, BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    VIDEO_LLM = True
+
+    def __init__(
+        self,
+        model_path: str,
+        min_pixels: int | None = None,
+        max_pixels: int | None = None,
+        total_pixels: int | None = None,
+        max_new_tokens: int = 32768,
+        top_p: float = 0.8,
+        top_k: int = 20,
+        temperature: float = 0.01,
+        repetition_penalty: float = 1.0,
+        presence_penalty: float = 1.5,
+        use_custom_prompt: bool = True,
+        system_prompt: str | None = None,
+        post_process: bool = False,
+        verbose: bool = False,
+        use_audio_in_video: bool = True,
+        **kwargs,
+    ) -> None:
+        super().__init__(use_custom_prompt=use_custom_prompt)
+        self.min_pixels = min_pixels
+        self.max_pixels = max_pixels
+        self.total_pixels = total_pixels
+        self.max_new_tokens = max_new_tokens
+        self.top_k = top_k
+        self.top_p = top_p
+        self.repetition_penalty = repetition_penalty
+        self.presence_penalty = presence_penalty
+        self.temperature = temperature
+        if self.total_pixels and self.total_pixels > 24576 * 32 * 32:
+            print('The total number of video tokens might too large, resulting in an overly long input sequence.')
+        self.generate_kwargs = dict(
+            max_new_tokens=self.max_new_tokens,
+            top_p=top_p,
+            top_k=top_k,
+            temperature=temperature,
+            repetition_penalty=repetition_penalty,
+        )
+        self.system_prompt = system_prompt
+        self.verbose = verbose
+        self.post_process = post_process
+        self.fps = kwargs.pop('fps', 2)
+        self.nframe = kwargs.pop('nframe', 128)
+        self.FRAME_FACTOR = 2
+        self.use_audio_in_video = use_audio_in_video
+
+        assert model_path is not None
+        self.model_path = model_path
+        from transformers import AutoModelForImageTextToText, AutoProcessor
+
+        # Use official Qwen3-Omni classes when model_path indicates omni
+        if listinstr(['omni'], model_path.lower()):
+            try:
+                from transformers import (Qwen3OmniMoeForConditionalGeneration,
+                                          Qwen3OmniMoeProcessor)
+            except Exception as err:
+                logging.critical("pip install git+https://github.com/huggingface/transformers")
+                raise err
+            self.processor = Qwen3OmniMoeProcessor.from_pretrained(model_path)
+        else:
+            self.processor = AutoProcessor.from_pretrained(model_path)
+
+        gpu_mems = get_gpu_memory()
+        max_gpu_mem = max(gpu_mems) if gpu_mems != [] else -1
+        assert max_gpu_mem > 0
+
+        self.use_vllm = kwargs.get('use_vllm', False)
+        self.use_lmdeploy = kwargs.get('use_lmdeploy', False)
+        self.limit_mm_per_prompt = VLLM_MAX_IMAGE_INPUT_NUM
+        os.environ['VLLM_WORKER_MULTIPROC_METHOD'] = 'spawn'
+        assert self.use_vllm + self.use_lmdeploy <= 1, "You can only set one flag `use_vllm` to True"
+        if self.use_vllm:
+            if listinstr(['omni'], self.model_path.lower()):
+                os.environ['VLLM_USE_V1'] = '0'
+            from vllm import LLM
+            gpu_count = torch.cuda.device_count()
+            tp_size = gpu_count if gpu_count > 0 else 1
+            logging.info(
+                f'Using vLLM for {self.model_path} inference with {tp_size} GPUs (available: {gpu_count})'
+            )
+            if os.environ.get('VLLM_WORKER_MULTIPROC_METHOD') != 'spawn':
+                logging.warning(
+                    "VLLM_WORKER_MULTIPROC_METHOD is not set to spawn. Use 'export VLLM_WORKER_MULTIPROC_METHOD=spawn'"
+                )
+            enable_expert_parallel = is_moe_model(self.model_path)
+            # For Qwen3-Omni, vLLM engine v1 is not supported yet
+            if listinstr(['omni'], self.model_path.lower()):
+                limit_mm = {"image": 3, "video": 3, "audio": 3}
+            else:
+                limit_mm = {"image": self.limit_mm_per_prompt}
+            self.llm = LLM(
+                model=self.model_path,
+                max_num_seqs=8,
+                limit_mm_per_prompt=limit_mm,
+                tensor_parallel_size=tp_size,
+                enable_expert_parallel=enable_expert_parallel,
+                seed=0,
+                gpu_memory_utilization=kwargs.get("gpu_utils", 0.9),
+                trust_remote_code=True,
+            )
+        else:
+            if listinstr(['omni'], model_path.lower()):
+                self.model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
+                    model_path, dtype='auto', device_map='auto', attn_implementation='flash_attention_2'
+                )
+            else:
+                self.model = AutoModelForImageTextToText.from_pretrained(
+                    model_path, torch_dtype='auto', device_map='auto', attn_implementation='flash_attention_2'
+                )
+            self.model.eval()
+
+        torch.cuda.empty_cache()
+
+    def _prepare_content(self, inputs: list[dict[str, str]], dataset: str | None = None) -> list[dict[str, str]]:
+        content = []
+        for s in inputs:
+            if s['type'] == 'image':
+                item = {'type': 'image', 'image': ensure_image_url(s['value'])}
+                if dataset == 'OCRBench':
+                    item['min_pixels'] = 10 * 10 * 32 * 32
+                    warnings.warn(f"OCRBench dataset uses custom min_pixels={item['min_pixels']}")
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item['min_pixels'] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+                for key in ['min_pixels', 'max_pixels', 'total_pixels', 'resized_height', 'resized_width']:
+                    if key in s and s[key] is not None:
+                        item[key] = s[key]
+            elif s['type'] == 'video':
+                value = s['value']
+                if isinstance(value, list):
+                    item = {
+                        'type': 'video',
+                        'video': [ensure_image_url(v) for v in value],
+                    }
+                else:
+                    item = {'type': 'video', 'video': ensure_video_url(value)}
+                if self.min_pixels is not None:
+                    item['min_pixels'] = self.min_pixels
+                if self.max_pixels is not None:
+                    item['max_pixels'] = self.max_pixels
+                if self.total_pixels is not None:
+                    item['total_pixels'] = self.total_pixels
+                for key in ['resized_height', 'resized_width', 'fps', 'nframes', 'sample_fps']:
+                    if key in s and s[key] is not None:
+                        item[key] = s[key]
+                if not isinstance(value, list):
+                    if self.fps is not None and 'fps' not in item:
+                        item['fps'] = self.fps
+                    elif self.nframe is not None and 'nframes' not in item:
+                        import cv2
+                        video = cv2.VideoCapture(s['value'])
+                        frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                        video.release()
+                        if frame_count < self.nframe:
+                            new_frame_count = frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR
+                            print(f"use {new_frame_count} for {s['value']}")
+                            item['nframes'] = new_frame_count
+                        else:
+                            item['nframes'] = self.nframe
+            elif s['type'] == 'audio':
+                item = {'type': 'audio', 'audio': s['value']}
+            elif s['type'] == 'text':
+                item = {'type': 'text', 'text': s['value']}
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def generate_inner_transformers(self, message, dataset=None):
+        is_omni = listinstr(['omni'], self.model_path.lower())
+        if is_omni:
+            try:
+                from qwen_omni_utils import process_mm_info
+            except Exception as err:
+                logging.critical("Please install it via 'pip install qwen-omni-utils[decord]'")
+                raise err
+        else:
+            try:
+                from qwen_vl_utils import process_vision_info
+            except Exception as err:
+                logging.critical("Please install it via 'pip install qwen-vl-utils'")
+                raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({'role': 'system', 'content': self.system_prompt})
+        messages.append({'role': 'user', 'content': self._prepare_content(message, dataset=dataset)})
+        if self.verbose:
+            print(f'\033[31m{messages}\033[0m')
+
+        if is_omni:
+            # For Qwen3-Omni, messages is a list of dicts
+            text = self.processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+            audios, images, videos = process_mm_info(messages, use_audio_in_video=self.use_audio_in_video)
+            inputs = self.processor(
+                text=text,
+                audio=audios,
+                images=images,
+                videos=videos,
+                return_tensors='pt',
+                padding=True,
+                use_audio_in_video=self.use_audio_in_video,
+            )
+        else:
+            text = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+            images, videos, video_kwargs = process_vision_info(
+                messages,
+                image_patch_size=16,
+                return_video_kwargs=True,
+                return_video_metadata=True,
+            )
+
+            video_metadatas = None
+            if videos is not None:
+                videos, video_metadatas = zip(*videos)
+                videos, video_metadatas = list(videos), list(video_metadatas)
+
+            inputs = self.processor(
+                text=text,
+                images=images,
+                videos=videos,
+                video_metadata=video_metadatas,
+                do_resize=False,
+                return_tensors='pt',
+                **(video_kwargs or {}),
+            )
+        try:
+            inputs = inputs.to(self.model.device)
+            if hasattr(self.model, 'dtype'):
+                inputs = inputs.to(self.model.dtype)
+        except Exception:
+            inputs = inputs.to('cuda')
+
+        if is_omni:
+            try:
+                text_ids, _ = self.model.generate(
+                    **inputs,
+                    return_audio=False,
+                    thinker_return_dict_in_generate=True,
+                    use_audio_in_video=self.use_audio_in_video,
+                )
+            except TypeError:
+                text_ids, _ = self.model.generate(
+                    **inputs,
+                    return_audio=False,
+                    use_audio_in_video=self.use_audio_in_video,
+                )
+            response = self.processor.batch_decode(
+                text_ids.sequences[:, inputs["input_ids"].shape[1]:],
+                skip_special_tokens=True,
+                clean_up_tokenization_spaces=False,
+            )[0]
+        else:
+            generated_ids = self.model.generate(
+                **inputs,
+                **self.generate_kwargs,
+            )
+            generated_ids = [
+                output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
+            ]
+            out = self.processor.tokenizer.batch_decode(
+                generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
+            )
+            response = out[0]
+        if self.post_process:
+            resp = response.split('\\boxed{')[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == '{':
+                    counter += 1
+                elif resp[i] == '}':
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                response = resp[:end]
+
+        if self.verbose:
+            print(f'\033[32m{response}\033[0m')
+        return response
+
+    def generate_inner_vllm(self, message, dataset=None):
+        from vllm import SamplingParams
+        is_omni = listinstr(['omni'], self.model_path.lower())
+        if is_omni:
+            try:
+                from qwen_omni_utils import process_mm_info
+            except Exception as err:
+                logging.critical("qwen_omni_utils not found, 'pip install qwen-omni-utils[decord]'")
+                raise err
+        else:
+            try:
+                from qwen_vl_utils import process_vision_info
+            except Exception as err:
+                logging.critical("qwen_vl_utils not found, 'pip install qwen-vl-utils'")
+                raise err
+
+        messages = []
+        if self.system_prompt is not None:
+            messages.append({'role': 'system', 'content': self.system_prompt})
+        messages.append({'role': 'user', 'content': self._prepare_content(message, dataset=dataset)})
+        if self.verbose:
+            print(f'\033[31m{messages}\033[0m')
+
+        text = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+        if is_omni:
+            audios, image_inputs, video_inputs = process_mm_info(messages, use_audio_in_video=self.use_audio_in_video)
+        else:
+            image_inputs, video_inputs, video_kwargs = process_vision_info(
+                messages,
+                image_patch_size=16,
+                return_video_kwargs=True,
+                return_video_metadata=True,
+            )
+
+        sampling_params = SamplingParams(
+            temperature=self.temperature,
+            max_tokens=self.max_new_tokens,
+            top_p=self.top_p,
+            top_k=self.top_k,
+            repetition_penalty=self.repetition_penalty,
+            presence_penalty=self.presence_penalty,
+            stop_token_ids=None
+        )
+        mm_data = {}
+        if image_inputs is not None:
+            mm_data['image'] = image_inputs
+        if video_inputs is not None:
+            mm_data['video'] = video_inputs
+        if is_omni and 'audios' in locals() and audios is not None:
+            mm_data['audio'] = audios
+
+        req = {'prompt': text}
+        if mm_data:
+            req['multi_modal_data'] = mm_data
+        if is_omni:
+            req['mm_processor_kwargs'] = {"use_audio_in_video": self.use_audio_in_video}
+        elif video_kwargs is not None:
+            req['mm_processor_kwargs'] = video_kwargs
+
+        outputs = self.llm.generate([req], sampling_params=sampling_params)
+
+        for o in outputs:
+            generated_text = o.outputs[0].text
+
+        if self.post_process:
+            resp = generated_text.split('\\boxed{')[-1]
+            lt = len(resp)
+            counter, end = 1, None
+            for i in range(lt):
+                if resp[i] == '{':
+                    counter += 1
+                elif resp[i] == '}':
+                    counter -= 1
+                if counter == 0:
+                    end = i
+                    break
+                elif i == lt - 1:
+                    end = lt
+                    break
+            if end is not None:
+                generated_text = resp[:end]
+
+        if self.verbose:
+            print(f'\033[32m{generated_text}\033[0m')
+        return generated_text
+
+    def generate_inner(self, message, dataset=None):
+        if self.use_vllm:
+            return self.generate_inner_vllm(message, dataset=dataset)
+        else:
+            return self.generate_inner_transformers(message, dataset=dataset)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..22b3651a25258d43df026e21b1a89561589f2866
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/qwen3_vl/prompt.py
@@ -0,0 +1,162 @@
+from __future__ import annotations
+
+
+class Qwen3VLPromptMixin:
+    """
+    Mixin class for Qwen3VLChat to build prompts consistent with Qwen3-VL README.
+
+    Requires the following methods to be implemented in the subclass:
+        - dump_image(line, dataset: str) -> str | list[str]
+
+    Implements the following methods:
+        - use_custom_prompt(dataset: str) -> bool
+        - build_prompt(line, dataset: str) -> list[dict[str, str]]
+    """
+
+    def __init__(self, *args, use_custom_prompt: bool = True, **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        self._use_custom_prompt = use_custom_prompt
+
+    def set_dump_image(self, dump_image_func):
+        self.dump_image_func = dump_image_func
+
+    def dump_image(self, line, dataset):
+        return self.dump_image_func(line)
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        from vlmeval.dataset import DATASET_TYPE
+        dataset_type = DATASET_TYPE(dataset, default=None)
+
+        if not self._use_custom_prompt:
+            return False
+        if dataset in {"SSI_Bench"}:
+            return False
+        # Follow Qwen3-VL convention: apply concise, task-specified prompts for MCQ/YN/VQA
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return True
+        if dataset_type == 'MCQ':
+            return True
+        if dataset_type == 'Y/N':
+            return True
+        if dataset_type == 'VQA':
+            return True
+        return False
+
+    def build_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        from vlmeval.dataset import DATASET_TYPE
+
+        if dataset in {'MMMU_DEV_VAL', 'MMMU_TEST'}:
+            return self._build_mmmu_prompt(line, dataset)
+
+        dataset_type = DATASET_TYPE(dataset, default=None)
+        if dataset_type == 'MCQ':
+            return self._build_mcq_prompt(line, dataset)
+        if dataset_type == 'Y/N':
+            return self._build_yorn_prompt(line, dataset)
+        if dataset_type == 'VQA':
+            return self._build_vqa_prompt(line, dataset)
+        raise ValueError(f'Unsupported dataset: {dataset}')
+
+    def _build_mmmu_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """Keep all images at the beginning; then a single user text message.
+        Matches Qwen3-VL multi-image style shown in README examples.
+        """
+        import string
+
+        import pandas as pd
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+
+        options_prompt = ''
+        if len(options):
+            options_prompt = 'Options:\n'
+            for key, item in options.items():
+                options_prompt += f'{key}. {item}\n'
+
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += 'Please select the correct answer from the options above.'
+        prompt = prompt.rstrip()
+
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_mcq_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """Multi-choice prompt: include options and require a single option letter.
+        Keep images before the text per Qwen3-VL convention.
+        """
+        import string
+
+        import pandas as pd
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        options = {cand: line[cand] for cand in string.ascii_uppercase if cand in line and not pd.isna(line[cand])}
+
+        options_prompt = ''
+        if len(options):
+            options_prompt = 'Options:\n'
+            for key, item in options.items():
+                options_prompt += f'{key}. {item}\n'
+
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        prompt = ''
+        if hint is not None:
+            prompt += f'Hint: {hint}\n'
+        prompt += f'Question: {question}\n'
+        if len(options):
+            prompt += options_prompt
+            prompt += 'Answer with the option letter only.'
+        prompt = prompt.rstrip()
+
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_yorn_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """Yes/No prompt: require explicit yes or no answer only.
+        Keep images before the text.
+        """
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        prompt = f'{question} Please answer yes or no.'
+
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
+
+    def _build_vqa_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """VQA prompt: concise question with preference for short answers.
+        Keep images before the text.
+        """
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        prompt = question + '\nPlease answer concisely with short words or phrases when possible.'
+
+        msgs = []
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type='image', value=tgt_path)]
+        msgs.append(dict(type='text', value=prompt))
+        return msgs
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..932f3c16bbf4722706693393913da8efc7c683b2
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/__init__.py
@@ -0,0 +1,2 @@
+from .model import Thyme  # noqa: F401
+from .prompt import ThymePromptMixin  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/model.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..dd3fc12fa3d8f0e47ca0731acaa139197ba7d1dd
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/model.py
@@ -0,0 +1,612 @@
+from __future__ import annotations
+import copy
+import logging
+import os
+import re
+import warnings
+
+import torch
+from transformers.cache_utils import DynamicCache
+
+from ..base import BaseModel
+from .prompt import ThymePromptMixin
+from .sandbox import execute_code_in_sandbox
+from .utils import (REASONING_SYS_PROMPT, SIMPLE_SYS_PROMPT, SPECIAL_STRING_LIST,
+                    generate_prompt_final_qa, generate_prompt_simple_qa)
+
+
+def ensure_image_url(image: str) -> str:
+    prefixes = ["http://", "https://", "file://", "data:image;"]
+    if any(image.startswith(prefix) for prefix in prefixes):
+        return image
+    if os.path.exists(image):
+        return "file://" + image
+    raise ValueError(f"Invalid image: {image}")
+
+
+def ensure_video_url(video: str) -> str:
+    prefixes = ["http://", "https://", "file://", "data:video;"]
+    if any(video.startswith(prefix) for prefix in prefixes):
+        return video
+    if os.path.exists(video):
+        return "file://" + video
+    raise ValueError(f"Invalid video: {video}")
+
+
+class Thyme(ThymePromptMixin, BaseModel):
+    INSTALL_REQ = False
+    INTERLEAVE = True
+    VIDEO_LLM = True
+
+    def __init__(
+        self,
+        model_path: str,
+        min_pixels: int | None = None,
+        max_pixels: int | None = None,
+        max_new_tokens=2048,
+        top_p=0.001,
+        top_k=1,
+        temperature=0.01,
+        repetition_penalty=1.0,
+        # pandayin: rounds of intermediate steps before reaching final answer.
+        max_iterations=5,
+        # pandayin: max retry before reaching a valid answer.
+        max_retry=5,
+        use_custom_prompt: bool = True,
+        system_prompt: str | None = "You are a helpful assistant.",
+        post_process: bool = True,
+        # if True, will try to only extract stuff wrapped in <answer> &
+        # </answer>.
+        verbose: bool = True,
+        auto_cleanup: bool = True,
+        # Clean up intermediate images generated by sandbox code.
+        **kwargs,
+    ):
+        from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
+        super().__init__(use_custom_prompt=use_custom_prompt)
+        self.min_pixels = min_pixels
+        self.max_pixels = max_pixels
+        self.top_p = top_p
+        self.top_k = top_k
+        self.temperature = temperature
+        self.system_prompt = system_prompt
+        self.max_iterations = max_iterations
+        self.max_retry = max_retry
+        self.verbose = verbose
+        self.post_process = post_process
+        self.auto_cleanup = auto_cleanup
+        self.fps = 2.0
+        self.nframe = 64
+        self.FRAME_FACTOR = 2
+        assert model_path is not None
+        self.model_path = model_path
+        self.processor = AutoProcessor.from_pretrained(model_path)
+
+        self.generate_kwargs = dict(
+            max_new_tokens=max_new_tokens,
+            top_p=top_p,
+            top_k=top_k,
+            temperature=temperature,
+            repetition_penalty=repetition_penalty,
+            stop_strings=SPECIAL_STRING_LIST,
+            eos_token_id=self.processor.tokenizer.eos_token_id,
+            tokenizer=self.processor.tokenizer,
+        )
+
+        self.model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
+            model_path,
+            torch_dtype="auto",
+            device_map="auto",
+            attn_implementation="sdpa",
+        )
+        self.model.eval()
+
+        torch.cuda.empty_cache()
+
+    def _extract_image_path(self, contents: list[dict[str, str]]):
+        user_image_path = ""
+        content_history = copy.deepcopy(contents)
+        for rou in content_history:
+            if rou["type"] != "image":
+                continue
+            user_image_path = rou["value"]
+            break
+        return user_image_path
+
+    def _prepare_content(
+        self, inputs: list[dict[str, str]], dataset: str | None = None
+    ) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        user_image_path = self._extract_image_path(inputs)
+        content = []
+        for s in inputs:
+            if s["type"] == "image":
+                item = {"type": "image", "image": ensure_image_url(s["value"])}
+                if dataset == "OCRBench":
+                    item["min_pixels"] = 10 * 10 * 28 * 28
+                    warnings.warn(
+                        f"OCRBench dataset uses custom min_pixels={item['min_pixels']}"
+                    )
+                    if self.max_pixels is not None:
+                        item["max_pixels"] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item["min_pixels"] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item["max_pixels"] = self.max_pixels
+            elif s["type"] == "video":
+                item = {
+                    "type": "video",
+                    "video": ensure_video_url(s["value"]),
+                    "min_pixels": self.min_pixels,
+                    "max_pixels": self.max_pixels,
+                }
+                if self.fps is not None:
+                    item["fps"] = self.fps
+                elif self.nframe is not None:
+                    import cv2
+
+                    video = cv2.VideoCapture(s["value"])
+                    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                    video.release()
+                    if frame_count < self.nframe:
+                        new_frame_count = (
+                            frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR)
+                        print(f"use {new_frame_count} for {s['value']}")
+                        item["nframes"] = new_frame_count
+                    else:
+                        item["nframes"] = self.nframe
+            elif s["type"] == "text":
+                item = {
+                    "type": "text",
+                    "text": generate_prompt_final_qa(
+                        s["value"],
+                        user_image_path),
+                }
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def _prepare_content_simple(
+        self, inputs: list[dict[str, str]], dataset: str | None = None
+    ) -> list[dict[str, str]]:
+        """
+        inputs list[dict[str, str]], each dict has keys: ['type', 'value']
+        """
+        content = []
+        for s in inputs:
+            if s["type"] == "image":
+                item = {"type": "image", "image": ensure_image_url(s["value"])}
+                if dataset == "OCRBench":
+                    item["min_pixels"] = 10 * 10 * 28 * 28
+                    warnings.warn(
+                        f"OCRBench dataset uses custom min_pixels={item['min_pixels']}"
+                    )
+                    if self.max_pixels is not None:
+                        item["max_pixels"] = self.max_pixels
+                else:
+                    if self.min_pixels is not None:
+                        item["min_pixels"] = self.min_pixels
+                    if self.max_pixels is not None:
+                        item["max_pixels"] = self.max_pixels
+            elif s["type"] == "video":
+                item = {
+                    "type": "video",
+                    "video": ensure_video_url(s["value"]),
+                    "min_pixels": self.min_pixels,
+                    "max_pixels": self.max_pixels,
+                }
+                if self.fps is not None:
+                    item["fps"] = self.fps
+                elif self.nframe is not None:
+                    import cv2
+
+                    video = cv2.VideoCapture(s["value"])
+                    frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
+                    video.release()
+                    if frame_count < self.nframe:
+                        new_frame_count = (
+                            frame_count // self.FRAME_FACTOR * self.FRAME_FACTOR)
+                        print(f"use {new_frame_count} for {s['value']}")
+                        item["nframes"] = new_frame_count
+                    else:
+                        item["nframes"] = self.nframe
+            elif s["type"] == "text":
+                item = {
+                    "type": "text",
+                    "text": generate_prompt_simple_qa(
+                        s["value"])}
+            else:
+                raise ValueError(f"Invalid message type: {s['type']}, {s}")
+            content.append(item)
+        return content
+
+    def _extract_box_answer(self, response):
+        resp = response.split("\\boxed{")[-1]
+        lt = len(resp)
+        counter, end = 1, None
+        for i in range(lt):
+            if resp[i] == "{":
+                counter += 1
+            elif resp[i] == "}":
+                counter -= 1
+            if counter == 0:
+                end = i
+                break
+            elif i == lt - 1:
+                end = lt
+                break
+        if end is not None:
+            response = resp[:end]
+        return response
+
+    def _remove_unpickable_values(self, dictionary):
+        import pickle
+
+        def is_pickable(obj):
+            try:
+                pickle.dumps(obj)
+                return True
+            except (pickle.PicklingError, TypeError, AttributeError):
+                return False
+
+        keys_to_remove = []
+        for key, value in dictionary.items():
+            if isinstance(value, dict):
+                self._remove_unpickable_values(value)
+            elif not is_pickable(value):
+                keys_to_remove.append(key)
+        for key in keys_to_remove:
+            del dictionary[key]
+        return dictionary
+
+    def generate_inner_transformers(
+            self,
+            message,
+            dataset=None,
+            temp_output_dir=None):
+        try:
+            from qwen_vl_utils import process_vision_info
+        except Exception as err:
+            logging.critical(
+                "qwen_vl_utils not found, please install it via 'pip install qwen-vl-utils'"
+            )  # noqa: E501
+            raise err
+
+        user_image_path = self._extract_image_path(message)
+
+        messages = []
+        messages.append({"role": "system", "content": REASONING_SYS_PROMPT})
+        messages.append(
+            {"role": "user", "content": self._prepare_content(message, dataset=dataset)}
+        )
+
+        if self.verbose:
+            print(f"\033[31m{messages}\033[0m")
+
+        #   -------   outer loop. retry multiple times if fail to reach a valid
+        retry_generations = self.max_retry
+        has_valid_answer = False
+        while (retry_generations > 0) and (not has_valid_answer):
+            # pandayin: main logic/ work flow for generation.
+            # The gist is to pause at special tokens (</code> & </answer>) and
+            # maybe perform code execution.
+            conversation_history = copy.deepcopy(messages)
+
+            # For each generation, we initialize a KV-Cache to speed up
+            # inference.
+            kv_cache = DynamicCache()
+            # Maintain a dictionary to save context (local & global vars.) for
+            # code execution.
+            previous_execution_context = {}
+            if self.verbose:
+                print(
+                    f"\033[32m\n--- Generation {self.max_retry - retry_generations + 1} ---\033[0m"
+                )
+
+            #   -------   inner loop. generate multiple steps until reaching a
+            retry_iterations = self.max_iterations
+            # We assume each answer round is limited to a few code (usually 1)
+            # execution.
+            while retry_iterations > 0:
+                retry_iterations -= 1
+                generated_content = []
+                if self.verbose:
+                    print(
+                        f"\033[32m\n--- Iteration {self.max_iterations - retry_iterations} ---\033[0m"
+                    )
+
+                text = self.processor.apply_chat_template(
+                    [conversation_history], tokenize=False, add_generation_prompt=(
+                        retry_iterations == self.max_iterations - 1), )
+
+                if retry_iterations != self.max_iterations - 1:
+                    if text[0].endswith("<|im_end|>\n"):
+                        text[0] = text[0][: -len("<|im_end|>\n")]
+                images, videos = process_vision_info([conversation_history])
+                inputs = self.processor(
+                    text=text,
+                    images=images,
+                    videos=videos,
+                    padding=True,
+                    return_tensors="pt",
+                )
+                inputs = inputs.to("cuda")
+
+                # just in case this iteration is invalid, we need to roll back,
+                # thus making a backup.
+                last_kv_cache = copy.deepcopy(kv_cache)
+                # bkup context. roll back when we fail to execute the generated
+                # code.
+                last_execution_context = copy.deepcopy(
+                    self._remove_unpickable_values(previous_execution_context)
+                )
+                generated_ids = self.model.generate(
+                    **inputs, **self.generate_kwargs, past_key_values=kv_cache
+                )
+                generated_ids = [
+                    output_ids[len(input_ids):]
+                    for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
+                ]
+                out = self.processor.tokenizer.batch_decode(
+                    generated_ids,
+                    skip_special_tokens=True,
+                    clean_up_tokenization_spaces=False,
+                )
+                generated_text_segment = out[0]
+
+                # Case 1: directly give answer
+                if "</answer>" in generated_text_segment:
+                    generated_content.append(
+                        {"type": "text", "text": generated_text_segment},
+                    )
+
+                # Case 2: reach code generation.
+                # parse current result. Two cases: reach </code> or reach
+                # </answer>
+                code_regex = re.compile(
+                    r"<code>\s*(?:```\s*)?(?:python\s*)?([\s\S]*?)\s*(?:```\s*)?</code>",
+                    re.IGNORECASE,
+                )
+
+                code_match = code_regex.search(generated_text_segment)
+
+                # execute code and return result.
+                if code_match:
+                    code_to_execute = code_match.group(1).strip()
+                    if self.verbose:
+                        print(
+                            f"\033[31m--- Found Code Block ---\n"
+                            f"{generated_text_segment}\n"
+                            f",-------------------------\033[0m"
+                        )
+
+                    (
+                        processed_img_paths,
+                        captured_stdout,
+                        error_msg,
+                        current_execution_context,
+                    ) = execute_code_in_sandbox(
+                        code_to_execute,
+                        user_image_path,
+                        temp_output_dir=temp_output_dir,
+                        previous_execution_context=previous_execution_context,
+                    )
+                    previous_execution_context = current_execution_context
+                    if not processed_img_paths:
+                        # deemed as unsuccessful iteration. roll back status.
+                        kv_cache = last_kv_cache
+                        previous_execution_context = last_execution_context
+                        print(f"{error_msg}")
+                        continue
+
+                    has_valid_images = False
+                    generated_content += [
+                        {"type": "text", "text": generated_text_segment},
+                        {"type": "text", "text": "<sandbox_output>"},
+                    ]
+                    first_path = processed_img_paths[0]
+                    if os.path.exists(first_path):
+                        # Iterate through each path in the list
+                        for img_path in processed_img_paths:
+                            if os.path.exists(img_path):
+                                # Add text segments only once per sandbox
+                                # output block
+                                if not has_valid_images:
+                                    has_valid_images = True
+                                generated_content.append(
+                                    {"type": "image", "image": img_path}
+                                )
+                    else:
+                        generated_content.append(
+                            {"type": "text", "text": first_path})
+
+                    if has_valid_images or not os.path.exists(first_path):
+                        generated_content.append(
+                            {"type": "text", "text": "</sandbox_output>"}
+                        )
+                    else:
+                        # pandayin: a failed code execution/generation doesn't
+                        # count as an intermedia step.
+                        print(
+                            "skip this generation due to error and adapt the temperature"
+                        )
+                        self.generate_kwargs["temperature"] = 1.0
+                        continue
+                else:
+                    # wo code. wo </answer>, assume repetition generated,
+                    # break.
+                    if "</answer>" not in generated_text_segment:
+                        print("wo code. wo </answer>")
+                        print(generated_text_segment)
+                        self.generate_kwargs["temperature"] = 1.0
+                        break
+
+                # Update conversation_history with the latest generated segment
+                # If the last message was 'user', start a new 'assistant'
+                # message
+                if conversation_history[-1]["role"] == "user":
+                    conversation_history.append(
+                        {"role": "assistant", "content": generated_content}
+                    )
+                # If the last message was 'assistant', append to its last text
+                # content item
+                elif conversation_history[-1]["role"] == "assistant":
+                    conversation_history[-1]["content"] += generated_content
+
+                # --- Check for final answer tag if no code was processed in this segment ---
+                if "</answer>" in generated_text_segment:
+                    has_valid_answer = True
+                    print("\033[32m--- Final answer tag found. ---\033[0m")
+                    break
+
+                # If the model produced an EOS token and no code/answer, it
+                # might be finished
+                if generated_ids[0][-1] == self.processor.tokenizer.eos_token_id:
+                    if self.verbose:
+                        print(
+                            "\033[32m--- Model generated EOS and no further actions (code/answer)."
+                            "Assuming completion. ---\033[0m"
+                        )
+                    break
+
+            # End of a generation. Maybe successfully find a valid answer, or
+            # start a new generation.
+            if self.verbose:
+                if has_valid_answer:
+                    print(
+                        f"\033[32m\n--- End of processing (max iterations: {self.max_iterations},"
+                        f"actual: {self.max_iterations - retry_iterations + 1}) ---\033[0m"
+                    )
+                    break
+                else:
+                    print(
+                        f"\033[32m\n --- Fail to find a valid answer. (max retrys: {self.max_retry},"
+                        f"actual: {self.max_retry - retry_generations + 1})---\033[0m"
+                    )
+
+            retry_generations -= 1
+            # pandayin: Adjust/reset generation_kwargs here. So more
+            # explorations could be done to find a valid answer.
+            print("Fail to find a valid answer and adapt the temperature")
+            self.generate_kwargs["temperature"] = 1.0
+
+        # reset generation hyper-param.
+        self.generate_kwargs["temperature"] = self.temperature
+
+        # pandayin: If we still fail after max_try generations, try a simple
+        # prompt.
+        if not has_valid_answer:
+            print(
+                f"\033[32m\n --- Fail to find a valid answer after {self.max_retry} retrys."
+                f"Falling back to simple prompt.---\033[0m"
+            )
+            del self.generate_kwargs["stop_strings"]
+
+            messages = []
+            if self.system_prompt is not None:
+                messages.append(
+                    {"role": "system", "content": SIMPLE_SYS_PROMPT})
+            messages.append(
+                {
+                    "role": "user",
+                    "content": self._prepare_content_simple(message, dataset=dataset),
+                }
+            )
+            conversation_history = copy.deepcopy(messages)
+            text = self.processor.apply_chat_template(
+                [conversation_history], tokenize=False, add_generation_prompt=True)
+
+            images, videos = process_vision_info([conversation_history])
+            inputs = self.processor(
+                text=text,
+                images=images,
+                videos=videos,
+                padding=True,
+                return_tensors="pt",
+            )
+            inputs = inputs.to("cuda")
+            generated_ids = self.model.generate(
+                **inputs,
+                **self.generate_kwargs,
+            )
+            generated_ids = [
+                output_ids[len(input_ids):]
+                for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
+            ]
+            out = self.processor.tokenizer.batch_decode(
+                generated_ids,
+                skip_special_tokens=True,
+                clean_up_tokenization_spaces=False,
+            )
+            generated_text_segment = out[0]
+
+            self.generate_kwargs["stop_strings"] = SPECIAL_STRING_LIST
+
+            # to align with the following processing procedure. wrap a <answer>
+            # bracket.
+            answer_match = re.search(
+                r"<answer>(.*?)</answer>", generated_text_segment, re.DOTALL
+            )
+            if not answer_match:
+                generated_text_segment = (
+                    "<answer>" + generated_text_segment + "</answer>"
+                )
+            conversation_history.append(
+                {
+                    "role": "assistant",
+                    "content": [{"type": "text", "text": generated_text_segment}],
+                }
+            )
+
+        final_assistant_response = ""
+        for msg in reversed(conversation_history):
+            if msg["role"] != "assistant":
+                continue
+            current_content_str = ""
+            for item in msg["content"]:
+                if item["type"] == "text":
+                    current_content_str += item["text"]
+            # Get the last full response from assistant
+            final_assistant_response = current_content_str
+            break
+
+        if self.post_process:
+            print(
+                f"\033[31m--- Final response ---\n{final_assistant_response}\n-------------------------\033[0m"
+            )
+            # Extract content within <answer> tags from the final assistant
+            # response
+            answer_match = re.search(
+                r"<answer>(.*?)</answer>", final_assistant_response, re.DOTALL
+            )
+            if answer_match:
+                final_answer = answer_match.group(1).strip()
+            else:
+                final_answer = "No answer tag found in the final output."
+
+            # Sometimes the answer is still wrapped in \boxed{}, keeping the behaviour of Qwen2.5-VL.
+            # We extract the answer within this.
+            match = re.search(r"\\boxed\{(.*?)\}", final_answer)
+            if match:
+                final_answer = self._extract_box_answer(final_answer)
+
+            if self.verbose:
+                print(f"\033[32m{final_answer}\033[0m")
+            return final_answer
+        else:
+            return final_assistant_response
+
+    def generate_inner(self, message, dataset=None):
+        if self.auto_cleanup:
+            import tempfile
+
+            with tempfile.TemporaryDirectory() as temp_dir:
+                return self.generate_inner_transformers(
+                    message, dataset=dataset, temp_output_dir=temp_dir
+                )
+        else:
+            return self.generate_inner_transformers(message, dataset=dataset)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/prompt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/prompt.py
new file mode 100644
index 0000000000000000000000000000000000000000..b8c93ec83a882fc08492908749b2f438b45e1212
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/prompt.py
@@ -0,0 +1,270 @@
+from __future__ import annotations
+
+from vlmeval.smp import listinstr
+
+
+class ThymePromptMixin:
+    """
+    Mixin class for Thyme to build custom prompt for different datasets.
+
+    Requires the following methods to be implemented in the subclass:
+        - dump_image(line, dataset: str) -> str | list[str]
+
+    Implements the following methods:
+        - use_custom_prompt(dataset: str) -> bool
+        - build_prompt(line, dataset: str) -> list[dict[str, str]]
+    """
+
+    def __init__(
+            self,
+            *args,
+            use_custom_prompt: bool = True,
+            **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        self._use_custom_prompt = use_custom_prompt
+
+    def set_dump_image(self, dump_image_func):
+        self.dump_image_func = dump_image_func
+
+    def dump_image(self, line, dataset):
+        return self.dump_image_func(line)
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        return self._use_custom_prompt
+
+    def build_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        from vlmeval.dataset import DATASET_TYPE
+
+        if dataset in {"MMMU_DEV_VAL", "MMMU_TEST"}:
+            return self._build_mmmu_prompt(line, dataset)
+        if "OlympiadBench" in dataset:
+            return self._build_olympiad_prompt(line, dataset)
+        dataset_type = DATASET_TYPE(dataset, default=None)
+        if dataset_type in ["MCQ", "MMERealWorld"]:
+            return self._build_mcq_prompt(line, dataset)
+        if dataset_type == "Y/N":
+            return self._build_yorn_prompt(line, dataset)
+        if dataset_type == "VQA":
+            return self._build_vqa_prompt(line, dataset)
+        # Fall back to default prompt
+        return self._build_default_prompt(line, dataset)
+
+    def _build_mmmu_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MMMU dataset: keep all images at beginning."""
+
+        import string
+
+        import pandas as pd
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line["question"]
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        options_prompt = "Options:\n"
+        for key, item in options.items():
+            options_prompt += f"{key}. {item}\n"
+        hint = line["hint"] if (
+            "hint" in line and not pd.isna(
+                line["hint"])) else None
+        prompt = ""
+        if hint is not None:
+            prompt += f"Hint: {hint}\n"
+        prompt += f"Question: {question}\n"
+        if len(options):
+            prompt += options_prompt
+            prompt += "Please select the correct answer from the options above. \n"
+        prompt = prompt.rstrip()
+        msgs = []
+
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=prompt))
+        return msgs
+
+    def _build_mcq_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for MCQ dataset: use chinese prompt if the question contains chinese characters."""
+        MCQ_CN_PROMPT = "请直接回答选项字母。"
+        MCQ_EN_PROMPT = "Please select the correct answer from the options above."
+
+        import string
+
+        import pandas as pd
+
+        def cn_string(s):
+            import re
+
+            if re.search("[\u4e00-\u9fff]", s):
+                return True
+            return False
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line["question"]
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        options_prompt = "Options:\n"
+        for key, item in options.items():
+            options_prompt += f"{key}. {item}\n"
+        hint = line["hint"] if (
+            "hint" in line and not pd.isna(
+                line["hint"])) else None
+        prompt = ""
+        if hint is not None:
+            prompt += f"Hint: {hint}\n"
+        prompt += f"Question: {question}\n"
+        if len(options):
+            prompt += options_prompt
+            prompt += MCQ_CN_PROMPT if cn_string(prompt) else MCQ_EN_PROMPT
+        prompt = prompt.rstrip()
+        msgs = []
+
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=prompt))
+        return msgs
+
+    def _build_yorn_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for YORN dataset:"""
+        YORN_PROMPT = " Please answer yes or no."
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line["question"]
+        msgs = []
+
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=question))
+        assert msgs[-1]["type"] == "text"
+        if not listinstr(["MME"], dataset):
+            msgs[-1]["value"] += YORN_PROMPT
+        return msgs
+
+    def _build_vqa_prompt(self, line, dataset: str) -> list[dict[str, str]]:
+        """change the prompt for VQA dataset:"""
+        VQA_PROMPT = "\nPlease try to answer the question with short words or phrases if possible."
+
+        tgt_path = self.dump_image(line, dataset)
+        question = line["question"]
+        msgs = []
+
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=question))
+        assert msgs[-1]["type"] == "text"
+        if not listinstr(["MMVet"], dataset):
+            msgs[-1]["value"] += VQA_PROMPT
+        return msgs
+
+    def _build_olympiad_prompt(
+            self, line, dataset: str) -> list[dict[str, str]]:
+
+        from vlmeval.dataset.utils.olympiadbench import get_answer_type_text, make_input
+
+        self.is_chinese = "zh" in line["source"]
+        self.is_math = "maths" in line["source"]
+        self.is_theorem_proving = "TP" in line["source"]
+
+        if self.is_chinese:
+            subject_content = "数学" if self.is_math else "物理"
+            if self.is_theorem_proving:
+                prompt = (
+                    f"以下是中国{subject_content}竞赛中的证明题。请根据题目的要求，运用逻辑推理及常用定理证明题目中的命题。"
+                    "证明过程中使用的变量和公式请使用LaTeX格式表示。")
+            else:
+                answer_type_text = get_answer_type_text(
+                    line["answer_type"],
+                    is_chinese=True,
+                    multiple_answer=line["is_multiple_answer"],
+                )
+                if line["is_multiple_answer"]:
+                    multiple_answer_text = "\\boxed{用英文逗号连接的多个答案}"
+                else:
+                    multiple_answer_text = "\\boxed{答案}"
+                unit_text = ""
+                if line["unit"]:
+                    multiple_answer_text += "(单位)"
+                    unit_text = "，注意答案的单位不要放在\\boxed{}中"
+                prompt = (
+                    f"以下是中国{subject_content}竞赛中的解答题{answer_type_text}。请根据题目的要求和所提供的信息计算得出答案。"
+                    f"解答过程和结果中使用的变量和公式请使用LaTeX格式表示。请在最后以“所以最终答案是{multiple_answer_text}。”"
+                    f"显式给出结果{unit_text}。")
+        else:
+            subject_content = "Math" if self.is_math else "Physics"
+            if self.is_theorem_proving:
+                prompt = (
+                    f"The following is a theorem proving problem from an International {subject_content} competition. "
+                    "Please use logical reasoning and common theorems to prove the proposition in the problem "
+                    "according to the given requirements. "
+                    "Please use LaTeX format to represent the variables and formulas used in the proof.")
+            else:
+                if line["is_multiple_answer"]:
+                    multiple_answer_text = (
+                        "\\boxed{multiple answers connected with commas}"
+                    )
+                else:
+                    multiple_answer_text = "\\boxed{answer}"
+                unit_text = ""
+                if line["unit"]:
+                    multiple_answer_text += "(unit)"
+                    unit_text = ", note that the unit of the answer should not be included in \\boxed{}"
+                answer_type_text = get_answer_type_text(
+                    line["answer_type"],
+                    is_chinese=False,
+                    multiple_answer=line["is_multiple_answer"],
+                )
+                prompt = (
+                    f"The following is an open-ended problem from an International {subject_content} competition. "
+                    f"{answer_type_text}Please calculate the answer according to the given requirements and "
+                    "the information provided. Please use LaTeX format to represent the variables and formulas "
+                    'used in the solution process and results. Please end your solution with "So the final answer '
+                    f'is {multiple_answer_text}." and give the result explicitly{unit_text}.')
+
+        if self.is_math:
+            input = make_input(prompt, line["question"])
+        else:
+            if (
+                "context" in line.keys() and str(line["context"]) != "nan"
+            ):  # cannot be null
+                input = make_input(
+                    prompt, line["context"] + "\n" + line["question"])
+            else:
+                input = make_input(prompt, line["question"])
+
+        tgt_path = self.dump_image(line, dataset)
+
+        msgs = []
+
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=input))
+
+        return msgs
+
+    def _build_default_prompt(
+            self, line, dataset: str) -> list[dict[str, str]]:
+        """For non-customized datasets, use a simple default prompt."""
+        tgt_path = self.dump_image(line, dataset)
+        question = line["question"]
+        msgs = []
+
+        if isinstance(tgt_path, list):
+            msgs.extend([dict(type="image", value=p) for p in tgt_path])
+        else:
+            msgs = [dict(type="image", value=tgt_path)]
+        msgs.append(dict(type="text", value=question))
+        return msgs
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/sandbox.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/sandbox.py
new file mode 100644
index 0000000000000000000000000000000000000000..660798988b7cb6cfbc99f8f4a6f0b30bfade3453
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/sandbox.py
@@ -0,0 +1,990 @@
+import ast
+import contextlib
+import glob
+import importlib
+import io
+import json
+import math
+import os
+import pickle
+import random
+import re
+import shutil
+import stat
+import string
+import sys
+import textwrap
+import types
+
+import numpy
+import timeout_decorator  # For efficient timeout control
+from PIL import Image, UnidentifiedImageError
+
+# Attempt to import cv2, but make it optional if not strictly needed by
+# all sandbox uses
+try:
+    import cv2
+except ImportError:
+    cv2 = None  # Keep track if cv2 is not available
+    print(
+        "Warning: OpenCV (cv2) not found. cv2-dependent sandboxed code will fail.",
+        file=sys.stderr,
+    )
+
+
+# --- Enhanced Security: Prohibit dangerous system calls ---
+DANGEROUS_PATTERNS = [
+    r"\bsys\.",
+    r"\bsocket\.",
+    r"\bsubprocess\.",
+    r"\bexec\(",
+    r"\beval\(",
+    r"\bcompile\(",
+    r"\b__import__\(",
+    r"\bos\.(remove|unlink|rmdir)\b",
+    r"\bshutil\.rmtree\b",
+    r"\bshutil\.move\b",
+    r"\bos\.(rename|renames)\b",
+]
+
+
+def check_dangerous_code(code_string):
+    """
+    Performs a basic static analysis to detect dangerous patterns in the code.
+    Returns True if dangerous patterns are found, False otherwise.
+    """
+    for pattern in DANGEROUS_PATTERNS:
+        if re.search(pattern, code_string, re.IGNORECASE):
+            return True
+    return False
+
+
+# pandayin: set a time limit to avoid dead loop, e.x., while(1) type loops.
+EXEC_TIME_LIMIT = 10
+# Define the temporary folder for processed images
+# IMPORTANT: Ensure this directory exists and is writable by the script.
+# For local testing, you might want to change this, e.g.:
+TEMP_PROCESSED_IMAGES_DIR = "./temp_processed_images/"
+
+
+class ReadOnlyPath:
+    def __init__(self, path):
+        self.path = path if isinstance(path, str) else None
+        self.original_permissions = None
+
+    def __enter__(self):
+        if self.path and os.path.isfile(self.path):
+            try:
+                self.original_permissions = os.stat(self.path).st_mode
+                read_only_permissions = self.original_permissions & ~(
+                    stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH
+                )
+                if self.original_permissions != read_only_permissions:
+                    os.chmod(self.path, read_only_permissions)
+            except OSError as e:
+                print(
+                    f"Warning: Could not make '{self.path}' read-only: {e}",
+                    file=sys.stderr,
+                )
+                self.original_permissions = None  # 确保我们不会在退出时尝试恢复权限
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        if (
+            self.path
+            and self.original_permissions is not None
+            and os.path.isfile(self.path)
+        ):
+            try:
+                os.chmod(self.path, self.original_permissions)
+            except OSError as e:
+                print(
+                    f"Warning: Could not restore original permissions for '{self.path}': {e}",
+                    file=sys.stderr,
+                )
+
+
+def align_first_line_to_second(code_string: str) -> str:
+    lines = code_string.splitlines()
+
+    first_line_info = None
+    second_line_info = None
+
+    for index, line_content in enumerate(lines):
+        if line_content.strip():
+            if first_line_info is None:
+                first_line_info = {"index": index, "content": line_content}
+            elif second_line_info is None:
+                second_line_info = {"index": index, "content": line_content}
+                break
+
+    if not first_line_info or not second_line_info:
+        return code_string
+
+    first_line_content = first_line_info["content"]
+    second_line_content = second_line_info["content"]
+
+    first_line_indent = " " * (
+        len(first_line_content) - len(first_line_content.lstrip(" "))
+    )
+    second_line_indent = " " * (
+        len(second_line_content) - len(second_line_content.lstrip(" "))
+    )
+
+    if first_line_indent != second_line_indent:
+        original_index = first_line_info["index"]
+        stripped_content = first_line_content.lstrip(" ")
+        lines[original_index] = second_line_indent + stripped_content
+
+    return "\n".join(lines)
+
+
+def get_image_paths(temp_output_dir):
+    extensions = ["jpg", "jpeg", "png", "bmp", "gif", "tiff"]
+    image_paths = []
+
+    for ext in extensions:
+        pattern = os.path.join(temp_output_dir, f"*.{ext}")
+        image_paths.extend(glob.glob(pattern))
+
+    return image_paths
+
+
+VALID_IMAGE_EXTENSIONS = (
+    ".png",
+    ".jpg",
+    ".jpeg",
+    ".gif",
+    ".bmp",
+    ".tiff",
+    ".webp")
+ORIGINAL_OUTPUT_PREFIX = "/mnt/data/temp_processed_images/"
+
+
+class ImagePathTransformer(ast.NodeTransformer):
+    def __init__(self, replacement_path):
+        self.replacement_path = replacement_path
+        self.path_was_replaced = False
+
+    def visit_Assign(self, node):
+        # Process simple assignments: target_variable = "string_literal"
+        if len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
+            target_variable_name = node.targets[0].id
+
+            if target_variable_name == "image_path":  # Target specific variable name
+                current_path_value = None
+                # Extract string value from ast.Constant (Python 3.8+) or
+                # ast.Str (older Python)
+                if isinstance(node.value, ast.Constant) and isinstance(
+                    node.value.value, str
+                ):
+                    current_path_value = node.value.value
+                elif isinstance(node.value, ast.Str):  # For Python < 3.8
+                    current_path_value = node.value.s
+
+                if current_path_value is not None:
+                    is_valid_image_on_disk = False
+                    if (
+                        os.path.exists(current_path_value)
+                        and os.path.isfile(current_path_value)
+                        and any(
+                            current_path_value.lower().endswith(ext)
+                            for ext in VALID_IMAGE_EXTENSIONS
+                        )
+                    ):
+                        is_valid_image_on_disk = True
+
+                    if not is_valid_image_on_disk:
+                        # Replace the value of the AST node
+                        if hasattr(ast, "Constant"):  # Python 3.8+
+                            node.value = ast.Constant(
+                                value=self.replacement_path)
+                        else:  # Python < 3.8
+                            node.value = ast.Str(s=self.replacement_path)
+                        self.path_was_replaced = True
+
+        return self.generic_visit(node)  # Continue traversing
+
+
+MIN_CROP_DIMENSION = 64
+
+# Define a minimum dimension for the crop
+# MIN_CROP_DIMENSION = 64 # Duplicate definition
+
+
+class CropCoordinateTransformer(ast.NodeTransformer):
+    def __init__(self, image_width, image_height,
+                 main_image_variable_name="image"):
+        self.image_width = image_width
+        self.image_height = image_height
+        self.main_image_variable_name = main_image_variable_name  # New parameter
+        self.coordinates_clamped = False
+        # Add specific variable names to look for
+        self.known_coord_var_names = [
+            "crop_box",
+            "bbox",
+            "box",
+            "coordinates",
+            "coords",
+            "crop_coords",
+            "left_crop_coords",
+            "right_crop_coords",
+            "top_crop_coords",
+            "bottom_crop_coords",
+            # Add any other common names you anticipate
+        ]
+
+    def _get_numeric_value(self, node_value):
+        if isinstance(node_value, ast.Constant) and isinstance(
+            node_value.value, (int, float)
+        ):
+            return node_value.value
+        elif isinstance(node_value, ast.Num):  # Python < 3.8
+            return node_value.n
+        return None
+
+    def _create_numeric_node(self, value):
+        int_value = int(round(value))
+        if hasattr(ast, "Constant"):  # Python 3.8+
+            return ast.Constant(value=int_value)
+        else:  # Python < 3.8
+            return ast.Num(n=int_value)
+
+    def _clamp_coordinates(self, v_x1, v_y1, v_x2, v_y2):
+        img_w, img_h = float(self.image_width), float(self.image_height)
+
+        cx1 = min(v_x1, v_x2)
+        cx2 = max(v_x1, v_x2)
+        cy1 = min(v_y1, v_y2)
+        cy2 = max(v_y1, v_y2)
+
+        final_x1 = max(0.0, cx1)
+        final_x1 = min(final_x1, img_w - 1.0 if img_w > 0 else 0.0)
+        final_x2 = max(0.0, cx2)
+        final_x2 = min(final_x2, img_w)
+        if final_x1 >= final_x2 and img_w > 0:
+            final_x1 = max(0.0, final_x2 - 1.0)
+            if final_x2 <= final_x1:
+                final_x2 = min(img_w, final_x1 + 1.0)
+
+        final_y1 = max(0.0, cy1)
+        final_y1 = min(final_y1, img_h - 1.0 if img_h > 0 else 0.0)
+        final_y2 = max(0.0, cy2)
+        final_y2 = min(final_y2, img_h)
+        if final_y1 >= final_y2 and img_h > 0:
+            final_y1 = max(0.0, final_y2 - 1.0)
+            if final_y2 <= final_y1:
+                final_y2 = min(img_h, final_y1 + 1.0)
+
+        current_width = final_x2 - final_x1
+        if img_w > 0 and current_width < MIN_CROP_DIMENSION:
+            if img_w < MIN_CROP_DIMENSION:
+                final_x1 = 0.0
+                final_x2 = img_w
+            else:
+                final_x2 = final_x1 + MIN_CROP_DIMENSION
+                if final_x2 > img_w:
+                    final_x2 = img_w
+                    final_x1 = max(0.0, final_x2 - MIN_CROP_DIMENSION)
+
+        current_height = final_y2 - final_y1
+        if img_h > 0 and current_height < MIN_CROP_DIMENSION:
+            if img_h < MIN_CROP_DIMENSION:
+                final_y1 = 0.0
+                final_y2 = img_h
+            else:
+                final_y2 = final_y1 + MIN_CROP_DIMENSION
+                if final_y2 > img_h:
+                    final_y2 = img_h
+                    final_y1 = max(0.0, final_y2 - MIN_CROP_DIMENSION)
+
+        if img_w > 0 and final_x1 >= final_x2:
+            final_x2 = min(img_w, final_x1 + 1.0)
+        if img_h > 0 and final_y1 >= final_y2:
+            final_y2 = min(img_h, final_y1 + 1.0)
+
+        return [final_x1, final_y1, final_x2, final_y2]
+
+    def visit_Assign(self, node):
+        processed_node_values = False
+
+        # Case 1: Direct unpacking e.g., x1, y1, x2, y2 = (v1, v2, v3, v4)
+        # or var1, var2, var3, var4 = (v1, v2, v3, v4) if we assume any 4-tuple assign is coords.
+        # For safety, let's stick to explicit x1,y1,x2,y2 or similar, or rely
+        # on Case 2 for named tuples.
+        if (
+            len(node.targets) == 1
+            and isinstance(node.targets[0], ast.Tuple)
+            and len(node.targets[0].elts) == 4
+            and all(isinstance(t, ast.Name) for t in node.targets[0].elts)
+            and isinstance(node.value, ast.Tuple)
+            # Optional: check if target names match common patterns like x1,y1,x2,y2
+            # For now, any 4-tuple assignment to 4 variables is considered.
+            and len(node.value.elts) == 4
+        ):
+
+            target_names = [t.id.lower() for t in node.targets[0].elts]
+            # A heuristic: if names like 'x1', 'y1', 'x2', 'y2' or 'left',
+            # 'top', 'right', 'bottom' are used
+            is_likely_coords_unpacking = any(
+                name in target_names
+                for name in ["x1", "y1", "x2", "y2", "left", "top", "right", "bottom"]
+            )
+
+            if (
+                is_likely_coords_unpacking
+            ):  # Process if it seems like coordinate unpacking
+                raw_coords_from_code = [
+                    self._get_numeric_value(v) for v in node.value.elts
+                ]
+
+                if all(c is not None for c in raw_coords_from_code):
+                    # Assume order from the tuple assignment e.g. x1,y1,x2,y2
+                    current_x1, current_y1, current_x2, current_y2 = (
+                        raw_coords_from_code
+                    )
+                    clamped_values = self._clamp_coordinates(
+                        current_x1, current_y1, current_x2, current_y2
+                    )
+
+                    original_rounded = [int(round(c))
+                                        for c in raw_coords_from_code]
+                    final_rounded = [int(round(c)) for c in clamped_values]
+
+                    if final_rounded != original_rounded:
+                        self.coordinates_clamped = True
+                        for i in range(4):
+                            node.value.elts[i] = self._create_numeric_node(
+                                clamped_values[i]
+                            )
+                    processed_node_values = True
+
+        # Case 2: Assignment to a known variable name, e.g., crop_box = (v1, v2, v3, v4)
+        # This will now specifically catch 'left_crop_coords = ...' and 'right_crop_coords = ...'
+        # if they are in self.known_coord_var_names.
+        elif (
+            not processed_node_values
+            and len(node.targets) == 1
+            and isinstance(node.targets[0], ast.Name)
+            # Check against the list
+            and node.targets[0].id in self.known_coord_var_names
+            and isinstance(node.value, ast.Tuple)
+            and len(node.value.elts) == 4
+        ):
+
+            raw_coords_from_code = [
+                self._get_numeric_value(v) for v in node.value.elts]
+
+            if all(c is not None for c in raw_coords_from_code):
+                current_x1, current_y1, current_x2, current_y2 = raw_coords_from_code
+                clamped_values = self._clamp_coordinates(
+                    current_x1, current_y1, current_x2, current_y2
+                )
+
+                original_rounded = [int(round(c))
+                                    for c in raw_coords_from_code]
+                final_rounded = [int(round(c)) for c in clamped_values]
+
+                if final_rounded != original_rounded:
+                    self.coordinates_clamped = True
+                    for i in range(4):
+                        node.value.elts[i] = self._create_numeric_node(
+                            clamped_values[i]
+                        )
+
+        # Case 3: Direct slicing assignment, e.g., cropped_var =
+        # image_var[y1:y2, x1:x2]
+        if (
+            len(node.targets) == 1
+            and isinstance(node.targets[0], ast.Name)  # e.g. sony_crop = ...
+            # ... image[...]
+            and isinstance(node.value, ast.Subscript)
+            and isinstance(node.value.value, ast.Name)  # ... image variable ...
+            and node.value.value.id == self.main_image_variable_name  # ... is the one we care about
+            and isinstance(node.value.slice, ast.Index)  # ... [<slice_tuple>] ...
+            and isinstance(node.value.slice.value, ast.Tuple)  # ... (slice_y, slice_x)
+            and len(node.value.slice.value.elts) == 2
+            and all(isinstance(s, ast.Slice) for s in node.value.slice.value.elts)
+        ):  # both are Slices
+
+            slice_tuple_node = node.value.slice.value
+            y_slice_node = slice_tuple_node.elts[0]  # First slice is y-dim
+            x_slice_node = slice_tuple_node.elts[1]  # Second slice is x-dim
+
+            # Extract numeric values, converting None to 0 or full dimension
+            y1_val = self._get_numeric_value(y_slice_node.lower)
+            if y1_val is None and y_slice_node.lower is None:
+                y1_val = 0.0  # If original was None, treat as 0
+
+            y2_val = self._get_numeric_value(y_slice_node.upper)
+            if y2_val is None and y_slice_node.upper is None:
+                # If original was None, treat as full height
+                y2_val = float(self.image_height)
+
+            x1_val = self._get_numeric_value(x_slice_node.lower)
+            if x1_val is None and x_slice_node.lower is None:
+                x1_val = 0.0
+
+            x2_val = self._get_numeric_value(x_slice_node.upper)
+            if x2_val is None and x_slice_node.upper is None:
+                x2_val = float(self.image_width)
+
+            # If any original value was not a number or None (e.g. a variable
+            # name in slice), skip
+            if not all(isinstance(v, (int, float))
+                       for v in [y1_val, y2_val, x1_val, x2_val]):
+                # print(f"INFO: Skipping slice clamping for '{node.targets[0].id}' due to non-numeric slice parts.")
+                return self.generic_visit(node)
+
+            # Keep this order for comparison
+            # original_coords = [x1_val, y1_val, x2_val, y2_val]
+            clamped_x1, clamped_y1, clamped_x2, clamped_y2 = self._clamp_coordinates(
+                x1_val, y1_val, x2_val, y2_val)
+
+            final_coords_for_ast = [
+                clamped_y1,
+                clamped_y2,
+                clamped_x1,
+                clamped_x2,
+            ]  # y first for slice node update
+            original_coords_for_ast_compare = [y1_val, y2_val, x1_val, x2_val]
+
+            # Round for comparison
+            original_rounded = [int(round(c))
+                                for c in original_coords_for_ast_compare]
+            final_rounded = [int(round(c)) for c in final_coords_for_ast]
+
+            if final_rounded != original_rounded:
+                self.coordinates_clamped = True
+                # Update the AST Slice nodes
+                y_slice_node.lower = self._create_numeric_node(clamped_y1)
+                y_slice_node.upper = self._create_numeric_node(clamped_y2)
+                x_slice_node.lower = self._create_numeric_node(clamped_x1)
+                x_slice_node.upper = self._create_numeric_node(clamped_x2)
+
+        return self.generic_visit(node)  # Important to call for other nodes
+
+
+class OpenCVNamespaceTransformer(ast.NodeTransformer):
+    """
+    Transforms incorrect OpenCV namespace calls (e.g., cv., cv4.) to cv2.
+    """
+
+    def __init__(self, incorrect_prefixes=None, correct_prefix="cv2"):
+        if incorrect_prefixes is None:
+            self.incorrect_prefixes = [
+                "cv",
+                "cv4",
+                "cv3",
+                "CV",
+            ]  # Common incorrect ones
+        else:
+            self.incorrect_prefixes = incorrect_prefixes
+        self.correct_prefix = correct_prefix
+        self.namespace_updated = False
+
+    def visit_Attribute(self, node):
+        """
+        Called for attribute access like object.attribute or object.sub_object.attribute
+        We are interested in cases like:
+        - cv.imread(...) -> node.value is Name(id='cv'), node.attr is 'imread'
+        - cv.some_module.CONSTANT -> node.value is Name(id='cv'), node.attr is 'some_module'
+                                     (this will be visited first)
+                                     then later, cv2.some_module.CONSTANT,
+                                     node.value is Attribute(value=Name(id='cv2'), attr='some_module')
+        """
+        # Check if the base of the attribute access (e.g., 'cv' in 'cv.imread')
+        # is an ast.Name node and matches one of the incorrect prefixes.
+        if (
+            isinstance(node.value, ast.Name)
+            and node.value.id in self.incorrect_prefixes
+        ):
+            # original_prefix = node.value.id
+            # Change the id of the ast.Name node to the correct prefix
+            node.value.id = self.correct_prefix
+            self.namespace_updated = True
+        # Important: continue visiting child nodes.
+        # This handles cases like cv.submodule.function, ensuring 'cv' is
+        # replaced.
+        return self.generic_visit(node)
+
+
+def ensure_temp_dir(temp_output_dir):
+    """Ensures the temporary directory for processed images exists."""
+    os.makedirs(temp_output_dir, exist_ok=True)
+    # Also ensure the directory is writable (basic check)
+    if not os.access(temp_output_dir, os.W_OK):
+        raise PermissionError(
+            f"Temporary directory {temp_output_dir} is not writable.")
+
+
+# (Default) For Linux/Unix system, signal mechanism could be used (high-efficiency).
+# For Windows where signal support is invalid, this function will be
+# executed in a separate process via multi-processing mechanism.
+
+
+@timeout_decorator.timeout(EXEC_TIME_LIMIT, use_signals=True)
+def _sandboxed_execution_target(
+    code_to_execute,
+    input_image_path,
+    temp_output_dir,
+    item_id,
+    previous_execution_context=None,
+):
+    """
+    Target function for sandboxed code execution in a subprocess.
+    Results are put into return_dict.
+    `previous_execution_context` is a dict {'globals': ..., 'locals': ...}
+    """
+    return_dict = {}
+
+    # Prepare a restricted environment for exec()
+    allowed_builtins = {
+        "print": print,
+        "len": len,
+        "str": str,
+        "int": int,
+        "float": float,
+        "list": list,
+        "dict": dict,
+        "tuple": tuple,
+        "range": range,
+        "round": round,
+        "abs": abs,
+        "min": min,
+        "max": max,
+        "sum": sum,
+        "sorted": sorted,
+        "any": any,
+        "all": all,
+        "zip": zip,
+        "map": map,
+        "filter": filter,
+        "True": True,
+        "False": False,
+        "None": None,
+        "isinstance": isinstance,
+        "issubclass": issubclass,
+        "Exception": Exception,
+        "ValueError": ValueError,
+        "TypeError": TypeError,
+        "AttributeError": AttributeError,
+        "IndexError": IndexError,
+        "KeyError": KeyError,
+        "NotImplementedError": NotImplementedError,
+        "enumerate": enumerate,
+        "pow": pow,
+        "divmod": divmod,
+        "bin": bin,
+        "oct": oct,
+        "hex": hex,
+        "complex": complex,
+        "__import__": __import__,
+        "globals": globals,
+        "locals": locals,
+        "open": open,
+        # Modules accessible via __import__ or provided in sandbox_globals
+        "os": os,
+        "shutil": shutil,
+        "itertools": __import__("itertools"),
+        "re": __import__("re"),
+        "time": __import__("time"),
+        "datetime": __import__("datetime"),
+        "math": __import__("math"),
+        "cmath": __import__("cmath"),
+        "collections": __import__("collections"),
+        "json": json,
+        "PIL": __import__("PIL"),
+        "random": random,
+        "UnidentifiedImageError": UnidentifiedImageError,
+    }
+
+    sandbox_globals = {
+        "__builtins__": allowed_builtins,
+        "os": os,
+        "random": random,
+        "string": string,
+        "math": math,
+        "Image": Image,
+        "UnidentifiedImageError": UnidentifiedImageError,
+        "numpy": numpy,
+        "np": numpy,  # TODO: pandayin: Check this? this seems like a duplicate
+        "json": json,  # Make json directly available
+        "re": re,  # Make re directly available
+        # PIL is available via __import__ or Image class
+    }
+
+    if cv2:
+        sandbox_globals["cv2"] = cv2
+
+    sandbox_globals_always = sandbox_globals
+    sandbox_locals = {
+        "image_path": input_image_path,
+        "temp_output_dir": temp_output_dir,
+    }
+
+    if previous_execution_context:
+        # Restore picklable local variables from the previous step
+        sandbox_locals.update(previous_execution_context.get("locals", {}))
+
+        # Re-create imported modules from the previous step
+        imports_to_recreate = previous_execution_context.get("globals", {})
+        for alias, module_name in imports_to_recreate.items():
+            try:
+                # Re-import the module and add it to the current step's locals
+                module_obj = importlib.import_module(module_name)
+                sandbox_locals[alias] = module_obj
+            except ImportError:
+                print(
+                    f"Warning: Could not re-import module '{module_name}' (as '{alias}') from previous step."
+                )
+
+    code_to_execute = align_first_line_to_second(code_to_execute)
+    try:
+        import autopep8
+        dedented_code = textwrap.dedent(code_to_execute).strip()
+        code_to_execute = dedented_code
+        formatted_code = autopep8.fix_code(
+            code_to_execute, options={"aggressive": 2}
+        )
+        # if formatted_code.strip() != code_to_execute.strip():
+        #     print(f"INFO: Attempted to auto-format code for {item_id} using autopep8.")
+        code_to_execute = formatted_code
+    except Exception:
+        pass
+
+    # ... (AST transformations: ImagePathTransformer, CropCoordinateTransformer, OpenCVNamespaceTransformer) ...
+    # (These transformations should operate on code_to_execute before exec)
+    if input_image_path and isinstance(input_image_path, str):
+        try:
+            tree = ast.parse(code_to_execute)
+            transformer = ImagePathTransformer(input_image_path)
+            new_tree = transformer.visit(tree)
+            if transformer.path_was_replaced:
+                if hasattr(ast, "unparse"):
+                    code_to_execute = ast.unparse(new_tree)
+                else:
+                    print(
+                        "Warning: ast.unparse not available (requires Python 3.9+). "
+                        "Code for image_path replacement not updated. Consider installing"
+                        "'astor' for older Python versions or upgrading Python."
+                    )
+        except SyntaxError as e:
+            print(
+                f"Syntax error when parsing code for image_path replacement: {e}")
+        except Exception as e:
+            print(
+                f"An error occurred during image_path AST transformation: {e}")
+
+    actual_image_width, actual_image_height = None, None
+    if input_image_path and os.path.isfile(input_image_path):
+        try:
+            with Image.open(input_image_path) as img_obj:
+                actual_image_width, actual_image_height = img_obj.size
+            if actual_image_width is None or actual_image_height is None:
+                print(
+                    f"WARNING: Could not determine dimensions for '{input_image_path}' using PIL."
+                )
+        except Exception as e:
+            print(
+                f"WARNING: Could not read image dimensions from '{input_image_path}' using PIL: {e}"
+            )
+
+    if actual_image_width is not None and actual_image_height is not None:
+        if actual_image_width == 0 or actual_image_height == 0:
+            print(
+                f"WARNING: Input image '{input_image_path}' has zero width or height"
+                f"({actual_image_width}x{actual_image_height}). Clamping might result in zero-size crops."
+            )
+        try:
+            crop_ast_tree = ast.parse(code_to_execute)
+            crop_transformer = CropCoordinateTransformer(
+                actual_image_width, actual_image_height
+            )
+            new_crop_ast_tree = crop_transformer.visit(crop_ast_tree)
+            if crop_transformer.coordinates_clamped:
+                if hasattr(ast, "unparse"):
+                    code_to_execute = ast.unparse(new_crop_ast_tree)
+                else:
+                    print(
+                        "WARNING: ast.unparse not available (Python 3.9+)."
+                        "Crop coordinate clamping not fully updated in code string."
+                    )
+        except SyntaxError:
+            # print(f"ERROR: Syntax error when parsing code for crop coordinate clamping: {e}")
+            pass
+        except Exception as e:
+            print(
+                f"ERROR: An error occurred during crop coordinate AST transformation: {e}"
+            )
+            pass
+    else:
+        if input_image_path:
+            # print(
+            #     f"INFO: Skipping crop coordinate clamping because image dimensions"
+            #     f"could not be obtained for '{input_image_path}'.")
+            pass
+
+    try:  # Wrap AST parsing for OpenCV transformer in try-except
+        tree = ast.parse(code_to_execute)
+        cv_transformer = OpenCVNamespaceTransformer(correct_prefix="cv2")
+        new_tree = cv_transformer.visit(tree)
+
+        if cv_transformer.namespace_updated:
+            print("OpenCV namespace references were updated.")
+            if hasattr(ast, "unparse"):
+                code_to_execute = ast.unparse(new_tree)
+                print("Code updated using ast.unparse.")
+            else:
+                try:
+                    import astor
+
+                    code_to_execute = astor.to_source(new_tree)
+                    print("Code updated using astor.to_source.")
+                except ImportError:
+                    print(
+                        "WARNING: ast.unparse not available (Python 3.9+), and astor not installed."
+                    )
+                    print(
+                        "The AST was modified, but the code_to_execute string was not updated."
+                    )
+    except SyntaxError as e:
+        print(
+            f"ERROR: Syntax error when parsing code for OpenCV namespace transformation: {e}."
+        )
+    except Exception as e:
+        print(
+            f"ERROR: An error occurred during OpenCV namespace AST transformation: {e}"
+        )
+
+    captured_stdout = io.StringIO()
+    processed_path_from_code = None
+    processed_paths_list = []
+    error_msg = None
+    full_print_output = None
+
+    try:
+        with contextlib.redirect_stdout(captured_stdout):
+            original_prefix = "/mnt/data/temp_processed_images/"
+            replacement_target_path = temp_output_dir
+            if original_prefix.endswith(
+                    "/") and not temp_output_dir.endswith("/"):
+                replacement_target_path = temp_output_dir + "/"
+
+            # Create subdirectories if implied by paths in code
+            quoted_path_pattern_str = (
+                r"(['\"])(" + re.escape(original_prefix) + r"[^'\"]*)\1"
+            )
+            quoted_path_pattern = re.compile(quoted_path_pattern_str)
+            unique_paths_found = set()
+            for match in quoted_path_pattern.finditer(code_to_execute):
+                unique_paths_found.add(match.group(2))
+            for full_path_str in unique_paths_found:
+                if full_path_str.startswith(original_prefix):
+                    # relative_path_suffix = full_path_str[len(original_prefix):]
+                    # print('full_path_str', unique_paths_found)
+                    relative_path_suffix = full_path_str[
+                        len(original_prefix):
+                    ].replace(" ", "")
+                    if relative_path_suffix:
+                        path_directory_part = os.path.dirname(
+                            relative_path_suffix)
+                        if path_directory_part:
+                            target_subdir_to_create = os.path.join(
+                                temp_output_dir, path_directory_part
+                            )
+                            os.makedirs(target_subdir_to_create, exist_ok=True)
+
+            code_to_execute = code_to_execute.replace(
+                original_prefix, replacement_target_path
+            )
+            code_to_execute = re.sub(r":\.\d{1}f}", ":.8f}", code_to_execute)
+            exec(code_to_execute, sandbox_globals, sandbox_locals)
+
+        full_print_output = captured_stdout.getvalue().strip()
+
+        if "processed_path" in sandbox_locals and (
+            not previous_execution_context
+            or "processed_path" not in previous_execution_context.get("locals", {})
+        ):
+            processed_path_from_code = sandbox_locals["processed_path"]
+            if (
+                isinstance(processed_path_from_code, str)
+                and processed_path_from_code.startswith(temp_output_dir)
+                and os.path.isfile(processed_path_from_code)
+            ):
+                processed_paths_list.append(processed_path_from_code)
+            elif (
+                isinstance(processed_path_from_code, str)
+                and processed_path_from_code.startswith(temp_output_dir)
+                and not os.path.exists(processed_path_from_code)
+            ):
+                error_msg = (
+                    f"Sandbox for {item_id}: 'processed_path' variable set to "
+                    f"'{processed_path_from_code}', but file does not exist."
+                )
+                processed_path_from_code = None
+            elif processed_path_from_code is not None:
+                error_msg = (
+                    f"Sandbox for {item_id}: 'processed_path' variable was "
+                    f"'{processed_path_from_code}', which is not a valid file path in {temp_output_dir}."
+                )
+                processed_path_from_code = None
+        # print(full_print_output)
+        if full_print_output:
+            path_search_pattern = rf"({re.escape(temp_output_dir)}[^\s\'\"]+\.(?:jpg|jpeg|png|bmp|gif|tiff))"
+            possible_image_path_list = get_image_paths(temp_output_dir)
+            possible_error_msg = ""
+            num_parse_images = 0
+            for match in re.finditer(path_search_pattern, full_print_output):
+                potential_path_from_print = match.group(1)
+                num_parse_images += 1
+                if os.path.isfile(potential_path_from_print):
+                    if potential_path_from_print not in processed_paths_list:
+                        processed_paths_list.append(potential_path_from_print)
+                elif not error_msg:
+                    possible_error_msg = (
+                        f"Sandbox for {item_id}: Path '{potential_path_from_print}' "
+                        "found in print, but file does not exist or is not a file."
+                    )
+            if len(processed_paths_list) == 0:
+                if num_parse_images == len(possible_image_path_list):
+                    processed_paths_list = possible_image_path_list
+                else:
+                    path_search_pattern = (
+                        r"([^\s\'\"]+\.(?:jpg|jpeg|png|bmp|gif|tiff))"
+                    )
+                    list_of_all_matches = re.findall(
+                        path_search_pattern, full_print_output
+                    )
+                    if len(list_of_all_matches) == len(
+                            possible_image_path_list):
+                        processed_paths_list = possible_image_path_list
+                    else:
+                        error_msg = possible_error_msg
+
+            if not processed_paths_list and temp_output_dir not in code_to_execute:
+                try:
+                    processed_paths_list.append(full_print_output)
+                except ValueError:
+                    pass
+
+    except ImportError as e:
+        error_msg = (
+            f"Sandbox for {item_id}: Code execution failed due to ImportError. "
+            f"Ensure all required modules are available and correctly named: {e}"
+        )
+        if "cv2" in str(e).lower() and not cv2:
+            error_msg += (
+                f"(Note: cv2 was not available in the sandbox host environment): {e}"
+            )
+    except MemoryError as e:
+        error_msg = (
+            f"Sandbox for {item_id}: Code execution failed due to MemoryError. "
+            f"The operation likely consumed too much memory: {e}"
+        )
+    except SyntaxError as e:  # Catch syntax errors from exec itself
+        error_msg = (
+            f"Sandbox for {item_id}: Code execution failed due to SyntaxError: {e}"
+        )
+    except Exception as e:
+        error_msg = f"Sandbox for {item_id}: Code execution failed: {e}"
+
+    return_dict["processed_paths_list"] = processed_paths_list
+    return_dict["print_output"] = full_print_output
+    return_dict["error"] = error_msg
+
+    if full_print_output is not None and not processed_paths_list:
+        error_msg = f"Sandbox for {item_id}: Path/result output error, unable to match save path"
+
+    picklable_variables = {}
+    imports_to_persist = {}
+
+    for name, value in sandbox_locals.items():
+        if name == "__builtins__":
+            continue
+        if name in sandbox_globals_always.keys():
+            continue
+        if isinstance(value, types.ModuleType):
+            imports_to_persist[name] = value.__name__
+        else:
+            try:
+                # pickle.dumps(value)
+                picklable_variables[name] = value
+            except (pickle.PicklingError, TypeError):
+                print(
+                    f"Warning: Var '{name}' of type {type(value).__name__} is not picklable.",
+                    file=sys.stderr,
+                )
+
+    # Store the final state of globals and locals
+    return_dict["execution_context"] = {
+        "globals": imports_to_persist,
+        "locals": picklable_variables,
+    }
+    return return_dict
+
+
+def execute_code_in_sandbox(
+    code_to_execute,
+    input_image_path,
+    item_id="N/A",
+    temp_output_dir=None,
+    previous_execution_context=None,
+):
+    """
+    Executes Python code in a restricted sandbox using timeout_decorator.
+
+    Args:
+        code_to_execute (str): The Python code string to execute.
+        input_image_path (str): Path to the user image.
+        item_id (str): Identifier for logging.
+        temp_output_dir (str): Directory for processed images.
+        previous_execution_context (dict, optional): Context from a previous execution.
+
+    Returns:
+        tuple: (processed_file_paths_list, captured_print_output,
+                error_message_or_none, current_execution_context)
+    """
+    if temp_output_dir is None:
+        temp_output_dir = TEMP_PROCESSED_IMAGES_DIR
+    if check_dangerous_code(code_to_execute):
+        return (
+            [],
+            "",
+            (
+                f"Sandbox for {item_id}: Code contains potentially dangerous system operations "
+                "such as remove. Execution denied.",
+            ),
+            None,
+        )
+
+    ensure_temp_dir(temp_output_dir)
+    with ReadOnlyPath(input_image_path):
+        try:
+            # Directly call the decorated function
+            result_dict = _sandboxed_execution_target(
+                code_to_execute,
+                input_image_path,
+                temp_output_dir,
+                item_id,
+                previous_execution_context,
+            )
+
+            # Unpack results from the returned dictionary
+            processed_paths_list = result_dict.get("processed_paths_list", [])
+            full_print_output = result_dict.get("print_output", "")
+            error_msg = result_dict.get("error", None)
+            current_execution_context = result_dict.get(
+                "execution_context", {"globals": {}, "locals": {}}
+            )
+
+        except timeout_decorator.TimeoutError:
+            # Handle the timeout gracefully
+            error_msg = f"Sandbox for {item_id}: Execution timed out after {EXEC_TIME_LIMIT} seconds."
+            print(error_msg)
+            processed_paths_list = []
+            full_print_output = ""
+            current_execution_context = None
+
+    return processed_paths_list, full_print_output, error_msg, current_execution_context
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/utils.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..81907d84523c742c048071dd61a096fe18ccf908
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/thyme/utils.py
@@ -0,0 +1,65 @@
+from PIL import Image
+
+REASONING_SYS_PROMPT = (
+    "You are a helpful assistant.\n\n"
+    "Solve the following problem step by step, and optionally write Python code "
+    "for image manipulation to enhance your reasoning process. The Python code "
+    "will be executed by an external sandbox, and the processed image or result "
+    "(wrapped in <sandbox_output></sandbox_output>) can be returned to aid your "
+    "reasoning and help you arrive at the final answer.\n\n"
+    "**Reasoning & Image Manipulation (Optional but Encouraged):**\n"
+    "    * You have the capability to write executable Python code to perform "
+    "image manipulations (e.g., cropping to a Region of Interest (ROI), "
+    "resizing, rotation, adjusting contrast) or perform calculation for better "
+    "reasoning.\n"
+    "    * The code will be executed in a secure sandbox, and its output will be "
+    "provided back to you for further analysis.\n"
+    "    * All Python code snippets **must** be wrapped as follows:\n"
+    "    <code>\n"
+    "    ```python\n"
+    "    # your code.\n"
+    "    ```\n"
+    "    </code>\n"
+    "    * At the end of the code, print the path of the processed image "
+    "(processed_path) or the result for further processing in a sandbox "
+    "environment."
+)
+
+
+SIMPLE_SYS_PROMPT = "You are a helpful assistant."
+
+
+def generate_prompt_simple_qa(user_question):
+    # Construct the prompt based on the given requirements
+    prompt = (
+        "You are an advanced AI assistant specializing in visual question "
+        "answering (VQA). You don't need to perform any image manipulation "
+        "or reasoning. Give the answer to the following question directly.\n"
+        f"**User's Question:** \"{user_question}\""
+    )
+    return prompt
+
+
+def generate_prompt_final_qa(user_question, user_image_path):
+    # Construct the prompt based on the given requirements
+    try:
+        with Image.open(user_image_path) as img:
+            user_image_size = f"{img.width}x{img.height}"
+    except Exception:
+        user_image_size = "Unable to determine (error reading image)"
+
+    prompt = f"""<image>
+{user_question}
+
+### User Image Path:** "{user_image_path}"
+### User Image Size:** "{user_image_size}"
+
+### **Output Format (strict adherence required):**
+
+<think>Your detailed reasoning process, including any code, should go here.</think>
+<answer>Your final answer to the user's question goes here.</answer>
+"""
+    return prompt
+
+
+SPECIAL_STRING_LIST = ["</code>", "</answer>"]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..a10f3055584db6145df67bd875502046f7ab6dff
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/__init__.py
@@ -0,0 +1 @@
+from .ursa_chat import UrsaChat
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_chat.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_chat.py
new file mode 100644
index 0000000000000000000000000000000000000000..8d8068063b2d68043b6d0a5ea77093a1eb34d43f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_chat.py
@@ -0,0 +1,169 @@
+import logging
+import re
+from typing import Dict, List, Optional, Union
+
+import torch
+from PIL import Image
+from transformers import AutoProcessor, AutoTokenizer, set_seed
+
+from vlmeval.dataset import DATASET_TYPE
+from ..base import BaseModel
+
+
+class UrsaChat(BaseModel):
+    def __init__(self, model_path: str, **kwargs) -> None:
+        from .ursa_model import UrsaForConditionalGeneration, UrsaProcessor
+        super().__init__()
+        self.model: UrsaForConditionalGeneration = UrsaForConditionalGeneration.from_pretrained(model_path,
+                                                                                                torch_dtype=torch.bfloat16).to('cuda')
+        self.image_processor: UrsaProcessor = UrsaProcessor.from_pretrained(model_path)
+        self.prompts = {
+            'SYSTEM_PROMPT' : 'You are a helpful assistant.',
+            'ORIGINAL_PROMPT' : 'you are given a math problem image, please solve the problem step by step. \nQuestion:',
+            'EXTRACT_PROMPT' : 'you are given a math problem image, please solve the problem step by step. When you get an answer, please return the correspond option instead of the text content.\nQuestion:',
+        }
+
+
+    def use_custom_prompt(self, dataset: str) -> bool:
+        """Whether to use custom prompt for the dataset.
+
+        Args:
+            dataset (str): The name of the dataset.
+
+        Returns:
+            bool: Whether to use custom prompt for the dataset.
+        """
+        if 'VQA' in DATASET_TYPE(dataset):
+            return True
+        return False
+
+
+    def build_prompt(self, line: str, dataset: str) -> List[dict]:
+        """Build prompt for multi-choice dataset.
+
+        Args:
+            line (str): one line of the dataset.
+            dataset (str): The name of the dataset.
+
+        Returns:
+            List[dict]: A list of elements constructed for current line.
+        """
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+        question = line['question']
+        # use only question
+        if dataset in ['MathVista_MINI', 'MathVista', 'MathVision']:
+            question = question[question.find('Question: ') + len('Question: ') : ]
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            question += f'\n{key}. {item}'
+        prompt = question
+        if dataset in ['DynaMath']:
+            prompt = '{}\n{}'.format(self.prompts['EXTRACT_PROMPT'], prompt)
+        else:
+            prompt = '{}\n{}'.format(self.prompts['ORIGINAL_PROMPT'], prompt)
+        message = [dict(type='image', value=s) for s in tgt_path]
+        message.append(dict(type='text', value=prompt))
+        return message
+
+
+    def extract_answer(self, text:str) -> str:
+        """Extract final answer which places after '†Answer:'
+
+        Args:
+            text (str): original response
+
+        Returns:
+            str: the extracted answer
+        """
+        patterns = [
+            r'Answer: \\boxed\{(.*?)\}',
+            r'\\boxed\{(.*?)\}',
+            r'†Answer:(.*?)(?:\n|$)',  # Match †Answer: to the next newline or end of string
+            r'†Answer: (.*?)(?:\n|$)',  # Match †Answer: to the next newline or end of string
+            r'†Answer: \n(.*?)(?:\n|$)',  # Match †Answer: to the next newline or end of string
+            r'correct answer：(.*?)(?:\n|$)',  # Match correct answer：to the next newline or end of string
+            r'Correct answer：(.*?)(?:\n|$)',  # Match Correct answer：to the next newline or end of string
+            r'Answer：(.*?)(?:\n|$)',
+            r"correct answer is:(.*?)(?:\n|$)",
+            r"correct answer is:\n(.*?)(?:\n|$)",
+            r"correct answer is:\n\n(.*?)(?:\n|$)",
+            r"correct answer is:\n\n\n(.*?)(?:\n|$)",
+            r'correct answer:(.*?)(?:\n|$)',  # Match correct answer：to the next newline or end of string
+            r'Correct answer:(.*?)(?:\n|$)',  # Match Correct answer：to the next newline or end of string
+            r'correct answer is(.*?)(?:\n|$)',  # Match correct answer is to the next newline or end of string
+            r'Answer:(.*?)(?:\n|$)',
+            r'†(.*?)(?:\n|$)',  # Match † to the next newline or end of string
+            r'Answer: (.*?)(?:\n|$)',
+            r'The answer is (.*?)(?:\n|$)'
+        ]
+
+        res_list = []
+
+        for i, pattern in enumerate(patterns):
+            matches = re.findall(pattern, text, re.IGNORECASE)
+            if matches:
+                if i == 0:  # Only concatenate results for the first pattern
+                    for match in matches:
+                        res = match.strip()
+                        if len(res) == 1 and (not res.isalpha()) and (not res.isdigit()):
+                            continue
+                        if len(res) == 0:
+                            continue
+                        res_list.append(res)
+                    if res_list:
+                        return '\n'.join(res_list)
+                else:
+                    for match in matches:
+                        res = match.strip()
+                        if len(res) == 1 and (not res.isalpha()) and (not res.isdigit()):
+                            continue
+                        if len(res) == 0:
+                            continue
+                        return res
+
+        return text
+
+
+    def generate_inner(self, message: List[dict], dataset: str=None) -> str:
+        """Generate response for the given message.
+
+        Args:
+            message (List[dict]): A list of elements constructed for
+                current line.
+            dataset (str): The name of the dataset.
+
+        Returns:
+            str: The generated response.
+        """
+        prompt, image_path = self.message_to_promptimg(message)
+        conv = [{"role": "system", "content": self.prompts['SYSTEM_PROMPT']}]
+        conv.append({
+            "role": "user",
+            "content": "<|image|>" + prompt
+        })
+        prompt = self.image_processor.apply_chat_template(conv, add_generation_prompt=True)
+        if isinstance(image_path, str):
+            raw_image = Image.open(image_path).convert('RGB')
+        elif isinstance(image_path, Image.Image):
+            raw_image = image_path.convert('RGB')
+        else:
+            raw_image = None
+        raw_image = [raw_image]
+        inputs = self.image_processor(prompt, raw_image, return_tensors='pt').to('cuda', torch.bfloat16)
+        generation_config = dict(
+            max_new_tokens=2048,
+            do_sample=False,
+            temperature=0.,
+            pad_token_id=self.image_processor.tokenizer.eos_token_id,
+        )
+        outputs = self.model.generate(**inputs, **generation_config)
+        response = self.image_processor.decode(outputs[0], skip_special_tokens=True).split('assistant\n')[-1]
+        if dataset in ['MathVista_MINI', 'MathVista', 'DynaMath', 'MathVision', 'LogicVista', 'WeMath', 'MathVerse_MINI_Vision_Only', 'MathVerse']:
+            response = 'Answer: {}'.format(self.extract_answer(response))
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..c7e5da1080fd49a8c3bb55ff009b4af88da8e94c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/__init__.py
@@ -0,0 +1,30 @@
+# Copyright (2025) Bytedance Ltd. and/or its affiliates
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .configuration_ursa import AlignerConfig, UrsaConfig, VisionConfig
+from .image_processing_vlm import VLMImageProcessor, VLMImageProcessorConfig
+from .modeling_ursa import UrsaForConditionalGeneration, UrsaForTokenClassification
+from .processing_ursa import UrsaProcessor
+from .projector import MlpProjector
+
+__all__ = [
+    "VLMImageProcessor",
+    "UrsaProcessor",
+    "UrsaForConditionalGeneration",
+    "UrsaForTokenClassification",
+    "VLMImageProcessorConfig",
+    "VisionConfig",
+    "MlpProjector",
+    "AlignerConfig",
+    "UrsaConfig"
+]
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/clip_encoder.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/clip_encoder.py
new file mode 100644
index 0000000000000000000000000000000000000000..8695701a81afb597d91da5e5540041a41e9f4214
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/clip_encoder.py
@@ -0,0 +1,242 @@
+# Copyright (c) 2023-2024 DeepSeek.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of
+# this software and associated documentation files (the "Software"), to deal in
+# the Software without restriction, including without limitation the rights to
+# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+# the Software, and to permit persons to whom the Software is furnished to do so,
+# subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+from typing import Dict, List, Literal, Optional, Tuple, Union
+
+import torch
+import torch.nn as nn
+import torchvision.transforms
+from einops import rearrange
+
+from .sam import create_sam_vit
+from .siglip_vit import create_siglip_vit
+
+
+class CLIPVisionTower(nn.Module):
+    def __init__(
+        self,
+        model_name: str = "siglip_large_patch16_384",
+        image_size: Union[Tuple[int, int], int] = 336,
+        select_feature: str = "patch",
+        select_layer: int = -2,
+        select_layers: list = None,
+        ckpt_path: str = "",
+        pixel_mean: Optional[List[float]] = None,
+        pixel_std: Optional[List[float]] = None,
+        **kwargs,
+    ):
+        super().__init__()
+
+        self.model_name = model_name
+        self.select_feature = select_feature
+        self.select_layer = select_layer
+        self.select_layers = select_layers
+
+        vision_tower_params = {
+            "model_name": model_name,
+            "image_size": image_size,
+            "ckpt_path": ckpt_path,
+            "select_layer": select_layer,
+        }
+        vision_tower_params.update(kwargs)
+        self.vision_tower, self.forward_kwargs = self.build_vision_tower(
+            vision_tower_params
+        )
+
+        if pixel_mean is not None and pixel_std is not None:
+            image_norm = torchvision.transforms.Normalize(
+                mean=pixel_mean, std=pixel_std
+            )
+        else:
+            image_norm = None
+
+        self.image_norm = image_norm
+
+    def build_vision_tower(self, vision_tower_params):
+        if self.model_name.startswith("siglip"):
+            self.select_feature = "same"
+            vision_tower = create_siglip_vit(**vision_tower_params)
+            forward_kwargs = dict()
+
+        elif self.model_name.startswith("sam"):
+            vision_tower = create_sam_vit(**vision_tower_params)
+            forward_kwargs = dict()
+
+        else:  # huggingface
+            from transformers import CLIPVisionModel
+
+            vision_tower = CLIPVisionModel.from_pretrained(**vision_tower_params)
+            forward_kwargs = dict(output_hidden_states=True)
+
+        return vision_tower, forward_kwargs
+
+    def feature_select(self, image_forward_outs):
+        if isinstance(image_forward_outs, torch.Tensor):
+            # the output has been the self.select_layer"s features
+            image_features = image_forward_outs
+        else:
+            image_features = image_forward_outs.hidden_states[self.select_layer]
+
+        if self.select_feature == "patch":
+            # if the output has cls_token
+            image_features = image_features[:, 1:]
+        elif self.select_feature == "cls_patch":
+            image_features = image_features
+        elif self.select_feature == "same":
+            image_features = image_features
+
+        else:
+            raise ValueError(f"Unexpected select feature: {self.select_feature}")
+        return image_features
+
+    def forward(self, images):
+        """
+
+        Args:
+            images (torch.Tensor): [b, 3, H, W]
+
+        Returns:
+            image_features (torch.Tensor): [b, n_patch, d]
+        """
+
+        if self.image_norm is not None:
+            images = self.image_norm(images)
+
+        image_forward_outs = self.vision_tower(images, **self.forward_kwargs)
+        image_features = self.feature_select(image_forward_outs)
+        return image_features
+
+
+class HybridVisionTower(nn.Module):
+    def __init__(
+        self,
+        high_res_cfg: Dict,
+        low_res_cfg: Dict,
+        freeze_high: bool = False,
+        freeze_low: bool = False,
+        concat_type: Literal["feature", "sequence", "add", "tuple"] = "tuple",
+        **ignore_kwargs,
+    ):
+        super().__init__()
+
+        self.vision_tower_high = CLIPVisionTower(**high_res_cfg)
+        self.vision_tower_low = CLIPVisionTower(**low_res_cfg)
+        self.low_res_size = low_res_cfg["image_size"]
+        self.concat_type = concat_type
+
+        self.high_layer_norm = nn.LayerNorm(high_res_cfg.get("output_dim", 1024))
+        self.low_layer_norm = nn.LayerNorm(low_res_cfg.get("output_dim", 1024))
+
+        if freeze_high:
+            for p_name, p in self.vision_tower_high.named_parameters():
+                p.requires_grad = False
+            self.vision_tower_high = self.vision_tower_high.eval()
+        else:
+            # train donwsamples and neck
+            for p_name, p in self.vision_tower_high.named_parameters():
+                if "downsamples" in p_name or "neck" in p_name:
+                    p.requires_grad = True
+                else:
+                    p.requires_grad = False
+
+        if freeze_low:
+            for p in self.vision_tower_low.parameters():
+                p.requires_grad = False
+            self.vision_tower_low = self.vision_tower_low.eval()
+
+        self.resize = torchvision.transforms.Resize(self.low_res_size, antialias=True)
+
+    def forward(self, images: torch.Tensor):
+        """
+
+        Args:
+            images (torch.Tensor): [bs, 3, H, W]
+
+        Returns:
+            res (torch.Tensor): [bs, t, c]
+        """
+
+        # [bs, c, h, w]
+        high_images = images
+
+        # [bs, c, h_low, w_low]
+        low_images = self.resize(images)
+
+        # separately run two vision towers
+        # run high_res vision tower
+        high_res = self.vision_tower_high(high_images)
+        # [bs, c, h, w] -> [bs, h*w, c]
+        high_res = rearrange(high_res, "b c h w -> b (h w) c")
+        # run low_res vision tower
+        low_res = self.vision_tower_low(low_images)
+
+        if self.concat_type == "feature":
+            images_features = torch.cat([high_res, low_res], dim=-1)
+        elif self.concat_type == "sequence":
+            images_features = torch.cat([high_res, low_res], dim=1)
+        elif self.concat_type == "add":
+            images_features = high_res + low_res
+        elif self.concat_type == "tuple":
+            images_features = (high_res, low_res)
+
+        else:
+            raise ValueError(
+                "Currently only support `feature`, `sequence`, `add` and `tuple` concat type."
+            )
+
+        return images_features
+
+
+if __name__ == "__main__":
+    image_size = 1024
+    x = torch.zeros(2, 3, image_size, image_size).bfloat16().cuda()
+
+    high_res_cfg = dict(
+        model_name="sam_b_downsample",
+        select_feature="same",
+        image_size=image_size,
+        pixel_mean=(0.48145466, 0.4578275, 0.40821073),
+        pixel_std=(0.26862954, 0.26130258, 0.27577711),
+        select_layer=-1,
+        ckpt_path="",
+    )
+
+    low_res_cfg = dict(
+        model_name="siglip_large_patch16_384",
+        select_feature="same",
+        image_size=384,
+        pixel_mean=(0.5, 0.5, 0.5),
+        pixel_std=(0.5, 0.5, 0.5),
+        select_layer=-1,
+        ckpt_path="",
+    )
+
+    net = (
+        HybridVisionTower(
+            high_res_cfg=high_res_cfg,
+            low_res_cfg=low_res_cfg,
+            freeze_high=True,
+            freeze_low=True,
+            concat_type="tuple",
+        )
+        .bfloat16()
+        .cuda()
+    )
+    high_x, low_x = net(x)
+    print(x.shape, high_x.shape, low_x.shape)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/configuration_ursa.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/configuration_ursa.py
new file mode 100644
index 0000000000000000000000000000000000000000..6d4b860e0f76a8175b6d44afb28930ded897f37d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/configuration_ursa.py
@@ -0,0 +1,145 @@
+# Copyright (2025) Bytedance Ltd. and/or its affiliates
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+
+if sys.version_info >= (3, 10):
+    print("Python version is above 3.10, patching the collections module.")
+    import collections
+    import collections.abc
+
+    for type_name in collections.abc.__all__:
+        setattr(collections, type_name, getattr(collections.abc, type_name))
+
+from attrdict import AttrDict
+from transformers import CONFIG_MAPPING
+from transformers.configuration_utils import PretrainedConfig
+from transformers.utils import logging
+
+logger = logging.get_logger(__name__)
+
+
+class VisionConfig(PretrainedConfig):
+    model_type = "vision"
+    cls: str = ""
+    params: AttrDict = {}
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+        self.cls = kwargs.get("cls", "")
+        if not isinstance(self.cls, str):
+            self.cls = self.cls.__name__
+
+        self.params = AttrDict(kwargs.get("params", {}))
+
+
+class AlignerConfig(PretrainedConfig):
+    model_type = "aligner"
+    cls: str = ""
+    params: AttrDict = {}
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+        self.cls = kwargs.get("cls", "")
+        if not isinstance(self.cls, str):
+            self.cls = self.cls.__name__
+
+        self.params = AttrDict(kwargs.get("params", {}))
+
+class UrsaConfig(PretrainedConfig):
+    model_type = "ursa"
+    is_composition = False
+    vision_config: VisionConfig
+    aligner_config: AlignerConfig
+    text_config: PretrainedConfig
+
+    def __init__(
+        self,
+        vision_config=None,
+        aligner_config=None,
+        text_config=None,
+        ignore_index=-100,
+        image_token_index=32000,
+        projector_hidden_act="gelu",
+        vision_feature_select_strategy="default",
+        vision_feature_layer=-2,
+        **kwargs,
+    ):
+        self.ignore_index = ignore_index
+        self.image_token_index = image_token_index
+        self.projector_hidden_act = projector_hidden_act
+
+        if vision_feature_select_strategy not in ["default", "full"]:
+            raise ValueError(
+                "vision_feature_select_strategy should be one of 'default', 'full'."
+                f"Got: {vision_feature_select_strategy}"
+            )
+
+        self.vision_feature_select_strategy = vision_feature_select_strategy
+        self.vision_feature_layer = vision_feature_layer
+
+        if vision_config is None:
+            vision_config = VisionConfig()
+            vision_config.cls = "HybridVisionTower"
+            vision_config.params = {
+                                        "concat_type": "tuple",
+                                        "high_res_cfg": {
+                                            "ckpt_path": "",
+                                            "image_size": 1024,
+                                            "model_name": "sam_b_downsample",
+                                            "output_dim": 1024,
+                                            "pixel_mean": [
+                                            0.48145466,
+                                            0.4578275,
+                                            0.40821073
+                                            ],
+                                            "pixel_std": [
+                                            0.26862954,
+                                            0.26130258,
+                                            0.27577711
+                                            ],
+                                            "select_feature": "same",
+                                            "select_layer": -1
+                                        },
+                                        "low_res_cfg": {
+                                            "ckpt_path": "",
+                                            "image_size": 384,
+                                            "model_name": "siglip_large_patch16_384",
+                                            "output_dim": 1024,
+                                            "pixel_mean": [
+                                            0.5,
+                                            0.5,
+                                            0.5
+                                            ],
+                                            "pixel_std": [
+                                            0.5,
+                                            0.5,
+                                            0.5
+                                            ],
+                                            "select_feature": "same",
+                                            "select_layer": -1
+                                        }
+                                    }
+        self.vision_config = vision_config
+        self.aligner_config = aligner_config
+        if isinstance(text_config, dict):
+            text_config["model_type"] = text_config["model_type"] if "model_type" in text_config else "llama"
+            text_config = CONFIG_MAPPING[text_config["model_type"]](**text_config)
+        elif text_config is None:
+            text_config = CONFIG_MAPPING["llama"]()
+
+        self.text_config = text_config
+
+        super().__init__(**kwargs)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/image_processing_vlm.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/image_processing_vlm.py
new file mode 100644
index 0000000000000000000000000000000000000000..367dee106453669e94f91cd364228fbbfa38e8bf
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/image_processing_vlm.py
@@ -0,0 +1,208 @@
+# Copyright (c) 2023-2024 DeepSeek.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of
+# this software and associated documentation files (the "Software"), to deal in
+# the Software without restriction, including without limitation the rights to
+# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+# the Software, and to permit persons to whom the Software is furnished to do so,
+# subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+from typing import List, Tuple, Union
+
+import numpy as np
+import torch
+import torchvision
+import torchvision.transforms.functional
+from PIL import Image
+from transformers import AutoImageProcessor, PretrainedConfig
+from transformers.image_processing_utils import BaseImageProcessor, BatchFeature
+from transformers.image_utils import to_numpy_array
+from transformers.utils import logging
+
+logger = logging.get_logger(__name__)
+
+ImageType = Union[np.ndarray, torch.Tensor, Image.Image]
+IMAGENET_MEAN = (0.48145466, 0.4578275, 0.40821073)
+IMAGENET_STD = (0.26862954, 0.26130258, 0.27577711)
+IMAGENET_INCEPTION_MEAN = (0.5, 0.5, 0.5)
+IMAGENET_INCEPTION_STD = (0.5, 0.5, 0.5)
+
+
+def expand2square(pil_img, background_color):
+    width, height = pil_img.size
+    if width == height:
+        return pil_img
+    elif width > height:
+        result = Image.new(pil_img.mode, (width, width), background_color)
+        result.paste(pil_img, (0, (width - height) // 2))
+        return result
+    else:
+        result = Image.new(pil_img.mode, (height, height), background_color)
+        result.paste(pil_img, ((height - width) // 2, 0))
+        return result
+
+
+class VLMImageProcessorConfig(PretrainedConfig):
+    model_type = "deepseek_vlm"
+    image_size: int
+    min_size: int
+    image_mean: Union[Tuple[float, float, float], List[float]]
+    image_std: Union[Tuple[float, float, float], List[float]]
+    rescale_factor: float
+    do_normalize: bool
+
+    def __init__(
+        self,
+        image_size: int,
+        min_size: int = 14,
+        image_mean: Union[Tuple[float, float, float], List[float]] = (
+            0.48145466,
+            0.4578275,
+            0.40821073,
+        ),
+        image_std: Union[Tuple[float, float, float], List[float]] = (
+            0.26862954,
+            0.26130258,
+            0.27577711,
+        ),
+        rescale_factor: float = 1.0 / 255.0,
+        do_normalize: bool = True,
+        **kwargs,
+    ):
+        self.image_size = image_size
+        self.min_size = min_size
+        self.image_mean = image_mean
+        self.image_std = image_std
+        self.rescale_factor = rescale_factor
+        self.do_normalize = do_normalize
+
+        super().__init__(**kwargs)
+
+
+class VLMImageProcessor(BaseImageProcessor):
+    model_input_names = ["pixel_values"]
+
+    def __init__(
+        self,
+        image_size: int,
+        min_size: int = 14,
+        image_mean: Union[Tuple[float, float, float], List[float]] = (
+            0.48145466,
+            0.4578275,
+            0.40821073,
+        ),
+        image_std: Union[Tuple[float, float, float], List[float]] = (
+            0.26862954,
+            0.26130258,
+            0.27577711,
+        ),
+        rescale_factor: float = 1.0 / 255.0,
+        do_normalize: bool = True,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+
+        self.image_size = image_size
+        self.rescale_factor = rescale_factor
+        self.image_mean = image_mean
+        self.image_std = image_std
+        self.min_size = min_size
+        self.do_normalize = do_normalize
+
+        if image_mean is None:
+            self.background_color = (127, 127, 127)
+        else:
+            self.background_color = tuple([int(x * 255) for x in image_mean])
+
+    def resize(self, pil_img: Image) -> np.ndarray:
+        """
+
+        Args:
+            pil_img (PIL.Image): [H, W, 3] in PIL.Image in RGB
+
+        Returns:
+            x (np.ndarray): [3, self.image_size, self.image_size]
+        """
+
+        width, height = pil_img.size
+        max_size = max(width, height)
+
+        size = [
+            max(int(height / max_size * self.image_size), self.min_size),
+            max(int(width / max_size * self.image_size), self.min_size),
+        ]
+
+        if width <= 0 or height <= 0 or size[0] <= 0 or size[1] <= 0:
+            print(f"orig size = {pil_img.size}, new size = {size}")
+            raise ValueError("Invalid size!")
+
+        pil_img = torchvision.transforms.functional.resize(
+            pil_img,
+            size,
+            interpolation=torchvision.transforms.functional.InterpolationMode.BICUBIC,
+            antialias=True,
+        )
+
+        pil_img = expand2square(pil_img, self.background_color)
+        x = to_numpy_array(pil_img)
+
+        # [H, W, 3] -> [3, H, W]
+        x = np.transpose(x, (2, 0, 1))
+
+        return x
+
+    def preprocess(self, images, return_tensors: str = "pt", **kwargs) -> BatchFeature:
+        # resize and pad to [self.image_size, self.image_size]
+        # then convert from [H, W, 3] to [3, H, W]
+        images: List[np.ndarray] = [self.resize(image) for image in images]
+
+        # resacle from [0, 255] -> [0, 1]
+        images = [
+            self.rescale(
+                image=image,
+                scale=self.rescale_factor,
+                input_data_format="channels_first",
+            )
+            for image in images
+        ]
+
+        # normalize
+        if self.do_normalize:
+            images = [
+                self.normalize(
+                    image=image,
+                    mean=self.image_mean,
+                    std=self.image_std,
+                    input_data_format="channels_first",
+                )
+                for image in images
+            ]
+
+        data = {"pixel_values": images}
+        return BatchFeature(data=data, tensor_type=return_tensors)
+
+    @property
+    def default_shape(self):
+        return [3, self.image_size, self.image_size]
+
+
+AutoImageProcessor.register(VLMImageProcessorConfig, VLMImageProcessor)
+
+
+if __name__ == "__main__":
+    image_processor = VLMImageProcessor(
+        image_size=1024,
+        image_mean=IMAGENET_INCEPTION_MEAN,
+        image_std=IMAGENET_INCEPTION_STD,
+        do_normalize=True,
+    )
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/modeling_ursa.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/modeling_ursa.py
new file mode 100644
index 0000000000000000000000000000000000000000..cd71604636fd3a89b1b2a9f840a6b3d6a4a42510
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/modeling_ursa.py
@@ -0,0 +1,704 @@
+# Copyright (2025) Bytedance Ltd. and/or its affiliates
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from dataclasses import dataclass
+from typing import List, Optional, Tuple, Union
+
+import torch
+import torch.utils.checkpoint
+from torch import nn
+from transformers import AutoModel, AutoModelForCausalLM, PreTrainedModel
+from transformers.activations import ACT2FN
+from transformers.cache_utils import Cache
+from transformers.modeling_outputs import ModelOutput
+
+from .clip_encoder import CLIPVisionTower, HybridVisionTower
+from .configuration_ursa import AlignerConfig, UrsaConfig, VisionConfig
+from .projector import MlpProjector
+
+
+@dataclass
+class UrsaCausalLMOutputWithPast(ModelOutput):
+    loss: Optional[torch.FloatTensor] = None
+    logits: torch.FloatTensor = None
+    past_key_values: Optional[List[torch.FloatTensor]] = None
+    hidden_states: Optional[Tuple[torch.FloatTensor]] = None
+    attentions: Optional[Tuple[torch.FloatTensor]] = None
+    image_hidden_states: Optional[Tuple[torch.FloatTensor]] = None
+    labels: Optional[Tuple[torch.FloatTensor]] = None
+
+
+class UrsaPreTrainedModel(PreTrainedModel):
+    config_class = UrsaConfig
+    base_model_prefix = "model"
+    supports_gradient_checkpointing = True
+    _no_split_modules = ["qwen2vlmVisionAttention"]
+    _skip_keys_device_placement = "past_key_values"
+    _supports_flash_attn_2 = True
+    _supports_cache_class = True
+
+    def _init_weights(self, module):
+        std = (
+            self.config.initializer_range
+            if hasattr(self.config, "initializer_range")
+            else self.config.text_config.initializer_range
+        )
+        if hasattr(module, "class_embedding"):
+            module.class_embedding.data.normal_(mean=0.0, std=std)
+        if isinstance(module, (nn.Linear, nn.Conv2d)):
+            module.weight.data.normal_(mean=0.0, std=std)
+            if module.bias is not None:
+                module.bias.data.zero_()
+        elif isinstance(module, nn.Embedding):
+            module.weight.data.normal_(mean=0.0, std=std)
+            if module.padding_idx is not None:
+                module.weight.data[module.padding_idx].zero_()
+
+    @property
+    def _supports_sdpa(self):
+        """
+        Retrieve language_model's attribute to check whether the model supports
+        SDPA or not.
+        """
+        return self.language_model._supports_sdpa
+
+
+class UrsaForConditionalGeneration(UrsaPreTrainedModel):
+    def __init__(self, config: UrsaConfig):
+        super().__init__(config)
+        # print(config)
+        # print(type(config.vision_config))
+        # print(type(config.aligner_config))
+        self.vision_model = HybridVisionTower(**config.vision_config["params"])
+        # print(config.aligner_config)
+
+        # print(config.aligner_config.params)
+        aligner_config = AlignerConfig(**config.aligner_config)
+        self.aligner = MlpProjector(aligner_config.params)
+        self.vocab_size = config.text_config.vocab_size
+        self.language_model = AutoModelForCausalLM.from_config(
+            config.text_config, attn_implementation=config._attn_implementation
+        )
+        self.pad_token_id = self.config.pad_token_id if self.config.pad_token_id is not None else -1
+        self.post_init()
+
+    def get_input_embeddings(self):
+        return self.language_model.get_input_embeddings()
+
+    def set_input_embeddings(self, value):
+        self.language_model.set_input_embeddings(value)
+
+    def get_output_embeddings(self):
+        return self.language_model.get_output_embeddings()
+
+    def set_output_embeddings(self, new_embeddings):
+        self.language_model.set_output_embeddings(new_embeddings)
+
+    def set_decoder(self, decoder):
+        self.language_model.set_decoder(decoder)
+
+    def get_decoder(self):
+        return self.language_model.get_decoder()
+
+    def tie_weights(self):
+        return self.language_model.tie_weights()
+
+    def resize_token_embeddings(self, new_num_tokens: Optional[int] = None, pad_to_multiple_of=None) -> nn.Embedding:
+        model_embeds = self.language_model.resize_token_embeddings(new_num_tokens, pad_to_multiple_of)
+        # update vocab size
+        self.config.text_config.vocab_size = model_embeds.num_embeddings
+        self.vocab_size = model_embeds.num_embeddings
+        return model_embeds
+
+    def _merge_input_ids_with_image_features(self, image_features, inputs_embeds, input_ids, attention_mask, labels):
+        num_images, num_image_patches, embed_dim = image_features.shape
+        batch_size, sequence_length = input_ids.shape
+        left_padding = not torch.sum(input_ids[:, -1] == torch.tensor(self.pad_token_id))
+        # 1. Create a mask to know where special image tokens are
+        special_image_token_mask = input_ids == self.config.image_token_index
+        num_special_image_tokens = torch.sum(special_image_token_mask, dim=-1)
+        # Compute the maximum embed dimension
+        max_embed_dim = (num_special_image_tokens.max() * (num_image_patches - 1)) + sequence_length
+        batch_indices, non_image_indices = torch.where(input_ids != self.config.image_token_index)
+
+        # 2. Compute the positions where text should be written
+        # Calculate new positions for text tokens in merged image-text sequence.
+        # `special_image_token_mask` identifies image tokens. Each image token will be replaced by `nb_text_tokens_per_images - 1` text tokens.
+        # `torch.cumsum` computes how each image token shifts subsequent text token positions.
+        # - 1 to adjust for zero-based indexing, as `cumsum` inherently increases indices by one.
+        new_token_positions = torch.cumsum((special_image_token_mask * (num_image_patches - 1) + 1), -1) - 1
+        nb_image_pad = max_embed_dim - 1 - new_token_positions[:, -1]
+        if left_padding:
+            new_token_positions += nb_image_pad[:, None]  # offset for left padding
+        text_to_overwrite = new_token_positions[batch_indices, non_image_indices]
+
+        # 3. Create the full embedding, already padded to the maximum position
+        final_embedding = torch.zeros(
+            batch_size, max_embed_dim, embed_dim, dtype=inputs_embeds.dtype, device=inputs_embeds.device
+        )
+        final_attention_mask = torch.zeros(
+            batch_size, max_embed_dim, dtype=attention_mask.dtype, device=inputs_embeds.device
+        )
+        if labels is not None:
+            final_labels = torch.full(
+                (batch_size, max_embed_dim), self.config.ignore_index, dtype=input_ids.dtype, device=input_ids.device
+            )
+        # In case the Vision model or the Language model has been offloaded to CPU, we need to manually
+        # set the corresponding tensors into their correct target device.
+        target_device = inputs_embeds.device
+        batch_indices, non_image_indices, text_to_overwrite = (
+            batch_indices.to(target_device),
+            non_image_indices.to(target_device),
+            text_to_overwrite.to(target_device),
+        )
+        attention_mask = attention_mask.to(target_device)
+
+        # 4. Fill the embeddings based on the mask. If we have ["hey" "<image>", "how", "are"]
+        # we need to index copy on [0, 577, 578, 579] for the text and [1:576] for the image features
+        final_embedding[batch_indices, text_to_overwrite] = inputs_embeds[batch_indices, non_image_indices]
+        final_attention_mask[batch_indices, text_to_overwrite] = attention_mask[batch_indices, non_image_indices]
+        if labels is not None:
+            final_labels[batch_indices, text_to_overwrite] = labels[batch_indices, non_image_indices]
+
+        # 5. Fill the embeddings corresponding to the images. Anything that is not `text_positions` needs filling (#29835)
+        image_to_overwrite = torch.full(
+            (batch_size, max_embed_dim), True, dtype=torch.bool, device=inputs_embeds.device
+        )
+        image_to_overwrite[batch_indices, text_to_overwrite] = False
+        image_to_overwrite &= image_to_overwrite.cumsum(-1) - 1 >= nb_image_pad[:, None].to(target_device)
+
+        if image_to_overwrite.sum() != image_features.shape[:-1].numel():
+            raise ValueError(
+                f"The input provided to the model are wrong. The number of image tokens is {torch.sum(special_image_token_mask)} while"
+                f" the number of image given to the model is {num_images}. This prevents correct indexing and breaks batch generation."
+            )
+
+        final_embedding[image_to_overwrite] = image_features.contiguous().reshape(-1, embed_dim).to(target_device)
+        final_attention_mask |= image_to_overwrite
+        position_ids = (final_attention_mask.cumsum(-1) - 1).masked_fill_((final_attention_mask == 0), 1)
+
+        # 6. Mask out the embedding at padding positions, as we later use the past_key_value value to determine the non-attended tokens.
+        batch_indices, pad_indices = torch.where(input_ids == self.pad_token_id)
+        indices_to_mask = new_token_positions[batch_indices, pad_indices]
+
+        final_embedding[batch_indices, indices_to_mask] = 0
+
+        if labels is None:
+            final_labels = None
+
+        return final_embedding, final_attention_mask, final_labels, position_ids
+
+    def forward(
+        self,
+        input_ids: torch.LongTensor = None,
+        pixel_values: torch.FloatTensor = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[List[torch.FloatTensor]] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        vision_feature_layer: Optional[int] = None,
+        vision_feature_select_strategy: Optional[str] = None,
+        labels: Optional[torch.LongTensor] = None,
+        use_cache: Optional[bool] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+    ) -> Union[Tuple, UrsaCausalLMOutputWithPast]:
+        output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
+        output_hidden_states = (
+            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
+        )
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+        vision_feature_layer = (
+            vision_feature_layer if vision_feature_layer is not None else self.config.vision_feature_layer
+        )
+        vision_feature_select_strategy = (
+            vision_feature_select_strategy
+            if vision_feature_select_strategy is not None
+            else self.config.vision_feature_select_strategy
+        )
+
+        if inputs_embeds is None:
+            # 1. Extra the input embeddings
+            inputs_embeds = self.get_input_embeddings()(input_ids)
+
+            # 2. Merge text and images
+            if pixel_values is not None and input_ids.shape[1] != 1:
+                image_outputs = self.vision_model(pixel_values)
+                # this is not memory efficient at all (output_hidden_states=True) will save all the hidden stated.
+                # selected_image_feature = image_outputs.hidden_states[vision_feature_layer]
+
+                # if vision_feature_select_strategy == "default":
+                #     selected_image_feature = selected_image_feature[:, 1:]
+                # elif vision_feature_select_strategy == "full":
+                #     selected_image_feature = selected_image_feature
+                # else:
+                #     raise ValueError(
+                #         f"Unexpected select feature strategy: {self.config.vision_feature_select_strategy}"
+                #     )
+
+                image_features = self.aligner(image_outputs)
+                inputs_embeds = inputs_embeds.to(image_features.dtype)
+                inputs_embeds, attention_mask, labels, position_ids = self._merge_input_ids_with_image_features(
+                    image_features, inputs_embeds, input_ids, attention_mask, labels
+                )
+
+            # In case input_ids.shape[1] == 1 & pixel_values==None & past_key_values != None, we are in the case of
+            # generation with cache
+            elif past_key_values is not None and pixel_values is not None and input_ids.shape[1] == 1:
+                # Retrieve the first layer to inspect the logits and mask out the hidden states
+                # that are set to 0
+                first_layer_past_key_value = past_key_values[0][0][:, :, :, 0]
+
+                # Sum all dimensions of head_dim (-2) to avoid random errors such as: https://github.com/huggingface/transformers/pull/28032#issuecomment-1863691941
+                batch_index, non_attended_tokens = torch.where(first_layer_past_key_value.float().sum(-2) == 0)
+
+                # Get the target length
+                target_length = input_ids.shape[1]
+                past_length = first_layer_past_key_value.shape[-1]
+
+                extended_attention_mask = torch.ones(
+                    (attention_mask.shape[0], past_length),
+                    dtype=attention_mask.dtype,
+                    device=attention_mask.device,
+                )
+
+                # Filter out only the tokens that can be un-attended, this can happen
+                # if one uses qwen2vlm + Fused modules where the cache on the
+                # first iteration is already big enough, or if one passes custom cache
+                valid_indices = non_attended_tokens < extended_attention_mask.size(-1)
+                new_batch_index = batch_index[valid_indices]
+                new_non_attended_tokens = non_attended_tokens[valid_indices]
+
+                # Zero-out the places where we don't need to attend
+                extended_attention_mask[new_batch_index, new_non_attended_tokens] = 0
+
+                attention_mask = torch.cat((extended_attention_mask, attention_mask[:, -target_length:]), dim=1)
+                position_ids = torch.sum(attention_mask, dim=1).unsqueeze(-1) - 1
+
+        outputs = self.language_model(
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            use_cache=use_cache,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+
+        logits = outputs[0]
+
+        loss = None
+        if labels is not None:
+            # Shift so that tokens < n predict n
+            if attention_mask is not None:
+                shift_attention_mask = attention_mask[..., 1:]
+                shift_logits = logits[..., :-1, :][shift_attention_mask.to(logits.device) != 0].contiguous()
+                shift_labels = labels[..., 1:][shift_attention_mask.to(labels.device) != 0].contiguous()
+            else:
+                shift_logits = logits[..., :-1, :].contiguous()
+                shift_labels = labels[..., 1:].contiguous()
+            # Flatten the tokens
+            loss_fct = nn.CrossEntropyLoss()
+            loss = loss_fct(
+                shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1).to(shift_logits.device)
+            )
+
+        if not return_dict:
+            output = (logits,) + outputs[1:]
+            return (loss,) + output if loss is not None else output
+
+        return UrsaCausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+        )
+
+    def prepare_inputs_for_generation(
+        self, input_ids, past_key_values=None, inputs_embeds=None, pixel_values=None, attention_mask=None, **kwargs
+    ):
+        if past_key_values is not None:
+            if isinstance(past_key_values, Cache):
+                cache_length = past_key_values.get_seq_length()
+                past_length = past_key_values.seen_tokens
+            else:
+                cache_length = past_length = past_key_values[0][0].shape[2]
+
+            # Keep only the unprocessed tokens:
+            # 1 - If the length of the attention_mask exceeds the length of input_ids, then we are in a setting where
+            # some of the inputs are exclusively passed as part of the cache (e.g. when passing input_embeds as
+            # input)
+            if attention_mask is not None and attention_mask.shape[1] > input_ids.shape[1]:
+                input_ids = input_ids[:, -(attention_mask.shape[1] - past_length) :]
+            # 2 - If the past_length is smaller than input_ids', then input_ids holds all input tokens. We can discard
+            # input_ids based on the past_length.
+            elif past_length < input_ids.shape[1]:
+                input_ids = input_ids[:, past_length:]
+            # 3 - Otherwise (past_length >= input_ids.shape[1]), let's assume input_ids only has unprocessed tokens.
+            elif self.config.image_token_index in input_ids:
+                input_ids = input_ids[:, input_ids.shape[1] - 1 :]
+            # If the cache has seen more tokens than it can hold, then the cache has a size limit. Let's discard the
+            # older attention values, as their corresponding values are not part of the input.
+            if cache_length < past_length and attention_mask is not None:
+                attention_mask = attention_mask[:, -(cache_length + input_ids.shape[1]) :]
+
+        position_ids = kwargs.get("position_ids", None)
+        if attention_mask is not None and position_ids is None:
+            # create position_ids on the fly for batch generation
+            position_ids = attention_mask.long().cumsum(-1) - 1
+            position_ids.masked_fill_(attention_mask == 0, 1)
+            if past_key_values:
+                position_ids = position_ids[:, -input_ids.shape[1] :]
+
+        # if `inputs_embeds` are passed, we only want to use them in the 1st generation step
+        if inputs_embeds is not None and past_key_values is None:
+            model_inputs = {"inputs_embeds": inputs_embeds}
+        else:
+            model_inputs = {"input_ids": input_ids}
+
+        model_inputs.update(
+            {
+                "position_ids": position_ids,
+                "past_key_values": past_key_values,
+                "use_cache": kwargs.get("use_cache"),
+                "attention_mask": attention_mask,
+                "pixel_values": pixel_values,
+            }
+        )
+        return model_inputs
+
+    def _reorder_cache(self, *args, **kwargs):
+        return self.language_model._reorder_cache(*args, **kwargs)
+
+
+class UrsaForTokenClassification(UrsaPreTrainedModel):
+    def __init__(self, config: UrsaConfig):
+        super().__init__(config)
+        self.vision_model = HybridVisionTower(**config.vision_config["params"])
+        aligner_config = AlignerConfig(**config.aligner_config)
+        self.aligner = MlpProjector(aligner_config.params)
+        self.vocab_size = config.text_config.vocab_size
+        self.language_model = AutoModelForCausalLM.from_config(
+            config.text_config, attn_implementation=config._attn_implementation
+        )
+        self.pad_token_id = self.config.pad_token_id if self.config.pad_token_id is not None else -1
+
+        self.dropout = nn.Dropout(0.1)
+        self.score = nn.Linear(self.language_model.config.hidden_size, 1)
+        self.post_init()
+
+    def _init_weights(self, module):
+        """Initialize the weights."""
+        if isinstance(module, nn.Linear):
+            # print(module)
+            nn.init.xavier_uniform_(module.weight)
+            nn.init.zeros_(module.bias)
+
+    def get_input_embeddings(self):
+        return self.language_model.get_input_embeddings()
+
+    def set_input_embeddings(self, value):
+        self.language_model.set_input_embeddings(value)
+
+    def get_output_embeddings(self):
+        return self.language_model.get_output_embeddings()
+
+    def set_output_embeddings(self, new_embeddings):
+        self.language_model.set_output_embeddings(new_embeddings)
+
+    def set_decoder(self, decoder):
+        self.language_model.set_decoder(decoder)
+
+    def get_decoder(self):
+        return self.language_model.get_decoder()
+
+    def tie_weights(self):
+        return self.language_model.tie_weights()
+
+    def resize_token_embeddings(self, new_num_tokens: Optional[int] = None, pad_to_multiple_of=None) -> nn.Embedding:
+        model_embeds = self.language_model.resize_token_embeddings(new_num_tokens, pad_to_multiple_of)
+        # update vocab size
+        self.config.text_config.vocab_size = model_embeds.num_embeddings
+        self.vocab_size = model_embeds.num_embeddings
+        return model_embeds
+
+    def _merge_input_ids_with_image_features(self, image_features, inputs_embeds, input_ids, attention_mask, labels):
+        num_images, num_image_patches, embed_dim = image_features.shape
+        batch_size, sequence_length = input_ids.shape
+        left_padding = not torch.sum(input_ids[:, -1] == torch.tensor(self.pad_token_id))
+        # 1. Create a mask to know where special image tokens are
+        special_image_token_mask = input_ids == self.config.image_token_index
+        num_special_image_tokens = torch.sum(special_image_token_mask, dim=-1)
+        # Compute the maximum embed dimension
+        max_embed_dim = (num_special_image_tokens.max() * (num_image_patches - 1)) + sequence_length
+        batch_indices, non_image_indices = torch.where(input_ids != self.config.image_token_index)
+
+        # 2. Compute the positions where text should be written
+        # Calculate new positions for text tokens in merged image-text sequence.
+        # `special_image_token_mask` identifies image tokens. Each image token will be replaced by `nb_text_tokens_per_images - 1` text tokens.
+        # `torch.cumsum` computes how each image token shifts subsequent text token positions.
+        # - 1 to adjust for zero-based indexing, as `cumsum` inherently increases indices by one.
+        new_token_positions = torch.cumsum((special_image_token_mask * (num_image_patches - 1) + 1), -1) - 1
+        nb_image_pad = max_embed_dim - 1 - new_token_positions[:, -1]
+        if left_padding:
+            new_token_positions += nb_image_pad[:, None]  # offset for left padding
+        text_to_overwrite = new_token_positions[batch_indices, non_image_indices]
+
+        # 3. Create the full embedding, already padded to the maximum position
+        final_embedding = torch.zeros(
+            batch_size, max_embed_dim, embed_dim, dtype=inputs_embeds.dtype, device=inputs_embeds.device
+        )
+        final_attention_mask = torch.zeros(
+            batch_size, max_embed_dim, dtype=attention_mask.dtype, device=inputs_embeds.device
+        )
+        if labels is not None:
+            final_labels = torch.full(
+                (batch_size, max_embed_dim), self.config.ignore_index, dtype=input_ids.dtype, device=input_ids.device
+            )
+        # In case the Vision model or the Language model has been offloaded to CPU, we need to manually
+        # set the corresponding tensors into their correct target device.
+        target_device = inputs_embeds.device
+        batch_indices, non_image_indices, text_to_overwrite = (
+            batch_indices.to(target_device),
+            non_image_indices.to(target_device),
+            text_to_overwrite.to(target_device),
+        )
+        attention_mask = attention_mask.to(target_device)
+
+        # 4. Fill the embeddings based on the mask. If we have ["hey" "<image>", "how", "are"]
+        # we need to index copy on [0, 577, 578, 579] for the text and [1:576] for the image features
+        final_embedding[batch_indices, text_to_overwrite] = inputs_embeds[batch_indices, non_image_indices]
+        final_attention_mask[batch_indices, text_to_overwrite] = attention_mask[batch_indices, non_image_indices]
+        if labels is not None:
+            final_labels[batch_indices, text_to_overwrite] = labels[batch_indices, non_image_indices]
+        # 5. Fill the embeddings corresponding to the images. Anything that is not `text_positions` needs filling (#29835)
+        image_to_overwrite = torch.full(
+            (batch_size, max_embed_dim), True, dtype=torch.bool, device=inputs_embeds.device
+        )
+        image_to_overwrite[batch_indices, text_to_overwrite] = False
+        image_to_overwrite &= image_to_overwrite.cumsum(-1) - 1 >= nb_image_pad[:, None].to(target_device)
+
+        if image_to_overwrite.sum() != image_features.shape[:-1].numel():
+            raise ValueError(
+                f"The input provided to the model are wrong. The number of image tokens is {torch.sum(special_image_token_mask)} while"
+                f" the number of image given to the model is {num_images}. This prevents correct indexing and breaks batch generation."
+            )
+
+        final_embedding[image_to_overwrite] = image_features.contiguous().reshape(-1, embed_dim).to(target_device)
+        final_attention_mask |= image_to_overwrite
+        position_ids = (final_attention_mask.cumsum(-1) - 1).masked_fill_((final_attention_mask == 0), 1)
+
+        # 6. Mask out the embedding at padding positions, as we later use the past_key_value value to determine the non-attended tokens.
+        batch_indices, pad_indices = torch.where(input_ids == self.pad_token_id)
+        indices_to_mask = new_token_positions[batch_indices, pad_indices]
+
+        final_embedding[batch_indices, indices_to_mask] = 0
+
+        if labels is None:
+            final_labels = None
+
+        return final_embedding, final_attention_mask, final_labels, position_ids
+
+    def forward(
+        self,
+        input_ids: torch.LongTensor = None,
+        pixel_values: torch.FloatTensor = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        past_key_values: Optional[List[torch.FloatTensor]] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        vision_feature_layer: Optional[int] = None,
+        vision_feature_select_strategy: Optional[str] = None,
+        labels: Optional[torch.LongTensor] = None,
+        use_cache: Optional[bool] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+    ) -> Union[Tuple, UrsaCausalLMOutputWithPast]:
+        output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
+        output_hidden_states = (
+            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
+        )
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+        vision_feature_layer = (
+            vision_feature_layer if vision_feature_layer is not None else self.config.vision_feature_layer
+        )
+        vision_feature_select_strategy = (
+            vision_feature_select_strategy
+            if vision_feature_select_strategy is not None
+            else self.config.vision_feature_select_strategy
+        )
+        if inputs_embeds is None:
+            # 1. Extra the input embeddings
+            inputs_embeds = self.get_input_embeddings()(input_ids)
+
+            # 2. Merge text and images
+            if pixel_values is not None and input_ids.shape[1] != 1:
+                image_outputs = self.vision_model(pixel_values)
+                # this is not memory efficient at all (output_hidden_states=True) will save all the hidden stated.
+                # selected_image_feature = image_outputs.hidden_states[vision_feature_layer]
+
+                # if vision_feature_select_strategy == "default":
+                #     selected_image_feature = selected_image_feature[:, 1:]
+                # elif vision_feature_select_strategy == "full":
+                #     selected_image_feature = selected_image_feature
+                # else:
+                #     raise ValueError(
+                #         f"Unexpected select feature strategy: {self.config.vision_feature_select_strategy}"
+                #     )
+
+                image_features = self.aligner(image_outputs)
+                inputs_embeds = inputs_embeds.to(image_features.dtype)
+                inputs_embeds, attention_mask, labels, position_ids = self._merge_input_ids_with_image_features(
+                    image_features, inputs_embeds, input_ids, attention_mask, labels
+                )
+
+            # In case input_ids.shape[1] == 1 & pixel_values==None & past_key_values != None, we are in the case of
+            # generation with cache
+            elif past_key_values is not None and pixel_values is not None and input_ids.shape[1] == 1:
+                # Retrieve the first layer to inspect the logits and mask out the hidden states
+                # that are set to 0
+                first_layer_past_key_value = past_key_values[0][0][:, :, :, 0]
+
+                # Sum all dimensions of head_dim (-2) to avoid random errors such as: https://github.com/huggingface/transformers/pull/28032#issuecomment-1863691941
+                batch_index, non_attended_tokens = torch.where(first_layer_past_key_value.float().sum(-2) == 0)
+
+                # Get the target length
+                target_length = input_ids.shape[1]
+                past_length = first_layer_past_key_value.shape[-1]
+
+                extended_attention_mask = torch.ones(
+                    (attention_mask.shape[0], past_length),
+                    dtype=attention_mask.dtype,
+                    device=attention_mask.device,
+                )
+
+                # Filter out only the tokens that can be un-attended, this can happen
+                # if one uses qwen2vlm + Fused modules where the cache on the
+                # first iteration is already big enough, or if one passes custom cache
+                valid_indices = non_attended_tokens < extended_attention_mask.size(-1)
+                new_batch_index = batch_index[valid_indices]
+                new_non_attended_tokens = non_attended_tokens[valid_indices]
+
+                # Zero-out the places where we don't need to attend
+                extended_attention_mask[new_batch_index, new_non_attended_tokens] = 0
+
+                attention_mask = torch.cat((extended_attention_mask, attention_mask[:, -target_length:]), dim=1)
+                position_ids = torch.sum(attention_mask, dim=1).unsqueeze(-1) - 1
+
+        outputs = self.language_model(
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            use_cache=use_cache,
+            output_attentions=output_attentions,
+            return_dict=return_dict,
+            output_hidden_states=True
+        )
+
+        # logits = outputs[0]
+        logits = outputs.hidden_states[-1]
+        logits = self.dropout(logits)
+        logits = self.score(logits)
+
+
+        loss = None
+        if labels is not None:
+            # Shift so that tokens < n predict n
+            # if attention_mask is not None:
+            #     shift_attention_mask = attention_mask[..., 1:]
+            #     shift_logits = logits[..., :-1, :][shift_attention_mask.to(logits.device) != 0].contiguous()
+            #     shift_labels = labels[..., 1:][shift_attention_mask.to(labels.device) != 0].contiguous()
+            # else:
+            #     shift_logits = logits[..., :-1, :].contiguous()
+            #     shift_labels = labels[..., 1:].contiguous()
+            # # Flatten the tokens
+            # loss_fct = nn.CrossEntropyLoss()
+            # loss = loss_fct(
+            #     shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1).to(shift_logits.device)
+            # )
+            # loss_fct = nn.CrossEntropyLoss()
+            # loss = loss_fct(logits.view(-1, 1), labels.view(-1))
+            loss = None
+        if not return_dict:
+            output = (logits,) + outputs[1:]
+            return (loss,) + output if loss is not None else output
+
+        return UrsaCausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            # past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            labels=labels
+            # attentions=outputs.attentions,
+        )
+
+    def prepare_inputs_for_generation(
+        self, input_ids, past_key_values=None, inputs_embeds=None, pixel_values=None, attention_mask=None, **kwargs
+    ):
+        if past_key_values is not None:
+            if isinstance(past_key_values, Cache):
+                cache_length = past_key_values.get_seq_length()
+                past_length = past_key_values.seen_tokens
+            else:
+                cache_length = past_length = past_key_values[0][0].shape[2]
+
+            # Keep only the unprocessed tokens:
+            # 1 - If the length of the attention_mask exceeds the length of input_ids, then we are in a setting where
+            # some of the inputs are exclusively passed as part of the cache (e.g. when passing input_embeds as
+            # input)
+            if attention_mask is not None and attention_mask.shape[1] > input_ids.shape[1]:
+                input_ids = input_ids[:, -(attention_mask.shape[1] - past_length) :]
+            # 2 - If the past_length is smaller than input_ids', then input_ids holds all input tokens. We can discard
+            # input_ids based on the past_length.
+            elif past_length < input_ids.shape[1]:
+                input_ids = input_ids[:, past_length:]
+            # 3 - Otherwise (past_length >= input_ids.shape[1]), let's assume input_ids only has unprocessed tokens.
+            elif self.config.image_token_index in input_ids:
+                input_ids = input_ids[:, input_ids.shape[1] - 1 :]
+            # If the cache has seen more tokens than it can hold, then the cache has a size limit. Let's discard the
+            # older attention values, as their corresponding values are not part of the input.
+            if cache_length < past_length and attention_mask is not None:
+                attention_mask = attention_mask[:, -(cache_length + input_ids.shape[1]) :]
+
+        position_ids = kwargs.get("position_ids", None)
+        if attention_mask is not None and position_ids is None:
+            # create position_ids on the fly for batch generation
+            position_ids = attention_mask.long().cumsum(-1) - 1
+            position_ids.masked_fill_(attention_mask == 0, 1)
+            if past_key_values:
+                position_ids = position_ids[:, -input_ids.shape[1] :]
+
+        # if `inputs_embeds` are passed, we only want to use them in the 1st generation step
+        if inputs_embeds is not None and past_key_values is None:
+            model_inputs = {"inputs_embeds": inputs_embeds}
+        else:
+            model_inputs = {"input_ids": input_ids}
+
+        model_inputs.update(
+            {
+                "position_ids": position_ids,
+                "past_key_values": past_key_values,
+                "use_cache": kwargs.get("use_cache"),
+                "attention_mask": attention_mask,
+                "pixel_values": pixel_values,
+            }
+        )
+        return model_inputs
+
+    def _reorder_cache(self, *args, **kwargs):
+        return self.language_model._reorder_cache(*args, **kwargs)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/processing_ursa.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/processing_ursa.py
new file mode 100644
index 0000000000000000000000000000000000000000..fe774751f345878d78ddf46edf23445ab57b62e8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/processing_ursa.py
@@ -0,0 +1,61 @@
+# Copyright (2025) Bytedance Ltd. and/or its affiliates
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import List, Optional, Union
+
+from transformers.feature_extraction_utils import BatchFeature
+from transformers.image_utils import ImageInput
+from transformers.processing_utils import ProcessorMixin
+from transformers.tokenization_utils_base import (PaddingStrategy, PreTokenizedInput, TextInput,
+                                                  TruncationStrategy)
+from transformers.utils import TensorType
+
+
+class UrsaProcessor(ProcessorMixin):
+    attributes = ["image_processor", "tokenizer"]
+    valid_kwargs = ["chat_template"]
+    image_processor_class = "AutoImageProcessor"
+    tokenizer_class = "AutoTokenizer"
+
+    def __init__(self, image_processor=None, tokenizer=None, chat_template=None, **kwargs):
+        super().__init__(image_processor, tokenizer, chat_template=chat_template, **kwargs)
+
+    def __call__(
+        self,
+        text: Union[TextInput, PreTokenizedInput, List[TextInput], List[PreTokenizedInput]] = None,
+        images: ImageInput = None,
+        padding: Union[bool, str, PaddingStrategy] = False,
+        truncation: Union[bool, str, TruncationStrategy] = None,
+        max_length=None,
+        return_tensors: Optional[Union[str, TensorType]] = None,    # or TensorType.PYTORCH
+    ) -> BatchFeature:
+        image_inputs = {}
+        if images is not None:
+            image_inputs = self.image_processor(images, return_tensors=return_tensors)
+        text_inputs = self.tokenizer(
+            text, return_tensors=return_tensors, padding=padding, truncation=truncation, max_length=max_length
+        )
+        return BatchFeature(data={**text_inputs, **image_inputs})
+
+    def decode(self, *args, **kwargs):
+        return self.tokenizer.decode(*args, **kwargs)
+
+    def batch_decode(self, *args, **kwargs):
+        return self.tokenizer.batch_decode(*args, **kwargs)
+
+    @property
+    # Copied from transformers.models.clip.processing_clip.CLIPProcessor.model_input_names
+    def model_input_names(self):
+        tokenizer_input_names = self.tokenizer.model_input_names
+        image_processor_input_names = self.image_processor.model_input_names
+        return list(dict.fromkeys(tokenizer_input_names + image_processor_input_names))
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/projector.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/projector.py
new file mode 100644
index 0000000000000000000000000000000000000000..15f4ca3b61678667b8f472f566f7412f68828213
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/projector.py
@@ -0,0 +1,100 @@
+# Copyright (c) 2023-2024 DeepSeek.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of
+# this software and associated documentation files (the "Software"), to deal in
+# the Software without restriction, including without limitation the rights to
+# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+# the Software, and to permit persons to whom the Software is furnished to do so,
+# subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+from typing import Tuple, Union
+
+import torch
+import torch.nn as nn
+from attrdict import AttrDict
+
+
+class MlpProjector(nn.Module):
+    def __init__(self, cfg):
+        super().__init__()
+
+        self.cfg = cfg
+
+        if cfg.projector_type == "identity":
+            modules = nn.Identity()
+
+        elif cfg.projector_type == "linear":
+            modules = nn.Linear(cfg.input_dim, cfg.n_embed)
+
+        elif cfg.projector_type == "mlp_gelu":
+            mlp_depth = cfg.get("depth", 1)
+            modules = [nn.Linear(cfg.input_dim, cfg.n_embed)]
+            for _ in range(1, mlp_depth):
+                modules.append(nn.GELU())
+                modules.append(nn.Linear(cfg.n_embed, cfg.n_embed))
+            modules = nn.Sequential(*modules)
+
+        elif cfg.projector_type == "low_high_hybrid_split_mlp_gelu":
+            mlp_depth = cfg.get("depth", 1)
+            self.high_up_proj = nn.Linear(cfg.input_dim, cfg.n_embed // 2)
+            self.low_up_proj = nn.Linear(cfg.input_dim, cfg.n_embed // 2)
+
+            modules = []
+            for _ in range(1, mlp_depth):
+                modules.append(nn.GELU())
+                modules.append(nn.Linear(cfg.n_embed, cfg.n_embed))
+            modules = nn.Sequential(*modules)
+
+        else:
+            raise ValueError(f"Unknown projector type: {cfg.projector_type}")
+
+        self.layers = modules
+
+    def forward(
+        self, x_or_tuple: Union[Tuple[torch.Tensor, torch.Tensor], torch.Tensor]
+    ):
+        """
+
+        Args:
+            x_or_tuple (Union[Tuple[torch.Tensor, torch.Tensor], torch.Tensor]:  if it is a tuple of torch.Tensor,
+                then it comes from the hybrid vision encoder, and x = high_res_x, low_res_x);
+                otherwise it is the feature from the single vision encoder.
+
+        Returns:
+            x (torch.Tensor): [b, s, c]
+        """
+
+        if isinstance(x_or_tuple, tuple):
+            # self.cfg.projector_type == "low_high_hybrid_split_mlp_gelu":
+            high_x, low_x = x_or_tuple
+            high_x = self.high_up_proj(high_x)
+            low_x = self.low_up_proj(low_x)
+            x = torch.concat([high_x, low_x], dim=-1)
+        else:
+            x = x_or_tuple
+
+        return self.layers(x)
+
+
+if __name__ == "__main__":
+    cfg = AttrDict(
+        input_dim=1024,
+        n_embed=2048,
+        depth=2,
+        projector_type="low_high_hybrid_split_mlp_gelu",
+    )
+    inputs = (torch.rand(4, 576, 1024), torch.rand(4, 576, 1024))
+
+    m = MlpProjector(cfg)
+    out = m(inputs)
+    print(out.shape)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/sam.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/sam.py
new file mode 100644
index 0000000000000000000000000000000000000000..31159a944c928e874391883f546c3f090f40af65
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/sam.py
@@ -0,0 +1,593 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.
+
+import copy
+from dataclasses import dataclass
+from functools import partial
+from typing import List, Optional, Tuple, Type, Union
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class MLPBlock(nn.Module):
+    def __init__(
+        self,
+        embedding_dim: int,
+        mlp_dim: int,
+        act: Type[nn.Module] = nn.GELU,
+    ) -> None:
+        super().__init__()
+        self.lin1 = nn.Linear(embedding_dim, mlp_dim)
+        self.lin2 = nn.Linear(mlp_dim, embedding_dim)
+        self.act = act()
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.lin2(self.act(self.lin1(x)))
+
+
+# From https://github.com/facebookresearch/detectron2/blob/main/detectron2/layers/batch_norm.py # noqa
+# Itself from https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119  # noqa
+class LayerNorm2d(nn.Module):
+    def __init__(self, num_channels: int, eps: float = 1e-6) -> None:
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(num_channels))
+        self.bias = nn.Parameter(torch.zeros(num_channels))
+        self.eps = eps
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        u = x.mean(1, keepdim=True)
+        s = (x - u).pow(2).mean(1, keepdim=True)
+        x = (x - u) / torch.sqrt(s + self.eps)
+        x = self.weight[:, None, None] * x + self.bias[:, None, None]
+        return x
+
+
+# This class and its supporting functions below lightly adapted from the ViTDet backbone available at: https://github.com/facebookresearch/detectron2/blob/main/detectron2/modeling/backbone/vit.py # noqa
+class ImageEncoderViT(nn.Module):
+    def __init__(
+        self,
+        img_size: int = 1024,
+        patch_size: int = 16,
+        in_chans: int = 3,
+        embed_dim: int = 768,
+        depth: int = 12,
+        num_heads: int = 12,
+        mlp_ratio: float = 4.0,
+        out_chans: int = 256,
+        qkv_bias: bool = True,
+        norm_layer: Type[nn.Module] = nn.LayerNorm,
+        act_layer: Type[nn.Module] = nn.GELU,
+        use_abs_pos: bool = True,
+        use_rel_pos: bool = False,
+        rel_pos_zero_init: bool = True,
+        window_size: int = 0,
+        global_attn_indexes: Tuple[int, ...] = (),
+        downsample_channels: Tuple[int, ...] = (512, 1024),
+    ) -> None:
+        """
+        Args:
+            img_size (int): Input image size.
+            patch_size (int): Patch size.
+            in_chans (int): Number of input image channels.
+            embed_dim (int): Patch embedding dimension.
+            depth (int): Depth of ViT.
+            num_heads (int): Number of attention heads in each ViT block.
+            mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.
+            qkv_bias (bool): If True, add a learnable bias to query, key, value.
+            norm_layer (nn.Module): Normalization layer.
+            act_layer (nn.Module): Activation layer.
+            use_abs_pos (bool): If True, use absolute positional embeddings.
+            use_rel_pos (bool): If True, add relative positional embeddings to the attention map.
+            rel_pos_zero_init (bool): If True, zero initialize relative positional parameters.
+            window_size (int): Window size for window attention blocks.
+            global_attn_indexes (list): Indexes for blocks using global attention.
+            downsample_channels (list): Channels for downsampling layers.
+        """
+        super().__init__()
+        self.img_size = img_size
+
+        self.patch_embed = PatchEmbed(
+            kernel_size=(patch_size, patch_size),
+            stride=(patch_size, patch_size),
+            in_chans=in_chans,
+            embed_dim=embed_dim,
+        )
+
+        self.pos_embed: Optional[nn.Parameter] = None
+        if use_abs_pos:
+            # Initialize absolute positional embedding with pretrain image size.
+            self.pos_embed = nn.Parameter(
+                torch.zeros(
+                    1, img_size // patch_size, img_size // patch_size, embed_dim
+                )
+            )
+
+        self.blocks = nn.ModuleList()
+        for i in range(depth):
+            block = Block(
+                dim=embed_dim,
+                num_heads=num_heads,
+                mlp_ratio=mlp_ratio,
+                qkv_bias=qkv_bias,
+                norm_layer=norm_layer,
+                act_layer=act_layer,
+                use_rel_pos=use_rel_pos,
+                rel_pos_zero_init=rel_pos_zero_init,
+                window_size=window_size if i not in global_attn_indexes else 0,
+                input_size=(img_size // patch_size, img_size // patch_size),
+            )
+            self.blocks.append(block)
+
+        self.neck = nn.Sequential(
+            nn.Conv2d(
+                embed_dim,
+                out_chans,
+                kernel_size=1,
+                bias=False,
+            ),
+            LayerNorm2d(out_chans),
+            nn.Conv2d(
+                out_chans,
+                out_chans,
+                kernel_size=3,
+                padding=1,
+                bias=False,
+            ),
+            LayerNorm2d(out_chans),
+        )
+
+        in_channels = out_chans
+        downsamples = []
+        for i in range(len(downsample_channels)):
+            out_channels = downsample_channels[i]
+            downsamples.append(
+                nn.Conv2d(
+                    in_channels,
+                    out_channels,
+                    kernel_size=3,
+                    stride=2,
+                    padding=1,
+                    bias=False,
+                )
+            )
+            in_channels = out_channels
+        self.downsamples = nn.Sequential(*downsamples)
+
+        self.sam_hd = True
+        if self.sam_hd:
+            self.hd_alpha_downsamples = nn.Parameter(torch.zeros(1))
+            # self.neck_hd = nn.Linear(embed_dim, embed_dim)
+            self.neck_hd = copy.deepcopy(self.neck)
+            # self.downsamples_hd = copy.deepcopy(self.downsamples)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.patch_embed(x)
+        if self.pos_embed is not None:
+            x = x + self.pos_embed
+
+        global_features = []
+        for i, blk in enumerate(self.blocks):
+            x = blk(x)
+            if self.sam_hd and blk.window_size == 0:
+                global_features.append(x)
+
+        x = self.neck(x.permute(0, 3, 1, 2))
+        x_dtype = x.dtype
+        x = F.interpolate(
+            x.float(), size=(96, 96), mode="bilinear", align_corners=False
+        ).to(x_dtype)
+        x = self.downsamples(x)
+
+        if self.sam_hd:
+            first_global_feature = self.neck_hd(global_features[0].permute(0, 3, 1, 2))
+            x_dtype = first_global_feature.dtype
+            first_global_feature = F.interpolate(
+                first_global_feature.float(),
+                size=(96, 96),
+                mode="bilinear",
+                align_corners=False,
+            )
+            first_global_feature = self.downsamples(first_global_feature.to(x_dtype))
+            x = x + first_global_feature * self.hd_alpha_downsamples
+
+        return x
+
+
+class Block(nn.Module):
+    """Transformer blocks with support of window attention and residual propagation blocks"""
+
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int,
+        mlp_ratio: float = 4.0,
+        qkv_bias: bool = True,
+        norm_layer: Type[nn.Module] = nn.LayerNorm,
+        act_layer: Type[nn.Module] = nn.GELU,
+        use_rel_pos: bool = False,
+        rel_pos_zero_init: bool = True,
+        window_size: int = 0,
+        input_size: Optional[Tuple[int, int]] = None,
+    ) -> None:
+        """
+        Args:
+            dim (int): Number of input channels.
+            num_heads (int): Number of attention heads in each ViT block.
+            mlp_ratio (float): Ratio of mlp hidden dim to embedding dim.
+            qkv_bias (bool): If True, add a learnable bias to query, key, value.
+            norm_layer (nn.Module): Normalization layer.
+            act_layer (nn.Module): Activation layer.
+            use_rel_pos (bool): If True, add relative positional embeddings to the attention map.
+            rel_pos_zero_init (bool): If True, zero initialize relative positional parameters.
+            window_size (int): Window size for window attention blocks. If it equals 0, then
+                use global attention.
+            input_size (tuple(int, int) or None): Input resolution for calculating the relative
+                positional parameter size.
+        """
+        super().__init__()
+        self.norm1 = norm_layer(dim)
+        self.attn = Attention(
+            dim,
+            num_heads=num_heads,
+            qkv_bias=qkv_bias,
+            use_rel_pos=use_rel_pos,
+            rel_pos_zero_init=rel_pos_zero_init,
+            input_size=input_size if window_size == 0 else (window_size, window_size),
+        )
+
+        self.norm2 = norm_layer(dim)
+        self.mlp = MLPBlock(
+            embedding_dim=dim, mlp_dim=int(dim * mlp_ratio), act=act_layer
+        )
+
+        self.window_size = window_size
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        shortcut = x
+        x = self.norm1(x)
+        # Window partition
+        if self.window_size > 0:
+            H, W = x.shape[1], x.shape[2]
+            x, pad_hw = window_partition(x, self.window_size)
+
+        x = self.attn(x)
+        # Reverse window partition
+        if self.window_size > 0:
+            x = window_unpartition(x, self.window_size, pad_hw, (H, W))
+
+        x = shortcut + x
+        x = x + self.mlp(self.norm2(x))
+
+        return x
+
+
+class Attention(nn.Module):
+    """Multi-head Attention block with relative position embeddings."""
+
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int = 8,
+        qkv_bias: bool = True,
+        use_rel_pos: bool = False,
+        rel_pos_zero_init: bool = True,
+        input_size: Optional[Tuple[int, int]] = None,
+    ) -> None:
+        """
+        Args:
+            dim (int): Number of input channels.
+            num_heads (int): Number of attention heads.
+            qkv_bias (bool):  If True, add a learnable bias to query, key, value.
+            rel_pos (bool): If True, add relative positional embeddings to the attention map.
+            rel_pos_zero_init (bool): If True, zero initialize relative positional parameters.
+            input_size (tuple(int, int) or None): Input resolution for calculating the relative
+                positional parameter size.
+        """
+        super().__init__()
+        self.num_heads = num_heads
+        head_dim = dim // num_heads
+        self.scale = head_dim**-0.5
+
+        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
+        self.proj = nn.Linear(dim, dim)
+
+        self.use_rel_pos = use_rel_pos
+        if self.use_rel_pos:
+            assert (
+                input_size is not None
+            ), "Input size must be provided if using relative positional encoding."
+            # initialize relative positional embeddings
+            self.rel_pos_h = nn.Parameter(torch.zeros(2 * input_size[0] - 1, head_dim))
+            self.rel_pos_w = nn.Parameter(torch.zeros(2 * input_size[1] - 1, head_dim))
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        B, H, W, _ = x.shape
+        # qkv with shape (3, B, nHead, H * W, C)
+        qkv = (
+            self.qkv(x).reshape(B, H * W, 3, self.num_heads, -1).permute(2, 0, 3, 1, 4)
+        )
+        # q, k, v with shape (B * nHead, H * W, C)
+        q, k, v = qkv.reshape(3, B * self.num_heads, H * W, -1).unbind(0)
+
+        def do_attention(q, k, v):
+            attn = (q * self.scale) @ k.transpose(-2, -1)
+            if self.use_rel_pos:
+                attn = add_decomposed_rel_pos(
+                    attn, q, self.rel_pos_h, self.rel_pos_w, (H, W), (H, W)
+                )
+
+            attn = attn.softmax(dim=-1)
+            x = (
+                (attn @ v)
+                .view(B, self.num_heads, H, W, -1)
+                .permute(0, 2, 3, 1, 4)
+                .reshape(B, H, W, -1)
+            )
+
+            return x
+
+        # from haiscale.utils import on_demand_checkpoint
+        # x = on_demand_checkpoint(do_attention, q, k, v)
+        x = do_attention(q, k, v)
+        x = self.proj(x)
+
+        return x
+
+
+def window_partition(
+    x: torch.Tensor, window_size: int
+) -> Tuple[torch.Tensor, Tuple[int, int]]:
+    """
+    Partition into non-overlapping windows with padding if needed.
+    Args:
+        x (tensor): input tokens with [B, H, W, C].
+        window_size (int): window size.
+
+    Returns:
+        windows: windows after partition with [B * num_windows, window_size, window_size, C].
+        (Hp, Wp): padded height and width before partition
+    """
+    B, H, W, C = x.shape
+
+    pad_h = (window_size - H % window_size) % window_size
+    pad_w = (window_size - W % window_size) % window_size
+    if pad_h > 0 or pad_w > 0:
+        x = F.pad(x, (0, 0, 0, pad_w, 0, pad_h))
+    Hp, Wp = H + pad_h, W + pad_w
+
+    x = x.view(B, Hp // window_size, window_size, Wp // window_size, window_size, C)
+    windows = (
+        x.permute(0, 1, 3, 2, 4, 5).contiguous().view(-1, window_size, window_size, C)
+    )
+    return windows, (Hp, Wp)
+
+
+def window_unpartition(
+    windows: torch.Tensor,
+    window_size: int,
+    pad_hw: Tuple[int, int],
+    hw: Tuple[int, int],
+) -> torch.Tensor:
+    """
+    Window unpartition into original sequences and removing padding.
+    Args:
+        windows (tensor): input tokens with [B * num_windows, window_size, window_size, C].
+        window_size (int): window size.
+        pad_hw (Tuple): padded height and width (Hp, Wp).
+        hw (Tuple): original height and width (H, W) before padding.
+
+    Returns:
+        x: unpartitioned sequences with [B, H, W, C].
+    """
+    Hp, Wp = pad_hw
+    H, W = hw
+    B = windows.shape[0] // (Hp * Wp // window_size // window_size)
+    x = windows.view(
+        B, Hp // window_size, Wp // window_size, window_size, window_size, -1
+    )
+    x = x.permute(0, 1, 3, 2, 4, 5).contiguous().view(B, Hp, Wp, -1)
+
+    if Hp > H or Wp > W:
+        x = x[:, :H, :W, :].contiguous()
+    return x
+
+
+def get_rel_pos(q_size: int, k_size: int, rel_pos: torch.Tensor) -> torch.Tensor:
+    """
+    Get relative positional embeddings according to the relative positions of
+        query and key sizes.
+    Args:
+        q_size (int): size of query q.
+        k_size (int): size of key k.
+        rel_pos (Tensor): relative position embeddings (L, C).
+
+    Returns:
+        Extracted positional embeddings according to relative positions.
+    """
+    max_rel_dist = int(2 * max(q_size, k_size) - 1)
+    # Interpolate rel pos if needed.
+    if rel_pos.shape[0] != max_rel_dist:
+        # Interpolate rel pos.
+        rel_pos_resized = F.interpolate(
+            rel_pos.reshape(1, rel_pos.shape[0], -1).permute(0, 2, 1),
+            size=max_rel_dist,
+            mode="linear",
+        )
+        rel_pos_resized = rel_pos_resized.reshape(-1, max_rel_dist).permute(1, 0)
+    else:
+        rel_pos_resized = rel_pos
+
+    # Scale the coords with short length if shapes for q and k are different.
+    q_coords = torch.arange(q_size)[:, None] * max(k_size / q_size, 1.0)
+    k_coords = torch.arange(k_size)[None, :] * max(q_size / k_size, 1.0)
+    relative_coords = (q_coords - k_coords) + (k_size - 1) * max(q_size / k_size, 1.0)
+
+    return rel_pos_resized[relative_coords.long()]
+
+
+def add_decomposed_rel_pos(
+    attn: torch.Tensor,
+    q: torch.Tensor,
+    rel_pos_h: torch.Tensor,
+    rel_pos_w: torch.Tensor,
+    q_size: Tuple[int, int],
+    k_size: Tuple[int, int],
+) -> torch.Tensor:
+    """
+    Calculate decomposed Relative Positional Embeddings from :paper:`mvitv2`.
+    https://github.com/facebookresearch/mvit/blob/19786631e330df9f3622e5402b4a419a263a2c80/mvit/models/attention.py   # noqa B950
+    Args:
+        attn (Tensor): attention map.
+        q (Tensor): query q in the attention layer with shape (B, q_h * q_w, C).
+        rel_pos_h (Tensor): relative position embeddings (Lh, C) for height axis.
+        rel_pos_w (Tensor): relative position embeddings (Lw, C) for width axis.
+        q_size (Tuple): spatial sequence size of query q with (q_h, q_w).
+        k_size (Tuple): spatial sequence size of key k with (k_h, k_w).
+
+    Returns:
+        attn (Tensor): attention map with added relative positional embeddings.
+    """
+    q_h, q_w = q_size
+    k_h, k_w = k_size
+    Rh = get_rel_pos(q_h, k_h, rel_pos_h)
+    Rw = get_rel_pos(q_w, k_w, rel_pos_w)
+
+    B, _, dim = q.shape
+    r_q = q.reshape(B, q_h, q_w, dim)
+    rel_h = torch.einsum("bhwc,hkc->bhwk", r_q, Rh)
+    rel_w = torch.einsum("bhwc,wkc->bhwk", r_q, Rw)
+
+    attn = (
+        attn.view(B, q_h, q_w, k_h, k_w)
+        + rel_h[:, :, :, :, None]
+        + rel_w[:, :, :, None, :]
+    ).view(B, q_h * q_w, k_h * k_w)
+
+    return attn
+
+
+class PatchEmbed(nn.Module):
+    """
+    Image to Patch Embedding.
+    """
+
+    def __init__(
+        self,
+        kernel_size: Tuple[int, int] = (16, 16),
+        stride: Tuple[int, int] = (16, 16),
+        padding: Tuple[int, int] = (0, 0),
+        in_chans: int = 3,
+        embed_dim: int = 768,
+    ) -> None:
+        """
+        Args:
+            kernel_size (Tuple): kernel size of the projection layer.
+            stride (Tuple): stride of the projection layer.
+            padding (Tuple): padding size of the projection layer.
+            in_chans (int): Number of input image channels.
+            embed_dim (int): Patch embedding dimension.
+        """
+        super().__init__()
+
+        self.proj = nn.Conv2d(
+            in_chans, embed_dim, kernel_size=kernel_size, stride=stride, padding=padding
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.proj(x)
+        # B C H W -> B H W C
+        x = x.permute(0, 2, 3, 1)
+        return x
+
+
+@dataclass
+class SAMViTCfg:
+    image_size: Union[Tuple[int, int], int] = 1024
+    width: int = 1024
+    layers: int = 23
+    heads: int = 16
+    patch_size: int = 16
+    window_size: int = 14
+    prompt_embed_dim: int = 256
+    global_attn_indexes: Union[List[int], Tuple[int]] = (5, 11, 17, 23)
+    downsample_channels: Union[List[int], Tuple[int]] = (512, 1024)
+
+
+SAM_MODEL_CONFIG = {
+    "sam_vit_b": {
+        "width": 768,
+        "layers": 12,
+        "heads": 12,
+        "global_attn_indexes": [2, 5, 8, 11],
+        "downsample_channels": (),
+    },
+    "sam_b_downsample": {
+        "width": 768,
+        "layers": 12,
+        "heads": 12,
+        "global_attn_indexes": [2, 5, 8, 11],
+        "downsample_channels": (512, 1024),
+    },
+    "sam_vit_l": {
+        "width": 1024,
+        "layers": 24,
+        "heads": 16,
+        "global_attn_indexes": [5, 11, 17, 23],
+        "downsample_channels": (),
+    },
+    "sam_vit_h": {
+        "width": 1280,
+        "layers": 32,
+        "heads": 16,
+        "global_attn_indexes": [7, 15, 23, 31],
+        "downsample_channels": (),
+    },
+}
+
+
+def create_sam_vit(
+    model_name: str = "sam_b_downsample",
+    image_size: int = 1024,
+    ckpt_path: str = "",
+    **kwargs,
+):
+    assert (
+        model_name in SAM_MODEL_CONFIG.keys()
+    ), f"model name: {model_name} should be in {SAM_MODEL_CONFIG.keys()}"
+
+    sam_cfg = SAMViTCfg(**SAM_MODEL_CONFIG[model_name])
+    image_encoder = ImageEncoderViT(
+        depth=sam_cfg.layers,
+        embed_dim=sam_cfg.width,
+        img_size=image_size,
+        mlp_ratio=4,
+        norm_layer=partial(torch.nn.LayerNorm, eps=1e-6),
+        num_heads=sam_cfg.heads,
+        patch_size=sam_cfg.patch_size,
+        qkv_bias=True,
+        use_rel_pos=True,
+        global_attn_indexes=sam_cfg.global_attn_indexes,
+        window_size=14,
+        out_chans=sam_cfg.prompt_embed_dim,
+        downsample_channels=sam_cfg.downsample_channels,
+    )
+
+    if ckpt_path:
+        state_dict = torch.load(ckpt_path)
+        image_encoder.load_state_dict(state_dict, strict=False)
+        print(f"SAM-ViT restores from {ckpt_path}")
+
+    return image_encoder
+
+
+if __name__ == "__main__":
+    x = torch.zeros(2, 3, 1024, 1024).bfloat16()
+    # x.permute(0, 3, 1, 2)
+    net = create_sam_vit().bfloat16()
+    out = net(x)
+    print(x.shape, out.shape)
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py
new file mode 100644
index 0000000000000000000000000000000000000000..a0bee4e45028971772177a0acac31f18a03ce6bd
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py
@@ -0,0 +1,663 @@
+# Copyright (c) 2023-2024 DeepSeek.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of
+# this software and associated documentation files (the "Software"), to deal in
+# the Software without restriction, including without limitation the rights to
+# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+# the Software, and to permit persons to whom the Software is furnished to do so,
+# subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+# https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/vision_transformer.py
+import math
+import warnings
+from dataclasses import dataclass
+from functools import partial
+from typing import (Callable, Dict, Final, List, Literal, Optional, Sequence, Set, Tuple, Type,
+                    Union)
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from timm.layers import (AttentionPoolLatent, DropPath, LayerType, Mlp, PatchDropout, PatchEmbed,
+                         resample_abs_pos_embed)
+from timm.models._manipulate import checkpoint_seq, named_apply
+
+
+def _no_grad_trunc_normal_(tensor, mean, std, a, b):
+    # Cut & paste from PyTorch official master until it's in a few official releases - RW
+    # Method based on https://people.sc.fsu.edu/~jburkardt/presentations/truncated_normal.pdf
+    def norm_cdf(x):
+        # Computes standard normal cumulative distribution function
+        return (1.0 + math.erf(x / math.sqrt(2.0))) / 2.0
+
+    if (mean < a - 2 * std) or (mean > b + 2 * std):
+        warnings.warn(
+            "mean is more than 2 std from [a, b] in nn.init.trunc_normal_. "
+            "The distribution of values may be incorrect.",
+            stacklevel=2,
+        )
+
+    with torch.no_grad():
+        # Values are generated by using a truncated uniform distribution and
+        # then using the inverse CDF for the normal distribution.
+        # Get upper and lower cdf values
+        l = norm_cdf((a - mean) / std)  # noqa: E741
+        u = norm_cdf((b - mean) / std)
+
+        # Uniformly fill tensor with values from [l, u], then translate to
+        # [2l-1, 2u-1].
+        tensor.uniform_(2 * l - 1, 2 * u - 1)
+
+        # Use inverse cdf transform for normal distribution to get truncated
+        # standard normal
+        tensor.erfinv_()
+
+        # Transform to proper mean, std
+        tensor.mul_(std * math.sqrt(2.0))
+        tensor.add_(mean)
+
+        # Clamp to ensure it's in the proper range
+        tensor.clamp_(min=a, max=b)
+        return tensor
+
+
+def trunc_normal_(tensor, mean=0.0, std=1.0, a=-2.0, b=2.0):
+    # type: (torch.Tensor, float, float, float, float) -> torch.Tensor
+    r"""The original timm.models.layers.weight_init.trunc_normal_ can not handle bfloat16 yet, here we first
+    convert the tensor to float32, apply the trunc_normal_() in float32, and then convert it back to its orignal dtype.
+    Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn
+    from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)`
+    with values outside :math:`[a, b]` redrawn until they are within
+    the bounds. The method used for generating the random values works
+    best when :math:`a \leq \text{mean} \leq b`.
+    Args:
+        tensor: an n-dimensional `torch.Tensor`
+        mean: the mean of the normal distribution
+        std: the standard deviation of the normal distribution
+        a: the minimum cutoff value
+        b: the maximum cutoff value
+    Examples:
+        >>> w = torch.empty(3, 5)
+        >>> nn.init.trunc_normal_(w)
+    """
+
+    with torch.no_grad():
+        dtype = tensor.dtype
+        tensor_fp32 = tensor.float()
+        tensor_fp32 = _no_grad_trunc_normal_(tensor_fp32, mean, std, a, b)
+        tensor_dtype = tensor_fp32.to(dtype=dtype)
+        tensor.copy_(tensor_dtype)
+
+
+def init_weights(self):
+    if self.pos_embed is not None:
+        trunc_normal_(self.pos_embed, std=self.pos_embed.shape[1] ** -0.5)
+    trunc_normal_(self.latent, std=self.latent_dim**-0.5)
+
+
+def init_weights_vit_timm(module: nn.Module, name: str = "") -> None:
+    """ViT weight initialization, original timm impl (for reproducibility)"""
+    if isinstance(module, nn.Linear):
+        trunc_normal_(module.weight, std=0.02)
+        if module.bias is not None:
+            nn.init.zeros_(module.bias)
+    elif hasattr(module, "init_weights"):
+        module.init_weights()
+
+
+class Attention(nn.Module):
+    fused_attn: Final[bool]
+
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int = 8,
+        qkv_bias: bool = False,
+        qk_norm: bool = False,
+        attn_drop: float = 0.0,
+        proj_drop: float = 0.0,
+        norm_layer: nn.Module = nn.LayerNorm,
+    ) -> None:
+        super().__init__()
+        assert dim % num_heads == 0, "dim should be divisible by num_heads"
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.scale = self.head_dim**-0.5
+        # self.fused_attn = use_fused_attn()
+        self.fused_attn = True
+
+        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
+        self.q_norm = norm_layer(self.head_dim) if qk_norm else nn.Identity()
+        self.k_norm = norm_layer(self.head_dim) if qk_norm else nn.Identity()
+        self.attn_drop = nn.Dropout(attn_drop)
+        self.proj = nn.Linear(dim, dim)
+        self.proj_drop = nn.Dropout(proj_drop) if proj_drop > 0.0 else nn.Identity()
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        B, N, C = x.shape
+        qkv = (
+            self.qkv(x)
+            .reshape(B, N, 3, self.num_heads, self.head_dim)
+            .permute(2, 0, 3, 1, 4)
+        )
+        q, k, v = qkv.unbind(0)
+        q, k = self.q_norm(q), self.k_norm(k)
+
+        if self.fused_attn:
+            x = F.scaled_dot_product_attention(
+                q,
+                k,
+                v,
+                dropout_p=self.attn_drop.p if self.training else 0.0,
+            )
+        else:
+            q = q * self.scale
+            attn = q @ k.transpose(-2, -1)
+            attn = attn.softmax(dim=-1)
+            attn = self.attn_drop(attn)
+            x = attn @ v
+
+        x = x.transpose(1, 2).reshape(B, N, C)
+        x = self.proj(x)
+        x = self.proj_drop(x)
+        return x
+
+
+class LayerScale(nn.Module):
+    def __init__(
+        self,
+        dim: int,
+        init_values: float = 1e-5,
+        inplace: bool = False,
+    ) -> None:
+        super().__init__()
+        self.inplace = inplace
+        self.gamma = nn.Parameter(init_values * torch.ones(dim))
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return x.mul_(self.gamma) if self.inplace else x * self.gamma
+
+
+class Block(nn.Module):
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int,
+        mlp_ratio: float = 4.0,
+        qkv_bias: bool = False,
+        qk_norm: bool = False,
+        proj_drop: float = 0.0,
+        attn_drop: float = 0.0,
+        init_values: Optional[float] = None,
+        drop_path: float = 0.0,
+        act_layer: nn.Module = nn.GELU,
+        norm_layer: nn.Module = nn.LayerNorm,
+        mlp_layer: nn.Module = Mlp,
+    ) -> None:
+        super().__init__()
+        self.norm1 = norm_layer(dim)
+        self.attn = Attention(
+            dim,
+            num_heads=num_heads,
+            qkv_bias=qkv_bias,
+            qk_norm=qk_norm,
+            attn_drop=attn_drop,
+            proj_drop=proj_drop,
+            norm_layer=norm_layer,
+        )
+        self.ls1 = (
+            LayerScale(dim, init_values=init_values) if init_values else nn.Identity()
+        )
+        self.drop_path1 = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+
+        self.norm2 = norm_layer(dim)
+        self.mlp = mlp_layer(
+            in_features=dim,
+            hidden_features=int(dim * mlp_ratio),
+            act_layer=act_layer,
+            drop=proj_drop,
+        )
+        self.ls2 = (
+            LayerScale(dim, init_values=init_values) if init_values else nn.Identity()
+        )
+        self.drop_path2 = DropPath(drop_path) if drop_path > 0.0 else nn.Identity()
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = x + self.drop_path1(self.ls1(self.attn(self.norm1(x))))
+        x = x + self.drop_path2(self.ls2(self.mlp(self.norm2(x))))
+        return x
+
+
+class VisionTransformer(nn.Module):
+    """Vision Transformer
+
+    A PyTorch impl of : `An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale`
+        - https://arxiv.org/abs/2010.11929
+    """
+
+    dynamic_img_size: Final[bool]
+
+    def __init__(
+        self,
+        img_size: Union[int, Tuple[int, int]] = 224,
+        patch_size: Union[int, Tuple[int, int]] = 16,
+        in_chans: int = 3,
+        num_classes: int = 1000,
+        global_pool: Literal["", "avg", "token", "map"] = "token",
+        embed_dim: int = 768,
+        depth: int = 12,
+        num_heads: int = 12,
+        mlp_ratio: float = 4.0,
+        qkv_bias: bool = True,
+        qk_norm: bool = False,
+        init_values: Optional[float] = None,
+        class_token: bool = True,
+        no_embed_class: bool = False,
+        reg_tokens: int = 0,
+        pre_norm: bool = False,
+        fc_norm: Optional[bool] = None,
+        dynamic_img_size: bool = False,
+        dynamic_img_pad: bool = False,
+        drop_rate: float = 0.0,
+        pos_drop_rate: float = 0.0,
+        patch_drop_rate: float = 0.0,
+        proj_drop_rate: float = 0.0,
+        attn_drop_rate: float = 0.0,
+        drop_path_rate: float = 0.0,
+        weight_init: Literal["skip", "jax", "jax_nlhb", "moco", ""] = "",
+        embed_layer: Callable = PatchEmbed,
+        norm_layer: Optional[LayerType] = None,
+        act_layer: Optional[LayerType] = None,
+        block_fn: Type[nn.Module] = Block,
+        mlp_layer: Type[nn.Module] = Mlp,
+        ignore_head: bool = False,
+    ) -> None:
+        """
+        Args:
+            img_size: Input image size.
+            patch_size: Patch size.
+            in_chans: Number of image input channels.
+            num_classes: Mumber of classes for classification head.
+            global_pool: Type of global pooling for final sequence (default: 'token').
+            embed_dim: Transformer embedding dimension.
+            depth: Depth of transformer.
+            num_heads: Number of attention heads.
+            mlp_ratio: Ratio of mlp hidden dim to embedding dim.
+            qkv_bias: Enable bias for qkv projections if True.
+            init_values: Layer-scale init values (layer-scale enabled if not None).
+            class_token: Use class token.
+            no_embed_class: Don't include position embeddings for class (or reg) tokens.
+            reg_tokens: Number of register tokens.
+            fc_norm: Pre head norm after pool (instead of before), if None, enabled when global_pool == 'avg'.
+            drop_rate: Head dropout rate.
+            pos_drop_rate: Position embedding dropout rate.
+            attn_drop_rate: Attention dropout rate.
+            drop_path_rate: Stochastic depth rate.
+            weight_init: Weight initialization scheme.
+            embed_layer: Patch embedding layer.
+            norm_layer: Normalization layer.
+            act_layer: MLP activation layer.
+            block_fn: Transformer block layer.
+        """
+        super().__init__()
+        assert global_pool in ("", "avg", "token", "map")
+        assert class_token or global_pool != "token"
+        use_fc_norm = global_pool == "avg" if fc_norm is None else fc_norm
+        # norm_layer = get_norm_layer(norm_layer) or partial(nn.LayerNorm, eps=1e-6)
+        # act_layer = get_act_layer(act_layer) or nn.GELU
+        norm_layer = partial(nn.LayerNorm, eps=1e-6)
+        act_layer = nn.GELU
+
+        self.num_classes = num_classes
+        self.global_pool = global_pool
+        self.num_features = self.embed_dim = (
+            embed_dim  # num_features for consistency with other models
+        )
+        self.num_prefix_tokens = 1 if class_token else 0
+        self.num_prefix_tokens += reg_tokens
+        self.num_reg_tokens = reg_tokens
+        self.has_class_token = class_token
+        self.no_embed_class = (
+            no_embed_class  # don't embed prefix positions (includes reg)
+        )
+        self.dynamic_img_size = dynamic_img_size
+        self.grad_checkpointing = False
+        self.ignore_head = ignore_head
+
+        embed_args = {}
+        if dynamic_img_size:
+            # flatten deferred until after pos embed
+            embed_args.update(dict(strict_img_size=False, output_fmt="NHWC"))
+        self.patch_embed = embed_layer(
+            img_size=img_size,
+            patch_size=patch_size,
+            in_chans=in_chans,
+            embed_dim=embed_dim,
+            bias=not pre_norm,  # disable bias if pre-norm is used (e.g. CLIP)
+            dynamic_img_pad=dynamic_img_pad,
+            **embed_args,
+        )
+        num_patches = self.patch_embed.num_patches
+
+        self.cls_token = (
+            nn.Parameter(torch.zeros(1, 1, embed_dim)) if class_token else None
+        )
+        self.reg_token = (
+            nn.Parameter(torch.zeros(1, reg_tokens, embed_dim)) if reg_tokens else None
+        )
+        embed_len = (
+            num_patches if no_embed_class else num_patches + self.num_prefix_tokens
+        )
+        self.pos_embed = nn.Parameter(torch.randn(1, embed_len, embed_dim) * 0.02)
+        self.pos_drop = nn.Dropout(p=pos_drop_rate)
+        if patch_drop_rate > 0:
+            self.patch_drop = PatchDropout(
+                patch_drop_rate,
+                num_prefix_tokens=self.num_prefix_tokens,
+            )
+        else:
+            self.patch_drop = nn.Identity()
+        self.norm_pre = norm_layer(embed_dim) if pre_norm else nn.Identity()
+
+        dpr = [
+            x.item() for x in torch.linspace(0, drop_path_rate, depth)
+        ]  # stochastic depth decay rule
+        self.blocks = nn.Sequential(
+            *[
+                block_fn(
+                    dim=embed_dim,
+                    num_heads=num_heads,
+                    mlp_ratio=mlp_ratio,
+                    qkv_bias=qkv_bias,
+                    qk_norm=qk_norm,
+                    init_values=init_values,
+                    proj_drop=proj_drop_rate,
+                    attn_drop=attn_drop_rate,
+                    drop_path=dpr[i],
+                    norm_layer=norm_layer,
+                    act_layer=act_layer,
+                    mlp_layer=mlp_layer,
+                )
+                for i in range(depth)
+            ]
+        )
+        self.norm = norm_layer(embed_dim) if not use_fc_norm else nn.Identity()
+
+        # Classifier Head
+        if global_pool == "map":
+            AttentionPoolLatent.init_weights = init_weights
+            self.attn_pool = AttentionPoolLatent(
+                self.embed_dim,
+                num_heads=num_heads,
+                mlp_ratio=mlp_ratio,
+                norm_layer=norm_layer,
+            )
+        else:
+            self.attn_pool = None
+        self.fc_norm = norm_layer(embed_dim) if use_fc_norm else nn.Identity()
+        self.head_drop = nn.Dropout(drop_rate)
+        self.head = (
+            nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()
+        )
+
+        if weight_init != "skip":
+            self.init_weights(weight_init)
+
+    def init_weights(self, mode: Literal["jax", "jax_nlhb", "moco", ""] = "") -> None:
+        assert mode in ("jax", "jax_nlhb", "moco", "")
+        # head_bias = -math.log(self.num_classes) if "nlhb" in mode else 0.0
+        trunc_normal_(self.pos_embed, std=0.02)
+        if self.cls_token is not None:
+            nn.init.normal_(self.cls_token, std=1e-6)
+        named_apply(init_weights_vit_timm, self)
+
+    @torch.jit.ignore
+    def no_weight_decay(self) -> Set:
+        return {"pos_embed", "cls_token", "dist_token"}
+
+    @torch.jit.ignore
+    def group_matcher(self, coarse: bool = False) -> Dict:
+        return dict(
+            stem=r"^cls_token|pos_embed|patch_embed",  # stem and embed
+            blocks=[(r"^blocks\.(\d+)", None), (r"^norm", (99999,))],
+        )
+
+    @torch.jit.ignore
+    def set_grad_checkpointing(self, enable: bool = True) -> None:
+        self.grad_checkpointing = enable
+
+    @torch.jit.ignore
+    def get_classifier(self) -> nn.Module:
+        return self.head
+
+    def reset_classifier(self, num_classes: int, global_pool=None) -> None:
+        self.num_classes = num_classes
+        if global_pool is not None:
+            assert global_pool in ("", "avg", "token", "map")
+            if global_pool == "map" and self.attn_pool is None:
+                assert (
+                    False
+                ), "Cannot currently add attention pooling in reset_classifier()."
+            elif global_pool != "map " and self.attn_pool is not None:
+                self.attn_pool = None  # remove attention pooling
+            self.global_pool = global_pool
+        self.head = (
+            nn.Linear(self.embed_dim, num_classes) if num_classes > 0 else nn.Identity()
+        )
+
+    def _pos_embed(self, x: torch.Tensor) -> torch.Tensor:
+        if self.dynamic_img_size:
+            B, H, W, C = x.shape
+            pos_embed = resample_abs_pos_embed(
+                self.pos_embed,
+                (H, W),
+                num_prefix_tokens=0 if self.no_embed_class else self.num_prefix_tokens,
+            )
+            x = x.view(B, -1, C)
+        else:
+            pos_embed = self.pos_embed
+
+        to_cat = []
+        if self.cls_token is not None:
+            to_cat.append(self.cls_token.expand(x.shape[0], -1, -1))
+        if self.reg_token is not None:
+            to_cat.append(self.reg_token.expand(x.shape[0], -1, -1))
+
+        if self.no_embed_class:
+            # deit-3, updated JAX (big vision)
+            # position embedding does not overlap with class token, add then concat
+            x = x + pos_embed
+            if to_cat:
+                x = torch.cat(to_cat + [x], dim=1)
+        else:
+            # original timm, JAX, and deit vit impl
+            # pos_embed has entry for class token, concat then add
+            if to_cat:
+                x = torch.cat(to_cat + [x], dim=1)
+            x = x + pos_embed
+
+        return self.pos_drop(x)
+
+    def _intermediate_layers(
+        self,
+        x: torch.Tensor,
+        n: Union[int, Sequence] = 1,
+    ) -> List[torch.Tensor]:
+        outputs, num_blocks = [], len(self.blocks)
+        take_indices = set(
+            range(num_blocks - n, num_blocks) if isinstance(n, int) else n
+        )
+
+        # forward pass
+        x = self.patch_embed(x)
+        x = self._pos_embed(x)
+        x = self.patch_drop(x)
+        x = self.norm_pre(x)
+        for i, blk in enumerate(self.blocks):
+            x = blk(x)
+            if i in take_indices:
+                outputs.append(x)
+
+        return outputs
+
+    def get_intermediate_layers(
+        self,
+        x: torch.Tensor,
+        n: Union[int, Sequence] = 1,
+        reshape: bool = False,
+        return_prefix_tokens: bool = False,
+        norm: bool = False,
+    ) -> Tuple[Union[torch.Tensor, Tuple[torch.Tensor]]]:
+        """Intermediate layer accessor (NOTE: This is a WIP experiment).
+        Inspired by DINO / DINOv2 interface
+        """
+        # take last n blocks if n is an int, if in is a sequence, select by matching indices
+        outputs = self._intermediate_layers(x, n)
+        if norm:
+            outputs = [self.norm(out) for out in outputs]
+        prefix_tokens = [out[:, 0 : self.num_prefix_tokens] for out in outputs]
+        outputs = [out[:, self.num_prefix_tokens :] for out in outputs]
+
+        if reshape:
+            grid_size = self.patch_embed.grid_size
+            outputs = [
+                out.reshape(x.shape[0], grid_size[0], grid_size[1], -1)
+                .permute(0, 3, 1, 2)
+                .contiguous()
+                for out in outputs
+            ]
+
+        if return_prefix_tokens:
+            return tuple(zip(outputs, prefix_tokens))
+        return tuple(outputs)
+
+    def forward_features(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.patch_embed(x)
+        x = self._pos_embed(x)
+        x = self.patch_drop(x)
+        x = self.norm_pre(x)
+        if self.grad_checkpointing and not torch.jit.is_scripting():
+            x = checkpoint_seq(self.blocks, x)
+        else:
+            x = self.blocks(x)
+        x = self.norm(x)
+        return x
+
+    def forward_head(self, x: torch.Tensor, pre_logits: bool = False) -> torch.Tensor:
+        if self.attn_pool is not None:
+            x = self.attn_pool(x)
+        elif self.global_pool == "avg":
+            x = x[:, self.num_prefix_tokens :].mean(dim=1)
+        elif self.global_pool:
+            x = x[:, 0]  # class token
+        x = self.fc_norm(x)
+        x = self.head_drop(x)
+        return x if pre_logits else self.head(x)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.forward_features(x)
+        if not self.ignore_head:
+            x = self.forward_head(x)
+        return x
+
+
+@dataclass
+class SigLIPVisionCfg:
+    width: int = 1152
+    layers: Union[Tuple[int, int, int, int], int] = 27
+    heads: int = 16
+    patch_size: int = 14
+    image_size: Union[Tuple[int, int], int] = 336
+    global_pool: str = "map"
+    mlp_ratio: float = 3.7362
+    class_token: bool = False
+    num_classes: int = 0
+    use_checkpoint: bool = False
+
+
+SigLIP_MODEL_CONFIG = {
+    "siglip_so400m_patch14_384": {
+        "image_size": 336,
+        "patch_size": 14,
+        "width": 1152,
+        "layers": 27,
+        "heads": 16,
+        "mlp_ratio": 3.7362,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+    "siglip_so400m_patch14_224": {
+        "image_size": 224,
+        "patch_size": 14,
+        "width": 1152,
+        "layers": 27,
+        "heads": 16,
+        "mlp_ratio": 3.7362,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+    "siglip_large_patch16_384": {
+        "image_size": 384,
+        "patch_size": 16,
+        "width": 1024,
+        "layers": 24,
+        "heads": 16,
+        "mlp_ratio": 4,
+        "global_pool": "map",
+        "use_checkpoint": False,
+    },
+}
+
+
+def create_siglip_vit(
+    model_name: str = "siglip_so400m_patch14_384",
+    image_size: int = 384,
+    select_layer: int = -1,
+    ckpt_path: str = "",
+    **kwargs,
+):
+    assert (
+        model_name in SigLIP_MODEL_CONFIG.keys()
+    ), f"model name should be in {SigLIP_MODEL_CONFIG.keys()}"
+
+    vision_cfg = SigLIPVisionCfg(**SigLIP_MODEL_CONFIG[model_name])
+
+    if select_layer <= 0:
+        layers = min(vision_cfg.layers, vision_cfg.layers + select_layer + 1)
+    else:
+        layers = min(vision_cfg.layers, select_layer)
+
+    model = VisionTransformer(
+        img_size=image_size,
+        patch_size=vision_cfg.patch_size,
+        embed_dim=vision_cfg.width,
+        depth=layers,
+        num_heads=vision_cfg.heads,
+        mlp_ratio=vision_cfg.mlp_ratio,
+        class_token=vision_cfg.class_token,
+        global_pool=vision_cfg.global_pool,
+        ignore_head=kwargs.get("ignore_head", True),
+        weight_init=kwargs.get("weight_init", "skip"),
+        num_classes=0,
+    )
+
+    if ckpt_path:
+        state_dict = torch.load(ckpt_path, map_location="cpu")
+
+        incompatible_keys = model.load_state_dict(state_dict, strict=False)
+        print(
+            f"SigLIP-ViT restores from {ckpt_path},\n"
+            f"\tincompatible_keys:', {incompatible_keys}."
+        )
+
+    return model
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..c9780c22884a73442a510b924fd340a46db589d8
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/__init__.py
@@ -0,0 +1,2 @@
+from .valley2 import Valley2Chat  # noqa: F401
+from .valley3 import Valley3Chat  # noqa: F401
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/requirements_valley.txt b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/requirements_valley.txt
new file mode 100644
index 0000000000000000000000000000000000000000..009d124d93b41463a5ed4efbf5747921478e2601
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/requirements_valley.txt
@@ -0,0 +1,33 @@
+accelerate==1.1.0
+bert-score==0.3.13
+datasets==2.21.0
+einops==0.8.0
+evaluate==0.4.3
+fastapi==0.115.0
+flash-attn==2.7.3
+ftfy==6.2.3
+markdown2==2.5.0
+ninja==1.11.1.1
+nltk==3.9.1
+numpy==1.26.4
+omegaconf==2.3.0
+openai==1.70.0
+opencv-python-headless==4.10.0.84
+packaging==24.1
+pandas==2.2.2
+peft==0.5.0
+prettytable==3.11.0
+protobuf==3.20.3
+pyarrow==15.0.0
+pydantic==1.10.14
+qwen-vl-utils==0.0.8
+requests==2.32.3
+rouge-score==0.1.2
+scikit-image==0.24.0
+scikit-learn==1.5.2
+sentencepiece==0.1.97
+timm==1.0.13
+tokenizers==0.21.1
+torchmetrics==1.6.0
+transformers==4.51.2
+uvicorn==0.30.6
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/valley2.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/valley2.py
new file mode 100644
index 0000000000000000000000000000000000000000..7bdb52285e669f4d5c79a8681de5da786d918d10
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/valley2.py
@@ -0,0 +1,245 @@
+import logging
+import re
+import string
+
+import pandas as pd
+import torch
+from transformers import AutoModel, AutoProcessor, set_seed
+
+from vlmeval.dataset import DATASET_TYPE
+from ..base import BaseModel
+
+
+class Valley2Chat(BaseModel):
+    """
+    A multimodal chat model interface for the Valley2 architecture. (https://arxiv.org/abs/2501.05901)
+
+    Attributes:
+        - seed (int): Random seed used for reproducibility. Default is 42.
+        - torch_dtype (torch.dtype): Data type for model weights (e.g., torch.bfloat16). Default is bfloat16.
+        - model_path (str): Hugging Face model path or local checkpoint directory for loading
+            the Valley2-DPO model. Default is 'bytedance-research/Valley2-DPO'.
+        - cot_output_trunc (bool): Whether to truncate model output by extracting the part between
+            <CONCLUSION> and </CONCLUSION> tags, if present. This is useful for structured
+            chain-of-thought outputs. Default is False.
+        - max_new_tokens (int): Maximum number of tokens to be generated during inference. Default is 2048.
+        - min_pixels (int): Minimum number of image pixels allowed after preprocessing. Default is 1280*28*28.
+        - max_pixels (int): Maximum number of image pixels allowed after preprocessing. Default is 16384*28*28.
+        - use_llava_cot_prompt (bool): Whether to enable LLaVA-CoT (Chain-of-Thought) prompting. If enabled,
+            "Please think step by step." is appended to the prompt to trigger detailed reasoning-style
+            outputs as in LLaVA-CoT (https://arxiv.org/abs/2411.10440). Default is False.
+        - use_custom_prompt (bool): Whether to enable custom prompts tailored to specific benchmarks. These prompts
+            have been manually optimized for individual datasets. Can be combined with `use_llava_cot_prompt`.
+            Default is True.
+        - use_custom_pixels (bool): Whether to enable benchmark-specific custom image pixel limits during
+            preprocessing. Some datasets benefit from lower or higher resolution constraints. Default is True.
+    """
+
+    def __init__(self,
+                 model_path='bytedance-research/Valley2-DPO',
+                 cot_output_trunc=False,
+                 max_new_tokens: int = 2048,
+                 seed=42,
+                 torch_dtype=torch.float16,
+                 min_pixels=1280 * 28 * 28,
+                 max_pixels=16384 * 28 * 28,
+                 use_llava_cot_prompt=False,
+                 use_custom_prompt=True,
+                 use_custom_pixels_limit=True,
+                 **kwargs):
+        set_seed(seed)
+        self.torch_dtype = torch_dtype
+        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+        logging.info(f"Start loading valley model from {model_path}")
+        self.model = AutoModel.from_pretrained(model_path, torch_dtype=self.torch_dtype, trust_remote_code=True)
+        self.model = self.model.to(self.device).half()
+        self.processor = AutoProcessor.from_pretrained(
+            model_path,
+            anyres=self.model.config.anyres,
+            max_pixels=max_pixels,
+            min_pixels=min_pixels,
+            trust_remote_code=True
+        )
+
+        self._use_llava_cot_prompt = use_llava_cot_prompt
+        self._use_custom_prompt = use_custom_prompt
+        self._use_custom_pixels_limit = use_custom_pixels_limit
+        self.cot_output_trunc = cot_output_trunc
+        self.max_pixels = max_pixels
+        self.min_pixels = min_pixels
+
+        kwargs_default = dict(do_sample=False, max_new_tokens=max_new_tokens, repetition_penalty=1.0) # noqa E501
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+
+    def use_custom_prompt(self, dataset):
+        if not self._use_custom_prompt:
+            return False
+        if any(dataset.startswith(prefix) for prefix in ['TextVQA', 'ChartQA', 'InfoVQA', 'DocVQA', 'CMMMU']):
+            return True
+        if any(dataset.startswith(prefix) for prefix in ['MMVet', 'MathVista', 'MathVerse', 'MathVision']):
+            return True
+        # if DATASET_TYPE(dataset) == 'Y/N' or DATASET_TYPE(dataset) == 'MCQ':
+        #     return True
+        return False
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += '\nAnswer the question using a single word or phrase.'
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+
+        if len(options):
+            prompt += "\nAnswer with the option's letter from the given choices directly."
+
+        return prompt
+
+    def build_mmvet_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += "\nProvide a step-by-step solution to the problem carefully."
+        return prompt
+
+    def build_math_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += (
+            "\nProvide a step-by-step solution to the problem, "
+            "and conclude with 'the answer is' followed by the final solution."
+        )
+        return prompt
+
+    def build_ocr_cmmmu_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += "\nPlease try to answer the question with short words or phrases if possible."
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        if dataset == 'MMVet':
+            prompt = self.build_mmvet_prompt(line, dataset)
+        elif any(dataset.startswith(prefix) for prefix in ('TextVQA', 'ChartQA', 'InfoVQA', 'DocVQA', 'CMMMU')):
+            prompt = self.build_ocr_cmmmu_prompt(line, dataset)
+        elif any(dataset.startswith(prefix) for prefix in ('MathVista', 'MathVerse', 'MathVision')):
+            prompt = self.build_math_prompt(line, dataset)
+        # elif DATASET_TYPE(dataset) == 'Y/N':
+        #     prompt = self.build_yorn_prompt(line, dataset)
+        # elif DATASET_TYPE(dataset) == 'MCQ':
+        #     prompt = self.build_multi_choice_prompt(line, dataset)
+        else:
+            raise RuntimeError(f'Invalid dataset type: {DATASET_TYPE(dataset)}')
+
+        # add image to message
+        message = []
+        if isinstance(tgt_path, list):
+            message = [dict(type='image', value=p) for p in tgt_path]
+        else:
+            message = [dict(type='image', value=tgt_path)]
+
+        # add text to message
+        message.append(dict(type='text', value=prompt))
+        return message
+
+    def post_process(self, generation_text, dataset=None):
+        if any(dataset.startswith(prefix) for prefix in ('TextVQA', 'ChartQA', 'InfoVQA', 'DocVQA', 'CMMMU')):
+            self.cot_output_trunc = True  # froce truncate cot output
+
+        if self.cot_output_trunc:
+            pattern = r"(?:<CONCLUSION>)([\s\S]*?)(?:</CONCLUSION>)"
+            match = re.search(pattern, generation_text)
+            if match:
+                generation_text = match.group(1)
+
+        if any(dataset.startswith(prefix) for prefix in ('TextVQA', 'ChartQA', 'InfoVQA', 'DocVQA', 'CMMMU')):
+            generation_text = generation_text.strip()
+            if generation_text.endswith('.'):
+                generation_text = generation_text[:-1]
+        return generation_text
+
+    def get_pixels_limit(self, dataset=None):
+        if self._use_custom_pixels_limit and \
+                any(dataset.startswith(prefix) for prefix in ['OCRBench', 'HallusionBench', 'MMMU', 'MMBench']):
+            min_pixels = 100 * 28 * 28
+            max_pixels = 1280 * 28 * 28
+        else:
+            min_pixels = self.min_pixels
+            max_pixels = self.max_pixels
+        return min_pixels, max_pixels
+
+    def generate_inner(self, message, dataset=None):
+        # Construct messages and images
+        messages = []
+        text, images = '', []
+        for item in message:
+            if item['type'] == 'text':
+                text += item['value']
+            elif item['type'] == 'image':
+                text += ' <image> '
+                images.append(item['value'])
+
+        # Add CoT prompt, trigger the llava-cot output format
+        LLAVA_COT_PROMPT = "\nPlease think step by step."
+        if self._use_llava_cot_prompt:
+            messages.append({"role": 'user', "content": text + LLAVA_COT_PROMPT})
+        elif self._use_custom_prompt and dataset in ["MMMU_DEV_VAL", "MMStar", "OCRBench", "MMVet"]:
+            messages.append({"role": 'user', "content": text + LLAVA_COT_PROMPT})
+        else:
+            messages.append({"role": 'user', "content": text})
+
+        # Preprocess the messages and images
+        min_pixels, max_pixels = self.get_pixels_limit(dataset)
+        data_dict = self.processor(
+            {
+                "conversations": messages,
+                "images": images
+            },
+            min_pixels=min_pixels,
+            max_pixels=max_pixels
+        )
+
+        # Inference
+        with torch.inference_mode():
+            self.model.to(dtype=self.torch_dtype, device=self.device)
+
+            if type(data_dict["images"][0]) is list:
+                images = [
+                    [item.to(dtype=self.torch_dtype, device=self.device) for item in img]
+                    for img in data_dict["images"]
+                ]
+            else:
+                images = [img.to(dtype=self.torch_dtype, device=self.device) for img in data_dict["images"]]
+
+            output_ids = self.model.generate(
+                input_ids=data_dict["input_ids"].to(self.device),
+                images=images,
+                image_sizes=data_dict["image_sizes"],
+                pixel_values=data_dict["pixel_values"].to(dtype=self.torch_dtype, device=self.device),
+                image_grid_thw=data_dict["image_grid_thw"].to(self.device),
+                do_sample=self.kwargs["do_sample"],
+                max_new_tokens=self.kwargs["max_new_tokens"],
+                repetition_penalty=self.kwargs["repetition_penalty"],
+                return_dict_in_generate=True,
+                output_scores=True,
+            )
+        input_token_len = data_dict["input_ids"].shape[1]
+        generation_text = self.processor.batch_decode(output_ids.sequences[:, input_token_len:])[0]
+        generation_text = generation_text.replace("<|im_end|>", "")
+
+        # Postprocess
+        generation_text = self.post_process(generation_text, dataset)
+        return generation_text
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/valley3.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/valley3.py
new file mode 100644
index 0000000000000000000000000000000000000000..e9796ed66497cf7bbcadf901067357e841fabea5
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/valley/valley3.py
@@ -0,0 +1,418 @@
+import logging
+import re
+import string
+
+import pandas as pd
+import torch
+from transformers import AutoModel, AutoProcessor, set_seed
+
+from vlmeval.dataset import DATASET_TYPE
+from ..base import BaseModel
+
+GTHINKER_SYS_PROMPT = (
+    "A conversation between User and Assistant. The user asks a question, and the Assistant solves it. "
+    "The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. "
+    "The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, "
+    "respectively, i.e., <think> reasoning process here </think><answer> answer here </answer>. "
+    "In the reasoning process enclosed within <think> </think>, each specific visual cue is enclosed within "
+    "<vcues_*>...</vcues_*>, where * indicates the index of the specific cue. Before concluding the final answer, "
+    "pause for a quick consistency check: verify whether the visual cues support the reasoning "
+    "and whether each step logically follows from what is seen. If correct, conclude the answer; "
+    "otherwise, revise the visual cues and reasoning, then conclude."
+)
+
+
+class Valley3Chat(BaseModel):
+    """
+    A multimodal chat model interface for the Valley2 architecture. (https://arxiv.org/abs/2501.05901)
+
+    Attributes:
+        - seed (int): Random seed used for reproducibility. Default is 42.
+        - torch_dtype (torch.dtype): Data type for model weights (e.g., torch.bfloat16). Default is bfloat16.
+        - model_path (str): Hugging Face model path or local checkpoint directory for loading
+            the Valley2-DPO model. Default is 'bytedance-research/Valley2-DPO'.
+        - max_new_tokens (int): Maximum number of tokens to be generated during inference. Default is 2048.
+        - min_pixels (int): Minimum number of image pixels allowed after preprocessing. Default is 1280*28*28.
+        - max_pixels (int): Maximum number of image pixels allowed after preprocessing. Default is 16384*28*28.
+        - use_custom_prompt (bool): Whether to enable custom prompts tailored to specific benchmarks. These prompts
+            have been manually optimized for individual datasets.
+        - use_custom_pixels (bool): Whether to enable benchmark-specific custom image pixel limits during
+            preprocessing. Some datasets benefit from lower or higher resolution constraints. Default is True.
+    """
+
+    def __init__(
+        self,
+        model_path='bytedance-research/Valley3',
+        max_new_tokens: int = 2048,
+        seed=42,
+        torch_dtype=torch.bfloat16,
+        min_pixels=1280 * 28 * 28,
+        max_pixels=16384 * 28 * 28,
+        use_custom_prompt=True,
+        use_custom_pixels_limit=True,
+        use_gthinker_thinking=False,
+        **kwargs
+    ):
+        set_seed(seed)
+        self.torch_dtype = torch_dtype
+        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+        logging.info(f"Start loading valley model from {model_path}")
+        self.model = AutoModel.from_pretrained(model_path, torch_dtype=self.torch_dtype, trust_remote_code=True)
+        self.model = self.model.to(self.device)
+        self.processor = AutoProcessor.from_pretrained(
+            model_path,
+            anyres=self.model.config.anyres,
+            max_pixels=max_pixels,
+            min_pixels=min_pixels,
+            trust_remote_code=True
+        )
+
+        self._use_custom_prompt = use_custom_prompt
+        self._use_custom_pixels_limit = use_custom_pixels_limit
+        self._use_gthinker_thinking = use_gthinker_thinking
+        self.max_pixels = max_pixels
+        self.min_pixels = min_pixels
+        self.screen_parse = True
+
+        kwargs_default = dict(do_sample=False, max_new_tokens=max_new_tokens, repetition_penalty=1.0) # noqa E501
+        kwargs_default.update(kwargs)
+        self.kwargs = kwargs_default
+
+    def use_custom_prompt(self, dataset):
+        return self._use_custom_prompt
+
+    def build_yorn_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += '\nAnswer the question using a single word or phrase.'
+        return prompt
+
+    def build_yorn_prompt_gthinker(self, line, dataset=None):
+        prompt = line['question']
+        prompt += (
+            '\nAnswer this question and the answer enclosed by '
+            '<answer> </answer> should be a single word or phrase.'
+        )
+
+        return prompt
+
+    def build_multi_choice_prompt(self, line, dataset=None):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+
+        if len(options):
+            prompt += "\nAnswer with the option's letter from the given choices directly."
+
+        return prompt
+
+    def build_multi_choice_prompt_gthinker(self, line, dataset=None):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+
+        if len(options):
+            prompt += (
+                "\nAnswer the multiple-choice question. "
+                "The answer enclosed by <answer> </answer> should be "
+                "'$LETTER' (without quotes) where LETTER is one of the options. "
+            )
+
+        return prompt
+
+    def build_mmvet_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += "\nProvide a step-by-step solution to the problem carefully.\nPlease think step by step."
+        return prompt
+
+    def build_mmvet_prompt_gthinker(self, line, dataset=None):
+        prompt = line['question']
+        prompt += "\nProvide a step-by-step solution to the problem carefully.\nPlease think step by step."
+        return prompt
+
+    def build_math_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += (
+            "\nProvide a step-by-step solution to the problem, "
+            "and conclude with 'the answer is' followed by the final solution."
+        )
+        return prompt
+
+    def build_mathvista_prompt_gthinker(self, line, dataset=None):
+        prompt = line["question"]
+        type_ = line["question_type"]
+        if type_ == "multi_choice":
+            prompt = prompt + (
+                "\nAnswer the multiple-choice question. The answer enclosed by <answer> </answer> "
+                "should be '$LETTER' (without quotes) where LETTER is one of the options."
+            )
+        elif type_ == "free_form":
+            prompt = prompt + (
+                "\nAnswer this question and the answer enclosed by <answer> </answer> "
+                "should be single word or phrase as indicated by the hint."
+            )
+        else:
+            raise AssertionError
+        # return msgs
+        return prompt
+
+    def build_mathvision_prompt_gthinker(self, line, dataset=None):
+
+        prompt = line["question"]
+        hint = prompt.split("\n")[0]
+        if hint == (
+            "Hint: Please answer the question and provide the correct option letter,"
+            " e.g., A, B, C, D, at the end."
+        ):
+            prompt = prompt.replace(hint, "") + (
+                "\nAnswer the multiple-choice question. The answer enclosed by <answer> </answer> "
+                "should be '$LETTER' (without quotes) where LETTER is one of the options."
+            )
+            prompt = prompt.replace("(A)", "A.").replace("(B)", "B.").replace("(C)", "C.")
+            prompt = prompt.replace("(D)", "D.").replace("(E)", "E.")
+        elif hint == 'Hint: Please answer the question and provide the final answer at the end.':
+            prompt = prompt.replace(hint, "") + (
+                "\nAnswer this question and the answer enclosed by <answer> </answer> should be single word or phrase."
+            )
+        else:
+            raise AssertionError
+        return prompt
+
+    def build_ocr_cmmmu_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += "\nPlease try to answer the question with short words or phrases if possible."
+        return prompt
+
+    def build_ocrbench_prompt(self, line, dataset=None):
+        prompt = line['question']
+        prompt += "\nPlease think step by step."
+        return prompt
+
+    def build_ocrbench_prompt_gthinker(self, line, dataset=None):
+        prompt = line["question"]
+
+        if line["category"] == "Handwritten Mathematical Expression Recognition":
+            prompt += (
+                "\nAnswer this question withou solving the math problem and the answer enclosed by <answer> </answer>"
+                " should be the OCR result without missing any character."
+            )
+        else:
+            prompt += (
+                "\nAnswer this question and the answer enclosed by <answer> </answer> "
+                "should be the OCR result without missing any character."
+            )
+        return prompt
+
+    def build_m3cot_prompt(self, line, dataset=None, use_cot=False):
+        prompt = line['question']
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        if hint is not None:
+            prompt = hint + '\n' + prompt
+
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        for key, item in options.items():
+            prompt += f'\n{key}. {item}'
+        prompt += (
+            "\nProvide a step-by-step solution to the problem, and conclude with 'the answer is' followed by "
+            "the '$LETTER' (without quotes) where LETTER is one of the options."
+        )
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert self.use_custom_prompt(dataset)
+        assert isinstance(dataset, str)
+        tgt_path = self.dump_image(line, dataset)
+
+        # ========== thinking ========== #
+        if self._use_gthinker_thinking:
+            if any(dataset.startswith(prefix) for prefix in (
+                'MMStar', 'AI2D_TEST', 'AI2D_TEST_NO_MASK', 'MMMU', 'MMBench', 'M3CoT', 'MME-RealWorld'
+            )):
+                prompt = self.build_multi_choice_prompt_gthinker(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('HallusionBench',)):
+                prompt = self.build_yorn_prompt_gthinker(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('MathVista_MINI',)):
+                prompt = self.build_mathvista_prompt_gthinker(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('MathVision',)):
+                prompt = self.build_mathvision_prompt_gthinker(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('MMVet',)):
+                prompt = self.build_mmvet_prompt_gthinker(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('OCRBench',)):
+                prompt = self.build_ocrbench_prompt_gthinker(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('ScreenSpot',)):
+                prompt = self.build_screen_prompt_gthinker(line, dataset)
+            elif DATASET_TYPE(dataset) == 'Y/N':
+                prompt = self.build_yorn_prompt_gthinker(line, dataset)
+            elif DATASET_TYPE(dataset) == 'MCQ':
+                prompt = self.build_multi_choice_prompt_gthinker(line, dataset)
+            else:
+                prompt = line['question']
+
+        # ========== no_thinking ==========
+        else:
+            if dataset == 'MMVet':
+                prompt = self.build_mmvet_prompt(line, dataset)
+            elif dataset == 'OCRBench':
+                prompt = self.build_ocrbench_prompt(line, dataset)
+            elif dataset == 'M3CoT_Test':
+                prompt = self.build_m3cot_prompt(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('TextVQA', 'ChartQA', 'InfoVQA', 'DocVQA', 'CMMMU')):
+                prompt = self.build_ocr_cmmmu_prompt(line, dataset)
+            elif any(dataset.startswith(prefix) for prefix in ('MathVista', 'MathVerse', 'MathVision')):
+                prompt = self.build_math_prompt(line, dataset)
+            elif DATASET_TYPE(dataset) == 'Y/N':
+                prompt = self.build_yorn_prompt(line, dataset)
+            elif DATASET_TYPE(dataset) == 'MCQ':
+                prompt = self.build_multi_choice_prompt(line, dataset)
+            else:
+                prompt = line['question']
+
+        # add image to message
+        message = []
+        if isinstance(tgt_path, list):
+            message.extend([dict(type='image', value=p) for p in tgt_path])
+        else:
+            message.extend([dict(type='image', value=tgt_path)])
+
+        # add text to message
+        message.append(dict(type='text', value=prompt))
+        return message
+
+    def post_process(self, generation_text, dataset=None):
+        # print(f">>> RAW RESPONSE:\n{generation_text}")
+        if dataset.startswith("MMVet"):
+            generation_text += "(Please only output the final score.)"
+            return generation_text
+
+        if dataset.startswith("OCRBench"):
+            return generation_text
+
+        # Extract answer content
+        pattern = r"(?:<answer>)([\s\S]*?)(?:</answer>)"
+        match = re.search(pattern, generation_text)
+        if match:
+            generation_text = match.group(1)
+
+        if any(dataset.startswith(prefix) for prefix in ('TextVQA', 'ChartQA', 'InfoVQA', 'DocVQA', 'CMMMU')):
+            generation_text = generation_text.strip()
+            if generation_text.endswith('.'):
+                generation_text = generation_text[:-1]
+
+        return generation_text
+
+    def get_pixels_limit(self, dataset=None):
+        if self._use_custom_pixels_limit and \
+                any(dataset.startswith(prefix) for prefix in ['OCRBench', 'HallusionBench', 'MMMU']):
+            min_pixels = 100 * 28 * 28
+            max_pixels = 1280 * 28 * 28
+        else:
+            min_pixels = self.min_pixels
+            max_pixels = self.max_pixels
+        return min_pixels, max_pixels
+
+    def generate_inner(self, message, dataset=None):
+        # Construct messages and images
+        # print(f">>> ORIGIN_MSG: {message}")
+        messages = []
+        text, images = '', []
+        for item in message:
+            if item['type'] == 'text':
+                text += item['value']
+            elif item['type'] == 'image':
+                text += ' <image> '
+                images.append(item['value'])
+
+        if self._use_gthinker_thinking:
+            gthinker_system_prompt = None
+
+            if any(dataset.startswith(prefix) for prefix in ('MMBench', 'MMMU', 'AI2D')):
+                gthinker_system_prompt = GTHINKER_SYS_PROMPT
+            elif any(dataset.startswith(prefix) for prefix in ('MMStar', 'MathVista', 'HallusionBench')):
+                gthinker_system_prompt = \
+                    GTHINKER_SYS_PROMPT.replace("<think>", "<reason>").replace("</think>", "</reason>")
+            else:
+                gthinker_system_prompt = GTHINKER_SYS_PROMPT
+
+            if gthinker_system_prompt is not None:
+                messages.append({"role": 'system', "content": gthinker_system_prompt})
+
+        messages.append({"role": 'user', "content": text})
+
+        min_pixels, max_pixels = self.get_pixels_limit(dataset)
+        # print(f">>> min_pixels: {min_pixels}, max_pixels: {max_pixels}")
+        # print(f">>> MSG: {messages}")
+        # print(f">>> IMAGES: {images}")
+        # print(f">>> ENABLE_THINKING: {self._use_gthinker_thinking}")
+        data_dict = self.processor(
+            {
+                "conversations": messages,
+                "images": images
+            },
+            min_pixels=min_pixels // len(images),
+            max_pixels=max_pixels // len(images),
+            enable_thinking=self._use_gthinker_thinking
+        )
+
+        # Inference
+        with torch.inference_mode():
+            self.model.to(dtype=self.torch_dtype, device=self.device)
+
+            if type(data_dict["images"][0]) is list:
+                images = [
+                    [item.to(dtype=self.torch_dtype, device=self.device) for item in img]
+                    for img in data_dict["images"]
+                ]
+            else:
+                images = [img.to(dtype=self.torch_dtype, device=self.device) for img in data_dict["images"]]
+
+            # print(f'pixel_values.shape: {data_dict["pixel_values"].shape}')
+
+            output_ids = self.model.generate(
+                input_ids=data_dict["input_ids"].to(self.device),
+                images=images,
+                image_sizes=data_dict["image_sizes"],
+                pixel_values=data_dict["pixel_values"].to(dtype=self.torch_dtype, device=self.device),
+                image_grid_thw=data_dict["image_grid_thw"].to(self.device),
+                do_sample=self.kwargs["do_sample"],
+                max_new_tokens=self.kwargs["max_new_tokens"],
+                repetition_penalty=self.kwargs["repetition_penalty"],
+                return_dict_in_generate=True,
+                output_scores=True,
+            )
+        input_token_len = data_dict["input_ids"].shape[1]
+
+        # sequences = output_ids.sequences.clone()
+        # sequences[sequences < 0] = 0
+        # full_seq = self.processor.batch_decode(sequences)[0]
+        # print(f">>> FULL_SEQ: {full_seq}")
+
+        generation_text = self.processor.batch_decode(output_ids.sequences[:, input_token_len:])[0]
+        generation_text = generation_text.replace("<|im_end|>", "")
+
+        # Postprocess
+        generation_text = self.post_process(generation_text, dataset)
+
+        # print(f">>> POST_PROCESS: {generation_text}")
+        return generation_text
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..a2715b17e0d04682ec50c594b16fbc74ced1e64d
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/__init__.py
@@ -0,0 +1,8 @@
+from .chat_uni_vi import Chatunivi
+from .llama_vid import LLaMAVID
+from .pllava import PLLaVA
+from .video_chatgpt import VideoChatGPT
+from .video_llava import VideoLLaVA, VideoLLaVA_HF
+from .videochat2 import VideoChat2_HD
+
+__all__ = ['VideoLLaVA', 'VideoLLaVA_HF', 'Chatunivi', 'VideoChatGPT', 'LLaMAVID', 'VideoChat2_HD', 'PLLaVA']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/chat_uni_vi.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/chat_uni_vi.py
new file mode 100644
index 0000000000000000000000000000000000000000..60b1352e2d78ea10107ce77994653452765ac667
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/chat_uni_vi.py
@@ -0,0 +1,197 @@
+import logging
+import os
+
+import numpy as np
+import torch
+from PIL import Image
+
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+
+def _get_rawvideo_dec(
+    video_path,
+    image_processor,
+    max_frames=64,
+    image_resolution=224,
+    video_framerate=1,
+    s=None,
+    e=None,
+):
+    # speed up video decode via decord.
+    video_mask = np.zeros(max_frames, dtype=np.int64)
+    max_video_length = 0
+
+    # T x 3 x H x W
+    video = np.zeros((max_frames, 3, image_resolution, image_resolution), dtype=np.float64)
+
+    if s is None:
+        start_time, end_time = None, None
+    else:
+        start_time = int(s)
+        end_time = int(e)
+        start_time = start_time if start_time >= 0.0 else 0.0
+        end_time = end_time if end_time >= 0.0 else 0.0
+        if start_time > end_time:
+            start_time, end_time = end_time, start_time
+        elif start_time == end_time:
+            end_time = start_time + 1
+
+    if os.path.exists(video_path):
+        from decord import VideoReader, cpu
+        vreader = VideoReader(video_path, ctx=cpu(0))
+    else:
+        print(video_path)
+        raise FileNotFoundError
+
+    fps = vreader.get_avg_fps()
+    f_start = 0 if start_time is None else int(start_time * fps)
+    f_end = int(min(1000000000 if end_time is None else end_time * fps, len(vreader) - 1))
+    num_frames = f_end - f_start + 1
+    if num_frames > 0:
+        # T x 3 x H x W
+        sample_fps = int(video_framerate)
+        t_stride = int(round(float(fps) / sample_fps))
+
+        all_pos = list(range(f_start, f_end + 1, t_stride))
+        if len(all_pos) > max_frames:
+            sample_pos = [all_pos[_] for _ in np.linspace(0, len(all_pos) - 1, num=max_frames, dtype=int)]
+        else:
+            sample_pos = all_pos
+
+        patch_images = [Image.fromarray(f) for f in vreader.get_batch(sample_pos).asnumpy()]
+
+        patch_images = torch.stack(
+            [image_processor.preprocess(img, return_tensors='pt')['pixel_values'][0] for img in patch_images]
+        )
+        slice_len = patch_images.shape[0]
+
+        max_video_length = max_video_length if max_video_length > slice_len else slice_len
+        if slice_len < 1:
+            pass
+        else:
+            video[:slice_len, ...] = patch_images
+
+        return patch_images, slice_len
+    else:
+        print('video path: {} error.'.format(video_path))
+
+    video_mask[:max_video_length] = [1] * max_video_length
+
+    return torch.from_numpy(video), video_mask
+
+
+class Chatunivi(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = False
+    VIDEO_LLM = True
+    # sample 1 fps (maximum 64 frames) from the video
+
+    def __init__(self, model_path='Chat-UniVi/Chat-UniVi', **kwargs):
+        assert model_path is not None
+        try:
+            from ChatUniVi.model.builder import load_pretrained_model
+        except Exception as err:
+            logging.critical('Please install Chat-UniVi from https://github.com/PKU-YuanGroup/Chat-UniVi.git.')
+            raise err
+
+        model_name = 'ChatUniVi'
+        tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name)
+        self.tokenizer = tokenizer
+        self.model = model
+        vision_tower = model.get_vision_tower()
+        if not vision_tower.is_loaded:
+            vision_tower.load_model()
+        image_processor = vision_tower.image_processor
+        self.processor = image_processor
+        self.context_len = context_len
+        self.kwargs = kwargs
+        self.fps = 1
+        self.resolution = 224
+        if 'v1.5' in model_path:
+            self.resolution = 336
+
+    def get_model_output(self, model, video_processor, tokenizer, video, qs):
+        from ChatUniVi.constants import (DEFAULT_IM_END_TOKEN, DEFAULT_IM_START_TOKEN,
+                                         DEFAULT_IMAGE_PATCH_TOKEN, DEFAULT_IMAGE_TOKEN,
+                                         IMAGE_TOKEN_INDEX, MAX_IMAGE_LENGTH)
+        from ChatUniVi.conversation import SeparatorStyle, conv_templates
+        from ChatUniVi.mm_utils import KeywordsStoppingCriteria, tokenizer_image_token
+
+        mm_use_im_start_end = getattr(model.config, 'mm_use_im_start_end', False)
+        mm_use_im_patch_token = getattr(model.config, 'mm_use_im_patch_token', True)
+        if mm_use_im_patch_token:
+            tokenizer.add_tokens([DEFAULT_IMAGE_PATCH_TOKEN], special_tokens=True)
+        if mm_use_im_start_end:
+            tokenizer.add_tokens([DEFAULT_IM_START_TOKEN, DEFAULT_IM_END_TOKEN], special_tokens=True)
+        model.resize_token_embeddings(len(tokenizer))
+
+        if model.config.config['use_cluster']:
+            for n, m in model.named_modules():
+                m = m.to(dtype=torch.bfloat16)
+
+        video_frames, slice_len = _get_rawvideo_dec(
+            video, video_processor, max_frames=MAX_IMAGE_LENGTH,
+            image_resolution=self.resolution, video_framerate=self.fps
+        )
+
+        if model.config.mm_use_im_start_end:
+            qs = DEFAULT_IM_START_TOKEN + DEFAULT_IMAGE_TOKEN * slice_len + DEFAULT_IM_END_TOKEN + '\n' + qs
+        if type(qs) is dict and 'user' in qs:
+            qs['user'] = DEFAULT_IMAGE_TOKEN * slice_len + '\n' + qs['user']
+        else:
+            qs = DEFAULT_IMAGE_TOKEN * slice_len + '\n' + qs
+
+        conv = conv_templates['v1'].copy()
+        if type(qs) is dict and 'system' in qs:
+            conv.system = qs['system']
+        if type(qs) is dict and 'user' in qs:
+            conv.append_message(conv.roles[0], qs['user'])
+        else:
+            conv.append_message(conv.roles[0], qs)
+        if type(qs) is dict and 'assistant' in qs:
+            conv.append_message(conv.roles[1], qs['assistant'])
+        else:
+            conv.append_message(conv.roles[1], None)
+        prompt = conv.get_prompt().strip('</s>')
+
+        input_ids = tokenizer_image_token(prompt, tokenizer, IMAGE_TOKEN_INDEX, return_tensors='pt').unsqueeze(
+            0).cuda()
+
+        stop_str = conv.sep if conv.sep_style != SeparatorStyle.TWO else conv.sep2
+        keywords = [stop_str]
+        stopping_criteria = KeywordsStoppingCriteria(keywords, tokenizer, input_ids)
+
+        with torch.inference_mode():
+            output_ids = model.generate(
+                input_ids,
+                images=video_frames.half().cuda(),
+                do_sample=True,
+                temperature=0.2,
+                top_p=None,
+                num_beams=1,
+                output_scores=True,
+                return_dict_in_generate=True,
+                max_new_tokens=1024,
+                use_cache=True,
+                stopping_criteria=[stopping_criteria])
+
+        output_ids = output_ids.sequences
+        input_token_len = input_ids.shape[1]
+        n_diff_input_output = (input_ids != output_ids[:, :input_token_len]).sum().item()
+        if n_diff_input_output > 0:
+            print(f'[Warning] {n_diff_input_output} output_ids are not the same as the input_ids')
+        outputs = tokenizer.batch_decode(output_ids[:, input_token_len:], skip_special_tokens=True)[0]
+        outputs = outputs.strip()
+        if outputs.endswith(stop_str):
+            outputs = outputs[:-len(stop_str)]
+        outputs = outputs.strip()
+        return outputs
+
+    def generate_inner(self, message, dataset=None):
+        if listinstr(['MLVU', 'MVBench'], dataset):
+            question, video = self.message_to_promptvideo_withrole(message, dataset)
+        else:
+            question, video = self.message_to_promptvideo(message)
+        response = self.get_model_output(self.model, self.processor, self.tokenizer, video, question)
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/llama_vid.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/llama_vid.py
new file mode 100644
index 0000000000000000000000000000000000000000..41b216ae1823dd51b252c212b4252d81e3beec89
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/llama_vid.py
@@ -0,0 +1,139 @@
+import copy as cp
+import logging
+import os
+
+import torch
+from huggingface_hub import snapshot_download
+
+from vlmeval.smp import download_file, dump, listinstr, load
+from ..base import BaseModel
+
+
+def load_video(video_path, setting_fps):
+    from decord import VideoReader, cpu
+    vr = VideoReader(video_path, ctx=cpu(0))
+    total_frame_num = len(vr)
+    fps = round(vr.get_avg_fps())
+    frame_idx = [i for i in range(0, total_frame_num, int(fps / setting_fps))]
+    spare_frames = vr.get_batch(frame_idx).asnumpy()
+    return spare_frames
+
+
+def change_file(file_path, mm_vision_tower):
+    org_data = load(file_path)
+    org_data['image_processor'] = './vlmeval/vlm/video_llm/configs/llama_vid/processor/clip-patch14-224'
+    org_data['mm_vision_tower'] = mm_vision_tower
+    dump(org_data, file_path)
+
+
+class LLaMAVID(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = False
+    VIDEO_LLM = True
+    # sample 1 fps from the video
+
+    def __init__(self, model_path='YanweiLi/llama-vid-7b-full-224-video-fps-1', **kwargs):
+        assert model_path is not None
+        try:
+            from llamavid.model.builder import load_pretrained_model
+            from llava.mm_utils import get_model_name_from_path
+        except Exception as err:
+            logging.critical('Please install LLaMA-VID from https://github.com/dvlab-research/LLaMA-VID.')
+            raise err
+
+        model_base = None
+        model_name = get_model_name_from_path(model_path)
+
+        eva_vit_g_url = 'https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/eva_vit_g.pth'
+        true_model_path = snapshot_download(model_path)
+        eva_vit_path = os.path.join(true_model_path, 'eva_vit_g.pth')
+        if not os.path.exists(eva_vit_path):
+            download_file(eva_vit_g_url, eva_vit_path)
+        config_path = os.path.join(true_model_path, 'config.json')
+        change_file(config_path, eva_vit_path)
+
+        tokenizer, model, image_processor, context_len = load_pretrained_model(
+            true_model_path, model_base, model_name, None, device_map='cpu', device='cpu'
+        )
+        model.cuda()
+        self.tokenizer = tokenizer
+        self.model = model
+        self.processor = image_processor
+        self.context_len = context_len
+        self.kwargs = kwargs
+        self.fps = 1
+
+    def get_model_output(self, model, video_processor, tokenizer, video, qs):
+        from llamavid.constants import (DEFAULT_IM_END_TOKEN, DEFAULT_IM_START_TOKEN,
+                                        DEFAULT_IMAGE_TOKEN, IMAGE_TOKEN_INDEX)
+        from llamavid.conversation import SeparatorStyle, conv_templates
+        from llava.mm_utils import KeywordsStoppingCriteria, tokenizer_image_token
+
+        if type(qs) is dict:
+            original_qs = cp.deepcopy(qs['user'])
+        else:
+            original_qs = cp.deepcopy(qs)
+        if model.config.mm_use_im_start_end:
+            qs = DEFAULT_IM_START_TOKEN + DEFAULT_IMAGE_TOKEN + DEFAULT_IM_END_TOKEN + '\n' + qs
+        elif type(qs) is dict and 'user' in qs:
+            qs['user'] = DEFAULT_IMAGE_TOKEN + '\n' + qs['user']
+        else:
+            qs = DEFAULT_IMAGE_TOKEN + '\n' + qs
+
+        conv_mode = 'vicuna_v1'
+        conv = conv_templates[conv_mode].copy()
+        if type(qs) is dict and 'system' in qs:
+            conv.system = qs['system']
+        if type(qs) is dict and 'user' in qs:
+            conv.append_message(conv.roles[0], qs['user'])
+        else:
+            conv.append_message(conv.roles[0], qs)
+        if type(qs) is dict and 'assistant' in qs:
+            conv.append_message(conv.roles[1], qs['assistant'])
+        else:
+            conv.append_message(conv.roles[1], None)
+        prompt = conv.get_prompt().strip('</s>')
+
+        # Check if the video exists
+        if os.path.exists(video):
+            video = load_video(video, self.fps)
+            video = video_processor.preprocess(video, return_tensors='pt')['pixel_values'].half().cuda()
+            video = [video]
+
+        input_ids = tokenizer_image_token(prompt, tokenizer, IMAGE_TOKEN_INDEX, return_tensors='pt').unsqueeze(0).cuda()
+
+        stop_str = conv.sep if conv.sep_style != SeparatorStyle.TWO else conv.sep2
+        keywords = [stop_str]
+        stopping_criteria = KeywordsStoppingCriteria(keywords, tokenizer, input_ids)
+
+        cur_prompt = original_qs
+        with torch.inference_mode():
+            model.update_prompt([[cur_prompt]])
+            output_ids = model.generate(
+                input_ids,
+                images=video,
+                do_sample=True,
+                temperature=0.2,
+                max_new_tokens=1024,
+                use_cache=True,
+                stopping_criteria=[stopping_criteria],
+            )
+
+        input_token_len = input_ids.shape[1]
+        n_diff_input_output = (input_ids != output_ids[:, :input_token_len]).sum().item()
+        if n_diff_input_output > 0:
+            print(f'[Warning] {n_diff_input_output} output_ids are not the same as the input_ids')
+        outputs = tokenizer.batch_decode(output_ids[:, input_token_len:], skip_special_tokens=True)[0]
+        outputs = outputs.strip()
+        if outputs.endswith(stop_str):
+            outputs = outputs[: -len(stop_str)]
+        outputs = outputs.strip()
+        return outputs
+
+    def generate_inner(self, message, dataset=None):
+        if listinstr(['MLVU', 'MVBench'], dataset):
+            question, video = self.message_to_promptvideo_withrole(message, dataset)
+        else:
+            question, video = self.message_to_promptvideo(message)
+        response = self.get_model_output(self.model, self.processor, self.tokenizer, video, question)
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/pllava.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/pllava.py
new file mode 100644
index 0000000000000000000000000000000000000000..d971ba91fbdee9ce654510f8935c77020f077bf4
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/pllava.py
@@ -0,0 +1,99 @@
+import logging
+import sys
+
+import numpy as np
+import torchvision
+from huggingface_hub import snapshot_download
+from PIL import Image
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+
+class PLLaVA(BaseModel):
+
+    INSTALL_REQ = True
+    INTERLEAVE = False
+    VIDEO_LLM = True
+
+    def __init__(self, model_path='ermu2001/pllava-13b', dir_root=None, **kwargs):
+        sys.path.append(dir_root)
+        try:
+            from tasks.eval.model_utils import load_pllava
+        except Exception as err:
+            logging.critical(
+                'Please first install requirements and set the root path to use PLLaVA. \
+                Follow the instructions at https://github.com/magic-research/PLLaVA.'
+            )
+            raise err
+
+        self.nframe = 16
+        self.use_lora = True
+        self.lora_alpha = 4
+        self.pooling_shape = (16, 12, 12)
+        self.RESOLUTION = 672
+        self.model_path = model_path
+        # remind that, once the model goes larger (30B+) may cause the memory to be heavily used up. Even Tearing Nodes.
+        weight_dir = snapshot_download(model_path)
+        self.model, self.processor = load_pllava(
+            model_path, num_frames=self.nframe, use_lora=self.use_lora,
+            weight_dir=weight_dir, lora_alpha=self.lora_alpha, pooling_shape=self.pooling_shape
+        )
+
+        #  position embedding
+        self.model = self.model.to('cuda')
+        self.model = self.model.eval()
+
+    def load_video(self, video_path, num_segments=8, resolution=336):
+        from decord import VideoReader, cpu
+        transforms = torchvision.transforms.Resize(size=resolution)
+        vr = VideoReader(video_path, ctx=cpu(0), num_threads=1)
+        num_frames = len(vr)
+        frame_indices = self.get_index(num_frames, num_segments)
+        images_group = list()
+        for frame_index in frame_indices:
+            img = Image.fromarray(vr[frame_index].asnumpy())
+            images_group.append(transforms(img))
+        return images_group
+
+    def get_index(self, num_frames, num_segments):
+        seg_size = float(num_frames - 1) / num_segments
+        start = int(seg_size / 2)
+        offsets = np.array([
+            start + int(np.round(seg_size * idx)) for idx in range(num_segments)
+        ])
+        return offsets
+
+    def generate_inner(self, message, dataset=None):
+        from tasks.eval.eval_utils import conv_templates
+        from tasks.eval.model_utils import pllava_answer
+
+        question, video = self.message_to_promptvideo(message)
+
+        img_list = self.load_video(video, num_segments=self.nframe, resolution=self.RESOLUTION)
+
+        if self.model_path == 'ermu2001/pllava-34b':  # using slightly different conversation mode for 34b model
+            if listinstr(['Video-MCQ'], DATASET_TYPE(dataset)):  # MCQ dataset
+                conv_mode = 'eval_mvbench_llavanext'
+            else:  # VQA dataset
+                conv_mode = 'eval_videoqa_llavanext'
+        else:
+            if listinstr(['Video-MCQ'], DATASET_TYPE(dataset)):  # MCQ dataset
+                conv_mode = 'eval_mvbench'
+            else:  # VQA dataset
+                conv_mode = 'eval_videoqabench'
+
+        conv = conv_templates[conv_mode].copy()
+        if dataset in ['MVBench', 'MVBench_MP4']:
+            conv.user_query(message[1]['value'], message[0]['value'], message[-2]['value'], is_mm=True)
+            conv.assistant_response(message[-1]['value'])
+        else:
+            conv.user_query(question, is_mm=True)
+        llm_response, conv = pllava_answer(
+            conv=conv, model=self.model, processor=self.processor,
+            do_sample=False, img_list=img_list, max_new_tokens=512, print_res=False
+        )
+        if dataset in ['MVBench', 'MVBench_MP4']:
+            llm_response = '(' + ''.join(llm_response.split(message[-1]['value'])[1:])
+        return llm_response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/video_chatgpt.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/video_chatgpt.py
new file mode 100644
index 0000000000000000000000000000000000000000..2ab8c22060fadd58e16b872f1bebb19bc5eba6e1
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/video_chatgpt.py
@@ -0,0 +1,64 @@
+import logging
+import os
+import sys
+
+from huggingface_hub import snapshot_download
+
+from ..base import BaseModel
+
+
+class VideoChatGPT(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = False
+    VIDEO_LLM = True
+    # sample a video in 100 frames
+
+    def __init__(self, model_path='MBZUAI/Video-ChatGPT-7B', dir_root=None, **kwargs):
+        assert model_path is not None
+        sys.path.append(dir_root)
+        try:
+            from video_chatgpt.eval.model_utils import initialize_model
+        except Exception as err:
+            logging.critical(
+                'Please first install requirements and set the root path to use Video-ChatGPT. \
+                Follow the instructions at https://github.com/mbzuai-oryx/Video-ChatGPT.'
+            )
+            raise err
+        base_model_path = snapshot_download('mmaaz60/LLaVA-7B-Lightening-v1-1')
+        projection_path = snapshot_download(model_path)
+        projection_name = 'video_chatgpt-7B.bin'
+        projection_path = os.path.join(projection_path, projection_name)
+
+        model, vision_tower, tokenizer, image_processor, video_token_len = initialize_model(
+            base_model_path, projection_path
+        )
+        self.tokenizer = tokenizer
+        self.model = model
+        self.processor = image_processor
+        self.context_len = video_token_len
+        self.kwargs = kwargs
+        self.vision_tower = vision_tower
+
+    def get_model_output(self, model, video_processor, tokenizer, video, qs):
+        from video_chatgpt.eval.model_utils import load_video
+        from video_chatgpt.inference import video_chatgpt_infer
+        conv_mode = 'video-chatgpt_v1'
+
+        video_frames = load_video(video)
+        # Run inference on the video and questions
+        output = video_chatgpt_infer(
+            video_frames,
+            qs,
+            conv_mode,
+            model,
+            self.vision_tower,
+            tokenizer,
+            video_processor,
+            self.context_len,
+        )
+        return output
+
+    def generate_inner(self, message, dataset=None):
+        question, video = self.message_to_promptvideo(message)
+        response = self.get_model_output(self.model, self.processor, self.tokenizer, video, question)
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/video_llava.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/video_llava.py
new file mode 100644
index 0000000000000000000000000000000000000000..4069dfeda9d7ce523585beecd213803e32a2c999
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/video_llava.py
@@ -0,0 +1,179 @@
+import logging
+
+import numpy as np
+import torch
+
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+
+def read_video_pyav(container, indices):
+    frames = []
+    container.seek(0)
+    start_index = indices[0]
+    end_index = indices[-1]
+    for i, frame in enumerate(container.decode(video=0)):
+        if i > end_index:
+            break
+        if i >= start_index and i in indices:
+            frames.append(frame)
+    return np.stack([x.to_ndarray(format='rgb24') for x in frames])
+
+
+class VideoLLaVA_HF(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = False
+    VIDEO_LLM = True
+    # sample a video in 8 frames
+
+    def __init__(self, model_path='LanguageBind/Video-LLaVA-7B-hf', **kwargs):
+        try:
+            from transformers import VideoLlavaForConditionalGeneration, VideoLlavaProcessor
+        except Exception as err:
+            logging.critical('Please install the latest version transformers. \
+                          You can install by `pip install transformers==4.42.0` \
+                          or `pip install --upgrade git+https://github.com/huggingface/transformers.git`.')
+            raise err
+
+        assert model_path is not None
+        self.model_path = model_path
+        self.model = VideoLlavaForConditionalGeneration.from_pretrained(model_path)
+        self.model.eval().cuda()
+        self.processor = VideoLlavaProcessor.from_pretrained(model_path)
+        self.kwargs = kwargs
+        self.nframe = 8
+        torch.cuda.empty_cache()
+
+    def generate_inner(self, message, dataset=None):
+        import av
+        if self.nframe != 8:
+            raise Exception(f'Video-LLaVA only supported 8 frames to generate, you now set frame numbers to {self.nframe}')  # noqa
+        question, video = self.message_to_promptvideo(message)
+
+        container = av.open(video)
+
+        # sample uniformly 8 frames from the video
+        total_frames = container.streams.video[0].frames
+        indices = np.arange(0, total_frames, total_frames / self.nframe).astype(int)
+        clip = read_video_pyav(container, indices)
+
+        prompt = f'USER: <video>\n{question} ASSISTANT:'
+        inputs = self.processor(text=prompt, videos=clip, return_tensors='pt').to(self.model.device)
+
+        # Generate args -- deperecated
+        generation_args = {
+            'max_new_tokens': 1024,
+            'temperature': 0.0,
+            'do_sample': False,
+        }
+        generation_args.update(self.kwargs)
+
+        generate_ids = self.model.generate(**inputs, **generation_args)
+        generate_ids = generate_ids[:, inputs['input_ids'].shape[1]:]
+        response = self.processor.batch_decode(
+            generate_ids,
+            skip_special_tokens=True,
+            clean_up_tokenization_spaces=False
+        )[0]
+        return response
+
+
+class VideoLLaVA(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = False
+    VIDEO_LLM = True
+    # sample a video in 8 frames
+
+    def __init__(self, model_path='LanguageBind/Video-LLaVA-7B', **kwargs):
+        assert model_path is not None
+        try:
+            from videollava.constants import DEFAULT_IMAGE_TOKEN  # noqa: F401
+            from videollava.constants import DEFAULT_VID_END_TOKEN  # noqa: F401
+            from videollava.constants import DEFAULT_VID_START_TOKEN  # noqa: F401
+            from videollava.constants import IMAGE_TOKEN_INDEX  # noqa: F401
+            from videollava.conversation import SeparatorStyle, conv_templates  # noqa: F401
+            from videollava.mm_utils import KeywordsStoppingCriteria  # noqa: F401
+            from videollava.mm_utils import get_model_name_from_path  # noqa: F401
+            from videollava.mm_utils import tokenizer_image_token  # noqa: F401
+            from videollava.model.builder import load_pretrained_model  # noqa: F401
+            from videollava.model.language_model.llava_llama import \
+                LlavaLlamaForCausalLM  # noqa: F401
+            from videollava.train.train import smart_tokenizer_and_embedding_resize  # noqa: F401
+        except Exception as err:
+            logging.critical('Please install Video-LLaVA from https://github.com/FangXinyu-0913/Video-LLaVA.')
+            raise err
+
+        model_base = None
+        model_name = model_path.split('/')[-1]
+        tokenizer, model, processor, context_len = load_pretrained_model(model_path, model_base, model_name)
+        self.tokenizer = tokenizer
+        self.model = model
+        self.processor = processor
+        self.context_len = context_len
+        self.kwargs = kwargs
+        self.nframe = 8
+
+    def get_model_output(self, model, video_processor, tokenizer, video, qs):
+        from videollava.constants import DEFAULT_IMAGE_TOKEN, IMAGE_TOKEN_INDEX
+        from videollava.conversation import SeparatorStyle, conv_templates
+        from videollava.mm_utils import KeywordsStoppingCriteria, tokenizer_image_token
+
+        if type(qs) is dict and 'user' in qs:
+            qs['user'] = ''.join([DEFAULT_IMAGE_TOKEN] * self.nframe) + '\n' + qs['user']
+        else:
+            qs = ''.join([DEFAULT_IMAGE_TOKEN] * self.nframe) + '\n' + qs
+
+        conv_mode = 'llava_v1'
+        device = torch.device('cuda')
+        conv = conv_templates[conv_mode].copy()
+        if type(qs) is dict and 'system' in qs:
+            conv.system = qs['system']
+        if type(qs) is dict and 'user' in qs:
+            conv.append_message(conv.roles[0], qs['user'])
+        else:
+            conv.append_message(conv.roles[0], qs)
+        if type(qs) is dict and 'assistant' in qs:
+            conv.append_message(conv.roles[1], qs['assistant'])
+        else:
+            conv.append_message(conv.roles[1], None)
+        prompt = conv.get_prompt().strip('</s>')
+
+        video_tensor = video_processor.preprocess(video, return_tensors='pt')['pixel_values'][0].half().to(device)
+        input_ids = tokenizer_image_token(prompt, tokenizer,
+                                          IMAGE_TOKEN_INDEX, return_tensors='pt').unsqueeze(0).to(device)
+
+        stop_str = conv.sep if conv.sep_style != SeparatorStyle.TWO else conv.sep2
+        keywords = [stop_str]
+        stopping_criteria = KeywordsStoppingCriteria(keywords, tokenizer, input_ids)
+
+        with torch.inference_mode():
+            output_ids = model.generate(
+                input_ids,
+                images=[video_tensor],
+                do_sample=False,
+                temperature=0.0,
+                max_new_tokens=1024,
+                use_cache=True,
+                stopping_criteria=[stopping_criteria])
+
+        input_token_len = input_ids.shape[1]
+        n_diff_input_output = (input_ids != output_ids[:, :input_token_len]).sum().item()
+        if n_diff_input_output > 0:
+            print(f'[Warning] {n_diff_input_output} output_ids are not the same as the input_ids')
+        outputs = tokenizer.batch_decode(output_ids[:, input_token_len:], skip_special_tokens=True)[0]
+        outputs = outputs.strip()
+        if outputs.endswith(stop_str):
+            outputs = outputs[:-len(stop_str)]
+        outputs = outputs.strip()
+        return outputs
+
+    def generate_inner(self, message, dataset=None):
+        if self.nframe != 8:
+            raise Exception(f'Video-LLaVA only supported 8 frames to generate, you now set frame numbers to {self.nframe}')  # noqa
+        if listinstr(['MLVU', 'MVBench'], dataset):
+            question, video = self.message_to_promptvideo_withrole(message, dataset)
+        else:
+            question, video = self.message_to_promptvideo(message)
+        response = self.get_model_output(self.model, self.processor['video'], self.tokenizer, video, question)
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/videochat2.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/videochat2.py
new file mode 100644
index 0000000000000000000000000000000000000000..3887b4e8bd5607b9a2ea7ece7f724a29783bb4a7
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/video_llm/videochat2.py
@@ -0,0 +1,438 @@
+import logging
+import os
+import os.path as osp
+import shutil
+import sys
+
+import huggingface_hub
+import numpy as np
+import requests
+import torch
+from huggingface_hub import snapshot_download
+from torchvision import transforms
+from transformers import StoppingCriteria, StoppingCriteriaList
+
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+
+def get_prompt(conv):
+    ret = conv.system + conv.sep
+    for role, message in conv.messages:
+        if message:
+            ret += role + ' ' + message + ' ' + conv.sep
+        else:
+            ret += role
+    return ret
+
+
+def get_prompt2(conv):
+    ret = conv.system + conv.sep
+    count = 0
+    for role, message in conv.messages:
+        count += 1
+        if count == len(conv.messages):
+            ret += role + ' ' + message
+        else:
+            if message:
+                ret += role + ' ' + message + ' ' + conv.sep
+            else:
+                ret += role
+    return ret
+
+
+class StoppingCriteriaSub(StoppingCriteria):
+    def __init__(self, stops=[], encounters=1):
+        super().__init__()
+        self.stops = stops
+
+    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor):
+        for stop in self.stops:
+            if torch.all((stop == input_ids[0][-len(stop):])).item():
+                return True
+        return False
+
+
+class VideoChat2_HD(BaseModel):
+    INSTALL_REQ = True
+    INTERLEAVE = False
+    VIDEO_LLM = True
+
+    def __init__(self, model_path='OpenGVLab/VideoChat2_HD_stage4_Mistral_7B',
+                 root='./Ask-Anything', config_file='./configs/videochat2_hd.json',
+                 **kwargs):
+
+        from peft import LoraConfig, TaskType, get_peft_model
+        self.config_file = config_file
+        self.root = root
+        self.model_path = model_path
+
+        if root is None:
+            raise ValueError('Please set `root` to Ask-Anything directory, \
+                          which is cloned from here: https://github.com/OpenGVLab/Ask-Anything')
+
+        sys.path.append(osp.join(root, 'video_chat2'))
+        try:
+            from dataset.hd_utils import HD_transform_no_padding  # noqa: F401
+            from dataset.hd_utils import HD_transform_padding  # noqa: F401
+            from models import VideoChat2_it_hd_mistral
+            from utils.config import Config
+            from utils.easydict import EasyDict  # noqa: F401
+        except Exception as err:
+            logging.critical(
+                'Please first install VideoChat2 and set the root path to use VideoChat2, '
+                'which is cloned from here: https://github.com/OpenGVLab/Ask-Anything '
+            )
+            raise err
+
+        cfg = Config.from_file(self.config_file)
+
+        def download_file(url, pth):
+            destination_folder = pth
+            # 确保目标文件夹存在
+            if not os.path.exists(destination_folder):
+                os.makedirs(destination_folder)
+
+            # 获取文件名
+            filename = os.path.basename(url)
+            destination_path = os.path.join(destination_folder, filename)
+            if os.path.exists(destination_path):
+                print(f'File downloaded! No repeat download needed. Saved in {destination_path}')
+                return
+
+            # 下载文件
+            response = requests.get(url, stream=True)
+            if response.status_code == 200:
+                with open(destination_path, 'wb') as file:
+                    response.raw.decode_content = True
+                    shutil.copyfileobj(response.raw, file)
+                print(f'File downloaded and saved to {destination_path}')
+            else:
+                print(f'Download failed, status code: {response.status_code}')
+
+        hf_token = os.environ.get('HUGGINGFACE_TOKEN')
+        huggingface_hub.login(hf_token)
+        videochat2_model_path = snapshot_download(repo_id=cfg.model.videochat2_model_path, repo_type='model')
+        cfg.model.videochat2_model_path = osp.join(videochat2_model_path, 'videochat2_mistral_7b_stage2.pth')
+
+        mistral_model_path = snapshot_download(repo_id=cfg.model.mistral_model_path, repo_type='model')
+        cfg.model.mistral_model_path = mistral_model_path
+
+        vit_blip_model_path = snapshot_download(repo_id=cfg.model.vit_blip_model_path, repo_type='model')
+        cfg.model.vit_blip_model_path = osp.join(vit_blip_model_path, 'umt_l16_qformer.pth')
+
+        model = VideoChat2_it_hd_mistral(config=cfg.model)
+        peft_config = LoraConfig(
+            task_type=TaskType.CAUSAL_LM, inference_mode=False,
+            r=16, lora_alpha=32, lora_dropout=0.,
+            target_modules=[
+                'q_proj', 'k_proj', 'v_proj', 'o_proj',
+                'gate_proj', 'up_proj', 'down_proj', 'lm_head'
+            ]
+        )
+        model.mistral_model = get_peft_model(model.mistral_model, peft_config)
+        stage4_model_path = snapshot_download(repo_id=model_path, repo_type='model')
+        state_dict = torch.load(osp.join(stage4_model_path, 'videochat2_hd_mistral_7b_stage4.pth'), 'cuda')
+
+        if 'model' in state_dict.keys():
+            model.load_state_dict(state_dict['model'], strict=False)
+        else:
+            model.load_state_dict(state_dict, strict=False)
+
+        model = model.to(torch.device('cuda'))
+        model = model.eval()
+        self.model = model
+
+        #  position embedding
+        self.nframe = 16
+        self.resolution = 224
+        self.hd_num = 6
+        new_pos_emb = self.get_sinusoid_encoding_table(
+            n_position=(self.resolution // 16) ** 2 * self.nframe,
+            cur_frame=self.nframe
+        )
+        self.model.vision_encoder.encoder.pos_embed = new_pos_emb
+
+        self.hd_transform = HD_transform_no_padding
+
+        mean = (0.485, 0.456, 0.406)
+        std = (0.229, 0.224, 0.225)
+        self.transform = transforms.Compose([
+            transforms.Lambda(lambda x: x.float().div(255.0)),
+            transforms.Normalize(mean, std)
+        ])
+
+    def get_sinusoid_encoding_table(self, n_position=784, d_hid=1024,
+                                    cur_frame=8, ckpt_num_frame=4,
+                                    pre_n_position=784):
+        ''' Sinusoid position encoding table '''
+        # TODO: make it with torch instead of numpy
+        def get_position_angle_vec(position):
+            return [position / np.power(10000, 2 * (hid_j // 2) / d_hid) for hid_j in range(d_hid)]
+
+        # generate checkpoint position embedding
+        sinusoid_table = np.array([get_position_angle_vec(pos_i) for pos_i in range(pre_n_position)])
+        sinusoid_table[:, 0::2] = np.sin(sinusoid_table[:, 0::2])  # dim 2i
+        sinusoid_table[:, 1::2] = np.cos(sinusoid_table[:, 1::2])  # dim 2i+1
+        sinusoid_table = torch.tensor(sinusoid_table, dtype=torch.float, requires_grad=False).unsqueeze(0)
+
+        print(f'n_position: {n_position}')
+        print(f'pre_n_position: {pre_n_position}')
+
+        if n_position != pre_n_position:
+            T = ckpt_num_frame  # checkpoint frame
+            P = 14  # checkpoint size
+            C = d_hid
+            new_P = int((n_position // cur_frame) ** 0.5)  # testing size
+            if new_P != 14:
+                print(f'Pretraining uses 14x14, but current version is {new_P}x{new_P}')
+                print('Interpolate the position embedding')
+                sinusoid_table = sinusoid_table.reshape(-1, T, P, P, C)
+                sinusoid_table = sinusoid_table.reshape(-1, P, P, C).permute(0, 3, 1, 2)
+                sinusoid_table = torch.nn.functional.interpolate(
+                    sinusoid_table, size=(new_P, new_P), mode='bicubic', align_corners=False)
+                # BT, C, H, W -> BT, H, W, C ->  B, T, H, W, C
+                sinusoid_table = sinusoid_table.permute(0, 2, 3, 1).reshape(-1, T, new_P, new_P, C)
+                sinusoid_table = sinusoid_table.flatten(1, 3)  # B, THW, C
+
+        if cur_frame != ckpt_num_frame:
+            print(f'Pretraining uses 4 frames, but current frame is {cur_frame}')
+            print('Interpolate the position embedding')
+            T = ckpt_num_frame  # checkpoint frame
+            new_T = cur_frame  # testing frame
+            # interpolate
+            P = int((n_position // cur_frame) ** 0.5)  # testing size
+            C = d_hid
+            sinusoid_table = sinusoid_table.reshape(-1, T, P, P, C)
+            sinusoid_table = sinusoid_table.permute(0, 2, 3, 4, 1).reshape(-1, C, T)  # BHW, C, T
+            sinusoid_table = torch.nn.functional.interpolate(sinusoid_table, size=new_T, mode='linear')
+            sinusoid_table = sinusoid_table.reshape(1, P, P, C, new_T).permute(0, 4, 1, 2, 3)  # B, T, H, W, C
+            sinusoid_table = sinusoid_table.flatten(1, 3)  # B, THW, C
+
+        return sinusoid_table
+
+    def get_index(self, bound, fps, max_frame, first_idx=0):
+        if bound:
+            start, end = bound[0], bound[1]
+        else:
+            start, end = -100000, 100000
+        start_idx = max(first_idx, round(start * fps))
+        end_idx = min(round(end * fps), max_frame)
+        seg_size = float(end_idx - start_idx) / self.nframe
+        frame_indices = np.array([
+            int(start_idx + (seg_size / 2) + np.round(seg_size * idx))
+            for idx in range(self.nframe)
+        ])
+        return frame_indices
+
+    def read_video(self, video_path, bound=None):
+        from decord import VideoReader, cpu
+        vr = VideoReader(video_path, ctx=cpu(0), num_threads=1)
+        max_frame = len(vr) - 1
+        fps = float(vr.get_avg_fps())
+        frame_indices = self.get_index(bound, fps, max_frame, first_idx=0)
+
+        frames = vr.get_batch(frame_indices)
+        frames = frames.permute(0, 3, 1, 2)
+        frames = self.hd_transform(frames.float(), image_size=self.resolution, hd_num=self.hd_num)
+        torch_imgs = self.transform(frames)
+        return torch_imgs
+
+    def ask(self, text, conv):
+        conv.messages.append([conv.roles[0], text])
+
+    def get_context_emb(self, conv, model, img_list, answer_prompt=None, print_res=False):
+        if answer_prompt:
+            prompt = get_prompt2(conv)
+        else:
+            prompt = get_prompt(conv)
+        if print_res:
+            print(prompt)
+        if '<VideoHere>' in prompt:
+            prompt_segs = prompt.split('<VideoHere>')
+        else:
+            prompt_segs = prompt.split('<ImageHere>')
+        assert len(prompt_segs) == len(img_list) + 1, 'Unmatched numbers of image placeholders and images.'
+        with torch.no_grad():
+            seg_tokens = [
+                model.mistral_tokenizer(
+                    seg, return_tensors='pt', add_special_tokens=i == 0).to('cuda').input_ids
+                # only add bos to the first seg
+                for i, seg in enumerate(prompt_segs)
+            ]
+            seg_embs = [model.mistral_model.base_model.model.model.embed_tokens(seg_t) for seg_t in seg_tokens]
+    #         seg_embs = [model.mistral_model.model.embed_tokens(seg_t) for seg_t in seg_tokens]
+        mixed_embs = [emb for pair in zip(seg_embs[:-1], img_list) for emb in pair] + [seg_embs[-1]]
+        mixed_embs = torch.cat(mixed_embs, dim=1)
+        return mixed_embs
+
+    def answer(self, conv, model, img_list, do_sample=True, max_new_tokens=500, num_beams=1, min_length=1, top_p=0.9,
+               repetition_penalty=1.0, length_penalty=1, temperature=1.0, answer_prompt=None, print_res=False):
+        stop_words_ids = [
+            torch.tensor([2]).to('cuda'),
+            torch.tensor([29871, 2]).to('cuda')]  # '</s>' can be encoded in two different ways.
+        stopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])
+
+        conv.messages.append([conv.roles[1], answer_prompt])
+        embs = self.get_context_emb(conv, model, img_list, answer_prompt=answer_prompt, print_res=print_res)
+        with torch.no_grad():
+            outputs = model.mistral_model.generate(
+                inputs_embeds=embs,
+                max_new_tokens=max_new_tokens,
+                stopping_criteria=stopping_criteria,
+                num_beams=num_beams,
+                do_sample=do_sample,
+                min_length=min_length,
+                top_p=top_p,
+                repetition_penalty=repetition_penalty,
+                length_penalty=length_penalty,
+                temperature=temperature,
+            )
+        output_token = outputs[0]
+        if output_token[0] == 0:  # the model might output a unknow token <unk> at the beginning. remove it
+            output_token = output_token[1:]
+        if output_token[0] == 1:  # some users find that there is a start token <s> at the beginning. remove it
+            output_token = output_token[1:]
+        output_text = model.mistral_tokenizer.decode(output_token, add_special_tokens=False)
+        output_text = output_text.split('</s>')[0]  # remove the stop sign </s>
+    #     output_text = output_text.split('[/INST]')[-1].strip()
+        conv.messages[-1][1] = output_text + '</s>'
+        return output_text, output_token.cpu().numpy()
+
+    def infer_data(
+        self, data_sample, system=' ',
+        question_prompt='',  # add in the end of question
+        answer_prompt=None,  # add in the begining of answer
+        system_q=False,  # whether add question in the system prompt for QFormer
+        print_res=True,
+        system_llm=False
+    ):
+        assert system_q is False, 'do not support system_q now'
+        video = data_sample['video']
+        T_, C, H, W = video.shape
+        video = video.reshape(1, T_, C, H, W).to('cuda')
+
+        video_list = []
+        with torch.no_grad():
+            if system_q:
+                raise NotImplementedError
+            else:
+                video_emb, _, _ = self.model.encode_img(video, system)
+        video_list.append(video_emb[0])
+        question = data_sample['question']
+
+        from utils.easydict import EasyDict
+        chat = EasyDict({
+            'system': system,
+            'roles': ('[INST]', '[/INST]'),
+            'messages': [],
+            'sep': ''
+        })
+
+        if data_sample['subtitle'] != '':
+            subtitle = f"This video's subtitles are listed below: {data_sample['subtitle']}"
+            chat.messages.append([chat.roles[0], f'{subtitle}\n<Video><VideoHere></Video> [/INST]'])
+        else:
+            chat.messages.append([chat.roles[0], '<Video><VideoHere></Video> [/INST]'])
+
+        if system_llm:
+            prompt = system + question + question_prompt
+        else:
+            prompt = question + question_prompt
+
+        self.ask(prompt, chat)
+
+        llm_message = self.answer(
+            conv=chat, model=self.model, do_sample=False,
+            img_list=video_list, max_new_tokens=100,
+            answer_prompt=answer_prompt, print_res=print_res
+        )[0]
+
+        return llm_message.strip()
+
+    def qa_template(self, data):
+        question = data.split('Answer:')[0].split('\n')[0] + '\n'
+        question += 'Options:\n'
+        choices = data.split('Answer:')[0].split('\n')[1:]
+        choices = [item for item in choices if item != '']  # remove blank space
+        for idx, c in enumerate(choices):
+            cur_choice, cur_text = c[0], c[3:]
+            question += f'({cur_choice}) {cur_text}\n'
+        question = question.rstrip()
+        return question
+
+    def split_subtitle(self, data):
+        if 'This video\'s subtitles are listed below' in data:
+            # 找到subtitle的起始和结束位置
+            start_marker = 'This video\'s subtitles are listed below:'
+            end_marker = 'Select the best answer to the following multiple-choice question based on the video.'
+
+            start_index = data.find(start_marker) + len(start_marker)
+            end_index = data.find(end_marker)
+
+            # 提取subtitle部分
+            subtitle = data[start_index:end_index].strip()
+            return subtitle
+        else:
+            return ''
+
+    def generate_inner(self, message, dataset=None):
+        if dataset == 'Video-MME':
+            _, video = self.message_to_promptvideo(message)
+            torch_imgs = self.read_video(video)
+            subtitle = self.split_subtitle(message[-2]['value'])
+            question = self.qa_template(message[-1]['value'])
+            example = {
+                'subtitle': subtitle,
+                'video': torch_imgs,
+                'question': question
+            }
+            pred_option = self.infer_data(
+                example,
+                ' ',
+                question_prompt='\nOnly give the best option.',
+                answer_prompt='Best option:(',
+                system_q=False,
+                print_res=False,
+                system_llm=True
+            )
+            return_message = '(' + pred_option.split('\n')[0]
+            return return_message
+        elif listinstr(['MLVU', 'MVBench', 'TempCompass'], dataset):
+            question, video = self.message_to_promptvideo_withrole(message, dataset)
+            torch_imgs = self.read_video(video)
+            example = {
+                'subtitle': '',
+                'video': torch_imgs,
+                'question': question['user']
+            }
+            if 'assistant' not in question:
+                question['assistant'] = None
+            if question['system'] == '':
+                question['system'] = ' '
+            pred_option = self.infer_data(
+                example,
+                question['system'],
+                answer_prompt=question['assistant'],
+                system_q=False,
+                print_res=False,
+                system_llm=True
+            )
+            return_message = '(' + pred_option.split('\n')[0]
+            return return_message
+        else:
+            question, video = self.message_to_promptvideo(message)
+            torch_imgs = self.read_video(video)
+            example = {
+                'subtitle': '',
+                'video': torch_imgs,
+                'question': f'Question:{question}\nAnswer:'
+            }
+            pred_result = self.infer_data(
+                example,
+                ' ',
+                system_q=False,
+                print_res=False,
+                system_llm=False
+            )
+            return pred_result
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/__init__.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..dbf4db400e3b7d60bde0214fb0e6a8a3fd3e0f5c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/__init__.py
@@ -0,0 +1,7 @@
+from .sharecaptioner import ShareCaptioner
+from .xcomposer import XComposer
+from .xcomposer2 import XComposer2
+from .xcomposer2_4KHD import XComposer2_4KHD
+from .xcomposer2d5 import XComposer2d5
+
+__all__ = ['ShareCaptioner', 'XComposer', 'XComposer2', 'XComposer2_4KHD', 'XComposer2d5']
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/sharecaptioner.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/sharecaptioner.py
new file mode 100644
index 0000000000000000000000000000000000000000..4b2320cc227d873a8705644272ab8d4bd9532c5f
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/sharecaptioner.py
@@ -0,0 +1,97 @@
+import string
+
+import pandas as pd
+import torch
+from PIL import Image
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import cn_string
+from ..base import BaseModel
+
+
+class ShareCaptioner(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = False
+
+    def __init__(self, model_path='Lin-Chen/ShareCaptioner', **kwargs):
+        assert model_path is not None
+        tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
+        self.model = AutoModelForCausalLM.from_pretrained(
+            model_path, device_map='cuda', trust_remote_code=True).eval()
+        self.model.tokenizer = tokenizer
+        self.model.cuda()
+        self.model.half()
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert dataset is None or isinstance(dataset, str)
+        assert self.use_custom_prompt(dataset)
+        tgt_path = self.dump_image(line, dataset)
+
+        if dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+            question = line['question']
+            hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+            if hint is not None:
+                question = hint + '\n' + question
+
+            option_candidate = string.ascii_uppercase
+            options = {
+                cand: line[cand]
+                for cand in option_candidate
+                if cand in line and not pd.isna(line[cand])
+            }
+            for key, item in options.items():
+                question += f'\n{key}. {item}'
+            prompt = question
+
+            if not cn_string(prompt):
+                prompt = prompt + '\n' + "Answer with the option's letter from the given choices directly."
+            else:
+                prompt = prompt + '\n' + '请直接回答选项字母。'
+        else:
+            prompt = line['question']
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+        return message
+
+    def generate_inner(self, message, dataset=None):
+        prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+        seg1 = '<|User|>:'
+        seg2 = f'{prompt}{self.model.eoh}\n<|Bot|>:'
+        self.seg_emb1 = self.model.encode_text(seg1, add_special_tokens=True)
+        self.seg_emb2 = self.model.encode_text(seg2, add_special_tokens=False)
+
+        image = Image.open(image_path).convert('RGB')
+        image = self.model.vis_processor(image).unsqueeze(0)
+        image = image.to(self.model.device)
+        tmp_bs = image.shape[0]
+        tmp_seg_emb1 = self.seg_emb1.repeat(tmp_bs, 1, 1)
+        tmp_seg_emb2 = self.seg_emb2.repeat(tmp_bs, 1, 1)
+        with torch.cuda.amp.autocast():
+            with torch.no_grad():
+                image = self.model.encode_img(image)
+                input_emb = torch.cat(
+                    [tmp_seg_emb1, image, tmp_seg_emb2], dim=1)
+                out_embeds = self.model.internlm_model.generate(
+                    inputs_embeds=input_emb,
+                    max_length=500,
+                    num_beams=3,
+                    min_length=1,
+                    do_sample=True,
+                    repetition_penalty=1.5,
+                    length_penalty=1.0,
+                    temperature=1.,
+                    eos_token_id=self.model.tokenizer.eos_token_id,
+                    num_return_sequences=1)
+
+        for j, out in enumerate(out_embeds):
+            out[out == -1] = 2
+            response = self.model.decode_text([out])
+        return response
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer.py
new file mode 100644
index 0000000000000000000000000000000000000000..07d8fe6dacef0e60dfa6b4663f3b7a5817da10c0
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer.py
@@ -0,0 +1,151 @@
+import copy as cp
+import string
+
+import pandas as pd
+import torch
+from PIL import Image
+from transformers import AutoModel, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
+
+from vlmeval.dataset import DATASET_TYPE
+from ..base import BaseModel
+
+
+class StoppingCriteriaSub(StoppingCriteria):
+    def __init__(self, stops=[], encounters=1):
+        super().__init__()
+        self.stops = stops
+
+    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor):
+        for stop in self.stops:
+            if torch.all((stop == input_ids[0][-len(stop):])).item():
+                return True
+
+        return False
+
+
+class XComposer(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = False
+
+    def __init__(self, model_path='internlm/internlm-xcomposer-vl-7b', **kwargs):
+        assert model_path is not None
+        self.model_path = model_path
+
+        model = AutoModel.from_pretrained(self.model_path, device_map='cpu', trust_remote_code=True).cuda().eval()
+        tokenizer = AutoTokenizer.from_pretrained(self.model_path, trust_remote_code=True)
+        model.tokenizer = tokenizer
+        self.model = model
+        self.device = self.model.internlm_model.model.embed_tokens.weight.device
+        self.eoh = '<TOKENS_UNUSED_0>'
+        self.eoa = '<TOKENS_UNUSED_1>'
+        stop_words_ids = [
+            torch.tensor([103027]).to(self.device),  # end of human
+            torch.tensor([103028]).to(self.device),  # end of bot
+        ]
+        default_kwargs = {
+            'max_new_tokens': 512, 'num_beams': 5, 'do_sample': False,
+            'min_length': 1, 'repetition_penalty': 1.5, 'length_penalty': 1.0
+        }
+        default_kwargs.update(kwargs)
+        self.kwargs = default_kwargs
+        self.stopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])
+
+    def generate_inner(self, message, dataset=None):
+        if len(message) == 2:
+            if message[0]['type'] == 'text' and message[1]['type'] == 'image':
+                message = [message[1], message[0]]
+        kwargs = cp.deepcopy(self.kwargs)
+        if dataset is not None:
+            if DATASET_TYPE(dataset) == 'MCQ':
+                kwargs['max_new_tokens'] = 5
+                kwargs['num_beams'] = 5
+
+        with torch.cuda.amp.autocast():
+            with torch.no_grad():
+                prompt_embs = self.message_to_prompt_embs(message, dataset)
+                outputs = self.model.internlm_model.generate(
+                    inputs_embeds=prompt_embs,
+                    stopping_criteria=self.stopping_criteria,
+                    **kwargs
+                )
+
+        output_token = outputs[0]
+        if output_token[0] == 0:
+            output_token = output_token[1:]
+        if output_token[0] == 1:
+            output_token = output_token[1:]
+        output_text = self.model.tokenizer.decode(output_token, add_special_tokens=False)
+
+        output_text = output_text.split(self.model.eoa)[0]
+        output_text = output_text.split('<|Bot|>')[-1].strip()
+        return output_text
+
+    def message_to_prompt_embs(self, message, dataset=None):
+        assert isinstance(message, list)
+        img_embeds = []
+        prompt_full = '<|User|>: '
+        for msg in message:
+            if msg['type'] == 'text':
+                prompt_full += msg['value']
+            elif msg['type'] == 'image':
+                image = Image.open(msg['value']).convert('RGB')
+                image = self.model.vis_processor(image).unsqueeze(0).to(self.device)
+                img_embeds.append(self.model.encode_img(image))
+                prompt_full += '<ImageHere>'
+
+        prompt_full += self.model.eoh + ' <|Bot|>: '
+        if dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+            prompt_full += 'Answer: The answer is '
+        elif dataset is not None and DATASET_TYPE(dataset) in ['VQA', 'QA', 'Y/N']:
+            prompt_full += 'Answer: '
+
+        prompt_segs = prompt_full.split('<ImageHere>')
+        assert len(prompt_segs) == len(img_embeds) + 1
+
+        prompt_seg_tokens = [
+            self.model.tokenizer(seg, return_tensors='pt', add_special_tokens=(i == 0)).to(self.device).input_ids.long()
+            for i, seg in enumerate(prompt_segs)
+        ]
+        prompt_seg_embs = [self.model.internlm_model.model.embed_tokens(seg) for seg in prompt_seg_tokens]
+        all_embeddings = []
+        for i in range(len(img_embeds)):
+            all_embeddings.extend([prompt_seg_embs[i], img_embeds[i]])
+        all_embeddings.append(prompt_seg_embs[-1])
+        prompt_embs = torch.cat(all_embeddings, dim=1)
+        return prompt_embs
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'MCQ':
+            return True
+        return False
+
+    def build_prompt(self, line, dataset=None):
+        assert dataset is None or isinstance(dataset, str)
+        assert self.use_custom_prompt(dataset)
+        tgt_path = self.dump_image(line, dataset)
+
+        question = line['question']
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        options_prompt = ''
+        for key, item in options.items():
+            options_prompt += f'{key}. {item}\n'
+        hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+        context = 'N/A' if hint is None else hint
+        mid_prompt = 'Context: ' + context + '\nQuestion: ' + question
+        if len(options_prompt):
+            mid_prompt += '\nOptions: ' + options_prompt
+
+        if len(options):
+            txt_prompt = 'Please answer this question by choosing the correct choice.'
+        else:
+            txt_prompt = 'Please answer this question directly. '
+        prompt = txt_prompt + mid_prompt
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+        return message
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2.py
new file mode 100644
index 0000000000000000000000000000000000000000..17e90d21dc6796d4bbe09e2a36a0d5b19ef441ac
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2.py
@@ -0,0 +1,218 @@
+import re
+import string
+
+import pandas as pd
+import torch
+import torchvision
+from PIL import Image
+from transformers import AutoModel, AutoTokenizer
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+pattern = re.compile(r'[A-Z]')
+
+
+def __padding__(image):
+    width, height = image.size
+    tar = max(width, height)
+    top_padding = int((tar - height) / 2)
+    bottom_padding = tar - height - top_padding
+    left_padding = int((tar - width) / 2)
+    right_padding = tar - width - left_padding
+    image = torchvision.transforms.functional.pad(image, [left_padding, top_padding, right_padding, bottom_padding])
+    return image
+
+
+meta_instruction = """
+You are an AI assistant whose name is InternLM-XComposer (浦语·灵笔).
+- InternLM-XComposer (浦语·灵笔) is a multi-modality conversational language model that is developed by
+Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
+- InternLM-XComposer (浦语·灵笔) can understand and communicate fluently in the language
+chosen by the user such as English and 中文.
+- InternLM-XComposer (浦语·灵笔) is capable of comprehending and articulating responses effectively
+based on the provided image.
+"""
+
+
+def model_gen(model, text, images, need_bos=True, padding=False, beams=3, max_token=500):
+    pt1 = 0
+    embeds = []
+    im_mask = []
+    images = [images]
+    images_loc = [0]
+    for i, pts in enumerate(images_loc + [len(text)]):
+        subtext = text[pt1:pts]
+        if need_bos or len(subtext) > 0:
+            text_embeds = model.encode_text(subtext, add_special_tokens=need_bos)
+            embeds.append(text_embeds)
+            im_mask.append(torch.zeros(text_embeds.shape[:2]).cuda())
+            need_bos = False
+        if i < len(images):
+            try:
+                image = Image.open(images[i]).convert('RGB')
+            except Exception:
+                image = images[i].convert('RGB')
+            if padding:
+                image = __padding__(image)
+            image = model.vis_processor(image).unsqueeze(0).cuda()
+            image_embeds = model.encode_img(image)
+            embeds.append(image_embeds)
+            im_mask.append(torch.ones(image_embeds.shape[:2]).cuda())
+        pt1 = pts
+    embeds = torch.cat(embeds, dim=1)
+    im_mask = torch.cat(im_mask, dim=1)
+    im_mask = im_mask.bool()
+
+    outputs = model.generate(
+        inputs_embeds=embeds,
+        im_mask=im_mask,
+        temperature=1.0,
+        max_new_tokens=max_token,
+        num_beams=beams,
+        do_sample=False,
+        repetition_penalty=1.0)
+
+    output_token = outputs[0]
+    if output_token[0] == 0 or output_token[0] == 1:
+        output_token = output_token[1:]
+    output_text = model.tokenizer.decode(output_token, add_special_tokens=False)
+    output_text = output_text.split('[UNUSED_TOKEN_145]')[0].strip()
+    return output_text
+
+
+class XComposer2(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = False
+
+    def __init__(self, model_path='internlm/internlm-xcomposer2-vl-7b', **kwargs):
+        assert model_path is not None
+        self.model_path = model_path
+
+        model = AutoModel.from_pretrained(self.model_path, device_map='cpu', trust_remote_code=True).cuda().eval()
+        model.half()
+        tokenizer = AutoTokenizer.from_pretrained(self.model_path, trust_remote_code=True)
+        model.tokenizer = tokenizer
+        self.model = model
+        self.device = self.model.model.tok_embeddings.weight.device
+
+    def generate_mme(self, image_path, text):
+        text = text.split('Please answer')[0].strip()
+        text = f'{text} Answer this question briefly'
+        text = f'[UNUSED_TOKEN_146]user\n{text}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+
+        return model_gen(self.model, text, image_path, need_bos=True, padding=True, beams=5)
+
+    def generate_multichoice(self, image_path, text, dataset):
+        out = model_gen(self.model, text, image_path, need_bos=True, padding=False, beams=5, max_token=5)
+        if 'mmmu' in dataset.lower():
+            return out
+        res = pattern.findall(out)
+        if len(res) == 0:
+            print('Error:', out)
+            res = 'Z'
+        return res[0]
+
+    def generate_vqa(self, image_path, text):
+        out = model_gen(self.model, text, image_path, need_bos=True)
+        return out
+
+    def generate_vanilla(self, image_path, text):
+        text = (
+            '[UNUSED_TOKEN_146]system\n{}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]user\n{}'
+            'Answer this question in detail.[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+        ).format(meta_instruction, text)
+        out = model_gen(self.model, text, image_path, need_bos=True, max_token=500)
+        return out
+
+    def generate_brief(self, image_path, text):
+        text = (
+            '[UNUSED_TOKEN_146]user\nAnswer the question using a single word or phrase.{}'
+            '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+        ).format(text)
+        out = model_gen(self.model, text, image_path, need_bos=True, max_token=10)
+        return out
+
+    def generate_driectly(self, image_path, text):
+        text = '[UNUSED_TOKEN_146]user\n{}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'.format(text)
+        out = model_gen(self.model, text, image_path, need_bos=True, max_token=500)
+        return out
+
+    def generate_inner(self, message, dataset=None):
+        prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+        with torch.cuda.amp.autocast():
+            if dataset is None:
+                return self.generate_vanilla(image_path, prompt)
+            assert isinstance(dataset, str)
+            if dataset == 'MME':
+                return self.generate_mme(image_path, prompt)
+
+            elif listinstr(['hallu'], dataset.lower()):
+                return self.generate_brief(image_path, prompt)
+
+            elif listinstr(['llava'], dataset.lower()):
+                return self.generate_vanilla(image_path, prompt)
+
+            elif dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+                return self.generate_multichoice(image_path, prompt, dataset)
+
+            elif dataset is not None and DATASET_TYPE(dataset) == 'VQA':
+                return self.generate_vqa(image_path, prompt)
+
+            else:
+                return self.generate_vanilla(image_path, prompt)
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'MCQ' or DATASET_TYPE(dataset) == 'VQA':
+            return True
+        return False
+
+    def build_mcqa(self, line):
+        question = line['question']
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        img_prompt = '[UNUSED_TOKEN_146]user\n'
+        if len(options):
+            options_prompt = ''
+            for key, item in options.items():
+                options_prompt += f'{key}. {item} '
+            options_prompt = options_prompt.strip()
+            hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+
+            context = 'N/A' if hint is None else hint
+            mid_prompt = 'Question: ' + question + '\nContext: ' + context + '\nOptions: ' + options_prompt
+            ans_prompt = '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\nThe answer is'
+            prompt = img_prompt + mid_prompt + ans_prompt
+        else:
+            mid_prompt = f'Answer the question using a single word or phrase.{question}'
+            ans_prompt = '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            prompt = img_prompt + mid_prompt + ans_prompt
+
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert dataset is None or isinstance(dataset, str)
+        assert self.use_custom_prompt(dataset)
+        tgt_path = self.dump_image(line, dataset)
+
+        if DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_mcqa(line)
+        elif DATASET_TYPE(dataset) == 'VQA':
+            if 'mathvista' in dataset.lower():
+                q = line['question']
+                prompt = f'[UNUSED_TOKEN_146]user\n{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            else:
+                q = line['question']
+                prompt = (
+                    f'[UNUSED_TOKEN_146]user\nAnswer the question using a single word or phrase.{q}'
+                    '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+                )
+        message = [dict(type='text', value=prompt)]
+        message.extend([dict(type='image', value=s) for s in tgt_path])
+        return message
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2_4KHD.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2_4KHD.py
new file mode 100644
index 0000000000000000000000000000000000000000..e99a102cbea3a71f25a6a1f01dd2f49f56bdc000
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2_4KHD.py
@@ -0,0 +1,244 @@
+import re
+import string
+
+import numpy as np
+import pandas as pd
+import torch
+import torchvision.transforms as transforms
+from PIL import Image
+from transformers import AutoModel, AutoTokenizer
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+pattern = re.compile(r'[A-Z]')
+
+
+def padding_336(b):
+    width, height = b.size
+    tar = int(np.ceil(height / 336) * 336)
+    top_padding = int((tar - height) / 2)
+    bottom_padding = tar - height - top_padding
+    left_padding = 0
+    right_padding = 0
+    b = transforms.functional.pad(b, [left_padding, top_padding, right_padding, bottom_padding], fill=[255, 255, 255])
+
+    return b
+
+
+def HD_transform(img, im_num=16):
+    width, height = img.size
+    trans = False
+    if width < height:
+        img = img.transpose(Image.TRANSPOSE)
+        trans = True
+        width, height = img.size
+    ratio = (width / height)
+    scale = 1
+    while scale * np.ceil(scale / ratio) <= im_num:
+        scale += 1
+    scale -= 1
+    new_w = int(scale * 336)
+    new_h = int(new_w / ratio)
+
+    img = transforms.functional.resize(img, [new_h, new_w],)
+    img = padding_336(img)
+    width, height = img.size
+    assert width * height <= im_num * 336 * 336
+    if trans:
+        img = img.transpose(Image.TRANSPOSE)
+
+    return img
+
+
+meta_instruction = """You are an AI assistant whose name is InternLM-XComposer (浦语·灵笔).
+- InternLM-XComposer (浦语·灵笔) is a multi-modality conversational language model that is developed\
+ by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
+- InternLM-XComposer (浦语·灵笔) can understand and communicate fluently in the language chosen by\
+ the user such as English and 中文.
+- InternLM-XComposer (浦语·灵笔) is capable of comprehending and articulating responses\
+ effectively based on the provided image."""
+
+
+def model_gen(model, text, images, need_bos=True, padding=False, beams=3, max_token=500):
+    pt1 = 0
+    embeds = []
+    im_mask = []
+    images = [images]
+    images_loc = [0]
+    for i, pts in enumerate(images_loc + [len(text)]):
+        subtext = text[pt1:pts]
+        if need_bos or len(subtext) > 0:
+            text_embeds = model.encode_text(subtext, add_special_tokens=need_bos)
+            embeds.append(text_embeds)
+            im_mask.append(torch.zeros(text_embeds.shape[:2]).cuda())
+            need_bos = False
+        if i < len(images):
+            try:
+                image = Image.open(images[i]).convert('RGB')
+            except Exception:
+                image = images[i].convert('RGB')
+
+            image = HD_transform(image, im_num=model.hd_num)
+            image = model.vis_processor(image).unsqueeze(0).cuda()
+            image_embeds = model.encode_img(image)
+            embeds.append(image_embeds)
+            im_mask.append(torch.ones(image_embeds.shape[:2]).cuda())
+        pt1 = pts
+    embeds = torch.cat(embeds, dim=1)
+    im_mask = torch.cat(im_mask, dim=1)
+    im_mask = im_mask.bool()
+
+    outputs = model.generate(inputs_embeds=embeds, im_mask=im_mask,
+                             temperature=1.0, max_new_tokens=max_token, num_beams=beams,
+                             do_sample=False, repetition_penalty=1.0)
+    output_token = outputs[0]
+    if output_token[0] == 0 or output_token[0] == 1:
+        output_token = output_token[1:]
+    output_text = model.tokenizer.decode(output_token, add_special_tokens=False)
+    output_text = output_text.split('[UNUSED_TOKEN_145]')[0].strip()
+    return output_text
+
+
+class XComposer2_4KHD(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = False
+
+    def __init__(self, model_path='internlm/internlm-xcomposer2-4khd-7b', **kwargs):
+        assert model_path is not None
+        self.model_path = model_path
+
+        model = AutoModel.from_pretrained(self.model_path, device_map='cpu', trust_remote_code=True).cuda().eval()
+        model.half()
+        tokenizer = AutoTokenizer.from_pretrained(self.model_path, trust_remote_code=True)
+        model.tokenizer = tokenizer
+        self.model = model
+        self.device = self.model.model.tok_embeddings.weight.device
+        self.model.hd_num = 25
+
+    def generate_mme(self, image_path, text):
+        text = text.split('Please answer')[0].strip()
+        text = f'{text} Answer this question briefly'
+        text = f'[UNUSED_TOKEN_146]user\n{text}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+
+        return model_gen(self.model, text, image_path, need_bos=True, padding=True, beams=5)
+
+    def generate_multichoice(self, image_path, text, dataset):
+        out = model_gen(self.model, text, image_path, need_bos=True, padding=False, beams=5, max_token=5)
+        if 'mmmu' in dataset.lower():
+            return out
+        res = pattern.findall(out)
+        if len(res) == 0:
+            print('Error:', out)
+            res = 'Z'
+        return res[0]
+
+    def generate_vqa(self, image_path, text):
+        out = model_gen(self.model, text, image_path, need_bos=True, max_token=100)
+        return out
+
+    def generate_vanilla(self, image_path, text):
+        out = model_gen(self.model, text, image_path, need_bos=True, max_token=500)
+        return out
+
+    def generate_brief(self, image_path, text):
+        text = '[UNUSED_TOKEN_146]user\nAnswer the question using a single word or phrase.{}\
+               [UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'.format(text)
+        out = model_gen(self.model, text, image_path, need_bos=True, max_token=10)
+        return out
+
+    def generate(self, message, dataset=None):
+        prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+        if listinstr(['docvqa_test', 'infovqa_test'], dataset.lower()):
+            self.model.hd_num = 65
+        elif listinstr(['docvqa_val', 'infovqa_val', 'OCRBench'], dataset.lower()):
+            self.model.hd_num = 55
+        elif listinstr(['mmlongbench_doc'], dataset.lower()):
+            self.model.hd_num = 45
+        elif listinstr(['mmmu', 'mmbench', 'mmvet'], dataset.lower()):
+            self.model.hd_num = 16
+        else:
+            self.model.hd_num = 25
+
+        with torch.cuda.amp.autocast():
+            if dataset is None:
+                return self.generate_vanilla(image_path, prompt)
+            assert isinstance(dataset, str)
+            if dataset == 'MME':
+                return self.generate_mme(image_path, prompt)
+
+            elif listinstr(['hallu'], dataset.lower()):
+                return self.generate_brief(image_path, prompt)
+
+            elif listinstr(['llava', 'mmvet'], dataset.lower()):
+                return self.generate_vanilla(image_path, prompt)
+
+            elif dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+                return self.generate_multichoice(image_path, prompt, dataset)
+
+            elif dataset is not None and DATASET_TYPE(dataset) == 'VQA':
+                return self.generate_vqa(image_path, prompt)
+
+            else:
+                return self.generate_vanilla(image_path, prompt)
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'MCQ' or DATASET_TYPE(dataset) == 'VQA':
+            return True
+        return False
+
+    def build_mcqa(self, line):
+        question = line['question']
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        img_prompt = '[UNUSED_TOKEN_146]user\n'
+        if len(options):
+            options_prompt = ''
+            for key, item in options.items():
+                options_prompt += f'{key}. {item} '
+            options_prompt = options_prompt.strip()
+            hint = line['hint'] if ('hint' in line and not pd.isna(line['hint'])) else None
+
+            context = 'N/A' if hint is None else hint
+            mid_prompt = 'Question: ' + question + '\nContext: ' + context + '\nOptions: ' + options_prompt
+            ans_prompt = '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\nThe answer is'
+            prompt = img_prompt + mid_prompt + ans_prompt
+        else:
+            mid_prompt = f'Answer the question using a single word or phrase.{question}'
+            ans_prompt = '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            prompt = img_prompt + mid_prompt + ans_prompt
+
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert dataset is None or isinstance(dataset, str)
+        assert self.use_custom_prompt(dataset)
+        tgt_path = self.dump_image(line, dataset)
+
+        if DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_mcqa(line)
+        elif DATASET_TYPE(dataset) == 'VQA':
+            if 'mathvista' in dataset.lower():
+                q = line['question']
+                prompt = f'[UNUSED_TOKEN_146]user\n{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            elif listinstr(['llava', 'mmvet'], dataset.lower()):
+                q = line['question']
+                prompt = '[UNUSED_TOKEN_146]system\n{}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]user\n{}\
+                         Answer this question in detail.[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]\
+                         assistant\n'.format(meta_instruction, q)
+            elif listinstr(['mmlongbench_doc'], dataset.lower()):
+                q = line['question']
+                prompt = f'[UNUSED_TOKEN_146]user\n{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            else:
+                q = line['question']
+                prompt = f'[UNUSED_TOKEN_146]user\nAnswer the question using a single word or phrase.\
+                          {q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+        ret = [dict(type='text', value=prompt)]
+        ret.extend([dict(type='image', value=s) for s in tgt_path])
+        return ret
diff --git a/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2d5.py b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2d5.py
new file mode 100644
index 0000000000000000000000000000000000000000..88215a616c63a995c73b42027828c7acd4678f5c
--- /dev/null
+++ b/reference/code/Fast-dLLM/third_party/VLMEvalKit/vlmeval/vlm/xcomposer/xcomposer2d5.py
@@ -0,0 +1,412 @@
+import logging
+import re
+import string
+from urllib.request import urlopen
+
+import numpy as np
+import pandas as pd
+import torch
+import torchvision.transforms as transforms
+from PIL import Image, ImageDraw, ImageFont
+from transformers import AutoModel, AutoTokenizer
+
+from vlmeval.dataset import DATASET_TYPE
+from vlmeval.smp import listinstr
+from ..base import BaseModel
+
+pattern = re.compile(r'[A-Z]')
+conv_pattern = '\\[UNUSED_TOKEN_146\\]user\\\n|\\[UNUSED_TOKEN_146\\]assistant\\\n|\\[UNUSED_TOKEN_145\\]'
+
+
+def get_font():
+    try:
+        truetype_url = "https://opencompass.openxlab.space/utils/Fonts/SimHei.ttf"
+        ff = urlopen(truetype_url)
+        font = ImageFont.truetype(ff, size=40)
+    except Exception as e:
+        logging.warning(f'{type(e)}: {e}')
+        logging.warning("Fail to download the font. Use the default one.")
+        font = ImageFont.load_default(size=40)
+    return font
+
+
+def padding_560(b):
+    width, height = b.size
+    tar = int(np.ceil(height / 560) * 560)
+    top_padding = int((tar - height) / 2)
+    bottom_padding = tar - height - top_padding
+    left_padding = 0
+    right_padding = 0
+    b = transforms.functional.pad(
+        b, [left_padding, top_padding, right_padding, bottom_padding], fill=[255, 255, 255])
+
+    return b
+
+
+def Identity_transform(img, hd_num=25):
+    width, height = img.size
+    trans = False
+    if width < height:
+        img = img.transpose(Image.TRANSPOSE)
+        trans = True
+        width, height = img.size
+    ratio = (width / height)
+    scale = 1
+    new_h = int(scale * 560)
+    new_w = int(new_h * ratio)
+    # print (new_h, new_w)
+
+    img = transforms.functional.resize(img, [new_h, new_w],)
+    img = img.transpose(Image.TRANSPOSE)
+    img = padding_560(img)
+    width, height = img.size
+    if not trans:
+        img = img.transpose(Image.TRANSPOSE)
+
+    return img
+
+
+def HD_transform(img, im_num=36, id_scale=1.5):
+    width, height = img.size
+    trans = False
+    if width < height:
+        img = img.transpose(Image.TRANSPOSE)
+        trans = True
+        width, height = img.size
+    ratio = (width / height)
+    scale = 1
+    while scale * np.ceil(scale / ratio) <= im_num:
+        scale += 1
+    scale -= 1
+
+    scale = min(np.ceil(width * id_scale / 560), scale)
+    new_w = int(scale * 560)
+    new_h = int(new_w / ratio)
+
+    try:
+        img = transforms.functional.resize(img, [new_h, new_w])
+    except Exception:
+        print((height, width))
+        print((new_h, new_w))
+        print(im_num)
+    img = padding_560(img)
+    width, height = img.size
+    assert width * height <= im_num * 560 * 560
+    if trans:
+        img = img.transpose(Image.TRANSPOSE)
+
+    return img
+
+
+def img_process(imgs):
+    new_imgs = []
+    for img in imgs:
+        w, h = img.size
+        scale = w / h
+        if w > h:
+            new_w = 560 * 2
+            new_h = int(560 * 2 / scale)
+        else:
+            new_w = int(560 * 2 * scale)
+            new_h = 560 * 2
+        img = transforms.functional.resize(img, [new_h, new_w],)
+        new_imgs.append(img)
+    imgs = new_imgs
+    new_w = 0
+    new_h = 0
+    pad = 40
+    if w > h:
+        for im in imgs:
+            w, h = im.size
+            new_w = max(new_w, w)
+            new_h += h + 10 + pad
+        font = get_font()
+        new_img = Image.new('RGB', (new_w, new_h), 'white')
+        draw = ImageDraw.Draw(new_img)
+        curr_h = 0
+        for idx, im in enumerate(imgs):
+            w, h = im.size
+            new_img.paste(im, (0, pad + curr_h))
+            draw.text((0, curr_h), f'<IMAGE {idx}>', font=font, fill='black')
+            if idx + 1 < len(imgs):
+                draw.line([(0, pad + curr_h + h + 5), (new_w, pad + curr_h + h + 5)], fill='black', width=2)
+            curr_h += h + 10 + pad
+        # print (new_w, new_h)
+    else:
+        for im in imgs:
+            w, h = im.size
+            new_w += w + 10
+            new_h = max(new_h, h)
+        new_h += pad
+        font = get_font()
+        new_img = Image.new('RGB', (new_w, new_h), 'white')
+        draw = ImageDraw.Draw(new_img)
+        curr_w = 0
+        for idx, im in enumerate(imgs):
+            w, h = im.size
+            new_img.paste(im, (curr_w, pad))
+            draw.text((curr_w, 0), f'<IMAGE {idx}>', font=font, fill='black')
+            if idx + 1 < len(imgs):
+                draw.line([(curr_w + w + 5, 0), (curr_w + w + 5, new_h)], fill='black', width=2)
+            curr_w += w + 10
+    return new_img
+
+
+meta_instruction = """You are an AI assistant whose name is InternLM (书生·浦语).\n" + "- InternLM (书生·浦语) \
+is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室).
+It is designed to be helpful, honest, and harmless.\n"+"- InternLM (书生·浦语) \
+can understand and communicate fluently in the language chosen by the user such as English and 中文."""
+
+
+def model_gen(model, text, images, need_bos=True, padding=False, beams=3, max_token=2048, video_input=False):
+    embeds = []
+    im_mask = []
+    # print(text)
+
+    im_idx = 0
+    sub_q = text.split('<IM_POS>')
+    add_im = len(sub_q) - 1
+    for subtext in sub_q:
+        if need_bos or len(subtext) > 0:
+            text_embeds = model.encode_text(
+                subtext, add_special_tokens=need_bos)
+            embeds.append(text_embeds)
+            im_mask.append(torch.zeros(text_embeds.shape[:2]).to(model.device))
+            need_bos = False
+
+        if im_idx < len(images) and add_im:
+            image = images[im_idx]
+            if video_input:
+                image = Identity_transform(image)
+            else:
+                if len(images) > 1:
+                    image = HD_transform(image, im_num=max(4, model.hd_num // len(images)), id_scale=model.id_scale)
+                else:
+                    image = HD_transform(
+                        image, im_num=model.hd_num, id_scale=model.id_scale)
+            # print(image.size)
+            image = model.vis_processor(image).unsqueeze(0).to(model.device)
+            image_embeds = model.encode_img(image)
+            im_idx += 1
+            add_im -= 1
+            embeds.append(image_embeds)
+            im_mask.append(torch.ones(
+                image_embeds.shape[:2], dtype=torch.long).to(model.device))
+
+    embeds = torch.cat(embeds, dim=1)
+    im_mask = torch.cat(im_mask, dim=1)
+    im_mask = im_mask.bool()
+
+    outputs = model.generate(inputs_embeds=embeds, im_mask=im_mask,
+                             temperature=1.0, max_new_tokens=max_token, num_beams=beams,
+                             do_sample=False, repetition_penalty=1.0)
+
+    output_token = outputs[0]
+    if output_token[0] == 0 or output_token[0] == 1:
+        output_token = output_token[1:]
+    output_text = model.tokenizer.decode(output_token, add_special_tokens=False)
+    output_text = output_text.split('[UNUSED_TOKEN_145]')[0].strip().split('<|im_end|>')[0].strip().split('The answer is')[-1].strip()  # noqa
+    # print(output_text)
+    return output_text
+
+
+class XComposer2d5(BaseModel):
+
+    INSTALL_REQ = False
+    INTERLEAVE = True
+
+    def __init__(self, model_path='internlm/internlm-xcomposer2d5-7b', id_scale=1.5, beam=3, **kwargs):
+        assert model_path is not None
+        self.model_path = model_path
+        self.id_scale = id_scale
+        self.beam = beam
+
+        model = AutoModel.from_pretrained(
+            self.model_path, device_map='cpu', trust_remote_code=True, local_files_only=True).cuda().eval()
+        model.half()
+        tokenizer = AutoTokenizer.from_pretrained(
+            self.model_path, trust_remote_code=True)
+        model.tokenizer = tokenizer
+        self.model = model
+        self.device = self.model.model.tok_embeddings.weight.device
+        self.org_hd_num = 36
+        self.model.hd_num = 36
+        self.model.id_scale = self.id_scale
+
+    def message_to_promptimg(self, message, dataset=None, video_input=False):
+        num_images = len([x for x in message if x['type'] == 'image'])
+        if num_images == 0:
+            prompt = '\n'.join([x['value']
+                               for x in message if x['type'] == 'text'])
+            image = None
+
+        else:
+            image = [Image.open(x['value']).convert('RGB') for x in message if x['type'] == 'image']
+            if len(image) > 1:
+                self.model.hd_num = max(1, self.org_hd_num // len(image))
+            else:
+                self.set_max_num(dataset)
+
+            if video_input:
+                im_prompt = '<IM_POS>Here are some frames of a video.'
+                if len(image) > 64:
+                    step = len(image) / 64
+                    image = [image[int(i * step)] for i in range(64)]
+                image = [img_process(image)]
+
+            else:
+                if len(image) > 1:
+                    im_prompt = ' '.join([
+                        f'Image{im_idx + 1}: <IM_POS>;' for im_idx in range(len(image))])
+                else:
+                    im_prompt = '<IM_POS>'
+
+            prompt = ''
+            for x in message:
+                if x['type'] == 'text' and x.get('role', '') != 'system':
+                    prompt += x['value']
+            sp = [i for i in re.split(conv_pattern, prompt) if i != '' and i != '\n']
+            assert len(sp) <= 2
+            q = sp[0]
+            prompt = f'[UNUSED_TOKEN_146]user\n{im_prompt}{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+
+            for idx in range(10):
+                idx = chr(65 + idx)
+                prompt = prompt.replace(f'({idx})', f'{idx}.')
+
+        return prompt, image
+
+    def generate_mme(self, image_path, text):
+        text = text.split('Please answer')[0].strip()
+        text = f'{text} Answer this question briefly'
+        text = f'[UNUSED_TOKEN_146]user\n{text}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+
+        return model_gen(self.model, text, image_path, need_bos=True, padding=True, beams=self.beam)
+
+    def generate_multichoice(self, image_path, text, dataset):
+        out = model_gen(self.model, text, image_path,
+                        need_bos=True, padding=False, beams=self.beam, max_token=5)
+        if 'mmmu' in dataset.lower():
+            return out
+        res = pattern.findall(out)
+        if len(res) == 0:
+            print('Error:', out)
+            res = 'Z'
+        return res[0]
+
+    def generate_vqa(self, image_path, text):
+        out = model_gen(self.model, text, image_path, beams=self.beam,
+                        need_bos=True, max_token=100)
+        return out
+
+    def generate_vanilla(self, image_path, text):
+        out = model_gen(self.model, text, image_path, beams=self.beam,
+                        need_bos=True, max_token=500)
+        return out
+
+    def generate_brief(self, image_path, text):
+        text = '[UNUSED_TOKEN_146]user\nAnswer the question using a single word or phrase.{}\
+               [UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'.format(text)
+        out = model_gen(self.model, text, image_path, beams=self.beam,
+                        need_bos=True, max_token=10)
+        return out
+
+    def generate_video(self, image_path, text):
+        out = model_gen(
+            self.model, text, image_path, beams=1,  # self.beam,
+            need_bos=True, max_token=100, video_input=True)
+        return out
+
+    def set_max_num(self, dataset):
+        if dataset is not None and listinstr(['MME-RealWorld', 'MME-RealWorld-CN'], dataset):
+            self.model.hd_num = 25
+        else:
+            self.model.hd_num = self.org_hd_num
+
+    def generate_inner(self, message, dataset=None):
+        with torch.cuda.amp.autocast():
+            if dataset is None:
+                prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+                return self.generate_vanilla(image_path, prompt)
+            assert isinstance(dataset, str)
+
+            if listinstr(['video', 'mvbench'], dataset.lower()):
+                prompt, image_path = self.message_to_promptimg(message, dataset=dataset, video_input=True)
+                return self.generate_video(image_path, prompt)
+            else:
+                prompt, image_path = self.message_to_promptimg(message, dataset=dataset)
+                if dataset == 'MME':
+                    return self.generate_mme(image_path, prompt)
+                elif listinstr(['hallu', 'pope'], dataset.lower()):
+                    return self.generate_brief(image_path, prompt)
+                elif listinstr(['llava', 'mmvet'], dataset.lower()):
+                    return self.generate_vanilla(image_path, prompt)
+                elif dataset is not None and DATASET_TYPE(dataset) == 'MCQ':
+                    return self.generate_multichoice(image_path, prompt, dataset)
+                elif listinstr(['MME-RealWorld', 'MME-RealWorld-CN'], dataset):
+                    return self.generate_multichoice(image_path, prompt, dataset)
+                elif dataset is not None and DATASET_TYPE(dataset) == 'VQA':
+                    return self.generate_vqa(image_path, prompt)
+                else:
+                    return self.generate_vanilla(image_path, prompt)
+
+    def use_custom_prompt(self, dataset):
+        assert dataset is not None
+        if DATASET_TYPE(dataset) == 'MCQ' or DATASET_TYPE(dataset) == 'VQA':
+            return True
+        return False
+
+    def build_mcqa(self, line):
+        question = line['question']
+        options = {
+            cand: line[cand]
+            for cand in string.ascii_uppercase
+            if cand in line and not pd.isna(line[cand])
+        }
+        img_prompt = '[UNUSED_TOKEN_146]user\n'
+        if len(options):
+            options_prompt = ''
+            for key, item in options.items():
+                options_prompt += f'{key}. {item} '
+            options_prompt = options_prompt.strip()
+            hint = line['hint'] if (
+                'hint' in line and not pd.isna(line['hint'])) else None
+
+            context = 'N/A' if hint is None else hint
+            mid_prompt = 'Question: ' + question + '\nContext: ' + \
+                context + '\nOptions: ' + options_prompt
+            ans_prompt = '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\nThe answer is'
+            prompt = img_prompt + mid_prompt + ans_prompt
+        else:
+            mid_prompt = f'Answer the question using a single word or phrase.{question}'
+            ans_prompt = '[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            prompt = img_prompt + mid_prompt + ans_prompt
+
+        return prompt
+
+    def build_prompt(self, line, dataset=None):
+        assert dataset is None or isinstance(dataset, str)
+        assert self.use_custom_prompt(dataset)
+        tgt_path = self.dump_image(line, dataset)
+
+        if DATASET_TYPE(dataset) == 'MCQ':
+            prompt = self.build_mcqa(line)
+        elif DATASET_TYPE(dataset) == 'VQA':
+            if 'mathvista' in dataset.lower():
+                q = line['question']
+                prompt = f'[UNUSED_TOKEN_146]user\n{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            elif listinstr(['llava', 'mmvet'], dataset.lower()):
+                q = line['question']
+                prompt = '[UNUSED_TOKEN_146]system\n{}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]user\n{}\
+                         Answer this question in detail.[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]\
+                         assistant\n'.format(meta_instruction, q)
+            elif listinstr(['mmlongbench_doc', 'dude', 'slidevqa'], dataset.lower()):
+                q = line['question']
+                prompt = f'[UNUSED_TOKEN_146]user\n{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+            else:
+                q = line['question']
+                prefix = 'Answer the question using a single word or phrase.'
+                prompt = f'[UNUSED_TOKEN_146]user\n{prefix}{q}[UNUSED_TOKEN_145]\n[UNUSED_TOKEN_146]assistant\n'
+        ret = [dict(type='text', value=prompt)]
+        ret.extend([dict(type='image', value=s) for s in tgt_path])
+        return ret