Update README.md

README.md

@@ -1,141 +1 @@
----
-license: mit
-pipeline_tag: any-to-any
-library_name: transformers
-datasets:
-- wyddmw/OpenREAD
-base_model:
-- Qwen/Qwen3-VL-8B-Instruct
----
-
-<div align="center">
-
-<h1>
-OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic
-</h1>
-
-<p align="center">
-  [![arxiv paper](https://img.shields.io/badge/arXiv-Paper-red)](https://arxiv.org/abs/2512.01830)
-  [![🤗 HuggingFace Paper](https://img.shields.io/badge/HuggingFace%F0%9F%A4%97-Paper-orange)](https://huggingface.co/papers/2512.01830)
-  [![Project Page](https://img.shields.io/badge/Project_Page-%F0%9F%8F%A0-blue)](https://wyddmw.github.io/OpenREAD)
-  [![GitHub Code](https://img.shields.io/badge/GitHub-Code-181717.svg?logo=github)](https://github.com/wyddmw/OpenREAD)
-  [![🤗 HuggingFace Models](https://img.shields.io/badge/HuggingFace%F0%9F%A4%97-Models-orange)](https://huggingface.co/wyddmw/OpenREAD/tree/main)
-  [![🤗 HuggingFace Datasets](https://img.shields.io/badge/HuggingFace%F0%9F%A4%97-Datasets-orange)](https://huggingface.co/datasets/wyddmw/OpenREAD)
-</p>
-
-This repository hosts the OpenREAD model, an OPEN-ended REasoning reinforced vision-language model (VLM)-based autonomous driving (AD) framework, as presented in the paper [OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic](https://huggingface.co/papers/2512.01830).
-
-OpenREAD enables end-to-end Reinforcement Fine-Tuning (RFT) across the full spectrum from high-level reasoning to low-level trajectory planning. It constructs large-scale Chain-of-Thought (CoT) annotations and employs the powerful Qwen3 Large Language Model (LLM) as the critic in RFT to quantify reasoning quality for open-ended questions during reward modeling. Extensive experiments confirm that joint end-to-end RFT yields substantial improvements in both upstream and downstream tasks, enabling OpenREAD to achieve state-of-the-art performance on reasoning and planning benchmarks.
-
-<img src="https://huggingface.co/wyddmw/OpenREAD/resolve/main/asset/framework.png"/><br>
-An overview of the framework of our OpenREAD.
-
-</div>
-
-## ✨Capabilities
-
-<img src="https://huggingface.co/wyddmw/OpenREAD/resolve/main/asset/planning.png"/>
-<img src="https://huggingface.co/wyddmw/OpenREAD/resolve/main/asset/lingoqa.png"/>
-
-An overview of the capability of our proposed OpenREAD, a vision-language model tailored for autonomous driving by reinforcement learning with GRPO. Besides the trajectory planning, our OpenREAD is also capable of providing reasoning-enhanced response for open-ended scenario understanding, action analysis, *etc*.
-
-## 🦙 Data & Model Zoo
-Our OpenREAD is built upon the [Qwen3-VL-8B](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) and finetuned on a mixture of datasets including LingoQA, OmniDrive, and NuScenes datasets. Our OpenREAD is now available at [huggingface](https://huggingface.co/wyddmw/OpenREAD). Enjoy playing with it!
-
-<img src="https://huggingface.co/wyddmw/OpenREAD/resolve/main/asset/annotations.png"/>
-
-To facilitate the learning of reasoning capability at the cold start stage, we construct a large scale of CoT annotations on the LingoQA and NuScenes datasets as shown above. We further extend the amount of annotations for LingoQA from 7K to 11K. All the CoT annotations are available [here](https://huggingface.co/datasets/wyddmw/OpenREAD).
-
-## 🛠️ Install
-
-1.  Clone this repository and navigate to OpenREAD folder
-    ```bash
-    git clone https://github.com/wyddmw/OpenREAD
-    cd OpenREAD
-    ```
-
-2.  Install `ms-swift` package
-    ```shell
-    conda create -n openread python=3.10 -y
-    conda activate openread
-    pip install -e .
-    ```
-
-3.  Install Flash-Attention.
-    ```shell
-    pip install flash_attn=2.8.3 --no-build-isolation
-    ```
-    If the installation is not compatible for your device and environment, please refer to the [source code](https://github.com/Dao-AILab/flash-attention/releases) and install the suitable version.
-
-4.  Install Qwen3-VL dependencies.
-    ```shell
-    pip install "transformers==4.57" "qwen_vl_utils==0.0.14"
-    ```
-
-## 🪜 Training & Evaluation
-
-### Datasets
-The datasets used to train OpenREAD are as follows:
-*   [NuScenes](https://www.nuscenes.org/nuscenes)
-*   [LingoQA](https://github.com/wayveai/LingoQA)
-*   [OmniDrive](https://github.com/NVlabs/OmniDrive)
-
-Please download our pre-processed [Lidar-BEV](https://huggingface.co/datasets/wyddmw/NuScenes_LidarBev) images for the NuScenes dataset. For trajectory evaluation, we use the GT cache introduced in [GPT-Driver](https://github.com/PointsCoder/GPT-Driver). Please download the GT cache from [Google Drive](https://drive.google.com/drive/folders/1NCqPtdK8agPi1q3sr9-8-vPdYj08OCAE)
-The datasets are organized in the following structure:
-```
-data
-├── LingoQA
-│   ├── action
-│   │   └── images
-│   ├── evaluation
-│   │   │── images
-│   │   └── val.parquet
-│   ├── scenery
-│   │   └── images
-│   ├── training_data.json
-│   └── evaluation_data.json
-├── nuscenes
-│   ├── samples
-│   │   ├── CAM_FRONT
-│   │   ├── LIDAR_BEV
-│   ├── gt
-│   │   │── vad_gt_seg.pkl
-│   │   └── gt_traj_mask.pkl
-│   traj_val_bev_ego_status.json
-│   
-```
-
-It is recommended to symlink your dataset root to `data`:
-
-### Evaluate on the LingoQA dataset.
-Before running the evaluation script, please first download the pretrained [Lingo-Judge](wayveai/Lingo-Judge).
-Check the path of LingoQA dataset and LingoJudge pretrained model in the `eval/LingoQA/eval_lingo.sh`.
-```shell
-sh eval/LingoQA/eval_lingo.sh
-```
-The predictions, Lingo-Judge, CIDEr, Meteor, and BLEU metrics will be saved to the `eval/LingoQA/lingoqa_results_OpenREAD.json`.
-
-### Evaluation on the NuScenes Trajectory Benchmark
-We also provide scripts to evaluate trajectory prediction quality on the NuScenes validation set using both STP-3 and UniAD metrics. Update the trained model path, eval_file path, training mode, and inference outputs path in the `eval/Trajectory/infer_trajs_dist.sh`, then run trajectory inference:
-```shell
-bash eval/Trajectory/infer_trajs_dist.sh
-```
-This script generates trajectory prediction JSON files under the directory specified by inference outputs path. Next, update the trajectory inference outputs path inside `eval/Trajectory/eval_trajs.py`, Then compute both STP-3 and UniAD metrics by running:
-```shell
-python eval/Trajectory/eval_trajs.py
-```
-
-## Acknowledgment
-
-We appreciate the awesome open-source project of [ms-swift](https://github.com/modelscope/ms-swift), [OmniDrive](https://github.com/NVlabs/OmniDrive), and [GPT-Driver](https://github.com/PointsCoder/GPT-Driver).
-
-## ✏️ Citation
-If you find our work helpful or inspiring, please feel free to cite it.
-```bibtex
-@article{zhang2025openread,
-  title={OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic},
-  author={Zhang, Songyan and Huang, Wenhui and Chen, Zhan and Collister, Chua Jiahao and Huang, Qihang and Lv, Chen},
-  journal={arXiv preprint arXiv:2512.01830},
-  year={2025}
-}
-```
+This is the model ckpts for our WiseAD. For more details, please refer to our github repo https://github.com/wyddmw/WiseAD