Improve: Update model card with abstract, correct license, and add tags/resources

README.md

@@ -1,13 +1,24 @@
 ---
 base_model: OpenGVLab/InternVL2-4B
 library_name: transformers
-license: apache-2.0
+license: mit
 pipeline_tag: image-text-to-text
+tags:
+  - vlm
+  - gui-agent
+  - multimodal
+  - android
+  - web
+  - computer-vision
+  - natural-language-processing
 ---
 
 # OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
 
-This model is described in the paper [OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis](https://huggingface.co/papers/2412.19723)
+This model is described in the paper [OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis](https://huggingface.co/papers/2412.19723).
+
+## Abstract
+Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. Despite their utility in advancing digital automation, a critical bottleneck persists: collecting high-quality trajectory data for training. Common practices for collecting such data rely on human supervision or synthetic data generation through executing pre-defined tasks, which are either resource-intensive or unable to guarantee data quality. Moreover, these methods suffer from limited data diversity and significant gaps between synthetic data and real-world environments. To address these challenges, we propose OS-Genesis, a novel GUI data synthesis pipeline that reverses the conventional trajectory collection process. Instead of relying on pre-defined tasks, OS-Genesis enables agents first to perceive environments and perform step-wise interactions, then retrospectively derive high-quality tasks to enable trajectory-level exploration. A trajectory reward model is then employed to ensure the quality of the generated trajectories. We demonstrate that training GUI agents with OS-Genesis significantly improves their performance on highly challenging online benchmarks. In-depth analysis further validates OS-Genesis's efficiency and its superior data quality and diversity compared to existing synthesis methods. Our codes, data, and checkpoints are available at this https URL .
 
 <div align="center">
 
@@ -23,8 +34,8 @@ We introduce OS-Genesis, an interaction-driven pipeline that synthesizes high-qu
 ## Quick Start
 OS-Genesis-8B-WA is a mobile action model finetuned from [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B).
 
-###  OS-Genesis AC Family Models
-In the following table, we provide an overview of the OS-Genesis AC Family Models used for evaluating the AndroidControl Benchmark.
+### OS-Genesis WebArena Family Models
+In the following table, we provide an overview of the OS-Genesis WA Family Models used for evaluating the WebArena Benchmark.
 
 |   Model Name    |                           Base Model                                            |                           Training Data                                            |                           HF Link                           |
 | :-------------: | :-------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------: | :---------------------------------------------------------: |
@@ -150,8 +161,23 @@ print(f'User: {question}
 Assistant: {response}')
 ```
 
+## More Resources
+### Raw collected triples
+
+In addition to our complete trajectory data on HuggingFace, we also provide collected raw `<s_pre, a, s_post>` triples. You can use them to reproduce the process of reverse task synthesis directly, without re-collecting them from emulators yourself 😄. The screenshots and corresponding texts (with SoM info contained) are provided below:
+
+|   Data Type    |                           Screenshots                                            |                           Data JSON                                            |
+| :-------------: | :-------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------: |
+| Mobile | [Screenshots](https://drive.google.com/file/d/1ILyz_-DDOdAk32kue1lEPaV50YzQ5c4v/view?usp=sharing) | [Data JSON](https://drive.google.com/file/d/1dSxNf-co4LGh93NoiUgWKdbcf8Mo_VWG/view?usp=sharing) |
+| Web | [Screenshots](https://drive.google.com/file/d/1X2QktZ51OUofZ43vDGB4RuAPlXbdf5ua/view?usp=sharing) | [Data JSON](https://drive.google.com/file/d/1mDxhonGnd3wZbNQgWMVpYEkPW26_FVg8/view?usp=sharing) |
+
+Feel free to email me if you require additional data of this kind.
+
+## FAQ ❓
+
+We have collected some questions from emails, Hugging Face, and WeChat communications. Please check the [FAQ](https://github.com/OS-Copilot/OS-Genesis/blob/main/faq.md) 🤖
 
-## Citation
+## Citation 📖
 If you find this repository helpful, feel free to cite our paper:
 ```bibtex
 @article{sun2024osgenesis,