File size: 5,246 Bytes
79960aa 3874ec8 670988f 3874ec8 79960aa d12caac 79960aa 3874ec8 79960aa 3874ec8 79960aa 58b01a9 79960aa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
---
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
datasets:
- luzimu/webgen-agent_train_step-grpo
- luzimu/webgen-agent_train_sft
license: mit
pipeline_tag: image-text-to-text
library_name: transformers
---
# WebGen-Agent
WebGen-Agent is an advanced website generation agent designed to autonomously create websites from natural language instructions. It was introduced in the paper [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://arxiv.org/pdf/2509.22644v1).
Code: https://github.com/mnluzimu/WebGen-Agent
## Project Overview
WebGen-Agent combines state-of-the-art language models with specialized training techniques to create a powerful website generation tool. The agent can understand natural language instructions specifying appearance and functional requirements, iteratively generate website codebases, and refine them using visual and functional feedback.
## Resources
Links to the data and model parameters are as follows:
| **Data** | **HF Link** |
|----------|------|
| **webgen-agent_train_sft** | 🤗 [luzimu/webgen-agent_train_sft](https://huggingface.co/datasets/luzimu/webgen-agent_train_sft) |
| **webgen-agent_train_step-grpo** | 🤗 [luzimu/webgen-agent_train_step-grpo](https://huggingface.co/datasets/luzimu/webgen-agent_train_step-grpo) |
| **Model** | **HF Link** |
|----------|------|
| **WebGenAgent-LM-7B-SFT** | 🤗 [luzimu/WebGenAgent-LM-7B-SFT](https://huggingface.co/luzimu/WebGenAgent-LM-7B-SFT) |
| **WebGenAgent-LM-7B-Step-GRPO** | 🤗 [luzimu/WebGenAgent-LM-7B-Step-GRPO](https://huggingface.co/luzimu/WebGenAgent-LM-7B-Step-GRPO) |
| **WebGenAgent-LM-8B-SFT** | 🤗 [luzimu/WebGenAgent-LM-8B-SFT](https://huggingface.co/luzimu/WebGenAgent-LM-8B-SFT) |
| **WebGenAgent-LM-8B-Step-GRPO** | 🤗 [luzimu/WebGenAgent-LM-8B-Step-GRPO](https://huggingface.co/luzimu/WebGenAgent-LM-8B-Step-GRPO) |
## How WebGen-Agent Works
WebGen-Agent follows an iterative, multi-step paradigm for website generation:
1. **Code Generation**: The agent generates code to create or edit website files based on natural language instructions
2. **Code Execution**: Dependencies are installed and the website service is started
3. **Feedback Gathering**:
- A screenshot of the website is captured
- A Visual Language Model (VLM) provides appearance feedback and scores
- A GUI-agent tests the website functionality and provides functional feedback
4. **Refinement**: Based on the feedback, the agent continues to improve the website until it meets requirements
## Step-GRPO with Screenshot and GUI-agent Feedback
The Step-GRPO with Screenshot and GUI-agent Feedback approach uses the screenshot and GUI-agent scores inherently produced in the WebGen-Agent workflow as step-level rewards:
- **Screenshot Score**: Quantifies the visual appeal and aesthetics of the website
- **GUI-agent Score**: Measures how well the website meets functional requirements
These dual rewards provide dense, reliable process supervision that significantly improves the model's ability to generate high-quality websites.
## Sample Usage
Before running inference, you should rename `.env.template` to `.env` and set the base urls and api keys for the agent-engine LLM and feedback VLM. They can be obtained from any openai-compatible providers such as [openrouter](https://openrouter.ai/), [modelscope](https://www.modelscope.cn/my/overview), [bailian](https://bailian.console.aliyun.com/#/home), and [llmprovider](https://llmprovider.ai/).
You can also deploy open-source VLMs and LLMs by running `src/scripts/deploy_qwenvl_32b.sh` and `src/scripts/deploy.sh`. Scripts for single inference and batch inference can be found at `src/scripts/infer_single.sh` and `src/scripts/infer_batch.sh`.
```bash
python src/infer_single.py \
--model deepseek-chat \
--vlm_model Qwen/Qwen2.5-VL-32B-Instruct \
--instruction "Please implement a wheel of fortune website." \
--workspace-dir workspaces_root/test \
--log-dir service_logs/test \
--max-iter 20 \
--overwrite \
--error-limit 5
```
## Citation
If you find our project useful, please cite:
```
@misc{lu2025webgenagentenhancinginteractivewebsite,
title={WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning},
author={Zimu Lu and Houxing Ren and Yunqiao Yang and Ke Wang and Zhuofan Zong and Junting Pan and Mingjie Zhan and Hongsheng Li},
year={2025},
eprint={2509.22644},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2509.22644},
}
@misc{lu2025webgenbenchevaluatingllmsgenerating,
title={WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch},
author={Zimu Lu and Yunqiao Yang and Houxing Ren and Haotian Hou and Han Xiao and Ke Wang and Weikang Shi and Aojun Zhou and Mingjie Zhan and Hongsheng Li},
year={2025},
eprint={2505.03733},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.03733},
}
``` |