Instructions to use luzimu/WebGenAgent-LM-8B-Step-GRPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use luzimu/WebGenAgent-LM-8B-Step-GRPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="luzimu/WebGenAgent-LM-8B-Step-GRPO")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("luzimu/WebGenAgent-LM-8B-Step-GRPO")
model = AutoModelForCausalLM.from_pretrained("luzimu/WebGenAgent-LM-8B-Step-GRPO")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use luzimu/WebGenAgent-LM-8B-Step-GRPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "luzimu/WebGenAgent-LM-8B-Step-GRPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luzimu/WebGenAgent-LM-8B-Step-GRPO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/luzimu/WebGenAgent-LM-8B-Step-GRPO

SGLang

How to use luzimu/WebGenAgent-LM-8B-Step-GRPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "luzimu/WebGenAgent-LM-8B-Step-GRPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luzimu/WebGenAgent-LM-8B-Step-GRPO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "luzimu/WebGenAgent-LM-8B-Step-GRPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "luzimu/WebGenAgent-LM-8B-Step-GRPO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use luzimu/WebGenAgent-LM-8B-Step-GRPO with Docker Model Runner:
```
docker model run hf.co/luzimu/WebGenAgent-LM-8B-Step-GRPO
```

luzimu commited on Aug 31, 2025

Commit

771caa6

verified ·

1 Parent(s): ac06d32

Update README.md

Browse files

Files changed (1) hide show

README.md +74 -3

README.md CHANGED Viewed

@@ -1,3 +1,74 @@
----
-license: mit
----

+---
+license: mit
+---
+# WebGen-Agent
+WebGen-Agent is an advanced website generation agent designed to autonomously create websites from natural language instructions. It was introduced in the paper [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](fig/WebGen_Agent.pdf).
+## Project Overview
+WebGen-Agent combines state-of-the-art language models with specialized training techniques to create a powerful website generation tool. The agent can understand natural language instructions specifying appearance and functional requirements, iteratively generate website codebases, and refine them using visual and functional feedback.
+![WebGen-Agent Workflow](fig/webgen-agent.png)
+![Step-GRPO with Screenshot and GUI-agent Feedback](fig/step-grpo.png)
+## Resources
+Links to the data and model parameters are as follows:
+| Data | HF Link |
+|----------|------|
+| webgen-agent_train_sft | 🤗 [luzimu/webgen-agent_train_sft](https://huggingface.co/datasets/luzimu/webgen-agent_train_sft) |
+| webgen-agent_train_step-grpo | 🤗 [luzimu/webgen-agent_train_step-grpo](https://huggingface.co/datasets/luzimu/webgen-agent_train_step-grpo) |
+| Model | HF Link |
+|----------|------|
+| WebGenAgent-LM-7B-SFT | 🤗 [luzimu/WebGenAgent-LM-7B-SFT](https://huggingface.co/luzimu/WebGenAgent-LM-7B-SFT) |
+| WebGenAgent-LM-7B-Step-GRPO | 🤗 [luzimu/WebGenAgent-LM-7B-Step-GRPO](https://huggingface.co/luzimu/WebGenAgent-LM-7B-Step-GRPO) |
+| WebGenAgent-LM-8B-SFT | 🤗 [luzimu/WebGenAgent-LM-8B-SFT](https://huggingface.co/luzimu/WebGenAgent-LM-8B-SFT) |
+| WebGenAgent-LM-8B-Step-GRPO | 🤗 [luzimu/WebGenAgent-LM-8B-Step-GRPO](https://huggingface.co/luzimu/WebGenAgent-LM-8B-Step-GRPO) |
+## How WebGen-Agent Works
+WebGen-Agent follows an iterative, multi-step paradigm for website generation:
+1. **Code Generation**: The agent generates code to create or edit website files based on natural language instructions
+2. **Code Execution**: Dependencies are installed and the website service is started
+3. **Feedback Gathering**:
+   - A screenshot of the website is captured
+   - A Visual Language Model (VLM) provides appearance feedback and scores
+   - A GUI-agent tests the website functionality and provides functional feedback
+4. **Refinement**: Based on the feedback, the agent continues to improve the website until it meets requirements
+## Key Features
+- **Iterative Refinement**: Continuously improves website appearance and functionality
+- **Feedback Integration**: Uses both visual and functional feedback for enhanced performance
+- **Backtracking Mechanism**: Reverts to previous states when encountering persistent errors
+- **Best Step Selection**: Selects the optimal version based on screenshot and GUI-agent scores
+## Step-GRPO with Screenshot and GUI-agent Feedback
+The Step-GRPO with Screenshot and GUI-agent Feedback approach uses the screenshot and GUI-agent scores inherently produced in the WebGen-Agent workflow as step-level rewards:
+- **Screenshot Score**: Quantifies the visual appeal and aesthetics of the website
+- **GUI-agent Score**: Measures how well the website meets functional requirements
+These dual rewards provide dense, reliable process supervision that significantly improves the model's ability to generate high-quality websites.
+## Citation
+If you find our project useful, please cite:
+```
+@misc{lu2025webgenbenchevaluatingllmsgenerating,
+      title={WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch},
+      author={Zimu Lu and Yunqiao Yang and Houxing Ren and Haotian Hou and Han Xiao and Ke Wang and Weikang Shi and Aojun Zhou and Mingjie Zhan and Hongsheng Li},
+      year={2025},
+      eprint={2505.03733},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2505.03733},
+}
+```