Create README.md

README.md

@@ -0,0 +1,313 @@
+---
+base_model:
+- Qwen/Qwen3-4B-Instruct-2507
+---
+
+# Code Aesthetics with Agentic Reward Feedback
+<div align="center">
+<a href='https://bangx7.github.io/code-aesthetics/'><img src='https://img.shields.io/badge/Project-Page-Green'></a>
+  <br>
+  <a href="https://arxiv.org/abs/2510.23272"><b>Paper Link</b>👁️</a>
+</div>
+<div align="center">
+<p>
+<sup>1,2</sup><a href="https://bangx7.github.io" target="_blank">Bang Xiao</a><sup>#</sup>,</span>
+                <span class="author-block">
+                  <sup>1,3</sup><a href="https://github.com/JackLingjie" target="_blank">Lingjie Jiang</a><sup>#</sup>,</span>
+                  <span class="author-block">
+                    <sup>1</sup><a href="https://www.microsoft.com/en-us/research/people/shaohanh/" target="_blank">Shaohan Huang</a><sup>✉</sup>,</span>
+                    <span class="author-block">
+                      <sup>1</sup><a href="https://www.microsoft.com/en-us/research/people/tengchaolv/" target="_blank">Tengchao Lv</a>,
+                    </span>
+                    <span class="author-block">
+                      <sup>1</sup><a href="https://www.microsoft.com/en-us/research/people/yupanhuang/" target="_blank">Yupan Huang</a>,
+                    </span>
+                    <span class="author-block">
+                      <sup>1</sup><a href="https://yushuiwx.github.io/" target="_blank">Xun Wu</a>,
+                    </span>
+                    <span class="author-block">
+                      <sup>1</sup><a href="https://www.microsoft.com/en-us/research/people/lecu/" target="_blank">Lei Cui</a>,
+                    </span>
+                    <span class="author-block">
+                      <sup>1</sup><a href="https://www.microsoft.com/en-us/research/people/fuwei/" target="_blank">Furu Wei</a>
+                    </span>
+</p>
+  <p>
+    <sup>1</sup>Microsoft Research Asia &nbsp;&nbsp;
+    <sup>2</sup>Zhiyuan College, Shanghai Jiao Tong University &nbsp;&nbsp;
+    <sup>3</sup>Peking University<br>
+    <sup>#</sup>Equal Contribution
+    <sup>✉</sup>Corresponding author
+  </p>
+</div>
+
+**Note: This is the version of AesCoder-4B model for only webpage design.**
+
+## Quickstart
+### VLLM deployment (Recommended)
+
+We recommend using `vllm>=0.8.5` for efficient inference and deployment. Here's how to get started:
+
+**Installation:**
+```bash
+pip install vllm>=0.8.5
+```
+
+**API Server Deployment:**
+To create an OpenAI-compatible API endpoint:
+```bash
+vllm serve SamuelBang/AesCoder-4B --max-model-len 262144
+```
+
+**Using with OpenAI Client:**
+```python
+from openai import OpenAI
+
+# Initialize the client
+client = OpenAI(
+    base_url="http://localhost:8000/v1",
+    api_key="None"
+)
+
+# Generate completion
+response = client.chat.completions.create(
+    model="SamuelBang/AesCoder-4B",
+    messages=[
+        {"role": "user", "content": "Create a user-friendly website for a landing page dedicated to selling dog-related products."}
+    ],
+    temperature=0.8,
+    max_tokens=16384
+)
+
+# Get the generated content
+content = response.choices[0].message.content
+print("Generated content:", content)
+```
+
+**Basic vLLM Usage:**
+```python
+from vllm import LLM, SamplingParams
+
+model_name = "SamuelBang/AesCoder-4B"
+
+# Initialize the model
+llm = LLM(
+    model=model_name,
+    max_model_len=262144,  # Maximum context length
+    tensor_parallel_size=1,  # Adjust based on your GPU setup
+)
+
+# Define sampling parameters
+sampling_params = SamplingParams(
+    temperature=0.8,
+    top_p=0.8,
+    top_k=20,
+    min_p=0,
+    max_tokens=16384
+)
+
+# Prepare the prompt
+prompt = "Create a user-friendly website for a landing page dedicated to selling dog-related products. "
+messages = [
+    {"role": "user", "content": prompt}
+]
+
+# Apply chat template (you'll need to get the tokenizer for this)
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+# Generate text
+outputs = llm.generate([text], sampling_params)
+
+# Get the result
+content = outputs[0].outputs[0].text
+print("Generated content:", content)
+```
+
+**Note:** If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as `32,768` or adjusting the `tensor_parallel_size` based on your available GPU memory.
+
+
+
+### SGLang Deployment
+You can use `sglang>=0.4.6.post1` to create an OpenAI-compatible API endpoint:
+
+```shell
+python -m sglang.launch_server --model-path SamuelBang/AesCoder-4B --context-length 262144
+```
+
+
+**Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as `32,768`.**
+### Use with origin `transformer` package (NOT recommended, very slow)
+
+The code of Qwen3 has been in the latest Hugging Face `transformers` and we advise you to use the latest version of `transformers`.
+
+With `transformers<4.51.0`, you will encounter the following error:
+```
+KeyError: 'qwen3'
+```
+
+The following contains a code snippet illustrating how to use the model generate content based on given inputs. 
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "SamuelBang/AesCoder-4B"
+
+# load the tokenizer and the model
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+
+# prepare the model input
+prompt = "Create a user-friendly website for a landing page dedicated to selling dog-related products. "
+messages = [
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+# conduct text completion
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=16384,
+    temperature=0.8
+)
+output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 
+
+content = tokenizer.decode(output_ids, skip_special_tokens=True)
+
+print("content:", content)
+```
+
+## Best Practices
+### Sampling Parameters
+To achieve optimal performance, we suggest using `Temperature=0.8`, `TopP=0.8`, `TopK=20`, and `MinP=0` for sampling. 
+
+### System Prompt
+To achieve better webpage generation results with our model, please use appropriate system prompts. We have categorized webpage generation into 5 main categories, and the recommended system prompts for each category are as follows (reference to: https://designarena.ai):
+
+Website: 
+```txt
+You are an expert web developer and designer specializing in modern websites. Create a complete, working HTML page with embedded CSS and JavaScript if needed. Feel free to use lightweight libraries like Tailwind CSS to enhance the design as long as they can be rendered in an iframe.
+ Requirements:
+ 1. Create a fully functional, modern, and responsive website design
+ 2. Use only HTML, CSS, and JavaScript, but feel free to use libraries like Tailwind CSS to make the design better. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts as additional libraries can be imported.
+ 3. Include all styles inline within <style> tags
+ 4. Focus on clean layouts, typography, and user experience
+ 5. Implement modern web design trends (gradients, shadows, smooth animations)
+ 6. Ensure excellent mobile responsiveness
+ 7. Include interactive elements where appropriate
+ 8. Make it production-ready and professional
+ 9. You must include all relevant script tags for libraries to work properly.
+ Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.
+```
+
+Game Development: 
+```txt
+You are an expert game developer specializing in browser-based games. Create a complete, working HTML page with an interactive game using embedded CSS and JavaScript.
+ Requirements:
+ 1. Create a fully functional, playable browser game
+ 2. Use HTML, CSS, and JavaScript, but feel free to use libraries Tailwind CSS to make the game better as long as they will render in iframe. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts (and others) can be imported.
+ 3. Include all styles inline within <style> tags
+ 4. Implement game mechanics, controls, and scoring systems
+ 5. Include game states (start screen, gameplay, game over)
+ 6. Add visual feedback, animations, and sound effects using Web Audio API if needed
+ 7. Ensure responsive design that works on both desktop and mobile
+ 8. Make the game engaging and fun to play
+ 9. You must include all relevant script tags for libraries to work properly.
+ Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.
+ ```
+
+ 3D Design
+ ```txt
+ You are an expert in 3D graphics and WebGL. Create a complete, working HTML page with 3D graphics and animations using embedded CSS and JavaScript.
+ Requirements:
+ 1. Create a fully functional 3D scene or application
+ 2. Use only CSS, and vanilla JavaScript with WebGL or CSS 3D transforms. Feel free to use lightweight libraries like Three.js or Babylon.js to make the 3D design better as long as they can be rendered in an iframe. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts (and others) can be imported.
+ 3. Include all styles inline within <style> tags
+ 4. Implement 3D models, animations, and interactive controls
+ 5. Add proper lighting, materials, and camera controls
+ 6. Include smooth animations and user interaction
+ 7. Ensure good performance and responsive design
+ 8. Make it visually impressive and production-ready
+ 9. You must include all relevant script tags for libraries to work properly.
+ Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.
+ ```
+
+
+ Data Visualization
+ ```txt
+ You are an expert in data visualization and interactive charts. Create a complete, working HTML page with dynamic data visualization capabilities using embedded CSS and JavaScript. Feel free to use lightwight libraries (as long as it can be rendered in an iframe) such as Tailwind CSS.
+
+ Requirements:
+ 1. Create a fully functional data visualization application with interactive charts and graphs
+ 2. Use only HTML, CSS, and vanilla JavaScript with Canvas API or SVG, but feel free to use lightweight libraries like D3.js or Chart.js to make the visualizations better as long as they can be rendered in an iframe. Libraries such as ThreeJS, react three fiber, drei, @react-three/postprocessing, @react-three/cannon, d3, and recharts (and others) can be imported.
+ 3. Include all styles inline within <style> tags
+ 4. Ensure responsive design that adapts to different screen sizes
+ 5. Your goal is to make the design of the data visualization top-notch.
+ 6. You must include all relevant script tags for libraries to work properly.
+
+ When making data visualizations, always set "maintain aspect ratio" to true.
+
+ Return ONLY the complete HTML code, starting with <!DOCTYPE html> and ending with </html>. Do not include any explanations or markdown formatting.
+ ```
+
+ UI Component
+ ```txt
+ You are a world-class UI/UX designer and frontend engineer with a sharp eye for aesthetics, accessibility, and modern interaction design. Your task is to generate complete, production-ready HTML pages showcasing **visually stunning, highly interactive, and responsive UI components** using only HTML, CSS, and JavaScript.
+
+**Guidelines:**
+1. Deliver a single, fully functional UI component as a complete HTML page
+2. Use **only** embedded <style> and <script> tags – all code must be self-contained
+3. You may use:
+ - Tailwind CSS (via CDN)
+ - Lightweight icon libraries (e.g., Heroicons)
+ - Three.js, react-three-fiber, drei, d3, recharts (for advanced visuals), and others if you import them properly
+4. Ensure **fully responsive design**, supporting both desktop and mobile layouts
+5. Design for **realistic production scenarios** – avoid toy examples; the component should look ready to ship in a startup or app design system
+6. You must include all relevant script tags for libraries to work properly.
+
+**Design Requirements (unless the user specifies otherwise):**
+- Contemporary visual style: Soft shadows, rounded corners, clean typography, subtle gradients
+- State handling: Show all interactive states (hover, active, loading, disabled, success)
+- Microinteractions: Include smooth transitions and animations for interactive elements
+- Accessibility: Use semantic HTML and ARIA roles where appropriate
+- Use thoughtful spacing, sizing, and visual hierarchy
+
+**Bonus:**
+- Add delight through animations, hover effects, or clever color usage
+- Incorporate a beautiful and functional layout structure, not just the component
+
+Final Output:
+Return ONLY the full, standalone HTML code (starting with <!DOCTYPE html>) and nothing else. No explanations, no markdown formatting.
+
+If the user specifies a particular style (e.g., glassmorphism, brutalism, Material Design), follow their style instructions instead of the default design preferences.
+```
+
+
+
+
+## &#x1F4DA; Citation
+If you find this codebase useful for your research, please use the following entry.
+```BibTeX
+@misc{xiao2025codeaestheticsagenticreward,
+      title={Code Aesthetics with Agentic Reward Feedback}, 
+      author={Bang Xiao and Lingjie Jiang and Shaohan Huang and Tengchao Lv and Yupan Huang and Xun Wu and Lei Cui and Furu Wei},
+      year={2025},
+      eprint={2510.23272},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2510.23272}, 
+}
+```