Spaces:
Running
๐จ DesignGym 2.0 โ Teaching an LLM to Think Like a Designer
What if a machine didn't just generate a layout โ but learned how to improve one?
The Big Idea
Imagine a blank canvas. On it, you can place anything โ a headline, a hero image, a call-to-action, a sofa, a warehouse shelf, a hospital bed, a circuit component. The problem of arrangement is universal.
DesignGym 2.0 asks a radical question:
Can we train an LLM to reason about space โ iteratively, purposefully, the way a designer does?
Not a one-shot generator. Not a template-picker. An agent that looks at a layout, diagnoses what's wrong, chooses a meaningful edit, and gets better over time through feedback.
That's what this project is. And it's bigger than posters.
Figure 1 โ End-to-end architecture: the OpenEnv-compliant DesignGym environment, the heuristic planner that bootstraps SFT data, the SFT adapter that locks in the action interface, and the GRPO trainer that learns design preference from verifiable reward.
๐ The Vision: A Canvas Is a Universal Problem
Here's what hit me building this: graphic design is just one instance of a much deeper problem.
Every time humans arrange objects under constraints to serve a goal, they are solving a layout problem:
- ๐๏ธ Interior Design โ furniture in a room, flow, light, balance
- ๐ญ Warehouse Planning โ shelf placement, picking efficiency, safety zones
- ๐ฅ Hospital Floor Plans โ patient flow, emergency access, quiet zones
- ๐ป UI Dashboards โ information hierarchy, click efficiency, accessibility
- ๐ฅ๏ธ Chip Placement (VLSI) โ routing, thermal zones (a billion-dollar optimization problem)
- ๐ซ Classroom Seating โ learning groups, teacher sightlines, collaborative zones
- ๐ช Retail Layouts โ foot traffic, product discovery, impulse zones
- ๐ฐ Editorial Design โ reading order, visual weight, column balance
- ๐๏ธ Urban Zoning โ density, transit access, green space allocation
The abstract structure is always the same:
DesignGym proves this framework works for graphic design. The architecture is a foundation for any domain where arrangement matters.
๐ Project Links
| Resource | Link |
|---|---|
| ๐ Environment (HF Space) | DesignGym Environment Server |
| ๐ป GitHub Repo | canboyedits/DesignGym |
| ๐ง SFT Trained Adapter | designgym2-sft-qwen05-lora |
| ๐ GRPO Training Notebook | grpo_train_colab.ipynb |
| ๐ SFT Training Notebook | SFT_training_script.ipynb |
| ๐ Evaluation Notebook | evaluate_base_vs_sft.ipynb |
| ๐ Training Logs | HF Training Job |
๐ค Why Not Just Generate the Layout?
Because that's not how it works in the real world.
A good designer doesn't conjure the final poster from thin air. They start somewhere, step back, ask "what's wrong here?", make a targeted edit, look again, and repeat. The process is:
bad layout โ structural choice โ placement โ refinement โ polish โ final design
DesignGym 2.0 formalizes that process as a reinforcement learning environment. The agent learns the workflow of design, not just the output.
This matters because layout improvement is an optimization problem, not a generation problem. And RL is the right tool for optimization.
๐ฌ Round 1 โ Round 2: What Changed?
| Area | DesignGym Round 1 | DesignGym 2.0 |
|---|---|---|
| Core goal | Optimize layouts with structured actions | Learn the process of design over multiple phases |
| Task style | Short-horizon layout refinement | Long-horizon planning + instruction following |
| Agent behavior | Local edit selection | Structure โ placement โ refinement โ polish |
| Reward | Geometry + aesthetic deltas | Delta-aware, instruction-aware, phase-aware |
| Learning | Environment-ready foundation | SFT + GRPO training pipeline |
| Evaluation | Layout quality + validity | Base vs heuristic vs SFT vs GRPO comparison |
Round 1 answered: Can layout design be a real RL environment?
Round 2 answers: Can an LLM learn to act like a designer?
๐งฉ The Environment: How It Works
State
Each design is a canvas of normalized elements. Every element $b_i$ has geometry:
A layout is:
The state given to the agent includes: task ID, step count, current score, best score so far, instruction score, phase score, metric vector, weakest metrics, focus elements, action history, full geometry, and current phase. The agent can reason not just about what the layout looks like but where it is in the design process.
The RL loop:
Actions
The action space spans both pixel-level and structural edits โ because real design isn't just moving boxes:
Primitive actions: move, resize, align, distribute, snap, swap_z
Structural actions: apply_template, promote, reflow_group, anchor_to_region, finalize
For example:
- "Make the title louder" โ
promote - "Clean up the spacing" โ
distributeorreflow_group - "Move the CTA to the footer" โ
anchor_to_region - "Choose a stronger layout" โ
apply_template
Hard Constraints
The feasible layout space is:
Only $L \in \mathcal{F}$ are accepted. Invalid actions are rejected โ forcing the agent to learn legal creativity.
๐ฏ Solving Aesthetics: The Math of "Good Design"
This is where DesignGym gets serious. Rather than asking "does it look good?" (subjective), we ask "what makes it measurably good?" (computable).
My research into computational aesthetics โ from Birkhoff's 1933 aesthetic measure $M = O/C$ to modern Visual Moment Equilibrium models โ confirmed that most design quality signals are mathematically expressible. DesignGym implements seven of them:
The Composite Score
Each $g_k$ is an independent, interpretable aesthetic signal. Together they give a holistic quality score $U(L) \in [0,1]$.
1. Overlap โ Do elements collide?
Exponential penalty for any intersection. Clean layouts score near 1.
2. Alignment โ Do things line up intentionally?
$d_q$ is the distance from element anchors (left, center, right edges) to the nearest alignment guide. Lower distance โ higher score.
3. Spacing โ Is the rhythm intentional?
Coefficient of variation on gaps $\Delta$:
Low variation = consistent rhythm = high score. This captures the difference between accidental spacing and deliberate structure.
4. Balance โ Does visual weight feel stable?
Each element carries visual mass $m_i = a_i \cdot p_i$ (area ร semantic importance). The center of mass:
Balance rewards layouts centered around the canvas midpoint:
Inspired by the Visual Moment Equilibrium model โ treating layout as a physics problem where visual torques must cancel. This achieves Pearson $r = 0.942$ correlation with human perception benchmarks.
5. Hierarchy โ Do big things matter more?
Visual salience per element:
Hierarchy is measured by rank agreement with semantic importance $p_i$ via Spearman correlation:
If the biggest element is also the most important, the layout looks right because it is right.
6. Reading Order โ Does the eye flow correctly?
Given required reading relations $\mathcal{R} = {(i,j)}$:
$i \prec j$ means element $i$ appears before $j$ in the intended scan path (F-pattern, Z-pattern, etc.).
7. Instruction Fit โ Does the layout follow the brief?
Tasks have semantic placement goals (CTA in lower-right, masthead at top, etc.). Given target region center $r_i$ and element center $c_i$:
This is the bridge from pure geometry to design intention โ an element can be perfectly aligned yet still be in the wrong place.
๐ The Reward Function: Teaching Taste
This is the most important design decision in the whole project. A flat reward teaches nothing.
After SFT, nearly every action is valid JSON โ so rewarding "valid JSON" gives GRPO a flat signal with no learning gradient. The reward had to be richer, more informative, and process-aware.
Delta-Sensitive Shaping
Let $S_t$, $I_t$, $P_t$, $B_t$ be layout score, instruction score, phase score, and best-so-far score at step $t$:
With a clipped scaling function that makes small improvements legible:
The full shaped reward:
Where $b_{\text{process}}$ is a phase-aware bonus and $\pi_t$ is a penalty for invalid, useless, or premature actions.
Process-Aware Reward: Diagnosing Before Acting
The reward checks not just whether the layout improved, but whether the agent chose the right kind of action for the current weakness:
| Weak Metric | Preferred Actions |
|---|---|
| Overlap / crowding | move, resize, reflow_group |
| Spacing / rhythm | distribute, align, reflow_group |
| Hierarchy | promote, resize, apply_template |
| Instruction fit | anchor_to_region, promote |
| Early structure phase | apply_template |
| Late polish phase | align, snap, finalize |
This teaches the agent to diagnose โ choose relevantly โ improve โ not just make random valid edits.
Early Finalize Penalty
The agent must not give up early. finalize is only rewarded if:
Otherwise $\pi_{\text{finalize}} > 0$. This enforces long-horizon behavior and prevents the agent from prematurely declaring victory.
๐ Training Pipeline: From Language to Design Policy
Base Model โ Heuristic Planner โ SFT Model โ GRPO Model
Figure 2 โ The heuristic planner solving an episode end-to-end. It diagnoses the weakest metric, picks a phase-appropriate action, applies it, and re-scores โ generating the (state โ action) pairs we use as SFT training data. This is how the agent gets a warm start without human labels.
Why SFT First?
The base model (Qwen 0.5B) understands design language but cannot speak the environment's action format. It says things like:
"Move the title higher and make the CTA stronger."
Semantically meaningful โ but not executable. The environment needs:
{
"action_type": "anchor_to_region",
"element_id": "cta",
"region": "lower_right"
}
SFT teaches the model the interface. The goal is not creativity โ it's action correctness.
Result: valid_json_rate: 0% โ valid_json_rate: 100%. That's not a fine-tune. That's a capability phase transition.
Figure 3 โ SFT training evidence on Qwen-0.5B: loss converges cleanly, validity rates jump to 100%, and the action distribution learned by the model mirrors the heuristic teacher. After SFT the model can act in DesignGym; before SFT it cannot.
Why GRPO After?
Once the model can act, GRPO teaches it which valid action is better.
GRPO uses environment reward directly โ no static labels, no human annotation on every trajectory. The loop:
- Sample multiple candidate actions
- Execute them in DesignGym
- Score each with the verifier
- Increase probability of higher-reward actions
- Decrease probability of lower-reward actions
This works because DesignGym has verifiable rewards โ the environment is the oracle. No reward model needed.
The learning story:
๐ Results: Evidence of Learning
| Policy | Final Score | Instruction Score | Valid JSON | Early Finalize |
|---|---|---|---|---|
| Base Qwen 0.5B | 0.6948 | 0.5360 | 0% | 100% |
| SFT Qwen 0.5B | 0.7101 | 0.6263 | 100% | 0% |
| SFT @ Best-of-4 | 0.7057 | 0.6672 | 100% | 0% |
| GRPO Qwen 0.5B | 0.6717 | 0.5483 | 98% | 67% |
| GRPO @ Best-of-4 โ | 0.6781 | 0.5817 | 100% | 17% |
โ = shipped headline configuration. BaseโFinal pipeline gain on instruction score: +8.5%, on valid JSON: 0% โ 100%, on premature finalizes: down 83%.
The most important result is the jump from 0% to 100% valid actions. That's the model crossing from "knows about design" to "can act in a design environment."
GRPO then learns which valid actions are better โ a harder problem that continues to improve with more training. The infrastructure and reward signal are proven.
๐ ๏ธ Tasks Supported
poster_basic_v1 โ Headline hierarchy, hero image placement, CTA placement, clean spacing, reading order.
editorial_cover_v1 โ Masthead preservation, headline stack, visual balance, editorial hierarchy.
dense_flyer_v1 โ Crowded layout repair, support group reflow, spacing under density, caption alignment.
Three very different design problems. A model that improves across all three has learned general layout reasoning, not template memorization.
๐ฎ How Big Can This Get?
The framework underlying DesignGym isn't about posters. It's about this general structure:
Swap the metrics. Swap the domain. The agent still learns to optimize arrangement through sequential decision-making.
Interior Design: empty room โ zoning โ furniture placement โ spacing โ style polish
VLSI Chip Placement: component placement โ routing โ thermal zones โ signal integrity
Warehouse Logistics: shelf placement โ picking paths โ safety zones โ density optimization
Hospital Layout: patient flow โ emergency access โ staff circulation โ quiet zones
UI Design: information hierarchy โ click paths โ accessibility โ responsive stability
The same RL loop. A different reward function. The canvas is always the problem, and the canvas is everywhere.
๐๏ธ Architecture
.
โโโ client.py # OpenEnv client interface
โโโ inference.py # Policy inference runner
โโโ models.py # Typed action/observation models
โโโ notebooks/
โ โโโ grpo_train_colab.ipynb # GRPO training โ start here
โ โโโ SFT_training_script.ipynb # SFT training
โ โโโ evaluate_base_vs_sft_designgym2.ipynb
โโโ training/
โ โโโ grpo_train.py # GRPO training script
โ โโโ generate_sft_data.py # SFT data generation
โ โโโ run_grpo.sh # Smoke test runner
โโโ server/
โ โโโ app.py # FastAPI wrapper
โ โโโ DesignGym_environment.py # Core environment logic
โ โโโ rewards.py # Reward functions
โ โโโ phases.py # Phase-aware logic
โ โโโ requirements.txt
โโโ data/sft/ # SFT train/eval data
โโโ assets/ # Plots and diagrams
โโโ web/ # Demo web UI
โโโ Dockerfile
โโโ pyproject.toml
โโโ openenv.yaml # OpenEnv manifest
โโโ README.md
โก OpenEnv Compatibility
DesignGym is fully OpenEnv-compatible: reset(), step(action), typed models, FastAPI server, Docker + HF Space deployment, deterministic seeded episodes.
๐ Running It
Install:
pip install -e .
openenv validate
python server/app.py
Docker:
docker build -t designgym-env .
docker run --rm -p 8000:8000 designgym-env
Inference:
HF_TOKEN=your_token \
API_BASE_URL=https://router.huggingface.co/v1 \
MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct:scaleway \
DESIGNGYM_TASK=poster_basic_v1 \
python inference.py
๐งช What's Next
- Image-Aware Reward โ Saliency maps, OCR regions, protected focal zones
- Preference-Trained Reward Model โ Learn from human pairwise layout comparisons
- Style-Conditioned Design โ Luxury, editorial, minimal, corporate as goal inputs
- Interior DesignGym โ Same framework extended to room/furniture layout
- Multi-Agent Critique โ Designer agent proposes โ Critic agent diagnoses โ Designer improves
- Multi-Page Layout โ Magazines, slide decks, reports, product catalogs
๐ก The Bottom Line
Design is usually treated as subjective and unteachable by machine.
DesignGym proves that's not true. Many aspects of layout quality are computable, verifiable, and learnable. And because the underlying framework is general, the same approach extends to any domain where arrangement matters.
The final vision: models that don't just generate designs โ but learn how to improve them, step by step, the way a skilled designer does.
A good designer does not just produce a layout.
A good designer repeatedly diagnoses, edits, compares, and improves.DesignGym 2.0 teaches that process to a machine.
Built for the OpenEnv Hackathon ยท India 2026
Math rendered with KaTeX / MathJax
