DesignCoder

Checkpoint collection for DesignCoder, a family of full-parameter SFT models for UI design research and end-to-end HTML/CSS/JavaScript implementation.

Each subfolder in this repository is a self-contained, directly loadable checkpoint.

Naming convention

designcoder_{basemodel}_{size}_{optimizer}_bs{global_batch}[_{extra_axes}]_step{global_step}

• basemodel / size: base model family and parameter scale
• optimizer: muon or adamw
• bs: global batch size (per_device × grad_accum × world_size)
• extra_axes: any hyper-parameter that deviates from the default recipe, e.g. wd0.05 (weight decay, default 0.0), ep20 (epochs, default 2), or data41287 (dataset revision)
• step: trainer global_step of the exported weights

Dataset revisions

Checkpoints in this repository come from two different dataset revisions. Scores and loss values are only comparable within the same revision.

Tag	Samples	Used by
(untagged) data37865	37,865	*_step1900, *_step3800
data41287	41,287	*_data41287_step200, *_data41287_step400

Checkpoints

Subfolder	Base model	Optimizer	LR	Global batch	Dataset	Step	bench-200 V5.9.2 (n=200)	Notes
designcoder_qwen3.5_4b_muon_bs32_step1900	Qwen3.5-4B	Muon	1e-5	32	37,865	1900	–	smallest of the first release
designcoder_qwen3.5_9b_muon_bs16_step3800	Qwen3.5-9B	Muon	1e-5	16	37,865	3800	–	optimizer ablation (Muon arm)
designcoder_qwen3.5_9b_adamw_bs16_step3800	Qwen3.5-9B	AdamW	2e-5	16	37,865	3800	–	optimizer ablation (AdamW arm)
designcoder_qwen3.6_27b_adamw_bs32_step1900	Qwen3.6-27B	AdamW	1e-5	32	37,865	1900	–	largest of the first release
designcoder_qwen3.5_4b_adamw_bs256_data41287_step200	Qwen3.5-4B	AdamW	2e-5	256	41,287	200	80.06	best 4B / AdamW
designcoder_qwen3.5_4b_muon_bs256_data41287_step200	Qwen3.5-4B	Muon	2e-5	256	41,287	200	71.34	best 4B / Muon
designcoder_qwen3.5_9b_adamw_bs256_data41287_step200	Qwen3.5-9B	AdamW	2e-5	256	41,287	200	82.14	best 9B
designcoder_qwen3.8_27b_adamw_bs128_data41287_step400	Qwen3.8-27B	AdamW	1e-5	128	41,287	400	87.04	strongest checkpoint in the collection

All four data41287 scores are final full-benchmark runs: 200/200 rollouts, 200/200 screenshot captures, 200/200 judge evaluations per model, scored with the complete V5.9.2 rubric set (all three families) — no subsetting, no omitted rubric family.

Benchmark

bench-200 is the frozen 200-case DesignCoder benchmark (100 Track A landing, 40 Track A dashboard, 30 Track B landing, 30 Track B dashboard; Track A cases specify a style, Track B cases are style-free).

Rubric composition. Scoring uses three independent rubric families. Only the first varies per case; the other two are fixed for every case of a given surface.

Family	Scope	Size	Scale
Frozen	per case	23–25 checks/case, 4,983 total (184 cases carry 25, 15 carry 24, 1 carries 23)	binary 0/1
Prompt Fit & Product	fixed per surface	5 rubrics	0/1/2
Static	fixed per surface	landing 27 d_* + 8 q_*; dashboard 25 d_* + 9 q_*	d_* 2/0/N-A, q_* 0/1/2/N-A

Frozen checks are distributed across six dimensions: Components 1,517 (30.4%), Layout 842 (16.9%), Aesthetics 782 (15.7%), Typography 642 (12.9%), Alignment 616 (12.4%), Assets 584 (11.7%). Every check is check_with=screenshot.

overall_score (0–100) is the unweighted mean of top-level slots: Prompt Fit (1 slot), each active Static dimension (1 slot each), and the whole Frozen family (1 slot). Aggregation is performed by the reference collector DesignEvaluator-Skill/scripts/collect_unified.py, not by a reimplementation. Judge: gpt-5.6-sol (vision) with structured JSON output.

Full-run results (n=200 per model)

Model	Overall (V5.9.2)	Prompt Fit	Static	Frozen	Landing	Dashboard	Track A	Track B	Render fails
27B AdamW step400	87.04	84.10	87.31	88.52	89.65	82.19	86.56	88.16	0/200
9B AdamW step200	82.14	78.35	82.20	85.41	86.96	73.19	82.71	80.82	1/200
4B AdamW step200	80.06	75.25	80.40	83.27	85.70	69.59	80.61	78.78	1/200
4B Muon step200	71.34	64.60	70.96	79.79	78.48	58.06	72.05	69.68	3/200

Scores are monotone in scale. Five of the six pairwise differences are significant (paired bootstrap 10k-resample 95% CI excludes 0 and Wilcoxon p < 4e-7); 9B vs 4B AdamW is not significant (mean diff +2.08, CI [-0.03, 4.18], p = 0.065). See eval/significance_tests_v592.json.

Two observations that only the full rubric set exposes:

• Dashboards are the bottleneck, and they degrade faster than landings. The 27B loses 7.5 points moving from landing to dashboard; the 4B Muon loses 20.4. The Static family's chart (d_data_*) and workflow (d_work_*) checks catch empty or non-functional charts that the Frozen checks largely miss.
• The Frozen family alone compresses the ranking. Across the four models Frozen spans only 8.7 points (88.52 → 79.79) while Prompt Fit spans 19.5 and Static spans 16.4. Reporting Frozen-heavy scores therefore understates the gap between scales.

Evaluation artifacts (eval/)

File	Content
eval/benchmark_summary_v592.csv	per-model aggregates: overall, three family scores, Track/Surface splits, Static and Frozen dimensions
eval/benchmark_per_case_v592.csv	long-form per-case scores (overall + three families) for all 4 models × 200 cases
eval/significance_tests_v592.json	paired bootstrap (10k resamples) + Wilcoxon signed-rank for all 6 model pairs
eval/rubric_stats.json	composition of all three rubric families and the aggregation rule
eval/reports.html	self-contained interactive HTML report: model comparison, dimension heatmap, score distributions, per-case tables

Checkpoint selection

The data41287 checkpoints were selected by running the benchmark, not by taking the lowest training loss. In all four runs the best checkpoint sits at roughly 75% of training, and loss kept improving while benchmark scores fell. The table below shows the 8-case selection subset (used only to rank checkpoints, not comparable to the final full-run numbers in the tables above):

Run	Step	Train loss	subset bench (n=8, Frozen+Prompt-Fit only)	final V5.9.2 (n=200)
4B AdamW	200	0.2696	84.22	80.06
4B AdamW	266	0.2682	68.35	–
4B Muon	200	0.3339	83.36	71.34
4B Muon	266	0.3349	81.27	–
9B AdamW	200	0.2518	84.40	82.14
9B AdamW	266	0.2504	lowest of the three	–
27B AdamW	400	0.2067	91.19	87.04
27B AdamW	530	0.2059	86.37	–

The 4B AdamW pair is the clearest example: loss improved from 0.2696 to 0.2682 while the subset score collapsed from 84.22 to 68.35. Do not pick checkpoints from this family by loss.

The two score columns are not comparable: the selection subset used 8 cases and only two of the three rubric families, and it overestimates by 4–12 points, with the largest error on the weakest model. It is reliable enough to rank checkpoints within a run, which is all it was used for — every number reported elsewhere in this card is the full 200-case V5.9.2 score.

Shared training setup

• Objective: full-parameter supervised fine-tuning (no LoRA / adapters)
• Dataset: designcoder_sft_v2_train in ShareGPT format (see revision table above)
• Chat template: qwen3_5 with thinking enabled
• Context length: 32,768
• Sequence packing: enabled, with neat packing (no cross-sample attention)
• LR schedule: cosine, warmup ratio 0.1

Usage

from transformers import AutoModelForCausalLM, AutoProcessor

repo = "xingxm/DesignCoder"
subfolder = "designcoder_qwen3.8_27b_adamw_bs128_data41287_step400"

model = AutoModelForCausalLM.from_pretrained(repo, subfolder=subfolder, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(repo, subfolder=subfolder)

To download a single checkpoint only:

hf download xingxm/DesignCoder --include "designcoder_qwen3.8_27b_adamw_bs128_data41287_step400/*" --local-dir ./DesignCoder

Inference contract

These models are trained as tool-using agents, not single-turn generators. A case runs design_search → (websearch, landing only) → a final answer containing exactly three code blocks in the order html, css, js. Reproduce the system prompts and tool observation format from examples/designcoder/runtime/infer_designcoder.py; prompting with a bare instruction and no tool turns does not match the training distribution and will score far below the numbers above.

Provenance

Each subfolder additionally ships trainer_state.json / trainer_log.jsonl (and training_loss.png where available) so that the loss curve and exact step schedule of the run can be recovered from the checkpoint itself.