---
license: apache-2.0
library_name: transformers
base_model: Qwen/Qwen3.5-4B
pipeline_tag: image-text-to-text
tags:
  - designcoder
  - qwen3_5
  - sft
  - lora
  - merged
  - web-design
---

# DesignCoder 4B — AdamW SFT, step 12588

This repository contains the **merged BF16 model** from a locally trained
Qwen3.5-4B LoRA SFT checkpoint, `qwen35-4b-design/lora-r64-sft/checkpoint-12588`.
The root directory is a complete Hugging Face model export; loading it does not
require PEFT or a separate base-model download.

This is the AdamW checkpoint selected for DesignCoder RL initialization. It is
an SFT model, not an RL-trained checkpoint. It replaces the previously uploaded
Muon step1900 model on the main branch; that version remains in repository history.

## Source and training

| Item | Value |
| --- | --- |
| Base | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B/tree/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a) |
| Base revision | `851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a` |
| Optimizer | `adamw_torch` |
| Checkpoint | step 12588, 2 epochs |
| Dataset identifier | `design_sft_sharegpt_with_think` |
| LoRA | rank 64, alpha 128, dropout 0.05 |
| Learning rate | `1e-4`, cosine schedule, warmup ratio 0.03 |
| Training batch | 1 per device, 8 accumulation steps, effective batch 8 |
| Training cutoff | 16384 tokens |
| Export | FP32 LoRA merge with `safe_merge=True`, then BF16 safetensors |

The tokenizer, processor configuration and chat template are copied from the
selected training checkpoint. The vocabulary matches the base model. Standard
Transformers loading omits the base checkpoint's auxiliary MTP tensors, which
are not used by `Qwen3_5ForConditionalGeneration`.

The original adapter weights are also included under [`adapter/`](adapter/).
Their SHA256 is
`7d8305739df470c3d7906241a14ae37b342e1ba61f34b3d0341e69b97f85f12b`.
The adapter config uses the public base-model ID and pinned revision instead of
the original local filesystem path. Do not apply this adapter to the already
merged model at the repository root.

## Load the merged model

The export was checked with Transformers 5.8.0, PEFT 0.18.1 and PyTorch 2.6.0.

```python
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration

model_id = "chjdl1816/designcoder-mc"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id, dtype="auto", device_map="auto"
)
messages = [{"role": "user", "content": "Build a minimal portfolio homepage."}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

For reproducible training, pin the repository commit when downloading.

## Validation and artifacts

- All source adapter tensors were compared against their loaded values.
- FP32 inference before and after the LoRA merge was compared on a short text prompt.
- The BF16 export was reloaded and its logits were checked for exact equality
  with the in-memory BF16 model on that prompt.
- Tokenizer, processor and model configuration loading were checked.
- [`merge_validation.json`](merge_validation.json) contains the numerical results;
  [`training_summary.json`](training_summary.json) records training metadata;
  [`upload_manifest.json`](upload_manifest.json) records source and artifact hashes.

These checks validate export integrity. No new benchmark, GPU inference or
vLLM end-to-end RL evaluation was performed for this upload.

## License

Apache-2.0, following the Qwen3.5-4B base model. See [LICENSE](LICENSE).