---
license: other
base_model: qwen/Qwen3-4B
tags:
- text-generation
- 3d
- mesh
---

# output_40k_cont4ep

Qwen3-4B fine-tuned for **3D mesh understanding & generation** (SPAC pipeline, Spatial-PE enabled), continued from the `output_40k_cont2ep` weights for 4 more epochs.

## Training data (`bpe_40k_mix`, 40k samples)

- 20k mesh -> text: given a mesh-token sequence, produce a text description (understanding).
- 20k text -> mesh: given a caption + "Reconstruct this 3D shape in mesh token format:", output the mesh token sequence (generation).

Both directions are paired on the same shapes.

## Training details

- Full fine-tune (all params), DeepSpeed ZeRO-3, bf16, FlashAttention-2, packing (8192, neat).
- +4 epochs continuation from `output_40k_cont2ep` (fresh optimizer, fresh LR schedule).
- LR 5e-5 cosine decay to 5e-6, warmup 3%, 4 GPUs, batch 1 each.
- Train loss: epoch means 0.844 -> 0.660 -> 0.254 -> 0.052 (whole-run avg 0.4524).

## Inference notes

- Chat template: `qwen3_nothink` (no thinking). Mesh shapes are tokenized as `<mesh_start><morton_*><mesh_*>` tokens (see `added_tokens.json`).
- Requires the SPAC Spatial-PE inference wrapper (`spatial_pe_scale 0.08`, merge table from `Jack04810/spac-qwen3-4b-spatial-pe-8ep`) to run with the spatial positional encoding used in training.