--- license: apache-2.0 language: - en tags: - caid - blueprint - hardware - electronics - maker - structured-generation - json - qwen3.5 - vision - unsloth pipeline_tag: image-text-to-text base_model: Qwen/Qwen3.5-9B base_model_relation: finetune library_name: transformers --- # Parti-Vision Base — Qwen3.5-9B **Parti-Vision turns a hardware idea — a sentence, a short brief, even a sketch — into a complete build blueprint.** Tell it what you want to build — *"a USB-powered desk lamp with touch dimming"* — and optionally hand it a short description document and/or a concept image (a hand-drawn sketch or a product render). It gives back one structured blueprint: the parts list, a pin-level wiring map, step-by-step build instructions, costs, and a description of how the finished product should look. Everything comes out as one clean JSON object that an app can read and build on. This is the **all-in-one model** — it runs on its own, no add-ons needed. > **Early research preview.** Great for drafting and exploring ideas — not a replacement for > real engineering, CAD software, or safety review. By [caid-technologies](https://huggingface.co/caid-technologies). --- ## What it can do Give it a hardware idea (and optionally a brief and/or a picture) and it produces **one complete project blueprint** containing: - 📋 a **parts list** (components) with roles and key specs - 🔌 a **pin-level wiring map** — which pin connects to which, net by net - 🛠️ ordered **build instructions**, from fabrication through assembly and testing - 💲 a **sourcing table** with per-part costs that add up to a stated total - 🎨 an **appearance spec** plus a ready-to-use **image-generation prompt** for concept renders - 🖼️ and it can **read images**: show it a napkin sketch or a product render and it designs to match Unlike a chat model, it answers with the whole plan at once — a single JSON object with ten sections (`project, requirements, components, relationships, circuit, fabrication, instructions, appearance, image_generation_prompt, sourcing`). ## What you can give it - ✍️ **A plain-English request** (always) — one or two sentences describing what you want to build. - 📄 **A short description document** (optional) — plain text or Markdown (a design brief, notes, a requirements list). The model can't open files itself: paste the document's text into the user message. It was trained with the document wrapped in markers, so use the same shape: ```text --- ATTACHED DOCUMENT: brief.md --- --- END DOCUMENT --- ``` - 🖼️ **One concept image** (optional) — a **hand-drawn sketch** or a **product render**, passed as a regular vision input. Training images were PNGs; any format your image loader opens (PNG, JPEG, WebP, …) works, since it's decoded to pixels before the model sees it. One image per request is what it was trained on. Any combination works — prompt alone, prompt + document, prompt + image, or all three (the strongest setup in our testing). PDFs, CAD files, and spreadsheets aren't supported directly — convert them to text or an image first. ## What it's good for — and not ✅ **Good for:** brainstorming maker/electronics products, turning a rough idea (or a sketch) into an organized starting plan, and powering apps that need machine-readable project plans. 🚫 **Not for:** final engineering decisions, real CAD models, electrical safety, or anything safety-critical. Treat the output as a helpful **first draft to review**, not a finished design. ## Try it The model **answers in JSON directly** — no reasoning preamble to strip. (It ships a chat template that defaults to no-think; pass `enable_thinking=True` only if you deliberately want a reasoning trace.) ```python from unsloth import FastVisionModel REPO = "caid-technologies/parti-vision" model, tok = FastVisionModel.from_pretrained(REPO, load_in_4bit=False) FastVisionModel.for_inference(model) SYSTEM_PROMPT = ( "You design maker/electronics products. Reply with one JSON object with exactly these " "10 keys: project, requirements, components, relationships, circuit, fabrication, " "instructions, appearance, image_generation_prompt, sourcing. Output only the JSON." ) messages = [ {"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]}, {"role": "user", "content": [{"type": "text", "text": "Design a USB desk lamp with touch dimming."}]}, ] text = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False) inputs = tok(text=text, return_tensors="pt").to("cuda") out = model.generate(**inputs, max_new_tokens=13000, do_sample=False, repetition_penalty=1.1) print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) ``` 💡 **Tip:** keep `do_sample=False` (greedy decoding), keep `max_new_tokens` high (≥ 13,000 — blueprints are long, and most parse failures are just plans cut off mid-sentence), and keep `repetition_penalty=1.1`. To include an image, add `{"type": "image", "image": your_image}` to the user content and pass `images=` to the tokenizer. 🚀 **Serving it for real?** Run it under **vLLM with structured output** (`guided_json` constrained to the ten-key blueprint schema). That's the intended production path — it takes valid-blueprint rates on unseen prompts from ~61% to **~97%** (see below). The bundled chat template carries over to vLLM automatically; if you override it, re-add `chat_template_kwargs={"enable_thinking": false}`. ## What it learned from It was trained on **150 synthetic maker/electronics products** — desk gadgets, lab tools, smart-home devices, audio gear and more, weighted toward hobbyist electronics. Every training record was machine-checked before inclusion: the circuit's pins had to exist on the named parts, the sourcing table had to add up, and the build steps had to come in a sensible order. Each product appears in **four flavors** — prompt only, prompt + brief, prompt + sketch, and prompt + brief + render — which is how the model learned to read documents and look at concept images, not just react to bare prompts. ## Good to know (limitations) - **English prompts, maker/electronics domain.** Off-topic requests still tend to get a blueprint rather than a refusal. - It **proposes** designs; it doesn't verify them. The plans are plausible engineering, not tested builds — review before you solder. - Roughly **half of outputs have at least one deep design slip** (a pin on the wrong net, costs that don't quite add up, a misordered step). Treat every blueprint as a draft and re-validate it in your app. - Very long plans can get **cut off at the token cap** — raise `max_new_tokens` and keep the repetition penalty on. - Contradictory or physically impossible requests can produce **confidently wrong** blueprints. ## How well it works We tested it on **60 held-out examples** (15 products it never saw in training × 4 input flavors) plus **42 fresh out-of-distribution prompts**, scoring every output against the real blueprint schema and semantic design checks. How often it produced a valid blueprint: | How you run it | In-distribution | Unseen realistic prompts | |---|---|---| | 📦 Plain (free) decoding | ~67% | ~61% | | ✅ With `guided_json` (recommended) | **~83%** | **~97%** | More context helps: the strongest input flavor (prompt + brief + render image) reached ~73% valid under plain decoding, with ~93% fidelity to what the prompt asked for. The deeper design checks (correct pins, costs that add up, ordered steps) pass on ~40–48% of outputs — which is why re-validating and regenerating on failure is part of the intended workflow. For comparison, the stock base model produced a valid blueprint **0%** of the time. ---
Technical details (for ML folks) - **Base model:** `Qwen/Qwen3.5-9B` (Apache-2.0); this repo is the **fine-tune merged to 16-bit** (standalone, no adapter needed). - **Method:** LoRA with Unsloth (r=16, α=16, bf16) over language attention+MLP layers, vision encoder frozen, then merged. - **Training:** 3 epochs, max_seq_len 16384, on 150 schema- and gate-validated synthetic records × 4 input variants. Records authored by Claude Sonnet 5 and DeepSeek-V4-Pro; concept images (render + hand-drawn sketch per record) by gpt-image-1. - **Chat template:** patched to default to `enable_thinking=False` so the model emits JSON directly under transformers, Unsloth, and vLLM alike. - **Evaluation** (vLLM; strict parse = raw `json.loads`, no salvage; schema-valid = full pydantic `Product` validation; gates = pin existence, sourcing arithmetic, phase order): In-distribution, 60 held-out examples: | Metric | Fine-tuned — free | Fine-tuned — `guided_json` | Base Qwen3.5-9B | |---|---|---|---| | Strict JSON parse | 0.85 | 0.83 | 0.97 | | Schema-valid (`Product`) | 0.67 | **0.83** | 0.00 | | Gates-clean | 0.40 | 0.48 | 0.00 | | Prompt-echo fidelity | 0.82 | 0.82 | 0.00 | Out-of-distribution, 42 fresh prompts (free decoding): | kind | n | parse | schema | |---|---|---|---| | realistic paraphrase | 10 | 0.90 | 0.70 | | new domain | 10 | 0.90 | 0.70 | | vague / underspecified | 5 | 0.80 | 0.40 | | image-grounded | 6 | 0.83 | 0.50 | | adversarial | 7 | 0.43 | 0.29 | | off-task | 4 | 1.00 | 0.75 | Across realistic OOD rows, schema-valid is 0.61 free / **0.97** under `guided_json`. Sampling robustness at `temperature=0.7` (30 generations, 3 seeds): strict parse 0.97, schema-valid 0.57. Parse failures are overwhelmingly truncation at the 13k-token cap (10/126 free generations; 0 thinking-leaks, 0 markdown fences, 1 trailing-data). - **Inference:** `do_sample=False`, `repetition_penalty≈1.1`, `max_new_tokens≥13000`; serve with vLLM `guided_json` for production. ```bibtex @misc{parti_vision_base, title = {Parti-Vision Base: Qwen3.5-9B for multimodal hardware blueprint generation}, author = {Caid Technologies}, year = {2026}, howpublished = {\url{https://huggingface.co/caid-technologies}} } ``` Built with [Unsloth](https://github.com/unslothai/unsloth) and 🤗 Transformers / PEFT / TRL.