README.md

---
license: apache-2.0
language:
  - en
tags:
  - caid
  - blueprint
  - hardware
  - electronics
  - maker
  - structured-generation
  - json
  - qwen3.5
  - vision
  - unsloth
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3.5-9B
base_model_relation: finetune
library_name: transformers
---

# Parti-Vision Base — Qwen3.5-9B

**Parti-Vision turns a hardware idea — a sentence, a short brief, even a sketch — into a complete build blueprint.**

Tell it what you want to build — *"a USB-powered desk lamp with touch dimming"* — and optionally
hand it a short description document and/or a concept image (a hand-drawn sketch or a product
render). It gives back one structured blueprint: the parts list, a pin-level wiring map,
step-by-step build instructions, costs, and a description of how the finished product should
look. Everything comes out as one clean JSON object that an app can read and build on.

This is the **all-in-one model** — it runs on its own, no add-ons needed.

> **Early research preview.** Great for drafting and exploring ideas — not a replacement for
> real engineering, CAD software, or safety review.

By [caid-technologies](https://huggingface.co/caid-technologies).

---

## What it can do

Give it a hardware idea (and optionally a brief and/or a picture) and it produces **one complete
project blueprint** containing:

- 📋 a **parts list** (components) with roles and key specs
- 🔌 a **pin-level wiring map** — which pin connects to which, net by net
- 🛠️ ordered **build instructions**, from fabrication through assembly and testing
- 💲 a **sourcing table** with per-part costs that add up to a stated total
- 🎨 an **appearance spec** plus a ready-to-use **image-generation prompt** for concept renders
- 🖼️ and it can **read images**: show it a napkin sketch or a product render and it designs to match

Unlike a chat model, it answers with the whole plan at once — a single JSON object with ten
sections (`project, requirements, components, relationships, circuit, fabrication, instructions,
appearance, image_generation_prompt, sourcing`).

## What you can give it

- ✍️ **A plain-English request** (always) — one or two sentences describing what you want to build.
- 📄 **A short description document** (optional) — plain text or Markdown (a design brief, notes,
  a requirements list). The model can't open files itself: paste the document's text into the
  user message. It was trained with the document wrapped in markers, so use the same shape:

  ```text
  <your request>

  --- ATTACHED DOCUMENT: brief.md ---
  <the document text>
  --- END DOCUMENT ---
  ```

- 🖼️ **One concept image** (optional) — a **hand-drawn sketch** or a **product render**, passed
  as a regular vision input. Training images were PNGs; any format your image loader opens
  (PNG, JPEG, WebP, …) works, since it's decoded to pixels before the model sees it. One image
  per request is what it was trained on.

Any combination works — prompt alone, prompt + document, prompt + image, or all three (the
strongest setup in our testing). PDFs, CAD files, and spreadsheets aren't supported directly —
convert them to text or an image first.

## What it's good for — and not

✅ **Good for:** brainstorming maker/electronics products, turning a rough idea (or a sketch)
into an organized starting plan, and powering apps that need machine-readable project plans.

🚫 **Not for:** final engineering decisions, real CAD models, electrical safety, or anything
safety-critical. Treat the output as a helpful **first draft to review**, not a finished design.

## Try it

The model **answers in JSON directly** — no reasoning preamble to strip. (It ships a chat
template that defaults to no-think; pass `enable_thinking=True` only if you deliberately want
a reasoning trace.)

```python
from unsloth import FastVisionModel

REPO = "caid-technologies/parti-vision"
model, tok = FastVisionModel.from_pretrained(REPO, load_in_4bit=False)
FastVisionModel.for_inference(model)

SYSTEM_PROMPT = (
    "You design maker/electronics products. Reply with one JSON object with exactly these "
    "10 keys: project, requirements, components, relationships, circuit, fabrication, "
    "instructions, appearance, image_generation_prompt, sourcing. Output only the JSON."
)
messages = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},
    {"role": "user",   "content": [{"type": "text", "text": "Design a USB desk lamp with touch dimming."}]},
]
text = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tok(text=text, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=13000, do_sample=False, repetition_penalty=1.1)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

💡 **Tip:** keep `do_sample=False` (greedy decoding), keep `max_new_tokens` high (≥ 13,000 —
blueprints are long, and most parse failures are just plans cut off mid-sentence), and keep
`repetition_penalty=1.1`. To include an image, add
`{"type": "image", "image": your_image}` to the user content and pass `images=` to the tokenizer.

🚀 **Serving it for real?** Run it under **vLLM with structured output** (`guided_json`
constrained to the ten-key blueprint schema). That's the intended production path — it takes
valid-blueprint rates on unseen prompts from ~61% to **~97%** (see below). The bundled chat
template carries over to vLLM automatically; if you override it, re-add
`chat_template_kwargs={"enable_thinking": false}`.

## What it learned from

It was trained on **150 synthetic maker/electronics products** — desk gadgets, lab tools,
smart-home devices, audio gear and more, weighted toward hobbyist electronics. Every training
record was machine-checked before inclusion: the circuit's pins had to exist on the named parts,
the sourcing table had to add up, and the build steps had to come in a sensible order.

Each product appears in **four flavors** — prompt only, prompt + brief, prompt + sketch, and
prompt + brief + render — which is how the model learned to read documents and look at concept
images, not just react to bare prompts.

## Good to know (limitations)

- **English prompts, maker/electronics domain.** Off-topic requests still tend to get a
  blueprint rather than a refusal.
- It **proposes** designs; it doesn't verify them. The plans are plausible engineering, not
  tested builds — review before you solder.
- Roughly **half of outputs have at least one deep design slip** (a pin on the wrong net,
  costs that don't quite add up, a misordered step). Treat every blueprint as a draft and
  re-validate it in your app.
- Very long plans can get **cut off at the token cap** — raise `max_new_tokens` and keep the
  repetition penalty on.
- Contradictory or physically impossible requests can produce **confidently wrong** blueprints.

## How well it works

We tested it on **60 held-out examples** (15 products it never saw in training × 4 input
flavors) plus **42 fresh out-of-distribution prompts**, scoring every output against the real
blueprint schema and semantic design checks. How often it produced a valid blueprint:

| How you run it | In-distribution | Unseen realistic prompts |
|---|---|---|
| 📦 Plain (free) decoding | ~67% | ~61% |
| ✅ With `guided_json` (recommended) | **~83%** | **~97%** |

More context helps: the strongest input flavor (prompt + brief + render image) reached ~73%
valid under plain decoding, with ~93% fidelity to what the prompt asked for. The deeper design
checks (correct pins, costs that add up, ordered steps) pass on ~40–48% of outputs — which is
why re-validating and regenerating on failure is part of the intended workflow. For comparison,
the stock base model produced a valid blueprint **0%** of the time.

---

<details>
<summary><b>Technical details</b> (for ML folks)</summary>

- **Base model:** `Qwen/Qwen3.5-9B` (Apache-2.0); this repo is the **fine-tune merged to
  16-bit** (standalone, no adapter needed).
- **Method:** LoRA with Unsloth (r=16, α=16, bf16) over language attention+MLP layers, vision
  encoder frozen, then merged.
- **Training:** 3 epochs, max_seq_len 16384, on 150 schema- and gate-validated synthetic
  records × 4 input variants. Records authored by Claude Sonnet 5 and DeepSeek-V4-Pro; concept
  images (render + hand-drawn sketch per record) by gpt-image-1.
- **Chat template:** patched to default to `enable_thinking=False` so the model emits JSON
  directly under transformers, Unsloth, and vLLM alike.
- **Evaluation** (vLLM; strict parse = raw `json.loads`, no salvage; schema-valid = full
  pydantic `Product` validation; gates = pin existence, sourcing arithmetic, phase order):

  In-distribution, 60 held-out examples:

  | Metric | Fine-tuned — free | Fine-tuned — `guided_json` | Base Qwen3.5-9B |
  |---|---|---|---|
  | Strict JSON parse | 0.85 | 0.83 | 0.97 |
  | Schema-valid (`Product`) | 0.67 | **0.83** | 0.00 |
  | Gates-clean | 0.40 | 0.48 | 0.00 |
  | Prompt-echo fidelity | 0.82 | 0.82 | 0.00 |

  Out-of-distribution, 42 fresh prompts (free decoding):

  | kind | n | parse | schema |
  |---|---|---|---|
  | realistic paraphrase | 10 | 0.90 | 0.70 |
  | new domain | 10 | 0.90 | 0.70 |
  | vague / underspecified | 5 | 0.80 | 0.40 |
  | image-grounded | 6 | 0.83 | 0.50 |
  | adversarial | 7 | 0.43 | 0.29 |
  | off-task | 4 | 1.00 | 0.75 |

  Across realistic OOD rows, schema-valid is 0.61 free / **0.97** under `guided_json`.
  Sampling robustness at `temperature=0.7` (30 generations, 3 seeds): strict parse 0.97,
  schema-valid 0.57. Parse failures are overwhelmingly truncation at the 13k-token cap
  (10/126 free generations; 0 thinking-leaks, 0 markdown fences, 1 trailing-data).
- **Inference:** `do_sample=False`, `repetition_penalty≈1.1`, `max_new_tokens≥13000`; serve
  with vLLM `guided_json` for production.

```bibtex
@misc{parti_vision_base,
  title  = {Parti-Vision Base: Qwen3.5-9B for multimodal hardware blueprint generation},
  author = {Caid Technologies},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/caid-technologies}}
}
```

Built with [Unsloth](https://github.com/unslothai/unsloth) and 🤗 Transformers / PEFT / TRL.

</details>