Instructions to use Chaman1234/Fine-Tuned-Blender with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Chaman1234/Fine-Tuned-Blender with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct") model = PeftModel.from_pretrained(base_model, "Chaman1234/Fine-Tuned-Blender") - Notebooks
- Google Colab
- Kaggle
- Fine-Tuned-Blender (V11 β Least-Path Curvature Pruning + Long Training)
Fine-Tuned-Blender (V11 β Least-Path Curvature Pruning + Long Training)
A Qwen2.5-Coder-1.5B-Instruct LoRA adapter (r=4, lora_alpha=8 on q_proj and v_proj) paired with repeated blockwise Optimal Brain Surgeon (OBS) curvature-pruned FP16 query projections (pruned_q_proj.safetensors) for headless Blender 5.1 bpy scripting.
V11 warm-starts from the V10 adapter (benchmark/v10/) and runs a 3-stage, 228-step (3 epochs over 76 training examples) Least-Path pruning + masked LoRA recovery schedule combining:
- Least-Path repeated activation-Hessian blockwise column pruning & least-squares repair (Zenodo 24115016) across 4 evenly spaced attention query projections (
model.layers.{0,9,18,27}.self_attn.q_proj) to 50.0% exact column sparsity (768 / 1536surviving input columns per pruned projection). - Least-Change weight-displacement regularization (Zenodo 24074524) with
strength=50000.0, keeping final adapter squared displacement at5.120e-08while enforcing exact zero columns onlora_A[:, ~mask]after every step.
On the Local Blender 5.1 Workflow Suite (28 held-out tasks in headless Blender 5.1.0), V11 improves from V10's 21 / 28 (75.00%) to 25 / 28 (89.29%), while cutting 50% of input columns from the 4 targeted q_proj layers. Across all 40 combined held-out Blender 5.1 tasks (28 workflow + 12 legacy single-object tasks), V11 passes 33 / 40 (82.50%) vs V10's 30 / 40 (75.00%).
Storage & Architecture Caveat (pruned_q_proj.safetensors)
- Dense FP16 storage (no claimed inference speedup): The 4 pruned projections (
layers[0, 9, 18, 27].self_attn.q_proj) are stored as densetorch.float16[1536, 1536]matrices alongside[1536]FP16 biases and[1536]uint8column masks inpruned_q_proj.safetensors. Exact 50.0% structural column sparsity and0.0effective contribution from pruned columns (base + LoRA) are verified inbenchmark/v11/effective_sparsity_check.json, without claiming sparse-kernel throughput or latency gains. - Loading
pruned_q_proj.safetensorsvs. unpruned NF4 base: The 25 / 28 and 8 / 12 Blender execution scores were generated withpruned_q_proj.safetensorsapplied in FP16 and remaining layers quantized to 4-bit NF4. Evaluating the V11 LoRA adapter alone on the unpruned 4-bit NF4 base yields closely matching teacher-forced NLL (1.1985OOD validation /0.0155test vs1.2050/0.0161with pruned FP16q_proj; seebenchmark/v11/unpruned_base_check.json), though separate unpruned Blender generations were not run.
External Benchmark Coverage
External public Blender/3D benchmarks (BlenderGym, BlenderWorld, 3DCodeBench, 3DHarnessBench, 3DCodeEdit, and BlenderBench) were NOT run on this model. All reported pass rates come exclusively from the local headless Blender 5.1.0 fixture suites committed under benchmark/v11/.
| Suite | Status in this repo | Notes |
|---|---|---|
Local Blender 5.1 Workflow Suite |
Executed (28 tasks) | Scene-fixture multi-step & new API tasks (cases 1β28); reference scripts verified 28 / 28. |
Local Legacy Single-Object Suite (v3) |
Executed (12 tasks) | Original 8 test + 4 held-out single-object tasks; reference scripts verified 12 / 12. |
BlenderBench / BlenderGym / BlenderWorld / 3DCodeBench / 3DHarnessBench / 3DCodeEdit |
Not executed | No external benchmark scores are claimed for V11 or prior releases. |
Local Blender 5.1 Workflow Suite (V11 vs V10)
All completions were generated with greedy decoding (do_sample=False, max_new_tokens=220, batch_size=1, enable_thinking=False) on an NVIDIA GeForce GTX 1650 and graded identically in headless Blender 5.1.0 (--factory-startup) using benchmark/v11/check_workflows.py and benchmark/v11/check_scenes.py.
| Suite | V10 Baseline | V11 Least-Path (50% Pruned) | V10 Mean Answer Tokens | V11 Mean Answer Tokens |
|---|---|---|---|---|
| Local Blender 5.1 Workflow Suite (28 tasks) | 21 / 28 (75.00%) | 25 / 28 (89.29%) | 79.54 |
81.46 |
β³ Prior workflows (cases 1β20) |
20 / 20 (100.00%) | 19 / 20 (95.00%) | β | β |
β³ New V11 workflows (cases 21β28) |
1 / 8 (12.50%) | 6 / 8 (75.00%) | β | β |
Legacy Single-Object Suite (v3, 12 tasks) |
9 / 12 (75.00%) | 8 / 12 (66.67%) | 49.83 |
50.50 |
| Combined Held-Out Blender 5.1 Tasks (40) | 30 / 40 (75.00%) | 33 / 40 (82.50%) | β | β |
New Workflow Breakdown (cases 21β28) & Error Analysis
- Spot lights (
cases 21β22): V100 / 2(invalid enum"LIGHT_SPOT") β V112 / 2(StageSpotA,RimSpotB). - 3D Poly curves (
cases 23β24): V100 / 2('Curve' object has no attribute 'points'/AssertionError) β V112 / 2(CableA,CableB). - Multi-material polygon slots (
cases 27β28): V101 / 2(case 27 mistyped"CraneA"instead of"CrateA") β V112 / 2(CrateA,CrateB). - Vertex groups (
cases 25β26): V100 / 2, V110 / 2. Both models look up lowercase"sleeveA"/"sleeveB"instead of exact-case"SleeveA"/"SleeveB"(while V11 emits validvertex_groups.new/vg.addcalls). - Remaining regressions in V11: Workflow case
18(ShellBsubdivision + smooth shading) raisedbpy_prop_collection: attribute "load" not found, andv3task5(EvalTriangle) failed linking aMeshdatablock instead of anObject. Legacyv3tasks4,6, and12failed in both V10 and V11 (v3_regression_scores.json).
Training, Pruning & Honest NLL Report
Training & Repeated Pruning Setup
- Base model & starting weights:
Qwen/Qwen2.5-Coder-1.5B-Instructin 4-bit NF4 (bnb_4bit_compute_dtype=torch.float16, double quantization), warm-started fromblender-assistant-v10-workflows/least_change/adapter_model.safetensors(seed=42). - Training set (
76disjoint examples,max_length=320):12single-object replays +24prior workflow replays +40new Spot light, 3D Poly curve, Vertex group, and Multi-Material slot examples. All 76 scripts passedast.parseand were constructed disjoint from the28test tasks and2untouched OOD validation tasks (ico_spheremesh primitive andMIRRORmodifier). - Schedule (
228steps total,3epochs):- Cut 1 (
0% β 25%): Dequantizelayers[0, 9, 18, 27].self_attn.q_projto FP16, collect input second-moment Hessians over all 76 sequences (13,666calibration tokens), prune4lowest-saliency columns perblock_size=16(1,152 / 1,536surviving), apply exact local least-squares repair ($\Delta_S = W_D H_{DS} H_{SS}^{-1}$), zero prunedlora_A[:, ~mask], then run Steps 1β76 masked LoRA recovery (lr=2e-4,strength=50000.0, logit chunk4). - Cut 2 (
25% β 50%): Re-calibrate activations on the Cut 1 recovered model (13,666tokens), prune another4surviving columns per block (768 / 1,536surviving), repair surviving columns, zerolora_A[:, ~mask], then run Steps 77β152 masked recovery. - Long Continuation (
Steps 153β228): Continue column-masked Least-Change LoRA training for a full 3rd epoch to Step 228.
- Cut 1 (
Stage-by-Stage Teacher-Forced NLL & Displacement Trajectory
NLL Caveat (Prompt-Template Overlap): Very low
test_28NLL (0.0161across2,248tokens on28cases) occurs because those 28 workflow prompts reuse training-seen structural API families and prompt phrasing with held-out names/parameters, whereas the2untouched out-of-distribution (OOD) validation templates (ico_sphere,MIRROR,50tokens total) stay at1.2050NLL. Pair both numbers rather than interpretingtest_28NLL as generalbpyperplexity.
| Stage | Surviving Cols (q_proj) |
OOD Val NLL (2 ex / 50 tok) | Workflow Test NLL (28 ex / 2,248 tok) | Legacy v3 NLL (12 ex / 546 tok) |
Squared Weight Change (drift) |
|---|---|---|---|---|---|
| V10 starting baseline | 1536 / 1536 (0% cut) |
1.1180 |
0.1759 |
0.4868 |
β |
| Cut 1 unrecovered | 1152 / 1536 (25% cut) |
1.2022 |
0.1754 |
β | β |
| Cut 1 recovered (step 76) | 1152 / 1536 (25% cut) |
1.0766 |
0.0198 |
0.4579 |
5.511e-08 |
| Cut 2 unrecovered | 768 / 1536 (50% cut) |
1.0038 |
0.0186 |
β | β |
| Cut 2 recovered (step 152) | 768 / 1536 (50% cut) |
1.1485 |
0.0324 |
0.4405 |
5.464e-08 |
Final V11 (step 228, pruned FP16 q_proj) |
768 / 1536 (50% cut) |
1.2050 |
0.0161 |
0.5037 |
5.120e-08 |
Unpruned NF4 base + V11 LoRA (no pruned_q_proj) |
1536 / 1536 (unpruned) |
1.1985 |
0.0155 |
β | 5.120e-08 |
Verified Sparsity Per Pruned Layer (effective_sparsity_check.json)
| Module | Surviving Columns | Base Zero Fraction | Effective Dead-Column Max Abs (W_eff[:, ~mask]) |
|---|---|---|---|
model.layers.0.self_attn.q_proj |
768 / 1536 |
0.500014 |
0.0 |
model.layers.9.self_attn.q_proj |
768 / 1536 |
0.500000 |
0.0 |
model.layers.18.self_attn.q_proj |
768 / 1536 |
0.500001 |
0.0 |
model.layers.27.self_attn.q_proj |
768 / 1536 |
0.500003 |
0.0 |
Reproducibility & Decoding Logs
Every published completion file has a matching generation script and per-case decoding log with exact rendered_chat_prompt, generated_token_ids, stopped_on_eos, timings, environment versions (torch 2.1.2+cu118, transformers 4.46.3, peft 0.13.2, bitsandbytes 0.45.5), and SHA256 hashes:
- Exact generation scripts:
benchmark/v11/generate_eval.py(standalone Hugging Face / local CLI runner) andbenchmark/v11/eval_v11_models.py(exact provenance runner recorded in the logs). - Artifact SHA256 hashes:
- V11
adapter_model.safetensors:ef0641bd2fae8d4166da24c7ba20e9353af277a3128fc2b13a8a7892c4e9d720 - V11
pruned_q_proj.safetensors:32bbc65f8e36c4718dac863cf90bae65f9e4b241dab1015d6119f21e6a540a6d - V10 adapter (baseline in logs):
ea849a0e32ae6807787f0bd2359226e2925f135d096e6df12c47f5dc9e85c0af
- V11
- Logged artifacts in
benchmark/v11/:- Workflow suite (28 cases):
v11-28_completions.jsonl,v11-28-decoding-log.json,v10-28_completions.jsonl,v10-28-decoding-log.json,workflow_scores.json - Legacy
v3suite (12 cases):v11-v3_completions.jsonl,v11-v3-decoding-log.json,v10-v3_completions.jsonl,v10-v3-decoding-log.json,v3_regression_scores.json
- Workflow suite (28 cases):
Exact Prompt Template
Write a Blender 5.1 Python script using bpy. {task}
Keep it concise. Output code only, with no comments or Markdown.
Return only the Python solution.
Rendered via tokenizer.apply_chat_template([{"role": "user", "content": content}], tokenize=True, add_generation_prompt=True, enable_thinking=False).
Reproduce Completions & Blender 5.1 Grading
# 1. Regenerate greedy completions with pruned_q_proj.safetensors applied
python benchmark/v11/generate_eval.py \
--base-model Qwen/Qwen2.5-Coder-1.5B-Instruct \
--adapter . \
--cases benchmark/v11/cases.jsonl \
--output v11-repro.jsonl
# 2. Grade workflow suite (28 tasks) in headless Blender 5.1
blender -b --factory-startup --python benchmark/v11/check_workflows.py -- \
--cases benchmark/v11/cases.jsonl benchmark/v11/v11-28_completions.jsonl
# 3. Grade legacy single-object suite (12 tasks) in headless Blender 5.1
blender -b --factory-startup --python benchmark/v11/check_scenes.py -- \
--extended benchmark/v11/v11-v3_completions.jsonl
Quick Start
To reproduce V11's exact evaluated behavior, load pruned_q_proj.safetensors onto layers[0, 9, 18, 27].self_attn.q_proj before attaching the PEFT LoRA adapter:
import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from safetensors.torch import load_file
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
base_id = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
repo_id = "Chaman1234/Fine-Tuned-Blender"
tokenizer = AutoTokenizer.from_pretrained(base_id)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.float16,
)
base = AutoModelForCausalLM.from_pretrained(
base_id,
quantization_config=bnb_config,
torch_dtype=torch.float16,
device_map={"": 0},
)
pruned_path = hf_hub_download(repo_id, "pruned_q_proj.safetensors")
tensors = load_file(pruned_path, device="cpu")
for idx in (0, 9, 18, 27):
attn = base.model.layers[idx].self_attn
old = attn.q_proj
weight = tensors[f"layer_{idx}.weight_fp16"]
bias = tensors[f"layer_{idx}.bias_fp16"]
linear = torch.nn.Linear(weight.shape[1], weight.shape[0], bias=True, dtype=torch.float16)
with torch.no_grad():
linear.weight.copy_(weight)
linear.bias.copy_(bias)
linear.requires_grad_(False)
attn.q_proj = linear.to(old.weight.device)
model = PeftModel.from_pretrained(base, repo_id).eval()
task = "Create a spot light named StageSpotA at (3, -4, 5) with 750 watts, color (1.0, 0.9, 0.7), and spot angle 45 degrees."
prompt = (
f"Write a Blender 5.1 Python script using bpy. {task}\n"
"Keep it concise. Output code only, with no comments or Markdown.\n"
"Return only the Python solution."
)
ids = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
)
input_ids = torch.tensor([ids], device=next(model.parameters()).device)
attention_mask = torch.ones_like(input_ids)
with torch.inference_mode():
out = model.generate(
input_ids=input_ids,
attention_mask=attention_mask,
pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
max_new_tokens=220,
do_sample=False,
temperature=None,
top_p=None,
top_k=None,
)
print(tokenizer.decode(out[0][len(ids):], skip_special_tokens=True))
Related Zenodo records
- Weight-displacement least-change objective: https://doi.org/10.5281/zenodo.24074524
- Least-Path curvature-guided pruning and recovery (
hessian_repeated): https://doi.org/10.5281/zenodo.24115016
- Downloads last month
- 97
Model tree for Chaman1234/Fine-Tuned-Blender
Base model
Qwen/Qwen2.5-1.5BEvaluation results
- Headless Blender 5.1 Execution Pass Rate (25/28) on Local Blender 5.1 Workflow Suite (28 Tasks)self-reported89.290
- Headless Blender 5.1 Execution Pass Rate (8/12) on Local Legacy Single-Object Suite (v3, 12 Tasks)self-reported66.670