ccore

/

Qwen3.5-4B-3D-Coder

Like 632

Also, I noticed the dataset has complexity levels 1–10, and the report describes them as annotations. How did you decide what level to assign each example? Was it based on things like the number of objects, spatial relationships, interaction, code length, or something else, and were the labels generated by the teacher model or assigned during curation?

Did you use those levels to control which examples the model saw at different stages, or to evaluate/select checkpoints, or were they only added as metadata? I’m trying to figure out how to build a useful complexity progression for an agentic 9B model, so I’d be really interested in any practical heuristics you found worked.

So far around 3000 max tokens is proving the most performant for me, longer and it has chance of looping thinking, if that is avoided with repeat penalty lookback might need to be longer than default 64 tokens to try to get more out of longer thinking budgets. If anyone else has tested it across your own prompts and found any other successful very specific settings like an exact highest performance temperature or what that would be great, this model kicks way above its weight and shows promise of narrow models being high value near term. Im interest in training a similar agentic harness-based system with qwen-3.5-9b. Any tips or tricks you found along the way training on https://huggingface.co/datasets/ccore/ThreeJS-3D-Scenes would be great. I assume most of the helpful insights are in the write up.

thanks for the detailed feedback! the thinking loops are definitely something worth investigating, especially when increasing the generation budget.

i'm actually training two more models that i'm planning to release this weekend, along with another dataset.

one is another 4b model focused on a more specific type of 3d generation, which i'll explain in more detail once it's released.

the other is similar to the current model, but much smaller, just 0.8b. interestingly, with some changes to the training approach, i'm getting close to the same eval loss as the 4b. i'm still evaluating whether that translates into comparable real world performance, but the initial results are pretty promising.

the images i'm sharing here are actually from the current 0.8b Q4_K_M checkpoint, so you can get an idea of what it's already capable of.

right now, my main focus is understanding the actual capabilities and limits of these small models, and how much we can improve them through training alone.

as for the agentic harness, based on other projects i've built in specialized domains, i don't think that part will be particularly difficult. the bigger challenge, in my opinion, is making sure the underlying model has actually learned the capabilities it needs.

one thing i'd definitely recommend for your 9b experiment is paying close attention to the quality and structure of the training data, not just the quantity. that's been one of the biggest lessons from my experiments so far.

i'm still waiting for the final results before drawing conclusions, but i'll share more details about the training approaches and what i've learned in the technical report. i'm especially curious to see how much of this can be reproduced with different model sizes and architectures.

Also, I noticed the dataset has complexity levels 1–10, and the report describes them as annotations. How did you decide what level to assign each example? Was it based on things like the number of objects, spatial relationships, interaction, code length, or something else, and were the labels generated by the teacher model or assigned during curation?

Did you use those levels to control which examples the model saw at different stages, or to evaluate/select checkpoints, or were they only added as metadata? I’m trying to figure out how to build a useful complexity progression for an agentic 9B model, so I’d be really interested in any practical heuristics you found worked.

The 1–10 levels are more of a relative complexity scale than a strict benchmark. They reflect the overall demands of a scene, including things like structural complexity, detail, and interactions.

The main idea was to make sure the dataset had a good variety of examples, from relatively simple scenes to much more challenging ones.

One interesting thing I've noticed is that complexity doesn't always translate directly into difficulty for the model. Sometimes a visually simple scene can actually be harder to generate correctly than a much more elaborate one.

I'm really curious to see how your 9B experiment turns out.

0.8b looks like its coming along great. Im filtering the ThreeJS-3D-Scenes dataset using qwen3.8-27B as a visual judge for quality, and some complexity features as you described. Was there some level of domain variety as opposed to compositional variety that you found to be more important? My guess is both matter differently one for instilling knowledge like components and one more so for managing many moving parts in one scene, then there would be domain variety even within a single scene, like objects associated with people vs machines in a warehouse scene or something where compositionally the spaces are themselves even modularized. Ive been testing some automated object itemized list rubric generation for llm-as-a-judge rl so that the model learns quality and compositionality through rl, where then quality comes from scale of samples generated during rl iterations, test time augmentations at rl rollouts, and the size of the model that is judging.
Also slightly related, there was a paper on compositionality and functionality in geometric subspaces inside of diffusion models that I remember from like 2 years ago that did hint at their orthogonality IIIRC. I find that managing space particularly relative empty spaces also seems like a generally week point slms have with 3d. Also have you tested around with any common namespace-like as in designing a voxel engine that can be reused generally so that the model adapts to use that specifically, like a secondary cdn extension of threejs.
Note compositionality, functionality, and variety all seem like independent axes for which the dataset can be aligned against. From your successful checkpoints, did you see any of these axes laying more or less of a hand in the performance gains between checkpoints?

By the way 2048 is near performance of 3072, and generally is faster, so it is pareto from my few tests Ive run, Im only running like 16 different prompts, and scores seem better in quality with more thinking tokens (rubrics grade slightly higher) but also more thinking yields higher failure rate it seems. Things like missing variables, it might have defined it in thinking but then just used it like it had in its response. Have you thought about submitting to 3D-Arena? Or did you do that already?

0.8b looks like its coming along great. Im filtering the ThreeJS-3D-Scenes dataset using qwen3.8-27B as a visual judge for quality, and some complexity features as you described. Was there some level of domain variety as opposed to compositional variety that you found to be more important? My guess is both matter differently one for instilling knowledge like components and one more so for managing many moving parts in one scene, then there would be domain variety even within a single scene, like objects associated with people vs machines in a warehouse scene or something where compositionally the spaces are themselves even modularized. Ive been testing some automated object itemized list rubric generation for llm-as-a-judge rl so that the model learns quality and compositionality through rl, where then quality comes from scale of samples generated during rl iterations, test time augmentations at rl rollouts, and the size of the model that is judging.
Also slightly related, there was a paper on compositionality and functionality in geometric subspaces inside of diffusion models that I remember from like 2 years ago that did hint at their orthogonality IIIRC. I find that managing space particularly relative empty spaces also seems like a generally week point slms have with 3d. Also have you tested around with any common namespace-like as in designing a voxel engine that can be reused generally so that the model adapts to use that specifically, like a secondary cdn extension of threejs.
Note compositionality, functionality, and variety all seem like independent axes for which the dataset can be aligned against. From your successful checkpoints, did you see any of these axes laying more or less of a hand in the performance gains between checkpoints?

that's really interesting! i think both domain variety and compositional variety matter, but in different ways. there's quite a bit behind how i structured the datasets, and i'd rather explain those decisions properly in a more detailed technical report than try to cover everything in a comment haha

as for reusable 3d structures, i'm actually releasing another model today called qwen glb coder. it's focused on generating reusable 3d assets that can be used in games and other applications, rather than just complete three.js scenes.

i think it might be pretty relevant to what you're exploring!

0.8b looks like its coming along great. Im filtering the ThreeJS-3D-Scenes dataset using qwen3.8-27B as a visual judge for quality, and some complexity features as you described. Was there some level of domain variety as opposed to compositional variety that you found to be more important? My guess is both matter differently one for instilling knowledge like components and one more so for managing many moving parts in one scene, then there would be domain variety even within a single scene, like objects associated with people vs machines in a warehouse scene or something where compositionally the spaces are themselves even modularized. Ive been testing some automated object itemized list rubric generation for llm-as-a-judge rl so that the model learns quality and compositionality through rl, where then quality comes from scale of samples generated during rl iterations, test time augmentations at rl rollouts, and the size of the model that is judging.
Also slightly related, there was a paper on compositionality and functionality in geometric subspaces inside of diffusion models that I remember from like 2 years ago that did hint at their orthogonality IIIRC. I find that managing space particularly relative empty spaces also seems like a generally week point slms have with 3d. Also have you tested around with any common namespace-like as in designing a voxel engine that can be reused generally so that the model adapts to use that specifically, like a secondary cdn extension of threejs.
Note compositionality, functionality, and variety all seem like independent axes for which the dataset can be aligned against. From your successful checkpoints, did you see any of these axes laying more or less of a hand in the performance gains between checkpoints?
that's actually a really interesting observation! i've noticed something similar, where more thinking doesn't necessarily mean better results. sometimes the model seems to lose track of things it already reasoned about when generating the final code.

i'm still exploring how far these small models can go before adding more complex inference workflows.

and thanks for the 3d arena suggestion! that could be a really interesting way to compare the model against others, especially with independent evaluations.

Those look amazing, I cant wait to see more from you.