Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
DedeProGamesΒ 
posted an update 1 day ago
Post
1,891
Hi everyone! I want to share a few updates about GRM development.

*GRM-3.4 has been canceled*

Although GRM-3.4 showed strong reasoning and coding capabilities, testing revealed several issues. It tended to overthink, and its code often contained errors that GRM-3.2-Sky handled correctly. Its tool use also needed improvement, and its terminal performance did not reach the level we were aiming for.

We repeatedly saw it rewrite files from scratch instead of using editing tools, and use shell commands to create or modify files when dedicated tools were available.

*Looking ahead to GRM-3.6*

GRM-3.6 is a possible next generation for the GRM models. We are still deciding whether it will be a single large model or a complete model family. The flagship is set to be **GRM-3.6-Moon**, a **35B-A3B** model.

*Experimental models*

Orion LLM Labs may release experimental models more regularly from now on. These experiments will help us research and test ideas for future GRM models.

Thanks for following the project!

Interesting to see the tool-use failure mode (rewriting files from scratch instead of using edit tools) as a primary cancellation reason. In my own small-model experiments, I've found that models often default to "generate the whole thing" because it's the most statistically safe continuation β€” partial edits require the model to attend to and preserve existing context, which is harder than regenerating from scratch.

For GRM-3.6, if you're exploring tool-use reliability, one cheap diagnostic: measure the ratio of "full-file rewrite" vs "targeted edit" actions across a fixed set of code-editing prompts, and track how that ratio changes with training. It can reveal whether the model learned the tool schema but not the preference for minimal edits.

Good luck with the 35B-A3B flagship! πŸ€—

Rewriting a file from scratch seems worth tracking separately from whether the tests pass: it can also remove comments or behavior outside the requested change. That makes the contrast with 3.2-Sky useful for understanding the cancellation. In the comparisons you inspected, did 3.4 still favor full rewrites when both models had the same editing tools and system prompt?