My experience using and evaluating this model in coding tasks (the results are simply amazing).
Here’s what I noticed when comparing other “fine-tuned” QWEN 3.5 models: 27B and 35B-A3B. Finally, this model doesn’t break at 32K of context; my working context size is 1M, and it works correctly at that size—there are no issues with YARN scaling! The others break immediately (they lose context, tool calling fails) as soon as the context size exceeds 32–64K. I need to generate documentation and tests for a huge project—no problem! Your work is simply amazing! I’d love to see your version of the 122B-A10B. People have also mentioned in other discussions that "This model has been trained on broken Qwen3.5-35B-A3B official weight release that Alibaba Group shipped to people. It's not issue with training or learning data. It's issue with model itself. Same problem exists in Qwen3.6-35B-A3B base weights. It's main reason why this model gets stuck in a recursive loop even after full finetune in BF16 safetensors - expert tensors in block 0 has 40% of noisy zero blocks that are visible only on Q8_0 quantization. Three ssm_con1vd tensors have enormous big signal scale and accumulate cascade error distributing during training on entire neural network.". How did you manage to get around that? In any case, your work is simply wonderful—thank you so much!
During RL training, we observed that within a certain range, the increase in sequence length is accompanied by rising reward values. Therefore, accuracy does not degrade at larger lengths, indicating that the model is capable of handling tasks within this length range. The open-sourced checkpoint in this work supports a slightly longer sequence length compared to the original checkpoint while maintaining favorable metrics, which may account for this phenomenon.