Dont use this model

#1
by danbrima - opened

We deployed RedHatAI/GLM-5.3-NVFP4 on vLLM (B300, 262K context) and found it unusable for production. On basic tasks the model often never reaches an answer: it keeps reasoning until the entire token budget is used up and returns no final output. The same tasks are routine for the base model, so this doesn’t look like normal GLM-5.3 behavior.

The model card only evaluates short-context math/QA benchmarks, and even there IFEval drops to 95.5% recovery. There is no evaluation of agentic, tool-calling, or long-context use, which is where we saw the failures.

Has anyone else seen runaway reasoning with this checkpoint? Would be good to see an evaluation of reasoning termination and agentic tasks against zai-org/GLM-5.3 FP8.

Red Hat AI org

Hi @danbrima ,

Sorry to hear that this model is underperforming. What kind of basic tasks have you found that this model suffers on? We have found that the base model also has some wierd behavior sometimes, but regardless we are going to test this model in long-context and agentic evals to investigate.

Even on basic tasks, like making an edit in a moderately sized codebase, the model got stuck reasoning until it hit context compaction, several times in a row, without making a single tool call.

@danbrima share your vLLM command and vLLM version.

Red Hat AI org

Yes @danbrima if you could share vllm run command + version it would be very helpful!

Sign up or log in to comment