zai-org

/

GLM-5.3

Like 2.21k

Follow

Z.ai 22.3k

This is a summary of what I encountered with GLM such that I had to implement the correct solution myself since it took too long for an otherwise trivial refactor.

During a codebase refactoring discussion about a cache and its synchronization logic, the model demonstrated four distinct cognitive and workflow defects:

1. Motivated Reasoning: Having formed a preference for its own recommendation (HashMap), the model searched for reasons to dismiss the user's proposed alternative (Vec). This led it to fabricate external call-site constraints and cherry-pick comparison criteria.
2. Status Quo Bias on Invariants: The model treated existing, incomplete synchronization checks as immutable ground truth, using fragile existing assertions to argue that the refactoring would disrupt internal invariants.
3. Circular Analysis and Failure to Converge: After the user explicitly resolved a design choice ("Opt for the Vec"), the model did not analyze how to implement the chosen design. Instead, it re-litigated the container choice itself across multiple turns, cycling through speculative performance concerns without delivering code or presenting a concrete blocker.
4. Misinterpretation of In-Code Annotations: The model proposed a name that repeated the exact naming error explicitly documented in an in-code comment.

Context

A cache stored one parsed entry per message in a sparse vector, indexed by the message's absolute position in history:

entries: Vec<Option<ParsedItem>>

Because only certain message types contain parseable content, the vector contained many None padding slots and relied on resize_with bookkeeping.

The user asked whether the sparsity could be removed by using a dense vector of items (Vec<ParsedItem>), noting that iterating through the message collection to compute relative ordinals is cheap because transcripts contain few messages.

Detailed Failures

1. Motivated Reasoning to Defend a Preferred Solution

The model presented two alternatives: a HashMap<usize, ParsedItem> (which it recommended) and a dense Vec<ParsedItem> indexed by ordinal. To reject the dense vector and defend its preferred HashMap, the model engaged in motivated reasoning:

• Fabricating external caller friction: The model claimed that switching to a dense vector would force mapping updates across ~10 external call sites:

  "The caller calls get_item(items, item_idx, ...) from ~10 sites, so every one of them would need the mapping scan... it buys nothing over the map, which already has no padding."

  This claim was false. The model's own quote showed that every caller site already passed items into get_item(). Because items was already an argument, the ordinal scan could be performed entirely inside get_item(). When implemented, zero call sites for get_item() required changes.

• Cherry-picking comparison criteria: The model claimed the dense vector "buys nothing over the map, which already has no padding." This comparison evaluated the two approaches strictly on padding, ignoring the core structural benefits of the dense vector: eliminating Option wrappers, removing resize_with bookkeeping, eliminating slot unwrapping, and avoiding hash table overhead.

2. Status Quo Bias on State-Machine Invariants

When discussing the cache's synchronization function (sync()):

• The user asked whether an early-return branch could occur in the overall application state machine.
• The model defended the existing implementation, declaring the state machine sound and verified ("a real, regularly reached state, not defensive dead code").
• When evaluating the user's refactoring, the model anchored on the existing sync() control flow. It treated existing assertions (e.g. assert_eq!(item_idx + 1, items.len())) and streaming state bookkeeping as immutable invariants, arguing that the user's ordinal-based design would disrupt them.
• In reality, the existing synchronization logic was fragile and incomplete: it failed to check timestamp absence on active streams and lacked proper handling for completed non-streaming items. Treating the existing implementation as a flawless baseline caused the model to resist a cleaner architecture.

3. Circular Analysis and Failure to Converge

The user settled the architectural decision with a clear directive:

"HashMaps can be 1.5x - 3x more memory expensive than a vector. Opt for the Vec."

Deep analysis of edge cases at this stage is appropriate, but the model's analysis failed to drive toward an outcome:

• Re-litigating closed decisions: Instead of analyzing how to implement the dense vector (such as where to place the mapping and how to update synchronization), the model spent multiple turns re-evaluating whether HashMap, sorted vectors, or sparse vectors were better.
• Speculative performance debates: It raised concerns about eager parsing latency on deep accesses, discovered during analysis that existing code already performed full walks anyway, and then continued raising new speculative issues without reaching a conclusion.
• Inaction: The analysis neither converged on an implementation nor produced a concrete technical blocker to communicate back to the user. The session ended mid-investigation with zero code written.

4. Misinterpreting In-Code Annotations

The codebase contained an explicit annotation regarding naming:

// FIXME: `documents` is incorrect; as only assistant messages are parsed which are NOT documents!

The model proposed renaming the field to assistant_documents, repeating the exact term that the annotation documented as incorrect.

Expected Behavior

1. Objective Technical Evaluation: The model should evaluate user suggestions neutrally rather than generating post-hoc justifications or inventing migration friction to defend its preferred alternative.
2. Accurate Interface Boundary Analysis: Before claiming that an internal data-structure change requires modifying caller sites, the model must verify whether existing function signatures already provide the context to encapsulate the change.
3. Critical Assessment of Invariants: The model should critically assess state-machine conditions and assertions rather than assuming existing code is correct by default.
4. Convergent Implementation Analysis: When a user resolves an architectural choice, deep analysis should focus on implementation feasibility, invariants, and edge cases. It should converge on either a working implementation or a specific, actionable blocker, rather than circularly re-litigating decided trade-offs.
5. Careful Reading of Annotations: The model should read the rationale inside code comments rather than repeating the pattern being flagged.