When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference
Abstract
Quantized recurrent inference suffers from state write-back rules that suppress small updates, but error feedback and residual memory restore accuracy without retraining across GRU and LSTM architectures.
Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized state is stored and returned at the next time step, so the rule used to store that state can alter subsequent computations. Here, we introduce recurrent-state write-back to denote this rule and isolate its effect in a compact GRU encoder--decoder for fluorescence lifetime imaging, a molecular imaging modality used in quantitative biological imaging. A central task is estimating two lifetime parameters, the short-lived component τ1 and the long-lived component τ2, from high-noise time-resolved fluorescence signals. Holding the trained model fixed, replacing continuous state propagation with deterministic 4-bit state storage increases estimation errors for τ1 and τ2 by approximately 70x and 300x, respectively. Failure occurs when repeated small updates remain below the write threshold, leaving the stored state nearly fixed while the network continues to propose change. Error feedback, residual memory, and direction memory carry information from these suppressed updates across time and recover accuracy without retraining. Precision sweeps show that increasing state precision can worsen a fixed recurrent solution, while matched training shows that compatibility with the state interface can be learned. To test whether this behavior extends beyond the GRU, we repeat the post-training intervention in an independently trained LSTM, where coarse write-back reproduces the failure, error feedback restores accuracy, and state-specific interventions reveal greater sensitivity of the cell state than the hidden state. Our results establish recurrent-state write-back as a key determinant of low-precision recurrent dynamics and identify the state-storage interface as a central design consideration for quantized recurrent inference.
Community
We show that quantization doesn't just make individual numbers less precise in recurrent networks; it can quietly break the network's memory itself.
In GRUs and LSTMs, the internal state gets saved after every time step and fed back in for the next one. We found that when this state is stored with coarse 4-bit rounding, the network can keep proposing small updates in the same direction for dozens of steps in a row, but every single one of them gets rounded away. The state ends up frozen even though the network is still "trying" to move it, so it keeps computing on stale memory.
We call this the recurrent-state write-back problem. On a time- resolved fluorescence lifetime imaging task, changing only this storage rule (with the trained model completely unchanged otherwise) increased estimation errors by roughly 70x and 300x for two key parameters. Interestingly, just giving the state more bits doesn't reliably fix this; a model trained around a 4-bit interface can actually get worse when you store its state more precisely, because it adapted to the coarse interface during training. What matters is compatibility between the learned dynamics and the storage rule, not bit-width alone.
We also test some lightweight fixes: carrying the leftover rounding error forward, keeping a small residual, or just tracking the direction of repeated sub-threshold updates. All three recover most of the lost accuracy with zero retraining. We reproduce the same failure and rescue in an independently trained LSTM too, where the cell state turns out to be far more sensitive to this than the hidden state.
Happy to answer questions about the method or the write-back interventions.
Get this paper in your agent:
hf papers read 2609.04490 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper