Generic Triple-Latent Compression with Gated Associative Retrieval
Abstract
Generic triple-latent sequence models with token state and compressed pair-memory pathways enhance Transformer performance on text prediction tasks while a gated key-value retrieval extension improves recall at the cost of speed and stability.
We study generic triple-latent sequence models that maintain a running token state and compressed pair-memory pathway to capture higher-order token interactions without benchmark-specific parsing. The triple-latent family improves a small Transformer baseline on byte-level WikiText-2 and on a tokenizer-based MiniMind language-model benchmark, while a recall-focused gated key-value retrieval extension improves associative recall but remains seed-sensitive and much slower in the current reference implementation.
Get this paper in your agent:
hf papers read 2606.05175 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper