{
  "orid": "82EJxJzG6r",
  "arxiv_id": "2603.08859v1",
  "title": "Expressivity-Efficiency Tradeoffs for Hybrid Sequence Models",
  "claims": [
    {
      "index": 1,
      "text": "Theorem 3.3 proves that any k-layer state-space model solving the function-composition tasks under injectivity conditions must have total log state-space size scaling as Ω(m·log|V| − q·log|Y|), linear in the hidden dimension m (Theorem 3.3)."
    },
    {
      "index": 2,
      "text": "Theorem 3.7 proves that sliding-window Transformers solving the same tasks under a local-sensitivity condition require total window size scaling with the context-dependency range R (Theorem 3.7)."
    },
    {
      "index": 3,
      "text": "Theorem 4.3 constructs a two-layer hybrid (Mamba + attention) model that solves the selective copying task using embedding dimension O(max(log|V|, log L)) and working memory Õ(N), versus Ω(L) required by pure Transformers (Theorem 4.3)."
    },
    {
      "index": 4,
      "text": "Theorem 4.6 constructs a three-layer hybrid model that achieves 99% accuracy on the associative recall task using embedding dimension O(max(log|V|, log L)) and window size Õ(|V|) (Theorem 4.6)."
    },
    {
      "index": 5,
      "text": "On the selective copying task, the learned hybrid model reaches perfect accuracy with roughly 2,000 parameters while pure Transformer/SSM models need roughly 12,000 parameters to match it, a 6x parameter gap (Figure 4)."
    },
    {
      "index": 6,
      "text": "On multi-key associative recall, the hybrid model reaches 60% accuracy using 6x fewer parameters than pure Transformers, which plateau near 40% accuracy on single-key associative recall (Figures 5-6)."
    }
  ]
}