Goal-Conditioned Reachability Logit Masker: Guaranteed Goal Satisfaction for Constrained LLM Generation in $\mathcal{O}(1)$ Time
Author: ByeongUk An (ORCID: 0009-0007-5612-5602, hhjjkk7186@gmail.com)
Preprint: arXiv
Abstract
Constrained decoding frameworks (e.g., Outlines, SGLang, SynCode) have emerged as indispensable tools for forcing Large Language Models (LLMs) to adhere to strict syntactic schemas, such as JSON, SQL, or domain-specific grammars. However, existing methods rely almost exclusively on forward-looking Deterministic Finite Automata (DFA) transitions or infinite-horizon grammar reachability. Under realistic serving scenarios with strict token budgets ($T_{\max}$), these forward-only mechanisms suffer from a catastrophic structural vulnerability: they eagerly explore valid syntactic subtrees that cannot reach an accepting/closing state before the budget is exhausted, leading to truncated syntax failures (e.g., unclosed brackets) and dead-end traps.
In this paper, we propose the Goal-Conditioned Reachability Logit Masker (GCLM), an ultra-fast, strictly $\mathcal{O}(1)$ runtime constrained decoding engine that guarantees goal satisfaction within a finite token horizon. GCLM decouples grammar precomputation from inference: it computes a compact, 2D backward reachability bitmap $R \in \mathbb{B}^{(T_{\max}+1) \times |S|}$ via a one-time vectorized Breadth-First Search (BFS) offline. At runtime, GCLM evaluates whether candidate transitions can reach an accepting goal state within the remaining budget $T_{\text{rem}}$ using a single, in-place tensor slice in $\mathcal{O}(1)$ time.
Extensive experiments demonstrate that GCLM achieves a 100.0% valid JSON parsing rate across strict token limits ($T_{\max} \in [4, 16]$) where forward DFA baselines fail up to 54.4% of the time. In multi-step agent tool-calling benchmarks, GCLM eliminates infinite retry loops and traps, securing 100.0% goal completion compared to 16.8% for forward DFAs. Furthermore, empirical scaling experiments across state counts $|S| \in [10, 10^4]$ and vocabulary sizes up to 151,643 confirm that GCLM maintains a flat, strictly $\mathcal{O}(1)$ per-token latency ($< 0.4\text{ ms}$ on CPU, $< 0.05\text{ ms}$ on GPU), while reducing total generation latency by up to 40% via proactive early completion.
1. Introduction
Large Language Models (LLMs) are increasingly deployed in autonomous pipelines requiring structured outputs (e.g., JSON payloads, SQL queries, and tool invocations). Standard autoregressive sampling frequently outputs invalid syntax. To address this, constrained decoding algorithms intervene at each step by masking illegal vocabulary tokens based on a formal grammar or regular expression.
Despite their popularity, current state-of-the-art constrained decoding frameworks (e.g., Outlines, SGLang) share a fundamental structural limitation: time-agnostic forward exploration. They verify whether a candidate token $v$ creates a valid prefix ($\delta(s, v) \ge 0$), assuming an infinite generation horizon.
In practice, generation is bounded by a strict token budget $T_{\max}$. When $T_{\text{rem}}$ is small, entering deep syntactic branches (such as nested JSON keys) leads to abrupt budget exhaustion before closing delimiters (}) can be emitted. Similarly, in multi-step agent workflows, models may enter exploratory sub-trees or infinite retry loops, failing to reach the Finish() state.
Contributions
- Theoretical Formulation: We formalize the finite-horizon constrained decoding problem and define the time-bounded backward reachability tensor $R \in \mathbb{B}^{(T_{\max}+1) \times |S|}$.
- $\mathcal{O}(1)$ Vectorized Engine: We design an in-place PyTorch logit processor that checks candidate transitions against $R$ in $\mathcal{O}(1)$ time with zero memory allocation.
- Comprehensive Empirical Validation: We demonstrate 100% goal completion across synthetic dead-end traps, nested JSON schemas, agent tool-calling, asymptotic scaling ($|S| \le 10,000$), and real-world generation with
Qwen2.5-0.5B.
2. Related Work
- Forward DFA Maskers: Outlines (Willard & Louf, 2023) and SGLang (Zheng et al., 2024) compile regular expressions into DFAs. While fast, they are infinite-horizon and time-agnostic.
- Parser-Based Decoding: SynCode (Ugolotti et al., 2024) and PICARD (Scholak et al., 2021) maintain incremental parser state stacks, incurring variable per-token latency without deadline enforcement.
- Formal Reachability: While model checking (Baier & Katoen, 2008) utilizes backward reachability for verification, GCLM is the first to operationalize time-bounded backward reachability as an $\mathcal{O}(1)$ PyTorch logit mask for autoregressive LLMs.
3. Methodology
3.1 Finite-Horizon Backward Reachability
Let an FSM be defined as $\mathcal{M} = (S, \mathcal{V}, \delta, s_0, S_{\mathrm{goal}})$.
R[0, s] =
\begin{cases}
\mathrm{True} & \text{if } s \in S_{\mathrm{goal}} \\
\mathrm{False} & \text{otherwise}
\end{cases}
For step $t = 1, \dots, T_{\max}$:
R[t, s] = R[t-1, s] \;\lor\; \left( \exists v \in \mathcal{V} \text{ s.t. } \delta(s, v) \ge 0 \;\land\; R[t-1, \delta(s, v)] = \mathrm{True} \right)
3.2 $\mathcal{O}(1)$ Runtime Logits Processor
At decoding step $k$ with remaining budget $T_{\text{rem}} = T_{\max} - k$:
\mathrm{ValidTokens}(v) = (\delta(s_{\mathrm{curr}}, v) \ge 0) \;\land\; R\big[\min(T_{\text{rem}}-1, T_{\max}), \;\mathrm{clamp}(\delta(s_{\mathrm{curr}}, v), 0)\big]
\mathrm{Logits}[v] =
\begin{cases}
\mathrm{Logits}[v] & \text{if } \mathrm{ValidTokens}(v) = \mathrm{True} \\
-\infty & \text{otherwise}
\end{cases}
4. Empirical Evaluation
Exp 1: Dead-End Trap Avoidance (1,000 Trials)
| Method | Goal Reach Rate (%) | Trap Entry Rate (%) |
|---|---|---|
| Vanilla (Unconstrained) | 1.30% | 48.70% |
| Forward DFA (Outlines Style) | 50.20% | 49.80% |
| GCLM (Ours) | 100.00% | 0.00% |
Exp 2: Strict Budget JSON Schema Parsing (500 Trials/Cell)
| Budget | Vanilla | Forward DFA | GCLM (Ours) |
|---|---|---|---|
| $T_{\max} = 4$ | 2.4% | 55.4% | 100.0% (Forces early {} closure) |
| $T_{\max} = 6$ | 2.4% | 45.6% | 100.0% |
| $T_{\max} = 8$ | 2.2% | 65.2% | 100.0% |
| $T_{\max} = 12$ | 3.4% | 83.4% | 100.0% |
| $T_{\max} = 16$ | 1.4% | 91.8% | 100.0% |
Exp 3: Multi-Step Agent Tool-Calling Benchmark
| Action Budget | Vanilla | Forward DFA | GCLM (Ours) | Key Finding |
|---|---|---|---|---|
| 3 Actions | 0.00% | 16.80% | 100.00% | GCLM dynamically forces 3-step shortest path |
| 4 Actions | 0.00% | 33.20% | 100.00% | Prunes unfinishable deep search subtrees |
| 8 Actions | 0.60% | 65.20% | 100.00% | Completely avoids infinite retry trap loops |
Exp 4: Empirical $\mathcal{O}(1)$ Complexity Scaling
| Vocabulary Size $\vert\mathcal{V}\vert$ | State Count $\vert S\vert$ | Offline BFS Time | Memory Footprint | Online Latency per Token |
|---|---|---|---|---|
| $\vert\mathcal{V}\vert = 32,000$ (LLaMA) | $\vert S\vert = 10$ | 29.55 ms | 2.44 MB | 388.72 $\mu$s |
| $\vert\mathcal{V}\vert = 32,000$ | $\vert S\vert = 100$ | 240.10 ms | 24.42 MB | 335.10 $\mu$s |
| $\vert\mathcal{V}\vert = 32,000$ | $\vert S\vert = 1,000$ | 2,111.82 ms | 244.19 MB | 340.84 $\mu$s |
| $\vert\mathcal{V}\vert = 32,000$ | $\vert S\vert = 10,000$ | 25,790.14 ms | 2.44 GB | 356.29 $\mu$s ($\mathcal{O}(1)$ verified) |
| $\vert\mathcal{V}\vert = 151,643$ (Qwen2.5) | $\vert S\vert = 10$ | 159.29 ms | 11.57 MB | 601.92 $\mu$s |
| $\vert\mathcal{V}\vert = 151,643$ | $\vert S\vert = 10,000$ | 147,702.79 ms | 11.56 GB | 666.22 $\mu$s ($\mathcal{O}(1)$ verified) |
Exp 5: Real Lightweight LLM (Qwen2.5-0.5B, $|\mathcal{V}|=151,643$) End-to-End Generation
| Token Budget $T_{\max}$ | Vanilla Sampling | Forward DFA | GCLM (Ours) | Latency / Sample |
|---|---|---|---|---|
| $T_{\max} = 6$ tokens | 0.0% | 30.0% | 100.0% | 615.90 ms (Fastest) |
| $T_{\max} = 10$ tokens | 0.0% | 70.0% | 100.0% | 1,086.02 ms |
| $T_{\max} = 16$ tokens | 0.0% | 85.0% | 100.0% | 992.39 ms |
5. Conclusion
GCLM bridges formal reachability analysis and runtime logit masking for LLMs. By shifting time-bounded reachability to an offline vectorized BFS precomputation, GCLM guarantees goal satisfaction and syntactic closure under finite token horizons in strict $\mathcal{O}(1)$ time.