uuugi commited on
Commit
bc69298
·
verified ·
1 Parent(s): 6ff2546

Completely sanitize math and tables for Hugging Face Hub rendering

Browse files
Files changed (1) hide show
  1. README.md +32 -32
README.md CHANGED
@@ -20,8 +20,8 @@ pipeline_tag: text-generation
20
  [![Transformers 4.36+](https://img.shields.io/badge/Transformers-4.36+-yellow.svg)](https://huggingface.co/docs/transformers)
21
  [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
22
 
23
- An ultra-fast, strictly $O(1)$ runtime **Goal-Conditioned Reachability Logit Masking Engine** for Large Language Models.
24
- GCLM mathematically guarantees that an LLM will strictly reach designated goal/accepting states within a fixed token budget ($T_{\max}$), **fundamentally preventing dead-end traps and truncated syntax failures**.
25
 
26
  ---
27
 
@@ -40,38 +40,38 @@ GCLM mathematically guarantees that an LLM will strictly reach designated goal/a
40
 
41
  | Feature | Standard Forward DFA (Outlines / SGLang) | **GCLM (Ours)** |
42
  | :--- | :--- | :--- |
43
- | **Masking Basis** | Current state validity ($s_{\text{curr}} \to s'$) | **Time-bounded backward reachability** ($s_{\text{curr}} \to s' \to^* S_{\text{goal}}$ in $\le T_{\text{rem}}-1$ steps) |
44
  | **Dead-End Traps** | ❌ May enter valid forward branches that lead to dead-ends | ✅ **Preemptively masked** before entering trap |
45
- | **Token Budget Exceeded**| ❌ Outputs truncated/broken syntax when budget ends | ✅ **Forces early syntax closure** before budget exhaustion |
46
- | **Per-Token Overhead** | $O(1)$ table lookup | **Strict $O(1)$ vectorized PyTorch lookup (< 0.1ms)** |
47
- | **Complexity Scaling** | Scales with active state transitions | **Zero runtime dependence on state count (\|S\|)** |
48
 
49
  ---
50
 
51
  ## 📐 Mathematical Formulation
52
 
53
  ### 1. Offline Backward BFS Table Builder
54
- Given an FSM $(S, \Sigma, \delta, s_0, S_{\mathrm{goal}})$ and maximum token budget $T_{\max}$, we precompute a reachability tensor $R \in \mathbb{B}^{(T_{\max} + 1) \times |S|}$ via vectorized backward BFS:
55
 
56
  $$
57
- R[0, s] = \begin{cases} \text{True} & \text{if } s \in S_{\text{goal}} \\ \text{False} & \text{otherwise} \end{cases}
58
  $$
59
 
60
- For $t = 1, \dots, T_{\max}$:
61
 
62
  $$
63
- R[t, s] = R[t-1, s] \;\lor\; \left( \exists v \in \mathcal{V} \text{ s.t. } \delta(s, v) \ge 0 \;\land\; R[t-1, \delta(s, v)] = \text{True} \right)
64
  $$
65
 
66
- ### 2. Strict $O(1)$ Runtime Logits Masking
67
- At decoding step $k$ with remaining budget $T_{\text{rem}} = T_{\max} - k$:
68
 
69
  $$
70
- \text{ValidTokens}(v) = (\delta(s_{\text{curr}}, v) \ge 0) \;\land\; R\big[\min(T_{\text{rem}}-1, T_{\max}), \;\text{clamp}(\delta(s_{\text{curr}}, v), 0)\big]
71
  $$
72
 
73
  $$
74
- \text{Logits}[v] = \begin{cases} \text{Logits}[v] & \text{if } \text{ValidTokens}(v) = \text{True} \\ -\infty & \text{otherwise} \end{cases}
75
  $$
76
 
77
  ---
@@ -109,23 +109,23 @@ gclm_project/
109
  ### 1. Real Lightweight LLM End-to-End Benchmark (`Qwen2.5-0.5B`)
110
  > Tested on real model weights generating JSON responses under strict token limits.
111
 
112
- | Token Budget ($T_{\max}$) | Vanilla Sampling | Forward DFA (Outlines Style) | **GCLM (Ours)** | Latency / Sample (GCLM) |
113
  | :--- | :---: | :---: | :---: | :---: |
114
- | **$T_{\max} = 6$ tokens** | 0.0% | 30.0% | **100.0%** | **615.90 ms** (Fastest, early closure) |
115
- | **$T_{\max} = 10$ tokens**| 0.0% | 70.0% | **100.0%** | **1,086.02 ms** |
116
- | **$T_{\max} = 16$ tokens**| 0.0% | 85.0% | **100.0%** | **992.39 ms** |
117
 
118
  ---
119
 
120
  ### 2. Strict Budget JSON Schema Parsing Benchmark
121
  > Complex nested JSON schema tested across 500 trials per budget.
122
 
123
- | Budget ($T_{\max}$) | Vanilla | Forward DFA (Outlines Style) | **GCLM (Ours)** | Key Insight |
124
  | :--- | :---: | :---: | :---: | :--- |
125
- | **$T_{\max} = 4$** | 2.4% | 55.4% | **100.0%** | **Forces safe `{}` closure when fields cannot finish** |
126
- | **$T_{\max} = 6$** | 2.4% | 45.6% | **100.0%** | Prunes deep nested object paths |
127
- | **$T_{\max} = 8$** | 2.2% | 65.2% | **100.0%** | Eliminates dangling commas |
128
- | **$T_{\max} = 16$** | 1.4% | 91.8% | **100.0%** | Complete 100% parse rate across all budgets |
129
 
130
  ---
131
 
@@ -140,17 +140,17 @@ gclm_project/
140
 
141
  ---
142
 
143
- ### 4. FSM Complexity & Strict $O(1)$ Runtime Scaling
144
- > Scaling state count $|S|$ from 10 to 10,000 (1,000x increase). Plot saved as `paper_figure_scaling.png`.
145
 
146
- | Vocabulary Size (\|V\|) | State Count (\|S\|) | Offline BFS Time | Memory Footprint | Online Latency per Token |
147
  | :--- | :---: | :---: | :---: | :---: |
148
- | **\|V\| = 32,000 (LLaMA)** | \|S\| = 10 | 29.55 ms | 2.44 MB | **388.72 µs** |
149
- | \|V\| = 32,000 | \|S\| = 100 | 240.10 ms | 24.42 MB | **335.10 µs** |
150
- | \|V\| = 32,000 | \|S\| = 1,000 | 2,111.82 ms | 244.19 MB | **340.84 µs** |
151
- | \|V\| = 32,000 | **\|S\| = 10,000** | 25,790.14 ms | 2.44 GB | **356.29 µs** ($O(1)$ empirically verified) |
152
- | **\|V\| = 151,643 (Qwen2.5)** | \|S\| = 10 | 159.29 ms | 11.57 MB | **601.92 µs** |
153
- | \|V\| = 151,643 | **\|S\| = 10,000** | 147,702.79 ms | 11.56 GB | **666.22 µs** ($O(1)$ empirically verified) |
154
 
155
  ---
156
 
 
20
  [![Transformers 4.36+](https://img.shields.io/badge/Transformers-4.36+-yellow.svg)](https://huggingface.co/docs/transformers)
21
  [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
22
 
23
+ An ultra-fast, strictly \(O(1)\) runtime **Goal-Conditioned Reachability Logit Masking Engine** for Large Language Models.
24
+ GCLM mathematically guarantees that an LLM will strictly reach designated goal/accepting states within a fixed token budget (\(T_{\max}\)), **fundamentally preventing dead-end traps and truncated syntax failures**.
25
 
26
  ---
27
 
 
40
 
41
  | Feature | Standard Forward DFA (Outlines / SGLang) | **GCLM (Ours)** |
42
  | :--- | :--- | :--- |
43
+ | **Masking Basis** | Current state validity (`s_curr -> s'`) | **Time-bounded backward reachability** (`s_curr -> s' ->* S_goal` in `≤ T_rem - 1` steps) |
44
  | **Dead-End Traps** | ❌ May enter valid forward branches that lead to dead-ends | ✅ **Preemptively masked** before entering trap |
45
+ | **Token Budget Exceeded** | ❌ Outputs truncated/broken syntax when budget ends | ✅ **Forces early syntax closure** before budget exhaustion |
46
+ | **Per-Token Overhead** | O(1) table lookup | **Strict O(1) vectorized PyTorch lookup (< 0.1ms)** |
47
+ | **Complexity Scaling** | Scales with active state transitions | **Zero runtime dependence on state count (S)** |
48
 
49
  ---
50
 
51
  ## 📐 Mathematical Formulation
52
 
53
  ### 1. Offline Backward BFS Table Builder
54
+ Given an FSM \((S, \Sigma, \delta, s_0, S_{\mathrm{goal}})\) and maximum token budget \(T_{\max}\), we precompute a reachability tensor \(R \in \mathbb{B}^{(T_{\max} + 1) \times |S|}\) via vectorized backward BFS:
55
 
56
  $$
57
+ R[0, s] = \mathbf{1}(s \in S_{\mathrm{goal}})
58
  $$
59
 
60
+ For \(t = 1, \dots, T_{\max}\):
61
 
62
  $$
63
+ R[t, s] = R[t-1, s] \lor \left( \exists v \in \mathcal{V} \text{ s.t. } \delta(s, v) \ge 0 \land R[t-1, \delta(s, v)] = 1 \right)
64
  $$
65
 
66
+ ### 2. Strict \(O(1)\) Runtime Logits Masking
67
+ At decoding step \(k\) with remaining budget \(T_{\mathrm{rem}} = T_{\max} - k\):
68
 
69
  $$
70
+ \mathrm{ValidTokens}(v) = (\delta(s_{\mathrm{curr}}, v) \ge 0) \land R\big[\min(T_{\mathrm{rem}}-1, T_{\max}), \;\mathrm{clamp}(\delta(s_{\mathrm{curr}}, v), 0)\big]
71
  $$
72
 
73
  $$
74
+ \mathrm{Logits}[v] = \begin{cases} \mathrm{Logits}[v] & \text{if } \mathrm{ValidTokens}(v) = 1 \\ -\infty & \text{otherwise} \end{cases}
75
  $$
76
 
77
  ---
 
109
  ### 1. Real Lightweight LLM End-to-End Benchmark (`Qwen2.5-0.5B`)
110
  > Tested on real model weights generating JSON responses under strict token limits.
111
 
112
+ | Token Budget (T_max) | Vanilla Sampling | Forward DFA (Outlines Style) | **GCLM (Ours)** | Latency / Sample (GCLM) |
113
  | :--- | :---: | :---: | :---: | :---: |
114
+ | **T_max = 6 tokens** | 0.0% | 30.0% | **100.0%** | **615.90 ms** (Fastest, early closure) |
115
+ | **T_max = 10 tokens** | 0.0% | 70.0% | **100.0%** | **1,086.02 ms** |
116
+ | **T_max = 16 tokens** | 0.0% | 85.0% | **100.0%** | **992.39 ms** |
117
 
118
  ---
119
 
120
  ### 2. Strict Budget JSON Schema Parsing Benchmark
121
  > Complex nested JSON schema tested across 500 trials per budget.
122
 
123
+ | Budget (T_max) | Vanilla | Forward DFA (Outlines Style) | **GCLM (Ours)** | Key Insight |
124
  | :--- | :---: | :---: | :---: | :--- |
125
+ | **T_max = 4** | 2.4% | 55.4% | **100.0%** | **Forces safe `{}` closure when fields cannot finish** |
126
+ | **T_max = 6** | 2.4% | 45.6% | **100.0%** | Prunes deep nested object paths |
127
+ | **T_max = 8** | 2.2% | 65.2% | **100.0%** | Eliminates dangling commas |
128
+ | **T_max = 16** | 1.4% | 91.8% | **100.0%** | Complete 100% parse rate across all budgets |
129
 
130
  ---
131
 
 
140
 
141
  ---
142
 
143
+ ### 4. FSM Complexity & Strict O(1) Runtime Scaling
144
+ > Scaling state count \(|S|\) from 10 to 10,000 (1,000x increase). Plot saved as `paper_figure_scaling.png`.
145
 
146
+ | Vocabulary Size (V) | State Count (S) | Offline BFS Time | Memory Footprint | Online Latency per Token |
147
  | :--- | :---: | :---: | :---: | :---: |
148
+ | **V = 32,000 (LLaMA)** | S = 10 | 29.55 ms | 2.44 MB | **388.72 µs** |
149
+ | V = 32,000 | S = 100 | 240.10 ms | 24.42 MB | **335.10 µs** |
150
+ | V = 32,000 | S = 1,000 | 2,111.82 ms | 244.19 MB | **340.84 µs** |
151
+ | V = 32,000 | **S = 10,000** | 25,790.14 ms | 2.44 GB | **356.29 µs** (O(1) verified) |
152
+ | **V = 151,643 (Qwen2.5)** | S = 10 | 159.29 ms | 11.57 MB | **601.92 µs** |
153
+ | V = 151,643 | **S = 10,000** | 147,702.79 ms | 11.56 GB | **666.22 µs** (O(1) verified) |
154
 
155
  ---
156