Submission Type: Short Research Paper

# Introduction

Processing large document corpora through large language models requires hardware infrastructure that few research groups possess. Full self-attention scales as $O(N^2)$. At 65,536 tokens, inference costs confine deep textual analysis to institutions operating large GPU clusters. The bottleneck is algorithmic; its practical consequence is a dependency on hardware capital that most groups cannot meet.

In the foundational work , we established the theoretical foundation of Reduced Interaction Sampling (RIS), proving mathematically that the $O(N^2)$ attention bottleneck can be bypassed via stochastic sparsification while preserving factual retrieval. The present paper validates this mechanism empirically under LLM inference. Turning this theory into a functional inference engine required solving three engineering problems:
(1) generating sparse geometric masks in $O(N^2)$ space using lightweight boolean structures to avoid out-of-memory faults during allocation; (2) normalizing stochastic token fragments without diluting their competitive weight; and (3) preventing the collapse of positional encodings when extending windows far beyond native training limits. This paper describes the RIS-Kernel architecture, a systems-level implementation that injects runtime sparsity directly into unmodified language models through two sampling regimes:
*Stochastic Mode* for global coverage and *Structural Mode* for local community preservation.

Validation focuses on Qwen2-1.5B-Instruct  and TinyLlama-1.1B . Sub-2B parameter models define the most restrictive regime for long-context factual retrieval: compact enough to run on CPU hardware without GPU acceleration, yet architecturally complete enough to respond meaningfully to context delivered at inference time. Factual accuracy demands inference-time context delivery . RIS enables this on unaccelerated CPU hardware. Reducing attention time complexity to
$O(N \log N)$ allows a high-memory CPU server to process 65k-token contexts, removing the dependency on hardware accelerators for deep document retrieval.

# Results

## Hardware Profiling and CPU Ingestion Bounds

Experiments were conducted on two unaccelerated CPU servers: a high-memory Xeon workstation (ibteci: 2 sockets, 20 physical cores,
128 GB DDR4) and a desktop server (bioinfo: Intel Core i7-3770, 4
physical cores, 16 GB DDR4). The dominant bottleneck on CPU is memory bandwidth (L3 cache saturation). Float16 emulation on CPUs lacking native AVX-512 pushed single optimization steps past 60 hours, whereas Float32 restored predictable runtimes. Thread allocation was restricted near physical core boundaries (4 threads on bioinfo and 8 on ibteci) to prevent memory bus saturation and restore optimal throughput.

## Empirical Evaluation

The evaluation contexts are built by concatenating four scientific manuscripts covering distinct biological domains: *ajin­sha­nen­sis*,
*aom*, *genppi*, and *meta*, cleaned of metadata preceding the introduction. We use logit-based Q&A analysis to eliminate generative bias.

### Experiment A: Controlled Precision Comparison (32k Window) {#sec:exp_a}

At Qwen2's native limit of 32,768 tokens, full dense attention serves as the ground-truth target. The zero-context floor is 59.38% and the native dense target is 71.88% (a gap of 12.5 percentage points; the full grid breakdown across densities and seeds is provided in Table S2 of the Supplementary Material).

<figure id="fig:context_recovery_32k" data-latex-placement="t">
<div class="minipage">
<embed src="fig_heatmap_32k.pdf" />
</div>
<div class="minipage">
<embed src="fig_context_recovery_32k.pdf" />
</div>
<figcaption>Ensemble scaling and gap recovery rate on 32k window.</figcaption>
</figure>

As shown in Fig. [\[fig:heatmap_32k\]](#fig:heatmap_32k){reference-type="ref"
reference="fig:heatmap_32k"} and
[1](#fig:context_recovery_32k){reference-type="ref"
reference="fig:context_recovery_32k"}, RIS-Stochastic at 1% density and
70--80 seeds achieves 75.00% accuracy, outperforming the native dense baseline of 71.88% (125% gap recovery). The ensemble acts as an attention regularizer, pruning query-key noise. At 5% density, accuracy is capped at 71.88% (10 seeds) as distractor noise is reintroduced.
RIS-Structural converges faster: at 1% density and 10 seeds, it recovers
75.0% of the contextual gap (68.75% accuracy) whereas Stochastic requires 50 seeds. The block-clique geometry preferentially captures distal anchors under tight budgets.

### Experiment B: Scalability and Extrapolation (64k Window) {#sec:exp_b}

At 65,536 tokens ($2\times$ native limit), dense attention produces OOM faults. Scalability is evaluated using the expanded 64-question set,
with a zero-context floor of 51.56%.

<figure id="fig:qa_panel" data-latex-placement="t">
<div class="minipage">
<embed src="fig_heatmap_seeds_density.pdf" />
</div>
<div class="minipage">
<embed src="fig_qa_panel.pdf" />
</div>
<figcaption>Performance scaling of RIS modes at 65,536
tokens.</figcaption>
</figure>

Results diverge with the RoPE scaling method. Under linear interpolation, accuracy collapses to 15.6%--23.4% (random guessing baseline) with a single seed. Under YaRN scaling, the positional geometry is preserved and RIS-Kernel scales effectively: at 1% density and 40 seeds, accuracy reaches 57.8%, and peaks at 62.50% under 5%
density (40--60 seeds). This is confirmed as marginally significant under McNemar's paired test ($p = 0.078 < 0.10$).

### Ensemble Coverage Analysis

The ensemble scaling curve is concave for both modes. This concavity is governed by the total context coverage fraction of the ensembled key-value cache, defined as the union of the sparse indices selected by each seed. For $N$ independent seeds at attention density $d$, the union coverage fraction $U$ is given by $U = 1 - (1 - d)^N$. This explains why higher attention densities can degrade retrieval performance. Under RIS-Stochastic at 32k tokens, density $d=0.01$ ensembled over $N=70$
seeds yields $U \approx 50.5\%$: sufficient coverage to retrieve the target anchor tokens while pruning sequence-level distractors, allowing the model to exceed the dense baseline (75.00% vs. 71.88%). At $d=0.05$
with the same seed budget ($N=70$), $U \approx 97.2\%$ --- virtually dense attention that reintroduces sequence-level noise, capping accuracy at 71.88%.

## RoPE Scaling: Linear versus YaRN {#sec:rope}

How correctly positional encodings behave at 65,536 tokens bounds the RIS retrieval mechanism. Positional dilution under linear interpolation destroys positional coherence to the point where context is invisible,
yielding near-random accuracy (15.6%--23.4%) with a single seed. The stochastic ensemble partially compensates (recovering to 59.4% with 40
seeds at 5% density) by statistical voting, but YaRN positional preservation represents the superior scaling pathway, reaching 60.9% (2%
density) and 62.50% (5% density) under moderate seed counts.

<figure id="fig:rope_impact" data-latex-placement="t">
<embed src="fig_accuracy_window.pdf" style="width:45.0%" />
<figcaption>Accuracy vs context window under linear and YaRN scaling.</figcaption>
</figure>

TinyLlama-1.1B (native 2k limit) experiments under YaRN scaling show complete cognitive collapse beyond $4\times$ native training distribution (8k+ tokens) regardless of density or seeds. This sets a boundary condition: stochastic ensemble retrieval requires that the host model's positional encoding system remain at least partially functional at the target context length.

## Stochastic versus Structural Performance Comparison

RIS-Stochastic scales monotonically with density, peaking at 62.50% (5%
density). However, under severe budget limits (1% density),
RIS-Structural outperforms Stochastic, reaching an absolute peak of
65.62% (mean 56.53% vs. 52.13% for Stochastic). The block-clique geometry guarantees anchor neighborhood coverage under extreme sparsity,
whereas Stochastic uniform sampling has high variance. At higher densities (5%), the structural cliques over-allocate budget to local neighborhoods, causing Stochastic to dominate.

## Sweet-Spot Analysis: Sub-1% Structural Density at 64k {#sec:sweet_spot_sub1}

To determine whether the RIS-Structural mode yields useful retrieval at densities below the 1% baseline, we conducted a systematic grid search over effective structural block densities ranging from 0.1% to 0.9%
($B_{max} = 66$ to $590$ tokens), with ensemble seeds of 50, 100, 150,
and 200. All runs used YaRN scaling on Qwen2-1.5B at 65,536 tokens (the complete grid search results are reported in Table S1 of the Supplementary Material).

The search reveals a clear efficiency regime between 0.3% and 0.5%
density. In this range, the best accuracy stabilizes at 59.38%, achieved consistently with 150--200 seeds. At $d=0.3\%$ ($B_{max}=197$ tokens),
this represents a 70% reduction in structural attention cost relative to the 1.0% baseline ($B_{max}=655$) at a cost of only 6.25 percentage points in peak accuracy (59.38% vs 65.63%). The 0.5% configuration offers the most stable operating point: mean accuracy reaches 58.20% and the seed-sensitivity range narrows to 3.13 percentage points across
50--200 seeds.

Two observations warrant attention. First, the ensemble saturates between 150 and 200 seeds: adding seeds beyond 150 yields no further gain, consistent with stochastic coverage saturation. Second, the accuracy profile is non-monotonic in density: a local peak at 0.8%
(62.50%) followed by a marked drop at 0.9% (57.81%) indicates that the discrete values of $B_{max}$ interact with the geometric boundaries of the Qwen2 attention block structure. The sub-1% regime exposes a resource-efficiency frontier where the Structural mode retains over 90%
of the contextual retrieval signal at less than half the structural attention cost.

## Cross-Document Synthesis: Qualitative Validation {#sec:crossover}

To evaluate the capacity of RIS to perform conceptual synthesis across distinct documents, we designed a qualitative crossover experiment using the same four source manuscripts paired in six pairwise document combinations (ajinshanensis, aom, genppi, and meta), paired and cleaned of metadata preceding the introduction.
Table [1](#tab:crossover_metrics){reference-type="ref"
reference="tab:crossover_metrics"} summarizes the computational and generation metrics.

::: {#tab:crossover_metrics}
| Document | Context | Execution Time | Characters Gen. | Hallucinations |
| Pair | | | | |
| | (Tokens) | Phase 1 | Phase 2 | Phase 1 | Phase 2 | (Ph1/Ph2) |
| ajin - | 30,000 | 01:53:00 | 00:43:00 | 4,801 | 3,327 | 0 / 0 |
| aom | | | | | | |
| ajin - | 29,500 | 01:50:00 | 00:46:00 | 2,426 | 3,735 | 0 / 0 |
| gen | | | | | | |
| ajin - | 22,800 | 01:48:00 | 00:52:00 | 4,210 | 5,230 | 1 / 0 |
| meta | | | | | | |
| aom - | 32,400 | 01:50:00 | 00:16:00 | 3,491 | 907 | 0 / 0 |
| gen | | | | | | |
| aom - | 25,600 | 01:30:00 | 01:03:00 | 3,403 | 4,158 | 0 / 0 |
| meta | | | | | | |
| gen - | 25,200 | 01:36:00 | 00:27:00 | 3,672 | 2,362 | 0 / 0 |
| meta | | | | | | |

: Qualitative crossover experiment metrics (Phase 1 vs Phase 2).
:::

Across both phases, the model identified the conceptual intersections between the paired texts:

- ajinshanensis - aom: The model identified that both studies apply systems biology tools to characterize specialized metabolic modules in distinct extreme niches, noting the physical isolation of stingless bee larval food versus marine anaerobic methane oxidation.

- ajinshanensis - genppi: The model connected the interactome of *Acetilactobacillus jinshanensis* with the genomic context methods of GenPPi 1.5, noting that both studies rely on evolutionary conservation instead of experimental database matching.

- ajinshanensis - meta: The model linked the niche construction dynamics between early and late colonizers with the Jatai bee metagenomic dataset.

- aom - genppi: The model related the interactome construction of anaerobic methanotrophs with the ab initio predictive capabilities of the sampling algorithm.

- aom - meta: The model identified that both papers employ high-throughput sequencing to characterize metabolic capabilities of uncultivated microorganisms in extreme or specialized environments.

- genppi - meta: The model mapped the ab initio protein interaction prediction methods directly onto the metagenomic diversity analysis of the stingless bee microbiome.

In Phase 1, one minor semantic error occurred where the model substituted *Bacillus subtilis* for *Bacillus cereus* in the Jatai bee larval food context. Phase 1 outputs also exhibited syntax degradation and trailing symbol repetitions. In Phase 2, the increased attention density (3%) and the reduced seed count (30) resolved the formatting errors and lowered CPU execution times by 30% to 70% while maintaining zero semantic hallucinations.

The low rate of semantic errors (1 out of 12 runs) demonstrates that sparse attention preserves the model's capacity to retrieve and integrate facts across long contexts. The non-semantic formatting errors in Phase 1 likely stemmed from the interaction between high sparsity
(1%) and position embedding dilution under linear scaling, which was exacerbated by the low temperature (0.1) and high repetition penalty
(1.3). The regularizing effect of RIS appears to filter positional noise under linear scaling, allowing coherent retrieval beyond native context boundaries. Increasing the density to 3% in Phase 2 stabilized the attention mapping, providing sufficient positional cues to prevent syntax collapse.

# Discussion

## Controlled Parity with Native Dense Attention

The 32k controlled experiment
(Section [2.2.1](#sec:exp_a){reference-type="ref"
reference="sec:exp_a"}) provides the most direct test of the RIS-Kernel architecture. Where a full-attention dense baseline is computationally feasible, RIS-Stochastic at 1% density and 70--80 seeds exceeds the native dense attention baseline (75.00% vs 71.88%), while RIS-Stochastic at 5% density and 10 seeds achieves exact parity. The stochastic ensemble not only captures the complete factual retrieval signal of a dense $O(N^2)$ attention matrix at a fraction of the cost, but also acts as an attention regularizer. Sparsifying the attention matrix prunes the noisy query-key connections that accumulate over long sequences, letting the model focus on high-importance routing pathways. At 1% density and
10 seeds, the Structural mode captures 75% of the contextual gap (68.75%
accuracy), demonstrating that local community priors route attention to factual anchors effectively even under extreme sparsity.

## The Parametric Memory Floor Hypothesis

The zero-context baseline ($w=0$) achieves 59.38% accuracy on the balanced 32k set and 51.56% on the 64k set --- well above the 20% random floor. We attribute this elevated floor to parametric memory rather than data contamination. The scientific articles were published after the training cutoff. During pre-training on large biomedical and scientific corpora, the model encountered thousands of texts covering related bioinformatics, protein-protein interaction networks, and statistical methodologies. The model retains enough domain structure --- the biochemical role of heterodisulfide reductase complexes, the applicability of the Kolmogorov--Smirnov test --- to resolve plausible options from question semantics alone. Absolute accuracy is therefore an unreliable metric; the context recovery rate isolates the RIS contribution more cleanly.

## Architectural Advantages over Fixed-Geometry Sparse Attention

Unlike BigBird  or Longformer , which rely on fixed strided or block-local attention patterns, RIS employs a stochastic ensemble. A fixed block matrix fails whenever a key entity falls outside its predefined geometric stride. The RIS ensemble coverage formula
$1 - (1-\rho_\mathrm{sub})^{k \cdot N}$ circumvents this brittleness with a probabilistic net that holds regardless of the host model.
Doubling the context length doubles the computational cost, bypassing the quadratic explosion that blocks extended inference under dense attention.

## Hardware Context and Reproducibility

The primary 32k controlled evaluation was executed on the bioinfo server: a commodity desktop PC with 16 GB of DDR4 RAM, setting a low infrastructure floor for reproducibility. In both regimes, ingestion latency is the primary bottleneck, driven by memory bandwidth rather than arithmetic throughput. The $O(N \log N)$ geometry ensures that scaling the window does not multiply wall-clock cost superlinearly. The upfront compute cost to parse a large document set accurately is negligible compared to the financial and temporal cost of fine-tuning a model that will ultimately fail to memorize those same facts.

# Methods: The RIS Implementation

RIS replaces the attention kernel through instance method injection at runtime. Weights, tokenizer, and all remaining components are untouched . The same implementation runs on LLaMA and Qwen2 architectures without modification.

<figure id="fig:mask_generation" data-latex-placement="t">
<embed src="fig_flow_mask_generation.pdf" style="width:55.0%" />
<figcaption>Streaming mask generation.</figcaption>
</figure>

<figure id="fig:pfus_logic" data-latex-placement="t">
<embed src="fig_flow_pfus_logic.pdf" style="width:55.0%" />
<figcaption>PFUS index fusion.</figcaption>
</figure>

## Memory-Bounded Geometry Generation

Allocating 10 seed masks simultaneously at 65,536 tokens would require over 40 GB in boolean tensors. The streaming design avoids this by initializing masks sequentially: each seed's indices are drawn
(`torch.randint`), filtered for causality, merged into a single master mask, and discarded before the next seed is processed
(Figure [4](#fig:mask_generation){reference-type="ref"
reference="fig:mask_generation"}). Peak memory remains bound by a single
$O(N^2)$ boolean matrix.

## Sampling Modes: Stochastic and Structural

RIS-Kernel implements two sampling strategies. *Stochastic Mode* treats the sequence as a uniform pool and draws global neighbors per pivot,
with coverage scaling monotonically with density and seed count.
*Structural Mode* partitions the sequence into blocks of size
$B = \min(0.1N, B_{max})$ and connects each block as a clique before adding global redundant edges. To ensure strict computational parity,
the effective density budget $d_{\text{eff}}$ is partitioned such that
$d_{\text{eff}} = d_{\text{clique}} + d_{\text{global}}$, explicitly deducting the clique cost from the global allowance. The block-clique geometry preserves local community structure, which is effective at extreme sparsity (1--2% density) where uniform sampling misses distal anchors.

## Hybrid Anchor and Pre-Fusion Unified Softmax (PFUS)

Recomputing the full stochastic index union at every generation step would dominate CPU inference time. The Hybrid Anchor splits the index set into a static *Stochastic Anchor* (the union of all seed indices computed once at the first token) and a *Dynamic Local Window* covering recent tokens (Figure [5](#fig:pfus_logic){reference-type="ref"
reference="fig:pfus_logic"}). The two sets merge via `unique()` before Key/Value extraction.

PFUS uses a single pre-fusion softmax rather than separate normalization of sparse and dense components
. This ensures that a stochastically recovered token competes within the global distribution and carries the same competitive weight as a token from the dense window
.

## Dynamic RoPE Scaling and Validation Protocol

For context windows beyond the trained positional limit, RIS intercepts the model configuration and adjusts the RoPE factor dynamically. Linear interpolation and YaRN  are supported without external framework dependencies.

Evaluation employs a two-stage validation protocol (32 Q&A at 32k tokens, 64 Q&A at 64k tokens) using scientific manuscript contexts.
Answers are evaluated via discriminative logit analysis: the log-probability of option choices is compared directly, removing generative bias. Under RIS, the capture probability of a target token appearing $k$ times over $N$ independent seeds at sub-density
$\rho_{\text{sub}}$ is $1 - (1 - \rho_{\text{sub}})^{k \cdot N}$. PFUS ensures that these recovered tokens are not penalized by index-set dilution before softmax normalization.