Enhaced instructions and references

Hugging Face Static page

Hugging Face Static page

Hugging Face Static page

Hugging Face Static page

fix(colab): resolve Colab execution OOM and kwargs validation error

fix: keep _MASTER_RIS_MASK and base_past_key_values on CPU RAM to free 3.0GB VRAM and guarantee OOM-free inference

fix: recursively move all DynamicCache layers (cache.layers[i].keys) to target device to prevent CPU vs CUDA device mismatch during cat in generate

fix: pass explicit cache_position and attention_mask to model.generate() to prevent transformers from trimming input_ids to empty tensor when past_key_values has 38k tokens

fix: remove broken exec-based ris_attention patch and add full traceback to error handler

fix: remove stopping_criteria from model.generate - rely entirely on native eos_token_id for stopping

fix: set chunk_size=256 and patch ris_attention.py to remove float32 casting during prefill (50% VRAM saving)

fix: load cached KV-cache with map_location=cpu and stream to GPU to avoid PyTorch deserialization memory peak

fix: patch StopOnTokens and SemanticStoppingCriteria in inference_ris_v3.py against empty tensor index errors

feat: add persistent KV-cache caching (torch.save/torch.load) to skip document ingestion on repeat runs

fix: make StopOnTokens stopping criteria safe against empty tensors and remove static position_ids/attention_mask from generate call

fix: force-delete previous model objects and run CUDA ipc_collect before model load to ensure full 14.56GB VRAM is free on cell re-execution

fix: set TOTAL_CAPACITY=40000 and RIS_DENSITY=0.025 to optimize RIS matrix memory footprint to 2.9GB for zero OOM risk

fix: load 100% of Qwen2.5-3B-Instruct directly on cuda:0 to eliminate multi-device mismatch and maximize free VRAM headroom

feat: select Qwen2.5-3B-Instruct (bfloat16) for Kaggle Dual-GPU — guarantees 11.4GB free VRAM per GPU with zero OOM risk

fix: adjust dual-GPU split to 10 layers on GPU 0 and 18 layers on GPU 1 to prevent PyTorch safetensors materialization buffer peak

fix: use explicit 50/50 layer split device_map dictionary for bfloat16 7B model to guarantee equal 7.1GB VRAM balance on Dual-GPU

fix: set max_memory={0: '9GiB', 1: '14GiB'} for bfloat16 7B model to reserve 5.5GB free VRAM on GPU 0 for RIS geometry

fix: remove partial CPU mask patch in ris_attention.py to avoid device mismatch between CPU mask and GPU tensors

fix: pass device_map=auto in non-4bit branch of inference_ris_v3.py so bfloat16 models load on GPUs

fix: force cuda:0 for model inputs and add tqdm progress bar for context ingestion

fix: detect embed_tokens device dynamically after model load — resolves device mismatch with multi-GPU device_map=auto

fix: add import gc prior to gc.collect in patched inference_ris_v3.py

feat: switch dual-GPU to Qwen2.5-7B bfloat16 — fixes accelerate CPU-offload bug with NF4 models

refactor(notebook): separate into dedicated RIS-Kernel-Colab.ipynb and RIS-Kernel-Kaggle.ipynb files

refactor(notebook): separate raw Python format for Colab and JSON Notebook format for Kaggle

fix(notebook): set pip install retries=1 and timeout=5 for faster offline detection

feat(notebook): add multi-tier GPU/CPU strategy for Kaggle (Single T4 vs Dual T4) and Colab

feat(notebook): add adaptive CPU/GPU mode, optimize memory for CPU execution, and localize messages to US English

fix(notebook): remove container-restricted swapon calls and lock Qwen2.5-3B-Instruct for Colab

fix(notebook): use dd block allocation for swapfile to resolve swapon Invalid argument Linux error

feat(notebook): update Colab model selection to ultra-fast Qwen2.5-3B-Instruct (4-bit NF4) for 100% CPU RAM safety

feat(notebook): implement adaptive model strategy - Qwen2.5-7B-Instruct for Colab and Qwen2.5-14B-Instruct for Kaggle

feat(notebook): make notebook 100% universal for both Google Colab and Kaggle Notebooks

feat(notebook): integrate Qwen2.5-14B 4-bit NF4 with auto disk cleanup and memory optimizations

fix(transformers): bypass caching_allocator_warmup to prevent 8.94GiB dummy VRAM allocation

fix(ris_attention): calculate mask sum on CPU to eliminate PyTorch CUDA 11.92GiB int64 allocation

fix(quantization): set device_map={: 0} and enable fp32 cpu offload for 4-bit loading

fix(ris_attention): remove 5.96GB VRAM spike in ens_density calculation

feat(engine): add lazy 4-bit NF4 quantization support to load_ris_model

feat(notebook): add dynamic PDF/TXT upload scanning and optimize Colab memory footprint

docs: link Space to arXiv 2607.21927 for Hugging Face Papers

Key words

docs: update index.html and notebook metrics for 3% density and 15 seeds

perf: set optimal 3% density with 15 seeds for zero-OOM GPU T4