| --- |
| library_name: pytorch |
| license: other |
| tags: |
| - kv-cache |
| - llm |
| - rag |
| - vector-database |
| - model-compression |
| - qwen2 |
| - llama-3 |
| pipeline_tag: text-generation |
| --- |
| |
| # HyperCache AI — Unified KV-Cache & Vector DB Acceleration SDK |
|
|
| [](https://pypi.org/project/hypercache-ai/) |
| [](https://pypi.org/project/hypercache-ai/) |
| [](https://huggingface.co/Walid-Kotb-1/HyperCache-AI) |
| [](https://kotbster7.gumroad.com/l/hypercache-ai) |
|
|
| **Cut LLM KV-Cache VRAM by 58.2% (`+30%` faster decoding) AND compress Vector Database / RAG Embeddings by 57.3% (`2.34x` faster search) with zero model retraining.** |
|
|
| ```bash |
| pip install hypercache-ai |
| ``` |
|
|
| 👉 **[Get Your Instant Annual PRO License Key on Gumroad ($99/yr Launch Offer)](https://kotbster7.gumroad.com/l/hypercache-ai)** |
|
|
|  |
|
|
| HyperCache AI is a dual-engine runtime acceleration SDK for **PyTorch / Hugging Face LLMs** (`Qwen2.5`, `Llama-3`, `Mistral`, `Gemma`) and **Enterprise Vector Databases** (`Pinecone`, `Milvus`, `Qdrant`, `FAISS`, `pgvector`): |
| 1. **Engine 1 — LLM KV-Cache Runtime Compressor:** Preserves full-resolution semantic blocks while condensing historical background tokens on the fly, achieving **100% verbatim Needle-in-a-Haystack recall** while cutting GPU KV-Cache memory in half. |
| 2. **Engine 2 — Vector DB & RAG Embedding Compressor:** Compresses high-dimensional embedding vectors (e.g., `1536-D` OpenAI / Cohere / BGE embeddings down to `656-D`), cutting Vector Database RAM bills by **57.3%** while preserving **100.0% Top-1 match accuracy** and **99.92% cosine similarity correlation**. |
|
|
| --- |
|
|
| ## Verified Production Benchmarks |
|
|
| ### 1. Live 3B & 7B LLM Needle-in-a-Haystack Retrieval (NVIDIA Tesla T4 — FP16) |
| *Task: Retrieve a confidential vault authorization key (`'HC-ENTERPRISE-9940-X'`) and reserve liquidity threshold (`90.37 million USD`) buried at token ~160 inside a 2,765-token cloud infrastructure report.* |
|
|
| #### A) `Qwen/Qwen2.5-3B-Instruct` (100% Native GPU Resident on Tesla T4) |
|
|
| | Compression Method | Cached Tokens | KV-Cache VRAM | Decoding Speed | Exact Answer Retrieved? | Actual Model Output | |
| | :--- | :---: | :---: | :---: | :---: | :--- | |
| | **1. Uncompressed Baseline (100% RAM)** | 2,764 | 97.17 MB | 15.2 tok/s | **Yes (100%)** | `The primary vault authorization key is 'HC-ENTERPRISE-9940-X' and the reserve liquidity threshold is 90.37 million USD` | |
| | **2. HyperCache AI SDK (`-58.2% RAM`)** | **1,156** | **40.64 MB** | **19.7 tok/s (+30%)** | **Yes (100%)** | `The primary vault authorization key is 'HC-ENTERPRISE-9940-X' and the reserve liquidity threshold is 90.37 million USD` | |
| | **3. Standard Sliding Window (`-58.2% RAM`)** | 1,156 | 40.64 MB | 19.1 tok/s | **Failed (0%)** | `The primary vault authorization key is the exact primary vault authorization key...` *(Hallucinates & loops)* | |
|
|
| #### B) `Qwen/Qwen2.5-7B-Instruct` (Large-Scale VRAM Reduction) |
| * **Uncompressed Baseline:** `2,764 tokens` (**`151.16 MB VRAM`**) → Retrieves `'HC-ENTERPRISE-9940-X'` and `90.37 million USD`. |
| * **HyperCache AI SDK (`-58.2% RAM`):** **`1,156 tokens`** (**`63.22 MB VRAM`**) → Retrieves the **exact same verbatim output** while saving **`87.94 MB VRAM`** per single request! |
|
|
| --- |
|
|
| ### 2. Vector Database & RAG Embedding Benchmark (`5,000` Docs, `1536-D` → `656-D`) |
|
|
| | Vector DB Compression Method (At 42.7% RAM Budget) | Compressed Dim | RAM Saved | Top-1 Match (↑) | Recall@10 (↑) | Cosine Score Correlation (↑) | |
| | :--- | :---: | :---: | :---: | :---: | :---: | |
| | **1. Standard Dimension Truncation** | 656-D | 57.29% | 99.0% | 22.90% | 66.28% | |
| | **2. HyperCache Vector DB Engine (Ours)** | **656-D** | **57.29%** | **100.0%** | **95.20%** | **99.92%** | |
|
|
| --- |
|
|
| ### 3. Multi-Head Long-Context Attention Fidelity (`N = 4,096` Tokens, 16 Heads) |
|
|
| | Compression Method (At Identical 42.5% RAM Budget) | Output Cosine Similarity (↑) | Relative L2 Error (↓) | Fact Recall (↑) | Theoretical Speedup | |
| | :--- | :---: | :---: | :---: | :---: | |
| | **1. Standard Sliding Window** | 66.73% | 115.84% | 40.0% | 2.35x | |
| | **2. Uniform Strided Subsampling** | 63.47% | 114.45% | 20.0% | 2.35x | |
| | **3. HyperCache AI Engine (Ours)** | **99.96%** | **2.93%** | **100.0%** | **2.35x** | |
|
|
| --- |
|
|
| ## Instant 30-Second Installation & Evaluation (Free Trial Key Included) |
|
|
| You can install HyperCache AI directly from **Official PyPI** and evaluate both engines right now on your own machine or in Google Colab using our public evaluation key: |
| * **Public Evaluation Key:** `HYPERCACHE-FREE-TRIAL-2026` |
| *(Hardware-locked to **15 free evaluation compressions** per machine, up to **4,096 tokens/dimensions** per call).* |
|
|
| ### Step 1: One-Line Installation via `pip` |
| **Universal Python Version Support (3.8 – 3.13+):** `hypercache-ai` decrypts and compiles the runtime engine in volatile RAM directly on your target Python interpreter, ensuring seamless compatibility across **Python 3.8 through 3.13+** (including Google Colab, Kaggle, Linux, macOS, and Windows). |
|
|
| ```bash |
| pip install hypercache-ai torch transformers accelerate |
| ``` |
|
|
| ### Step 2: Use Both Engines in Python (3 Lines of Code) |
| ```python |
| import hypercache_sdk_client |
| |
| # 1. Unlock the AES-256-GCM dual engine in RAM |
| engine = hypercache_sdk_client.load_hypercache_engine( |
| license_key="HYPERCACHE-FREE-TRIAL-2026" |
| ) |
| |
| # 2A. Compress HuggingFace LLM KV-Cache on the fly (58.2% VRAM saved) |
| compressed_kv = engine.compress_torch_cache( |
| past_key_values, _inp_ids=input_ids, _mdl=model, _q_len=48 |
| ) |
| |
| # 2B. Compress Vector DB / RAG Document & Query Embeddings (1536-D -> 656-D, 57.3% RAM saved) |
| compressed_docs = engine.fit_compress_embeddings(doc_embeddings_1536d) |
| compressed_queries = engine.compress_query_embeddings(query_embeddings_1536d) |
| ``` |
|
|
| ### Step 3: Or Run the Full Benchmark Suite |
| ```bash |
| python quickstart_demo.py |
| ``` |
|
|
| --- |
|
|
| ## Annual Licensing, Early-Bird Pricing & Seat Management |
|
|
| HyperCache AI is distributed as an **AES-256-GCM authenticated commercial vault** (`hypercache-ai`) that decrypts strictly in volatile RAM upon license verification. |
|
|
| | Feature | Free Evaluation Trial | Developer / Startup PRO | Enterprise Cluster PRO | Enterprise Unlimited (K8s) | |
| | :--- | :---: | :---: | :---: | :---: | |
| | **Annual Price (Early-Bird Launch)** | **Free** | ~~$199/yr~~ **$99 / year** | ~~$999/yr~~ **$399 / year** | ~~$2,999/yr~~ **$1,499 / year** | |
| | **Purchase Instant License Key** | `HYPERCACHE-FREE-TRIAL-2026` | **[Buy Developer PRO ($99/yr)](https://kotbster7.gumroad.com/l/hypercache-ai)** | **[Buy Cluster PRO ($399/yr)](https://kotbster7.gumroad.com/l/hypercache-ai)** | **[Buy Unlimited K8s ($1,499/yr)](https://kotbster7.gumroad.com/l/hypercache-ai)** | |
| | **Active Machine Seats** | 1 Machine (Non-Transferable) | **2 Transferable Seats** *(Dev Laptop + 1 Prod Server)* | **10 Transferable Server Seats** | **Unlimited Nodes / Containers** *(Kubernetes Auto-Scaling)* | |
| | **Machine Transfer (`deactivate`)** | No | **Yes (`deactivate_current_machine()`)** | **Yes (`deactivate_current_machine()`)** | **Automatic (Zero Seat Lock)** | |
| | **Total Runs Allowed** | **15 Runs** (Hardware-Locked) | **Unlimited (24/7 Production)** | **Unlimited (24/7 Production)** | **Unlimited (24/7 Production)** | |
| | **Max Context / Vector DB Size** | Up to **4,096 Tokens / Dims** | **Unlimited (32k–1M+ Tokens)** | **Unlimited (32k–1M+ Tokens)** | **Unlimited (Cluster-Wide)** | |
|
|
| 👉 **[Click Here to Purchase Your Instant Annual PRO License Key on Gumroad](https://kotbster7.gumroad.com/l/hypercache-ai)** |
|
|
| ### Moving Your PRO License Between Machines |
| Both **Developer PRO (2 Seats)** and **Enterprise Cluster PRO (10 Seats)** allow you to transfer active seats between workstations and cloud servers at any time with a single line of Python: |
| ```python |
| # Frees the current machine seat immediately so you can activate on a new server: |
| print(engine.deactivate_current_machine()) |
| ``` |
| *(For dynamic Kubernetes / Docker auto-scaling environments where pods spin up and down continuously, choose **Enterprise Unlimited (K8s)** for zero seat-tracking overhead).* |
