Update README.md
Browse files
README.md
CHANGED
|
@@ -14,11 +14,20 @@ pipeline_tag: text-generation
|
|
| 14 |
|
| 15 |
# HyperCache AI β Unified KV-Cache & Vector DB Acceleration SDK
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
**Cut LLM KV-Cache VRAM by 58.2% (`+30%` faster decoding) AND compress Vector Database / RAG Embeddings by 57.3% (`2.34x` faster search) with zero model retraining.**
|
| 18 |
|
| 19 |
-
|
|
|
|
|
|
|
| 20 |
|
|
|
|
| 21 |
|
|
|
|
| 22 |
|
| 23 |
HyperCache AI is a dual-engine runtime acceleration SDK for **PyTorch / Hugging Face LLMs** (`Qwen2.5`, `Llama-3`, `Mistral`, `Gemma`) and **Enterprise Vector Databases** (`Pinecone`, `Milvus`, `Qdrant`, `FAISS`, `pgvector`):
|
| 24 |
1. **Engine 1 β LLM KV-Cache Runtime Compressor:** Preserves full-resolution semantic blocks while condensing historical background tokens on the fly, achieving **100% verbatim Needle-in-a-Haystack recall** while cutting GPU KV-Cache memory in half.
|
|
@@ -40,14 +49,14 @@ HyperCache AI is a dual-engine runtime acceleration SDK for **PyTorch / Hugging
|
|
| 40 |
| **3. Standard Sliding Window (`-58.2% RAM`)** | 1,156 | 40.64 MB | 19.1 tok/s | **Failed (0%)** | `The primary vault authorization key is the exact primary vault authorization key...` *(Hallucinates & loops)* |
|
| 41 |
|
| 42 |
#### B) `Qwen/Qwen2.5-7B-Instruct` (Large-Scale VRAM Reduction)
|
| 43 |
-
* **Uncompressed Baseline:** `2,764 tokens` (**`151.16 MB VRAM`**)
|
| 44 |
-
* **HyperCache AI SDK (`-58.2% RAM`):** **`1,156 tokens`** (**`63.22 MB VRAM`**)
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
-
### 2. Vector Database & RAG Embedding Benchmark (`5,000` Docs, `1536-D`
|
| 49 |
|
| 50 |
-
| Vector DB Compression Method (At 42.7% RAM Budget) | Compressed Dim | RAM Saved | Top-1 Match (
|
| 51 |
| :--- | :---: | :---: | :---: | :---: | :---: |
|
| 52 |
| **1. Standard Dimension Truncation** | 656-D | 57.29% | 99.0% | 22.90% | 66.28% |
|
| 53 |
| **2. HyperCache Vector DB Engine (Ours)** | **656-D** | **57.29%** | **100.0%** | **95.20%** | **99.92%** |
|
|
@@ -56,7 +65,7 @@ HyperCache AI is a dual-engine runtime acceleration SDK for **PyTorch / Hugging
|
|
| 56 |
|
| 57 |
### 3. Multi-Head Long-Context Attention Fidelity (`N = 4,096` Tokens, 16 Heads)
|
| 58 |
|
| 59 |
-
| Compression Method (At Identical 42.5% RAM Budget) | Output Cosine Similarity (
|
| 60 |
| :--- | :---: | :---: | :---: | :---: |
|
| 61 |
| **1. Standard Sliding Window** | 66.73% | 115.84% | 40.0% | 2.35x |
|
| 62 |
| **2. Uniform Strided Subsampling** | 63.47% | 114.45% | 20.0% | 2.35x |
|
|
@@ -64,26 +73,20 @@ HyperCache AI is a dual-engine runtime acceleration SDK for **PyTorch / Hugging
|
|
| 64 |
|
| 65 |
---
|
| 66 |
|
| 67 |
-
## Instant 30-Second Evaluation (Free Trial Key Included)
|
| 68 |
|
| 69 |
-
You can evaluate both engines right now on your own machine or in Google Colab using our public evaluation key:
|
| 70 |
* **Public Evaluation Key:** `HYPERCACHE-FREE-TRIAL-2026`
|
| 71 |
*(Hardware-locked to **15 free evaluation compressions** per machine, up to **4,096 tokens/dimensions** per call).*
|
| 72 |
|
| 73 |
-
### Step 1:
|
| 74 |
-
|
| 75 |
|
| 76 |
```bash
|
| 77 |
-
pip install
|
| 78 |
```
|
| 79 |
|
| 80 |
-
### Step 2:
|
| 81 |
-
Download `hypercache_sdk_client.py` and `quickstart_demo.py` into the same directory and run:
|
| 82 |
-
```bash
|
| 83 |
-
python3 quickstart_demo.py
|
| 84 |
-
```
|
| 85 |
-
|
| 86 |
-
### Step 3: Use Both Engines in Python (3 Lines of Code)
|
| 87 |
```python
|
| 88 |
import hypercache_sdk_client
|
| 89 |
|
|
@@ -102,22 +105,27 @@ compressed_docs = engine.fit_compress_embeddings(doc_embeddings_1536d)
|
|
| 102 |
compressed_queries = engine.compress_query_embeddings(query_embeddings_1536d)
|
| 103 |
```
|
| 104 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
---
|
| 106 |
|
| 107 |
## Annual Licensing, Early-Bird Pricing & Seat Management
|
| 108 |
|
| 109 |
-
HyperCache AI is distributed as an **AES-256-GCM authenticated commercial vault** (`
|
| 110 |
|
| 111 |
| Feature | Free Evaluation Trial | Developer / Startup PRO | Enterprise Cluster PRO | Enterprise Unlimited (K8s) |
|
| 112 |
| :--- | :---: | :---: | :---: | :---: |
|
| 113 |
| **Annual Price (Early-Bird Launch)** | **Free** | ~~$199/yr~~ **$99 / year** | ~~$999/yr~~ **$399 / year** | ~~$2,999/yr~~ **$1,499 / year** |
|
| 114 |
-
| **Purchase Instant License Key** | `HYPERCACHE-FREE-TRIAL-2026` | **[Buy Developer PRO (
|
| 115 |
| **Active Machine Seats** | 1 Machine (Non-Transferable) | **2 Transferable Seats** *(Dev Laptop + 1 Prod Server)* | **10 Transferable Server Seats** | **Unlimited Nodes / Containers** *(Kubernetes Auto-Scaling)* |
|
| 116 |
| **Machine Transfer (`deactivate`)** | No | **Yes (`deactivate_current_machine()`)** | **Yes (`deactivate_current_machine()`)** | **Automatic (Zero Seat Lock)** |
|
| 117 |
| **Total Runs Allowed** | **15 Runs** (Hardware-Locked) | **Unlimited (24/7 Production)** | **Unlimited (24/7 Production)** | **Unlimited (24/7 Production)** |
|
| 118 |
| **Max Context / Vector DB Size** | Up to **4,096 Tokens / Dims** | **Unlimited (32kβ1M+ Tokens)** | **Unlimited (32kβ1M+ Tokens)** | **Unlimited (Cluster-Wide)** |
|
| 119 |
|
| 120 |
-
π **[Click Here to Purchase Your Instant Annual PRO License Key on Gumroad](https://
|
| 121 |
|
| 122 |
### Moving Your PRO License Between Machines
|
| 123 |
Both **Developer PRO (2 Seats)** and **Enterprise Cluster PRO (10 Seats)** allow you to transfer active seats between workstations and cloud servers at any time with a single line of Python:
|
|
|
|
| 14 |
|
| 15 |
# HyperCache AI β Unified KV-Cache & Vector DB Acceleration SDK
|
| 16 |
|
| 17 |
+
[](https://pypi.org/project/hypercache-ai/)
|
| 18 |
+
[](https://pypi.org/project/hypercache-ai/)
|
| 19 |
+
[](https://huggingface.co/Walid-Kotb-1/HyperCache-AI)
|
| 20 |
+
[](https://kotbster7.gumroad.com/l/hypercache-ai)
|
| 21 |
+
|
| 22 |
**Cut LLM KV-Cache VRAM by 58.2% (`+30%` faster decoding) AND compress Vector Database / RAG Embeddings by 57.3% (`2.34x` faster search) with zero model retraining.**
|
| 23 |
|
| 24 |
+
```bash
|
| 25 |
+
pip install hypercache-ai
|
| 26 |
+
```
|
| 27 |
|
| 28 |
+
π **[Get Your Instant Annual PRO License Key on Gumroad ($99/yr Launch Offer)](https://kotbster7.gumroad.com/l/hypercache-ai)**
|
| 29 |
|
| 30 |
+

|
| 31 |
|
| 32 |
HyperCache AI is a dual-engine runtime acceleration SDK for **PyTorch / Hugging Face LLMs** (`Qwen2.5`, `Llama-3`, `Mistral`, `Gemma`) and **Enterprise Vector Databases** (`Pinecone`, `Milvus`, `Qdrant`, `FAISS`, `pgvector`):
|
| 33 |
1. **Engine 1 β LLM KV-Cache Runtime Compressor:** Preserves full-resolution semantic blocks while condensing historical background tokens on the fly, achieving **100% verbatim Needle-in-a-Haystack recall** while cutting GPU KV-Cache memory in half.
|
|
|
|
| 49 |
| **3. Standard Sliding Window (`-58.2% RAM`)** | 1,156 | 40.64 MB | 19.1 tok/s | **Failed (0%)** | `The primary vault authorization key is the exact primary vault authorization key...` *(Hallucinates & loops)* |
|
| 50 |
|
| 51 |
#### B) `Qwen/Qwen2.5-7B-Instruct` (Large-Scale VRAM Reduction)
|
| 52 |
+
* **Uncompressed Baseline:** `2,764 tokens` (**`151.16 MB VRAM`**) β Retrieves `'HC-ENTERPRISE-9940-X'` and `90.37 million USD`.
|
| 53 |
+
* **HyperCache AI SDK (`-58.2% RAM`):** **`1,156 tokens`** (**`63.22 MB VRAM`**) β Retrieves the **exact same verbatim output** while saving **`87.94 MB VRAM`** per single request!
|
| 54 |
|
| 55 |
---
|
| 56 |
|
| 57 |
+
### 2. Vector Database & RAG Embedding Benchmark (`5,000` Docs, `1536-D` β `656-D`)
|
| 58 |
|
| 59 |
+
| Vector DB Compression Method (At 42.7% RAM Budget) | Compressed Dim | RAM Saved | Top-1 Match (β) | Recall@10 (β) | Cosine Score Correlation (β) |
|
| 60 |
| :--- | :---: | :---: | :---: | :---: | :---: |
|
| 61 |
| **1. Standard Dimension Truncation** | 656-D | 57.29% | 99.0% | 22.90% | 66.28% |
|
| 62 |
| **2. HyperCache Vector DB Engine (Ours)** | **656-D** | **57.29%** | **100.0%** | **95.20%** | **99.92%** |
|
|
|
|
| 65 |
|
| 66 |
### 3. Multi-Head Long-Context Attention Fidelity (`N = 4,096` Tokens, 16 Heads)
|
| 67 |
|
| 68 |
+
| Compression Method (At Identical 42.5% RAM Budget) | Output Cosine Similarity (β) | Relative L2 Error (β) | Fact Recall (β) | Theoretical Speedup |
|
| 69 |
| :--- | :---: | :---: | :---: | :---: |
|
| 70 |
| **1. Standard Sliding Window** | 66.73% | 115.84% | 40.0% | 2.35x |
|
| 71 |
| **2. Uniform Strided Subsampling** | 63.47% | 114.45% | 20.0% | 2.35x |
|
|
|
|
| 73 |
|
| 74 |
---
|
| 75 |
|
| 76 |
+
## Instant 30-Second Installation & Evaluation (Free Trial Key Included)
|
| 77 |
|
| 78 |
+
You can install HyperCache AI directly from **Official PyPI** and evaluate both engines right now on your own machine or in Google Colab using our public evaluation key:
|
| 79 |
* **Public Evaluation Key:** `HYPERCACHE-FREE-TRIAL-2026`
|
| 80 |
*(Hardware-locked to **15 free evaluation compressions** per machine, up to **4,096 tokens/dimensions** per call).*
|
| 81 |
|
| 82 |
+
### Step 1: One-Line Installation via `pip`
|
| 83 |
+
**Universal Python Version Support (3.8 β 3.13+):** `hypercache-ai` decrypts and compiles the runtime engine in volatile RAM directly on your target Python interpreter, ensuring seamless compatibility across **Python 3.8 through 3.13+** (including Google Colab, Kaggle, Linux, macOS, and Windows).
|
| 84 |
|
| 85 |
```bash
|
| 86 |
+
pip install hypercache-ai torch transformers accelerate
|
| 87 |
```
|
| 88 |
|
| 89 |
+
### Step 2: Use Both Engines in Python (3 Lines of Code)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
```python
|
| 91 |
import hypercache_sdk_client
|
| 92 |
|
|
|
|
| 105 |
compressed_queries = engine.compress_query_embeddings(query_embeddings_1536d)
|
| 106 |
```
|
| 107 |
|
| 108 |
+
### Step 3: Or Run the Full Benchmark Suite
|
| 109 |
+
```bash
|
| 110 |
+
python quickstart_demo.py
|
| 111 |
+
```
|
| 112 |
+
|
| 113 |
---
|
| 114 |
|
| 115 |
## Annual Licensing, Early-Bird Pricing & Seat Management
|
| 116 |
|
| 117 |
+
HyperCache AI is distributed as an **AES-256-GCM authenticated commercial vault** (`hypercache-ai`) that decrypts strictly in volatile RAM upon license verification.
|
| 118 |
|
| 119 |
| Feature | Free Evaluation Trial | Developer / Startup PRO | Enterprise Cluster PRO | Enterprise Unlimited (K8s) |
|
| 120 |
| :--- | :---: | :---: | :---: | :---: |
|
| 121 |
| **Annual Price (Early-Bird Launch)** | **Free** | ~~$199/yr~~ **$99 / year** | ~~$999/yr~~ **$399 / year** | ~~$2,999/yr~~ **$1,499 / year** |
|
| 122 |
+
| **Purchase Instant License Key** | `HYPERCACHE-FREE-TRIAL-2026` | **[Buy Developer PRO ($99/yr)](https://kotbster7.gumroad.com/l/hypercache-ai)** | **[Buy Cluster PRO ($399/yr)](https://kotbster7.gumroad.com/l/hypercache-ai)** | **[Buy Unlimited K8s ($1,499/yr)](https://kotbster7.gumroad.com/l/hypercache-ai)** |
|
| 123 |
| **Active Machine Seats** | 1 Machine (Non-Transferable) | **2 Transferable Seats** *(Dev Laptop + 1 Prod Server)* | **10 Transferable Server Seats** | **Unlimited Nodes / Containers** *(Kubernetes Auto-Scaling)* |
|
| 124 |
| **Machine Transfer (`deactivate`)** | No | **Yes (`deactivate_current_machine()`)** | **Yes (`deactivate_current_machine()`)** | **Automatic (Zero Seat Lock)** |
|
| 125 |
| **Total Runs Allowed** | **15 Runs** (Hardware-Locked) | **Unlimited (24/7 Production)** | **Unlimited (24/7 Production)** | **Unlimited (24/7 Production)** |
|
| 126 |
| **Max Context / Vector DB Size** | Up to **4,096 Tokens / Dims** | **Unlimited (32kβ1M+ Tokens)** | **Unlimited (32kβ1M+ Tokens)** | **Unlimited (Cluster-Wide)** |
|
| 127 |
|
| 128 |
+
π **[Click Here to Purchase Your Instant Annual PRO License Key on Gumroad](https://kotbster7.gumroad.com/l/hypercache-ai)**
|
| 129 |
|
| 130 |
### Moving Your PRO License Between Machines
|
| 131 |
Both **Developer PRO (2 Seats)** and **Enterprise Cluster PRO (10 Seats)** allow you to transfer active seats between workstations and cloud servers at any time with a single line of Python:
|