Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,50 @@
+---
+library_name: nexusquant
+tags:
+  - kv-cache
+  - quantization
+  - e8-lattice
+  - llm
+  - inference
+  - compression
+license: mit
+---
+
+# NexusQuant: E8 Lattice KV Cache Compression
+
+Training-free KV cache compression for LLM inference. Uses E8 lattice vector quantization + Hadamard rotation. Calibration-free.
+
+## Headline
+
++0.276% wikitext PPL at 5.83x compression (Mistral-7B). NIAH retrieval preserved through 32K context. Validated on 9 architectures.
+
+## Head-to-head (Llama-3.1-8B-Instruct, 4K, n=30)
+
+| Method | bpe | NIAH |
+|---|---|---|
+| FP16 | 16.0 | 29/30 |
+| TurboQuant 2-bit | 2.125 | 0/30 |
+| NexusQuant K2V2 | 2.0 | **30/30** |
+
+## Install
+
+```
+pip install nexusquant-kv
+```
+
+## Usage
+
+```python
+from nexusquant import compress_kv_cache
+
+with compress_kv_cache(model, mode="quant_only", bits=2):
+    output = model.generate(input_ids, max_new_tokens=200)
+```
+
+## Links
+
+- [Live demo](https://huggingface.co/spaces/jmarquex/nexusquant-demo)
+- [GitHub](https://github.com/jagmarques/nexusquant)
+- [Paper PDF](https://github.com/jagmarques/nexusquant/blob/main/paper/nexusquant.pdf)
+- [llama.cpp PR](https://github.com/ggml-org/llama.cpp/pull/25352)
+- [vLLM PR](https://github.com/vllm-project/vllm/pull/47742)