Spaces:
Running
Running
File size: 2,605 Bytes
683efc3 eff7fd9 b9f9541 eff7fd9 b9f9541 eff7fd9 683efc3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | <!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Genrius AI</title>
<style>
body { margin: 0; background: #0b0b0c; color: #e8e6e1; font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif; line-height: 1.55; }
main { max-width: 760px; margin: 0 auto; padding: 40px 16px; }
h1 { color: #D4AF37; margin: 0 0 4px; font-size: 2rem; }
.tag { color: #b9b4a8; margin: 0 0 28px; }
h2 { color: #D4AF37; font-size: 1.15rem; margin-top: 28px; }
a { color: #D4AF37; }
img { width: 100%; height: auto; border-radius: 8px; margin: 16px 0; }
li { margin: 6px 0; }
</style>
</head>
<body>
<main>
<h1>Genrius AI</h1>
<p class="tag">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
<h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
<ul>
<li>85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.</li>
<li>At the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).</li>
<li>Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.</li>
</ul>
<img src="pingala8-poster.jpg" alt="Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU">
<h2>Links</h2>
<ul>
<li><a href="https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick" target="_blank" rel="noopener noreferrer">One-click Windows build</a></li>
<li><a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer">Code</a></li>
<li><a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">Paper</a></li>
<li><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B-Pingala8-2bit</a> and <a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B-Pingala8-2bit</a></li>
<li><a href="https://huggingface.co/datasets/Genrius-AI/pingala8-benchmarks" target="_blank" rel="noopener noreferrer">Benchmark tables</a></li>
</ul>
</main>
<script>document.addEventListener("click",function(e){var a=e.target.closest&&e.target.closest('a[href^="http"]');if(a){a.target="_blank";a.rel="noopener noreferrer";}},true);</script>
</body>
</html>
|