Genrius AI
Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.
Try it: Will it fit your GPU? (calculator) | One-click build, Windows and Linux, NVIDIA RTX 30/40/50
| Model | Size | WikiText-2 PPL, ours | IQ2_XXS, same size |
|---|---|---|---|
| Qwen3-4B-Pingala8-2bit | 1.24 GB | 16.64 | 28.61 |
| Llama-3.1-8B-Pingala8-2bit | 2.39 GB | 9.14 | 12.35 |
| Qwen3.5-4B-Pingala8-2bit | 1.51 GB | 10.97 | 14.75 |
Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them.
Pingala-8: 2-bit LLMs for llama.cpp, no retraining
- 85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
- At the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
- Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.
