README.md

metadata

title: README
colorFrom: yellow
colorTo: gray
sdk: static
pinned: false

Genrius AI

Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.

Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.

Pingala-8: 2-bit LLMs for llama.cpp, no retraining

• 85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
• Better at the same size: at the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
• Zero-shot (Qwen3-4B, 5-task average): we score 50.83, against 41.86 for IQ2_XXS. The 8-bit original scores 56.3.
• Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.

• Models: Qwen3-4B-Pingala8-2bit and Llama-3.1-8B-Pingala8-2bit
• Benchmarks: pingala8-benchmarks
• One-click Windows build: v2.4-oneclick
• Code: Genrius-AI/pingala8-llm
• Paper: doi.org/10.5281/zenodo.23481757