Spaces:
Running
Running
| title: README | |
| colorFrom: yellow | |
| colorTo: gray | |
| sdk: static | |
| pinned: false | |
| # Genrius AI | |
| **Generating Real Intelligence.** We build AI models and make them small and fast enough to run on hardware people already own. | |
| Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus. | |
| ## Pingala-8: 2-bit LLMs for llama.cpp, no retraining | |
| - **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB. | |
| - **Better at the same size:** at the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better). | |
| - **Zero-shot (Qwen3-4B, 5-task average):** we score 50.83, against 41.86 for IQ2_XXS. The 8-bit original scores 56.3. | |
| - **Where we lose:** Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B. | |
|  | |
| - **Models:** [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) and [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | |
| - **Benchmarks:** [pingala8-benchmarks](https://huggingface.co/datasets/Genrius-AI/pingala8-benchmarks) | |
| - **One-click Windows build:** [v2.4-oneclick](https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick) | |
| - **Code:** [Genrius-AI/pingala8-llm](https://github.com/Genrius-AI/pingala8-llm) | |
| - **Paper:** [doi.org/10.5281/zenodo.23481757](https://doi.org/10.5281/zenodo.23481757) | |