README.md

---
title: Pingala-8 2-bit LLMs
emoji: 📉
colorFrom: yellow
colorTo: gray
sdk: static
app_file: index.html
pinned: false
license: apache-2.0
short_description: 2-bit GGUF LLMs for llama.cpp, 85% smaller, no retraining
thumbnail: https://huggingface.co/spaces/Genrius-AI/pingala8/resolve/main/poster.jpg
tags:
- quantization
- 2-bit
- gguf
- llama.cpp
- trellis
- qwen3
- llama-3.1
- local-llm
- llm
- llama
- ai
- model-compression
- low-vram
models:
- Genrius-AI/Qwen3-4B-Pingala8-2bit
- Genrius-AI/Llama-3.1-8B-Pingala8-2bit
---

# Pingala-8: 2-bit LLMs for llama.cpp

Landing page for Pingala-8 by Genrius AI, a 2-bit GGUF format that compresses models you already have, with no retraining.

- Qwen3-4B: 8.05 GB to 1.24 GB (85% smaller). WikiText-2 perplexity 16.64 against 28.61 for llama.cpp IQ2_XXS and 18.15 for IQ2_KT at the same size (fp16: 12.46).
- Quiz average 50.8 against 41.9 for IQ2_XXS (+21%), keeping 90% of the 8-bit original's 56.3.
- Llama-3.1-8B (pre-registered): 8.44 against 9.36 for IQ2_KT.
- Limits: loses to IQ2_KT on Qwen3-14B (9.57 vs 9.40); prompt processing 0.74x of IQ2_XXS; needs our patched llama.cpp (CUDA). A retrained model (Ternary Bonsai 4B, 11.59 at 1.07 GB) scores higher.

This Space is a static page. There is no live chat here because the format needs a CUDA GPU and the patched runtime.

Links:

- One-click build (Windows and Linux): https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick
- Models: https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit and https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit
- Collection: https://huggingface.co/collections/Genrius-AI/pingala-8-2-bit-llms-for-llamacpp-6adcae08d49bbdd8dd446ca9
- Code: https://github.com/Genrius-AI/pingala8-llm
- Paper: https://doi.org/10.5281/zenodo.23481757

Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. Built with Llama.