Spaces:
Running
Running
| title: Pingala-8 2-bit LLMs | |
| emoji: 📉 | |
| colorFrom: yellow | |
| colorTo: gray | |
| sdk: static | |
| app_file: index.html | |
| pinned: false | |
| license: apache-2.0 | |
| short_description: 2-bit GGUF LLMs for llama.cpp, 85% smaller, no retraining | |
| thumbnail: https://huggingface.co/spaces/Genrius-AI/pingala8/resolve/main/poster.jpg | |
| tags: | |
| - quantization | |
| - 2-bit | |
| - gguf | |
| - llama.cpp | |
| - trellis | |
| - qwen3 | |
| - llama-3.1 | |
| - local-llm | |
| - llm | |
| - llama | |
| - ai | |
| - model-compression | |
| - low-vram | |
| models: | |
| - Genrius-AI/Qwen3-4B-Pingala8-2bit | |
| - Genrius-AI/Llama-3.1-8B-Pingala8-2bit | |
| # Pingala-8: 2-bit LLMs for llama.cpp | |
| Landing page for Pingala-8 by Genrius AI, a 2-bit GGUF format that compresses models you already have, with no retraining. | |
| - Qwen3-4B: 8.05 GB to 1.24 GB (85% smaller). WikiText-2 perplexity 16.64 against 28.61 for llama.cpp IQ2_XXS and 18.15 for IQ2_KT at the same size (fp16: 12.46). | |
| - Quiz average 50.8 against 41.9 for IQ2_XXS (+21%), keeping 90% of the 8-bit original's 56.3. | |
| - Llama-3.1-8B (pre-registered): 8.44 against 9.36 for IQ2_KT. | |
| - Limits: loses to IQ2_KT on Qwen3-14B (9.57 vs 9.40); prompt processing 0.74x of IQ2_XXS; needs our patched llama.cpp (CUDA). A retrained model (Ternary Bonsai 4B, 11.59 at 1.07 GB) scores higher. | |
| This Space is a static page. There is no live chat here because the format needs a CUDA GPU and the patched runtime. | |
| Links: | |
| - One-click build (Windows and Linux): https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick | |
| - Models: https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit and https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit | |
| - Collection: https://huggingface.co/collections/Genrius-AI/pingala-8-2-bit-llms-for-llamacpp-6adcae08d49bbdd8dd446ca9 | |
| - Code: https://github.com/Genrius-AI/pingala8-llm | |
| - Paper: https://doi.org/10.5281/zenodo.23481757 | |
| Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. Built with Llama. | |