Spaces:

Genrius-AI
/

pingala8

Running

App Files Files Community

pingala8 / README.md

harshit-genrius

Add discoverability tags

f3329c9 verified about 17 hours ago

preview code

Raw History Blame Contribute Delete

1.94 kB

metadata

title: Pingala-8 2-bit LLMs
emoji: 📉
colorFrom: yellow
colorTo: gray
sdk: static
app_file: index.html
pinned: false
license: apache-2.0
short_description: 2-bit GGUF LLMs for llama.cpp, 85% smaller, no retraining
thumbnail: https://huggingface.co/spaces/Genrius-AI/pingala8/resolve/main/poster.jpg
tags:
  - quantization
  - 2-bit
  - gguf
  - llama.cpp
  - trellis
  - qwen3
  - llama-3.1
  - local-llm
  - llm
  - llama
  - ai
  - model-compression
  - low-vram
models:
  - Genrius-AI/Qwen3-4B-Pingala8-2bit
  - Genrius-AI/Llama-3.1-8B-Pingala8-2bit

Pingala-8: 2-bit LLMs for llama.cpp

Landing page for Pingala-8 by Genrius AI, a 2-bit GGUF format that compresses models you already have, with no retraining.

Qwen3-4B: 8.05 GB to 1.24 GB (85% smaller). WikiText-2 perplexity 16.64 against 28.61 for llama.cpp IQ2_XXS and 18.15 for IQ2_KT at the same size (fp16: 12.46).
Quiz average 50.8 against 41.9 for IQ2_XXS (+21%), keeping 90% of the 8-bit original's 56.3.
Llama-3.1-8B (pre-registered): 8.44 against 9.36 for IQ2_KT.
Limits: loses to IQ2_KT on Qwen3-14B (9.57 vs 9.40); prompt processing 0.74x of IQ2_XXS; needs our patched llama.cpp (CUDA). A retrained model (Ternary Bonsai 4B, 11.59 at 1.07 GB) scores higher.

This Space is a static page. There is no live chat here because the format needs a CUDA GPU and the patched runtime.

Links:

One-click build (Windows and Linux): https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick
Models: https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit and https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit
Collection: https://huggingface.co/collections/Genrius-AI/pingala-8-2-bit-llms-for-llamacpp-6adcae08d49bbdd8dd446ca9
Code: https://github.com/Genrius-AI/pingala8-llm
Paper: https://doi.org/10.5281/zenodo.23481757