README.md

metadata

title: BitNet CPU Assistant
emoji: 🧠
colorFrom: blue
colorTo: indigo
sdk: streamlit
sdk_version: 1.53.1
app_file: app.py
pinned: false

🧠 BitNet CPU Assistant

Run Microsoft's 1-bit LLMs at lightning speeds on standard CPUs.

Features

• 1-bit Inference: Powered by bitnet.cpp for maximum efficiency.
• CPU Optimized: No GPU required, perfect for free-tier hosting.
• Low Memory: 3B models run in under 4GB of RAM.

Deployment Instructions

1. Create a new Streamlit Space on Hugging Face.
2. Upload all files from this directory.
3. The Space will automatically install dependencies from packages.txt and requirements.txt.
4. The first run will compile the BitNet kernels (approx 3-5 mins).

Local Development

pip install -r requirements.txt
streamlit run app.py