README.md

metadata

title: Nanbeige2.5 — Chat
emoji: 🦙
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 3.50.0
python_version: '3.10'
app_file: app.py
pinned: false

Nanbeige2.5 — Gradio Chat Space

Lightweight Gradio chat UI for PioTio/Nanbeige2.5 suitable for deployment on Hugging Face Spaces.

Features

• Streaming and non-streaming generation ✅
• Tokenizer ↔ model sanity fixes (avoids SentencePiece piece id errors) ✅
• 4-bit BitsAndBytes load when GPU + bitsandbytes are available ✅
• Optional LoRA adapter application (requires peft) ✅
• Controls: temperature, top-p, top-k, max tokens, max-history ✅

Quick CPU tip: This Space may be CPU-only. Full PioTio/Nanbeige2.5 on CPU is extremely slow — use the Load fast CPU demo (distilgpt2) button for quick responses, enable GPU in Space settings for production use, or check Force CPU generation to run Nanbeige on CPU (very slow and not recommended).

Deployment (Hugging Face Spaces)

1. Create a new Space (Gradio runtime).
2. Upload these files (app.py, requirements.txt, README.md).
3. In Space settings choose Hardware accelerator: GPU (recommended).

After pushing these files the Space will build automatically — open the Space page and monitor logs for errors. If you prefer, you can create the app.py directly in the web UI instead of pushing from Git.

Tip: keep bitsandbytes in requirements.txt if you plan to enable 4-bit loading on GPU; remove or pin it if the build log shows dependency issues.

• If you see piece id is out of range errors the app will attempt to auto-fix tokenizer/model alignment.
• To apply a LoRA adapter after starting the app, paste the adapter HF repo in the LoRA field and click Apply LoRA adapter.

Recommended hardware

• GPU (T4 / A10 / A100) for real-time streaming; CPU-only may be slow for inference.

Troubleshooting

• If model load fails on Spaces, check the logs for memory OOM; switch to GPU or enable bitsandbytes 4-bit.
• For adapter load failures, ensure adapter repo exists and peft is present in requirements.txt.

---