---
title: Nanbeige2.5 — Chat
emoji: 🦙
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: "3.50.0"
python_version: "3.10"
app_file: app.py
pinned: false
---

# Nanbeige2.5 — Gradio Chat Space

Lightweight Gradio chat UI for `PioTio/Nanbeige2.5` suitable for deployment on Hugging Face Spaces.

## Features
- Streaming and non-streaming generation ✅
- Tokenizer ↔ model sanity fixes (avoids SentencePiece `piece id` errors) ✅
- 4-bit BitsAndBytes load when GPU + `bitsandbytes` are available ✅
- Optional LoRA adapter application (requires `peft`) ✅
- Controls: temperature, top-p, top-k, max tokens, max-history ✅

**Quick CPU tip:** This Space **may be CPU-only**. Full `PioTio/Nanbeige2.5` on CPU is extremely slow — use the **Load fast CPU demo (distilgpt2)** button for quick responses, enable GPU in Space settings for production use, or check **Force CPU generation** to run Nanbeige on CPU (very slow and not recommended).

## Deployment (Hugging Face Spaces)
1. Create a new Space (Gradio runtime).
2. Upload these files (`app.py`, `requirements.txt`, `README.md`).
3. In Space settings choose **Hardware accelerator: GPU** (recommended).

After pushing these files the Space will build automatically — open the Space page and monitor logs for errors. If you prefer, you can create the `app.py` directly in the web UI instead of pushing from Git.

**Tip:** keep `bitsandbytes` in `requirements.txt` if you plan to enable 4-bit loading on GPU; remove or pin it if the build log shows dependency issues.
- If you see `piece id is out of range` errors the app will attempt to auto-fix tokenizer/model alignment.
- To apply a LoRA adapter after starting the app, paste the adapter HF repo in the LoRA field and click **Apply LoRA adapter**.

## Recommended hardware
- GPU (T4 / A10 / A100) for real-time streaming; CPU-only may be slow for inference.

## Troubleshooting
- If model load fails on Spaces, check the logs for memory OOM; switch to GPU or enable `bitsandbytes` 4-bit.
- For adapter load failures, ensure adapter repo exists and `peft` is present in `requirements.txt`.

---