README.md

metadata

title: Zero-GPU Side-by-Side LLM Comparison
emoji: 🤖
sdk: gradio
app_file: app.py

Hugging Face Space: Zero-GPU Side-by-Side LLM Comparison

This Space compares four instruction-tuned models side-by-side in a chat interface:

• Gemma4 31B-IT
• Gemma4 26B-IT
• Qwen 3.6 35B-IT
• Qwen 3.6 27B

The UI is designed for zero local GPU usage. It calls Hugging Face hosted inference endpoints via huggingface_hub.InferenceClient, so the Space itself only needs CPU resources.

What is included

• app.py: Gradio app with four parallel chat columns.
• requirements.txt: Runtime dependencies.

Important setup

1. Add your Hugging Face token in Space Secrets:
   • HF_API_TOKEN (or HUGGINGFACE_HUB_TOKEN or HF_TOKEN)
2. Ensure model IDs in MODEL_MATRIX are available through the Hugging Face Inference API.
3. This project is configured to use the fixed best Unsloth checkpoints for each family:
   • unsloth/gemma-4-31B-it-unsloth-bnb-4bit
   • unsloth/gemma-4-26B-A4B-it
   • unsloth/Qwen3.6-35B-A3B-GGUF
   • unsloth/Qwen3.6-27B-GGUF

Optional customization

• Add more precision variants in each model card inside MODEL_MATRIX.
• Change MODEL_MATRIX entries to your preferred checkpoints.
• If your model requires a different prompt format, edit _SYSTEM_PROMPT and _build_messages in app.py.

Notes

• For private or gated models, confirm your token has access.
• If a checkpoint does not support the chat endpoint, the app tries a text-
• ZeroGPU is the zero-a10g hardware flavor. In Spaces config, set this at the Space level (the README hardware: field is not always respected for creation).