Spaces:
Sleeping
Sleeping
| title: LLM Compare | |
| emoji: 🔍 | |
| colorFrom: blue | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: "6.9.0" | |
| app_file: app.py | |
| pinned: false | |
| # LLM Compare | |
| A Gradio web app for side-by-side LLM comparison. Compare your Dify application against reference models from OpenAI, Anthropic, Google Gemini, Qwen, and Yi. | |
| ## Features | |
| - **Two-column layout**: Your Dify app on the left, a selectable reference model on the right | |
| - **Multiple providers**: OpenAI (GPT-4o), Anthropic (Claude), Google Gemini, Qwen, Yi | |
| - **Overridable defaults**: Base URL and Model ID auto-fill from env vars but can be edited per-session | |
| - **Evaluation workflow**: Comment and grade (1–10) each model's response | |
| - **Nickname tracking**: All evaluations tagged with user nickname | |
| - **Excel export**: Download all evaluation data as `.xlsx` | |
| ## Setup | |
| ```bash | |
| pip install -r requirements.txt | |
| python app.py | |
| ``` | |
| ## Environment Variables | |
| Set these as **Hugging Face Spaces secrets** (Settings → Repository secrets) to provide defaults. | |
| Users can override Base URL / Model ID in the UI at runtime. | |
| ### API Keys (required for each provider you use) | |
| | Variable | Provider | | |
| |---|---| | |
| | `OPENAI_API_KEY` | OpenAI | | |
| | `ANTHROPIC_API_KEY` | Anthropic | | |
| | `GOOGLE_API_KEY` | Google Gemini | | |
| | `DASHSCOPE_API_KEY` | Qwen (DashScope / Alibaba) | | |
| | `YI_API_KEY` | Yi (01.AI) | | |
| ### Base URL overrides (optional) | |
| Override the default API endpoint for each provider. Useful for proxies or custom deployments. | |
| | Variable | Default | | |
| |---|---| | |
| | `OPENAI_BASE_URL` | *(uses OpenAI SDK default)* | | |
| | `ANTHROPIC_BASE_URL` | *(uses Anthropic SDK default)* | | |
| | `GOOGLE_BASE_URL` | *(uses Google GenAI SDK default)* | | |
| | `DASHSCOPE_BASE_URL` | `https://dashscope.aliyuncs.com/compatible-mode/v1` | | |
| | `YI_BASE_URL` | `https://api.01.ai/v1` | | |
| ### Model ID overrides (optional) | |
| Override the default model ID. Useful for switching to newer model versions without code changes. | |
| | Variable | Default | | |
| |---|---| | |
| | `OPENAI_MODEL_ID` | `gpt-4o` | | |
| | `OPENAI_MINI_MODEL_ID` | `gpt-4o-mini` | | |
| | `ANTHROPIC_MODEL_ID` | `claude-sonnet-4-20250514` | | |
| | `GOOGLE_MODEL_ID` | `gemini-2.0-flash` | | |
| | `DASHSCOPE_MODEL_ID` | `qwen-plus` | | |
| | `YI_MODEL_ID` | `yi-large` | | |
| ## How it works | |
| 1. Select a reference model from the dropdown — **Base URL** and **Model ID** auto-fill from env vars (or registry defaults) | |
| 2. Edit Base URL / Model ID if needed (changes apply to current session only) | |
| 3. Enter your prompt and click **Send to Both** | |
| 4. Grade and comment on each response, then **Submit Evaluation** | |
| ## Deployment | |
| Deploy on HuggingFace Spaces with Gradio SDK. Set the API keys and optional overrides as repository secrets in Settings. | |