README.md

metadata

title: Loan Collection
emoji: 👁
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 6.14.0
python_version: '3.13'
app_file: app.py
pinned: false

🆚 LLM Comparison Playground

Compare several chat backends side-by-side on the same system prompt and conversation, pick the best reply per turn, and save everything for later analysis. All identifying values (endpoints, model names, labels, prompts) are configuration — fill them in via environment variables / Space secrets; nothing company-specific is baked into the code.

Login

The UI login is toggleable. Set ENABLE_AUTH=false to disable it (open access), or leave it on (the default) to require a password. Credentials are read from the environment (APP_USERNAME / APP_PASSWORD, or APP_AUTH for multiple accounts) and never appear in the code or the UI. If auth is on but nothing is set, a default dev login (user / slm-demo) is used and a warning is printed — set real credentials before sharing.

Configure the backends

Set the values via environment variables / Space secrets (see .env.example). The defaults in config.py are neutral placeholders.

Variable	What it is
BACKEND_API_BASE_URL	OpenAI-compatible chat-completions URL for Backend A (auth via x-api-key)
BACKEND_API_KEY	API key for Backend A (secret)
BACKEND_A_LABEL / BACKEND_A_ENDPOINT_ID / BACKEND_A_MODEL	Backend A label, endpoint id, model
ENABLE_BACKEND_B	Feature flag — show the second backend for side-by-side comparison (default false)
AZURE_ENDPOINT / AZURE_API_KEY / AZURE_DEPLOYMENT	Azure OpenAI endpoint, key (secret), deployment for Backend B
BACKEND_B_LABEL	Backend B label
STT_API_KEY	Optional key for voice input (falls back to BACKEND_API_KEY)
ENABLE_AUTH / APP_USERNAME / APP_PASSWORD / APP_AUTH	UI login toggle + credentials
DATA_DIR	Where logs are written (set to /data on HF persistent storage)

By default only Backend A (the conversational model) is shown. Set ENABLE_BACKEND_B=true to turn on the second backend and the side-by-side "Preferred response" picker. To compare more or fewer models, edit the BACKENDS list in config.py (anonymous labels are assigned automatically: A, B, C, …).

Presets

A Preset dropdown in the sidebar loads ready-made system-prompt + intro pairs, each with its own fixed set of fill-in call-detail variables (customer name, amount, dates, etc.). Selecting one fills the fields and starts a fresh session. Edit the list in config.py (PRESETS).

Voice input

Switch Input mode to 🎤 Voice to record from your mic; the recording is transcribed into the message box (via the ringglabs Speech-to-Text SDK) so you can edit it before sending. The STT key comes from STT_API_KEY, or BACKEND_API_KEY if that's unset.

How it works

• One shared system prompt + intro message is broadcast to every backend.
• Each user message is sent to all backends; each keeps its own thread and continues from its own replies (threads diverge on assistant turns).
• All backends stream concurrently into their own panel, with per-model token counts (input / output / cached).
• Pick the preferred response (when comparing 2+ backends) and click Save preferred to record it.

Anonymity

Panels are shown anonymously ("Response A", "Response B", …) so the model identity never biases your pick. The mapping back to the real model is saved with each preferred pick (server-side only — users never see it).

Where data is saved

All under DATA_DIR (./data locally, /data on HF persistent storage):

• sessions/<session_id>.json — full session, rewritten every turn.
• requests.jsonl — one line per turn, every request made (appended unconditionally, including turns that errored).
• comparisons.jsonl — one line per turn you explicitly Save preferred.

All are gitignored by default (see .gitignore).

Run locally

pip install -r requirements.txt
cp .env.example .env        # fill in the values
python app.py

If a backend's key is missing, only that panel shows an error — the others still work.

Deploy to Hugging Face Spaces

1. Push these files to the Space (the .env and data/ contents are gitignored and must NOT be pushed).
2. In Settings → Variables and secrets, add the backend/Azure keys and any login credentials as secrets, and the non-secret URLs/labels as variables.
3. To keep saved logs across restarts, enable persistent storage and set DATA_DIR=/data. (Without it, the Space filesystem is wiped on every restart.)
4. The Space builds from requirements.txt and runs app.py.

Project structure

.
├── app.py              # Gradio entrypoint (side-by-side comparison UI)
├── config.py           # env-driven BACKENDS, defaults, DATA_DIR, presets
├── src/
│   ├── clients.py      # backend-agnostic streaming (custom + azure) + STT
│   ├── display.py      # tool-call/HTML-safe rendering
│   └── storage.py      # session JSON + requests/comparisons logs
├── requirements.txt
├── .env.example
└── data/               # logs (gitignored)