README.md

---
title: Loan Collection
emoji: 👁
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 6.14.0
python_version: '3.13'
app_file: app.py
pinned: false
---

# 🆚 LLM Comparison Playground

Compare several chat backends **side-by-side** on the same system prompt and
conversation, pick the best reply per turn, and save everything for later
analysis. All identifying values (endpoints, model names, labels, prompts) are
**configuration** — fill them in via environment variables / Space secrets;
nothing company-specific is baked into the code.

## Login
The UI login is toggleable. Set `ENABLE_AUTH=false` to disable it (open access),
or leave it on (the default) to require a password. Credentials are read from the
environment (`APP_USERNAME` / `APP_PASSWORD`, or `APP_AUTH` for multiple accounts)
and never appear in the code or the UI. If auth is on but nothing is set, a default
dev login (`user` / `slm-demo`) is used and a warning is printed — set real
credentials before sharing.

## Configure the backends
Set the values via environment variables / Space secrets (see `.env.example`). The
defaults in `config.py` are neutral placeholders.

| Variable | What it is |
|---|---|
| `BACKEND_API_BASE_URL` | OpenAI-compatible chat-completions URL for Backend A (auth via `x-api-key`) |
| `BACKEND_API_KEY` | API key for Backend A (secret) |
| `BACKEND_A_LABEL` / `BACKEND_A_ENDPOINT_ID` / `BACKEND_A_MODEL` | Backend A label, endpoint id, model |
| `ENABLE_BACKEND_B` | Feature flag — show the second backend for side-by-side comparison (default `false`) |
| `AZURE_ENDPOINT` / `AZURE_API_KEY` / `AZURE_DEPLOYMENT` | Azure OpenAI endpoint, key (secret), deployment for Backend B |
| `BACKEND_B_LABEL` | Backend B label |
| `STT_API_KEY` | Optional key for voice input (falls back to `BACKEND_API_KEY`) |
| `ENABLE_AUTH` / `APP_USERNAME` / `APP_PASSWORD` / `APP_AUTH` | UI login toggle + credentials |
| `DATA_DIR` | Where logs are written (set to `/data` on HF persistent storage) |

By default only **Backend A** (the conversational model) is shown. Set
`ENABLE_BACKEND_B=true` to turn on the second backend and the side-by-side
"Preferred response" picker. To compare more or fewer models, edit the `BACKENDS`
list in `config.py` (anonymous labels are assigned automatically: A, B, C, …).

## Presets
A **Preset** dropdown in the sidebar loads ready-made system-prompt + intro pairs,
each with its own fixed set of fill-in **call-detail variables** (customer name,
amount, dates, etc.). Selecting one fills the fields and starts a fresh session.
Edit the list in `config.py` (`PRESETS`).

## Voice input
Switch **Input mode** to 🎤 Voice to record from your mic; the recording is
transcribed into the message box (via the `ringglabs` Speech-to-Text SDK) so you
can edit it before sending. The STT key comes from `STT_API_KEY`, or
`BACKEND_API_KEY` if that's unset.

## How it works
- One shared **system prompt** + **intro message** is broadcast to every backend.
- Each user message is sent to all backends; **each keeps its own thread** and
  continues from its own replies (threads diverge on assistant turns).
- All backends **stream concurrently** into their own panel, with per-model
  **token counts** (input / output / cached).
- Pick the **preferred** response (when comparing 2+ backends) and click
  **Save preferred** to record it.

## Anonymity
Panels are shown **anonymously** ("Response A", "Response B", …) so the model
identity never biases your pick. The mapping back to the real model is saved with
each preferred pick (server-side only — users never see it).

## Where data is saved
All under `DATA_DIR` (`./data` locally, `/data` on HF persistent storage):
- `sessions/<session_id>.json` — full session, rewritten every turn.
- `requests.jsonl` — one line per turn, **every request made** (appended
  unconditionally, including turns that errored).
- `comparisons.jsonl` — one line per turn you explicitly **Save preferred**.

All are gitignored by default (see `.gitignore`).

## Run locally
```bash
pip install -r requirements.txt
cp .env.example .env        # fill in the values
python app.py
```

If a backend's key is missing, only that panel shows an error — the others still work.

## Deploy to Hugging Face Spaces
1. Push these files to the Space (the `.env` and `data/` contents are gitignored
   and must NOT be pushed).
2. In **Settings → Variables and secrets**, add the backend/Azure keys and any
   login credentials as **secrets**, and the non-secret URLs/labels as **variables**.
3. To keep saved logs across restarts, enable **persistent storage** and set
   `DATA_DIR=/data`. (Without it, the Space filesystem is wiped on every restart.)
4. The Space builds from `requirements.txt` and runs `app.py`.

## Project structure
```
.
├── app.py              # Gradio entrypoint (side-by-side comparison UI)
├── config.py           # env-driven BACKENDS, defaults, DATA_DIR, presets
├── src/
│   ├── clients.py      # backend-agnostic streaming (custom + azure) + STT
│   ├── display.py      # tool-call/HTML-safe rendering
│   └── storage.py      # session JSON + requests/comparisons logs
├── requirements.txt
├── .env.example
└── data/               # logs (gitignored)
```