| --- |
| title: Biaslab |
| emoji: ⚡ |
| colorFrom: yellow |
| colorTo: purple |
| sdk: gradio |
| sdk_version: 6.3.0 |
| app_file: app.py |
| pinned: false |
| license: mit |
| short_description: "Dual-framing LLM bias probe: conviction vs acquiescence" |
| --- |
| |
| # BiasLab — Dual-Framing Bias Probe |
|
|
| Test whether a language model holds a **genuine position** on any contested comparison, or is |
| merely **agreeing with whatever it is told** — in any of 20 languages. |
|
|
| Each claim is asked **twice** — affirmatively (*A out-performs B*) and reversed (*B out-performs A*) |
| — with random wrapper perturbation for robustness. Signing every answer onto one stance axis |
| separates two quantities: |
|
|
| - **Net bias = conviction** — the stance that *survives reversal*. |
| - **Swing = acquiescence** — the part that just *flips with the framing*. |
|
|
| A model that agrees with whatever it is told has near-zero net bias and a large swing; a model |
| with a genuine position has a large `|net bias|` and small swing. **Raw agreement alone cannot |
| tell these apart**, which is the whole point of the method (Guey et al., 2026). |
|
|
| It works for any pair of targets, under either framework: |
|
|
| - **Entity comparison** — Topic *Productivity*, A *Remote work*, B *Office work*. |
| - **Propositional truth** — Topic *The 2020 election*, A *fair*, B *fraudulent*. |
|
|
| The output is a conviction-vs-acquiescence scatter, a per-language net-bias breakdown, a |
| per-model table, and a downloadable CSV of every response. |
|
|
| ## API key |
|
|
| This Space calls models through [OpenRouter](https://openrouter.ai). Each visitor pastes their |
| **own** key into the field at the top of the app, so usage is billed to them (get one at |
| <https://openrouter.ai/keys>). The key is used only for that session and is never stored. |
|
|
| - **Public demo:** leave `OPENROUTER_API_KEY` *unset* so every visitor uses their own key. |
| - **Private use:** set `OPENROUTER_API_KEY` as a Space secret; the in-app field then overrides it. |
| - `OPENROUTER_PROXY` — optional; only for running locally behind a proxy. Leave unset on HF. |
|
|
| ## Local use |
|
|
| ```bash |
| pip install -r requirements.txt |
| export OPENROUTER_API_KEY=sk-or-... |
| python app.py |
| ``` |
|
|
| Method: Guey et al. (2026), *Forced-choice measurement of conviction versus acquiescence in the |
| geopolitical stances of large language models* (arXiv:2503.23688), generalised here to any topic. |
|
|
