README.md

metadata

title: Biaslab
emoji: ⚡
colorFrom: yellow
colorTo: purple
sdk: gradio
sdk_version: 6.3.0
app_file: app.py
pinned: false
license: mit
short_description: 'Dual-framing LLM bias probe: conviction vs acquiescence'

BiasLab — Dual-Framing Bias Probe

Test whether a language model holds a genuine position on any contested comparison, or is merely agreeing with whatever it is told — in any of 20 languages.

Each claim is asked twice — affirmatively (A out-performs B) and reversed (B out-performs A) — with random wrapper perturbation for robustness. Signing every answer onto one stance axis separates two quantities:

• Net bias = conviction — the stance that survives reversal.
• Swing = acquiescence — the part that just flips with the framing.

A model that agrees with whatever it is told has near-zero net bias and a large swing; a model with a genuine position has a large |net bias| and small swing. Raw agreement alone cannot tell these apart, which is the whole point of the method (Guey et al., 2026).

It works for any pair of targets, under either framework:

• Entity comparison — Topic Productivity, A Remote work, B Office work.
• Propositional truth — Topic The 2020 election, A fair, B fraudulent.

The output is a conviction-vs-acquiescence scatter, a per-language net-bias breakdown, a per-model table, and a downloadable CSV of every response.

API key

This Space calls models through OpenRouter. Each visitor pastes their own key into the field at the top of the app, so usage is billed to them (get one at https://openrouter.ai/keys). The key is used only for that session and is never stored.

• Public demo: leave OPENROUTER_API_KEY unset so every visitor uses their own key.
• Private use: set OPENROUTER_API_KEY as a Space secret; the in-app field then overrides it.
• OPENROUTER_PROXY — optional; only for running locally behind a proxy. Leave unset on HF.

Local use

pip install -r requirements.txt
export OPENROUTER_API_KEY=sk-or-...
python app.py

Method: Guey et al. (2026), Forced-choice measurement of conviction versus acquiescence in the geopolitical stances of large language models (arXiv:2503.23688), generalised here to any topic.