calerio/silent-signals-rqa

Binary disambiguator — coded vs literal use of a candidate dog-whistle term. RoBERTa-base.

Headline metric: disambiguation_124_f1 — F1 on the 124-row locked human-eval disambiguation set (see DESIGN_DEFENSE.md D7).

Variants

Each variant is checked into its own branch. Load with:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained("calerio/silent-signals-rqa", revision="<branch>")
tok   = AutoTokenizer.from_pretrained("calerio/silent-signals-rqa", revision="<branch>")

Branch	Variant id	disambiguation_124_f1	Notes
term-seed123	rqa_term_seed123	0.6880	—
term-seed42	rqa_term_seed42	0.7097	—
term-seed7	rqa_term_seed7	0.7246	default
term-enriched-def-seed123	rqa_term_enriched_def_seed123	0.6774	—
term-enriched-def-seed42	rqa_term_enriched_def_seed42	0.6992	—
term-enriched-def-seed7	rqa_term_enriched_def_seed7	0.7296	raw leader (not default — see below)

Default variant

rqa_term_seed7 — see the per-task default_variant_rationale in data/manifests/model_inventory.json of the project repo.

Picked the arm with the highest mean disambiguation_124_f1 across seeds (term = 0.7074), then the best seed within. Defends a per-arm comparison, not a single-seed peak. See docs/rq_a_report.md.

Where this came from

Bocconi 597 NLP group project on dog-whistle detection and disambiguation, on the silent_signals corpus (Kruk et al. 2024). Full methodology: docs/DESIGN_DEFENSE.md + per-RQ reports in the project repo. HF Space build write-up: docs/hf_space_report.md.