Quorum
Collection
Cheap, calibrated intent decisions from a jury of tiny open models. https://github.com/DataAgent-Lab/Quorum • 2 items • Updated
You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Quorum-Reader is shared for research and evaluation. Tell us a little about your use — it helps us prioritise what to build next. Access is granted automatically.
Log in or Sign Up to review the conditions and access this model content.
Quorum-Reader-Qwen3-4B-Adapter
A LoRA adapter for Qwen/Qwen3-4B — the in-context reader of the
Quorum 24-shot intent-classification ensemble.
Given 24 retrieved examples in the prompt (no weight update), it scores every candidate label in a single
forward pass. Combined (geometric mean) with a frozen BAAI/bge-large-en-v1.5 nearest-neighbour over the same
24 examples, the ensemble reaches 0.932 on Banking77 (full 3,080 test) — above the closed API Jev's 0.924
at Jev's own 24-shot protocol. In our study a further configuration (per-class descriptions + a
calibration-selected combination) reached 0.938 and significantly beat Jev on a paired McNemar test
(p = 0.0014).
Training
LoRA (r = 64) on Qwen/Qwen3-4B, fine-tuned on a mixture of public intent-classification datasets with
Banking77 held out. Single-forward answer-token option scoring.
Intended use & limitations
- Use as the reader in the Quorum 24-shot pipeline (see the repo). Requires the base model + PEFT to load.
- Contamination: the training mixture includes CLINC150, HWU64 and SNIPS — do not use this adapter to judge zero-shot generalization on those datasets.
- Not a general-purpose chat model; not a claim of open zero-shot state-of-the-art.
Usage
# https://github.com/DataAgent-Lab/Quorum
python scripts/eval_24shot.py --dataset banking77 --adapter DataAgent/Quorum-Reader-Qwen3-4B-Adapter
License
Adapter weights: Apache-2.0. The base model and datasets retain their own licenses.
- Downloads last month
- -