README.md

metadata

tags:
  - text-classification
  - multiple-choice
  - calibrated-classification
  - system-one

decision_maker

decision_maker is a BERT-Large decision encoder. It accepts a state, a question, and a variable list of options, then returns a calibrated probability distribution over those options in one encoder pass.

Quick start

pip install -r requirements.txt
python run.py

Edit STATE, QUESTION, and OPTIONS at the top of run.py.

Architecture

• bert-large-uncased bidirectional base encoder
• four-layer decision Transformer
• [DECIDE] query and [OPTION] key dot-product scoring
• global post-training temperature scaling

Evaluation

On the held-out development split used for this release, raw vs calibrated:

Metric	Raw	Calibrated
Accuracy	0.699	0.699
NLL	0.772	0.625
Brier	0.386	0.359
ECE	0.121	0.045

Limitations

This model has a 256-token structured input limit. Inputs that exceed that limit are not supported by the reference runner. Calibration is measured on the release validation distribution and may not transfer to every domain. Do not use it as the sole basis for high-stakes decisions.