frc-contract-drift
Sentence β the reason codes FinanceRateCalc's deterministic gate assigns when a published figure is restated in a way the record cannot support. Labels: CAUSALITY_NOT_ESTABLISHED, INDIVIDUAL_PREDICTION_PROHIBITED, LEGAL_CONCLUSION_NOT_SUPPORTED, LENDER_RECOMMENDATION, PROGRAM_MISMATCH, QUALIFIER_DELETED, SAME_APPLICANT_FRAMING, SCOPE_TOO_BROAD.
Trained on 8896 rule-generated sentences from 386 claim contracts (canonical sentence and faithful paraphrases β no code; rule-written distortions β the code), plus the citation-decision fixture candidates with the codes the gate gave them. No model wrote any training sentence. Held out by claim id: exact-match 0.999, per-label precision/recall in eval.json.
This model does not decide anything on financeratecalc.com β the gate does. It is a second reader for paraphrases the regexes miss, and a reproducible artefact of the record. It knows nothing about figures it was not shown; check the number with the receipt, not with this model.
Data: https://financeratecalc.com/eval/drift-training-set.jsonl Β· Spec: https://financeratecalc.com/rl-env/CITATION-DECISION-SPEC.md Β· Source: https://github.com/financeratecalc/financeratecalc.github.io
Out-of-template test (the number to quote)
The in-template figure (0.999) is measured on sentences from the same rule generator that wrote the training set. On 144 violating and 36 clean sentences written by hand outside those templates:
| violating sentence flagged with any code | 91% |
| flagged with the right code | 86% |
| clean sentence wrongly flagged | 11% |
On the held-out half (40 violating sentences): this model caught 33, the site's hand-written rule gate caught 26, the two together 37.
The test sentences were written by the publisher's assistant, not by independent annotators; small n. Weakest: same-applicant framing and whole-market conclusions. Data: https://financeratecalc.com/eval/drift-ood-set.jsonl, result: https://financeratecalc.com/eval/drift-model-ood.json
- Downloads last month
- 67
Model tree for FinanceRateCalc/frc-contract-drift
Base model
distilbert/distilroberta-base