GOVINDFROM/codx-elr-fusion

ELR-Fusion likelihood-ratio adapter

LoRA adapter that teaches a model to emit a well-formed, calibrated log-likelihood-ratio vector for a single clinical finding against 49 candidate pathologies. It is the trained component of ELR-Fusion, which combines per-finding evidence weights symbolically:

log-odds(d | E) = log-odds(d) + sum_i log LR(e_i | d)

Because the combination is a sum, the resulting posterior is a function of the evidence set. Order invariance is therefore exact by construction, for every model and every case — verified numerically at machine epsilon (mean pairwise JSD 5.2e-17 over 200 permutations).

Why it was trained

Zero-shot likelihood-ratio elicitation is ELR-Fusion's weak point. Measured on qwen3-4b-nothink, zero-shot fusion is exactly order-invariant but less accurate than direct prompting (top-1 0.033 vs 0.057) and markedly overconfident (entropy 3.11 vs 4.49 bits) — the compounding effect of summing log-LRs over correlated findings. This adapter targets that deficit.

Targets and the split that matters

Targets come from a likelihood table reconstructed from all 1,292,579 released DDXPlus patients (binomial SE ~3e-4 per entry). The table estimates the generator's per-finding conditionals; it is deliberately not used as a posterior, because naive recombination of those weights reproduces the shipped differentials poorly (see the data audit).

The train/test split is by finding, not by example. 103 of 516 findings are held out entirely — the adapter never sees them in any format or demographic framing — so the held-out loss measures generalisation to unseen findings rather than memorisation.

Training

Qwen3-8B, LoRA r=32 / alpha=64 / dropout=0.05 on all attention and MLP projections (87.3M trainable, 1.05% of parameters). Loss is masked to the assistant turn only, so it is not spent reproducing the 49-line candidate list that prefixes every prompt. Three elicitation formats are trained jointly — numeric LR, log-odds, and ordinal verbal bands — which is also ablation #7 of the paper.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# The base revision is pinned: `main` moves, and the adapter was trained
# against this commit. See ARTIFACTS.md in the source tree.
BASE, REV = "Qwen/Qwen3-8B", "b968826d9c46dd6066d109eabc6255188de91218"

base = AutoModelForCausalLM.from_pretrained(BASE, revision=REV, dtype="bfloat16")
model = PeftModel.from_pretrained(base, "GOVINDFROM/codx-elr-fusion")
tok = AutoTokenizer.from_pretrained(BASE, revision=REV)

The adapter alone is not the method. ELR-Fusion is the adapter plus the symbolic recombination step, and the order-invariance guarantee comes from the latter — summing log-LRs over a set cannot depend on their order. Prompt construction and the fusion step are in coherence/elicit/prompts.py and coherence/methods/elr_fusion.py.

Reproducing

• Code: https://github.com/GIND123/Probabilistic-Coherence-as-a-Test-of-Clinical-Reasoning-in-Language-Models
• Artifacts, battery and training set: GOVINDFROM/codx-clinical-coherence
• python -m coherence.hub.hf_pull pull train --adapter fetches the SFT set and this adapter into the layout the training and analysis code expects.

Honest limits

Measured accuracy cost is real: ELR-Fusion pays 5.6 top-1 points against direct prompting, and a fixed canonical presentation order reaches the same exact zero order effect at no accuracy cost. What this adapter buys over a canonical order is invariance to which order is chosen — different fixed orders sit 0.120–0.258 JSD apart — and a per-finding audit trail. Replacing the elicited log-LR table with one estimated from the corpus moves top-1 from 0.133 to 0.9765, which locates the remaining deficit in the elicited weights rather than in the fusion rule.