Dellboy/toppdblx-residual-parser

TopPDBLX residual parser — LoRA adapters

LoRA adapters for SmolLM2-360M-Instruct, trained to read the crystallisation conditions in the Protein Data Bank that a deterministic rule parser cannot.

Part of TopPDBLX: every exptl_crystal_grow.pdbx_details string in the PDB, turned into typed components with role, concentration and unit.

What problem this solves

A rule parser reads ~93% of the archive. The remaining 52,000 records are written as prose, across line breaks, or in notations no regex reaches. This model reads those.

Measured against 96 hand-labelled records (a crystallographer, one condition at a time):

Source	Precision	Recall	F1	F0.5
Rule parser alone	99.5%	72.8%	84.1	92.7
Rules + this model	99.3%	92.5%	95.8	97.8

Precision: of the reagents claimed, how many are really there. Recall: of the reagents really there, how many were found. F1: their harmonic mean. F0.5: the same, weighting precision twice as heavily — the right summary here, because a missed reagent is recoverable by re-reading the deposition and an invented one is not.

+19.7 points of recall, 58 reagents the rules never find, for 0.2 points of precision. Applied over the corpus it contributes 147,799 identified components and takes the classified share from 59.5% to 77.0%.

Revised 2026-08-03. The previous figures were too generous in one place and too harsh in another; both are explained below. Anything quoting this card before that date is stale.

The second gold batch, and what it corrected

The 96 records above were drawn at random. A second 96 were drawn where the pipeline and a 32B teacher disagree — adversarial by construction, and the precision signal a random sample cannot supply. It produced 32 rejections against the random batch's 1.

Contested 96	Precision	Recall	F1	F0.5
rules only	92.0%	56.5%	70.0	81.7
rules + this model (shipped)	92.7%	74.1%	82.3	88.2
rules + 32B teacher	91.2%	86.7%	88.9	90.3
union of both	91.3%	97.2%	94.2	92.4
rules + only where both agree	92.8%	63.6%	75.5	85.0

The rule parser is not 100% precise. Earlier versions of this card reported 100.0%, because the random batch never drew a contested record. On records that are actually disputed the rules are 92.0%. Precision is a property of the sample as much as of the parser, and a randomly drawn condition rarely challenges either reader.

The teacher beats this model on recall, decisively — 75 reagents found against 35 missed, p = 0.00017 — for 11 extra false positives against 3 avoided, p = 0.057.

This model's precision advantage largely evaporates on contested records: 92.7% against the teacher's 91.2%, not 99.3% against 92.0%. On a random sample it looks far more precise mostly because it rarely says anything contestable.

Neither batch is honest alone. The random 96 measures how good the pipeline is; the contested 96 measures which source is better. The contested numbers do not describe the corpus, where 57% of teacher-labelled records are contested rather than all of them.

The 2026-08-03 revision

Every figure on this card moved on 2026-08-03, from two changes that required no retraining:

• Lexicon 0.7.0 removed 29 entries that were never reagents. Corpus mining had promoted whole clauses to canonical entries — one of them was WELL_3_DROP_CRYSTALLIZATION_PLATE — and each then resolved, so each counted as a successful identification. It also merged 7 systematic names into molecules that already had entries, and added 42 real reagents.
• The pipeline began resolving aliases on this model's output. It emits PVP, TDP, MERCAPTOETHANOL and COH18N6, every one a reagent the lexicon knows under another name, and 658 correct readings were being discarded for spelling. Matching canonical ids exactly measured whether the model had guessed an internal identifier, which was never the question being asked of it.

Together: +5.1 points of recall with the adapters untouched. Rules precision fell from a reported 100.0% to 99.5% in the same pass, because a reagent that now resolves can now also be wrong — the old figure was partly an artefact of names that never resolved never being scored.

The model is unchanged. Only the measurement improved.

Rounds

Each subdirectory is one training round. All are rank-16 LoRA on 16 layers, prompt-masked, trained with MLX-LM on an M1 Max. Identification and grounding are measured on a frozen 2,000-record benchmark so rounds are comparable to each other; earlier rounds were scored against a live residual that shrank as curation improved and are marked accordingly.

Round	What changed	Identification	Grounding	Notes
round01	Bootstrap distillation from rule output, lexicon 0.1.0	87.0%	not measured	live residual, not comparable
round02	not_a_component class, confidence gate fixed	89.7%	not measured	live residual
round03	Cosine schedule, dropout, class rebalanced	88.4%	not measured	live residual
round04	Retrained on the 502-reagent lexicon	—	—	abandoned: trained on 36% duplicate rows
round05	Deduplicated training set, 95,818 distinct pairs	87.58%	93.41%	first frozen-benchmark round
round06	Full epoch, rank 16, 6,856 empty-answer examples	90.52% at iter 2,000	94.36%	the one to use — the final adapter, see below
round07	32B-teacher labels, 92.6% precise, 0.23 epochs	not run — scored on gold instead: 93.6% / 89.1%	—	regressed against round 06
round08	Same idea, labels 97.6% precise, 1.06 epochs	not run — scored on gold instead: 95.3% / 89.1%	—	regressed. See the confound below

Rounds 07 and 08 skipped the frozen benchmark deliberately. By then the hand-labelled gold set existed, and identification had already given three wrong answers to "how long should this train"; they were measured against labelled truth instead, which is the stricter test.

Which checkpoint to use, and a correction

Use round06/adapters.safetensors, the final adapter.

An earlier version of this card said to use round06/promoted_checkpoint_2000 instead, on the strength of a checkpoint sweep against a frozen 2,000-record benchmark. Measured against hand-labelled truth, that was wrong:

Round 06 adapter	Precision	Recall	F1	False positives
final, 6,000 iterations	99.6%	91.5%	95.4	1
checkpoint 2,000	95.7%	91.2%	93.4	12

Twelve extra false positives against one, p = 0.0034. The checkpoint is kept in the repo because the disagreement is the interesting part, not because it should be used.

Why the sweep chose wrongly. It ranked checkpoints on identification — whether an emitted reagent name exists in the curated lexicon. That metric cannot see a name which is real, present in the text, and simply not what the depositor meant, so a checkpoint that invents plausible chemistry scores well on it. Only labelled truth separates the two. The frozen benchmark remains useful for comparing rounds; it is not sufficient for choosing between them.

The sweep is still worth reading for what it shows about distillation:

Iteration	Fidelity to rules	Residual identification
500	80.60%	86.80%
2,000	89.60%	90.52%
6,000	93.60%	88.99%

Fidelity climbs monotonically while identification turns over: past a point the model spends its capacity imitating the rule parser — learning what is already in code — rather than reading the residual. Validation loss moved 0.003 across the whole span, which is why it is not the stopping signal here.

Rounds 07 and 08, and a correction to how they were judged

Both were attempts to get past the ceiling above by training on labels from a local Qwen2.5-32B teacher rather than from the rule parser. Both scored worse than round 06 — and the metric that said so was biased against them.

Five of the 96 hand-labelled records carried a reagent name the curated lexicon could not resolve, so their truth set was incomplete and a model naming that reagent correctly was scored as a false positive. The penalty falls hardest on whichever model says the most. After resolving those labels (lexicon 0.6.0) and excluding the records still incomplete:

	Precision	Recall	F1	F0.5
round 06 final	99.6%	87.4%	93.1	96.9
round 07 — labels 92.6% precise, 0.23 epochs	93.6%	89.1%	91.3	92.6
round 08 — labels 97.6% precise, 1.06 epochs	95.3%	89.1%	92.1	94.0

Held at lexicon 0.6.x on purpose. These rows predate the 2026-08-03 revision, which is worth roughly +5 points of recall to any row containing a student. Re-scoring round 06 alone would have it beat rounds 07 and 08 on a change none of them received, so the comparison is kept internally consistent at the version it was run. Current shipped figures are at the top.

On F0.5, which weights precision twice as heavily and is the right summary for a released dataset, round 06 leads by 2.9 points rather than F1's 1.0.

Round 06 remains the model to use: it leads on F1 and holds a 4.3-point precision advantage, and for a released scientific dataset missing chemistry is recoverable where invented chemistry is not. But the two teacher rounds do gain real recall, and the earlier version of this card called the approach closed on the strength of the biased numbers.

What the failures actually are. None of round 08's false positives names a reagent absent from the source text. Every one is a reagent genuinely present but not part of the crystallisation condition — a protein storage buffer, a soak, a cryoprotection step. The models are misattributing rather than inventing, which is a question about roles rather than chemistry.

They are also systematic: 11 of round 08's 14 false positives are the same reagents as round 07's, across independently filtered labels and four times the training.

Two teachers, measured

Correlated single-model error is what independent-model agreement removes and heuristic filters cannot, so a teacher-only find was kept only when a second, architecturally different 32B (Gemma-4-31b) named the same reagent unprompted:

	Precision	Recall	F1	F0.5
rules + this model (shipped)	99.6%	87.4%	93.1	96.9
rules + Gemma alone	96.3%	87.8%	91.8	94.4
+ every Qwen find	92.2%	92.9%	92.5	92.4
+ only where both agree	96.1%	91.8%	93.9	95.2

Also at lexicon 0.6.x, for the same reason as the table above.

Agreement keeps 13 of the 16 correct finds while cutting the wrong ones from 22 to 10 — the best F1 measured in this project, and still behind the shipped model on precision and on F0.5. It is useful as an inference-time ensemble, not as a training signal: the surviving additions are 13 correct to 10 wrong, and rounds 07 and 08 established that this student absorbs label noise rather than averaging it out.

Correction, 2026-08-03: Gemma is not the better teacher, and an earlier version of this card implied it was. The rules + Gemma alone row above sits beside a Qwen figure measured over a different set of records, which is not a comparison. Scored like for like — the 79 gold records both models answered and whose labels the lexicon can resolve:

Teacher, alone	Precision	Recall	F1	F0.5	False positives
Gemma-4-31b	89.7%	87.9%	88.8	89.4	24
Qwen2.5-32B	93.3%	87.9%	90.5	92.2	15

Both find exactly the same reagents — 210 true positives and 29 misses each. Gemma simply asserts nine more that are not in the text. It is also the less reliable harness: 82 of 96 generations parsed, against Qwen's 96 of 96, because its reasoning block consumes the token budget before it closes the JSON.

Gemma was chosen for architectural independence, which is what an agreement gate requires and which it does provide — Qwen2.5 and Gemma-4 fail differently, and that is the whole mechanism. It was never the stronger reader, and nothing above depends on it being one.

Using it

from mlx_lm import load, generate

model, tokenizer = load(
    "mlx-community/SmolLM2-360M-Instruct",
    adapter_path="round06",          # the final adapter, not promoted_checkpoint_2000
)

SYSTEM = (
    "You convert a PDB crystallisation condition string into JSON. "
    "Return only a JSON array. Each element has: role (precipitant, salt, buffer, additive, "
    "cryo, not_a_component or unknown), name (the canonical reagent, or null when the text "
    "names no reagent), amount (a number or null) and "
    "unit (percent_w_v, percent_v_v, molar, millimolar, mg_ml or null). "
    "Use not_a_component for text that names no reagent at all: method notes, screen "
    "references, or an unnamed protein, inhibitor or compound."
)

prompt = tokenizer.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": "0.1M HEPES pH 8.0, 10% PEG 8000, 10% 1-BUTANOL"}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

The emitted name is a canonical id from TopPDBLX's curated lexicon (574 reagents, 1,456 spellings). A name outside that lexicon should be treated as a hallucination and dropped — that guard is part of the pipeline, not the model.

Limitations

• It cannot discover chemistry the lexicon does not contain. It emits names it saw in training, so a genuinely novel reagent looks identical to a model error.
• Recall is the weaker half. 99.3% precision against 92.5% recall: it is likelier to miss a reagent than to invent one. Every metric used before the hand-labelled set was precision-shaped and blind to that.
• Its precision depends on how contested the record is. 99.3% on a random sample, 92.7% on records where a 32B teacher disagrees with it. Quote whichever matches your population, rather than whichever is higher.
• Roles are chemistry, not scope. The model labels a reagent buffer or salt whether it belonged to the drop, to a protein storage buffer or to a post-growth soak. The TopPDBLX pipeline now carries protein_buffer and soak roles and finds those passages itself; this model does not emit them.
• Trained on one corpus. Depositor conventions in the PDB; no claim beyond them.
• Concentrations are not covered by the 96-record evaluation, which measures reagent identity only.

Licence

MIT for the adapters. The base model carries its own licence.