README.md

---
license: mit
base_model: mlx-community/SmolLM2-360M-Instruct
tags:
  - mlx
  - lora
  - information-extraction
  - structural-biology
  - crystallography
language:
  - en
library_name: mlx
---

# TopPDBLX residual parser — LoRA adapters

LoRA adapters for **SmolLM2-360M-Instruct**, trained to read the crystallisation conditions in the
Protein Data Bank that a deterministic rule parser cannot.

Part of [TopPDBLX](https://github.com/bellcheddar/TopPDBLX): every `exptl_crystal_grow.pdbx_details`
string in the PDB, turned into typed components with role, concentration and unit.

## What problem this solves

A rule parser reads ~93% of the archive. The remaining **52,000 records** are written as prose,
across line breaks, or in notations no regex reaches. This model reads those.

Measured against **96 hand-labelled records** (a crystallographer, one condition at a time):

| Source | Precision | Recall | F1 | F0.5 |
|---|---|---|---|---|
| Rule parser alone | 99.5% | 72.8% | 84.1 | 92.7 |
| Rules + this model | **99.3%** | **92.5%** | **95.8** | **97.8** |

*Precision*: of the reagents claimed, how many are really there. *Recall*: of the reagents really
there, how many were found. *F1*: their harmonic mean. *F0.5*: the same, weighting precision twice
as heavily — the right summary here, because a missed reagent is recoverable by re-reading the
deposition and an invented one is not.

**+19.7 points of recall, 58 reagents the rules never find, for 0.2 points of precision.**
Applied over the corpus it contributes 147,799 identified components and takes the classified
share from 59.5% to 77.0%.

> **Revised 2026-08-03.** The previous figures were too generous in one place and too harsh
> in another; both are explained below. Anything quoting this card before that date is stale.

## The second gold batch, and what it corrected

The 96 records above were drawn **at random**. A second 96 were drawn **where the pipeline and a
32B teacher disagree** — adversarial by construction, and the precision signal a random sample
cannot supply. It produced 32 rejections against the random batch's 1.

| Contested 96 | Precision | Recall | F1 | F0.5 |
|---|---|---|---|---|
| rules only | 92.0% | 56.5% | 70.0 | 81.7 |
| rules + this model *(shipped)* | 92.7% | 74.1% | 82.3 | 88.2 |
| rules + 32B teacher | 91.2% | 86.7% | 88.9 | 90.3 |
| union of both | 91.3% | **97.2%** | **94.2** | 92.4 |
| rules + only where both agree | **92.8%** | 63.6% | 75.5 | 85.0 |

**The rule parser is not 100% precise.** Earlier versions of this card reported 100.0%, because
the random batch never drew a contested record. On records that are actually disputed the rules
are 92.0%. **Precision is a property of the sample as much as of the parser**, and a randomly
drawn condition rarely challenges either reader.

**The teacher beats this model on recall, decisively** — 75 reagents found against 35 missed,
p = 0.00017 — for 11 extra false positives against 3 avoided, p = 0.057.

**This model's precision advantage largely evaporates on contested records**: 92.7% against the
teacher's 91.2%, not 99.3% against 92.0%. On a random sample it looks far more precise mostly
because it rarely says anything contestable.

Neither batch is honest alone. The random 96 measures how good the pipeline is; the contested 96
measures which source is better. The contested numbers do not describe the corpus, where 57% of
teacher-labelled records are contested rather than all of them.

## The 2026-08-03 revision

Every figure on this card moved on 2026-08-03, from two changes that required no retraining:

- **Lexicon 0.7.0** removed 29 entries that were never reagents. Corpus mining had promoted whole
  clauses to canonical entries — one of them was `WELL_3_DROP_CRYSTALLIZATION_PLATE` — and each
  then *resolved*, so each counted as a successful identification. It also merged 7 systematic
  names into molecules that already had entries, and added 42 real reagents.
- **The pipeline began resolving aliases on this model's output.** It emits `PVP`, `TDP`,
  `MERCAPTOETHANOL` and `COH18N6`, every one a reagent the lexicon knows under another name, and
  658 correct readings were being discarded for spelling. Matching canonical ids exactly measured
  whether the model had guessed an internal identifier, which was never the question being asked
  of it.

Together: **+5.1 points of recall with the adapters untouched.** Rules precision fell from a
reported 100.0% to 99.5% in the same pass, because a reagent that now resolves can now also be
wrong — the old figure was partly an artefact of names that never resolved never being scored.

**The model is unchanged. Only the measurement improved.**

## Rounds

Each subdirectory is one training round. All are rank-16 LoRA on 16 layers, prompt-masked, trained
with MLX-LM on an M1 Max. Identification and grounding are measured on a **frozen 2,000-record
benchmark** so rounds are comparable to each other; earlier rounds were scored against a live
residual that shrank as curation improved and are marked accordingly.

| Round | What changed | Identification | Grounding | Notes |
|---|---|---|---|---|
| `round01` | Bootstrap distillation from rule output, lexicon 0.1.0 | 87.0% | not measured | live residual, not comparable |
| `round02` | `not_a_component` class, confidence gate fixed | 89.7% | not measured | live residual |
| `round03` | Cosine schedule, dropout, class rebalanced | 88.4% | not measured | live residual |
| `round04` | Retrained on the 502-reagent lexicon | — | — | **abandoned**: trained on 36% duplicate rows |
| `round05` | Deduplicated training set, 95,818 distinct pairs | 87.58% | 93.41% | first frozen-benchmark round |
| `round06` | Full epoch, rank 16, 6,856 empty-answer examples | **90.52%** at iter 2,000 | 94.36% | **the one to use** — the *final* adapter, see below |
| `round07` | 32B-teacher labels, 92.6% precise, 0.23 epochs | not run — scored on gold instead: 93.6% / 89.1% | — | **regressed** against round 06 |
| `round08` | Same idea, labels 97.6% precise, 1.06 epochs | not run — scored on gold instead: 95.3% / 89.1% | — | **regressed**. See the confound below |

Rounds 07 and 08 skipped the frozen benchmark deliberately. By then the hand-labelled gold set
existed, and `identification` had already given three wrong answers to "how long should this
train"; they were measured against labelled truth instead, which is the stricter test.

### Which checkpoint to use, and a correction

**Use `round06/adapters.safetensors`, the final adapter.**

An earlier version of this card said to use `round06/promoted_checkpoint_2000` instead, on the
strength of a checkpoint sweep against a frozen 2,000-record benchmark. Measured against
**hand-labelled truth**, that was wrong:

| Round 06 adapter | Precision | Recall | F1 | False positives |
|---|---|---|---|---|
| **final, 6,000 iterations** | **99.6%** | **91.5%** | **95.4** | **1** |
| checkpoint 2,000 | 95.7% | 91.2% | 93.4 | 12 |

Twelve extra false positives against one, p = 0.0034. The checkpoint is kept in the repo because
the disagreement is the interesting part, not because it should be used.

**Why the sweep chose wrongly.** It ranked checkpoints on *identification* — whether an emitted
reagent name exists in the curated lexicon. That metric cannot see a name which is real, present
in the text, and simply not what the depositor meant, so a checkpoint that invents plausible
chemistry scores well on it. Only labelled truth separates the two. The frozen benchmark remains
useful for comparing rounds; it is not sufficient for choosing between them.

The sweep is still worth reading for what it shows about distillation:

| Iteration | Fidelity to rules | Residual identification |
|---|---|---|
| 500 | 80.60% | 86.80% |
| 2,000 | 89.60% | 90.52% |
| 6,000 | 93.60% | 88.99% |

Fidelity climbs monotonically while identification turns over: past a point the model spends its
capacity imitating the rule parser — learning what is already in code — rather than reading the
residual. Validation loss moved 0.003 across the whole span, which is why it is not the stopping
signal here.

### Rounds 07 and 08, and a correction to how they were judged

Both were attempts to get past the ceiling above by training on labels from a local Qwen2.5-32B
teacher rather than from the rule parser. Both scored worse than round 06 — and the metric that
said so was biased against them.

Five of the 96 hand-labelled records carried a reagent name the curated lexicon could not resolve,
so their truth set was incomplete and a model naming that reagent *correctly* was scored as a false
positive. The penalty falls hardest on whichever model says the most. After resolving those labels
(lexicon 0.6.0) and excluding the records still incomplete:

| | Precision | Recall | F1 | F0.5 |
|---|---|---|---|---|
| **round 06 final** | **99.6%** | 87.4% | **93.1** | **96.9** |
| round 07 — labels 92.6% precise, 0.23 epochs | 93.6% | **89.1%** | 91.3 | 92.6 |
| round 08 — labels 97.6% precise, 1.06 epochs | 95.3% | **89.1%** | 92.1 | 94.0 |

> **Held at lexicon 0.6.x on purpose.** These rows predate the 2026-08-03 revision, which is worth
> roughly +5 points of recall to any row containing a student. Re-scoring round 06 alone would
> have it beat rounds 07 and 08 on a change none of them received, so the comparison is kept
> internally consistent at the version it was run. Current shipped figures are at the top.

On F0.5, which weights precision twice as heavily and is the right summary for a released dataset,
round 06 leads by 2.9 points rather than F1's 1.0.

**Round 06 remains the model to use**: it leads on F1 and holds a 4.3-point precision advantage,
and for a released scientific dataset missing chemistry is recoverable where invented chemistry is
not. But the two teacher rounds do gain real recall, and the earlier version of this card called
the approach closed on the strength of the biased numbers.

**What the failures actually are.** None of round 08's false positives names a reagent absent from
the source text. Every one is a reagent genuinely present but not part of the crystallisation
condition — a protein storage buffer, a soak, a cryoprotection step. The models are misattributing
rather than inventing, which is a question about roles rather than chemistry.

They are also systematic: 11 of round 08's 14 false positives are the same reagents as round 07's,
across independently filtered labels and four times the training.

### Two teachers, measured

Correlated single-model error is what independent-model agreement removes and heuristic filters
cannot, so a teacher-only find was kept only when a second, architecturally different 32B
(Gemma-4-31b) named the same reagent unprompted:

| | Precision | Recall | F1 | F0.5 |
|---|---|---|---|---|
| rules + this model *(shipped)* | **99.6%** | 87.4% | 93.1 | **96.9** |
| rules + Gemma alone | 96.3% | 87.8% | 91.8 | 94.4 |
| + every Qwen find | 92.2% | **92.9%** | 92.5 | 92.4 |
| **+ only where both agree** | 96.1% | 91.8% | **93.9** | 95.2 |

> Also at lexicon 0.6.x, for the same reason as the table above.

Agreement keeps 13 of the 16 correct finds while cutting the wrong ones from 22 to 10 — the best
F1 measured in this project, and still behind the shipped model on precision and on F0.5. It is
useful as an **inference-time ensemble**, not as a training signal: the surviving additions are 13
correct to 10 wrong, and rounds 07 and 08 established that this student absorbs label noise rather
than averaging it out.

**Correction, 2026-08-03: Gemma is not the better teacher, and an earlier version of this card
implied it was.** The `rules + Gemma alone` row above sits beside a Qwen figure measured over a
different set of records, which is not a comparison. Scored like for like — the 79 gold records
both models answered and whose labels the lexicon can resolve:

| Teacher, alone | Precision | Recall | F1 | F0.5 | False positives |
|---|---|---|---|---|---|
| Gemma-4-31b | 89.7% | 87.9% | 88.8 | 89.4 | 24 |
| **Qwen2.5-32B** | **93.3%** | 87.9% | **90.5** | **92.2** | **15** |

Both find exactly the same reagents — 210 true positives and 29 misses each. Gemma simply asserts
nine more that are not in the text. It is also the less reliable harness: **82 of 96 generations
parsed, against Qwen's 96 of 96**, because its reasoning block consumes the token budget before it
closes the JSON.

Gemma was chosen for **architectural independence**, which is what an agreement gate requires and
which it does provide — Qwen2.5 and Gemma-4 fail differently, and that is the whole mechanism. It
was never the stronger reader, and nothing above depends on it being one.

## Using it

```python
from mlx_lm import load, generate

model, tokenizer = load(
    "mlx-community/SmolLM2-360M-Instruct",
    adapter_path="round06",          # the final adapter, not promoted_checkpoint_2000
)

SYSTEM = (
    "You convert a PDB crystallisation condition string into JSON. "
    "Return only a JSON array. Each element has: role (precipitant, salt, buffer, additive, "
    "cryo, not_a_component or unknown), name (the canonical reagent, or null when the text "
    "names no reagent), amount (a number or null) and "
    "unit (percent_w_v, percent_v_v, molar, millimolar, mg_ml or null). "
    "Use not_a_component for text that names no reagent at all: method notes, screen "
    "references, or an unnamed protein, inhibitor or compound."
)

prompt = tokenizer.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": "0.1M HEPES pH 8.0, 10% PEG 8000, 10% 1-BUTANOL"}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```

The emitted `name` is a canonical id from TopPDBLX's curated lexicon (574 reagents, 1,456
spellings). A name outside that lexicon should be treated as a hallucination and dropped — that
guard is part of the pipeline, not the model.

## Limitations

- **It cannot discover chemistry the lexicon does not contain.** It emits names it saw in
  training, so a genuinely novel reagent looks identical to a model error.
- **Recall is the weaker half.** 99.3% precision against 92.5% recall: it is likelier to miss a
  reagent than to invent one. Every metric used before the hand-labelled set was
  precision-shaped and blind to that.
- **Its precision depends on how contested the record is.** 99.3% on a random sample, 92.7% on
  records where a 32B teacher disagrees with it. Quote whichever matches your population,
  rather than whichever is higher.
- **Roles are chemistry, not scope.** The model labels a reagent `buffer` or `salt` whether it
  belonged to the drop, to a protein storage buffer or to a post-growth soak. The TopPDBLX
  pipeline now carries `protein_buffer` and `soak` roles and finds those passages itself; this
  model does not emit them.
- **Trained on one corpus.** Depositor conventions in the PDB; no claim beyond them.
- **Concentrations are not covered by the 96-record evaluation**, which measures reagent identity
  only.

## Licence

MIT for the adapters. The base model carries its own licence.