promotion/polyedit-artifacts

PolyEdit artifacts

Models behind the results in POLYEDIT (results/results.md). The task: given a polymer, a property and a target interval, return a single-step matched-pair edit that lands the property inside the interval.

adapters/

LoRA on meta-llama/Llama-3.2-3B-Instruct (rank 32, alpha 64, all-linear, 3 epochs, 7,143 steps, assistant-only loss). Splits are component-based 80/20 over PolymerGraphID with cross-split pairs removed. training_meta.json in each folder carries the full configuration.

Folder	Training target	Interval	Arm
1x-product	{product, rules}	1x	A (free-form); B is this decoded rule-first
1x-rules	{rules}	1x	C (rule-only)
2x-product	{product, rules}	2x	A/B, headline numbers
2x-rules	{rules}	2x	C, headline numbers

edit_success on the held-out split (44,492 prompts, 917 source polymers, clustered CI):

Arm	1x train / 1x eval	2x train / 2x eval
A. free-form	9.08%	12.64% [11.70, 13.48]
B. rule-first	9.27%	14.26% [13.44, 15.01]
C. rule-only	7.08%	11.36% [10.55, 12.16]

edit_success requires both that the property lands in the interval and that the product is genuinely an edit of the source. Scoring the property alone credits a model for retrieving a memorised polymer: 40% of the free-form model's apparent successes were exactly that.

verifiers/

TransPolymer and polyBERT fine-tuned as property regressors. Each .json holds held-out metrics.

Folder	What	Note
v1	absolute values, MMP corpus only (1,375 polymers)	superseded
v2	absolute values, plus external per-property labels (2,696)	current
delta	property changes over 144,932 matched pairs	negative result

These cannot be used to judge success. Their RMSE is 1.1-12.4x the interval they would have to decide membership of, and doubling the training data does not change that -- a committee of both confirms only 0.5% of otherwise-unlabelled products. They are usable for ranking candidates, which is what a reward needs, though even there neighbouring edits of one source are much harder (0.735 on Egc) than random polymer pairs (0.898).

The delta models were an attempt to fix that by predicting changes instead of absolute values, using 62x more supervision. Neighbour discrimination did not move (0.734), and as a selector they are worse than v2 (10.26% vs 13.54%). Kept for reproducibility.

Usage

git clone https://github.com/promotion-kim/POLYEDIT
python scripts/eval_polyedit_llm.py \
  --base_model meta-llama/Llama-3.2-3B-Instruct \
  --adapter <path>/adapters/2x-rules \
  --answer rules --decoding rule_first --interval_scale 2 \
  --name run --out run.json

--decoding rule_first applies the model's own rule to the source with RDKit rather than trusting the product string it emits, which removes the retrieval route entirely: every valid product is then an edit by construction.

Data

Derived from a DFT-labelled matched-molecular-pair corpus of 95,186 pairs over 2,328 polymers (Egc, Egb, Eea, Ei, EPS, Nc, Xc, Eat). The corpus itself is not redistributed here.