---
license: apache-2.0
base_model: meta-llama/Llama-3.2-3B-Instruct
tags:
  - polymer
  - matched-molecular-pairs
  - chemistry
  - lora
---

# PolyEdit artifacts

Models behind the results in [POLYEDIT](https://github.com/promotion-kim/POLYEDIT)
(`results/results.md`). The task: given a polymer, a property and a target interval,
return a single-step matched-pair edit that lands the property inside the interval.

## adapters/

LoRA on `meta-llama/Llama-3.2-3B-Instruct` (rank 32, alpha 64, all-linear, 3 epochs,
7,143 steps, assistant-only loss). Splits are component-based 80/20 over PolymerGraphID with
cross-split pairs removed. `training_meta.json` in each folder carries the full configuration.

| Folder | Training target | Interval | Arm |
|---|---|---|---|
| `1x-product` | `{product, rules}` | 1x | A (free-form); B is this decoded rule-first |
| `1x-rules` | `{rules}` | 1x | C (rule-only) |
| `2x-product` | `{product, rules}` | 2x | A/B, headline numbers |
| `2x-rules` | `{rules}` | 2x | C, headline numbers |

`edit_success` on the held-out split (44,492 prompts, 917 source polymers, clustered CI):

| Arm | 1x train / 1x eval | 2x train / 2x eval |
|---|---:|---:|
| A. free-form | 9.08% | 12.64% [11.70, 13.48] |
| B. rule-first | 9.27% | **14.26% [13.44, 15.01]** |
| C. rule-only | 7.08% | 11.36% [10.55, 12.16] |

`edit_success` requires both that the property lands in the interval *and* that the product is
genuinely an edit of the source. Scoring the property alone credits a model for retrieving a
memorised polymer: 40% of the free-form model's apparent successes were exactly that.

## verifiers/

TransPolymer and polyBERT fine-tuned as property regressors. Each `.json` holds held-out
metrics.

| Folder | What | Note |
|---|---|---|
| `v1` | absolute values, MMP corpus only (1,375 polymers) | superseded |
| `v2` | absolute values, plus external per-property labels (2,696) | **current** |
| `delta` | property *changes* over 144,932 matched pairs | negative result |

**These cannot be used to judge success.** Their RMSE is 1.1-12.4x the interval they would have
to decide membership of, and doubling the training data does not change that -- a committee of
both confirms only 0.5% of otherwise-unlabelled products. They are usable for *ranking*
candidates, which is what a reward needs, though even there neighbouring edits of one source
are much harder (0.735 on Egc) than random polymer pairs (0.898).

The `delta` models were an attempt to fix that by predicting changes instead of absolute
values, using 62x more supervision. Neighbour discrimination did not move (0.734), and as a
selector they are worse than `v2` (10.26% vs 13.54%). Kept for reproducibility.

## Usage

```bash
git clone https://github.com/promotion-kim/POLYEDIT
python scripts/eval_polyedit_llm.py \
  --base_model meta-llama/Llama-3.2-3B-Instruct \
  --adapter <path>/adapters/2x-rules \
  --answer rules --decoding rule_first --interval_scale 2 \
  --name run --out run.json
```

`--decoding rule_first` applies the model's own rule to the source with RDKit rather than
trusting the product string it emits, which removes the retrieval route entirely: every valid
product is then an edit by construction.

## Data

Derived from a DFT-labelled matched-molecular-pair corpus of 95,186 pairs over 2,328 polymers
(Egc, Egb, Eea, Ei, EPS, Nc, Xc, Eat). The corpus itself is not redistributed here.