| --- |
| license: apache-2.0 |
| base_model: meta-llama/Llama-3.2-3B-Instruct |
| tags: |
| - polymer |
| - matched-molecular-pairs |
| - chemistry |
| - lora |
| --- |
| |
| # PolyEdit artifacts |
|
|
| Models behind the results in [POLYEDIT](https://github.com/promotion-kim/POLYEDIT) |
| (`results/results.md`). The task: given a polymer, a property and a target interval, |
| return a single-step matched-pair edit that lands the property inside the interval. |
|
|
| ## adapters/ |
|
|
| LoRA on `meta-llama/Llama-3.2-3B-Instruct` (rank 32, alpha 64, all-linear, 3 epochs, |
| 7,143 steps, assistant-only loss). Splits are component-based 80/20 over PolymerGraphID with |
| cross-split pairs removed. `training_meta.json` in each folder carries the full configuration. |
|
|
| | Folder | Training target | Interval | Arm | |
| |---|---|---|---| |
| | `1x-product` | `{product, rules}` | 1x | A (free-form); B is this decoded rule-first | |
| | `1x-rules` | `{rules}` | 1x | C (rule-only) | |
| | `2x-product` | `{product, rules}` | 2x | A/B, headline numbers | |
| | `2x-rules` | `{rules}` | 2x | C, headline numbers | |
|
|
| `edit_success` on the held-out split (44,492 prompts, 917 source polymers, clustered CI): |
|
|
| | Arm | 1x train / 1x eval | 2x train / 2x eval | |
| |---|---:|---:| |
| | A. free-form | 9.08% | 12.64% [11.70, 13.48] | |
| | B. rule-first | 9.27% | **14.26% [13.44, 15.01]** | |
| | C. rule-only | 7.08% | 11.36% [10.55, 12.16] | |
|
|
| `edit_success` requires both that the property lands in the interval *and* that the product is |
| genuinely an edit of the source. Scoring the property alone credits a model for retrieving a |
| memorised polymer: 40% of the free-form model's apparent successes were exactly that. |
|
|
| ## verifiers/ |
|
|
| TransPolymer and polyBERT fine-tuned as property regressors. Each `.json` holds held-out |
| metrics. |
|
|
| | Folder | What | Note | |
| |---|---|---| |
| | `v1` | absolute values, MMP corpus only (1,375 polymers) | superseded | |
| | `v2` | absolute values, plus external per-property labels (2,696) | **current** | |
| | `delta` | property *changes* over 144,932 matched pairs | negative result | |
|
|
| **These cannot be used to judge success.** Their RMSE is 1.1-12.4x the interval they would have |
| to decide membership of, and doubling the training data does not change that -- a committee of |
| both confirms only 0.5% of otherwise-unlabelled products. They are usable for *ranking* |
| candidates, which is what a reward needs, though even there neighbouring edits of one source |
| are much harder (0.735 on Egc) than random polymer pairs (0.898). |
|
|
| The `delta` models were an attempt to fix that by predicting changes instead of absolute |
| values, using 62x more supervision. Neighbour discrimination did not move (0.734), and as a |
| selector they are worse than `v2` (10.26% vs 13.54%). Kept for reproducibility. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/promotion-kim/POLYEDIT |
| python scripts/eval_polyedit_llm.py \ |
| --base_model meta-llama/Llama-3.2-3B-Instruct \ |
| --adapter <path>/adapters/2x-rules \ |
| --answer rules --decoding rule_first --interval_scale 2 \ |
| --name run --out run.json |
| ``` |
|
|
| `--decoding rule_first` applies the model's own rule to the source with RDKit rather than |
| trusting the product string it emits, which removes the retrieval route entirely: every valid |
| product is then an edit by construction. |
|
|
| ## Data |
|
|
| Derived from a DFT-labelled matched-molecular-pair corpus of 95,186 pairs over 2,328 polymers |
| (Egc, Egb, Eea, Ei, EPS, Nc, Xc, Eat). The corpus itself is not redistributed here. |
|
|
