Tonylmz/ParentDopingFlowFinal-checkpoints

ParentDopingFlowFinal Checkpoints

This repository stages the checkpoints and benchmark tables selected for external release.

Start here

Read PAPER_DRAFT_FLOWX_20260829.md. It is the complete current write-up: task, dataset construction, method, comparison methods, evaluation protocol, results, negative results, and four explicit retractions. Every number in it is tagged [5-fold] or [single]; do not quote a [single] number as validated.

Two results carry the paper:

1. checkpoints/flowx_dpo_warmup20_ep60_last.pt does both jobs at once -- xMAE 0.1495 against a 0.1808 parent baseline, direction 0.697, and a scaffold matching the parent in 39 of 39 held-out cases at 2.0% lattice error. This is one run on one split.
2. checkpoints/kfold_f{0..4}_tcfree_best.pt pool over 785 movable held-out cases to direction 0.715 (p=1.8e-34, sd 0.018) and xMAE 0.1879 against 0.2034. This is the validated number, but it validates a slightly different recipe than result 1 -- see the draft's limitations section.

There are three releases in this repository.

2026-08-28: flow-x, the doping amount as a generated variable

Read RESULTS_FLOWX_DPO_20260828.md, then TECHNICAL_REFERENCE.md sections 15.13-15.40. Use checkpoints/flowx_dpo_idw05_heldtc_ep12_last.pt: it is the only flow-x checkpoint whose stage-1 backbone never saw a Tc label of an evaluated structure.

Here x has no coordinate of its own. Per-site composition is the atom-type sphere row squared and normalised, so x is read out of the generated structure rather than predicted by a head, and a requested x enters only as soft conditioning.

Two capabilities, and they do not hold at once:

1. Follow a requested x. checkpoints/flowx_only_pretrained_last.pt reaches a realized-x span of 0.458 median / 0.922 max on held-out axes, 8 of 9 above 0.1. This needs a late checkpoint: the span is 0.000 at epoch 15 and 0.508 at 300, so never select for it by val_loss.
2. Generate the optimal x unprompted. The main checkpoint scores generated-x MAE 0.1832 against a 0.1808 parent baseline with direction accuracy 0.646 (p=1.3e-2) -- a tie on the aggregate, but 11 of 12 wins on the cases that need a move above 0.35.

Training for both at once destroys the first and drops the second to chance: checkpoints/flowx_dpo_unified_ep300_last.pt has span 0.068 and direction 0.506 (p=1.0). That checkpoint is included as the negative result.

Read the four |parent_x - optimum_x| bands, not the aggregate. Opposite effects cancel in the mean, which is how the preference term and the best-anchor pairing were both misjudged once already.

2026-08-15: concentration, edit selection and construction

Read PROJECT_STATUS.md first. It is the single current overview: scope, all three boards, contributions, limits and remaining work.

Use checkpoints/joint_v2_parentgroup_frozen_ep300.pt. It is trained with a parent-grouped split, so no parent compound appears on both sides of the holdout.

What this delivers

Given a parent structure and a set of candidate doping edits: pick the optimal concentration (mean optimal-x error 0.0122; 0.0135 on an unseen family), pick the best edit (regret 3.20 K, about half of random), and construct the chosen edit into a full structure with site occupancies (100% composition validity on oxides against a 99.6% real-data control).

What it does not deliver

Proposing edits absent from the dataset. Three independent routes fail: the CSP trunk copies atom types verbatim so the flow cannot generate composition; a learned proposer has 155 training examples over 35 parents against ~930 classes; and 88.6% of enumerated proposals have no known Tc, so they cannot be scored at all. See PROJECT_STATUS.md section 5.

Three things that are easy to misread

1. The dome metric fluctuates by up to 0.029 rho between epochs of the same run, so smaller differences are not real.
2. rho and top-1 already sit at the label-noise limit; only regret has headroom, and 89% of it is on parents whose best edit is multi-channel.
3. Under a protocol that permits memorisation a random forest on composition reaches rho 0.776 against this model's 0.745. It is nearest-neighbour retrieval (r=0.987), and on a held-out family it collapses to -0.142 while this model holds 0.839. Always quote the cross-family number alongside.

A finetune must set atom_type_manifold=null_manifold to match the pretrain. Mismatching it silently collapses the doping-axis metric from 0.730 to 0.229 while the training curves still look healthy.

Files marked SUPERSEDED are the July release, measured on a split where 91.7% of the benchmark cases were training data; they are kept only for reproducibility.

Included artifacts

File	Role	Notes
ABLATION_CONCLUSION.md	ablation_verdict	Which components are necessary and which are not, each with its evidence level. Necessary: FlowDPO, best-anchor pairing, the exact-amount projection, and the frozen warmup together with enough pairs per epoch. NOT necessary: the structural pretrain, amount resampling and consistency, the geometry anchor, the CSP rehearsal branch, Tc-weighted flow, gradient guidance, parent stay-pairs, and the Tc head itself. Two entries were moved out of the necessary list on 2026-08-31 after checking the validated recipe's actual parameters.
review/PROBLEM_ANSWERS_ROUND6.md	figure_review_answers	Latest round of the figure review. Confirms the structure numbers survive the LAPACK guard unchanged, and records that minimum-bond validity is blind to cell distortion: 97% valid at a median 2.40 A bond while the lattice is 199% off.
review/PROBLEM_ANSWERS_ROUND5.md	figure_review_answers_round5	Records that the requested x has almost no effect on generation: a request differing by 0.85 changes the lattice by 4.7e-7 A, so obedience is delivered entirely by the projection rather than by the network.
benchmark/control_freeze0_structure_board.json	freeze_control_structure_board	The single-variable freeze control, re-measured with the LAPACK guard and end-to-end request mode. Differs from the best checkpoint only in the warmup: 2 of 39 scaffolds against 39 of 39, 199.0% lattice error against 2.0%. Also the evidence that minimum-bond validity alone is not a structure metric.
benchmark/negative_beta01_aligned_fold0_board.json	beta01_negative_result	Aligning every knob of the best single run onto a leak-free fold does NOT reproduce it: direction 0.526 (p=0.29) against 0.697. With the structure board beside it (0 of 39 scaffolds, 87.1% lattice) this closes beta=0.1 on the Tc-free backbone on both metrics. Only beta=0.005 has five-fold evidence.
benchmark/negative_beta01_aligned_fold0_structure.json	beta01_negative_structure	Structure half of the beta=0.1 negative result: 0 of 39 scaffolds, 87.1% lattice, 11 of 39 crystals broken by the generator.
PAPER_DRAFT_FLOWX_20260829.md	paper_draft	START HERE. Full draft: task, dataset construction, method, comparison methods, evaluation protocol, results, negative results and four explicit retractions. Every number is tagged [5-fold] or [single]; do not quote a [single] number as validated.
checkpoints/flowx_dpo_warmup20_ep60_last.pt	best_single_x_and_structure	Best single configuration, and the only one that does both jobs at once: xMAE 0.1495 against a 0.1808 parent baseline, direction 0.697 (p=1.3e-4), and a scaffold matching the parent in 39 of 39 held-out cases at 2.0% lattice error and 1.05 deg angle error. The decisive setting is the 20-epoch frozen warmup: the identical recipe without it collapses to 1 of 39 and 114%. Use last.pt -- best.pt lands at epoch 5, inside the frozen phase where flow terms are zero.
checkpoints/kfold_f0_tcfree_best.pt	kfold_pooled_headline_fold0	Fold 0 of the five-fold headline. Pooled over 785 movable held-out cases the five best.pt checkpoints give direction 0.715 (p=1.8e-34, sd 0.018) and xMAE 0.1879 against a 0.2034 parent baseline. The backbone is Tc-free so no fold's test labels reached it. Per-fold direction 0.545 / 0.590 / 0.737 / 0.907 / 0.783 -- that spread is why single runs cannot be trusted here.
checkpoints/kfold_f1_tcfree_best.pt	kfold_pooled_headline_fold1	Fold 1 of the five-fold headline. See fold 0 for the pooled numbers.
checkpoints/kfold_f2_tcfree_best.pt	kfold_pooled_headline_fold2	Fold 2 of the five-fold headline. See fold 0 for the pooled numbers.
checkpoints/kfold_f3_tcfree_best.pt	kfold_pooled_headline_fold3	Fold 3 of the five-fold headline; the strongest fold, direction 0.907.
checkpoints/kfold_f4_tcfree_best.pt	kfold_pooled_headline_fold4	Fold 4 of the five-fold headline. See fold 0 for the pooled numbers.
benchmark/flowx_warmup20_ep60_holdout_board.json	best_single_board	Per-case rows for the best single configuration; every draft table is recomputable from these rows without the weights.
benchmark/flowx_warmup20_ep60_structure_board.json	best_single_structure_board	Structure board for the same checkpoint: scaffold 39/39, RMS 0.0006, lattice 2.0%, with the parent's own minimum bond recorded next to the generated one.
benchmark/baseline_x_head_regression_board.json	x_head_regression_baseline	The obvious alternative to generating x, and it is not competitive: direction 0.416 is BELOW chance (p=0.96), generated x 0.3161 and the head's own prediction 0.2992, against parent baselines of 0.1808 and 0.1504.
benchmark/baseline_no_structural_pretrain_board.json	no_pretrain_baseline	No structural pretrain, 30 epochs: direction 0.596 (p=4.5e-2), xMAE 0.1763. Quantifies what the 1840-epoch pretrain buys.
benchmark/control_parent_min_bond.json	split_site_measurement_control	Why absolute minimum-bond validity is not a model verdict: measured on the REAL parents with the same one-atom-per-site convention, 4 of 40 fall below 0.5 A, on the very axes flagged as model failures. Split sites, not model error.
benchmark/two_stage_handoff_board.json	two_stage_handoff_board	The superseded delivery route: this model picks x, the projection writes it in exactly, the structural pretrain builds geometry from a parent-initialised prior. Scaffold 10/12, lattice 7.4%, x error 0.0000. Superseded because one model now does both and scores higher.
benchmark/control_csp_reconstruction_composition_fixed.json	csp_protocol_valid_control	Control for the retracted CSP claim. A composition-FIXED checkpoint scores 0.825 with 1.35% lattice error, reproducing the 0.803 that was previously mis-attributed to a composition-generating pretrain.
benchmark/control_csp_reconstruction_composition_generated.json	csp_protocol_invalid_artifact	The same protocol on a composition-GENERATING checkpoint: match rate 0.000. Not a geometry verdict -- a random prior discards the given formula, 0 of 14 species survive on a Y123 cell. eval_csp_reconstruction.py now refuses this configuration by default.
RESULTS_FLOWX_DPO_20260828.md	flowx_dpo_entry_point	START HERE for flow-x. Main board on 89 non-degenerate held-out cases, the four
TECHNICAL_REFERENCE.md	engineering_and_retraction_log	Full mechanism and measurement log, sections 15.13-15.40. Includes every retracted conclusion and, in 15.38, the measurement mistakes that produced them. Read 15.39 before quoting any controllability number.
checkpoints/flowx_dpo_idw05_heldtc_ep12_last.pt	flowx_dpo_main_checkpoint	Main flow-x + FlowDPO checkpoint, and the only one whose backbone never saw a Tc label of the evaluated structures (structure_csp_pretrain_2000_heldtc, 113/113 coverage). identity_flow_weight 0.5. Generated-x MAE 0.1832 against a 0.1808 parent baseline, direction accuracy 0.646 (p=1.3e-2), rho +0.652, TcRegret 10.8 K; bands 8.89 / 1.65 / 1.07 / 0.50 with 11/12 wins on the cases needing a move above 0.35. It ties the parent baseline overall and wins where a large move is required -- do not quote it as beating the baseline on the aggregate.
checkpoints/flowx_dpo_idw05_leakybackbone_best.pt	flowx_dpo_leaky_backbone_reference	Same recipe warm-started from structure_csp_leakfree_pretrain, whose holdout list was built for a different board and covers only 73/113 test structures. Its better-looking xMAE 0.1611 and direction 0.700 are the reference for how much backbone Tc leakage is worth, not a result.
checkpoints/flowx_dpo_sampleonly_best.pt	flowdpo_sample_only_ablation	FlowDPO ablation: preference term on sampled pairs only. Kept because the aggregate board hides the effect -- the contribution is visible only in the band stratification.
checkpoints/flowx_dpo_beta005_ep10.pt	flowdpo_beta_ablation	FlowDPO beta ablation at 0.05, epoch 10.
checkpoints/flowx_controllability_from2000_ep300.pt	flowx_controllability_checkpoint	Long-trained flow-x without the preference term. Realized-x span grows with training (0.000 at epoch 15, 0.115 at 200, 0.508 at 300), so controllability must be measured on a late checkpoint and never picked by val_loss.
checkpoints/flowx_only_pretrained_last.pt	flowx_only_holdout_controllability	flow-x without DPO. On held-out axes the realized-x span is 0.458 median / 0.922 max with 8 of 9 axes above 0.1 and obey-MAE 0.329: requested x is genuinely followed on unseen axes.
checkpoints/flowx_dpo_unified_ep300_last.pt	flowx_dpo_unified_conflict_checkpoint	The negative result that closes the joint objective: adding DPO collapses controllability from 0.458 to 0.068 and direction accuracy to 0.506 (p=1.0) with every band above 1.29. Controllability and optimum-finding are not simultaneously achievable in this design.
checkpoints/structure_csp_pretrain_2000_heldtc_snapshot.pt	stage1_paired_backbone	Stage-1 warm start paired with the flow-x holdout: excludes all 465 ids of stage1_merged_holdout.json, so it covers 113/113 test structures. Any stage-2 run must use this, not structure_csp_leakfree_pretrain.
checkpoints/structure_csp_pretrain_2000_tcfree_last.pt	stage1_tc_free_backbone	Split-agnostic stage-1 backbone: never saw a Tc label at all. Required for k-fold, where no Tc-supervised pretrain covers every fold (stage1_merged reaches only 65-87% per fold).
metadata/flowx_idw05_heldtc_split_summary.json	flowx_idw05_heldtc_split_summary	Pair-row split for the main flow-x checkpoint; the holdout removes whole doping axes.
benchmark/flowx_dpo_idw05_heldtc_holdout_board.json	flowx_main_board_holdout	Main board, per-case rows, clean backbone. Read with the band stratification, not the aggregate.
benchmark/flowx_dpo_idw05_leakybackbone_holdout_board.json	flowx_board_leaky_backbone	Same board for the leaky-backbone run; the delta against the file above is the leakage cost.
benchmark/flowx_dpo_sampleonly_holdout_board.json	flowdpo_sampleonly_board	Sample-only FlowDPO ablation board.
benchmark/flowx_dpo_beta02_holdout_board.json	flowdpo_beta_board	beta 0.2 FlowDPO ablation board.
benchmark/flowx_dpo_unified_holdout_board.json	flowdpo_unified_board	Board for the joint controllability+optimum objective; direction accuracy is at chance.
benchmark/flowx_dpo_parent_pair_comparison.json	flowdpo_parent_pair_comparison	Including parent stay-pairs in the preference set is worse everywhere: only 9 such pairs survive a 5 K gap. A correct but sparse preference signal hurts.
benchmark/flowx_controllability_holdout.json	flowx_controllability_holdout	Realized-x span as the requested x sweeps 0->1, on held-out axes. Span, not rho: rho reaches +1.000 on a span of 0.001.
benchmark/flowx_controllability_late_epochs.json	flowx_controllability_epoch_trend	Controllability against epoch: 0.000 / 0.115 / 0.508 at 15 / 200 / 300.
benchmark/flowx_controllability_unified.json	flowx_controllability_after_dpo	Same probe on the unified run: span collapses to 0.068.
benchmark/flowx_sweep_enforce_amount_contrast.json	superseded_sweep_method_contrast	The superseded method: sweep requested x with exact-amount projection and rerank by the Tc head. xMAE 0.4372 against a 0.2355 baseline while the oracle over the same candidates is 0.0119, so the bottleneck there is entirely the Tc head. Kept as the contrast that motivates generating x directly.
benchmark/flowx_amount_unknown_board.json	flowx_amount_unknown_board	Board with --amount_unknown: no requested x is given, the model must generate it. This is the deliverable metric.
PROJECT_STATUS.md	project_status_entry_point	START HERE. Scope, all three boards, contributions, limits, artifacts.
checkpoints/joint_v2_propose_v1_ep300.pt	propose_mode_experiment_checkpoint	Trained with --mark_identity_condition and a learnable propose token to let generation choose the edit itself. The token did learn (norm 0.71) but behaviour is unchanged, because the CSP trunk copies atom types verbatim and cannot generate composition. Kept as the negative result.
benchmark/composition_baselines_leaky.json	composition_baselines_leaky	RF/SVR/Ridge on composition only. Under the memorisation-permitting protocol RF reaches rho 0.776 vs the flow model's 0.745; it must be reported alongside the cross-family result below.
benchmark/composition_baselines_cross_family.json	composition_baselines_cross_family	With cuprate_Y123 removed from baseline training, RF/SVR/Ridge score rho -0.142 / 0.054 / -0.005 on the 89 held-out Y123 series against the flow model's 0.839. RF is nearest-neighbour retrieval (r=0.987 vs its nearest training formula's Tc), so it has nothing to retrieve here.
benchmark/composition_validity.json	composition_validity	Charge-neutrality validity, oxides only. Conditioned construction 100%, real-data control 99.6%, unconditioned generation 66.7%. The criterion is a false negative for the intermetallic families.
benchmark/rf_vs_flow_scorer.json	rf_vs_flow_scorer	Same candidate pool, two scorers. RF picks the right channel slightly more often, which shows the discovery board cannot separate discovery from retrieval rather than that RF is better at discovery.
benchmark/de_novo_proposal.json	de_novo_proposal	Unconditioned generation returns the parent unchanged for 5 of 6 parents; the one exception is chemically impossible (O5Ba1Pb1).
benchmark/enumerate_construct_score.json	enumerate_construct_score	Enumerate, construct, score. 88.6% of top picks are edits absent from the data and therefore unscoreable: only 4 of 35 parents yield a regret.
benchmark/alignment_tolerance_ladder.json	alignment_coverage_limit_ladder	Why board B covers only 30 parents. Loose tolerances rescue 98% at the fit stage but those correspondences carry median 0.354 fractional drift.
benchmark/regret_attribution.json	regret_attribution	20 of 30 parents are solved exactly; multi-channel optima carry 89% of the remaining regret.
benchmark/channel_synergy.json	channel_synergy	With the primary channel fixed, true delta Tc still spans 13.59 K and the per-group
benchmark/ranking_ep300_mc2.json	ranking_board_ep300	Board B for all three splits plus four trivial baselines, all at epoch 300.
checkpoints/joint_v2_parentgroup_frozen_ep300.pt	delivery_checkpoint_leakage_free	Primary delivery checkpoint. Frozen backbone, parent-grouped split (0% parent overlap between train and val), epoch 300. Dome benchmark rho 0.745 / peak_x_err 0.0122 / regret 1.32 K.
checkpoints/joint_v2_familyholdout_y123_ep300.pt	cross_family_checkpoint	Cross-family run: every cuprate_Y123 doping pair held out of training. Scores rho 0.839 on the 89 held-out Y123 series vs 0.840 for the model trained on them.
checkpoints/joint_v2_pairrow_frozen_ep300.pt	leaky_split_reference_checkpoint	Same recipe on the historical pair-row split, kept as the leakage reference: rho 0.742 vs 0.745, i.e. removing leakage costs nothing.
checkpoints/structure_csp_all_tcfix_v2_best_tc.pt	structure_pretrain_warm_start	CSP-conditioned pretrain all three runs warm-start from. Trained with atom_type_manifold=null_manifold; a finetune must match that or the doping-axis metric collapses from 0.730 to 0.229.
metadata/joint_v2_parentgroup_split_summary.json	parentgroup_split_summary	Parent-grouped split, 0% parent overlap.
metadata/joint_v2_familyholdout_split_summary.json	familyholdout_split_summary	Family holdout split; val contains only cuprate_Y123.
benchmark/DELIVERABLE_BENCHMARK_AND_CONTRIBUTIONS_20260812.md	current_deliverable	Current benchmark definitions, results and contributions. Start here.
benchmark/MEASUREMENT_FIXES_AND_NEXT_ACTIONS_20260807.md	measurement_investigation_log	Full investigation log, including every retracted conclusion.
benchmark/dome_parentgroup_ep300_summary.json	dome_benchmark_summary	Benchmark A result for the delivery checkpoint (213 series).
benchmark/ranking_ep300_mc2.json	ranking_benchmark_summary	Benchmark B for all three splits plus four trivial baselines.
benchmark/label_noise_floor.json	label_noise_floor	Label-noise floor. rho 0.745 and top-1 0.700 match a perfect estimator limited by ~2 K label noise, so those metrics are already at the label resolution limit.
benchmark/channel_synergy.json	channel_synergy_ablation	Multi-channel synergy: with the primary channel fixed, true delta Tc still spans 13.59 K and the per-group
benchmark/regret_attribution.json	regret_attribution	20 of 30 parents are solved exactly; multi-channel optima carry 89% of the remaining regret.
benchmark/alignment_tolerance_ladder.json	alignment_coverage_limit	Why benchmark B covers only 30 parents: loose tolerances rescue 98% at the fit stage, but those correspondences carry median 0.354 fractional drift and are correctly rejected.
checkpoints/joint_v2_hardfam_preserve_best_tc.pt	main_guided_replay_checkpoint	SUPERSEDED (2026-08-13): measured on a leaky split. Kept for reproducibility.
checkpoints/joint_v2_with_others_best_tc.pt	baseline_checkpoint	Main baseline for guided replay comparison.
checkpoints/joint_v2_hardfam_v1_best.pt	hardfam_ablation_checkpoint	Hard-family weighted ablation checkpoint.
checkpoints/structure_csp_no_others_last.pt	structure_pretrain_checkpoint	Optional structure pretrain initialization checkpoint.
metadata/joint_v2_hardfam_preserve_pair_row_split_summary.json	preserve_split_summary	Train/val split summary for the preserve run.
metadata/joint_v2_hardfam_v1_pair_row_split_summary.json	hardfam_v1_split_summary	Train/val split summary for the hardfam v1 run.
metadata/joint_v2_with_others_pair_row_split_summary.json	baseline_split_summary	Train/val split summary for the baseline run.
benchmark/FINAL_BENCHMARK_TABLE_20260805.md	final_benchmark_markdown	SUPERSEDED (2026-08-13): all numbers are training-set scores. See DELIVERABLE_*.md.
benchmark/overall.csv	overall_benchmark_csv	Overall benchmark table in CSV.
benchmark/by_family.csv	family_benchmark_csv	Per-family benchmark table in CSV.

Recommended checkpoints

flow-x (2026-08-28):

• Generate the optimal x: checkpoints/flowx_dpo_idw05_heldtc_ep12_last.pt
• Follow a requested x: checkpoints/flowx_only_pretrained_last.pt
• Stage-1 warm start: checkpoints/structure_csp_pretrain_2000_heldtc_snapshot.pt, or checkpoints/structure_csp_pretrain_2000_tcfree_last.pt for k-fold

Concentration / edit selection (2026-08-15):

• Primary delivery checkpoint: checkpoints/joint_v2_parentgroup_frozen_ep300.pt
• Unseen family: checkpoints/joint_v2_familyholdout_y123_ep300.pt
• Leakage reference: checkpoints/joint_v2_pairrow_frozen_ep300.pt

Benchmark

flow-x boards are the benchmark/flowx_*.json files; read them alongside RESULTS_FLOWX_DPO_20260828.md. The three older boards are in benchmark/ranking_ep300_mc2.json, benchmark/dome_parentgroup_ep300_summary.json and benchmark/enumerate_construct_score.json, described in PROJECT_STATUS.md.