Random-initialized Rem3Di + COSMO: ten epochs, fifty BigSolDB probes
Completed 13/50 planned fifth-epoch frozen Kumo Large probes. Every learned Rem3Di preprocessor, encoder, denoising decoder and COSMO head started from its constructor initialization. No published Rem3Di or PCQM/COSMO checkpoint initializes this run. MACE-POLAR-1-M alone is pretrained and frozen. The encoder/decoder architecture and 640-dimensional pooled descriptor match the preceding experiments.
Two simultaneous streams: standard author atomic-embedding denoising on the retained PCQM500k structures, and 153 sigma-bin areas plus A/V reconstruction on the retained original PM6 COSMO structures. Each epoch covers both complete training streams; the shorter stream cycles. No CombiSolv head or covariance/decorrelation penalty is included.
The released denoising implementation, Gaussian sigma 0.3 and its original gradient paths are preserved. In particular the clean atomic reconstruction target remains differentiable, as in the author implementation. Raw and per-channel denoising losses, held-out descriptor variability and target variability are logged; low reconstruction loss alone is not treated as downstream quality.
Task uncertainty weighting uses sum(L/(2 exp(s)) + s/2). The unmodified raw denoising loss has 1088 channel terms, so its initial log variance is ln(1088)−0.5; COSMO starts at −0.5. This is mathematically equivalent, up to an additive constant, to starting both log variances at −0.5 with per-channel denoising units. All invariant feature normalization and COSMO log1p/ln target standardization were freshly fit using training rows only.
Held-out BigSolDB solute connectivity groups are excluded from both new training streams. PCQM and COSMO validation groups are also excluded across objectives. The pretrained MACE foundation can have prior exposure; this experiment makes no claim about removing its historical pretraining data. Exact memberships and all original coordinates are preserved in the prepared dataset.
Every fifth-epoch checkpoint is saved with all heads, task weights, AdamW state, OneCycle scheduler and RNG state, and verified against its Hugging Face manifest before probing. Probes run in separate processes and never change the training RNG, optimizer or schedule. There are exactly five probes per epoch, fifty total; the epoch-10 endpoint is the predeclared final checkpoint, and all intermediate test scores are reported without test-based tuning.
| Point | Epoch position | MAE | RMSE | R² |
|---|---|---|---|---|
| 1 | 0.20000 | 0.595141 | 0.778008 | 0.533688 |
| 2 | 0.40000 | 0.610717 | 0.789840 | 0.519397 |
| 3 | 0.60000 | 0.611143 | 0.786464 | 0.523497 |
| 4 | 0.80000 | 0.596699 | 0.779846 | 0.531483 |
| 5 | 1.00000 | 0.582410 | 0.757501 | 0.557947 |
| 6 | 1.20000 | 0.593677 | 0.775478 | 0.536716 |
| 7 | 1.40000 | 0.610561 | 0.789466 | 0.519853 |
| 8 | 1.60000 | 0.605837 | 0.777111 | 0.534763 |
| 9 | 1.80000 | 0.595558 | 0.778459 | 0.533148 |
| 10 | 2.00000 | 0.605233 | 0.784106 | 0.526350 |
| 11 | 2.20000 | 0.603681 | 0.794870 | 0.513256 |
| 12 | 2.40000 | 0.623854 | 0.807604 | 0.497536 |
| 13 | 2.60000 | 0.613172 | 0.800946 | 0.505786 |
Prior matched probes and literature values
| Model | Origin | MAE | RMSE | R² |
|---|---|---|---|---|
| Epoch 0: PCQMep2 → COSMOep10 + Kumo L | Prior matched predictions, revalidated | 0.594936 | 0.783643 | 0.526909 |
| Published Rem3Di POLAR + Kumo L | New matched frozen probe | 0.595827 | 0.757843 | 0.557547 |
| Monroe + Kumo L | Prior matched predictions, revalidated | 0.577354 | 0.745340 | 0.572026 |
| XGBoost-DFT | Published Solvaformer Table 1 (not rerun) | 0.621000 | 0.806000 | 0.499000 |
| XGBoost-CFP | Published Solvaformer Table 1 (not rerun) | 0.749000 | 0.943000 | 0.315000 |
| XGBoost-MMB | Published Solvaformer Table 1 (not rerun) | 0.710000 | 0.912000 | 0.360000 |
| SolvBERT | Published Solvaformer Table 1 (not rerun) | 0.871000 | 1.110000 | 0.052000 |
| MPNN | Published Solvaformer Table 1 (not rerun) | 0.668000 | 0.864000 | 0.425000 |
| EquiformerV2 | Published Solvaformer Table 1 (not rerun) | 0.691000 | 0.926000 | 0.345000 |
| Solvaformer | Published Solvaformer Table 1 (not rerun) | 0.643000 | 0.837000 | 0.460000 |
| MPNN+Charges | Published Solvaformer Table 1 (not rerun) | 0.629000 | 0.817000 | 0.486000 |
All measured probes use exactly the released 9,686 test rows and the 75,731-row non-test complement as full Kumo context. Fixed shared baseline conformers, 1281 ordered solute640/solvent640/temperature-in-Kelvin features, Kumo Large v1.0.1, four estimators, seed 42, fp16, query chunks 128. No additional transformation of the supplied log10 mol/L labels. Monroe uses its full 1441 features. Literature rows are copied reported values, not rerun models. The author prediction file has 9,686 rows and reproduces Table 1 metrics; the prose says 9,250. The private neural validation subset is unavailable.
Prepared data · Checkpoints · Probe W&B · Solvaformer Table 1
- Downloads last month
- -