lowdown-labs/fela-chemistry

Disclaimer

This model is a research preview. Some of the datasets it was trained on carry copyleft or attribution terms (in particular ChEMBL is ShareAlike, see "Training data" below), so review those terms before any commercial use. This model has not been validated in a laboratory environment, and any inference it produces is for demonstration only. Lowdown Labs built it in the interest of advancing public science.

FELA-Chem: a small chemistry model that runs on a plain CPU

FELA-Chem reads a molecule written as a SMILES string and gives you a number back. Hand it aspirin and ask whether it looks toxic, how soluble it is, or how a reaction might score for yield, and it answers. It also ranks which known reaction templates could build a target you want to make, and that ranking drives its route planner.

The whole thing runs on your own laptop or a CPU box. No GPU, no cloud. A screening pass or a route search calls the model thousands of times, so keeping it small and on your machine keeps it cheap. One thing to be clear about up front: it scores and ranks chemistry that already exists. It does not invent new molecules, so nothing new comes out as SMILES, only numbers and rankings.

What goes in, what comes out

• Input: a molecule as a SMILES string, the standard text form of a molecular graph. Aspirin, for instance, is CC(=O)Oc1ccccc1C(=O)O. For the reaction tasks (yield, retrosynthesis) you pass a full reaction SMILES: a dot between co reactants, and >> between the reactant and product sides. Inside, the model reads the string one atom or bond at a time over a window of up to 128 tokens.
• Output depends on which task head you load:
  • property classification: one probability per endpoint, say toxic or not, or blood brain barrier permeable or not. Take the probability, or the argmax for a label.
  • property regression: a real value in the property's own units, like log solubility in mol/L for ESOL, or predicted yield in percent for a reaction.
  • retrosynthesis template ranking: a ranked list of reaction templates with probabilities, which the route planner uses to decide what to try next.
• In practice that covers three things you might do without touching the cloud: triage a compound library for toxicity and ADMET style endpoints, estimate a reaction's yield from its full context, and drive a retrosynthesis planner that works backward from a target to buyable building blocks.

Why we built it this way

A normal attention model builds a table that grows with the square of the input length, which is what makes big sequence models hungry. The Fourier operator has no such table. Its working memory stays a fixed size no matter how long the molecule, so the model stays small, tens of millions of parameters, and stays cheap to call. That matters most for a route planner, which scores dozens of candidate steps per target and would run up a real bill on a per call API.

To train it, we first show the model a large pile of unlabeled SMILES and have it fill in hidden pieces of each string, so it learns the grammar of chemistry on its own. We also show it several valid ways to write the same molecule. Then we fine tune a light task head on top. It is the same recipe that worked for the FELA genomics model.

Performance

This model runs on CPU. The architecture is CPU native by design, with an FFT global convolution plus a fixed size state recall layer, no GPU kernel needed for inference.

The retrosynthesis planner ran CPU only, calling the 33M parameter policy as its expansion model. Mean search time was 10.9 seconds per target and median 3.5 seconds per target, at a mean of 43.7 policy expansions per target. A planner is the heaviest CPU consumer of this model (it scores many candidate states per target), so a single property or yield score is well below those numbers.

Sizes (fp32 on disk, approximate):

Variant	Params	fp32 disk	Device class
10M (property, smallest)	9.86M	about 39 MB	laptop or small CPU
33M (property default)	33.5M	about 134 MB	laptop or CPU server
116M (best retro and yield)	116M	about 464 MB	CPU server
286M (flagship backbone)	286.34M	about 1.1 GB	CPU server

The FNO plus recall backbone could export to ONNX for smaller CPU or ARM targets. The 286M flagship backbone was pretrained on 122M unique SMILES (PubChem plus ChEMBL) to a final masked token accuracy of 0.9613.

The downstream results are the reaction yield below (R2 0.960), the USPTO full 18,240 class template ranker (top 1 0.94, top 10 0.99), and the planner re run on that ranker. The flagship property scores are not reported here; the property tables cite the validated 33M and 116M results.

Accuracy

The short version - on property prediction FELA-Chem is competitive with the well known models and wins outright on a couple of tasks, while trailing the largest pretrained models on the hardest ones, with our current training budget.

On reaction yield it matches Yield-BERT. As a route planner it clears the standard open tool AiZynthFinder but sits below the strong academic planner retro* on retro*'s own targets.

All numbers are measured with the protocol stated per task. Property prediction uses the MoleculeNet datasets (Wu et al. 2018) with scaffold splits, which hold out molecules built on novel scaffolds and are the harder, more honest way to test generalization.

We keep a validation split for model selection and report test only for the best validation config, with no tuning on test. Baselines are the published numbers for the named models.

Property classification, ROC-AUC (higher is better)

Benchmark	FELA-Chem (best variant)	Named baselines
BACE	0.872 (FNO+GLA 33M)	D-MPNN 0.852, Uni-Mol 0.857, MolFormer-XL 0.882
ClinTox	0.894 (FNO+GLA 10M, mean of 4 seeds, best of 4 = 0.907)	Uni-Mol 0.919, ChemBERTa-2 0.563, MolFormer-XL 0.948
Tox21	0.784 (GDN 116M)	ChemBERTa-2 0.749, Uni-Mol 0.796, MolFormer-XL 0.847
SIDER	0.629 (GDN 33M)	Uni-Mol 0.659, MolFormer-XL 0.690
BBBP	0.750 (GDN 33M)	D-MPNN 0.886, MolFormer-XL 0.937
HIV	0.768 (FNO 65M)	Uni-Mol 0.808, MolFormer-XL 0.822

Property regression, RMSE (lower is better)

Benchmark	FELA-Chem (best variant)	Named baselines
ESOL (log solubility)	0.816 (FNO+GLA 33M)	ChemBERTa-2 0.889, Uni-Mol 0.788, MolFormer-XL 0.279
Lipophilicity	0.642 (GDN 33M)	ChemBERTa-2 0.758, Uni-Mol 0.603, MolFormer-XL 0.529
FreeSolv	2.248 (FNO+GLA 33M)	ChemBERTa-2 1.349, Uni-Mol 1.480, MolFormer-XL 0.231

Retrosynthesis template ranking, USPTO-50k (the planner policy)

The model ranks which of the USPTO-50k reaction templates apply to a product. This is the expansion policy of the route planner. Protocol: USPTO-50k template classification, top k accuracy on the held out split, templates extracted with rdchiral.

System	Top 1	Top 10
neuralsym (baseline)	0.443	0.787
GLN (graph baseline)	0.526	0.838
LocalRetro (context)	0.534	not reported
FELA-Chem FNO+GLA 33M	0.485	0.782
FELA-Chem GDN 33M	0.529	0.833
FELA-Chem GDN 116M	0.542	0.848

The planner needs a wider policy than the USPTO-50k 1,472 class ranker. The flagship policy scales this to the full USPTO template library, an 18,240 class ranker (305M), evaluated on its held out validation split (top k accuracy, templates extracted with rdchiral):

System	Top 1	Top 10
FELA-Chem USPTO full ranker (18,240 class, 305M)	0.94	0.99

This is the policy the planner below uses.

Reaction yield, Buchwald-Hartwig HTE (full reaction context)

Protocol: the Ahneman et al. 2018 Buchwald-Hartwig high throughput experimentation set, the standard FullCV_01 70/30 split, full reaction context input (aryl halide, amine, ligand, base, additive as one reaction SMILES), the same feature set Yield-BERT uses.

Benchmark	FELA-Chem (GDN 286M)	FELA-Chem (GDN 116M)	Baseline
Buchwald-Hartwig HTE yield, test R2	0.960	0.956	Yield-BERT about 0.95
Buchwald-Hartwig HTE yield, test RMSE (yield %)	5.486	5.73	Yield-BERT band about 5 to 8

The 286M flagship edges Yield-BERT on the same full reaction context input and FullCV_01 split.

Retrosynthesis planning (system level), USPTO held out products

The template ranker drops in as the expansion policy of a retro star style AND-OR tree search; rdchiral applies each ranked template as the operator (enforcing valid chemistry); the search composes routes and stops when every leaf is a molecule in the public AiZynthFinder ZINC in stock set (17.4M buyable molecules).

The protocol we adopted was to run 500 USPTO held out products that the policy never saw, ZINC stock, CPU only. We report two policies on the identical planner and the identical 500 targets, changing only the policy, which isolates the effect of policy strength.

Policy	Solved rate	Context baseline
Strong 18,240 class ranker (top 10 0.99)	about 0.89 with search budget; about 0.80 (393 of 500) at a conservative 30 second per target limit	retro* about 0.86 to 0.88 (own hard target set); AiZynthFinder about 0.55 to 0.70
This planner on retro*'s own 190 hard targets (retro*'s stock)	0.058 (11 of 190)	retro* about 0.86 to 0.88 (same 190 targets)
Weak frozen probe policy (top 10 0.659, ablation)	0.456 strict / 0.553 lenient	AiZynthFinder about 0.55 to 0.70 (same ZINC stock)

With the strong 18,240 class ranker the planner solves about 0.89 of held out USPTO targets with a search budget, and holds about 0.80 (393 of 500) at a conservative 30 second per target limit, both on our held out USPTO set with the ZINC stock. This clears the standard open planner AiZynthFinder (about 0.55 to 0.70) on the same ZINC stock, CPU only. We do not claim a same benchmark tie with the strong academic planner retro* (an ICML 2020 neural guided A* planner at about 0.86 to 0.88 on its own hard target benchmark): run like for like on retro*'s own 190 hard targets with retro*'s own stock, our planner solves only 5.8 percent (11 of 190), well below retro*, and the gap is a missing learned value function, which is a scoped next step if we can continue work on FELA-chem in the future.

The solved rate is sensitive to search budget, the 30 second per target limit is a reproducible operating point and not a hard ceiling, the rate rises with more compute per target, which is itself the point. A cheap policy buys more search per dollar.

The jump from 0.456 to about 0.80 comes from changing only the policy (weak probe to strong 18,240 class ranker) on the same planner and test set, which confirms that policy template recall, not search budget, was the binding constraint. The real advantage, as with other FELA models, seems to be the solution rate delivered CPU only and on premises.

Loading with standard tooling

The repo ships config.json (the architecture hyperparameters and per variant metadata) and a self contained modeling.py with load_model / from_pretrained and scoring helpers. A few lines load the model from a local directory or a Hugging Face repo:

from modeling import load_model, score_smiles
m = load_model("/path/to/weights_dir")                   # dir with the safetensors + config.json + tokenizer.json
label, prob = score_smiles(m, "CC(=O)Oc1ccccc1C(=O)O")   # aspirin -> (label, probability)

For the retrosynthesis policy checkpoint:

from modeling import load_model, rank_templates
policy = load_model("/path/to/weights_dir", variant="ranker")
ranked = rank_templates(policy, "CCC(C)N(C)C(=O)c1cc2ccccc2c(-c2ccccc2F)n1", top_k=10)
# -> [(template_class_id, probability), ...] for the planner

For a property head (classification or regression) or the reaction yield head:

from modeling import load_model, predict_property
tox = load_model("/path/to/weights_dir", variant="tox21")
print(predict_property(tox, "CC(=O)Oc1ccccc1C(=O)O"))   # per endpoint probabilities

esol = load_model("/path/to/weights_dir", variant="esol")
print(predict_property(esol, "CC(=O)Oc1ccccc1C(=O)O"))   # predicted log solubility (mol/L)

y = load_model("/path/to/weights_dir", variant="yield")
print(predict_property(y, "Brc1ccccc1.Nc1ccccc1>>c1ccc(Nc2ccccc2)cc1"))  # predicted yield (percent)

Every task ships its own safetensors weights and, where needed, its own tokenizer, all selected by the variant name. The shipped variants are: foundation (encoder), ranker (retrosynthesis template ranking), synth (synthesizability), cond_heads (reaction conditions), the nine property heads (bace, bbbp, clintox, sider, tox21, hiv, esol, freesolv, lipo), and yield (reaction yield). The property and yield heads are the exact fine tuned checkpoints behind the accuracy tables above, so each reported number is reproducible from the weights in this repo.

The weights ship as safetensors (not pickle).

How to run it

FELA-Chem is a small suite, not a single file. Each capability (the nine property heads, reaction yield, the retrosynthesis ranker, synthesizability, and reaction conditions) has its own variant in config.json, and you load the one you need with variant=. Point FELA_CHEM_WEIGHTS at the directory that holds the weights (with its config.json and the tokenizers) and run. The quickstart default scores aspirin with the foundation encoder.

See quickstart/ for a runnable, self contained example:

pip install -r quickstart/requirements.txt
export FELA_CHEM_WEIGHTS=/path/to/weights_dir
python quickstart/run.py

For an interactive playground, see the Hugging Face Space in space/. For a production serving path, the models run under the FELA server, a separate CPU native serving project (https://github.com/Lowdown-Labs/fela_server).

Formats

• fp32: reference and CPU.
• int8: on device deployment format (AVX512-VNNI on x86, NEON dot product on ARM).
• bf16: server and GPU inference only; most commodity ARM and microcontroller CPUs lack native bf16, so it is not the on device format.

Training data

• Pretraining corpus: ChEMBL 37 (chembl_37_chemreps.txt, the canonical_smiles field), 2,897,819 raw molecules, reduced to 2,872,185 unique valid canonical SMILES after RDKit canonicalization (2 to 120 heavy atoms, up to 250 characters). The 116M backbone adds PubChem (the CID-SMILES release) for a deduplicated 6,539,567 SMILES corpus. The 286M flagship backbone is pretrained on the full PubChem plus ChEMBL corpus, 122,249,322 unique valid canonical SMILES (final masked token accuracy 0.9613). Masked token self supervised pretraining (BERT style 15 percent masking) with SMILES enumeration augmentation (RDKit random SMILES, Bjerrum 2017). ChEMBL is released under CC BY-SA 3.0 (Zdrazil et al. 2024). PubChem records are public domain (NCBI).
• Tokenizer: the atom wise SMILES regex tokenizer (Schwaller et al. 2019), vocab 421 plus an appended mask id (the 116M backbone uses a 686 token vocab). Ship tokenizer.json with the weights so the token to id map matches.
• Property fine tuning and evaluation: MoleculeNet (Wu et al. 2018): BACE, BBBP, ClinTox, Tox21, SIDER, HIV (classification), ESOL, FreeSolv, Lipophilicity (regression), with Bemis-Murcko scaffold splits (Bemis and Murcko 1996), loaded from the DeepChem mirror.
• Retrosynthesis: USPTO-50k reaction templates (Lowe 2012; typed by Schneider et al. 2015), extracted and applied with rdchiral (Coley et al. 2019). The planner stock is the AiZynthFinder public ZINC in stock set, 17,422,831 buyable molecules (Genheden et al. 2020).
• Reaction yield: the Buchwald-Hartwig HTE dataset (Ahneman et al. 2018), FullCV_01 split.
• Reference baselines (we did not retrain them): D-MPNN / Chemprop (Yang et al. 2019), Uni-Mol (Zhou et al. 2023), MolFormer-XL (Ross et al. 2022), ChemBERTa-2 (Ahmad et al. 2022), neuralsym (Segler and Waller 2017), GLN (Dai et al. 2019), LocalRetro (Chen and Jung 2021), Yield-BERT (Schwaller et al. 2021), AiZynthFinder (Genheden et al. 2020), retro star (Chen et al. 2020).
• The GDN variant uses the Gated DeltaNet layer from the flash-linear-attention (fla) library; its Triton kernel is used for training, and CPU inference uses a pure torch recurrence.

Splits and licensing

Per component: the training dataset, the split definition (method, seed, sizes, and the reproducible split function in train.py), the dataset license, and the commercial verdict. Split sizes are reproduced from the training data by python train.py smoke (bare size assertions per component). One dataset carries a commercial use flag; see the note at the end.

• Backbone (masked LM pretrain). ChEMBL 37 (canonical_smiles field) plus, for the larger backbones, PubChem CID-SMILES. Corpus sizes: 33M and 65M backbones = ChEMBL only (2,872,185 unique valid canonical SMILES, 2 to 120 heavy atoms, up to 250 characters); 116M = deduplicated ChEMBL plus PubChem 6,539,567 SMILES; 286M flagship = full PubChem plus ChEMBL 122,249,322 unique valid canonical SMILES. No held out split: self supervised masked LM pretraining consumes the whole corpus (BERT style 15 percent masking, 80/10/10 corrupt, SMILES enumeration augmentation), so there is no train and test line to cite. Licenses: ChEMBL 37 is CC BY-SA 3.0 (Zdrazil et al. 2024); PubChem records are U.S. NCBI public domain. Commercial verdict: permitted with attribution; ChEMBL's ShareAlike is flagged below.
• Property finetunes (MoleculeNet). BACE, BBBP, ClinTox, Tox21, SIDER, HIV (classification), ESOL, FreeSolv, Lipophilicity (regression), from MoleculeNet (Wu et al. 2018), via the DeepChem mirror CSVs. Split: deterministic Bemis-Murcko scaffold split, frac (0.8, 0.1, 0.1), largest scaffold groups to train, defined in train.py scaffold_split. Reproduced test sizes: BACE 152, ClinTox 148, Tox21 783, SIDER 144, HIV 4112, ESOL 114, FreeSolv 65, Lipophilicity 420, BBBP 205. License: MoleculeNet via DeepChem is MIT. Commercial verdict: permitted (research benchmark; underlying assay data is public).
• Reaction yield head (Buchwald-Hartwig HTE). Ahneman et al., Science 2018 high throughput C-N coupling set, via the BH_input.xlsx feature file (Sandfort et al. 2020), sheet FullCV_01, n=3955. Split: the Yield-BERT protocol, a seed 0 70/30 random split (30 percent test), holding 10 percent of train as validation, defined in train.py yield_split, reproduced test size
  1186. Commercial verdict: permitted as a published research dataset; confirm reuse terms of the Science supplementary feature file for redistribution (flagged as verify terms).
• Retrosynthesis template ranker (the planner policy). USPTO reaction corpus (Lowe 1976 to Sep 2016 grants and applications, about 3.75M reactions), retro templates extracted with rdchiral. Flagship 18,240 class ranker: templates kept at min count 10, product classified, seed 0 80/10/10 random split over n=1,027,751 reactions, defined in train.py retro_full_split; reproduced classes 18,240 and val size 102,775 (the held out val used to report top 1 0.94, top 10 0.99). The USPTO-50k gate (1,472 class, min count 5) is the smaller variant. License: USPTO patent reaction data is U.S. public domain. Commercial verdict: permitted (public domain).
• Reaction condition heads (catalyst, solvent, reagent). The same USPTO grants and applications; the agent field is parsed into catalyst, solvent, and reagent by RDKit heuristics, agents stripped from the input so the model predicts conditions. Cap 400,000 reactions, top 50 label vocab per category plus none and other. Split: seed 0 shuffle, 80/10/10, defined in train.py cond_split; reproduced test size 40,000. License: USPTO public domain. Commercial verdict: permitted (public domain).
• Synthesizability head (SCScore style). Real molecules only: ChEMBL drug like SMILES plus USPTO reaction products; the head regresses the RDKit SAScore. Split: seed 0 permutation, 80/10/10, defined in train.py synth_split; on the 50,000 molecule build the test size is 5,000. The headline metric is the test Spearman of the FELA score against SAScore on real molecules: 0.970 on a fresh 8,000 molecule ChEMBL holdout (and 0.9165 on the harder mixed ChEMBL plus USPTO test). License: ChEMBL CC BY-SA 3.0 plus USPTO public domain; SAScore labels are computed with RDKit (BSD). Commercial verdict: permitted with attribution; the ChEMBL ShareAlike flag applies.

Licensing flag (ChEMBL ShareAlike): ChEMBL 37 is released under CC BY-SA 3.0. The ShareAlike term means a derivative of the ChEMBL data must itself be offered under CC BY-SA 3.0 (or a compatible license). ChEMBL SMILES appear in the pretraining corpus (all backbones) and in the synthesizability head's training molecules. Whether trained model weights constitute a derivative work of the underlying data under CC BY-SA is legally unsettled and is a business decision, not a technical one; it is flagged for legal review before any commercial redistribution, and it interacts with this repo's own Lowdown Labs Lovely License 1.0 (CC BY-NC 4.0 plus Hippocratic 3.0). PubChem, MoleculeNet (DeepChem, MIT), and USPTO (public domain) carry no such copyleft. The Buchwald-Hartwig BH_input.xlsx feature file's redistribution terms should be confirmed against the Science 2018 supplementary material.

The reproducible split definitions and size assertions live in train.py; run python train.py smoke (with FELA_CHEM_DATA pointing at the MoleculeNet CSVs and the optional FELA_CHEM_BH / FELA_CHEM_CONDDIR / FELA_CHEM_SYNTHDIR env vars for the reaction datasets) to rebuild each split and check the audited sizes.

Intended use, limitations, and safety

What it is for: research use. CPU, no cloud scoring of molecular properties (toxicity and ADMET style endpoints, solubility, lipophilicity), reaction yield estimation from a full reaction context, ranking which reaction templates could make a target, and serving as the expansion policy of a retrosynthesis planner that proposes routes to buyable building blocks.

What it is not for: it is not a molecule generator and does not invent new SMILES (it scores and ranks existing chemistry). It is not a clinical, diagnostic, or safety decision tool, and it is not a substitute for laboratory validation. Predicted properties, yields, and routes are statistical and must be checked by a chemist and by experiment before any synthesis, dosing, or safety decision.

Privacy: the model runs on the device. Proprietary structures and reactions do not have to leave your machine, which matters for unpublished or competitively sensitive chemistry.

Evaluated conditions and known failure modes:

• Property prediction was evaluated on MoleculeNet with scaffold splits; on the hardest scaffold generalization tasks (BBBP, HIV) and on the very low RMSE regression endpoints (ESOL, FreeSolv, Lipophilicity) the model trails MolFormer-XL and Uni-Mol, which are much larger and use far more pretraining data or 3D conformers. FreeSolv (642 molecules) is the weakest case and is small and noisy.
• The retrosynthesis planner was evaluated on 500 held out USPTO products against the ZINC stock; that set and stock differ from the retro star 190 hard target benchmark and the AiZynthFinder sets, so the solved rate clears the standard open planner AiZynthFinder (0.55 to 0.70) on the same ZINC stock but is not a same benchmark tie with retro* (0.86 to 0.88). On retro*'s own 190 hard targets with retro*'s own stock, our planner solves only 5.8 percent (11 of 190), the gap being a missing learned value function. The solved rate is a reproducible operating point at a 30 second per target limit, not a ceiling.
• Inputs must be valid SMILES; for reaction tasks they must be full reaction SMILES with the reaction context co reactants present (a product only yield input gives an R2 near zero, a feature set error, not a model limit). The loader validates SMILES and fails clearly on malformed input.

How to cite

@misc{lowdownlabs_felachem,
  title  = {FELA-Chem: a small CPU-runnable Fourier Neural Operator model for molecular
            property prediction, reaction scoring, and retrosynthesis planning},
  author = {Lowdown Labs},
  year   = {2026},
  note   = {Model card}
}

You must also cite the datasets, the reference benchmarks, and the libraries. Each entry below gives the reference and, for datasets, the governing license.

Datasets

• ChEMBL 37 (pretraining corpus and the synthesizability head's molecules). License: CC BY-SA 3.0. Zdrazil, B., Felix, E., Hunter, F., et al. (2024). The ChEMBL Database in 2023. Nucleic Acids Research, 52(D1), D1180-D1192. https://doi.org/10.1093/nar/gkad1004 . License text: https://creativecommons.org/licenses/by-sa/3.0/ . Flag: ShareAlike is copyleft; see the licensing flag above.
• PubChem (CID-SMILES release, 116M and 286M backbones). Status: U.S. NCBI public domain. Kim, S., Chen, J., Cheng, T., et al. (2023). PubChem 2023 update. Nucleic Acids Research, 51(D1), D1373-D1380. https://doi.org/10.1093/nar/gkac956 .
• USPTO patent reactions (retro ranker and reaction condition heads). Status: derived from U.S. patent text, public domain. Lowe, D. M. (2012). Extraction of chemical structures and reactions from the literature. PhD thesis, University of Cambridge. https://doi.org/10.17863/CAM.16293 . Dataset: https://doi.org/10.6084/m9.figshare.5104873.v1 . Typing: Schneider, N., Stiefl, N., Landrum, G. A. (2015). JCIM, 55(1), 39-53. https://doi.org/10.1021/ci5006614 .
• Buchwald-Hartwig HTE (reaction yield head). Ahneman, D. T., Estrada, J. G., Lin, S., Dreher, S. D., Doyle, A. G. (2018). Predicting reaction performance in C-N cross coupling using machine learning. Science, 360(6385), 186-190. https://doi.org/10.1126/science.aar5169 . Feature file: Sandfort, F., Strieth-Kalthoff, F., Kuhnemund, M., et al. (2020). Chem, 6(6), 1379-1390. https://doi.org/10.1016/j.chempr.2020.02.017 . Flag: confirm redistribution terms of the feature file (verify terms).
• MoleculeNet (property fine tuning and evaluation). Wu, Z., Ramsundar, B., Feinberg, E. N., et al. (2018). MoleculeNet: A Benchmark for Molecular Machine Learning. Chemical Science, 9, 513-530. https://doi.org/10.1039/C7SC02664A . Loaded via the DeepChem mirror (MIT).
• SAScore basis (synthesizability head's label). Ertl, P., Schuffenhauer, A. (2009). Estimation of synthetic accessibility score. Journal of Cheminformatics, 1, 8. https://doi.org/10.1186/1758-2946-1-8 . Computed with RDKit.
• Scaffold split, buyable stock, augmentation: Bemis, G. W., Murcko, M. A. (1996). J. Med. Chem., 39(15), 2887-2893. https://doi.org/10.1021/jm9602928 . AiZynthFinder ZINC in stock set: Genheden, S., Thakkar, A., Chadimova, V., et al. (2020). J. Cheminformatics, 12, 70 (Apache-2.0). Bjerrum, E. J. (2017). SMILES Enumeration as Data Augmentation. arXiv:1703.07076.

Methods, architecture, and libraries

• Fourier Neural Operator (the FNO sequence mixer): Li, Z., Kovachki, N., Azizzadenesheli, K., et al. (2021). ICLR. arXiv:2010.08895 .
• Gated DeltaNet (the FNO plus GDN recall layer): Yang, S., Kautz, J., Hatamizadeh, A. (2024). Gated Delta Networks: Improving Mamba2 with Delta Rule. arXiv:2412.06464 .
• Gated Linear Attention (the FNO plus GLA recall layer): Yang, S., Wang, B., Shen, Y., Panda, R., Kim, Y. (2024). arXiv:2312.06635 .
• flash-linear-attention (fla): the Gated DeltaNet Triton kernel used for GPU training of the GDN variant; CPU inference uses the pure torch recurrence in modeling.py. https://github.com/fla-org/flash-linear-attention (MIT).
• RDKit (SMILES canonicalization and validation, template extraction and application via rdchiral, SAScore labels): https://www.rdkit.org (BSD-3-Clause). rdchiral: Coley, C. W., Green, W. H., Jensen, K. F. (2019). JCIM, 59(6), 2529-2537. https://doi.org/10.1021/acs.jcim.9b00286 .
• Atom wise SMILES tokenizer: Schwaller, P., Laino, T., Gaudin, T., et al. (2019). Molecular Transformer. ACS Cent. Sci., 5(9), 1572-1583. https://doi.org/10.1021/acscentsci.9b00576 .
• PyTorch: Paszke, A., Gross, S., Massa, F., et al. (2019). NeurIPS. Weights ship as safetensors.

Note: Landmark attention is not used in FELA-Chem. The SSSL layer pattern in config.json interleaves the FNO mixer with a single recall layer (GLA or Gated DeltaNet) every fourth block; there is no landmark or sliding window attention layer in modeling.py.

Reference baselines (cited for comparison, not retrained by us)

• Yield-BERT: Schwaller, P., Vaucher, A. C., Laino, T., Reymond, J.-L. (2021). Mach. Learn.: Sci. Technol., 2, 015016. https://doi.org/10.1088/2632-2153/abc81d .
• LocalRetro: Chen, S., Jung, Y. (2021). JACS Au, 1(10), 1612-1620. https://doi.org/10.1021/jacsau.1c00246 .
• GLN: Dai, H., Li, C., Coley, C., Dai, B., Song, L. (2019). NeurIPS. arXiv:2001.01408 .
• neuralsym: Segler, M. H. S., Waller, M. P. (2017). Chem. Eur. J., 23(25), 5966-5971. https://doi.org/10.1002/chem.201605499 .
• retro star: Chen, B., Li, C., Dai, H., Song, L. (2020). ICML. arXiv:2006.15820 .
• D-MPNN / Chemprop: Yang, K., Swanson, K., Jin, W., et al. (2019). JCIM, 59(8), 3370-3388. https://doi.org/10.1021/acs.jcim.9b00237 .
• Uni-Mol: Zhou, G., Gao, Z., Ding, Q., et al. (2023). ICLR. https://openreview.net/forum?id=6K2RM6wVqKu .
• MolFormer-XL: Ross, J., Belgodere, B., Chenthamarakshan, V., et al. (2022). Nature Machine Intelligence, 4, 1256-1264. https://doi.org/10.1038/s42256-022-00580-7 .
• ChemBERTa-2: Ahmad, W., Simon, E., Chithrananda, S., Grand, G., Ramsundar, B. (2022). arXiv:2209.01712 .

Model family

This is part of the FELA family from Lowdown Labs: one FNO architecture across many modalities, all CPU native and subquadratic. This repo is pushed as lowdown-labs/fela-chemistry. The sibling repos are independently trained per modality and share no weights, so none carries a base_model link.

License

Released under the Lowdown Labs Lovely License 1.0 (CC BY-NC 4.0 plus Hippocratic License 3.0). See LICENSE. For most LL models, a commercial license may be available; contact Lowdown Labs.