Instructions to use Delower/molmistral-foundation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Keras
How to use Delower/molmistral-foundation with Keras:
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://Delower/molmistral-foundation") - Notebooks
- Google Colab
- Kaggle
MolMistral β SMILES foundation model (job 794 checkpoint, REPAIRED)
Foundation (pretrained, disease-agnostic) weights for a Mistral-style SMILES language model: 12 layers, 12 query heads, 1 KV head, hidden dim 512, intermediate dim 768, 21,831,680 parameters, DeepChem 2.4.0 SMILES vocabulary (621 tokens). Pretrained on a ~3.83M-molecule mixture (MOSES + ChEMBL + ZINC + PubChem, decontaminated against the DPP-4 fine-tune set).
File: best_foundation.keras β this is the file to fine-tune from for a
new target/disease. Do NOT fine-tune from any DPP-4 fine-tuned checkpoint;
those are DPP-4-specific.
β οΈ Defect found and repaired β read before use
This checkpoint comes from SPARCq job 794, which FAILED with NaN losses
after ~1 clean pretraining epoch of 10. A pre-submission tensor scan of the
saved best-val checkpoint (pretrain val_loss 0.2959) found exactly 2 NaN
values out of 21,831,680 model weights, both in the token-embedding row
for [C@H] (row 33, columns 357 and 415) β plus 12 further NaN/Β±Inf values
at the same two coordinates in the saved AdamW optimizer-state buffers.
[C@H] occurs in 41% of typical fine-tune sequences, so the unmodified
file poisons fine-tuning β it is the likely reason job 794's own fine-tune
stage went NaN from its first epoch.
All 14 entries were repaired in a copy by copying the values at the same
columns from the [C@@H] row (row 35, the stereoisomeric counterpart
token) within each tensor:
- weights [33, 357]: NaN β 0.003635966684669256
- weights [33, 415]: NaN β -0.008388345129787922
- optimizer buffers: 12 values patched identically (see
REPAIR_NOTES.md)
The repaired file was then re-scanned over the whole file (152,821,762
values): 0 non-finite values, and it reproduces the original
checkpoint's held-out validation loss (0.272177 vs 0.272176 on 256 rows).
Full details: REPAIR_NOTES.md, gate_scan.json (original scan),
repaired_gate_scan.json (post-repair scan).
Hashes
| File | Size (bytes) | SHA-256 |
|---|---|---|
best_foundation.keras (this repo, REPAIRED) |
611,953,638 | a27c247f9f064017f04b1d5c480a9f7800ce1745cbce77e8e5a8a28d72ce4f68 |
| Job 794 original (NOT uploaded, defective) | 611,953,638 | 8fa4e0f349e910158552f82d3c559859ddb39c9758cdd4600c74d53090dd4494 |
The unmodified original was deliberately not uploaded.
Caveat
The foundation saw only ~1 clean epoch of its planned 10-epoch pretraining before job 794 failed, so its quality is unverified pending a fine-tune test. Treat downstream results accordingly.
- Downloads last month
- 36