Upload 6 files

README.md

@@ -1,3 +1,61 @@
----
-license: bsd-3-clause
----
+---
+tags:
+- chemistry
+- molecular-design
+- transformer
+- generative-model
+- predictive-model
+license: bsd-3-clause
+datasets:
+- GuacaMol
+- ZINC
+- MoleculeNet
+gated: true
+extra_gated_fields:
+  Organization: text
+  Intended use: text
+  Contact person: text
+  E-mail: text
+  Country: country
+  Date: date_picker
+  I agree to use this model only for purposes that are non-malicious and ethically responsible: checkbox
+  I have read and accept the BSD 3-Clause license: checkbox
+---
+
+# Hyformer
+
+Hyformer is a joint transformer-based model that unifies a generative decoder with a predictive encoder. Depending on the task, Hyformer uses either a causal or a bidirectional mask, outputting token probabilities or predicted property values.
+
+## Model Details
+- **Paper:** [Synergistic Benefits of Joint Molecule Generation and Property Prediction](https://arxiv.org/abs/2504.16559)
+- **Authors:** Adam Izdebski, Jan Olszewski, Pankhil Gawade, Krzysztof Koras, Serra Korkmaz, Valentin Rauscher, Jakub M. Tomczak, Ewa Szczurek
+- **License:** BSD 3-Clause
+- **Repository:** [https://github.com/szczurek-lab/hyformer](https://github.com/szczurek-lab/hyformer)
+
+## Model checkpoints
+- **[Hyformer_molecules_8M](https://huggingface.co/SzczurekLab/hyformer_molecules_8M):** Trained on GuacaMol dataset ([Brown et al., 2019](https://jcheminf.biomedcentral.com/articles/10.1186/s13321-019-0351-9))
+- **[Hyformer_molecules_50M](https://huggingface.co/SzczurekLab/hyformer_molecules_50M):** Trained on 19M molecules from ZINC, ChEMBL, and other purchasable molecular datasets ([Zhou et al., 2023](https://openreview.net/forum?id=1pPpKc9wR0Y))
+- **[Hyformer_peptides_34M](https://huggingface.co/SzczurekLab/hyformer_peptides_34M):** Trained on 3.5M general-purpose and antimicrobial peptides
+- **[Hyformer_peptides_34M_MIC](https://huggingface.co/SzczurekLab/hyformer_peptides_34M_MIC):** `Hyformer_peptides_34M` jointly fine-tuned on minimal inhibitory concentration values (MIC) against E. coli bacteria
+
+## Gated Access
+This model is available with **gated access**. To request access, please use the Hugging Face gated request form.
+
+## Citation
+If you use this model, please cite:
+
+```
+@misc{izdebski2025synergisticbenefitsjointmolecule,
+      title={Synergistic Benefits of Joint Molecule Generation and Property Prediction}, 
+      author={Adam Izdebski and Jan Olszewski and Pankhil Gawade and Krzysztof Koras and Serra Korkmaz and Valentin Rauscher and Jakub M. Tomczak and Ewa Szczurek},
+      year={2025},
+      eprint={2504.16559},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2504.16559}, 
+}
+```
+
+## References
+- Brown, Nathan, et al. "GuacaMol: benchmarking models for de novo molecular design." Journal of chemical information and modeling, 2019.
+- Zhou, Gengmo, et al. "Uni-mol: A universal 3d molecular representation learning framework." ICLR, 2023.

ckpt.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd2084a900ef19cc01f8b18dbfc5be564993363ae50d8dcbbec47f3d094f5045
+size 204798416

model_config.json

@@ -0,0 +1,9 @@
+{
+  "model_type": "Hyformer",
+  "embedding_dim": 512,
+  "num_attention_heads": 8,
+  "num_transformer_layers": 8,
+  "vocab_size": 34,
+  "prediction_task_type": "regression",
+  "num_prediction_tasks": 1
+}

tokenizer_config.json

@@ -0,0 +1,4 @@
+{
+    "tokenizer_type": "AATokenizer",
+    "vocabulary_path": "data/vocabulary/aa.txt"
+}

trainer_config.json

@@ -0,0 +1,23 @@
+{
+    "batch_size": 64,
+    "learning_rate": 0.0001,
+    "weight_decay": 0.01,
+    "max_epochs": 60,
+    "tasks": {
+        "prediction": 0.6,
+        "lm": 0.4
+    },
+    "compile": true,
+    "enable_ddp": false,
+    "dtype": "float32",
+    "num_workers": 16,
+    "beta1": 0.9,
+    "beta2": 0.95,
+    "gradient_accumulation_steps": 1,
+    "grad_clip": 1.0,
+    "decay_lr": true,
+    "log_interval": 10,
+    "save_interval": 5,
+    "min_lr": 1e-06,
+    "warmup_iters": 54
+}

vocab.txt

@@ -0,0 +1,27 @@
+T
+I
+F
+L
+V
+E
+G
+A
+S
+Y
+Q
+C
+W
+H
+R
+K
+M
+D
+N
+P
+B
+U
+Z
+X
+O
+-
+.