Update README.md
Browse files
README.md
CHANGED
|
@@ -5,9 +5,7 @@ tags:
|
|
| 5 |
---
|
| 6 |
|
| 7 |
|
| 8 |
-
```markdown
|
| 9 |
# 🧬 MoLLaMA-Small
|
| 10 |
-
|
| 11 |
MoLLaMA-Small is a lightweight LLaMA-based causal language model (57.2M parameters) trained from scratch to generate valid chemical molecules using SMILES strings.
|
| 12 |
|
| 13 |
This model uses DeepChem's `SmilesTokenizer` and was trained on a combined dataset of ZINC15 and MuMOInstruct. It is designed for unconditional molecule generation.
|
|
@@ -31,11 +29,11 @@ A custom, scaled-down LLaMA architecture was used to optimize for chemical langu
|
|
| 31 |
* **Max Position Embeddings**: 1024
|
| 32 |
|
| 33 |
## 🚀 How to Use
|
| 34 |
-
|
| 35 |
You can easily load this model using the standard `transformers` library. The model generates SMILES strings by prompting it with the `[bos]` (Beginning of Sequence) token.
|
| 36 |
|
| 37 |
### Prerequisites
|
| 38 |
Make sure you have the required libraries installed:
|
|
|
|
| 39 |
```bash
|
| 40 |
pip install transformers torch deepchem
|
| 41 |
|
|
@@ -86,6 +84,4 @@ print(f"Generated SMILES: {generated_smiles}")
|
|
| 86 |
* **Batch Size**: 512 (with gradient accumulation steps of 4)
|
| 87 |
* **Learning Rate**: 1e-4 (Cosine scheduler, 10% Warmup)
|
| 88 |
* **Precision**: bf16 (Mixed Precision)
|
| 89 |
-
* **Early Stopping Patience**: 5 epochs
|
| 90 |
-
|
| 91 |
-
```
|
|
|
|
| 5 |
---
|
| 6 |
|
| 7 |
|
|
|
|
| 8 |
# 🧬 MoLLaMA-Small
|
|
|
|
| 9 |
MoLLaMA-Small is a lightweight LLaMA-based causal language model (57.2M parameters) trained from scratch to generate valid chemical molecules using SMILES strings.
|
| 10 |
|
| 11 |
This model uses DeepChem's `SmilesTokenizer` and was trained on a combined dataset of ZINC15 and MuMOInstruct. It is designed for unconditional molecule generation.
|
|
|
|
| 29 |
* **Max Position Embeddings**: 1024
|
| 30 |
|
| 31 |
## 🚀 How to Use
|
|
|
|
| 32 |
You can easily load this model using the standard `transformers` library. The model generates SMILES strings by prompting it with the `[bos]` (Beginning of Sequence) token.
|
| 33 |
|
| 34 |
### Prerequisites
|
| 35 |
Make sure you have the required libraries installed:
|
| 36 |
+
|
| 37 |
```bash
|
| 38 |
pip install transformers torch deepchem
|
| 39 |
|
|
|
|
| 84 |
* **Batch Size**: 512 (with gradient accumulation steps of 4)
|
| 85 |
* **Learning Rate**: 1e-4 (Cosine scheduler, 10% Warmup)
|
| 86 |
* **Precision**: bf16 (Mixed Precision)
|
| 87 |
+
* **Early Stopping Patience**: 5 epochs
|
|
|
|
|
|