---
tags:
- chemistry
- cheminformatics
- materials-science
- astra
- transformers
- gpt2
- text-generation
license: mit
---

# ASTRA: Initial SMILES Generator

This model is part of the **ASTRA (Advanced Solvation Transformer for Rational Additives)** framework. 
It is a GPT-2 based Causal Language Model (CausalLM) trained on a large-scale dataset of canonical SMILES strings. While this model has learned the fundamental grammar and structural rules necessary to generate chemically valid molecules autonomously, it has not yet been optimized for specific quantum chemical properties. 

This model serves as the starting point (pre-generator) for the ASTRA Active Learning loop.

## Model Details
- **Architecture:** GPT-2 (CausalLM)
- **Stage:** Initial (Before Active Learning)
- **Task:** Autoregressive SMILES Generation
- **Data:** Large-scale unlabelled canonical SMILES (~4M -> ~260K)

## Usage
```python
from astra.tokenizer import initial_gpt_tokenizer_with_vocabulary
from astra.model import GPTForCausalLM

# Load the tokenizer and model
vocab_file = "./vocab.txt"
tokenizer = initial_gpt_tokenizer_with_vocabulary(path=vocab_file)

pretrained_path = "<path/to/pretrained/model>" # Replace with your actual paths
model = GPTForCausalLM.from_pretrained(pretrained_gpt_path)
```

## More Information
For more details on data preparation, downstream fine-tuning, and the full active learning loop, please visit our [GitHub Repository](https://github.com/wkdghdus23/astra).