Update README.md

README.md

@@ -4,4 +4,34 @@ metrics:
 - accuracy
 tags:
 - chemistry
----
+---
+# Molecular BERT Pretrained Using ChEMBL Database
+
+This model has been pretrained based on the methodology outlined in the paper [Pushing the Boundaries of Molecular Property Prediction for Drug Discovery with Multitask Learning BERT Enhanced by SMILES Enumeration](https://spj.science.org/doi/10.34133/research.0004). While the original model was initially trained using custom code, it has been adapted for use within the Hugging Face Transformers framework in this project.
+
+## Model Details
+The model architecture utilized is based on BERT. Here are the key configuration details:
+
+```
+BertConfig(
+    vocab_size=len(tokenizer_pretrained.vocab),
+    hidden_size=256,
+    num_hidden_layers=8,
+    num_attention_heads=8,
+    intermediate_size=1024,
+    hidden_act="gelu",
+    hidden_dropout_prob=0.1,
+    attention_probs_dropout_prob=0.1,
+    max_position_embeddings=max_seq_len,
+    type_vocab_size=1,
+    pad_token_id=tokenizer_pretrained.vocab["[PAD]"],
+    position_embedding_type="absolute"
+)
+```
+
+## Pretraining Database
+The model was pretrained using data from the ChEMBL database, specifically version 33. You can download the database from [ChEMBL](https://ftp.ebi.ac.uk/pub/databases/chembl/ChEMBLdb/latest/).
+
+## Performance
+The accuracy score achieved by the pretrained model is 0.9672. The testing dataset used for evaluation constitutes 10% of the ChEMBL dataset.
+