Update README.md

README.md

@@ -12,37 +12,25 @@ should probably proofread and complete it, then remove this comment. -->
 
 # ProGemma
 
-This model is a fine-tuned version of [JuIm/ProGemma](https://huggingface.co/JuIm/ProGemma) on an unknown dataset.
+This is a custom configuration of Google's Gemma 2 model that is being pre-trained on amino acid sequences of lengths 0 to 512. Updates are made regularly as the model hits new checkpoints. As of 08.10.2024, the model has been trained on about 40% of the dataset.
 
-## Model description
+The model generates amino acids on a letter-by-letter basis.
 
-More information needed
+Current training loss is about 2.7. Preliminary evaluation of generated sequences on AlphaFold 3 shows pTM scores of ~0.4 and average pLLDT scores ~60. After training is complete, a proper evaluation will be done to see whether sequences result in proteins with a low free energy. Current perplexity scores are on par with NVIDIA's ProtGPT2. 
 
-## Intended uses & limitations
+The purpose of this model was to see whether I could develop an alternative to NVIDIA's ProtGPT2. ProGemma also serves as a stepping stone to a new model that will also utilize control tags to generate proteins based on function.
 
-More information needed
+To use this mode for yourself using the pipeline within the Transformers package, please see the code below:
 
-## Training and evaluation data
+from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
 
-More information needed
+model = AutoModelForCausalLM.from_pretrained("JuIm/ProGemma") tokenizer = AutoTokenizer.from_pretrained("JuIm/Amino-Acid-Sequence-Tokenizer")
 
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.001
-- train_batch_size: 1
-- eval_batch_size: 8
-- seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.4
-- training_steps: 7000
-
-### Training results
+progemma = pipeline("text-generation", model=model, tokenizer=tokenizer)
 
+sequence = progemma("\<bos>", top_k=950, max_length=100, num_return_sequences=1, do_sample=True, repetition_penalty=1.2, eos_token_id=21, pad_token_id=22, bos_token_id=20)
 
+print(sequence)
 
 ### Framework versions