Update README.md

README.md

@@ -12,37 +12,29 @@ should probably proofread and complete it, then remove this comment. -->
 
 # ProGemma2
 
-This model is a fine-tuned version of [JuIm/ProGemma2](https://huggingface.co/JuIm/ProGemma2) on an unknown dataset.
+This is a custom configuration of Google’s Gemma 2 LLM that is being pre-trained on amino acid sequences of 512 AA or less in length. Periodic updates are made to this page as training reaches new checkpoints.
 
-## Model description
+The purpose of this model was to investigate the differences between ProGemma and ProtGPT (GPT-2 architecture) as it pertains to sequence generation. As of 8.22.2024, ProGemma has been trained on ~80% of the training dataset and is still on epoch 1. Training loss is ~2.6. Perplexity scores as well as AlphaFold 3’s ptm, pLDDT, and iptm scores are generally in line with ProtGPT’s scores for sequence lengths < 250, although the testing phase is still very early. I have yet to do testing for sequence lengths > 250. More robust testing is also required for lengths < 250 AA. In my very preliminary testing, HHblit e-values of ~0.1 are achieved with relatively easily.
 
-More information needed
+Controlled generation is not a capability of this model, and therefore serves as a method to significantly improve generation as, in principal, a sequence that performs a given function or resides in a particular cellular location can be generated.
 
-## Intended uses & limitations
+In sequence generation, a top_k of 950 appears to work well as it prevents repetition. This is also seen in ProtGPT.
 
-More information needed
+Below is code using the Transformers library to generate sequences using ProGemma.
 
-## Training and evaluation data
+from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
 
-More information needed
+model = AutoModelForCausalLM.from_pretrained("JuIm/ProGemma")
 
-## Training procedure
+tokenizer = AutoTokenizer.from_pretrained("JuIm/Amino-Acid-Sequence-Tokenizer")
 
-### Training hyperparameters
+progemma = pipeline("text-generation", model=model, tokenizer=tokenizer)
 
-The following hyperparameters were used during training:
-- learning_rate: 0.001
-- train_batch_size: 2
-- eval_batch_size: 8
-- seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.4
-- training_steps: 3500
-
-### Training results
+sequence = progemma("\<bos>", top_k=950, max_length=100, num_return_sequences=1, do_sample=True, repetition_penalty=1.2, eos_token_id=21, pad_token_id=22, bos_token_id=20)
 
+s = sequence[0]['generated_text']
 
+print(s)
 
 ### Framework versions