Create README.md

README.md

@@ -0,0 +1,73 @@
+---
+datasets:
+- Mir-2002/python-google-style-docstrings
+language:
+- en
+metrics:
+- bleu
+- rouge
+base_model:
+- Salesforce/codet5p-220m-bimodal
+pipeline_tag: summarization
+tags:
+- code
+---
+
+# Overview
+
+This is a fine tuned CodeT5+ (220m) bimodal model tuned on a dataset consisting of 59,000 Python code-docstring pairs. The docstrings are in Google style format.
+A google style docstring is formatted as follows:
+```
+<Description of the code>
+
+Args:
+<var1> (<data-type>) : <description of var1>
+<var2> (<data_type>) : <description of var2>
+
+Returns:
+<var3> (<data-type>) : <description of var3>
+
+Raises:
+<var4> (<data-type>) : <description of var4>
+```
+
+For more information on my dataset, please see the included referenced dataset.
+
+# Hyperparameters
+
+MAX_SOURCE_LENGTH = 256
+MAX_TARGET_LENGTH = 128
+BATCH_SIZE = 16
+NUM_EPOCHS = 35
+LEARNING_RATE = 3e-5
+GRADIENT_ACCUMULATION_STEPS = 4
+EARLY_STOPPING_PATIENCE = 2
+WEIGHT_DECAY = 0.01
+OPTIMIZER = ADAFACTOR
+LR_SCHEDULER = LINEAR
+
+# Loss
+
+On the 35th epoch, the model achieved the following loss:
+
+Epoch	Training Loss	Validation Loss
+26	1.001400	1.288712
+27	0.983600	1.284895
+28	0.961300	1.277680
+29	0.940600	1.275018
+30	0.933600	1.275621
+31	0.918200	1.270074
+32	0.904700	1.268874
+33	0.908800	1.268534
+34	0.900600	1.268240
+*35*	*0.894800*	*1.268536*
+
+# BLEU and ROUGE Scores
+
+==================================================
+EVALUATION RESULTS
+==================================================
+BLEU Score: 0.3540
+ROUGE-1: 0.5855
+ROUGE-2: 0.3946
+ROUGE-L: 0.5243