v1 done

README.md

@@ -1,3 +1,46 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- bsu-slim/electra-tiny
+pipeline_tag: text-classification
+library_name: transformers
+---
+
+A pretrained [ELECTRA-Tiny](https://huggingface.co/bsu-slim/electra-tiny/tree/main) model. Pretraining [data](https://osf.io/5mk3x) 
+was from the [2024 BabyLM Challenge](https://babylm.github.io/index.html). Used personally to perform text classification 
+on the [Web of Science Dataset WOS-46985](https://data.mendeley.com/datasets/9rw3vkcfy4/6) but this model is not currently fine-tuned
+for that task.
+
+
+# Training
+Used pretraining pipeline as defined in this [repository](https://github.com/bakirgrbic/bblm).
+
+## Hyperparameters
+- Epochs: 1
+- Batch size: 8
+- Learning rate: 1e-4
+- Optimizer: AdamW
+
+## Resources Used
+- Compute: AWS Sagemaker ml.g4dn.xlarge
+- Time: About 7 hours
+
+# Evaluation (Web of Science)
+Used wos pipeline as defined in this [repository](https://github.com/bakirgrbic/bblm).
+
+## Results
+- 64% accuracy on the last epoch of the test set.
+
+## Hyperparameters
+- Epochs: 3
+- Batch size: 64
+- Learning rate: 2e-5
+- Optimizer: AdamW
+- Max Length: 128
+- Parameter Freezing: None
+
+## Resources Used
+- Compute: AWS Sagemaker ml.g4dn.xlarge
+- Time: About 5 minutes