Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,65 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: sshleifer/tiny-gpt2
|
| 4 |
+
tags:
|
| 5 |
+
- pasta-finetune
|
| 6 |
+
- knatware
|
| 7 |
+
- p
|
| 8 |
+
- causal
|
| 9 |
+
datasets:
|
| 10 |
+
- stanfordnlp/imdb
|
| 11 |
+
library_name: peft
|
| 12 |
+
pipeline_tag: text-generation
|
| 13 |
---
|
| 14 |
+
|
| 15 |
+
# knatware/knat
|
| 16 |
+
|
| 17 |
+
A sshleifer/tiny-gpt2 model fine-tuned with the Parameterised Efficiency (PEFT / LoRA) method, generated by the PASTA fine-tuning workflow.
|
| 18 |
+
|
| 19 |
+
## Model Description
|
| 20 |
+
|
| 21 |
+
This model was produced with the **Parameterised Efficiency (PEFT / LoRA)** method (PASTA framework) starting from the base model [`sshleifer/tiny-gpt2`](https://huggingface.co/sshleifer/tiny-gpt2), fine-tuned on a sample of the [`stanfordnlp/imdb`](https://huggingface.co/datasets/stanfordnlp/imdb) dataset.
|
| 22 |
+
|
| 23 |
+
- **Task type:** causal
|
| 24 |
+
- **Library:** peft
|
| 25 |
+
- **Generated by:** the PASTA fine-tuning Colab notebook generator
|
| 26 |
+
|
| 27 |
+
## Intended Uses & Limitations
|
| 28 |
+
|
| 29 |
+
This model was fine-tuned on a small sample for demonstration purposes. It has **not** been evaluated at scale and should not be used in production or safety-critical settings without further training, evaluation, and review. Behaviour is inherited from the base model and the (small) fine-tuning sample, and may reflect biases present in either.
|
| 30 |
+
|
| 31 |
+
## Training Procedure
|
| 32 |
+
|
| 33 |
+
### Hyperparameters
|
| 34 |
+
|
| 35 |
+
| Hyperparameter | Value |
|
| 36 |
+
|---|---|
|
| 37 |
+
| Method | Parameterised Efficiency (PEFT / LoRA) |
|
| 38 |
+
| Base model | `sshleifer/tiny-gpt2` |
|
| 39 |
+
| Dataset | `stanfordnlp/imdb` (`train[:200]`) |
|
| 40 |
+
| Epochs | 1 |
|
| 41 |
+
| Batch size | 4 |
|
| 42 |
+
| Learning rate | 0.0005 |
|
| 43 |
+
| Max steps | 20 |
|
| 44 |
+
| LoRA rank | 4 |
|
| 45 |
+
| LoRA alpha | 8 |
|
| 46 |
+
|
| 47 |
+
### Framework versions
|
| 48 |
+
|
| 49 |
+
See the `!pip install` cell in the training notebook for the exact package set used.
|
| 50 |
+
|
| 51 |
+
## How to Get Started
|
| 52 |
+
|
| 53 |
+
```python
|
| 54 |
+
from transformers import pipeline
|
| 55 |
+
|
| 56 |
+
gen = pipeline("text-generation", model="knatware/knat")
|
| 57 |
+
gen("Your prompt here")
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
## Testing
|
| 61 |
+
|
| 62 |
+
Before being pushed, this model was tested locally with a sample inference call, and was re-loaded and tested again directly from the Hub after pushing to confirm the upload was complete and usable.
|
| 63 |
+
|
| 64 |
+
---
|
| 65 |
+
© Knatware Technology UK. Developed by Kayode Okosi.
|