File size: 2,041 Bytes
37d156a e6d2fe2 37d156a e6d2fe2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
license: apache-2.0
base_model: sshleifer/tiny-gpt2
tags:
- pasta-finetune
- knatware
- p
- causal
datasets:
- stanfordnlp/imdb
library_name: peft
pipeline_tag: text-generation
---
# knatware/knat
A sshleifer/tiny-gpt2 model fine-tuned with the Parameterised Efficiency (PEFT / LoRA) method, generated by the PASTA fine-tuning workflow.
## Model Description
This model was produced with the **Parameterised Efficiency (PEFT / LoRA)** method (PASTA framework) starting from the base model [`sshleifer/tiny-gpt2`](https://huggingface.co/sshleifer/tiny-gpt2), fine-tuned on a sample of the [`stanfordnlp/imdb`](https://huggingface.co/datasets/stanfordnlp/imdb) dataset.
- **Task type:** causal
- **Library:** peft
- **Generated by:** the PASTA fine-tuning Colab notebook generator
## Intended Uses & Limitations
This model was fine-tuned on a small sample for demonstration purposes. It has **not** been evaluated at scale and should not be used in production or safety-critical settings without further training, evaluation, and review. Behaviour is inherited from the base model and the (small) fine-tuning sample, and may reflect biases present in either.
## Training Procedure
### Hyperparameters
| Hyperparameter | Value |
|---|---|
| Method | Parameterised Efficiency (PEFT / LoRA) |
| Base model | `sshleifer/tiny-gpt2` |
| Dataset | `stanfordnlp/imdb` (`train[:200]`) |
| Epochs | 1 |
| Batch size | 4 |
| Learning rate | 0.0005 |
| Max steps | 20 |
| LoRA rank | 4 |
| LoRA alpha | 8 |
### Framework versions
See the `!pip install` cell in the training notebook for the exact package set used.
## How to Get Started
```python
from transformers import pipeline
gen = pipeline("text-generation", model="knatware/knat")
gen("Your prompt here")
```
## Testing
Before being pushed, this model was tested locally with a sample inference call, and was re-loaded and tested again directly from the Hub after pushing to confirm the upload was complete and usable.
---
© Knatware Technology UK. Developed by Kayode Okosi.
|