| --- |
| license: apache-2.0 |
| base_model: sshleifer/tiny-gpt2 |
| tags: |
| - pasta-finetune |
| - knatware |
| - p |
| - causal |
| datasets: |
| - stanfordnlp/imdb |
| library_name: peft |
| pipeline_tag: text-generation |
| --- |
| |
| # knatware/knat |
|
|
| A sshleifer/tiny-gpt2 model fine-tuned with the Parameterised Efficiency (PEFT / LoRA) method, generated by the PASTA fine-tuning workflow. |
|
|
| ## Model Description |
|
|
| This model was produced with the **Parameterised Efficiency (PEFT / LoRA)** method (PASTA framework) starting from the base model [`sshleifer/tiny-gpt2`](https://huggingface.co/sshleifer/tiny-gpt2), fine-tuned on a sample of the [`stanfordnlp/imdb`](https://huggingface.co/datasets/stanfordnlp/imdb) dataset. |
|
|
| - **Task type:** causal |
| - **Library:** peft |
| - **Generated by:** the PASTA fine-tuning Colab notebook generator |
|
|
| ## Intended Uses & Limitations |
|
|
| This model was fine-tuned on a small sample for demonstration purposes. It has **not** been evaluated at scale and should not be used in production or safety-critical settings without further training, evaluation, and review. Behaviour is inherited from the base model and the (small) fine-tuning sample, and may reflect biases present in either. |
|
|
| ## Training Procedure |
|
|
| ### Hyperparameters |
|
|
| | Hyperparameter | Value | |
| |---|---| |
| | Method | Parameterised Efficiency (PEFT / LoRA) | |
| | Base model | `sshleifer/tiny-gpt2` | |
| | Dataset | `stanfordnlp/imdb` (`train[:200]`) | |
| | Epochs | 1 | |
| | Batch size | 4 | |
| | Learning rate | 0.0005 | |
| | Max steps | 20 | |
| | LoRA rank | 4 | |
| | LoRA alpha | 8 | |
|
|
| ### Framework versions |
|
|
| See the `!pip install` cell in the training notebook for the exact package set used. |
|
|
| ## How to Get Started |
|
|
| ```python |
| from transformers import pipeline |
| |
| gen = pipeline("text-generation", model="knatware/knat") |
| gen("Your prompt here") |
| ``` |
|
|
| ## Testing |
|
|
| Before being pushed, this model was tested locally with a sample inference call, and was re-loaded and tested again directly from the Hub after pushing to confirm the upload was complete and usable. |
|
|
| --- |
| © Knatware Technology UK. Developed by Kayode Okosi. |
|
|
