---
license: apache-2.0
base_model: sshleifer/tiny-gpt2
tags:
- pasta-finetune
- knatware
- p
- causal
datasets:
- stanfordnlp/imdb
library_name: peft
pipeline_tag: text-generation
---

# knatware/knat

A sshleifer/tiny-gpt2 model fine-tuned with the Parameterised Efficiency (PEFT / LoRA) method, generated by the PASTA fine-tuning workflow.

## Model Description

This model was produced with the **Parameterised Efficiency (PEFT / LoRA)** method (PASTA framework) starting from the base model [`sshleifer/tiny-gpt2`](https://huggingface.co/sshleifer/tiny-gpt2), fine-tuned on a sample of the [`stanfordnlp/imdb`](https://huggingface.co/datasets/stanfordnlp/imdb) dataset.

- **Task type:** causal
- **Library:** peft
- **Generated by:** the PASTA fine-tuning Colab notebook generator

## Intended Uses & Limitations

This model was fine-tuned on a small sample for demonstration purposes. It has **not** been evaluated at scale and should not be used in production or safety-critical settings without further training, evaluation, and review. Behaviour is inherited from the base model and the (small) fine-tuning sample, and may reflect biases present in either.

## Training Procedure

### Hyperparameters

| Hyperparameter | Value |
|---|---|
| Method | Parameterised Efficiency (PEFT / LoRA) |
| Base model | `sshleifer/tiny-gpt2` |
| Dataset | `stanfordnlp/imdb` (`train[:200]`) |
| Epochs | 1 |
| Batch size | 4 |
| Learning rate | 0.0005 |
| Max steps | 20 |
| LoRA rank | 4 |
| LoRA alpha | 8 |

### Framework versions

See the `!pip install` cell in the training notebook for the exact package set used.

## How to Get Started

```python
from transformers import pipeline

gen = pipeline("text-generation", model="knatware/knat")
gen("Your prompt here")
```

## Testing

Before being pushed, this model was tested locally with a sample inference call, and was re-loaded and tested again directly from the Hub after pushing to confirm the upload was complete and usable.

---
© Knatware Technology UK. Developed by Kayode Okosi.