| --- |
| license: apache-2.0 |
| base_model: HuggingFaceTB/SmolLM2-135M |
| tags: |
| - pasta-finetune |
| - knatware |
| - p |
| - instruction |
| datasets: |
| - tatsu-lab/alpaca |
| library_name: peft |
| pipeline_tag: text-generation |
| --- |
| |
| # knatware/SmolLM2-135M |
|
|
| SmolLM2-135M |
|
|
| ## Model Description |
|
|
| This model was produced with the **Parameterised Efficiency (PEFT / LoRA)** method (PASTA framework) starting from the base model [`HuggingFaceTB/SmolLM2-135M`](https://huggingface.co/HuggingFaceTB/SmolLM2-135M), fine-tuned on a sample of the [`tatsu-lab/alpaca`](https://huggingface.co/datasets/tatsu-lab/alpaca) dataset. |
|
|
| - **Task type:** instruction |
| - **Library:** peft |
| - **Generated by:** the PASTA fine-tuning Colab notebook generator |
|
|
| ## Intended Uses & Limitations |
|
|
| This model was fine-tuned on a small sample for demonstration purposes. It has **not** been evaluated at scale and should not be used in production or safety-critical settings without further training, evaluation, and review. Behaviour is inherited from the base model and the (small) fine-tuning sample, and may reflect biases present in either. |
|
|
| ## Training Procedure |
|
|
| ### Hyperparameters |
|
|
| | Hyperparameter | Value | |
| |---|---| |
| | Method | Parameterised Efficiency (PEFT / LoRA) | |
| | Base model | `HuggingFaceTB/SmolLM2-135M` | |
| | Dataset | `tatsu-lab/alpaca` (`train[:200]`) | |
| | Epochs | 1 | |
| | Batch size | 4 | |
| | Learning rate | 0.0005 | |
| | Max steps | 20 | |
| | LoRA rank | 4 | |
| | LoRA alpha | 8 | |
|
|
| ### Framework versions |
|
|
| See the `!pip install` cell in the training notebook for the exact package set used. |
|
|
| ## How to Get Started |
|
|
| ```python |
| from transformers import pipeline |
| |
| gen = pipeline("text-generation", model="knatware/SmolLM2-135M") |
| gen("Your prompt here") |
| ``` |
|
|
| ## Testing |
|
|
| Before being pushed, this model was tested locally with a sample inference call, and was re-loaded and tested again directly from the Hub after pushing to confirm the upload was complete and usable. |
|
|
| --- |
| © Knatware Technology UK. Developed by Kayode Okosi. |
|
|
