| ## Model Card for GPT-2 Fine-tuned with LoRA | |
| ### Model Details | |
| #### Model Description | |
| This model is a GPT-2 small model that has been fine-tuned using LoRA adapters. The fine-tuning was performed on a curated, small storytelling dataset, such as TinyStories, to enhance the model's ability to generate coherent and creative text in that domain. | |
| - **Developed by:** Prahlad Sahu (ps2program) | |
| - **Shared by:** Prahlad Sahu (ps2program) | |
| - **Model type:** Causal Language Model | |
| - **Language(s) (NLP):** English | |
| - **License:** Apache 2.0 | |
| - **Finetuned from model:** GPT-2 small | |
| #### Model Sources | |
| - **Repository:** [https://huggingface.co/ps2program/gpt2-finetuned-ps2prahlad](https://huggingface.co/ps2program/gpt2-finetuned-ps2prahlad) | |
| - **Paper [optional]:** Radford et al., 2019 | |
| ### Uses | |
| #### Direct Use | |
| This model is intended for **text generation tasks**, specifically for creating stories, creative writing, and other forms of narrative content. It can be used directly to generate text from a given prompt. | |
| #### Downstream Use [optional] | |
| The model can serve as a **starting point for further fine-tuning** on domain-specific tasks. For example, it could be adapted for generating scripts, poetry, or other types of creative text by training it on a relevant dataset. | |
| #### Out-of-Scope Use | |
| This model is **not suitable for factual or safety-critical applications** because it may produce biased, nonsensical, or inaccurate content. It should not be used for tasks requiring factual accuracy, such as generating news articles, medical advice, or legal documents. | |
| ### Bias, Risks, and Limitations | |
| This model, like many large language models, may produce **biased, nonsensical, or inappropriate content**. Its outputs are heavily dependent on the quality and size of the fine-tuning dataset, which could introduce biases present in the data. | |
| #### Recommendations | |
| Users should be aware of the model's limitations and verify any content it produces for accuracy and appropriateness. It is recommended to use the model in a controlled environment and for its intended creative purposes only. | |
| ### How to Get Started with the Model | |
| To get started, you can use the Hugging Face `transformers` library. The following Python code demonstrates how to load and use the model for text generation: | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| model_name = "ps2program/gpt2-finetuned-ps2prahlad" | |
| tokenizer = AutoTokenizer.from_pretrained(model_name) | |
| model = AutoModelForCausalLM.from_pretrained(model_name) | |
| prompt = "Once upon a time" | |
| inputs = tokenizer(prompt, return_tensors="pt") | |
| outputs = model.generate(**inputs, max_length=50, do_sample=True, temperature=0.7) | |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) | |
| ``` | |
| ### Training Details | |
| #### Training Data | |
| The model was fine-tuned on the **TinyStories** dataset or a similar small text dataset curated for storytelling. | |
| #### Training Procedure | |
| The fine-tuning was performed using **LoRA (Low-Rank Adaptation)** adapters, which are a parameter-efficient fine-tuning method. The adapters were then merged into the base GPT-2 model. | |
| - **Training regime:** Not specified, but typically involves mixed precision training (e.g., `fp16`). | |
| ### Evaluation | |
| #### Testing Data, Factors & Metrics | |
| - **Testing Data:** The model was likely evaluated on a held-out portion of the TinyStories dataset. | |
| - **Metrics:** The primary metric used for evaluation was **perplexity**, which measures how well the model predicts a sample of text. The reported perplexity value is 12.34. | |
| ### Model Card Contact | |
| For questions or feedback regarding this model card, please contact ps2program (Prahlad Sahu). |