| |
|
| | --- |
| | library_name: transformers |
| | tags: [text2text-generation, t5, lora, peft, pytorch, github-issues] |
| | --- |
| | |
| | # Model Card for `mayankpuvvala/lora-t5-pytorch-issues` |
| |
|
| | This model represents the LoRA adapter weights trained on a custom dataset of PyTorch GitHub issues. It is intended to be used with the base `t5-small` model to generate issue bodies from titles. |
| |
|
| | ## Model Details |
| |
|
| | ### Model Description |
| |
|
| | - **Developed by:** Mayank Puvvala |
| | - **Model type:** LoRA Adapter for Text-to-Text Generation |
| | - **Language(s):** English |
| | - **License:** MIT |
| | - **Fine-tuned from model:** [t5-small](https://huggingface.co/t5-small) |
| |
|
| | ### Model Sources |
| |
|
| | - **Repository:** [GitHub](https://github.com/mayankpuvvala/LLM_FineTune_GenAI) |
| |
|
| | ## How to Get Started with the Model |
| |
|
| | ```python |
| | from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
| | from peft import PeftModel |
| | |
| | base_model = AutoModelForSeq2SeqLM.from_pretrained("t5-small") |
| | model = PeftModel.from_pretrained(base_model, "mayankpuvvala/lora-t5-pytorch-issues") |
| | tokenizer = AutoTokenizer.from_pretrained("t5-small") |
| | |
| | input_text = "Memory leak when using DataLoader with num_workers > 0" |
| | inputs = tokenizer(input_text, return_tensors="pt") |
| | outputs = model.generate(**inputs) |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| | ## Training Details |
| | ### Training Data |
| | - Custom dataset of PyTorch GitHub issues, comprising titles and corresponding bodies. |
| | ### Training Procedure |
| | - Fine-tuned using PEFT with LoRA for 3 epochs. |
| | ### Training Hyperparameters |
| | - Epochs: 3 |
| | - Batch size: 8 |
| | ## Evaluation Metrics |
| | - ROUGE Precision: 53.12% |
| | - ROUGE F1 Score: 49.8% |
