---
library_name: transformers
tags: [text2text-generation, t5, lora, peft, pytorch, github-issues]
---

# Model Card for `mayankpuvvala/lora-t5-pytorch-issues`

This model represents the LoRA adapter weights trained on a custom dataset of PyTorch GitHub issues. It is intended to be used with the base `t5-small` model to generate issue bodies from titles.

## Model Details

### Model Description

- **Developed by:** Mayank Puvvala
- **Model type:** LoRA Adapter for Text-to-Text Generation
- **Language(s):** English
- **License:** MIT
- **Fine-tuned from model:** [t5-small](https://huggingface.co/t5-small)

### Model Sources

- **Repository:** [GitHub](https://github.com/mayankpuvvala/LLM_FineTune_GenAI)

## How to Get Started with the Model

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from peft import PeftModel

base_model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")
model = PeftModel.from_pretrained(base_model, "mayankpuvvala/lora-t5-pytorch-issues")
tokenizer = AutoTokenizer.from_pretrained("t5-small")

input_text = "Memory leak when using DataLoader with num_workers > 0"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
## Training Details
### Training Data
- Custom dataset of PyTorch GitHub issues, comprising titles and corresponding bodies.
### Training Procedure
- Fine-tuned using PEFT with LoRA for 3 epochs.
### Training Hyperparameters
- Epochs: 3
- Batch size: 8
## Evaluation Metrics
- ROUGE Precision: 53.12%
- ROUGE F1 Score: 49.8%