Model Card for mayankpuvvala/peft_lora_t5_merged_model_pytorch_issues

This model is a fine-tuned T5 encoder-decoder designed to generate GitHub issue bodies based on issue titles, specifically for the PyTorch repository. It utilizes Parameter-Efficient Fine-Tuning (PEFT) with Low-Rank Adaptation (LoRA) to achieve efficient training and inference.

Model Details

Model Description

• Developed by: Mayank Puvvala
• Model type: Text-to-Text Generation
• Language(s): English
• License: MIT
• Fine-tuned from model: t5-small

Model Sources

• Repository: GitHub
• Demo: Hugging Face Space

Uses

Direct Use

• Input: GitHub issue title (e.g., "Memory leak when using DataLoader with num_workers > 0")
• Output: Generated issue body describing the problem in detail.

Out-of-Scope Use

• Not suitable for generating issue bodies for repositories other than PyTorch without further fine-tuning.

Bias, Risks, and Limitations

• The model is trained on PyTorch issues and may not generalize well to other domains.
• Generated content should be reviewed for accuracy and relevance.

Recommendations

• Use the model as an assistant tool, with human oversight for the final content.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "mayankpuvvala/peft_lora_t5_merged_model_pytorch_issues"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

input_text = "Memory leak when using DataLoader with num_workers > 0"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

• Custom dataset of PyTorch GitHub issues, comprising titles and corresponding bodies.

Training Procedure

• Fine-tuned using PEFT with LoRA for 3 epochs.

Training Hyperparameters

• Epochs: 3
• Batch size: 8

Evaluation Metrics

• ROUGE Precision: 53.12%
• ROUGE F1 Score: 49.8%