Update README.md
Browse files
README.md
CHANGED
|
@@ -15,4 +15,53 @@ tags:
|
|
| 15 |
- prompt_refining
|
| 16 |
- indirect_requests
|
| 17 |
- pragmatics
|
| 18 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
- prompt_refining
|
| 16 |
- indirect_requests
|
| 17 |
- pragmatics
|
| 18 |
+
---
|
| 19 |
+
# PragmaticLM - T5 for Prompt Restructuring
|
| 20 |
+
|
| 21 |
+
## 📌 Overview
|
| 22 |
+
**PragmaticLM** is a fine-tuned T5 model designed to **restructure and reframe user prompts** for better understanding by downstream LLMs. The model enhances prompt clarity by leveraging **contextual restructuring** techniques.
|
| 23 |
+
|
| 24 |
+
## 🚀 Model Details
|
| 25 |
+
- **Base Model**: [T5-Base](https://huggingface.co/t5-base)
|
| 26 |
+
- **Training Data**: [Indirect Requests] (https://huggingface.co/datasets/msamogh/indirect-requests)
|
| 27 |
+
- **Task Type**: Text-to-text transformation
|
| 28 |
+
- **Library**: [Hugging Face Transformers](https://github.com/huggingface/transformers)
|
| 29 |
+
|
| 30 |
+
## 📊 Training Configuration
|
| 31 |
+
- **Epochs**: 10
|
| 32 |
+
- **Batch Size**: 8
|
| 33 |
+
- **Learning Rate**: Encoder: `1e-5`, Decoder: `3e-5`
|
| 34 |
+
- **Optimizer**: AdamW
|
| 35 |
+
- **Loss Function**: Cross-entropy loss
|
| 36 |
+
- **Hardware**: GPU (T4)
|
| 37 |
+
|
| 38 |
+
## ⚡ Usage
|
| 39 |
+
```python
|
| 40 |
+
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
| 41 |
+
import torch
|
| 42 |
+
|
| 43 |
+
tokenizer = AutoTokenizer.from_pretrained("aliMohammad16/pragmaticLM")
|
| 44 |
+
model = AutoModelForSeq2SeqLM.from_pretrained("aliMohammad16/pragmaticLM")
|
| 45 |
+
|
| 46 |
+
def restructure_prompt(input_prompt):
|
| 47 |
+
input_text = f"Restructure Prompt: {input_prompt}"
|
| 48 |
+
inputs = tokenizer(input_text, return_tensors="pt", padding=True)
|
| 49 |
+
|
| 50 |
+
output = model.generate(
|
| 51 |
+
inputs.input_ids,
|
| 52 |
+
max_length=64,
|
| 53 |
+
num_beams=4,
|
| 54 |
+
early_stopping=True
|
| 55 |
+
)
|
| 56 |
+
|
| 57 |
+
return tokenizer.decode(output[0], skip_special_tokens=True)
|
| 58 |
+
|
| 59 |
+
# Example Usage
|
| 60 |
+
test_prompt = "I am not feeeling well. I need to consult a doctor nearby."
|
| 61 |
+
print(restructure_prompt(test_prompt))
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
## ⏳ Improvements
|
| 65 |
+
- **Work in progress**: This is a work in progress. I am actively working on this model.
|
| 66 |
+
- **Update**: Next I am implementing a multimodular pipeline, integrating TinyLlama 1.1B and Llama Index RAG with `prompt-restructuring` model, to improve output generation.
|
| 67 |
+
|