Update README.md

README.md

@@ -1,3 +1,270 @@
----
-license: apache-2.0
----
+# TinyLlama Multi-Task LoRA (Sentiment + Translation)
+
+This repository contains a **LoRA adapter** trained on top of  
+**TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T**  
+to support **multiple tasks** via instruction-style prompting.
+
+---
+
+## 🔧 Base Model
+
+- **Base model**: `TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T`
+- **Architecture**: Decoder-only (LLaMA-style)
+- **Fine-tuning method**: LoRA (PEFT)
+- **Trainable parameters**: ~4M
+- **Total parameters**: ~1.1B
+
+This repository **only contains the LoRA adapter weights**, not the full base model.
+
+---
+
+## 📌 Supported Tasks
+
+### 1️⃣ Sentiment Analysis (IMDB)
+
+Binary sentiment classification for movie reviews:
+
+- `positive`
+- `negative`
+
+### 2️⃣ Translation (English → French)
+
+Neural machine translation from English to French.
+
+---
+
+## 🧠 Training Data
+
+All datasets are loaded directly from **Hugging Face Datasets**:
+
+| Task                | Dataset                         | Description                      |
+| ------------------- | ------------------------------- | -------------------------------- |
+| Sentiment Analysis  | `imdb`                          | Movie reviews with binary labels |
+| Translation (EN→FR) | `Helsinki-NLP/opus-100` (en-fr) | Parallel English–French corpus   |
+
+---
+
+## 🧩 Training Strategy
+
+- **Multi-task instruction tuning**
+- Tasks are distinguished via **explicit prompt headers**
+- All tasks are unified into a **causal language modeling** objective
+- Only LoRA parameters are updated; base model weights remain frozen
+
+---
+
+## 🧾 Prompt Format
+
+### Sentiment Analysis
+
+```text
+### Task: Sentiment Analysis
+### Review:
+This movie was absolutely fantastic!
+### Answer:
+positive
+
+### Task: Translation (English to French)
+### English:
+I love learning large language models.
+### French:
+J'aime apprendre les grands modèles de langage.
+
+
+```
+
+ **Load Base Model + LoRA Adapter**
+
+```python 
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base_model_name = "TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T"
+lora_repo = "BEncoderRT/tinyllama-multitask-lora"
+
+tokenizer = AutoTokenizer.from_pretrained(base_model_name)
+tokenizer.pad_token = tokenizer.eos_token
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_name,
+    device_map="auto"
+).eval()
+
+lora_model = AutoModelForCausalLM.from_pretrained(
+    base_model_name,
+    device_map="auto"
+)
+
+lora_model = PeftModel.from_pretrained(lora_model, lora_repo)
+lora_model.eval()
+
+def generate(
+    model,
+    prompt,
+    max_new_tokens=64,
+    temperature=0.3
+):
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+    with torch.no_grad():
+        outputs = model.generate(
+            **inputs,
+            max_new_tokens=max_new_tokens,
+            do_sample=True,
+            temperature=temperature,
+            pad_token_id=tokenizer.eos_token_id
+        )
+
+    return tokenizer.decode(outputs[0], skip_special_tokens=True)
+```
+
+
+
+**Inference Examples**
+
+Sentiment Analysis Example
+
+```python
+sentiment_tests = [
+    "I absolutely loved this movie. The acting was brilliant.",
+    "This film was boring, slow, and a complete waste of time.",
+    "The movie was okay, but nothing special.",
+]
+
+def test_sentiment_comparison(texts):
+    for text in texts:
+        prompt = (
+            "### Task: Sentiment Analysis\n"
+            "### Review:\n"
+            f"{text}\n"
+            "### Answer:\n"
+        )
+
+        base_out = generate(base_model, prompt, max_new_tokens=8)
+        lora_out = generate(lora_model, prompt, max_new_tokens=8)
+
+        print("=" * 80)
+        print("REVIEW:")
+        print(text)
+        print("\n[BASE MODEL OUTPUT]")
+        print(base_out)
+        print("\n[LORA MODEL OUTPUT]")
+        print(lora_out)
+```
+
+```python
+test_sentiment_comparison(sentiment_tests)
+```
+
+```
+================================================================================
+REVIEW:
+I absolutely loved this movie. The acting was brilliant.
+
+[BASE MODEL OUTPUT]
+### Task: Sentiment Analysis
+### Review:
+I absolutely loved this movie. The acting was brilliant.
+### Answer:
+I loved this movie. It was so
+
+[LORA MODEL OUTPUT]
+### Task: Sentiment Analysis
+### Review:
+I absolutely loved this movie. The acting was brilliant.
+### Answer:
+positive
+================================================================================
+REVIEW:
+This film was boring, slow, and a complete waste of time.
+
+[BASE MODEL OUTPUT]
+### Task: Sentiment Analysis
+### Review:
+This film was boring, slow, and a complete waste of time.
+...
+### Review:
+The movie was okay, but nothing special.
+### Answer:
+negative
+```
+
+
+
+
+
+Translation Example
+
+```python
+translation_tests = [
+
+  "I love learning large language models.",
+
+  "This movie was disappointing and boring.",
+
+  "Artificial intelligence is changing the world."
+
+]
+def test_translation_comparison(texts):
+    for text in texts:
+        prompt = (
+            "### Task: Translation (English to French)\n"
+            "### English:\n"
+            f"{text}\n"
+            "### French:\n"
+        )
+
+        base_out = generate(base_model, prompt, max_new_tokens=64)
+        lora_out = generate(lora_model, prompt, max_new_tokens=64)
+
+        print("=" * 80)
+        print("ENGLISH:")
+        print(text)
+        print("\n[BASE MODEL OUTPUT]")
+        print(base_out)
+        print("\n[LORA MODEL OUTPUT]")
+        print(lora_out)
+
+```
+
+```python
+test_sentiment_comparison(sentiment_tests)
+```
+
+```
+================================================================================
+ENGLISH:
+I love learning large language models.
+
+[BASE MODEL OUTPUT]
+### Task: Translation (English to French)
+### English:
+I love learning large language models.
+### French:
+J'adore apprendre les modèles de langage grand.
+
+### Translation:
+I love learning large language models.
+### Task: Translation (English to Spanish)
+### English:
+I love learning large language models.
+### Spanish:
+Me gusta
+
+[LORA MODEL OUTPUT]
+### Task: Translation (English to French)
+### English:
+I love learning large language models.
+### French:
+Je me passionne pour les modèles de langues grandes.
+================================================================================
+...
+### English:
+Artificial intelligence is changing the world.
+### French:
+L'intelligence artificielle change le monde.
+```
+
+
+