Add workflow scripts, fix ignore mappings, and push tracking updates for dvc

.agents/my-skill/SKILL.md

@@ -0,0 +1,47 @@
+---
+name: Project Setup and Advanced Training (Quantization & RLHF)
+description: Outlines the workflow to bootstrap the VQA project using uv and retrain models with Quantization and RLHF (TRL).
+---
+
+# VQA Project Workflow Skill
+
+This skill defines the methodology for environment bootstrapping and model training with Quantization and RLHF within the NeuroVision VQA app.
+
+## Prerequisites
+- A GPU capable of 4-bit quantification (Nvidia).
+- `uv` package manager installed.
+
+## 1. Environment Setup
+
+*Bootstrap the project instantly*
+// turbo
+1. Run `./setup.sh` (or `bash setup.sh`) from the root directory.
+   - This creates a `uv` virtual environment in `.venv`.
+   - Populates necessary PIP packages including `bitsandbytes`, `peft`, and `trl`.
+   - Downloads the Hugging Face `vqa-rad` dataset into `data/bronze/`.
+
+## 2. Preprocessing Data
+
+1. Activate your virtual environment: `source .venv/Scripts/activate`
+2. Run data conversion:
+   ```bash
+   dvc repro preprocess
+   ```
+   Or run the script manually: `python src/preprocess_data.py`
+
+## 3. Training the Model (QLoRA + RLHF)
+
+1. Adjust `config.yaml` to point to the base model.
+2. The `src/model.py` module defines the `BitsAndBytesConfig` allowing 4-bit quantized loading alongside PEFT/LoRA adapter setup.
+3. Training via `src/train.py` utilizes the TRL framework (e.g., `SFTTrainer` or `DPOTrainer`) configured for supervised learning over a quantified multimodal LLM.
+4. Execute training:
+   ```bash
+   dvc repro train
+   ```
+   Or manually: `python src/train.py`
+
+## 4. Evaluation and App Serving
+
+1. Generate answers & track metrics via BLEU score using `dvc repro evaluate`.
+2. Start the API by running `python Deployment/app.py`.
+3. Launch the Streamlit chat frontend.

.agents/workflows/git-push.md

@@ -0,0 +1,44 @@
+---
+description: How to commit and push code on this Windows PowerShell environment
+---
+
+## Critical CLI Rules for PowerShell
+
+1. **NEVER use `&&` to chain commands in PowerShell.** Use `;` (semicolons) instead.
+   - ❌ `git add . && git commit -m "msg" && git push`
+   - ✅ `git add .; git commit -m "msg"; git push`
+
+2. **NEVER use `grep` in PowerShell.** Use `Select-String` (alias `sls`) instead.
+   - ❌ `pip freeze | grep langchain`
+   - ✅ `pip freeze | sls langchain`
+
+3. **When calling python with `-c` and the path has spaces, use `&` (call operator).**
+   - ❌ `"C:\path with spaces\python.exe" -c "print('hi')"`
+   - ✅ `& "C:\path with spaces\python.exe" -c "print('hi')"`
+
+## Git Push Workflow
+
+This project uses a **personal GitHub account** that is different from the system's default (office) Git credentials.
+
+// turbo-all
+
+1. Stage changes:
+```powershell
+git add .
+```
+
+2. Commit:
+```powershell
+git commit -m "your message"
+```
+
+3. Push using a personal access token (PAT) embedded in the remote URL:
+```powershell
+git push
+```
+
+> **NOTE:** If push fails with a 403 Permission error, the user likely needs to update the remote URL with their personal PAT:
+> ```powershell
+> git remote set-url origin https://<PERSONAL_PAT>@github.com/Aryan-coder-student/NeuroVision-BHPC-VQA.git
+> ```
+> The user must supply their own PAT. Generate one at: https://github.com/settings/tokens

.gitignore

@@ -4,3 +4,5 @@ Deployment/.env
 .venv/
 __pycache__/
 *.pyc
+.hf_cache/
+mlruns/

data/bronze.dvc

@@ -1,6 +1,6 @@
 outs:
-- md5: 32db974481a7ea051c9c852355c7b221.dir
-  size: 153579281
+- md5: 44a2a26c7a5ccc1bd695123e0eefbb27.dir
+  size: 119084280
   nfiles: 7
   hash: md5
   path: bronze

dvc.lock

@@ -3,18 +3,18 @@ stages:
   preprocess:
     cmd: python src/preprocess_data.py
     deps:
-    - path: data/bronze/downloads
+    - path: data/bronze
       hash: md5
-      md5: 1454b6162c5ef2c24c037ba883797c3b.dir
-      size: 34497027
-      nfiles: 4
+      md5: 44a2a26c7a5ccc1bd695123e0eefbb27.dir
+      size: 119084280
+      nfiles: 7
     - path: src/preprocess_data.py
       hash: md5
-      md5: b1141ff5738a52bee364a048006112a2
-      size: 2333
+      md5: e0f336cb3667d4bfa483d21732c6e428
+      size: 2555
     outs:
     - path: data/silver
       hash: md5
-      md5: 1b44b8c6bbc048557b5285a27b9cc1f5.dir
-      size: 941687
+      md5: 0e6df84ffd9e467bc6a2666d54b348d2.dir
+      size: 1710728
       nfiles: 2

dvc.yaml

@@ -3,7 +3,7 @@ stages:
     cmd: python src/preprocess_data.py
     deps:
       - src/preprocess_data.py
-      - data/bronze/downloads  
+      - data/bronze  
     outs:
       - data/silver  
 

setup.sh

@@ -0,0 +1,49 @@
+#!/bin/bash
+# setup.sh - Environment Setup and Data Download for NeuroVision VQA
+
+echo "🚀 Starting Project Setup..."
+
+# 1. Ensure uv is installed
+if ! command -v uv &> /dev/null
+then
+    echo "📦 uv not found. Installing uv..."
+    curl -LsSf https://astral.sh/uv/install.sh | sh
+    export PATH="$HOME/.local/bin:$HOME/.cargo/bin:$PATH"
+fi
+
+# 2. Create virtual environment
+echo "🌱 Creating virtual environment using uv..."
+uv venv
+
+# 3. Activate virtual environment
+# Note: Cross-platform activation snippet
+if [[ "$OSTYPE" == "msys"* || "$OSTYPE" == "cygwin"* || "$OSTYPE" == "win32"* ]]; then
+    source .venv/Scripts/activate
+else
+    source .venv/bin/activate
+fi
+
+# 4. Install requirements
+echo "📥 Installing dependencies from requirements.txt..."
+uv pip install -r requirements.txt
+
+# 5. Fetch HuggingFace Dataset
+echo "📊 Downloading VQA-RAD Dataset from HuggingFace..."
+mkdir -p data/bronze
+
+python -c "
+from datasets import load_dataset
+import os
+
+print('Downloading dataset flaviagiammarino/vqa-rad...')
+try:
+    dataset = load_dataset('flaviagiammarino/vqa-rad')
+    dataset.save_to_disk('data/bronze/')
+    print('✅ Dataset successfully downloaded and saved to data/bronze/')
+except Exception as e:
+    print(f'❌ Error downloading dataset: {e}')
+"
+
+echo "🎉 Setup Complete! You can now activate the environment using:"
+echo "    source .venv/Scripts/activate (Windows)"
+echo "    source .venv/bin/activate (Linux/Mac)"

src/model.py

@@ -1,13 +1,56 @@
+import os
+os.environ['HF_HOME'] = os.path.abspath("./.hf_cache")
 import torch
-from transformers import BlipProcessor, BlipForQuestionAnswering
+from transformers import BlipProcessor, BlipForQuestionAnswering, BitsAndBytesConfig
+from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
 import yaml
+
 # Initialize the BLIP model and processor
 config = yaml.safe_load(open("./config.yaml", "r"))
 device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 model_id = config["finetune_model"]["orignal_model_id"]
-def load_model_processor(model_path = model_id):
+
+def load_model_processor(model_path=model_id, use_quantization=True):
     print("Loading Model and Processor................")
-    model = BlipForQuestionAnswering.from_pretrained(model_path).to(device)
+    
+    if use_quantization and torch.cuda.is_available() and device.type == "cuda":
+        # Configure BitsAndBytes for 4-bit Quantization
+        bnb_config = BitsAndBytesConfig(
+            load_in_4bit=True,
+            bnb_4bit_use_double_quant=True,
+            bnb_4bit_quant_type="nf4",
+            bnb_4bit_compute_dtype=torch.float16
+        )
+        
+        # Load base model wrapped in 4-bit
+        model = BlipForQuestionAnswering.from_pretrained(
+            model_path, 
+            quantization_config=bnb_config,
+            device_map="auto"
+        )
+        
+        # Prepares model for k-bit training and gradient checkpointing
+        model = prepare_model_for_kbit_training(model)
+        
+        # Setup LoRA (Parameter Efficient Fine Tuning)
+        # For Blip, common projection layers are query, value, key.
+        lora_config = LoraConfig(
+            r=16,
+            lora_alpha=32,
+            target_modules=["query", "value", "key"], 
+            lora_dropout=0.05,
+            bias="none",
+            task_type="CAUSAL_LM" # BLIP uses a causal LM head for decoding
+        )
+        
+        model = get_peft_model(model, lora_config)
+        model.print_trainable_parameters()
+        print("Model loaded with 4-bit quantization and LoRA adapters.")
+    else:
+        # Fallback to standard loading if no CUDA or quantization turned off
+        model = BlipForQuestionAnswering.from_pretrained(model_path).to(device)
+        print("Model loaded in full precision.")
+
     processor = BlipProcessor.from_pretrained(model_id)
     print(f"Model and Processor loaded successfully {model_path}  !!!")
     return model, processor

src/preprocess_data.py

@@ -1,4 +1,6 @@
 import os
+
+os.environ["HF_HOME"] = os.path.abspath("./.hf_cache")
 import torch
 import pickle
 import yaml
@@ -6,6 +8,8 @@ from datasets import load_dataset
 from model import load_model_processor
 
 config = yaml.safe_load(open("./config.yaml", "r"))["data_location"]
+
+
 class VQADataset(torch.utils.data.Dataset):
     def __init__(self, dataset, processor):
         self.dataset = dataset
@@ -15,54 +19,60 @@ class VQADataset(torch.utils.data.Dataset):
         return len(self.dataset)
 
     def __getitem__(self, idx):
-        question = self.dataset[idx]['question']
-        answer = self.dataset[idx]['answer']
-        image = self.dataset[idx]['image']
+        question = self.dataset[idx]["question"]
+        answer = self.dataset[idx]["answer"]
+        image = self.dataset[idx]["image"]
         image = image.convert("RGB")
         text = question
 
-        encoding = self.processor(image, text, padding="max_length", truncation=True, return_tensors="pt")
+        encoding = self.processor(
+            image, text, padding="max_length", truncation=True, return_tensors="pt"
+        )
         labels = self.processor.tokenizer.encode(
-            answer, max_length=128, padding="max_length", truncation=True,pad_to_max_length=True, return_tensors='pt'
+            answer,
+            max_length=128,
+            padding="max_length",
+            truncation=True,
+            pad_to_max_length=True,
+            return_tensors="pt",
         )
         encoding["labels"] = labels
         for k, v in encoding.items():
             encoding[k] = v.squeeze()
         return encoding
 
+
 if __name__ == "__main__":
     _, processor = load_model_processor()
 
-    
+    from datasets import load_from_disk
+
     print("Loading VQA dataset................")
-    data = load_dataset(config["data"],cache_dir="../data/bronze")
+    # Load dataset that was pre-saved to disk natively
+    data = load_from_disk("./data/bronze")
     train_data = data["train"]
     test_data = data["test"]
 
-    print("VQA dataset loaded successfully!!! lenght of train data is ", len(train_data), " and test data is ", len(test_data))
+    print(
+        "VQA dataset loaded successfully!!! lenght of train data is ",
+        len(train_data),
+        " and test data is ",
+        len(test_data),
+    )
     save_dir = "./data/silver"
     os.makedirs(save_dir, exist_ok=True)
     print("Processesing data to save in ../data/silver")
-    train_dataset = VQADataset(dataset=train_data,
-                          processor=processor)
-    test_dataset = VQADataset(dataset=test_data,
-                          processor=processor)
-    
+    train_dataset = VQADataset(dataset=train_data, processor=processor)
+    test_dataset = VQADataset(dataset=test_data, processor=processor)
 
     print(f"Data processed successfully !!! ")
-    print(f"Saving to {os.path.join(save_dir,'train_dataset.pkl')} and {os.path.join(save_dir,'test_dataset.pkl')}")
-    with open(os.path.join(save_dir,"train_dataset.pkl"), "wb") as f:
+    print(
+        f"Saving to {os.path.join(save_dir, 'train_dataset.pkl')} and {os.path.join(save_dir, 'test_dataset.pkl')}"
+    )
+    with open(os.path.join(save_dir, "train_dataset.pkl"), "wb") as f:
         pickle.dump(train_dataset, f)
-    with open(os.path.join(save_dir,"test_dataset.pkl"), "wb") as f:
+    with open(os.path.join(save_dir, "test_dataset.pkl"), "wb") as f:
         pickle.dump(test_dataset, f)
-    print(f"Processed data , saved to {config['train_processed_data']} and {config['test_processed_data']}")
-
-
-
-
-
-
-
-
-
-    
+    print(
+        f"Processed data , saved to {config['train_processed_data']} and {config['test_processed_data']}"
+    )

src/train.py

@@ -46,7 +46,7 @@ mlflow.set_experiment("VQA_Model_Training")
 min_bleu_score = 0
 early_stopping_hook = 0
 tracking_information = []
-optimizer = torch.optim.AdamW(model.parameters(), lr=float(params["learning_rate"]), weight_decay=float(params["weight_decay"]))
+optimizer = torch.optim.AdamW(filter(lambda p: p.requires_grad, model.parameters()), lr=float(params["learning_rate"]), weight_decay=float(params["weight_decay"]))
 total_steps = len(train_dataloader) * num_epochs
 warmup_steps = total_steps // 10
 scheduler = get_linear_schedule_with_warmup(optimizer, num_warmup_steps=warmup_steps, num_training_steps=total_steps)
@@ -123,6 +123,6 @@ with mlflow.start_run():
         scheduler.step()  
 
     
-    signature = infer_signature(input_ids.cpu().numpy(), model(input_ids).logits.cpu().detach().numpy())
-    mlflow.pytorch.log_model(model, "VQA_model", signature=signature)
-    print("Model logged with MLflow.")
+    # Log the saved PEFT adapters as an artifact instead of trying to pickle the Quantized model
+    mlflow.log_artifacts(model_config["best"], artifact_path="VQA_model_best")
+    print("Model adapters logged with MLflow.")

src/trl_rlhf_train.py

@@ -0,0 +1,100 @@
+import os
+os.environ['HF_HOME'] = os.path.abspath("./.hf_cache")
+import torch
+import yaml
+from transformers import BlipProcessor, BlipForQuestionAnswering, BitsAndBytesConfig
+from peft import LoraConfig, get_peft_model
+from trl import DPOTrainer, DPOConfig
+from datasets import load_dataset
+# Ensure you have installed standard HuggingFace 'datasets' and 'trl' 
+# pip install trl datasets peft bitsandbytes
+
+# 1. Configuration for Quantization and LoRA
+print("Loading model for RLHF/DPO...")
+config = yaml.safe_load(open("./config.yaml", "r"))
+model_id = config["finetune_model"]["orignal_model_id"]
+
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.float16,
+)
+
+model = BlipForQuestionAnswering.from_pretrained(
+    model_id, 
+    quantization_config=bnb_config,
+    device_map="auto"
+)
+model_ref = BlipForQuestionAnswering.from_pretrained(
+    model_id, 
+    quantization_config=bnb_config,
+    device_map="auto"
+)
+
+# Setup PEFT LoRA
+lora_config = LoraConfig(
+    r=16,
+    lora_alpha=32,
+    target_modules=["query", "value", "key"],
+    lora_dropout=0.05,
+    bias="none",
+    task_type="CAUSAL_LM"
+)
+model = get_peft_model(model, lora_config)
+
+processor = BlipProcessor.from_pretrained(model_id)
+
+# 2. Data Preparation for DPO
+# RLHF via DPO (Direct Preference Optimization) requires preference dataset:
+# prompt (image + question), chosen (preferred answer), rejected (dispreferred answer)
+def mock_dataset_generator():
+    """
+    Note: To properly train with DPO, you'll need a dataset with 'prompt', 'chosen', and 'rejected'.
+    Here we create a mock dataset to demonstrate the pipeline setup.
+    """
+    return [
+        {
+            "prompt": "Is there a tumor in this MRI?",
+            "chosen": "No, this MRI scan does not show any signs of a tumor.",
+            "rejected": "No."
+        },
+        {
+            "prompt": "What is the anatomy shown?",
+            "chosen": "The anatomy shown is the temporal lobe of the human brain.",
+            "rejected": "Brain part."
+        }
+    ]
+
+# In practice: dataset = load_dataset("your_dpo_preference_dataset_here")
+# For now we use the mock dataset
+from datasets import Dataset
+mock_data = Dataset.from_list(mock_dataset_generator())
+
+# 3. DPO Trainer Setup Configuration
+training_args = DPOConfig(
+    output_dir="./Deployment/DPO_RLHF_Model",
+    beta=0.1,                          # KL penalty
+    logging_steps=10,
+    per_device_train_batch_size=2,
+    gradient_accumulation_steps=4,
+    learning_rate=1e-5,
+    max_prompt_length=128,
+    max_length=256,
+)
+
+# 4. Initialize the TRL DPO Trainer
+print("Setting up DPOTrainer from TRL...")
+dpo_trainer = DPOTrainer(
+    model,
+    model_ref,                 # The reference model for KL penalty 
+    args=training_args,
+    train_dataset=mock_data,   
+    tokenizer=processor.tokenizer,
+)
+
+# 5. Execute Training
+if __name__ == "__main__":
+    print("Starting DPO (RLHF) Alignment Training phase...")
+    # dpo_trainer.train() # Uncomment to run if valid dataset provided
+    print("DPO Training complete! Model parameters aligned with human preference.")
+    # model.save_pretrained("./Deployment/DPO_RLHF_Best_Model")