README.md

---
license: mit
base_model: Qwen/Qwen3-4B-Instruct-2507
library_name: peft
pipeline_tag: text-generation
tags:
- unsloth
- lora
- peft
- qwen
- multiple-choice
- question-answering
datasets:
- dahaludba/QSolver_Train
metrics:
- map@3
---

# QSolver_Decoder_V16

QSolver_Decoder_V16 is a fine-tuned causal language model designed for context-driven scientific multiple-choice question answering. It utilizes 5-Fold Cross-Validation, Unsloth 4-bit quantization, and Low-Rank Adaptation (LoRA) on top of the base model `Qwen/Qwen3-4B-Instruct-2507`.

The model takes a context, question prompt, and five multiple-choice options (A, B, C, D, E), and ranks option token logits to produce predictions evaluated via Mean Average Precision at 3 (MAP@3).

---

## Model Details

- **Model Name:** QSolver_Decoder_V16
- **Repository ID:** `dahaludba/QSolver_Decoder_V16`
- **Base Model:** `Qwen/Qwen3-4B-Instruct-2507`
- **Dataset:** `dahaludba/QSolver_Train`
- **Fine-Tuning Architecture:** Low-Rank Adaptation (LoRA) via Unsloth (`FastLanguageModel`)
- **Quantization:** 4-bit NormalFloat (NF4)
- **Maximum Sequence Length:** 1024 tokens
- **Primary Metric:** MAP@3
- **Total Training Duration:** 18 hours 45 minutes
- **License:** MIT License

---

## Training Setup & Method

The repository contains adapter checkpoints trained across 5 folds (`fold_1` to `fold_5`). Each fold was trained using process isolation across available GPUs with dynamic memory management.

### Fine-Tuning Strategy
- **Prompt Token Masking:** Prompts were formatted with system and user blocks, and target answer completions were set while prompt tokens were masked with label ID `-100` so loss was calculated exclusively on completion target tokens.
- **Logit Extraction for Metrics:** Evaluation metrics computed option choice logit rankings specifically at the exact prediction index for option choices ('A', 'B', 'C', 'D', 'E') to calculate top-3 ranking performance without unnecessary GPU memory allocation.

---

## Hyperparameters

| Hyperparameter | Value |
| :--- | :--- |
| Base Model Quantization | 4-bit (BitsAndBytes / Unsloth) |
| LoRA Rank ($r$) | 32 |
| LoRA Alpha ($\alpha$) | 64 |
| LoRA Dropout | 0.05 |
| Target Modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
| Bias Term | `none` |
| Gradient Checkpointing | `unsloth` |
| Learning Rate | 1e-4 |
| Optimizer | AdamW |
| Learning Rate Schedule | Warmup Linear Decay |
| Warmup Ratio | 0.05 |
| Weight Decay | 0.01 |
| Per Device Train Batch Size | 4 |
| Per Device Eval Batch Size | 4 |
| Gradient Accumulation Steps | 4 (Effective Batch Size = 16) |
| Training Epochs | 4 per fold |
| Data Collator | `DataCollatorForSeq2Seq` (pad_to_multiple_of=8) |
| Evaluation Strategy | Epoch-based |
| Best Model Metric | MAP@3 (`greater_is_better=True`) |
| Seed | 42 |

---

## Experiment Tracking & Cross-Validation Results

The total training across all 5 folds completed in **18 hours and 45 minutes**. Individual run logs and metrics can be reviewed via the following Weights & Biases experiment links:

- **Fold 1:** [W&B Run hqnniqtc](https://wandb.ai/24f2002963-dl-genai-project/24f2002963-t22026/runs/hqnniqtc?nw=nwuser24f2002963)
- **Fold 2:** [W&B Run qgx597no](https://wandb.ai/24f2002963-dl-genai-project/24f2002963-t22026/runs/qgx597no?nw=nwuser24f2002963)
- **Fold 3:** [W&B Run 2eh7a9u8](https://wandb.ai/24f2002963-dl-genai-project/24f2002963-t22026/runs/2eh7a9u8?nw=nwuser24f2002963)
- **Fold 4:** [W&B Run 1jict6ux](https://wandb.ai/24f2002963-dl-genai-project/24f2002963-t22026/runs/1jict6ux?nw=nwuser24f2002963)
- **Fold 5:** [W&B Run fl2ot0l1](https://wandb.ai/24f2002963-dl-genai-project/24f2002963-t22026/runs/fl2ot0l1?nw=nwuser24f2002963)

---

## Prompt Template

Each input sample follows standard chat templates formatted as:

```text
<|im_start|>system
You are a scientific expert. Base your answer STRICTLY on the provided Context. Output ONLY the single letter corresponding to the correct option (A, B, C, D, or E).<|im_end|>
<|im_start|>user
Context: {context}
Question: {question}
A) {option_a}
B) {option_b}
C) {option_c}
D) {option_d}
E) {option_e}<|im_end|>
<|im_start|>assistant
{answer}<|im_end|>
```

---

## Evaluation Metric

Performance is measured using Mean Average Precision at 3 (MAP@3):

$$\text{MAP@3} = \frac{1}{U} \sum_{i=1}^{U} \sum_{k=1}^{\min(P, 3)} P(k) \times \text{rel}(k)$$

Where:
- $P(k)$ is the precision at rank $k$.
- $\text{rel}(k)$ is an indicator function returning 1 if the item at rank $k$ is correct, otherwise 0.
- Scoring weights: Rank 1 correct = 1.0, Rank 2 correct = 0.5, Rank 3 correct = 0.333, outside top 3 = 0.0.

---

## Usage Code Example

Below is an example script to load a fold adapter and run inference:

```python
import torch
from unsloth import FastLanguageModel
from peft import PeftModel

MODEL_REPO = "dahaludba/QSolver_Decoder_V16"
FOLD_SUBFOLDER = "fold_1"
MAX_SEQ_LENGTH = 1024

# Load Base Model & Tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Qwen/Qwen3-4B-Instruct-2507",
    max_seq_length=MAX_SEQ_LENGTH,
    dtype=None,
    load_in_4bit=True,
)

# Load PEFT Fold Adapter
model = PeftModel.from_pretrained(model, MODEL_REPO, subfolder=FOLD_SUBFOLDER)
FastLanguageModel.for_inference(model)

# Construct Input Prompt
prompt = (
    "<|im_start|>system\n"
    "You are a scientific expert. Base your answer STRICTLY on the provided Context. "
    "Output ONLY the single letter corresponding to the correct option (A, B, C, D, or E).<|im_end|>\n"
    "<|im_start|>user\n"
    "Context: Mitochondria generate most of the chemical energy needed to power the cell's biochemical reactions.\n"
    "Question: What organelle produces most cellular energy?\n"
    "A) Nucleus\nB) Mitochondria\nC) Ribosome\nD) Golgi Apparatus\nE) Endoplasmic Reticulum<|im_end|>\n"
    "<|im_start|>assistant\n"
)

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=2, use_cache=True)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("Predicted Option:", response.strip())
```

---

## License

This project and all model weight artifacts in this repository are distributed under the **MIT License**.