Blame for README.md

---
library_name: peft
tags:
- lora
- qlora
- cpp
- c++
- code
- chatbot
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
pipeline_tag: text-generation
---

# CppBuddy: a C++ tutor chatbot (QLoRA adapter for Qwen2.5-Coder-7B-Instruct)

CppBuddy is a small **LoRA adapter** that turns [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) into a friendly C++ tutor. It answers C++ concept questions and coding questions in a consistent style: a short explanation, a compilable code example, and a tip. It politely redirects off-topic questions back to C++.

This repository contains **only the adapter weights**, not the base model. You load it on top of Qwen2.5-Coder-7B-Instruct (see below).

## Model Details

- **Developed by:** mohitsharma9761
- **Model type:** LoRA adapter (QLoRA fine-tune) for a causal language model
- **Language:** English (code in C++)
- **License:** Apache-2.0 (same as the base model; please check the base model page for its current terms)
- **Fine-tuned from:** [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)

## Uses

### Direct use
- Asking C++ concept questions (pointers, RAII, smart pointers, STL, templates, move semantics, undefined behavior, and similar)
- Getting short C++ code examples for common tasks (searching, sorting, string handling, containers)
- Learning and interview practice

### Out-of-scope use
- Production or safety-critical code without human review
- General-knowledge chat. It was trained to redirect non-C++ questions, though this is not fully reliable.
- Anything that needs verified correctness. Always compile, test, and read the code it writes.

## Bias, Risks, and Limitations

- **It can be wrong.** Generated code may compile and still contain logic errors, or give outdated or unsafe advice.
- **Small fine-tuning dataset** (about 250 examples). The adapter mostly teaches *answer style and scope*; the C++ knowledge comes from the base model, so it inherits the base model's gaps.
- **Evaluation is small** (see below), so treat the numbers as a rough signal, not a benchmark.
- Off-topic redirection is learned from a handful of examples, so it will sometimes answer non-C++ questions anyway.

## How to Get Started with the Model

The system prompt below is the one used in training. Using a different one will make answers worse.

````python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"
ADAPTER = "mohitsharma9761/my-cpp-chatbot"

SYSTEM_PROMPT = "You are CppBuddy, a friendly expert C++ tutor and coding assistant (C++11 to C++23). Explain simply first, then show short, correct, compilable code in ```cpp blocks. Prefer modern C++ (smart pointers, STL, RAII). Point out undefined behavior and common mistakes. If a question is not about C++ or programming, politely steer back to C++."

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "what is RAII?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=600, do_sample=True, temperature=0.4, top_p=0.9)

print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
````

A 4-bit load fits on a free 16 GB GPU such as a Colab T4.

## Training Details

### Training data
About **250 curated question and answer pairs** about C++ (247 used for training, 10 held out for validation):

- Explanation-plus-code answers (short explanation, compilable example, tip). The built-in examples with code were shown twice per epoch so the model keeps writing code.
- Short definition-style answers for basic concepts.
- A small set of polite redirects for off-topic questions.
- A few corrective examples for topics where an earlier version of the model gave a confused answer (for example `new` vs `malloc`).

Full C++ programs in the answers were checked with `g++ -std=c++20 -fsyntax-only` and any that failed to compile were dropped.

### Training procedure

- **Method:** QLoRA (4-bit NF4 base model with double quantization, LoRA adapters)
- **LoRA:** r = 16, alpha = 32, dropout = 0.05, applied to `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
- **Loss:** computed on the assistant reply only (prompt tokens masked)
- **Epochs:** 2
- **Learning rate:** 2e-4
- **Batch size:** 1 per device, gradient accumulation 4 (effective batch size 4)
- **Optimizer:** `paged_adamw_8bit`
- **Max sequence length:** 1024 tokens
- **Training regime:** fp16 mixed precision with gradient checkpointing
- **Frameworks:** Transformers (`Trainer`), PEFT, bitsandbytes

#### Speeds, sizes, times
Roughly 10 minutes on a single Colab T4 GPU. The adapter file is small (tens of MB) compared with the 7B base model.

## Evaluation

### Training and validation loss
| Epoch | Training loss | Validation loss |
|---|---|---|
| 1 | 0.839 | 0.973 |
| 2 | 0.431 | 1.061 |

The validation set is only 10 short examples, so the validation loss is noisy. The small rise from epoch 1 to 2 hints at mild overfitting, which is why training was stopped at 2 epochs.

### Compile test
8 coding prompts that were **not** in the training data (for example merging sorted vectors, counting vowels, a stack class using `std::vector`, word frequency with `unordered_map`). Each reply's `cpp` code blocks were checked with `g++ -std=c++20 -fsyntax-only`.

- **7 of 8 code blocks compiled**
- **0 of 8 answers** were missing code

This is a syntax and type check only. It does not test whether the program gives correct output. It is a single sampled run (temperature 0.4) on a small prompt set, and it has not yet been compared against the plain base model.

## Environmental Impact

- **Hardware type:** NVIDIA T4 GPU (Google Colab)
- **Hours used:** about 0.2 hours per training run (approximate)
- **Cloud provider:** Google Colab
- **Carbon emitted:** not measured

## Technical Specifications

- **Architecture:** Qwen2.5 decoder-only transformer (7B parameters) with LoRA adapters
- **Objective:** causal language modeling (supervised fine-tuning on chat-formatted examples)
- **Software:** Transformers, PEFT, bitsandbytes, Datasets, PyTorch

## Acknowledgements

Built on [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) by the Qwen team.

## Model Card Contact

mohitsharma9761 on Hugging Face (use the Community tab of this repo).