Update README.md

0439d3b verified 22 days ago

7.01 kB

---
library_name: peft
tags:
- lora
- qlora
- cpp
- c++
- code
- chatbot
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
pipeline_tag: text-generation
---
# CppBuddy: a C++ tutor chatbot (QLoRA adapter for Qwen2.5-Coder-7B-Instruct)

CppBuddy is a small **LoRA adapter** that turns [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) into a friendly C++ tutor. It answers C++ concept questions and coding questions in a consistent style: a short explanation, a compilable code example, and a tip. It politely redirects off-topic questions back to C++.

This repository contains **only the adapter weights**, not the base model. You load it on top of Qwen2.5-Coder-7B-Instruct (see below).

## Model Details

- **Developed by:** mohitsharma9761
- **Model type:** LoRA adapter (QLoRA fine-tune) for a causal language model
- **Language:** English (code in C++)
- **License:** Apache-2.0 (same as the base model; please check the base model page for its current terms)
- **Fine-tuned from:** [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)

## Uses

### Direct use
- Asking C++ concept questions (pointers, RAII, smart pointers, STL, templates, move semantics, undefined behavior, and similar)
- Getting short C++ code examples for common tasks (searching, sorting, string handling, containers)
- Learning and interview practice

### Out-of-scope use
- Production or safety-critical code without human review
- General-knowledge chat. It was trained to redirect non-C++ questions, though this is not fully reliable.
- Anything that needs verified correctness. Always compile, test, and read the code it writes.

## Bias, Risks, and Limitations

- **It can be wrong.** Generated code may compile and still contain logic errors, or give outdated or unsafe advice.
- **Small fine-tuning dataset** (about 250 examples). The adapter mostly teaches *answer style and scope*; the C++ knowledge comes from the base model, so it inherits the base model's gaps.
- **Evaluation is small** (see below), so treat the numbers as a rough signal, not a benchmark.
- Off-topic redirection is learned from a handful of examples, so it will sometimes answer non-C++ questions anyway.

## How to Get Started with the Model

The system prompt below is the one used in training. Using a different one will make answers worse.

````python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"
ADAPTER = "mohitsharma9761/my-cpp-chatbot"
SYSTEM_PROMPT = "You are CppBuddy, a friendly expert C++ tutor and coding assistant (C++11 to C++23). Explain simply first, then show short, correct, compilable code in ```cpp blocks. Prefer modern C++ (smart pointers, STL, RAII). Point out undefined behavior and common mistakes. If a question is not about C++ or programming, politely steer back to C++."
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "what is RAII?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=600, do_sample=True, temperature=0.4, top_p=0.9)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
````

A 4-bit load fits on a free 16 GB GPU such as a Colab T4.

## Training Details

### Training data
About **250 curated question and answer pairs** about C++ (247 used for training, 10 held out for validation):

- Explanation-plus-code answers (short explanation, compilable example, tip). The built-in examples with code were shown twice per epoch so the model keeps writing code.
- Short definition-style answers for basic concepts.
- A small set of polite redirects for off-topic questions.
- A few corrective examples for topics where an earlier version of the model gave a confused answer (for example `new` vs `malloc`).

Full C++ programs in the answers were checked with `g++ -std=c++20 -fsyntax-only` and any that failed to compile were dropped.

### Training procedure

- **Method:** QLoRA (4-bit NF4 base model with double quantization, LoRA adapters)
- **LoRA:** r = 16, alpha = 32, dropout = 0.05, applied to `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
- **Loss:** computed on the assistant reply only (prompt tokens masked)
- **Epochs:** 2
- **Learning rate:** 2e-4
- **Batch size:** 1 per device, gradient accumulation 4 (effective batch size 4)
- **Optimizer:** `paged_adamw_8bit`
- **Max sequence length:** 1024 tokens
- **Training regime:** fp16 mixed precision with gradient checkpointing
- **Frameworks:** Transformers (`Trainer`), PEFT, bitsandbytes

#### Speeds, sizes, times
Roughly 10 minutes on a single Colab T4 GPU. The adapter file is small (tens of MB) compared with the 7B base model.

## Evaluation

### Training and validation loss
| Epoch | Training loss | Validation loss |
|---|---|---|
| 1 | 0.839 | 0.973 |
| 2 | 0.431 | 1.061 |

The validation set is only 10 short examples, so the validation loss is noisy. The small rise from epoch 1 to 2 hints at mild overfitting, which is why training was stopped at 2 epochs.

### Compile test
8 coding prompts that were **not** in the training data (for example merging sorted vectors, counting vowels, a stack class using `std::vector`, word frequency with `unordered_map`). Each reply's `cpp` code blocks were checked with `g++ -std=c++20 -fsyntax-only`.

- **7 of 8 code blocks compiled**
- **0 of 8 answers** were missing code

This is a syntax and type check only. It does not test whether the program gives correct output. It is a single sampled run (temperature 0.4) on a small prompt set, and it has not yet been compared against the plain base model.

## Environmental Impact

- **Hardware type:** NVIDIA T4 GPU (Google Colab)
- **Hours used:** about 0.2 hours per training run (approximate)
- **Cloud provider:** Google Colab
- **Carbon emitted:** not measured

## Technical Specifications

- **Architecture:** Qwen2.5 decoder-only transformer (7B parameters) with LoRA adapters
- **Objective:** causal language modeling (supervised fine-tuning on chat-formatted examples)
- **Software:** Transformers, PEFT, bitsandbytes, Datasets, PyTorch

## Acknowledgements

Built on [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) by the Qwen team.

## Model Card Contact

mohitsharma9761 on Hugging Face (use the Community tab of this repo).

	---
	library_name: peft
	tags:
	- lora
	- qlora
	- cpp
	- c++
	- code
	- chatbot
	license: apache-2.0
	language:
	- en
	base_model:
	- Qwen/Qwen2.5-Coder-7B-Instruct
	pipeline_tag: text-generation
	---

	# CppBuddy: a C++ tutor chatbot (QLoRA adapter for Qwen2.5-Coder-7B-Instruct)

	CppBuddy is a small LoRA adapter that turns [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) into a friendly C++ tutor. It answers C++ concept questions and coding questions in a consistent style: a short explanation, a compilable code example, and a tip. It politely redirects off-topic questions back to C++.

	This repository contains only the adapter weights, not the base model. You load it on top of Qwen2.5-Coder-7B-Instruct (see below).

	## Model Details

	- Developed by: mohitsharma9761
	- Model type: LoRA adapter (QLoRA fine-tune) for a causal language model
	- Language: English (code in C++)
	- License: Apache-2.0 (same as the base model; please check the base model page for its current terms)
	- Fine-tuned from: [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)

	## Uses

	### Direct use
	- Asking C++ concept questions (pointers, RAII, smart pointers, STL, templates, move semantics, undefined behavior, and similar)
	- Getting short C++ code examples for common tasks (searching, sorting, string handling, containers)
	- Learning and interview practice

	### Out-of-scope use
	- Production or safety-critical code without human review
	- General-knowledge chat. It was trained to redirect non-C++ questions, though this is not fully reliable.
	- Anything that needs verified correctness. Always compile, test, and read the code it writes.

	## Bias, Risks, and Limitations

	- It can be wrong. Generated code may compile and still contain logic errors, or give outdated or unsafe advice.
	- Small fine-tuning dataset (about 250 examples). The adapter mostly teaches answer style and scope; the C++ knowledge comes from the base model, so it inherits the base model's gaps.
	- Evaluation is small (see below), so treat the numbers as a rough signal, not a benchmark.
	- Off-topic redirection is learned from a handful of examples, so it will sometimes answer non-C++ questions anyway.

	## How to Get Started with the Model

	The system prompt below is the one used in training. Using a different one will make answers worse.

	````python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
	from peft import PeftModel

	BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"
	ADAPTER = "mohitsharma9761/my-cpp-chatbot"

	SYSTEM_PROMPT = "You are CppBuddy, a friendly expert C++ tutor and coding assistant (C++11 to C++23). Explain simply first, then show short, correct, compilable code in ```cpp blocks. Prefer modern C++ (smart pointers, STL, RAII). Point out undefined behavior and common mistakes. If a question is not about C++ or programming, politely steer back to C++."

	bnb = BitsAndBytesConfig(
	load_in_4bit=True,
	bnb_4bit_quant_type="nf4",
	bnb_4bit_compute_dtype=torch.float16,
	bnb_4bit_use_double_quant=True,
	)

	tokenizer = AutoTokenizer.from_pretrained(BASE)
	model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
	model = PeftModel.from_pretrained(model, ADAPTER)
	model.eval()

	messages = [
	{"role": "system", "content": SYSTEM_PROMPT},
	{"role": "user", "content": "what is RAII?"},
	]
	inputs = tokenizer.apply_chat_template(
	messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
	).to(model.device)

	with torch.no_grad():
	out = model.generate(**inputs, max_new_tokens=600, do_sample=True, temperature=0.4, top_p=0.9)

	print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
	````

	A 4-bit load fits on a free 16 GB GPU such as a Colab T4.

	## Training Details

	### Training data
	About 250 curated question and answer pairs about C++ (247 used for training, 10 held out for validation):

	- Explanation-plus-code answers (short explanation, compilable example, tip). The built-in examples with code were shown twice per epoch so the model keeps writing code.
	- Short definition-style answers for basic concepts.
	- A small set of polite redirects for off-topic questions.
	- A few corrective examples for topics where an earlier version of the model gave a confused answer (for example `new` vs `malloc`).

	Full C++ programs in the answers were checked with `g++ -std=c++20 -fsyntax-only` and any that failed to compile were dropped.

	### Training procedure

	- Method: QLoRA (4-bit NF4 base model with double quantization, LoRA adapters)
	- LoRA: r = 16, alpha = 32, dropout = 0.05, applied to `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
	- Loss: computed on the assistant reply only (prompt tokens masked)
	- Epochs: 2
	- Learning rate: 2e-4
	- Batch size: 1 per device, gradient accumulation 4 (effective batch size 4)
	- Optimizer: `paged_adamw_8bit`
	- Max sequence length: 1024 tokens
	- Training regime: fp16 mixed precision with gradient checkpointing
	- Frameworks: Transformers (`Trainer`), PEFT, bitsandbytes

	#### Speeds, sizes, times
	Roughly 10 minutes on a single Colab T4 GPU. The adapter file is small (tens of MB) compared with the 7B base model.

	## Evaluation

	### Training and validation loss
	\| Epoch \| Training loss \| Validation loss \|
	\|---\|---\|---\|
	\| 1 \| 0.839 \| 0.973 \|
	\| 2 \| 0.431 \| 1.061 \|

	The validation set is only 10 short examples, so the validation loss is noisy. The small rise from epoch 1 to 2 hints at mild overfitting, which is why training was stopped at 2 epochs.

	### Compile test
	8 coding prompts that were not in the training data (for example merging sorted vectors, counting vowels, a stack class using `std::vector`, word frequency with `unordered_map`). Each reply's `cpp` code blocks were checked with `g++ -std=c++20 -fsyntax-only`.

	- 7 of 8 code blocks compiled
	- 0 of 8 answers were missing code

	This is a syntax and type check only. It does not test whether the program gives correct output. It is a single sampled run (temperature 0.4) on a small prompt set, and it has not yet been compared against the plain base model.

	## Environmental Impact

	- Hardware type: NVIDIA T4 GPU (Google Colab)
	- Hours used: about 0.2 hours per training run (approximate)
	- Cloud provider: Google Colab
	- Carbon emitted: not measured

	## Technical Specifications

	- Architecture: Qwen2.5 decoder-only transformer (7B parameters) with LoRA adapters
	- Objective: causal language modeling (supervised fine-tuning on chat-formatted examples)
	- Software: Transformers, PEFT, bitsandbytes, Datasets, PyTorch

	## Acknowledgements

	Built on [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) by the Qwen team.

	## Model Card Contact

	mohitsharma9761 on Hugging Face (use the Community tab of this repo).