README.md

metadata

library_name: peft
tags:
  - lora
  - qlora
  - cpp
  - c++
  - code
  - chatbot
license: apache-2.0
language:
  - en
base_model:
  - Qwen/Qwen2.5-Coder-7B-Instruct
pipeline_tag: text-generation

CppBuddy: a C++ tutor chatbot (QLoRA adapter for Qwen2.5-Coder-7B-Instruct)

CppBuddy is a small LoRA adapter that turns Qwen2.5-Coder-7B-Instruct into a friendly C++ tutor. It answers C++ concept questions and coding questions in a consistent style: a short explanation, a compilable code example, and a tip. It politely redirects off-topic questions back to C++.

This repository contains only the adapter weights, not the base model. You load it on top of Qwen2.5-Coder-7B-Instruct (see below).

Model Details

• Developed by: mohitsharma9761
• Model type: LoRA adapter (QLoRA fine-tune) for a causal language model
• Language: English (code in C++)
• License: Apache-2.0 (same as the base model; please check the base model page for its current terms)
• Fine-tuned from: Qwen/Qwen2.5-Coder-7B-Instruct

Uses

Direct use

• Asking C++ concept questions (pointers, RAII, smart pointers, STL, templates, move semantics, undefined behavior, and similar)
• Getting short C++ code examples for common tasks (searching, sorting, string handling, containers)
• Learning and interview practice

Out-of-scope use

• Production or safety-critical code without human review
• General-knowledge chat. It was trained to redirect non-C++ questions, though this is not fully reliable.
• Anything that needs verified correctness. Always compile, test, and read the code it writes.

Bias, Risks, and Limitations

• It can be wrong. Generated code may compile and still contain logic errors, or give outdated or unsafe advice.
• Small fine-tuning dataset (about 250 examples). The adapter mostly teaches answer style and scope; the C++ knowledge comes from the base model, so it inherits the base model's gaps.
• Evaluation is small (see below), so treat the numbers as a rough signal, not a benchmark.
• Off-topic redirection is learned from a handful of examples, so it will sometimes answer non-C++ questions anyway.

How to Get Started with the Model

The system prompt below is the one used in training. Using a different one will make answers worse.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"
ADAPTER = "mohitsharma9761/my-cpp-chatbot"

SYSTEM_PROMPT = "You are CppBuddy, a friendly expert C++ tutor and coding assistant (C++11 to C++23). Explain simply first, then show short, correct, compilable code in ```cpp blocks. Prefer modern C++ (smart pointers, STL, RAII). Point out undefined behavior and common mistakes. If a question is not about C++ or programming, politely steer back to C++."

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "what is RAII?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=600, do_sample=True, temperature=0.4, top_p=0.9)

print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

A 4-bit load fits on a free 16 GB GPU such as a Colab T4.

Training Details

Training data

About 250 curated question and answer pairs about C++ (247 used for training, 10 held out for validation):

• Explanation-plus-code answers (short explanation, compilable example, tip). The built-in examples with code were shown twice per epoch so the model keeps writing code.
• Short definition-style answers for basic concepts.
• A small set of polite redirects for off-topic questions.
• A few corrective examples for topics where an earlier version of the model gave a confused answer (for example new vs malloc).

Full C++ programs in the answers were checked with g++ -std=c++20 -fsyntax-only and any that failed to compile were dropped.

Training procedure

• Method: QLoRA (4-bit NF4 base model with double quantization, LoRA adapters)
• LoRA: r = 16, alpha = 32, dropout = 0.05, applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Loss: computed on the assistant reply only (prompt tokens masked)
• Epochs: 2
• Learning rate: 2e-4
• Batch size: 1 per device, gradient accumulation 4 (effective batch size 4)
• Optimizer: paged_adamw_8bit
• Max sequence length: 1024 tokens
• Training regime: fp16 mixed precision with gradient checkpointing
• Frameworks: Transformers (Trainer), PEFT, bitsandbytes

Speeds, sizes, times

Roughly 10 minutes on a single Colab T4 GPU. The adapter file is small (tens of MB) compared with the 7B base model.

Evaluation

Training and validation loss

Epoch	Training loss	Validation loss
1	0.839	0.973
2	0.431	1.061

The validation set is only 10 short examples, so the validation loss is noisy. The small rise from epoch 1 to 2 hints at mild overfitting, which is why training was stopped at 2 epochs.

Compile test

8 coding prompts that were not in the training data (for example merging sorted vectors, counting vowels, a stack class using std::vector, word frequency with unordered_map). Each reply's cpp code blocks were checked with g++ -std=c++20 -fsyntax-only.

• 7 of 8 code blocks compiled
• 0 of 8 answers were missing code

This is a syntax and type check only. It does not test whether the program gives correct output. It is a single sampled run (temperature 0.4) on a small prompt set, and it has not yet been compared against the plain base model.

Environmental Impact

• Hardware type: NVIDIA T4 GPU (Google Colab)
• Hours used: about 0.2 hours per training run (approximate)
• Cloud provider: Google Colab
• Carbon emitted: not measured

Technical Specifications

• Architecture: Qwen2.5 decoder-only transformer (7B parameters) with LoRA adapters
• Objective: causal language modeling (supervised fine-tuning on chat-formatted examples)
• Software: Transformers, PEFT, bitsandbytes, Datasets, PyTorch

Acknowledgements

Built on Qwen2.5-Coder-7B-Instruct by the Qwen team.

Model Card Contact

mohitsharma9761 on Hugging Face (use the Community tab of this repo).