--- library_name: peft tags: - lora - qlora - cpp - c++ - code - chatbot license: apache-2.0 language: - en base_model: - Qwen/Qwen2.5-Coder-7B-Instruct pipeline_tag: text-generation --- # CppBuddy: a C++ tutor chatbot (QLoRA adapter for Qwen2.5-Coder-7B-Instruct) CppBuddy is a small **LoRA adapter** that turns [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) into a friendly C++ tutor. It answers C++ concept questions and coding questions in a consistent style: a short explanation, a compilable code example, and a tip. It politely redirects off-topic questions back to C++. This repository contains **only the adapter weights**, not the base model. You load it on top of Qwen2.5-Coder-7B-Instruct (see below). ## Model Details - **Developed by:** mohitsharma9761 - **Model type:** LoRA adapter (QLoRA fine-tune) for a causal language model - **Language:** English (code in C++) - **License:** Apache-2.0 (same as the base model; please check the base model page for its current terms) - **Fine-tuned from:** [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) ## Uses ### Direct use - Asking C++ concept questions (pointers, RAII, smart pointers, STL, templates, move semantics, undefined behavior, and similar) - Getting short C++ code examples for common tasks (searching, sorting, string handling, containers) - Learning and interview practice ### Out-of-scope use - Production or safety-critical code without human review - General-knowledge chat. It was trained to redirect non-C++ questions, though this is not fully reliable. - Anything that needs verified correctness. Always compile, test, and read the code it writes. ## Bias, Risks, and Limitations - **It can be wrong.** Generated code may compile and still contain logic errors, or give outdated or unsafe advice. - **Small fine-tuning dataset** (about 250 examples). The adapter mostly teaches *answer style and scope*; the C++ knowledge comes from the base model, so it inherits the base model's gaps. - **Evaluation is small** (see below), so treat the numbers as a rough signal, not a benchmark. - Off-topic redirection is learned from a handful of examples, so it will sometimes answer non-C++ questions anyway. ## How to Get Started with the Model The system prompt below is the one used in training. Using a different one will make answers worse. ````python import torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel BASE = "Qwen/Qwen2.5-Coder-7B-Instruct" ADAPTER = "mohitsharma9761/my-cpp-chatbot" SYSTEM_PROMPT = "You are CppBuddy, a friendly expert C++ tutor and coding assistant (C++11 to C++23). Explain simply first, then show short, correct, compilable code in ```cpp blocks. Prefer modern C++ (smart pointers, STL, RAII). Point out undefined behavior and common mistakes. If a question is not about C++ or programming, politely steer back to C++." bnb = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True, ) tokenizer = AutoTokenizer.from_pretrained(BASE) model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto") model = PeftModel.from_pretrained(model, ADAPTER) model.eval() messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": "what is RAII?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt", return_dict=True ).to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=600, do_sample=True, temperature=0.4, top_p=0.9) print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) ```` A 4-bit load fits on a free 16 GB GPU such as a Colab T4. ## Training Details ### Training data About **250 curated question and answer pairs** about C++ (247 used for training, 10 held out for validation): - Explanation-plus-code answers (short explanation, compilable example, tip). The built-in examples with code were shown twice per epoch so the model keeps writing code. - Short definition-style answers for basic concepts. - A small set of polite redirects for off-topic questions. - A few corrective examples for topics where an earlier version of the model gave a confused answer (for example `new` vs `malloc`). Full C++ programs in the answers were checked with `g++ -std=c++20 -fsyntax-only` and any that failed to compile were dropped. ### Training procedure - **Method:** QLoRA (4-bit NF4 base model with double quantization, LoRA adapters) - **LoRA:** r = 16, alpha = 32, dropout = 0.05, applied to `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` - **Loss:** computed on the assistant reply only (prompt tokens masked) - **Epochs:** 2 - **Learning rate:** 2e-4 - **Batch size:** 1 per device, gradient accumulation 4 (effective batch size 4) - **Optimizer:** `paged_adamw_8bit` - **Max sequence length:** 1024 tokens - **Training regime:** fp16 mixed precision with gradient checkpointing - **Frameworks:** Transformers (`Trainer`), PEFT, bitsandbytes #### Speeds, sizes, times Roughly 10 minutes on a single Colab T4 GPU. The adapter file is small (tens of MB) compared with the 7B base model. ## Evaluation ### Training and validation loss | Epoch | Training loss | Validation loss | |---|---|---| | 1 | 0.839 | 0.973 | | 2 | 0.431 | 1.061 | The validation set is only 10 short examples, so the validation loss is noisy. The small rise from epoch 1 to 2 hints at mild overfitting, which is why training was stopped at 2 epochs. ### Compile test 8 coding prompts that were **not** in the training data (for example merging sorted vectors, counting vowels, a stack class using `std::vector`, word frequency with `unordered_map`). Each reply's `cpp` code blocks were checked with `g++ -std=c++20 -fsyntax-only`. - **7 of 8 code blocks compiled** - **0 of 8 answers** were missing code This is a syntax and type check only. It does not test whether the program gives correct output. It is a single sampled run (temperature 0.4) on a small prompt set, and it has not yet been compared against the plain base model. ## Environmental Impact - **Hardware type:** NVIDIA T4 GPU (Google Colab) - **Hours used:** about 0.2 hours per training run (approximate) - **Cloud provider:** Google Colab - **Carbon emitted:** not measured ## Technical Specifications - **Architecture:** Qwen2.5 decoder-only transformer (7B parameters) with LoRA adapters - **Objective:** causal language modeling (supervised fine-tuning on chat-formatted examples) - **Software:** Transformers, PEFT, bitsandbytes, Datasets, PyTorch ## Acknowledgements Built on [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) by the Qwen team. ## Model Card Contact mohitsharma9761 on Hugging Face (use the Community tab of this repo).