| --- |
| license: mit |
| base_model: unsloth/Qwen2.5-3B-Instruct |
| library_name: peft |
| tags: |
| - qwen2.5 |
| - unsloth |
| - lora |
| - peft |
| - interview |
| - software-engineering |
| - text-generation |
| language: |
| - en |
| pipeline_tag: text-generation |
| datasets: |
| - shimogerald/interview-coach-dataset |
| --- |
| |
| # Interview Coach LoRA (Qwen2.5-3B-Instruct) |
|
|
| LoRA adapter fine-tuned for software-engineering interview Q&A coaching. |
|
|
| ## Model Details |
|
|
| - **Base model:** `unsloth/Qwen2.5-3B-Instruct` |
| - **Method:** QLoRA (4-bit) + LoRA via Unsloth |
| - **LoRA:** `r=16`, `lora_alpha=16`, `lora_dropout=0` |
| - **Target modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
| - **Context length:** 2048 |
| - **Language:** English |
|
|
| ## Training Data |
|
|
| Fine-tuned on [`shimogerald/interview-coach-dataset`](https://huggingface.co/datasets/shimogerald/interview-coach-dataset) (chat `messages` format, ~90/10 train/val). |
|
|
| ## Intended Use |
|
|
| Practice / coaching-style answers to technical interview questions (APIs, systems, coding concepts, behavioral, etc.). |
|
|
| ## Limitations |
|
|
| - Synthetic training data may contain errors |
| - Not a substitute for real interview feedback |
| - May hallucinate technical details |
| - English only |
|
|
| ## How to Use |
|
|
| ```python |
| from unsloth import FastLanguageModel |
| |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name="shimogerald/lora_interview_coach", |
| max_seq_length=2048, |
| load_in_4bit=True, |
| ) |
| FastLanguageModel.for_inference(model) |
| |
| messages = [{"role": "user", "content": "What is the difference between PUT and PATCH?"}] |
| prompt = tokenizer.apply_chat_template( |
| messages, tokenize=False, add_generation_prompt=True |
| ) |
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| out = model.generate(**inputs, max_new_tokens=256, do_sample=False) |
| print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)) |
| ``` |
|
|
| If loading the adapter separately fails, load the base model then attach this repo with PEFT `PeftModel.from_pretrained`. |
|
|
| ## Training Setup (summary) |
|
|
| - Optimizer: AdamW |
| - LR schedule: cosine with warmup |
| - Epochs: 3 |
| - Framework: Unsloth + Accelerate + Transformers |
|
|
|
|
| This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) |
|
|
| [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth) |
|
|
