Upload README.md with huggingface_hub

README.md

@@ -1,63 +1,78 @@
----
-base_model: unsloth/Qwen2.5-Coder-7B-Instruct
-library_name: peft
-model_name: codek-lora
-tags:
-- base_model:adapter:unsloth/Qwen2.5-Coder-7B-Instruct
-- lora
-- sft
-- transformers
-- trl
-- unsloth
-licence: license
-pipeline_tag: text-generation
----
-
-# Model Card for codek-lora
-
-This model is a fine-tuned version of [unsloth/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct).
-It has been trained using [TRL](https://github.com/huggingface/trl).
-
-## Quick start
-
-```python
-from transformers import pipeline
-
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
-```
-
-## Training procedure
-
- 
-
-
-This model was trained with SFT.
-
-### Framework versions
-
-- PEFT 0.18.1
-- TRL: 0.23.1
-- Transformers: 4.57.6
-- Pytorch: 2.10.0+cu128
-- Datasets: 4.3.0
-- Tokenizers: 0.22.2
-
-## Citations
-
-
-
-Cite TRL as:
-    
-```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
-}
-```
+---
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+  - lora
+  - peft
+  - qwen2.5
+  - qwen2.5-coder
+  - code
+  - reasoning
+  - pedagogy
+  - fine-tuned
+language:
+  - en
+license: apache-2.0
+datasets:
+  - mechramc/codek-v1
+---
+
+# CodeK LoRA -- Qwen2.5-Coder-7B-Instruct
+
+A LoRA adapter fine-tuned on the **CodeK v1** dataset: a reasoning-first, pedagogical coding dataset in Karpathy's register.
+
+## What it does
+
+Teaches the model to reason like an experienced engineer -- not just write code, but:
+- Decompose problems before implementing
+- Explain *why* a solution is the right one
+- Identify and diagnose subtle bugs
+- Contrast clean vs clunky implementations with precision
+- Apply a hypothesis->experiment->evaluate loop to code
+
+## Training
+
+| Setting | Value |
+|---------|-------|
+| Base model | `Qwen/Qwen2.5-Coder-7B-Instruct` |
+| Method | LoRA (RS-LoRA) |
+| Rank | 16, Alpha 32 |
+| Dropout | 0.05 |
+| Epochs | 3 |
+| Batch size | 2 (effective 8 with grad accum) |
+| Learning rate | 2e-4 |
+| Train pairs | 2,351 |
+| Val pairs | 262 |
+| Final eval loss | 0.0600 |
+| Hardware | RunPod A100 80GB |
+| Training time | 59 minutes |
+
+## Dataset
+
+[mechramc/codek-v1](https://huggingface.co/datasets/mechramc/codek-v1) -- 201 seeds, 2,613 ShareGPT pairs across 4 augmentation passes:
+
+- **Pass 1** -- Reasoning decomposition (intuition, plan, why this solution, alternatives)
+- **Pass 2** -- Debugging (introduce + diagnose a subtle one-line bug)
+- **Pass 3** -- Contrast (clunky vs clean, with specific explanation)
+- **Pass 4** -- Research loop (hypothesis, minimal test, success metric, simplicity check, abandon condition)
+
+## Usage
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
+model = PeftModel.from_pretrained(base, "mechramc/codek-qwen2.5-coder-7b-lora")
+tokenizer = AutoTokenizer.from_pretrained("mechramc/codek-qwen2.5-coder-7b-lora")
+```
+
+Or with Unsloth (2-5x faster inference):
+```python
+from unsloth import FastLanguageModel
+model, tokenizer = FastLanguageModel.from_pretrained(
+    "mechramc/codek-qwen2.5-coder-7b-lora",
+    max_seq_length=4096,
+)
+```