| license: apache-2.0 | |
| base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct | |
| tags: | |
| - angular | |
| - lora | |
| - peft | |
| library_name: peft | |
| # angular-coder-30b-a3b-lora | |
| The PEFT LoRA adapter behind `bharathmuppa/angular-coder-30b-a3b`. Apply it to `Qwen/Qwen3-Coder-30B-A3B-Instruct`: | |
| ```python | |
| from peft import PeftModel | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| base = "Qwen/Qwen3-Coder-30B-A3B-Instruct" | |
| tok = AutoTokenizer.from_pretrained(base) | |
| model = AutoModelForCausalLM.from_pretrained(base, dtype="auto", device_map="auto") | |
| model = PeftModel.from_pretrained(model, "bharathmuppa/angular-coder-30b-a3b-lora") | |
| ``` | |
| Rank 16, alpha 16, dropout 0.05, target modules: q/k/v/o projections of every | |
| layer. Trained 796 optimizer steps (2.0 epochs), final eval loss | |
| 1.1662. `train_summary.json` in this repo holds the run settings. See the | |
| merged model card for data and training details. | |