| base_model: Qwen/Qwen2.5-3B-Instruct | |
| library_name: peft | |
| license: mit | |
| tags: | |
| - lora | |
| - json | |
| - structured-output | |
| - text-generation | |
| - qwen2 | |
| - sft | |
| - trl | |
| language: | |
| - en | |
| pipeline_tag: text-generation | |
| # Qwen2.5-3B JSON Structured Output (LoRA) | |
| A LoRA adapter that enhances Qwen2.5-3B-Instruct for extracting structured JSON from unstructured text. | |
| ## Training Details | |
| | Parameter | Value | | |
| |-----------|-------| | |
| | Base Model | Qwen/Qwen2.5-3B-Instruct | | |
| | Method | QLoRA (4-bit NF4) | | |
| | LoRA Rank | 16 | | |
| | LoRA Alpha | 32 | | |
| | Target Modules | q,k,v,o,gate,up,down | | |
| | Trainable Params | 29.9M (1.76%) | | |
| | Training Examples | 20 curated | | |
| | Epochs | 5 | | |
| | Learning Rate | 5e-5 | | |
| | Final Loss | 1.506 | | |
| | Hardware | NVIDIA RTX 3090 x2 | | |
| ## Quick Start | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig | |
| from peft import PeftModel | |
| import torch, json | |
| bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16) | |
| tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct") | |
| model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", quantization_config=bnb, device_map="auto") | |
| model = PeftModel.from_pretrained(model, "2reb/Qwen2.5-3B-JSON-StructuredOutput") | |
| ``` | |
| ## License | |
| MIT | |