| --- |
| base_model: Qwen/Qwen3-4B-Base |
| library_name: peft |
| license: other |
| language: |
| - en |
| tags: |
| - lora |
| - peft |
| - adapter |
| - adaption |
| - autoscientist |
| --- |
| |
| # adaption_fim_event_classification |
| |
| ## Model Training |
| |
| A LORA adapter for `Qwen/Qwen3-4B-Base`. This model was trained with SFT using [Adaption](https://adaptionlabs.ai)'s AutoScientist on the fim_event_classification dataset. |
| |
| |
|  |
| |
| ### AutoScientist Config |
| |
| ```json |
| { |
| "job_id": "15245b3b-55a2-4540-9b8a-9ff40a459d60", |
| "training_experiment_id": "9642fe2d-3d48-4612-86cc-d9239c844595", |
| "original_model_name": "Qwen/Qwen3-4B-Base", |
| "trained_model_name": "adaption_fim_event_classification", |
| "training_method": "sft", |
| "training_type": "lora", |
| "data_format": "chat", |
| "hyperparams": { |
| "lora": "true", |
| "lora_r": 32, |
| "n_evals": 5, |
| "n_epochs": 5, |
| "batch_size": "max", |
| "lora_alpha": 64, |
| "lora_dropout": 0.05, |
| "min_lr_ratio": 0.2, |
| "warmup_ratio": 0.05, |
| "weight_decay": 0.01, |
| "learning_rate": 0.0001, |
| "max_grad_norm": 1, |
| "base_model_size": "4B", |
| "train_on_inputs": "false", |
| "training_method": "sft", |
| "lr_scheduler_type": "cosine", |
| "scheduler_num_cycles": 0.5, |
| "lora_trainable_modules": "k_proj,up_proj,o_proj,q_proj,down_proj,v_proj,gate_proj" |
| } |
| } |
| ``` |
| |
| ## Training Data |
|
|
| The model was trained on 27,254 rows of adapted data with the following domain distribution: technology (71%), science (5%), academic-education (4%), governance (2%), math (2%), history (2%), corporate-business (2%), code (1%), writing-editing-communication (1%), entertainment (1%), cooking (1%), culture (1%), geography (1%), sports (1%), language (1%), travel (1%), fitness-sports (1%), music (0%), medical (0%), how-to (0%), product-advice (0%), animal-nature (0%), religion (0%), transportation (0%), legal (0%), personal-finance (0%), data-analysis-visualization (0%), personal-growth (0%), career-workplace (0%), games (0%), agriculture (0%), parenting-family (0%), art (0%), news (0%), architecture-design (0%), dating (0%), fashion-beauty (0%), marketing (0%), social (0%), hr (0%), literature (0%), roleplay (0%), market-analysis (0%), other (0%), outdoor-recreation (0%), media (0%), beauty (0%), philosophy (0%), survival-skills (0%), humanitarian-aid (0%), survival-outdoors (0%), family-relationships (0%), academic-游戏副本 (0%), survival-preparedness (0%). |
|
|
| ## Model Evaluation |
|
|
| The model was evaluated on an in-distribution held-out test set as well as a broader domain-specific test set to measure generalization. |
|
|
|
|
|  |
|
|
| | Domain | Win rate vs. base model | |
| | --- | --- | |
| | technology | 71% | |
|
|
| ## How to use |
|
|
| ```bash |
| pip install torch transformers peft accelerate |
| ``` |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline |
| from peft import PeftModel |
| |
| BASE = "Qwen/Qwen3-4B-Base" |
| ADAPTER = "dowdryan/adaption_fim_event_classification" |
| |
| base = AutoModelForCausalLM.from_pretrained(BASE, dtype="auto", device_map="auto") |
| # Merge the LoRA weights into the base for faster inference |
| model = PeftModel.from_pretrained(base, ADAPTER).merge_and_unload() |
| model.eval() |
| tokenizer = AutoTokenizer.from_pretrained(BASE) |
| |
| pipe = pipeline("text-generation", model=model, tokenizer=tokenizer) |
| |
| messages = [{"role": "user", "content": "Hello!"}] |
| out = pipe(messages, max_new_tokens=512) |
| print(out[0]["generated_text"][-1]["content"]) |
| ``` |
|
|
