Initial model upload

7064d19 verified 8 months ago

6.78 kB

	---
	license: apache-2.0
	base_model: Qwen/Qwen2.5-7B-Instruct
	tags:
	- generated_from_trainer
	model-index:
	- name: prm
	results: []
	---

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment. -->

	[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
	<details><summary>See axolotl config</summary>

	axolotl version: `0.4.1`
	```yaml
	base_model: Qwen/Qwen2.5-7B-Instruct
	model_type: AutoModelForCausalLM
	tokenizer_type: AutoTokenizer

	load_in_8bit: false
	load_in_4bit: false
	strict: false

	datasets:
	- path: Jennny/strict_mc_label
	conversation: qwen-7b-chat
	type: sharegpt
	split: "train"
	train_on_split: "train"

	warmup_ratio: 0.05
	val_set_size: 0.0
	output_dir: ./prm
	wandb_project: preference-models
	# wandb_entity: domain-generalization
	wandb_watch:
	wandb_name: "qwen-7b-bs32_lr2e-6_prm"
	wandb_log_model:

	train_on_inputs: false

	save_safetensors: true
	#noisy_embedding_alpha: 10.0 # default for sharegpt type
	dataset_prepared_path: ~/data/preference-models/last_run_prepared

	dataset_processes: 48
	#torch_compile: true
	sequence_len: 8192
	sample_packing: true
	pad_to_sequence_len: true

	trust_remote_code: True
	adapter:
	lora_model_dir:
	#lora_r: 32
	#lora_alpha: 16
	#lora_dropout: 0.05
	#lora_target_linear: true
	#lora_fan_in_fan_out:

	gradient_checkpointing: True

	#warmup_ratio: 0.1
	gradient_accumulation_steps: 4
	micro_batch_size: 1
	num_epochs: 1
	#max_steps: 10
	#optimizer: adamw_torch_fused
	optimizer: paged_adamw_32bit
	#lr_scheduler: constant_with_warmup
	lr_scheduler: cosine
	learning_rate: 2.0e-6

	weight_decay: 0.0
	max_grad_norm: 1.0

	group_by_length: false
	bf16: auto
	fp16: false
	tf32: true

	early_stopping_patience:
	local_rank:
	logging_steps: 2
	xformers_attention:
	flash_attention: true

	eval_steps:
	eval_table_size:
	eval_table_max_new_tokens:
	#save_steps: 100
	save_strategy: "epoch"
	save_total_limit: 4
	#save_safetensors: false
	debug:

	ddp: #true
	deepspeed: #deepspeed/zero1.json # multi-gpu only

	fsdp:
	fsdp_config:
	special_tokens:
	pad_token: <\|end_of_text\|>

	```

	</details><br>

	# prm

	This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the None dataset.
	It achieves the following results on the evaluation set:
	- Loss: 0.0457

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- learning_rate: 1e-05
	- train_batch_size: 1
	- eval_batch_size: 1
	- seed: 42
	- distributed_type: multi-GPU
	- num_devices: 8
	- gradient_accumulation_steps: 4
	- total_train_batch_size: 32
	- total_eval_batch_size: 8
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: cosine
	- lr_scheduler_warmup_steps: 3
	- num_epochs: 2

	### Training results

	\| Training Loss \| Epoch \| Step \| Validation Loss \|
	\|:-------------:\|:------:\|:----:\|:---------------:\|
	\| No log \| 0.0296 \| 1 \| 3.5810 \|
	\| 3.6289 \| 0.0593 \| 2 \| 2.8980 \|
	\| 3.6289 \| 0.0889 \| 3 \| 1.3876 \|
	\| 2.1145 \| 0.1185 \| 4 \| 0.4411 \|
	\| 2.1145 \| 0.1481 \| 5 \| 0.2394 \|
	\| 0.3565 \| 0.1778 \| 6 \| 0.1183 \|
	\| 0.3565 \| 0.2074 \| 7 \| 0.0618 \|
	\| 0.0918 \| 0.2370 \| 8 \| 0.1148 \|
	\| 0.0918 \| 0.2667 \| 9 \| 0.0619 \|
	\| 0.0815 \| 0.2963 \| 10 \| 0.0790 \|
	\| 0.0815 \| 0.3259 \| 11 \| 0.0636 \|
	\| 0.0707 \| 0.3556 \| 12 \| 0.0596 \|
	\| 0.0707 \| 0.3852 \| 13 \| 0.0577 \|
	\| 0.0521 \| 0.4148 \| 14 \| 0.0554 \|
	\| 0.0521 \| 0.4444 \| 15 \| 0.0563 \|
	\| 0.0587 \| 0.4741 \| 16 \| 0.0559 \|
	\| 0.0587 \| 0.5037 \| 17 \| 0.0517 \|
	\| 0.0519 \| 0.5333 \| 18 \| 0.0529 \|
	\| 0.0519 \| 0.5630 \| 19 \| 0.0559 \|
	\| 0.0512 \| 0.5926 \| 20 \| 0.0524 \|
	\| 0.0512 \| 0.6222 \| 21 \| 0.0513 \|
	\| 0.0505 \| 0.6519 \| 22 \| 0.0513 \|
	\| 0.0505 \| 0.6815 \| 23 \| 0.0512 \|
	\| 0.1024 \| 0.7111 \| 24 \| 0.0521 \|
	\| 0.1024 \| 0.7407 \| 25 \| 0.0507 \|
	\| 0.0486 \| 0.7704 \| 26 \| 0.0498 \|
	\| 0.0486 \| 0.8 \| 27 \| 0.0505 \|
	\| 0.0485 \| 0.8296 \| 28 \| 0.0495 \|
	\| 0.0485 \| 0.8593 \| 29 \| 0.0482 \|
	\| 0.0497 \| 0.8889 \| 30 \| 0.0495 \|
	\| 0.0497 \| 0.9185 \| 31 \| 0.0511 \|
	\| 0.074 \| 0.9481 \| 32 \| 0.0485 \|
	\| 0.074 \| 0.9778 \| 33 \| 0.0484 \|
	\| 0.047 \| 1.0074 \| 34 \| 0.0493 \|
	\| 0.047 \| 1.0148 \| 35 \| 0.0483 \|
	\| 0.0448 \| 1.0444 \| 36 \| 0.0480 \|
	\| 0.0448 \| 1.0741 \| 37 \| 0.0483 \|
	\| 0.041 \| 1.1037 \| 38 \| 0.0477 \|
	\| 0.041 \| 1.1333 \| 39 \| 0.0471 \|
	\| 0.0463 \| 1.1630 \| 40 \| 0.0467 \|
	\| 0.0463 \| 1.1926 \| 41 \| 0.0465 \|
	\| 0.0435 \| 1.2222 \| 42 \| 0.0464 \|
	\| 0.0435 \| 1.2519 \| 43 \| 0.0469 \|
	\| 0.0457 \| 1.2815 \| 44 \| 0.0472 \|
	\| 0.0457 \| 1.3111 \| 45 \| 0.0467 \|
	\| 0.0484 \| 1.3407 \| 46 \| 0.0462 \|
	\| 0.0484 \| 1.3704 \| 47 \| 0.0461 \|
	\| 0.0449 \| 1.4 \| 48 \| 0.0459 \|
	\| 0.0449 \| 1.4296 \| 49 \| 0.0458 \|
	\| 0.0481 \| 1.4593 \| 50 \| 0.0458 \|
	\| 0.0481 \| 1.4889 \| 51 \| 0.0459 \|
	\| 0.0433 \| 1.5185 \| 52 \| 0.0461 \|
	\| 0.0433 \| 1.5481 \| 53 \| 0.0461 \|
	\| 0.042 \| 1.5778 \| 54 \| 0.0458 \|
	\| 0.042 \| 1.6074 \| 55 \| 0.0458 \|
	\| 0.0428 \| 1.6370 \| 56 \| 0.0457 \|
	\| 0.0428 \| 1.6667 \| 57 \| 0.0457 \|
	\| 0.0443 \| 1.6963 \| 58 \| 0.0457 \|
	\| 0.0443 \| 1.7259 \| 59 \| 0.0457 \|
	\| 0.0428 \| 1.7556 \| 60 \| 0.0457 \|
	\| 0.0428 \| 1.7852 \| 61 \| 0.0457 \|
	\| 0.042 \| 1.8148 \| 62 \| 0.0458 \|
	\| 0.042 \| 1.8444 \| 63 \| 0.0457 \|
	\| 0.0433 \| 1.8741 \| 64 \| 0.0457 \|
	\| 0.0433 \| 1.9037 \| 65 \| 0.0458 \|
	\| 0.0397 \| 1.9333 \| 66 \| 0.0457 \|


	### Framework versions

	- Transformers 4.43.3
	- Pytorch 2.1.2+cu121
	- Datasets 2.19.1
	- Tokenizers 0.19.1