[2025-12-21 13:06:18,420] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) Using the `WANDB_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none). 2025-12-21 13:06:20 - INFO - __main__ - Model parameters ModelArguments(base_model_revision=None, model_name_or_path='Qwen/Qwen2.5-0.5B', ref_model_name_or_path='Qwen/Qwen2.5-Math-1.5B-Instruct', model_revision='main', model_code_revision=None, torch_dtype='bfloat16', tokenizer_name_or_path=None, trust_remote_code=False, attn_implementation='sdpa', use_peft=False, lora_r=16, lora_alpha=32, lora_dropout=0.05, lora_target_modules=None, lora_modules_to_save=None, load_in_8bit=False, load_in_4bit=False, bnb_4bit_quant_type='nf4', use_bnb_nested_quant=False, bnb_4bit_quant_storage='uint8') 2025-12-21 13:06:20 - INFO - __main__ - Data parameters DataArguments(chat_template=None, dataset_mixer={'distillm-2-master/data/reformatted/qwen_math': 1.0}, text_column='text', dataset_splits=['train', 'test'], dataset_configs=None, preprocessing_num_workers=8, truncation_side=None, auto_insert_empty_system_msg=True) 2025-12-21 13:06:20 - INFO - __main__ - Training/evaluation parameters DPOConfig( _n_gpu=1, accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False}, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, batch_eval_metrics=False, beta=0.1, bf16=True, bf16_full_eval=False, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=0, dataloader_persistent_workers=False, dataloader_pin_memory=True, dataloader_prefetch_factor=None, dataset_num_proc=None, ddp_backend=None, ddp_broadcast_buffers=None, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=1800, debug=[], deepspeed=None, disable_dropout=True, disable_tqdm=False, dispatch_batches=None, do_eval=True, do_predict=False, do_train=False, eval_accumulation_steps=None, eval_delay=0, eval_do_concat_batches=True, eval_on_start=False, eval_steps=10000000, eval_strategy=no, eval_use_gather_object=False, evaluation_strategy=None, f_alpha_divergence_coef=1.0, f_divergence_type=FDivergenceType.REVERSE_KL, force_use_ref_model=False, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}, fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, generate_during_eval=False, gradient_accumulation_steps=4, gradient_checkpointing=False, gradient_checkpointing_kwargs=None, greater_is_better=None, group_by_length=False, half_precision_backend=auto, hub_always_push=False, hub_model_id=qwen, hub_model_revision=main, hub_private_repo=False, hub_strategy=every_save, hub_token=, ignore_data_skip=False, include_inputs_for_metrics=False, include_num_input_tokens_seen=False, include_tokens_per_second=False, is_encoder_decoder=None, jit_mode_eval=False, label_names=None, label_pad_token_id=-100, label_smoothing=0, label_smoothing_factor=0.0, learning_rate=5e-05, length_column_name=length, load_best_model_at_end=False, local_rank=0, log_level=info, log_level_replica=warning, log_on_each_node=True, logging_dir=./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/runs/Dec21_13-06-20_ai-train-a40-1, logging_first_step=True, logging_nan_inf_filter=True, logging_steps=10, logging_strategy=steps, loss_type=distillm_v2, lr_scheduler_kwargs={}, lr_scheduler_type=cosine, max_grad_norm=1.0, max_length=256, max_prompt_length=128, max_steps=-1, max_target_length=None, metric_for_best_model=None, model_adapter_name=None, model_init_kwargs=None, model_type=qwen, mp_parameters=, neftune_noise_alpha=None, no_cuda=False, num_train_epochs=2, optim=adamw_torch, optim_args=None, optim_target_modules=None, output_dir=./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2, overwrite_output_dir=False, padding_value=None, past_index=-1, per_device_eval_batch_size=2, per_device_train_batch_size=2, precompute_ref_log_probs=False, prediction_loss_only=False, proj_lr=0.0005, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=, ray_scope=last, ref_adapter_name=None, ref_model_init_kwargs=None, ref_model_mixup_alpha=0.9, ref_model_sync_steps=64, reference_free=False, remove_unused_columns=False, report_to=[], restore_callback_states_from_checkpoint=False, resume_from_checkpoint=None, rpo_alpha=None, run_name=qwen2.5-0.5b-span-distillm2, save_on_each_node=False, save_only_model=True, save_safetensors=True, save_steps=500, save_strategy=epoch, save_total_limit=3, seed=42, skip_memory_metrics=True, split_batches=None, split_layer_mapping=[0, 1, 6, 6], student_layer_mapping=[14, 16, 18, 20, 22, 24], sync_ref_model=False, teacher_layer_mapping=[18, 20, 22, 24, 26, 28], tf32=None, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, torch_empty_cache_steps=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, truncation_mode=keep_end, use_cpu=False, use_dsa=False, use_hs=False, use_ipex=False, use_legacy_prediction_loop=False, use_mps_device=False, w_span_loss=1.0, warmup_ratio=0.1, warmup_steps=0, weight_decay=0.0, ) 2025-12-21 13:06:20 - INFO - __main__ - Training on the following splits: ['train : 30000', 'test : 10'] [INFO|tokenization_utils_base.py:2289] 2025-12-21 13:06:21,074 >> loading file vocab.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/vocab.json [INFO|tokenization_utils_base.py:2289] 2025-12-21 13:06:21,074 >> loading file merges.txt from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/merges.txt [INFO|tokenization_utils_base.py:2289] 2025-12-21 13:06:21,074 >> loading file tokenizer.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/tokenizer.json [INFO|tokenization_utils_base.py:2289] 2025-12-21 13:06:21,074 >> loading file added_tokens.json from cache at None [INFO|tokenization_utils_base.py:2289] 2025-12-21 13:06:21,074 >> loading file special_tokens_map.json from cache at None [INFO|tokenization_utils_base.py:2289] 2025-12-21 13:06:21,074 >> loading file tokenizer_config.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/tokenizer_config.json [INFO|tokenization_utils_base.py:2533] 2025-12-21 13:06:21,345 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. 2025-12-21 13:06:21 - INFO - __main__ - Prompt sample 20952 of the raw training set: <|im_start|>system Put your final answer within \boxed{}.<|im_end|> <|im_start|>user A teacher has a class with $24$ students in it. If she wants to split the students into equal groups of at most $10$ students each, what is the least number of groups that she needs?<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Chosen sample 20952 of the raw training set: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>assistant To determine the least number of groups the teacher needs to split the students into equal groups of at most 10 students each, we can follow these steps: 1. Identify the total number of students, which is 24. 2. Identify the maximum number of students per group, which is 10. 3. Divide the total number of students by the maximum number of students per group to find out how many groups are needed. Perform the division: \[ 24 \div 10 = 2.4 \] Since we cannot have a fraction of a group, we need to round up to the next whole number. This is because even if there are some students left over after forming 2 groups of 10, we still need an additional group for these remaining students. Therefore, we round 2.4 up to 3. Thus, the least number of groups needed is \(\boxed{3}\).<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Rejected sample 20952 of the raw training set: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>assistant A bag holds 6 cards numbered from $1$ to $6$. After drawing cards that are even, what are the possible outcomes if one card from each draw is removed from the bag?ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(6, 3))";ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(1, 3))";ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(1, 3))"; ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(5, 1))";ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(6, 1))"; ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(1, 2))$ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(3, 1))";ViewById="text5" onclick="$($text5).html(giveTheRandomNumber(6, 2))"; ViewById="button1" onclick="$($text3).html($text3 + $text1);";ViewById="button1" onclick="$($text3).html($text3 * $text2);"; ViewById="button1" onclick="$($text3).html($text3 - $text2);"; ViewById="button1" onclick="$($text3).html($text3 * $text7);$ViewById="button1" onclick="$($text3).html($text1;#class="button1"> ViewById="button1" onclick="$($text3).html($text3 + $text7);";ViewById="button1" onclick="$($text3).html($text3 * $text7);$ViewById="button1" onclick="$($text3).html($text3 - $text9);"; ViewById="button1" onclick="$($text3).html($text4;#class="button1"> ViewById="button1" onclick="$($text3).html($text3 * $text4);$ViewById="button1" onclick="$($text3).html($text3 + $text5);$ViewById="button1" onclick="$($text3).html($text1* $text6);";ViewById="button1" onclick="$($text3).html($text6);$ViewById="button1" onclick="$($text3<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Prompt sample 3648 of the raw training set: <|im_start|>system Put your final answer within \boxed{}.<|im_end|> <|im_start|>user If a basket has a total capacity of 40 fruits and there are three times as many apples as oranges, what is the number of oranges in the basket?<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Chosen sample 3648 of the raw training set: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>assistant To determine the number of oranges in the basket, we start by defining variables for the number of apples and oranges. Let \( O \) represent the number of oranges, and since there are three times as many apples as oranges, we can represent the number of apples as \( 3O \). The total capacity of the basket is 40 fruits, so we can write the following equation: \[ O + 3O = 40 \] Combining like terms, we get: \[ 4O = 40 \] To find the value of \( O \), we divide both sides of the equation by 4: \[ O = \frac{40}{4} = 10 \] Therefore, the number of oranges in the basket is \(\boxed{10}\).<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Rejected sample 3648 of the raw training set: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>assistant In a jar there are 20 oranges. Mr. Smith buys 6 more. Also the number of apples is 10 more than the number of oranges. What is the number of fruits in the jar?鲕 鲕 In the park there are two ponds. One is for frogs and the other one is for squirrels. Today there are 6 frogs in the first pond and there are twice as many squirrels in the other one. How many more squirrels than frogs are in the pond for squirrels?鲕 鲕 One day there was 10 buckets in the garden. The gardeners took some buckets. Then there were 4 buckets empty. How many buckets were taken out of the garden??鲕 鲕 I have a small container filled with jelly beans. I need to divide them into 55 equal sets of 10 beans each. How many beans will I need?鲕 鲕 A group of 20 children from our class received the book ‘The Three Little Pigs’ on their birthdays. Now they read it 5 times a week. The following week, some children donated it to their classmate who read it 10 times a week. How many more times will they read the 500-page book in a month?鲕 鲕 The little girl started to feed the duck, which was 13 in number. The little duck came back to the kitchen 11 times. How many ducks were there before she came back to feed it?鲕 鲕 We are at the park to play. Each of us has ten tokens. Then I saw ½ of them run away, and ¼ from the remaining ones run away. How many tokens were there before the little boy came back to me?鲕 鲕 Yesterday I started to make some sandwiches. I made 12 before I started and I ate 8 of them. When I finished making them, I didn’t have 10 left. How many sandwiches were there to begin with?鲕 鲕 In one week, we ate 36 sandwiches. How many sandwiches were in the beginning?鲕 鲕 I have 10 apples, of which 3 are fresh. 1 apple is rotten. How many apples are not rotten?鲕 鲕 We eat 6. We are playing outside for 2 hours. At noon we get the news that there was a storm in our town. We eat a new one as soon as<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Prompt sample 819 of the raw training set: <|im_start|>system Put your final answer within \boxed{}.<|im_end|> <|im_start|>user Three marbles are randomly selected, without replacement, from a bag containing two red, two blue and two green marbles. What is the probability that one marble of each color is selected? Express your answer as a common fraction.<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Chosen sample 819 of the raw training set: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>assistant To determine the probability that one marble of each color (red, blue, and green) is selected when three marbles are randomly chosen without replacement from a bag containing two red, two blue, and two green marbles, we can follow these steps: 1. **Calculate the total number of ways to select 3 marbles out of 6 marbles:** The total number of ways to choose 3 marbles out of 6 is given by the combination formula: \[ \binom{6}{3} = \frac{6!}{3!(6-3)!} = \frac{6!}{3! \cdot 3!} \] Simplifying the factorials: \[ \binom{6}{3} = \frac{6 \times 5 \times 4}{3 \times 2 \times 1} = 20 \] 2. **Calculate the number of favorable outcomes (one marble of each color):** To have one marble of each color, we need to choose 1 red, 1 blue, and 1 green marble. The number of ways to choose 1 marble out of 2 for each color is: \[ \binom{2}{1} \times \binom{2}{1} \times \binom{2}{1} \] Each binomial coefficient is 2, so: \[ 2 \times 2 \times 2 = 8 \] 3. **Calculate the probability:** The probability is the ratio of the number of favorable outcomes to the total number of possible outcomes: \[ \text{Probability} = \frac{\text{Number of favorable outcomes}}{\text{Total number of outcomes}} = \frac{8}{20} = \frac{2}{5} \] Thus, the probability that one marble of each color is selected is: \[ \boxed{\frac{2}{5}} \]<|im_end|> 2025-12-21 13:06:21 - INFO - __main__ - Rejected sample 819 of the raw training set: <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>assistant A die is rolled. What is the probability that an odd number is rolled? Express your answer as a common fraction or decimal, rounded to two decimal places.enze enze Let $P(x)$ be the proposition $[(x+2)(x+3)(x+5)<0]$ and let $Q(x)$ be the proposition $(|x|+1>0)$. Which of the following is true?enze enze Prove each of the following propositions: \textbf{a)} $5,5,5,5$ can be arranged to form a $5$-pointed star.enze enze Which of the following relations is not reflexive, symmetric, or transitive?enze enze \textbf{Proposition}: For all $x \in \mathbb{R}^+$, $x > 3$.enze enze Let $f(x)=|x+1|$.Which of the following statements is true?enze enze If $y \ge 0$ and $y \ne 0$, which of the following is a true statement?enze enze Which of the following is always true when you add one to a left-angle bracketed expression?enze enze \begin{align*}a\big( 7 - 5 b\big) &= 13-\big(5 - 2a\big) && a, b \text{ are integers} && 5 b \ne 0 && b \text{ is not a multiple of 2} &&2 \le a \le 5. && b \ne -2 && b \ne 3. && 2 \le b \le 3. && a\big( 7 - 5 b\big) &= 17-5b && a\big( 7 - 5 b\big) < 17 && -6b > -17 && b> 17/6 && 2 \le b \le 7. && b \text{ is not a multiple of 3} && 2 \le b \le 7. && b\big(a - 5\big) < a && -b\le -5 && 5 | b \le 2. && 2 \le b \le 5. && 2 \le b \le 3. && -2 \le b < 0 && b \text<|im_end|> /mnt/phongdq/miniconda3/envs/span_distillm2/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:100: FutureWarning: Deprecated argument(s) used in '__init__': model_init_kwargs, ref_model_init_kwargs, max_length, max_prompt_length, loss_type, force_use_ref_model. Will not be supported from version '1.0.0'. Deprecated positional argument(s) used in DPOTrainer, please use the DPOConfig to set these arguments instead. warnings.warn(message, FutureWarning) /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:488: UserWarning: You passed `model_init_kwargs` to the DPOTrainer, the value you passed will override the one in the `DPOConfig`. warnings.warn( /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:516: UserWarning: You passed `ref_model_init_kwargs` to the DPOTrainer, the value you passed will override the one in the `DPOConfig`. warnings.warn( /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:543: UserWarning: You passed a model_id to the DPOTrainer. This will automatically create an `AutoModelForCausalLM` or a `PeftModel` (if you passed a `peft_config`) for you. warnings.warn( [INFO|configuration_utils.py:733] 2025-12-21 13:06:22,294 >> loading configuration file config.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/config.json [INFO|configuration_utils.py:800] 2025-12-21 13:06:22,295 >> Model config Qwen2Config { "_name_or_path": "Qwen/Qwen2.5-0.5B", "architectures": [ "Qwen2ForCausalLM" ], "attention_dropout": 0.0, "bos_token_id": 151643, "eos_token_id": 151643, "hidden_act": "silu", "hidden_size": 896, "initializer_range": 0.02, "intermediate_size": 4864, "max_position_embeddings": 32768, "max_window_layers": 24, "model_type": "qwen2", "num_attention_heads": 14, "num_hidden_layers": 24, "num_key_value_heads": 2, "rms_norm_eps": 1e-06, "rope_theta": 1000000.0, "sliding_window": null, "tie_word_embeddings": true, "torch_dtype": "bfloat16", "transformers_version": "4.43.2", "use_cache": true, "use_mrope": false, "use_sliding_window": false, "vocab_size": 151936 } [INFO|modeling_utils.py:3634] 2025-12-21 13:06:22,298 >> loading weights file model.safetensors from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/model.safetensors [INFO|modeling_utils.py:1572] 2025-12-21 13:06:22,310 >> Instantiating Qwen2ForCausalLM model under default dtype torch.bfloat16. [INFO|configuration_utils.py:1038] 2025-12-21 13:06:22,311 >> Generate config GenerationConfig { "bos_token_id": 151643, "eos_token_id": 151643 } [INFO|modeling_utils.py:4463] 2025-12-21 13:06:23,365 >> All model checkpoint weights were used when initializing Qwen2ForCausalLM. [INFO|modeling_utils.py:4471] 2025-12-21 13:06:23,365 >> All the weights of Qwen2ForCausalLM were initialized from the model checkpoint at Qwen/Qwen2.5-0.5B. If your task is similar to the task the model of the checkpoint was trained on, you can already use Qwen2ForCausalLM for predictions without further training. [INFO|configuration_utils.py:993] 2025-12-21 13:06:23,829 >> loading configuration file generation_config.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B/snapshots/060db6499f32faf8b98477b0a26969ef7d8b9987/generation_config.json [INFO|configuration_utils.py:1038] 2025-12-21 13:06:23,829 >> Generate config GenerationConfig { "bos_token_id": 151643, "eos_token_id": 151643, "max_new_tokens": 2048 } /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:550: UserWarning: You passed a ref model_id to the DPOTrainer. This will automatically create an `AutoModelForCausalLM` warnings.warn( [INFO|configuration_utils.py:733] 2025-12-21 13:06:24,296 >> loading configuration file config.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-Math-1.5B-Instruct/snapshots/aafeb0fc6f22cbf0eaeed126eff8be45b0360a35/config.json [INFO|configuration_utils.py:800] 2025-12-21 13:06:24,297 >> Model config Qwen2Config { "_name_or_path": "Qwen/Qwen2.5-Math-1.5B-Instruct", "architectures": [ "Qwen2ForCausalLM" ], "attention_dropout": 0.0, "bos_token_id": 151643, "eos_token_id": 151645, "hidden_act": "silu", "hidden_size": 1536, "initializer_range": 0.02, "intermediate_size": 8960, "max_position_embeddings": 4096, "max_window_layers": 21, "model_type": "qwen2", "num_attention_heads": 12, "num_hidden_layers": 28, "num_key_value_heads": 2, "rms_norm_eps": 1e-06, "rope_theta": 10000.0, "sliding_window": null, "tie_word_embeddings": true, "torch_dtype": "bfloat16", "transformers_version": "4.43.2", "use_cache": true, "use_sliding_window": false, "vocab_size": 151936 } [INFO|modeling_utils.py:3634] 2025-12-21 13:06:24,298 >> loading weights file model.safetensors from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-Math-1.5B-Instruct/snapshots/aafeb0fc6f22cbf0eaeed126eff8be45b0360a35/model.safetensors [INFO|modeling_utils.py:1572] 2025-12-21 13:06:24,311 >> Instantiating Qwen2ForCausalLM model under default dtype torch.bfloat16. [INFO|configuration_utils.py:1038] 2025-12-21 13:06:24,312 >> Generate config GenerationConfig { "bos_token_id": 151643, "eos_token_id": 151645 } [INFO|modeling_utils.py:4463] 2025-12-21 13:06:24,468 >> All model checkpoint weights were used when initializing Qwen2ForCausalLM. [INFO|modeling_utils.py:4471] 2025-12-21 13:06:24,468 >> All the weights of Qwen2ForCausalLM were initialized from the model checkpoint at Qwen/Qwen2.5-Math-1.5B-Instruct. If your task is similar to the task the model of the checkpoint was trained on, you can already use Qwen2ForCausalLM for predictions without further training. [INFO|configuration_utils.py:993] 2025-12-21 13:06:24,947 >> loading configuration file generation_config.json from cache at /mnt/phongdq/.cache/huggingface/hub/models--Qwen--Qwen2.5-Math-1.5B-Instruct/snapshots/aafeb0fc6f22cbf0eaeed126eff8be45b0360a35/generation_config.json [INFO|configuration_utils.py:1038] 2025-12-21 13:06:24,948 >> Generate config GenerationConfig { "bos_token_id": 151643, "eos_token_id": [ 151645, 151643 ], "pad_token_id": 151643 } /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:561: UserWarning: You passed `force_use_ref_model` to the DPOTrainer, the value you passed will override the one in the `DPOConfig`. warnings.warn( /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:710: UserWarning: You passed `max_length` to the DPOTrainer, the value you passed will override the one in the `DPOConfig`. warnings.warn( /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:723: UserWarning: You passed `max_prompt_length` to the DPOTrainer, the value you passed will override the one in the `DPOConfig`. warnings.warn( /mnt/phongdq/projects/span_fdd/distillm-2-master/src/span_distillm_trainer.py:808: UserWarning: You passed `loss_type` to the DPOTrainer, the value you passed will override the one in the `DPOConfig`. warnings.warn( Map: 0%| | 0/30000 [00:00> Using auto half precision backend [INFO|trainer.py:2134] 2025-12-21 13:08:48,319 >> ***** Running training ***** [INFO|trainer.py:2135] 2025-12-21 13:08:48,319 >> Num examples = 30,000 [INFO|trainer.py:2136] 2025-12-21 13:08:48,319 >> Num Epochs = 2 [INFO|trainer.py:2137] 2025-12-21 13:08:48,319 >> Instantaneous batch size per device = 2 [INFO|trainer.py:2140] 2025-12-21 13:08:48,319 >> Total train batch size (w. parallel, distributed & accumulation) = 8 [INFO|trainer.py:2141] 2025-12-21 13:08:48,319 >> Gradient Accumulation steps = 4 [INFO|trainer.py:2142] 2025-12-21 13:08:48,319 >> Total optimization steps = 7,500 [INFO|trainer.py:2143] 2025-12-21 13:08:48,320 >> Number of trainable parameters = 502,299,520 0%| | 0/7500 [00:00> Could not estimate the number of tokens of the input, floating-point operations will not be computed 0%| | 1/7500 [00:02<5:16:49, 2.53s/it] {'loss': 0.9428, 'grad_norm': 29.764192581176758, 'learning_rate': 6.666666666666667e-08, 'logqs/rejected': -1.6647133827209473, 'logqs/chosen': -1.0295268297195435, 'logqs_logps/rejected': 0.03778006136417389, 'logps_logqs/chosen': 0.28790903091430664, 'epoch': 0.0} 0%| | 1/7500 [00:02<5:16:49, 2.53s/it] 0%| | 2/7500 [00:04<4:33:22, 2.19s/it] 0%| | 3/7500 [00:06<4:23:16, 2.11s/it] 0%| | 4/7500 [00:08<4:19:05, 2.07s/it] 0%| | 5/7500 [00:10<4:20:47, 2.09s/it] 0%| | 6/7500 [00:12<4:21:07, 2.09s/it] 0%| | 7/7500 [00:14<4:13:22, 2.03s/it] 0%| | 8/7500 [00:16<4:10:15, 2.00s/it] 0%| | 9/7500 [00:18<4:09:53, 2.00s/it] 0%| | 10/7500 [00:20<4:10:22, 2.01s/it] {'loss': 0.9354, 'grad_norm': 27.89144515991211, 'learning_rate': 6.666666666666667e-07, 'logqs/rejected': -1.7942376136779785, 'logqs/chosen': -1.047080159187317, 'logqs_logps/rejected': 0.027770940214395523, 'logps_logqs/chosen': 0.26982536911964417, 'epoch': 0.0} 0%| | 10/7500 [00:20<4:10:22, 2.01s/it] 0%| | 11/7500 [00:22<4:11:50, 2.02s/it] 0%| | 12/7500 [00:24<4:13:35, 2.03s/it] 0%| | 13/7500 [00:26<4:12:12, 2.02s/it] 0%| | 14/7500 [00:28<4:10:55, 2.01s/it] 0%| | 15/7500 [00:30<4:11:40, 2.02s/it] 0%| | 16/7500 [00:32<4:17:12, 2.06s/it] 0%| | 17/7500 [00:34<4:11:52, 2.02s/it] 0%| | 18/7500 [00:36<4:16:19, 2.06s/it] 0%| | 19/7500 [00:38<4:14:15, 2.04s/it] 0%| | 20/7500 [00:41<4:15:13, 2.05s/it] {'loss': 0.962, 'grad_norm': 24.24286460876465, 'learning_rate': 1.3333333333333334e-06, 'logqs/rejected': -1.8679412603378296, 'logqs/chosen': -1.0209720134735107, 'logqs_logps/rejected': 0.03146592527627945, 'logps_logqs/chosen': 0.2823728919029236, 'epoch': 0.01} 0%| | 20/7500 [00:41<4:15:13, 2.05s/it] 0%| | 21/7500 [00:43<4:16:48, 2.06s/it] 0%| | 22/7500 [00:45<4:17:43, 2.07s/it] 0%| | 23/7500 [00:47<4:15:41, 2.05s/it] 0%| | 24/7500 [00:49<4:13:34, 2.04s/it] 0%| | 25/7500 [00:51<4:14:20, 2.04s/it] 0%| | 26/7500 [00:53<4:18:23, 2.07s/it] 0%| | 27/7500 [00:55<4:17:06, 2.06s/it] 0%| | 28/7500 [00:57<4:14:21, 2.04s/it] 0%| | 29/7500 [00:59<4:14:04, 2.04s/it] 0%| | 30/7500 [01:01<4:11:52, 2.02s/it] {'loss': 0.9434, 'grad_norm': 25.1871395111084, 'learning_rate': 2.0000000000000003e-06, 'logqs/rejected': -1.8080604076385498, 'logqs/chosen': -1.0094053745269775, 'logqs_logps/rejected': 0.030170166864991188, 'logps_logqs/chosen': 0.2721155881881714, 'epoch': 0.01} 0%| | 30/7500 [01:01<4:11:52, 2.02s/it] 0%| | 31/7500 [01:03<4:11:59, 2.02s/it] 0%| | 32/7500 [01:05<4:10:39, 2.01s/it] 0%| | 33/7500 [01:07<4:09:28, 2.00s/it] 0%| | 34/7500 [01:09<4:08:35, 2.00s/it] 0%| | 35/7500 [01:11<4:11:15, 2.02s/it] 0%| | 36/7500 [01:13<4:12:36, 2.03s/it] 0%| | 37/7500 [01:15<4:08:17, 2.00s/it] 1%| | 38/7500 [01:17<4:06:34, 1.98s/it] 1%| | 39/7500 [01:19<4:06:51, 1.99s/it] 1%| | 40/7500 [01:21<4:07:20, 1.99s/it] {'loss': 0.8809, 'grad_norm': 16.8751163482666, 'learning_rate': 2.666666666666667e-06, 'logqs/rejected': -1.8127353191375732, 'logqs/chosen': -0.9894005060195923, 'logqs_logps/rejected': 0.04225217550992966, 'logps_logqs/chosen': 0.2620899975299835, 'epoch': 0.01} 1%| | 40/7500 [01:21<4:07:20, 1.99s/it] 1%| | 41/7500 [01:23<4:07:19, 1.99s/it] 1%| | 42/7500 [01:25<4:07:42, 1.99s/it] 1%| | 43/7500 [01:27<4:09:01, 2.00s/it] 1%| | 44/7500 [01:29<4:06:39, 1.98s/it] 1%| | 45/7500 [01:31<4:10:10, 2.01s/it] 1%| | 46/7500 [01:33<4:08:59, 2.00s/it] 1%| | 47/7500 [01:35<4:10:26, 2.02s/it] 1%| | 48/7500 [01:37<4:16:29, 2.07s/it] 1%| | 49/7500 [01:39<4:14:43, 2.05s/it] 1%| | 50/7500 [01:41<4:10:12, 2.02s/it] {'loss': 0.8379, 'grad_norm': 14.210755348205566, 'learning_rate': 3.3333333333333333e-06, 'logqs/rejected': -1.9377574920654297, 'logqs/chosen': -0.9669780731201172, 'logqs_logps/rejected': 0.036098260432481766, 'logps_logqs/chosen': 0.2598077058792114, 'epoch': 0.01} 1%| | 50/7500 [01:41<4:10:12, 2.02s/it] 1%| | 51/7500 [01:43<4:10:31, 2.02s/it] 1%| | 52/7500 [01:45<4:08:53, 2.01s/it] 1%| | 53/7500 [01:47<4:03:49, 1.96s/it] 1%| | 54/7500 [01:49<4:07:53, 2.00s/it] 1%| | 55/7500 [01:51<4:06:09, 1.98s/it] 1%| | 56/7500 [01:53<4:07:22, 1.99s/it] 1%| | 57/7500 [01:55<4:05:47, 1.98s/it] 1%| | 58/7500 [01:57<4:09:48, 2.01s/it] 1%| | 59/7500 [01:59<4:10:56, 2.02s/it] 1%| | 60/7500 [02:01<4:14:19, 2.05s/it] {'loss': 0.7642, 'grad_norm': 16.88459587097168, 'learning_rate': 4.000000000000001e-06, 'logqs/rejected': -1.9515488147735596, 'logqs/chosen': -0.9645748138427734, 'logqs_logps/rejected': 0.024065136909484863, 'logps_logqs/chosen': 0.2574823498725891, 'epoch': 0.02} 1%| | 60/7500 [02:01<4:14:19, 2.05s/it] 1%| | 61/7500 [02:03<4:14:56, 2.06s/it] 1%| | 62/7500 [02:05<4:08:20, 2.00s/it] 1%| | 63/7500 [02:07<4:06:23, 1.99s/it] 1%| | 64/7500 [02:09<4:08:20, 2.00s/it] 1%| | 65/7500 [02:11<4:07:03, 1.99s/it] 1%| | 66/7500 [02:13<4:07:56, 2.00s/it] 1%| | 67/7500 [02:15<4:05:24, 1.98s/it] 1%| | 68/7500 [02:17<4:00:34, 1.94s/it] 1%| | 69/7500 [02:19<3:59:46, 1.94s/it] 1%| | 70/7500 [02:21<4:01:02, 1.95s/it] {'loss': 0.6695, 'grad_norm': 8.644386291503906, 'learning_rate': 4.666666666666667e-06, 'logqs/rejected': -1.9543263912200928, 'logqs/chosen': -0.9731027483940125, 'logqs_logps/rejected': 0.0027110478840768337, 'logps_logqs/chosen': 0.2506929934024811, 'epoch': 0.02} 1%| | 70/7500 [02:21<4:01:02, 1.95s/it] 1%| | 71/7500 [02:23<4:03:50, 1.97s/it] 1%| | 72/7500 [02:25<4:03:42, 1.97s/it] 1%| | 73/7500 [02:27<4:03:39, 1.97s/it] 1%| | 74/7500 [02:29<4:06:16, 1.99s/it] 1%| | 75/7500 [02:31<4:04:08, 1.97s/it] 1%| | 76/7500 [02:33<4:03:10, 1.97s/it] 1%| | 77/7500 [02:35<4:03:57, 1.97s/it] 1%| | 78/7500 [02:37<4:06:32, 1.99s/it] 1%| | 79/7500 [02:39<4:06:45, 2.00s/it] 1%| | 80/7500 [02:41<4:09:15, 2.02s/it] {'loss': 0.693, 'grad_norm': 8.631427764892578, 'learning_rate': 5.333333333333334e-06, 'logqs/rejected': -2.0324504375457764, 'logqs/chosen': -1.0226396322250366, 'logqs_logps/rejected': 0.0028457362204790115, 'logps_logqs/chosen': 0.26710745692253113, 'epoch': 0.02} 1%| | 80/7500 [02:41<4:09:15, 2.02s/it] 1%| | 81/7500 [02:43<4:11:39, 2.04s/it] 1%| | 82/7500 [02:45<4:09:41, 2.02s/it] 1%| | 83/7500 [02:47<4:09:17, 2.02s/it] 1%| | 84/7500 [02:49<4:06:19, 1.99s/it] 1%| | 85/7500 [02:51<4:05:32, 1.99s/it] 1%| | 86/7500 [02:53<4:04:38, 1.98s/it] 1%| | 87/7500 [02:55<4:03:47, 1.97s/it] 1%| | 88/7500 [02:57<4:02:21, 1.96s/it] 1%| | 89/7500 [02:59<4:08:03, 2.01s/it] 1%| | 90/7500 [03:01<4:09:36, 2.02s/it] {'loss': 0.6543, 'grad_norm': 7.290536880493164, 'learning_rate': 6e-06, 'logqs/rejected': -2.219325542449951, 'logqs/chosen': -1.0595362186431885, 'logqs_logps/rejected': -0.015365222468972206, 'logps_logqs/chosen': 0.2920253872871399, 'epoch': 0.02} 1%| | 90/7500 [03:01<4:09:36, 2.02s/it] 1%| | 91/7500 [03:03<4:10:09, 2.03s/it] 1%| | 92/7500 [03:05<4:08:10, 2.01s/it] 1%| | 93/7500 [03:07<4:13:11, 2.05s/it] 1%|▏ | 94/7500 [03:09<4:12:12, 2.04s/it] 1%|▏ | 95/7500 [03:11<4:13:51, 2.06s/it] 1%|▏ | 96/7500 [03:13<4:14:13, 2.06s/it] 1%|▏ | 97/7500 [03:15<4:13:58, 2.06s/it] 1%|▏ | 98/7500 [03:17<4:12:22, 2.05s/it] 1%|▏ | 99/7500 [03:19<4:10:36, 2.03s/it] 1%|▏ | 100/7500 [03:21<4:06:05, 2.00s/it] {'loss': 0.6405, 'grad_norm': 9.3944730758667, 'learning_rate': 6.666666666666667e-06, 'logqs/rejected': -2.300631523132324, 'logqs/chosen': -1.1049785614013672, 'logqs_logps/rejected': -0.023101428523659706, 'logps_logqs/chosen': 0.2965795397758484, 'epoch': 0.03} 1%|▏ | 100/7500 [03:21<4:06:05, 2.00s/it] 1%|▏ | 101/7500 [03:23<4:06:22, 2.00s/it] 1%|▏ | 102/7500 [03:25<4:08:37, 2.02s/it] 1%|▏ | 103/7500 [03:27<4:06:17, 2.00s/it] 1%|▏ | 104/7500 [03:29<4:05:42, 1.99s/it] 1%|▏ | 105/7500 [03:31<4:06:37, 2.00s/it] 1%|▏ | 106/7500 [03:33<4:05:52, 2.00s/it] 1%|▏ | 107/7500 [03:35<4:05:26, 1.99s/it] 1%|▏ | 108/7500 [03:37<4:06:56, 2.00s/it] 1%|▏ | 109/7500 [03:39<4:07:51, 2.01s/it] 1%|▏ | 110/7500 [03:41<4:05:32, 1.99s/it] {'loss': 0.6289, 'grad_norm': 8.687568664550781, 'learning_rate': 7.333333333333334e-06, 'logqs/rejected': -2.3293564319610596, 'logqs/chosen': -1.1121724843978882, 'logqs_logps/rejected': -0.027716603130102158, 'logps_logqs/chosen': 0.2973412871360779, 'epoch': 0.03} 1%|▏ | 110/7500 [03:41<4:05:32, 1.99s/it] 1%|▏ | 111/7500 [03:43<4:08:38, 2.02s/it] 1%|▏ | 112/7500 [03:45<4:06:50, 2.00s/it] 2%|▏ | 113/7500 [03:47<4:06:37, 2.00s/it] 2%|▏ | 114/7500 [03:49<4:06:05, 2.00s/it] 2%|▏ | 115/7500 [03:51<4:07:02, 2.01s/it] 2%|▏ | 116/7500 [03:53<4:05:06, 1.99s/it] 2%|▏ | 117/7500 [03:55<4:10:02, 2.03s/it] 2%|▏ | 118/7500 [03:57<4:08:55, 2.02s/it] 2%|▏ | 119/7500 [03:59<4:12:21, 2.05s/it] 2%|▏ | 120/7500 [04:01<4:10:51, 2.04s/it] {'loss': 0.6488, 'grad_norm': 6.604452610015869, 'learning_rate': 8.000000000000001e-06, 'logqs/rejected': -2.4410266876220703, 'logqs/chosen': -1.0949373245239258, 'logqs_logps/rejected': -0.020331386476755142, 'logps_logqs/chosen': 0.2931906580924988, 'epoch': 0.03} 2%|▏ | 120/7500 [04:01<4:10:51, 2.04s/it] 2%|▏ | 121/7500 [04:03<4:08:43, 2.02s/it] 2%|▏ | 122/7500 [04:05<4:07:03, 2.01s/it] 2%|▏ | 123/7500 [04:07<4:07:30, 2.01s/it] 2%|▏ | 124/7500 [04:09<4:03:58, 1.98s/it] 2%|▏ | 125/7500 [04:11<4:04:06, 1.99s/it] 2%|▏ | 126/7500 [04:13<4:04:37, 1.99s/it] 2%|▏ | 127/7500 [04:15<4:05:43, 2.00s/it] 2%|▏ | 128/7500 [04:17<4:03:36, 1.98s/it] 2%|▏ | 129/7500 [04:19<4:08:32, 2.02s/it] 2%|▏ | 130/7500 [04:21<4:08:05, 2.02s/it] {'loss': 0.6058, 'grad_norm': 8.580706596374512, 'learning_rate': 8.666666666666668e-06, 'logqs/rejected': -2.4868416786193848, 'logqs/chosen': -1.0912067890167236, 'logqs_logps/rejected': -0.03053124248981476, 'logps_logqs/chosen': 0.30285224318504333, 'epoch': 0.03} 2%|▏ | 130/7500 [04:21<4:08:05, 2.02s/it] 2%|▏ | 131/7500 [04:23<4:08:06, 2.02s/it] 2%|▏ | 132/7500 [04:25<4:07:02, 2.01s/it] 2%|▏ | 133/7500 [04:27<4:05:14, 2.00s/it] 2%|▏ | 134/7500 [04:29<4:03:23, 1.98s/it] 2%|▏ | 135/7500 [04:31<4:05:19, 2.00s/it] 2%|▏ | 136/7500 [04:33<4:03:00, 1.98s/it] 2%|▏ | 137/7500 [04:35<4:07:40, 2.02s/it] 2%|▏ | 138/7500 [04:37<4:07:04, 2.01s/it] 2%|▏ | 139/7500 [04:39<4:07:01, 2.01s/it] 2%|▏ | 140/7500 [04:41<4:07:56, 2.02s/it] {'loss': 0.6394, 'grad_norm': 7.0485100746154785, 'learning_rate': 9.333333333333334e-06, 'logqs/rejected': -2.3458447456359863, 'logqs/chosen': -1.0620067119598389, 'logqs_logps/rejected': -0.024010034278035164, 'logps_logqs/chosen': 0.2925134301185608, 'epoch': 0.04} 2%|▏ | 140/7500 [04:41<4:07:56, 2.02s/it] 2%|▏ | 141/7500 [04:43<4:03:16, 1.98s/it] 2%|▏ | 142/7500 [04:45<4:06:51, 2.01s/it] 2%|▏ | 143/7500 [04:47<4:03:51, 1.99s/it] 2%|▏ | 144/7500 [04:49<4:01:52, 1.97s/it] 2%|▏ | 145/7500 [04:51<4:03:51, 1.99s/it] 2%|▏ | 146/7500 [04:53<4:02:47, 1.98s/it] 2%|▏ | 147/7500 [04:55<4:09:15, 2.03s/it] 2%|▏ | 148/7500 [04:57<4:09:15, 2.03s/it] 2%|▏ | 149/7500 [05:00<4:11:34, 2.05s/it] 2%|▏ | 150/7500 [05:02<4:07:27, 2.02s/it] {'loss': 0.5929, 'grad_norm': 6.8027143478393555, 'learning_rate': 1e-05, 'logqs/rejected': -2.330878496170044, 'logqs/chosen': -1.130059838294983, 'logqs_logps/rejected': -0.02687019482254982, 'logps_logqs/chosen': 0.2891428470611572, 'epoch': 0.04} 2%|▏ | 150/7500 [05:02<4:07:27, 2.02s/it] 2%|▏ | 151/7500 [05:03<4:04:37, 2.00s/it] 2%|▏ | 152/7500 [05:06<4:06:24, 2.01s/it] 2%|▏ | 153/7500 [05:08<4:07:18, 2.02s/it] 2%|▏ | 154/7500 [05:10<4:06:40, 2.01s/it] 2%|▏ | 155/7500 [05:12<4:06:43, 2.02s/it] 2%|▏ | 156/7500 [05:14<4:04:19, 2.00s/it] 2%|▏ | 157/7500 [05:16<4:04:28, 2.00s/it] 2%|▏ | 158/7500 [05:18<4:04:51, 2.00s/it] 2%|▏ | 159/7500 [05:20<4:04:17, 2.00s/it] 2%|▏ | 160/7500 [05:21<4:02:43, 1.98s/it] {'loss': 0.6421, 'grad_norm': 7.170975208282471, 'learning_rate': 1.0666666666666667e-05, 'logqs/rejected': -2.591428279876709, 'logqs/chosen': -1.1310765743255615, 'logqs_logps/rejected': -0.02124052681028843, 'logps_logqs/chosen': 0.2953173518180847, 'epoch': 0.04} 2%|▏ | 160/7500 [05:22<4:02:43, 1.98s/it] 2%|▏ | 161/7500 [05:24<4:04:19, 2.00s/it] 2%|▏ | 162/7500 [05:26<4:06:17, 2.01s/it] 2%|▏ | 163/7500 [05:27<4:03:22, 1.99s/it] 2%|▏ | 164/7500 [05:30<4:07:11, 2.02s/it] 2%|▏ | 165/7500 [05:32<4:05:36, 2.01s/it] 2%|▏ | 166/7500 [05:34<4:04:47, 2.00s/it] 2%|▏ | 167/7500 [05:36<4:04:20, 2.00s/it] 2%|▏ | 168/7500 [05:38<4:03:11, 1.99s/it] 2%|▏ | 169/7500 [05:40<4:06:10, 2.01s/it] 2%|▏ | 170/7500 [05:42<4:06:20, 2.02s/it] {'loss': 0.5828, 'grad_norm': 8.273604393005371, 'learning_rate': 1.1333333333333334e-05, 'logqs/rejected': -2.5116522312164307, 'logqs/chosen': -1.0730812549591064, 'logqs_logps/rejected': -0.024451564997434616, 'logps_logqs/chosen': 0.29727694392204285, 'epoch': 0.05} 2%|▏ | 170/7500 [05:42<4:06:20, 2.02s/it] 2%|▏ | 171/7500 [05:43<4:01:44, 1.98s/it] 2%|▏ | 172/7500 [05:46<4:05:09, 2.01s/it] 2%|▏ | 173/7500 [05:48<4:06:05, 2.02s/it] 2%|▏ | 174/7500 [05:50<4:03:47, 2.00s/it] 2%|▏ | 175/7500 [05:52<4:02:02, 1.98s/it] 2%|▏ | 176/7500 [05:53<4:02:06, 1.98s/it] 2%|▏ | 177/7500 [05:56<4:03:23, 1.99s/it] 2%|▏ | 178/7500 [05:58<4:04:22, 2.00s/it] 2%|▏ | 179/7500 [06:00<4:04:59, 2.01s/it] 2%|▏ | 180/7500 [06:02<4:03:22, 1.99s/it] {'loss': 0.6001, 'grad_norm': 6.634832382202148, 'learning_rate': 1.2e-05, 'logqs/rejected': -2.4300060272216797, 'logqs/chosen': -1.1077553033828735, 'logqs_logps/rejected': -0.013239803723990917, 'logps_logqs/chosen': 0.2987092137336731, 'epoch': 0.05} 2%|▏ | 180/7500 [06:02<4:03:22, 1.99s/it] 2%|▏ | 181/7500 [06:04<4:03:00, 1.99s/it] 2%|▏ | 182/7500 [06:06<4:03:27, 2.00s/it] 2%|▏ | 183/7500 [06:07<4:01:44, 1.98s/it] 2%|▏ | 184/7500 [06:09<4:01:31, 1.98s/it] 2%|▏ | 185/7500 [06:11<4:01:39, 1.98s/it] 2%|▏ | 186/7500 [06:13<4:04:26, 2.01s/it] 2%|▏ | 187/7500 [06:15<4:01:41, 1.98s/it] 3%|▎ | 188/7500 [06:17<4:03:47, 2.00s/it] 3%|▎ | 189/7500 [06:19<4:00:38, 1.97s/it] 3%|▎ | 190/7500 [06:21<4:01:25, 1.98s/it] {'loss': 0.5523, 'grad_norm': 5.8110127449035645, 'learning_rate': 1.2666666666666668e-05, 'logqs/rejected': -2.533198118209839, 'logqs/chosen': -1.068535566329956, 'logqs_logps/rejected': -0.030217409133911133, 'logps_logqs/chosen': 0.29886165261268616, 'epoch': 0.05} 3%|▎ | 190/7500 [06:21<4:01:25, 1.98s/it] 3%|▎ | 191/7500 [06:23<4:00:35, 1.98s/it] 3%|▎ | 192/7500 [06:25<4:03:22, 2.00s/it] 3%|▎ | 193/7500 [06:27<4:04:24, 2.01s/it] 3%|▎ | 194/7500 [06:29<4:06:14, 2.02s/it] 3%|▎ | 195/7500 [06:32<4:08:41, 2.04s/it] 3%|▎ | 196/7500 [06:34<4:05:59, 2.02s/it] 3%|▎ | 197/7500 [06:35<4:03:25, 2.00s/it] 3%|▎ | 198/7500 [06:38<4:05:15, 2.02s/it] 3%|▎ | 199/7500 [06:39<4:02:33, 1.99s/it] 3%|▎ | 200/7500 [06:41<4:00:57, 1.98s/it] {'loss': 0.5301, 'grad_norm': 11.780774116516113, 'learning_rate': 1.3333333333333333e-05, 'logqs/rejected': -2.3357419967651367, 'logqs/chosen': -1.0702375173568726, 'logqs_logps/rejected': -0.036942124366760254, 'logps_logqs/chosen': 0.2910466194152832, 'epoch': 0.05} 3%|▎ | 200/7500 [06:41<4:00:57, 1.98s/it] 3%|▎ | 201/7500 [06:43<4:00:12, 1.97s/it] 3%|▎ | 202/7500 [06:45<4:01:15, 1.98s/it] 3%|▎ | 203/7500 [06:47<4:01:11, 1.98s/it] 3%|▎ | 204/7500 [06:49<4:00:22, 1.98s/it] 3%|▎ | 205/7500 [06:51<4:05:16, 2.02s/it] 3%|▎ | 206/7500 [06:53<4:02:25, 1.99s/it] 3%|▎ | 207/7500 [06:55<4:03:54, 2.01s/it] 3%|▎ | 208/7500 [06:57<3:59:57, 1.97s/it] 3%|▎ | 209/7500 [06:59<4:01:54, 1.99s/it] 3%|▎ | 210/7500 [07:01<4:00:40, 1.98s/it] {'loss': 0.5464, 'grad_norm': 6.795104503631592, 'learning_rate': 1.4000000000000001e-05, 'logqs/rejected': -2.3672537803649902, 'logqs/chosen': -1.0265859365463257, 'logqs_logps/rejected': -0.025169294327497482, 'logps_logqs/chosen': 0.2783769965171814, 'epoch': 0.06} 3%|▎ | 210/7500 [07:01<4:00:40, 1.98s/it] 3%|▎ | 211/7500 [07:03<4:02:04, 1.99s/it] 3%|▎ | 212/7500 [07:05<4:05:56, 2.02s/it] 3%|▎ | 213/7500 [07:07<4:05:46, 2.02s/it] 3%|▎ | 214/7500 [07:09<4:05:37, 2.02s/it] 3%|▎ | 215/7500 [07:11<4:05:09, 2.02s/it] 3%|▎ | 216/7500 [07:13<4:04:22, 2.01s/it] 3%|▎ | 217/7500 [07:16<4:06:19, 2.03s/it] 3%|▎ | 218/7500 [07:18<4:06:05, 2.03s/it] 3%|▎ | 219/7500 [07:20<4:04:34, 2.02s/it] 3%|▎ | 220/7500 [07:22<4:03:06, 2.00s/it] {'loss': 0.5592, 'grad_norm': 7.497940540313721, 'learning_rate': 1.4666666666666668e-05, 'logqs/rejected': -2.5574231147766113, 'logqs/chosen': -1.0265271663665771, 'logqs_logps/rejected': -0.021774400025606155, 'logps_logqs/chosen': 0.2690812647342682, 'epoch': 0.06} 3%|▎ | 220/7500 [07:22<4:03:06, 2.00s/it] 3%|▎ | 221/7500 [07:23<3:59:29, 1.97s/it] 3%|▎ | 222/7500 [07:26<4:03:43, 2.01s/it] 3%|▎ | 223/7500 [07:27<4:01:28, 1.99s/it] 3%|▎ | 224/7500 [07:30<4:02:57, 2.00s/it] 3%|▎ | 225/7500 [07:31<3:59:07, 1.97s/it] 3%|▎ | 226/7500 [07:33<4:03:28, 2.01s/it] 3%|▎ | 227/7500 [07:35<4:00:57, 1.99s/it] 3%|▎ | 228/7500 [07:37<4:02:57, 2.00s/it] 3%|▎ | 229/7500 [07:40<4:05:31, 2.03s/it] 3%|▎ | 230/7500 [07:42<4:06:02, 2.03s/it] {'loss': 0.5413, 'grad_norm': 10.683938980102539, 'learning_rate': 1.5333333333333334e-05, 'logqs/rejected': -2.444855213165283, 'logqs/chosen': -1.0505539178848267, 'logqs_logps/rejected': -0.018786568194627762, 'logps_logqs/chosen': 0.28283795714378357, 'epoch': 0.06} 3%|▎ | 230/7500 [07:42<4:06:02, 2.03s/it] 3%|▎ | 231/7500 [07:44<4:07:34, 2.04s/it] 3%|▎ | 232/7500 [07:46<4:07:47, 2.05s/it] 3%|▎ | 233/7500 [07:48<4:07:28, 2.04s/it] 3%|▎ | 234/7500 [07:50<4:06:16, 2.03s/it] 3%|▎ | 235/7500 [07:52<4:06:24, 2.03s/it] 3%|▎ | 236/7500 [07:54<4:04:29, 2.02s/it] 3%|▎ | 237/7500 [07:56<4:01:37, 2.00s/it] 3%|▎ | 238/7500 [07:58<3:59:38, 1.98s/it] 3%|▎ | 239/7500 [08:00<3:59:15, 1.98s/it] 3%|▎ | 240/7500 [08:02<3:59:33, 1.98s/it] {'loss': 0.5512, 'grad_norm': 6.709837436676025, 'learning_rate': 1.6000000000000003e-05, 'logqs/rejected': -2.4961647987365723, 'logqs/chosen': -1.0098893642425537, 'logqs_logps/rejected': -0.023810578510165215, 'logps_logqs/chosen': 0.278264582157135, 'epoch': 0.06} 3%|▎ | 240/7500 [08:02<3:59:33, 1.98s/it] 3%|▎ | 241/7500 [08:04<3:57:59, 1.97s/it] 3%|▎ | 242/7500 [08:05<3:56:14, 1.95s/it] 3%|▎ | 243/7500 [08:08<3:58:16, 1.97s/it] 3%|▎ | 244/7500 [08:10<4:02:12, 2.00s/it] 3%|▎ | 245/7500 [08:12<4:04:50, 2.02s/it] 3%|▎ | 246/7500 [08:14<4:02:00, 2.00s/it] 3%|▎ | 247/7500 [08:16<4:01:17, 2.00s/it] 3%|▎ | 248/7500 [08:18<4:00:56, 1.99s/it] 3%|▎ | 249/7500 [08:20<4:00:35, 1.99s/it] 3%|▎ | 250/7500 [08:22<4:01:22, 2.00s/it] {'loss': 0.5611, 'grad_norm': 10.417595863342285, 'learning_rate': 1.6666666666666667e-05, 'logqs/rejected': -2.424856662750244, 'logqs/chosen': -1.025957465171814, 'logqs_logps/rejected': -0.009128410369157791, 'logps_logqs/chosen': 0.2678135931491852, 'epoch': 0.07} 3%|▎ | 250/7500 [08:22<4:01:22, 2.00s/it] 3%|▎ | 251/7500 [08:24<4:01:09, 2.00s/it] 3%|▎ | 252/7500 [08:26<4:02:24, 2.01s/it] 3%|▎ | 253/7500 [08:28<4:05:14, 2.03s/it] 3%|▎ | 254/7500 [08:30<4:05:54, 2.04s/it] 3%|▎ | 255/7500 [08:32<4:04:35, 2.03s/it] 3%|▎ | 256/7500 [08:34<4:02:32, 2.01s/it] 3%|▎ | 257/7500 [08:36<4:01:34, 2.00s/it] 3%|▎ | 258/7500 [08:38<3:58:57, 1.98s/it] 3%|▎ | 259/7500 [08:40<4:01:23, 2.00s/it] 3%|▎ | 260/7500 [08:42<4:03:17, 2.02s/it] {'loss': 0.5065, 'grad_norm': 11.447784423828125, 'learning_rate': 1.7333333333333336e-05, 'logqs/rejected': -2.2210516929626465, 'logqs/chosen': -0.9617618322372437, 'logqs_logps/rejected': -0.015696104615926743, 'logps_logqs/chosen': 0.2636481523513794, 'epoch': 0.07} 3%|▎ | 260/7500 [08:42<4:03:17, 2.02s/it] 3%|▎ | 261/7500 [08:44<4:05:20, 2.03s/it] 3%|▎ | 262/7500 [08:46<4:04:21, 2.03s/it] 4%|▎ | 263/7500 [08:48<4:05:58, 2.04s/it] 4%|▎ | 264/7500 [08:50<3:59:24, 1.99s/it] 4%|▎ | 265/7500 [08:52<4:01:40, 2.00s/it] 4%|▎ | 266/7500 [08:54<3:58:06, 1.97s/it] 4%|▎ | 267/7500 [08:56<3:58:47, 1.98s/it] 4%|▎ | 268/7500 [08:58<4:00:08, 1.99s/it] 4%|▎ | 269/7500 [09:00<3:59:23, 1.99s/it] 4%|▎ | 270/7500 [09:02<3:59:02, 1.98s/it] {'loss': 0.5154, 'grad_norm': 5.684640884399414, 'learning_rate': 1.8e-05, 'logqs/rejected': -2.264528512954712, 'logqs/chosen': -1.001298189163208, 'logqs_logps/rejected': -0.022414974868297577, 'logps_logqs/chosen': 0.2744675278663635, 'epoch': 0.07} 4%|▎ | 270/7500 [09:02<3:59:02, 1.98s/it] 4%|▎ | 271/7500 [09:04<3:59:23, 1.99s/it] 4%|▎ | 272/7500 [09:06<4:00:43, 2.00s/it] 4%|▎ | 273/7500 [09:08<4:00:16, 1.99s/it] 4%|▎ | 274/7500 [09:10<4:02:02, 2.01s/it] 4%|▎ | 275/7500 [09:12<3:59:26, 1.99s/it] 4%|▎ | 276/7500 [09:14<4:00:48, 2.00s/it] 4%|▎ | 277/7500 [09:16<4:01:01, 2.00s/it] 4%|▎ | 278/7500 [09:18<3:59:43, 1.99s/it] 4%|▎ | 279/7500 [09:20<3:59:51, 1.99s/it] 4%|▎ | 280/7500 [09:22<3:59:26, 1.99s/it] {'loss': 0.5255, 'grad_norm': 7.654947757720947, 'learning_rate': 1.866666666666667e-05, 'logqs/rejected': -2.5268959999084473, 'logqs/chosen': -0.967708945274353, 'logqs_logps/rejected': -0.01886928454041481, 'logps_logqs/chosen': 0.2643360197544098, 'epoch': 0.07} 4%|▎ | 280/7500 [09:22<3:59:26, 1.99s/it] 4%|▎ | 281/7500 [09:24<4:04:12, 2.03s/it] 4%|▍ | 282/7500 [09:26<4:04:06, 2.03s/it] 4%|▍ | 283/7500 [09:28<4:03:38, 2.03s/it] 4%|▍ | 284/7500 [09:30<4:04:32, 2.03s/it] 4%|▍ | 285/7500 [09:32<4:04:19, 2.03s/it] 4%|▍ | 286/7500 [09:34<4:04:23, 2.03s/it] 4%|▍ | 287/7500 [09:36<4:04:13, 2.03s/it] 4%|▍ | 288/7500 [09:38<4:02:33, 2.02s/it] 4%|▍ | 289/7500 [09:40<4:01:02, 2.01s/it] 4%|▍ | 290/7500 [09:42<4:05:11, 2.04s/it] {'loss': 0.4898, 'grad_norm': 10.260374069213867, 'learning_rate': 1.9333333333333333e-05, 'logqs/rejected': -2.3921210765838623, 'logqs/chosen': -0.9607987403869629, 'logqs_logps/rejected': -0.02684609964489937, 'logps_logqs/chosen': 0.2606453597545624, 'epoch': 0.08} 4%|▍ | 290/7500 [09:42<4:05:11, 2.04s/it] 4%|▍ | 291/7500 [09:44<4:02:00, 2.01s/it] 4%|▍ | 292/7500 [09:46<4:04:03, 2.03s/it] 4%|▍ | 293/7500 [09:48<4:01:37, 2.01s/it] 4%|▍ | 294/7500 [09:50<4:00:02, 2.00s/it] 4%|▍ | 295/7500 [09:52<4:00:08, 2.00s/it] 4%|▍ | 296/7500 [09:54<4:00:50, 2.01s/it] 4%|▍ | 297/7500 [09:56<3:59:50, 2.00s/it] 4%|▍ | 298/7500 [09:58<3:57:03, 1.97s/it] 4%|▍ | 299/7500 [10:00<4:00:07, 2.00s/it] 4%|▍ | 300/7500 [10:02<4:04:36, 2.04s/it] {'loss': 0.513, 'grad_norm': 6.034065246582031, 'learning_rate': 2e-05, 'logqs/rejected': -2.3824775218963623, 'logqs/chosen': -0.9604107141494751, 'logqs_logps/rejected': -0.015484345145523548, 'logps_logqs/chosen': 0.257590651512146, 'epoch': 0.08} 4%|▍ | 300/7500 [10:02<4:04:36, 2.04s/it] 4%|▍ | 301/7500 [10:04<4:04:42, 2.04s/it] 4%|▍ | 302/7500 [10:06<4:03:24, 2.03s/it] 4%|▍ | 303/7500 [10:08<4:03:22, 2.03s/it] 4%|▍ | 304/7500 [10:10<4:05:06, 2.04s/it] 4%|▍ | 305/7500 [10:12<4:03:28, 2.03s/it] 4%|▍ | 306/7500 [10:14<4:03:42, 2.03s/it] 4%|▍ | 307/7500 [10:16<4:01:50, 2.02s/it] 4%|▍ | 308/7500 [10:18<4:01:49, 2.02s/it] 4%|▍ | 309/7500 [10:20<3:58:41, 1.99s/it] 4%|▍ | 310/7500 [10:22<4:00:34, 2.01s/it] {'loss': 0.5054, 'grad_norm': 5.6131415367126465, 'learning_rate': 2.0666666666666666e-05, 'logqs/rejected': -2.4027295112609863, 'logqs/chosen': -0.9498389959335327, 'logqs_logps/rejected': -0.01957717165350914, 'logps_logqs/chosen': 0.24774718284606934, 'epoch': 0.08} 4%|▍ | 310/7500 [10:22<4:00:34, 2.01s/it] 4%|▍ | 311/7500 [10:24<3:59:33, 2.00s/it] 4%|▍ | 312/7500 [10:26<3:59:50, 2.00s/it] 4%|▍ | 313/7500 [10:28<4:03:05, 2.03s/it] 4%|▍ | 314/7500 [10:30<4:03:47, 2.04s/it] 4%|▍ | 315/7500 [10:32<4:06:03, 2.05s/it] 4%|▍ | 316/7500 [10:35<4:06:03, 2.06s/it] 4%|▍ | 317/7500 [10:37<4:05:47, 2.05s/it] 4%|▍ | 318/7500 [10:39<4:03:29, 2.03s/it] 4%|▍ | 319/7500 [10:41<4:00:03, 2.01s/it] 4%|▍ | 320/7500 [10:43<4:02:58, 2.03s/it] {'loss': 0.5398, 'grad_norm': 10.175050735473633, 'learning_rate': 2.1333333333333335e-05, 'logqs/rejected': -2.415235757827759, 'logqs/chosen': -0.9518426060676575, 'logqs_logps/rejected': -0.00980916153639555, 'logps_logqs/chosen': 0.25366705656051636, 'epoch': 0.09} 4%|▍ | 320/7500 [10:43<4:02:58, 2.03s/it] 4%|▍ | 321/7500 [10:45<4:04:26, 2.04s/it] 4%|▍ | 322/7500 [10:47<4:07:34, 2.07s/it] 4%|▍ | 323/7500 [10:49<4:09:54, 2.09s/it] 4%|▍ | 324/7500 [10:51<4:08:52, 2.08s/it] 4%|▍ | 325/7500 [10:53<4:09:40, 2.09s/it] 4%|▍ | 326/7500 [10:55<4:10:08, 2.09s/it] 4%|▍ | 327/7500 [10:57<4:05:43, 2.06s/it] 4%|▍ | 328/7500 [10:59<4:05:04, 2.05s/it] 4%|▍ | 329/7500 [11:01<4:02:44, 2.03s/it] 4%|▍ | 330/7500 [11:03<4:03:44, 2.04s/it] {'loss': 0.5072, 'grad_norm': 8.246116638183594, 'learning_rate': 2.2000000000000003e-05, 'logqs/rejected': -2.455138683319092, 'logqs/chosen': -0.9147905111312866, 'logqs_logps/rejected': -0.01797662302851677, 'logps_logqs/chosen': 0.23924462497234344, 'epoch': 0.09} 4%|▍ | 330/7500 [11:03<4:03:44, 2.04s/it] 4%|▍ | 331/7500 [11:05<4:00:03, 2.01s/it] 4%|▍ | 332/7500 [11:07<3:58:46, 2.00s/it] 4%|▍ | 333/7500 [11:09<3:59:54, 2.01s/it] 4%|▍ | 334/7500 [11:11<4:01:14, 2.02s/it] 4%|▍ | 335/7500 [11:13<4:02:01, 2.03s/it] 4%|▍ | 336/7500 [11:15<4:00:14, 2.01s/it] 4%|▍ | 337/7500 [11:17<4:01:28, 2.02s/it] 5%|▍ | 338/7500 [11:19<4:01:28, 2.02s/it] 5%|▍ | 339/7500 [11:21<4:02:02, 2.03s/it] 5%|▍ | 340/7500 [11:23<3:59:02, 2.00s/it] {'loss': 0.5277, 'grad_norm': 5.650853157043457, 'learning_rate': 2.2666666666666668e-05, 'logqs/rejected': -2.6037790775299072, 'logqs/chosen': -0.9433015584945679, 'logqs_logps/rejected': -0.02114572934806347, 'logps_logqs/chosen': 0.25682130455970764, 'epoch': 0.09} 5%|▍ | 340/7500 [11:23<3:59:02, 2.00s/it] 5%|▍ | 341/7500 [11:25<3:56:10, 1.98s/it] 5%|▍ | 342/7500 [11:27<3:57:58, 1.99s/it] 5%|▍ | 343/7500 [11:29<3:58:31, 2.00s/it] 5%|▍ | 344/7500 [11:31<4:00:24, 2.02s/it] 5%|▍ | 345/7500 [11:33<3:58:45, 2.00s/it] 5%|▍ | 346/7500 [11:35<3:56:59, 1.99s/it] 5%|▍ | 347/7500 [11:37<3:58:47, 2.00s/it] 5%|▍ | 348/7500 [11:39<3:57:38, 1.99s/it] 5%|▍ | 349/7500 [11:41<4:02:48, 2.04s/it] 5%|▍ | 350/7500 [11:43<4:02:42, 2.04s/it] {'loss': 0.5225, 'grad_norm': 16.936059951782227, 'learning_rate': 2.3333333333333336e-05, 'logqs/rejected': -2.471881151199341, 'logqs/chosen': -0.9848663210868835, 'logqs_logps/rejected': -0.02244795486330986, 'logps_logqs/chosen': 0.25425708293914795, 'epoch': 0.09} 5%|▍ | 350/7500 [11:43<4:02:42, 2.04s/it] 5%|▍ | 351/7500 [11:45<3:58:44, 2.00s/it] 5%|▍ | 352/7500 [11:47<3:58:32, 2.00s/it] 5%|▍ | 353/7500 [11:49<3:57:02, 1.99s/it] 5%|▍ | 354/7500 [11:51<3:53:29, 1.96s/it] 5%|▍ | 355/7500 [11:53<3:58:00, 2.00s/it] 5%|▍ | 356/7500 [11:55<3:56:17, 1.98s/it] 5%|▍ | 357/7500 [11:57<3:54:02, 1.97s/it] 5%|▍ | 358/7500 [11:59<3:53:24, 1.96s/it] 5%|▍ | 359/7500 [12:01<3:54:24, 1.97s/it] 5%|▍ | 360/7500 [12:03<3:54:13, 1.97s/it] {'loss': 0.4766, 'grad_norm': 5.972349166870117, 'learning_rate': 2.4e-05, 'logqs/rejected': -2.4947867393493652, 'logqs/chosen': -0.9315992593765259, 'logqs_logps/rejected': -0.025729354470968246, 'logps_logqs/chosen': 0.24706986546516418, 'epoch': 0.1} 5%|▍ | 360/7500 [12:03<3:54:13, 1.97s/it] 5%|▍ | 361/7500 [12:05<3:58:43, 2.01s/it] 5%|▍ | 362/7500 [12:07<4:01:45, 2.03s/it] 5%|▍ | 363/7500 [12:09<3:58:14, 2.00s/it] 5%|▍ | 364/7500 [12:11<4:00:55, 2.03s/it] 5%|▍ | 365/7500 [12:13<3:56:55, 1.99s/it] 5%|▍ | 366/7500 [12:15<3:58:30, 2.01s/it] 5%|▍ | 367/7500 [12:17<3:56:52, 1.99s/it] 5%|▍ | 368/7500 [12:19<3:56:17, 1.99s/it] 5%|▍ | 369/7500 [12:21<3:57:22, 2.00s/it] 5%|▍ | 370/7500 [12:23<3:55:17, 1.98s/it] {'loss': 0.4715, 'grad_norm': 6.544869899749756, 'learning_rate': 2.466666666666667e-05, 'logqs/rejected': -2.3478152751922607, 'logqs/chosen': -0.9704171419143677, 'logqs_logps/rejected': -0.02512136660516262, 'logps_logqs/chosen': 0.25436487793922424, 'epoch': 0.1} 5%|▍ | 370/7500 [12:23<3:55:17, 1.98s/it] 5%|▍ | 371/7500 [12:25<3:57:44, 2.00s/it] 5%|▍ | 372/7500 [12:27<3:58:24, 2.01s/it] 5%|▍ | 373/7500 [12:29<3:57:37, 2.00s/it] 5%|▍ | 374/7500 [12:31<4:00:47, 2.03s/it] 5%|▌ | 375/7500 [12:33<3:55:17, 1.98s/it] 5%|▌ | 376/7500 [12:35<3:57:29, 2.00s/it] 5%|▌ | 377/7500 [12:37<3:56:44, 1.99s/it] 5%|▌ | 378/7500 [12:39<4:01:08, 2.03s/it] 5%|▌ | 379/7500 [12:41<4:01:19, 2.03s/it] 5%|▌ | 380/7500 [12:43<4:00:35, 2.03s/it] {'loss': 0.4852, 'grad_norm': 6.1360087394714355, 'learning_rate': 2.5333333333333337e-05, 'logqs/rejected': -2.3703601360321045, 'logqs/chosen': -0.9511802792549133, 'logqs_logps/rejected': -0.02413088083267212, 'logps_logqs/chosen': 0.24224667251110077, 'epoch': 0.1} 5%|▌ | 380/7500 [12:43<4:00:35, 2.03s/it] 5%|▌ | 381/7500 [12:45<4:01:14, 2.03s/it] 5%|▌ | 382/7500 [12:47<3:59:58, 2.02s/it] 5%|▌ | 383/7500 [12:49<3:59:14, 2.02s/it] 5%|▌ | 384/7500 [12:51<3:59:38, 2.02s/it] 5%|▌ | 385/7500 [12:53<3:57:00, 2.00s/it] 5%|▌ | 386/7500 [12:55<3:54:53, 1.98s/it] 5%|▌ | 387/7500 [12:57<3:55:13, 1.98s/it] 5%|▌ | 388/7500 [12:59<3:59:59, 2.02s/it] 5%|▌ | 389/7500 [13:01<4:00:47, 2.03s/it] 5%|▌ | 390/7500 [13:04<4:01:26, 2.04s/it] {'loss': 0.4969, 'grad_norm': 5.493791103363037, 'learning_rate': 2.6000000000000002e-05, 'logqs/rejected': -2.4148240089416504, 'logqs/chosen': -0.9340194463729858, 'logqs_logps/rejected': -0.028125155717134476, 'logps_logqs/chosen': 0.245690256357193, 'epoch': 0.1} 5%|▌ | 390/7500 [13:04<4:01:26, 2.04s/it] 5%|▌ | 391/7500 [13:06<4:01:31, 2.04s/it] 5%|▌ | 392/7500 [13:08<4:03:35, 2.06s/it] 5%|▌ | 393/7500 [13:10<4:01:19, 2.04s/it] 5%|▌ | 394/7500 [13:12<3:59:04, 2.02s/it] 5%|▌ | 395/7500 [13:14<3:58:47, 2.02s/it] 5%|▌ | 396/7500 [13:16<3:55:52, 1.99s/it] 5%|▌ | 397/7500 [13:18<3:58:29, 2.01s/it] 5%|▌ | 398/7500 [13:20<4:00:05, 2.03s/it] 5%|▌ | 399/7500 [13:22<4:03:12, 2.05s/it] 5%|▌ | 400/7500 [13:24<4:03:05, 2.05s/it] {'loss': 0.4958, 'grad_norm': 5.0760345458984375, 'learning_rate': 2.6666666666666667e-05, 'logqs/rejected': -2.3461880683898926, 'logqs/chosen': -0.9484657049179077, 'logqs_logps/rejected': -0.021294891834259033, 'logps_logqs/chosen': 0.24541568756103516, 'epoch': 0.11} 5%|▌ | 400/7500 [13:24<4:03:05, 2.05s/it] 5%|▌ | 401/7500 [13:26<4:01:00, 2.04s/it] 5%|▌ | 402/7500 [13:28<4:01:08, 2.04s/it] 5%|▌ | 403/7500 [13:30<4:05:25, 2.07s/it] 5%|▌ | 404/7500 [13:32<4:02:20, 2.05s/it] 5%|▌ | 405/7500 [13:34<4:02:30, 2.05s/it] 5%|▌ | 406/7500 [13:36<4:03:50, 2.06s/it] 5%|▌ | 407/7500 [13:38<4:02:00, 2.05s/it] 5%|▌ | 408/7500 [13:40<3:57:22, 2.01s/it] 5%|▌ | 409/7500 [13:42<4:01:15, 2.04s/it] 5%|▌ | 410/7500 [13:44<4:00:46, 2.04s/it] {'loss': 0.4501, 'grad_norm': 5.334781169891357, 'learning_rate': 2.733333333333333e-05, 'logqs/rejected': -2.1954762935638428, 'logqs/chosen': -0.9099380373954773, 'logqs_logps/rejected': -0.015746772289276123, 'logps_logqs/chosen': 0.23156630992889404, 'epoch': 0.11} 5%|▌ | 410/7500 [13:44<4:00:46, 2.04s/it] 5%|▌ | 411/7500 [13:46<3:58:45, 2.02s/it] 5%|▌ | 412/7500 [13:48<3:59:02, 2.02s/it] 6%|▌ | 413/7500 [13:50<4:00:06, 2.03s/it] 6%|▌ | 414/7500 [13:52<4:02:04, 2.05s/it] 6%|▌ | 415/7500 [13:54<4:01:06, 2.04s/it] 6%|▌ | 416/7500 [13:56<3:59:24, 2.03s/it] 6%|▌ | 417/7500 [13:59<4:00:36, 2.04s/it] 6%|▌ | 418/7500 [14:01<3:58:47, 2.02s/it] 6%|▌ | 419/7500 [14:03<4:00:12, 2.04s/it] 6%|▌ | 420/7500 [14:05<3:58:32, 2.02s/it] {'loss': 0.4904, 'grad_norm': 5.286783695220947, 'learning_rate': 2.8000000000000003e-05, 'logqs/rejected': -2.3820581436157227, 'logqs/chosen': -0.8889561891555786, 'logqs_logps/rejected': -0.015984565019607544, 'logps_logqs/chosen': 0.2339855432510376, 'epoch': 0.11} 6%|▌ | 420/7500 [14:05<3:58:32, 2.02s/it] 6%|▌ | 421/7500 [14:07<3:56:55, 2.01s/it] 6%|▌ | 422/7500 [14:09<3:56:47, 2.01s/it] 6%|▌ | 423/7500 [14:11<3:55:20, 2.00s/it] 6%|▌ | 424/7500 [14:12<3:52:42, 1.97s/it] 6%|▌ | 425/7500 [14:14<3:54:55, 1.99s/it] 6%|▌ | 426/7500 [14:17<3:57:06, 2.01s/it] 6%|▌ | 427/7500 [14:19<3:59:10, 2.03s/it] 6%|▌ | 428/7500 [14:21<3:58:21, 2.02s/it] 6%|▌ | 429/7500 [14:23<3:57:08, 2.01s/it] 6%|▌ | 430/7500 [14:25<3:57:25, 2.01s/it] {'loss': 0.5122, 'grad_norm': 6.226876735687256, 'learning_rate': 2.8666666666666668e-05, 'logqs/rejected': -2.5425946712493896, 'logqs/chosen': -0.9313942193984985, 'logqs_logps/rejected': -0.013821355998516083, 'logps_logqs/chosen': 0.24699290096759796, 'epoch': 0.11} 6%|▌ | 430/7500 [14:25<3:57:25, 2.01s/it] 6%|▌ | 431/7500 [14:27<3:55:54, 2.00s/it] 6%|▌ | 432/7500 [14:29<3:55:32, 2.00s/it] 6%|▌ | 433/7500 [14:31<3:56:00, 2.00s/it] 6%|▌ | 434/7500 [14:32<3:51:59, 1.97s/it] 6%|▌ | 435/7500 [14:34<3:51:30, 1.97s/it] 6%|▌ | 436/7500 [14:36<3:50:54, 1.96s/it] 6%|▌ | 437/7500 [14:38<3:55:07, 2.00s/it] 6%|▌ | 438/7500 [14:40<3:53:49, 1.99s/it] 6%|▌ | 439/7500 [14:42<3:52:03, 1.97s/it] 6%|▌ | 440/7500 [14:44<3:54:37, 1.99s/it] {'loss': 0.4977, 'grad_norm': 6.629444122314453, 'learning_rate': 2.9333333333333336e-05, 'logqs/rejected': -2.7565362453460693, 'logqs/chosen': -0.9572598338127136, 'logqs_logps/rejected': -0.027072230353951454, 'logps_logqs/chosen': 0.2567192614078522, 'epoch': 0.12} 6%|▌ | 440/7500 [14:44<3:54:37, 1.99s/it] 6%|▌ | 441/7500 [14:46<3:54:13, 1.99s/it] 6%|▌ | 442/7500 [14:48<3:54:48, 2.00s/it] 6%|▌ | 443/7500 [14:51<3:57:43, 2.02s/it] 6%|▌ | 444/7500 [14:52<3:56:03, 2.01s/it] 6%|▌ | 445/7500 [14:54<3:53:51, 1.99s/it] 6%|▌ | 446/7500 [14:56<3:49:46, 1.95s/it] 6%|▌ | 447/7500 [14:58<3:54:03, 1.99s/it] 6%|▌ | 448/7500 [15:00<3:53:22, 1.99s/it] 6%|▌ | 449/7500 [15:02<3:51:35, 1.97s/it] 6%|▌ | 450/7500 [15:04<3:54:31, 2.00s/it] {'loss': 0.4968, 'grad_norm': 5.350918769836426, 'learning_rate': 3e-05, 'logqs/rejected': -2.48307466506958, 'logqs/chosen': -0.9087201952934265, 'logqs_logps/rejected': -0.017550479620695114, 'logps_logqs/chosen': 0.23335862159729004, 'epoch': 0.12} 6%|▌ | 450/7500 [15:04<3:54:31, 2.00s/it] 6%|▌ | 451/7500 [15:06<3:53:23, 1.99s/it] 6%|▌ | 452/7500 [15:08<3:52:17, 1.98s/it] 6%|▌ | 453/7500 [15:10<3:51:34, 1.97s/it] 6%|▌ | 454/7500 [15:12<3:56:37, 2.01s/it] 6%|▌ | 455/7500 [15:14<3:52:10, 1.98s/it] 6%|▌ | 456/7500 [15:16<3:55:20, 2.00s/it] 6%|▌ | 457/7500 [15:18<3:54:05, 1.99s/it] 6%|▌ | 458/7500 [15:20<3:53:10, 1.99s/it] 6%|▌ | 459/7500 [15:22<3:55:13, 2.00s/it] 6%|▌ | 460/7500 [15:24<3:56:06, 2.01s/it] {'loss': 0.4811, 'grad_norm': 5.345128059387207, 'learning_rate': 3.066666666666667e-05, 'logqs/rejected': -2.4155147075653076, 'logqs/chosen': -0.939001739025116, 'logqs_logps/rejected': -0.02310023084282875, 'logps_logqs/chosen': 0.2432173788547516, 'epoch': 0.12} 6%|▌ | 460/7500 [15:24<3:56:06, 2.01s/it] 6%|▌ | 461/7500 [15:26<3:53:52, 1.99s/it] 6%|▌ | 462/7500 [15:28<3:54:20, 2.00s/it] 6%|▌ | 463/7500 [15:30<3:54:36, 2.00s/it] 6%|▌ | 464/7500 [15:32<3:55:24, 2.01s/it] 6%|▌ | 465/7500 [15:34<3:58:31, 2.03s/it] 6%|▌ | 466/7500 [15:36<3:53:22, 1.99s/it] 6%|▌ | 467/7500 [15:38<3:51:30, 1.98s/it] 6%|▌ | 468/7500 [15:40<3:53:51, 2.00s/it] 6%|▋ | 469/7500 [15:42<3:51:36, 1.98s/it] 6%|▋ | 470/7500 [15:44<3:49:10, 1.96s/it] {'loss': 0.4923, 'grad_norm': 5.721433162689209, 'learning_rate': 3.1333333333333334e-05, 'logqs/rejected': -2.3975024223327637, 'logqs/chosen': -0.9537062644958496, 'logqs_logps/rejected': -0.021088995039463043, 'logps_logqs/chosen': 0.24367618560791016, 'epoch': 0.13} 6%|▋ | 470/7500 [15:44<3:49:10, 1.96s/it] 6%|▋ | 471/7500 [15:46<3:49:50, 1.96s/it] 6%|▋ | 472/7500 [15:48<3:54:26, 2.00s/it] 6%|▋ | 473/7500 [15:50<3:54:49, 2.01s/it] 6%|▋ | 474/7500 [15:52<3:56:30, 2.02s/it] 6%|▋ | 475/7500 [15:54<3:53:34, 1.99s/it] 6%|▋ | 476/7500 [15:56<3:54:49, 2.01s/it] 6%|▋ | 477/7500 [15:58<3:58:39, 2.04s/it] 6%|▋ | 478/7500 [16:00<3:56:40, 2.02s/it] 6%|▋ | 479/7500 [16:02<3:55:24, 2.01s/it] 6%|▋ | 480/7500 [16:04<3:57:22, 2.03s/it] {'loss': 0.5106, 'grad_norm': 5.784280776977539, 'learning_rate': 3.2000000000000005e-05, 'logqs/rejected': -2.42152738571167, 'logqs/chosen': -0.922091007232666, 'logqs_logps/rejected': -0.013227418065071106, 'logps_logqs/chosen': 0.23579850792884827, 'epoch': 0.13} 6%|▋ | 480/7500 [16:04<3:57:22, 2.03s/it] 6%|▋ | 481/7500 [16:06<3:57:52, 2.03s/it] 6%|▋ | 482/7500 [16:08<3:56:02, 2.02s/it] 6%|▋ | 483/7500 [16:10<3:58:16, 2.04s/it] 6%|▋ | 484/7500 [16:13<3:58:36, 2.04s/it] 6%|▋ | 485/7500 [16:15<3:57:01, 2.03s/it] 6%|▋ | 486/7500 [16:17<3:56:00, 2.02s/it] 6%|▋ | 487/7500 [16:19<3:58:50, 2.04s/it] 7%|▋ | 488/7500 [16:21<3:57:31, 2.03s/it] 7%|▋ | 489/7500 [16:23<3:59:08, 2.05s/it] 7%|▋ | 490/7500 [16:25<3:58:19, 2.04s/it] {'loss': 0.4852, 'grad_norm': 6.127557754516602, 'learning_rate': 3.266666666666667e-05, 'logqs/rejected': -2.3716423511505127, 'logqs/chosen': -0.9458481073379517, 'logqs_logps/rejected': -0.01939021237194538, 'logps_logqs/chosen': 0.2360747754573822, 'epoch': 0.13} 7%|▋ | 490/7500 [16:25<3:58:19, 2.04s/it] 7%|▋ | 491/7500 [16:27<3:57:16, 2.03s/it] 7%|▋ | 492/7500 [16:29<3:55:57, 2.02s/it] 7%|▋ | 493/7500 [16:31<3:56:12, 2.02s/it] 7%|▋ | 494/7500 [16:33<3:55:51, 2.02s/it] 7%|▋ | 495/7500 [16:35<3:57:22, 2.03s/it] 7%|▋ | 496/7500 [16:37<3:54:26, 2.01s/it] 7%|▋ | 497/7500 [16:39<3:52:49, 1.99s/it] 7%|▋ | 498/7500 [16:41<3:53:14, 2.00s/it] 7%|▋ | 499/7500 [16:43<3:53:40, 2.00s/it] 7%|▋ | 500/7500 [16:45<3:55:55, 2.02s/it] {'loss': 0.4681, 'grad_norm': 4.976317882537842, 'learning_rate': 3.3333333333333335e-05, 'logqs/rejected': -2.3980276584625244, 'logqs/chosen': -0.9156268239021301, 'logqs_logps/rejected': -0.01939373090863228, 'logps_logqs/chosen': 0.24052485823631287, 'epoch': 0.13} 7%|▋ | 500/7500 [16:45<3:55:55, 2.02s/it] 7%|▋ | 501/7500 [16:47<3:55:13, 2.02s/it] 7%|▋ | 502/7500 [16:49<3:52:40, 1.99s/it] 7%|▋ | 503/7500 [16:51<3:53:14, 2.00s/it] 7%|▋ | 504/7500 [16:53<3:57:41, 2.04s/it] 7%|▋ | 505/7500 [16:55<3:56:47, 2.03s/it] 7%|▋ | 506/7500 [16:57<3:54:58, 2.02s/it] 7%|▋ | 507/7500 [16:59<3:54:56, 2.02s/it] 7%|▋ | 508/7500 [17:01<3:57:09, 2.04s/it] 7%|▋ | 509/7500 [17:03<3:55:48, 2.02s/it] 7%|▋ | 510/7500 [17:05<3:58:26, 2.05s/it] {'loss': 0.5218, 'grad_norm': 6.175568580627441, 'learning_rate': 3.4000000000000007e-05, 'logqs/rejected': -2.45735239982605, 'logqs/chosen': -0.9779577255249023, 'logqs_logps/rejected': -0.009660984389483929, 'logps_logqs/chosen': 0.24760368466377258, 'epoch': 0.14} 7%|▋ | 510/7500 [17:05<3:58:26, 2.05s/it] 7%|▋ | 511/7500 [17:07<3:55:58, 2.03s/it] 7%|▋ | 512/7500 [17:09<3:54:28, 2.01s/it] 7%|▋ | 513/7500 [17:11<3:50:12, 1.98s/it] 7%|▋ | 514/7500 [17:13<3:48:04, 1.96s/it] 7%|▋ | 515/7500 [17:15<3:49:15, 1.97s/it] 7%|▋ | 516/7500 [17:17<3:50:54, 1.98s/it] 7%|▋ | 517/7500 [17:19<3:53:56, 2.01s/it] 7%|▋ | 518/7500 [17:21<4:02:57, 2.09s/it] 7%|▋ | 519/7500 [17:23<3:59:37, 2.06s/it] 7%|▋ | 520/7500 [17:25<3:59:33, 2.06s/it] {'loss': 0.494, 'grad_norm': 5.903124809265137, 'learning_rate': 3.466666666666667e-05, 'logqs/rejected': -2.395267963409424, 'logqs/chosen': -0.9655419588088989, 'logqs_logps/rejected': -0.021599214524030685, 'logps_logqs/chosen': 0.2510065734386444, 'epoch': 0.14} 7%|▋ | 520/7500 [17:25<3:59:33, 2.06s/it] 7%|▋ | 521/7500 [17:27<3:56:20, 2.03s/it] 7%|▋ | 522/7500 [17:29<3:57:34, 2.04s/it] 7%|▋ | 523/7500 [17:31<3:54:33, 2.02s/it] 7%|▋ | 524/7500 [17:33<3:53:17, 2.01s/it] 7%|▋ | 525/7500 [17:35<3:51:46, 1.99s/it] 7%|▋ | 526/7500 [17:37<3:50:29, 1.98s/it] 7%|▋ | 527/7500 [17:39<3:51:46, 1.99s/it] 7%|▋ | 528/7500 [17:41<3:51:01, 1.99s/it] 7%|▋ | 529/7500 [17:43<3:53:13, 2.01s/it] 7%|▋ | 530/7500 [17:45<3:51:16, 1.99s/it] {'loss': 0.484, 'grad_norm': 5.954137325286865, 'learning_rate': 3.5333333333333336e-05, 'logqs/rejected': -2.269402265548706, 'logqs/chosen': -0.9670447111129761, 'logqs_logps/rejected': -0.02733471989631653, 'logps_logqs/chosen': 0.25284090638160706, 'epoch': 0.14} 7%|▋ | 530/7500 [17:45<3:51:16, 1.99s/it] 7%|▋ | 531/7500 [17:47<3:49:53, 1.98s/it] 7%|▋ | 532/7500 [17:49<3:55:37, 2.03s/it] 7%|▋ | 533/7500 [17:51<3:55:01, 2.02s/it] 7%|▋ | 534/7500 [17:53<3:55:16, 2.03s/it] 7%|▋ | 535/7500 [17:55<3:55:50, 2.03s/it] 7%|▋ | 536/7500 [17:57<3:54:52, 2.02s/it] 7%|▋ | 537/7500 [17:59<3:52:34, 2.00s/it] 7%|▋ | 538/7500 [18:01<3:53:28, 2.01s/it] 7%|▋ | 539/7500 [18:03<3:54:05, 2.02s/it] 7%|▋ | 540/7500 [18:05<3:51:14, 1.99s/it] {'loss': 0.5207, 'grad_norm': 8.412433624267578, 'learning_rate': 3.6e-05, 'logqs/rejected': -2.468135356903076, 'logqs/chosen': -0.9467852711677551, 'logqs_logps/rejected': -0.015826771035790443, 'logps_logqs/chosen': 0.24712534248828888, 'epoch': 0.14} 7%|▋ | 540/7500 [18:05<3:51:14, 1.99s/it] 7%|▋ | 541/7500 [18:07<3:50:54, 1.99s/it] 7%|▋ | 542/7500 [18:09<3:52:37, 2.01s/it] 7%|▋ | 543/7500 [18:11<3:53:37, 2.01s/it] 7%|▋ | 544/7500 [18:13<3:56:33, 2.04s/it] 7%|▋ | 545/7500 [18:15<3:52:33, 2.01s/it] 7%|▋ | 546/7500 [18:17<3:50:56, 1.99s/it] 7%|▋ | 547/7500 [18:19<3:53:52, 2.02s/it] 7%|▋ | 548/7500 [18:22<3:55:57, 2.04s/it] 7%|▋ | 549/7500 [18:24<3:57:46, 2.05s/it] 7%|▋ | 550/7500 [18:26<3:57:52, 2.05s/it] {'loss': 0.5122, 'grad_norm': 6.119631767272949, 'learning_rate': 3.6666666666666666e-05, 'logqs/rejected': -2.495227813720703, 'logqs/chosen': -0.9584010243415833, 'logqs_logps/rejected': -0.020123371854424477, 'logps_logqs/chosen': 0.254245400428772, 'epoch': 0.15} 7%|▋ | 550/7500 [18:26<3:57:52, 2.05s/it] 7%|▋ | 551/7500 [18:28<3:55:46, 2.04s/it] 7%|▋ | 552/7500 [18:30<3:57:10, 2.05s/it] 7%|▋ | 553/7500 [18:32<3:57:48, 2.05s/it] 7%|▋ | 554/7500 [18:34<3:55:19, 2.03s/it] 7%|▋ | 555/7500 [18:36<3:54:50, 2.03s/it] 7%|▋ | 556/7500 [18:38<3:56:27, 2.04s/it] 7%|▋ | 557/7500 [18:40<3:55:36, 2.04s/it] 7%|▋ | 558/7500 [18:42<3:55:47, 2.04s/it] 7%|▋ | 559/7500 [18:44<3:52:37, 2.01s/it] 7%|▋ | 560/7500 [18:46<3:50:18, 1.99s/it] {'loss': 0.4747, 'grad_norm': 5.061271667480469, 'learning_rate': 3.733333333333334e-05, 'logqs/rejected': -2.4116296768188477, 'logqs/chosen': -0.9331947565078735, 'logqs_logps/rejected': -0.01965726539492607, 'logps_logqs/chosen': 0.2412753403186798, 'epoch': 0.15} 7%|▋ | 560/7500 [18:46<3:50:18, 1.99s/it] 7%|▋ | 561/7500 [18:48<3:53:00, 2.01s/it] 7%|▋ | 562/7500 [18:50<3:54:30, 2.03s/it] 8%|▊ | 563/7500 [18:52<3:54:15, 2.03s/it] 8%|▊ | 564/7500 [18:54<3:52:22, 2.01s/it] 8%|▊ | 565/7500 [18:56<3:54:05, 2.03s/it] 8%|▊ | 566/7500 [18:58<3:55:07, 2.03s/it] 8%|▊ | 567/7500 [19:00<3:53:40, 2.02s/it] 8%|▊ | 568/7500 [19:02<3:51:34, 2.00s/it] 8%|▊ | 569/7500 [19:04<3:53:26, 2.02s/it] 8%|▊ | 570/7500 [19:06<3:51:58, 2.01s/it] {'loss': 0.5024, 'grad_norm': 5.405981540679932, 'learning_rate': 3.8e-05, 'logqs/rejected': -2.5944881439208984, 'logqs/chosen': -0.9310491681098938, 'logqs_logps/rejected': -0.021863818168640137, 'logps_logqs/chosen': 0.23880405724048615, 'epoch': 0.15} 8%|▊ | 570/7500 [19:06<3:51:58, 2.01s/it] 8%|▊ | 571/7500 [19:08<3:56:26, 2.05s/it] 8%|▊ | 572/7500 [19:10<3:53:48, 2.02s/it] 8%|▊ | 573/7500 [19:12<3:53:33, 2.02s/it] 8%|▊ | 574/7500 [19:14<3:52:57, 2.02s/it] 8%|▊ | 575/7500 [19:16<3:51:21, 2.00s/it] 8%|▊ | 576/7500 [19:18<3:51:56, 2.01s/it] 8%|▊ | 577/7500 [19:20<3:50:19, 2.00s/it] 8%|▊ | 578/7500 [19:22<3:50:36, 2.00s/it] 8%|▊ | 579/7500 [19:24<3:49:38, 1.99s/it] 8%|▊ | 580/7500 [19:26<3:50:36, 2.00s/it] {'loss': 0.4967, 'grad_norm': 5.482855796813965, 'learning_rate': 3.866666666666667e-05, 'logqs/rejected': -2.333350896835327, 'logqs/chosen': -0.964136004447937, 'logqs_logps/rejected': -0.026567641645669937, 'logps_logqs/chosen': 0.24904675781726837, 'epoch': 0.15} 8%|▊ | 580/7500 [19:26<3:50:36, 2.00s/it] 8%|▊ | 581/7500 [19:28<3:52:51, 2.02s/it] 8%|▊ | 582/7500 [19:30<3:56:43, 2.05s/it] 8%|▊ | 583/7500 [19:32<3:55:58, 2.05s/it] 8%|▊ | 584/7500 [19:35<3:57:35, 2.06s/it] 8%|▊ | 585/7500 [19:37<3:59:49, 2.08s/it] 8%|▊ | 586/7500 [19:39<3:58:57, 2.07s/it] 8%|▊ | 587/7500 [19:41<3:53:38, 2.03s/it] 8%|▊ | 588/7500 [19:43<3:50:59, 2.01s/it] 8%|▊ | 589/7500 [19:45<3:48:57, 1.99s/it] 8%|▊ | 590/7500 [19:47<3:50:14, 2.00s/it] {'loss': 0.5092, 'grad_norm': 5.288559913635254, 'learning_rate': 3.933333333333333e-05, 'logqs/rejected': -2.5763792991638184, 'logqs/chosen': -0.9730243682861328, 'logqs_logps/rejected': -0.015027359127998352, 'logps_logqs/chosen': 0.24375414848327637, 'epoch': 0.16} 8%|▊ | 590/7500 [19:47<3:50:14, 2.00s/it] 8%|▊ | 591/7500 [19:49<3:53:29, 2.03s/it] 8%|▊ | 592/7500 [19:51<3:51:50, 2.01s/it] 8%|▊ | 593/7500 [19:53<3:50:38, 2.00s/it] 8%|▊ | 594/7500 [19:55<3:52:32, 2.02s/it] 8%|▊ | 595/7500 [19:57<3:53:01, 2.02s/it] 8%|▊ | 596/7500 [19:59<3:51:17, 2.01s/it] 8%|▊ | 597/7500 [20:01<3:52:39, 2.02s/it] 8%|▊ | 598/7500 [20:03<3:55:36, 2.05s/it] 8%|▊ | 599/7500 [20:05<3:55:24, 2.05s/it] 8%|▊ | 600/7500 [20:07<3:51:20, 2.01s/it] {'loss': 0.4939, 'grad_norm': 5.3498663902282715, 'learning_rate': 4e-05, 'logqs/rejected': -2.382721185684204, 'logqs/chosen': -0.9347658157348633, 'logqs_logps/rejected': -0.0192805677652359, 'logps_logqs/chosen': 0.24788007140159607, 'epoch': 0.16} 8%|▊ | 600/7500 [20:07<3:51:20, 2.01s/it] 8%|▊ | 601/7500 [20:09<3:56:31, 2.06s/it] 8%|▊ | 602/7500 [20:11<3:55:35, 2.05s/it] 8%|▊ | 603/7500 [20:13<3:57:28, 2.07s/it] 8%|▊ | 604/7500 [20:15<3:54:48, 2.04s/it] 8%|▊ | 605/7500 [20:17<3:54:58, 2.04s/it] 8%|▊ | 606/7500 [20:19<3:51:45, 2.02s/it] 8%|▊ | 607/7500 [20:21<3:55:47, 2.05s/it] 8%|▊ | 608/7500 [20:23<3:56:19, 2.06s/it] 8%|▊ | 609/7500 [20:25<3:54:11, 2.04s/it] 8%|▊ | 610/7500 [20:27<3:51:39, 2.02s/it] {'loss': 0.5026, 'grad_norm': 5.840240478515625, 'learning_rate': 4.066666666666667e-05, 'logqs/rejected': -2.5596606731414795, 'logqs/chosen': -0.9763317108154297, 'logqs_logps/rejected': -0.021116625517606735, 'logps_logqs/chosen': 0.25435274839401245, 'epoch': 0.16} 8%|▊ | 610/7500 [20:27<3:51:39, 2.02s/it] 8%|▊ | 611/7500 [20:29<3:55:32, 2.05s/it] 8%|▊ | 612/7500 [20:31<3:54:12, 2.04s/it] 8%|▊ | 613/7500 [20:33<3:53:44, 2.04s/it] 8%|▊ | 614/7500 [20:35<3:52:31, 2.03s/it] 8%|▊ | 615/7500 [20:37<3:51:36, 2.02s/it] 8%|▊ | 616/7500 [20:39<3:50:39, 2.01s/it] 8%|▊ | 617/7500 [20:41<3:49:16, 2.00s/it] 8%|▊ | 618/7500 [20:43<3:51:03, 2.01s/it] 8%|▊ | 619/7500 [20:45<3:51:03, 2.01s/it] 8%|▊ | 620/7500 [20:47<3:50:17, 2.01s/it] {'loss': 0.4848, 'grad_norm': 5.897874355316162, 'learning_rate': 4.133333333333333e-05, 'logqs/rejected': -2.3973097801208496, 'logqs/chosen': -0.9331893920898438, 'logqs_logps/rejected': -0.027380798012018204, 'logps_logqs/chosen': 0.24578984081745148, 'epoch': 0.17} 8%|▊ | 620/7500 [20:47<3:50:17, 2.01s/it] 8%|▊ | 621/7500 [20:49<3:48:03, 1.99s/it] 8%|▊ | 622/7500 [20:51<3:47:49, 1.99s/it] 8%|▊ | 623/7500 [20:53<3:50:17, 2.01s/it] 8%|▊ | 624/7500 [20:55<3:50:18, 2.01s/it] 8%|▊ | 625/7500 [20:57<3:47:52, 1.99s/it] 8%|▊ | 626/7500 [20:59<3:51:00, 2.02s/it] 8%|▊ | 627/7500 [21:01<3:48:06, 1.99s/it] 8%|▊ | 628/7500 [21:03<3:50:40, 2.01s/it] 8%|▊ | 629/7500 [21:05<3:49:21, 2.00s/it] 8%|▊ | 630/7500 [21:07<3:43:52, 1.96s/it] {'loss': 0.5086, 'grad_norm': 5.586450576782227, 'learning_rate': 4.2e-05, 'logqs/rejected': -2.7544543743133545, 'logqs/chosen': -0.9990125894546509, 'logqs_logps/rejected': -0.022872211411595345, 'logps_logqs/chosen': 0.2602679431438446, 'epoch': 0.17} 8%|▊ | 630/7500 [21:07<3:43:52, 1.96s/it] 8%|▊ | 631/7500 [21:09<3:47:06, 1.98s/it] 8%|▊ | 632/7500 [21:11<3:46:56, 1.98s/it] 8%|▊ | 633/7500 [21:13<3:50:29, 2.01s/it] 8%|▊ | 634/7500 [21:15<3:49:37, 2.01s/it] 8%|▊ | 635/7500 [21:17<3:49:31, 2.01s/it] 8%|▊ | 636/7500 [21:19<3:49:54, 2.01s/it] 8%|▊ | 637/7500 [21:21<3:49:34, 2.01s/it] 9%|▊ | 638/7500 [21:23<3:50:38, 2.02s/it] 9%|▊ | 639/7500 [21:26<3:55:47, 2.06s/it] 9%|▊ | 640/7500 [21:28<3:57:55, 2.08s/it] {'loss': 0.5095, 'grad_norm': 4.999581813812256, 'learning_rate': 4.266666666666667e-05, 'logqs/rejected': -2.6451942920684814, 'logqs/chosen': -0.9507570266723633, 'logqs_logps/rejected': -0.02135433442890644, 'logps_logqs/chosen': 0.24992606043815613, 'epoch': 0.17} 9%|▊ | 640/7500 [21:28<3:57:55, 2.08s/it] 9%|▊ | 641/7500 [21:30<3:59:22, 2.09s/it] 9%|▊ | 642/7500 [21:32<3:53:03, 2.04s/it] 9%|▊ | 643/7500 [21:34<3:53:05, 2.04s/it] 9%|▊ | 644/7500 [21:36<3:51:59, 2.03s/it] 9%|▊ | 645/7500 [21:38<3:54:14, 2.05s/it] 9%|▊ | 646/7500 [21:40<3:52:28, 2.04s/it] 9%|▊ | 647/7500 [21:42<3:52:40, 2.04s/it] 9%|▊ | 648/7500 [21:44<3:52:17, 2.03s/it] 9%|▊ | 649/7500 [21:46<3:53:06, 2.04s/it] 9%|▊ | 650/7500 [21:48<3:55:15, 2.06s/it] {'loss': 0.5296, 'grad_norm': 6.569551944732666, 'learning_rate': 4.3333333333333334e-05, 'logqs/rejected': -2.779996633529663, 'logqs/chosen': -0.9697681665420532, 'logqs_logps/rejected': -0.025524616241455078, 'logps_logqs/chosen': 0.25383344292640686, 'epoch': 0.17} 9%|▊ | 650/7500 [21:48<3:55:15, 2.06s/it] 9%|▊ | 651/7500 [21:50<3:54:43, 2.06s/it] 9%|▊ | 652/7500 [21:52<3:57:27, 2.08s/it] 9%|▊ | 653/7500 [21:54<3:58:30, 2.09s/it] 9%|▊ | 654/7500 [21:56<3:54:56, 2.06s/it] 9%|▊ | 655/7500 [21:58<3:50:40, 2.02s/it] 9%|▊ | 656/7500 [22:01<3:54:09, 2.05s/it] 9%|▉ | 657/7500 [22:03<3:52:43, 2.04s/it] 9%|▉ | 658/7500 [22:04<3:50:15, 2.02s/it] 9%|▉ | 659/7500 [22:07<3:50:53, 2.03s/it] 9%|▉ | 660/7500 [22:09<3:50:47, 2.02s/it] {'loss': 0.4829, 'grad_norm': 5.204061985015869, 'learning_rate': 4.4000000000000006e-05, 'logqs/rejected': -2.0674662590026855, 'logqs/chosen': -0.9540399312973022, 'logqs_logps/rejected': -0.03230584040284157, 'logps_logqs/chosen': 0.2554740607738495, 'epoch': 0.18} 9%|▉ | 660/7500 [22:09<3:50:47, 2.02s/it] 9%|▉ | 661/7500 [22:11<3:48:29, 2.00s/it] 9%|▉ | 662/7500 [22:13<3:50:08, 2.02s/it] 9%|▉ | 663/7500 [22:15<3:52:33, 2.04s/it] 9%|▉ | 664/7500 [22:17<3:49:24, 2.01s/it] 9%|▉ | 665/7500 [22:19<3:46:59, 1.99s/it] 9%|▉ | 666/7500 [22:21<3:51:01, 2.03s/it] 9%|▉ | 667/7500 [22:23<3:51:29, 2.03s/it] 9%|▉ | 668/7500 [22:25<3:52:28, 2.04s/it] 9%|▉ | 669/7500 [22:27<3:52:49, 2.04s/it] 9%|▉ | 670/7500 [22:29<3:51:23, 2.03s/it] {'loss': 0.501, 'grad_norm': 5.752065658569336, 'learning_rate': 4.466666666666667e-05, 'logqs/rejected': -2.2433483600616455, 'logqs/chosen': -0.9674617052078247, 'logqs_logps/rejected': -0.0279021505266428, 'logps_logqs/chosen': 0.2530800402164459, 'epoch': 0.18} 9%|▉ | 670/7500 [22:29<3:51:23, 2.03s/it] 9%|▉ | 671/7500 [22:31<3:49:09, 2.01s/it] 9%|▉ | 672/7500 [22:33<3:49:56, 2.02s/it] 9%|▉ | 673/7500 [22:35<3:49:14, 2.01s/it] 9%|▉ | 674/7500 [22:37<3:45:01, 1.98s/it] 9%|▉ | 675/7500 [22:39<3:47:05, 2.00s/it] 9%|▉ | 676/7500 [22:41<3:49:12, 2.02s/it] 9%|▉ | 677/7500 [22:43<3:50:05, 2.02s/it] 9%|▉ | 678/7500 [22:45<3:50:12, 2.02s/it] 9%|▉ | 679/7500 [22:47<3:48:49, 2.01s/it] 9%|▉ | 680/7500 [22:49<3:50:26, 2.03s/it] {'loss': 0.5006, 'grad_norm': 5.972290992736816, 'learning_rate': 4.5333333333333335e-05, 'logqs/rejected': -2.348886013031006, 'logqs/chosen': -0.9398557543754578, 'logqs_logps/rejected': -0.02349594049155712, 'logps_logqs/chosen': 0.24784140288829803, 'epoch': 0.18} 9%|▉ | 680/7500 [22:49<3:50:26, 2.03s/it] 9%|▉ | 681/7500 [22:51<3:49:42, 2.02s/it] 9%|▉ | 682/7500 [22:53<3:51:15, 2.04s/it] 9%|▉ | 683/7500 [22:55<3:48:34, 2.01s/it] 9%|▉ | 684/7500 [22:57<3:51:14, 2.04s/it] 9%|▉ | 685/7500 [22:59<3:52:27, 2.05s/it] 9%|▉ | 686/7500 [23:01<3:49:22, 2.02s/it] 9%|▉ | 687/7500 [23:03<3:48:10, 2.01s/it] 9%|▉ | 688/7500 [23:05<3:44:09, 1.97s/it] 9%|▉ | 689/7500 [23:07<3:45:30, 1.99s/it] 9%|▉ | 690/7500 [23:09<3:46:44, 2.00s/it] {'loss': 0.52, 'grad_norm': 5.432327747344971, 'learning_rate': 4.600000000000001e-05, 'logqs/rejected': -2.6325345039367676, 'logqs/chosen': -0.9581176042556763, 'logqs_logps/rejected': -0.020027101039886475, 'logps_logqs/chosen': 0.25393056869506836, 'epoch': 0.18} 9%|▉ | 690/7500 [23:09<3:46:44, 2.00s/it] 9%|▉ | 691/7500 [23:11<3:45:35, 1.99s/it] 9%|▉ | 692/7500 [23:13<3:43:59, 1.97s/it] 9%|▉ | 693/7500 [23:15<3:46:09, 1.99s/it] 9%|▉ | 694/7500 [23:17<3:46:57, 2.00s/it] 9%|▉ | 695/7500 [23:19<3:51:16, 2.04s/it] 9%|▉ | 696/7500 [23:21<3:47:06, 2.00s/it] 9%|▉ | 697/7500 [23:23<3:47:30, 2.01s/it] 9%|▉ | 698/7500 [23:25<3:45:55, 1.99s/it] 9%|▉ | 699/7500 [23:27<3:46:08, 2.00s/it] 9%|▉ | 700/7500 [23:29<3:44:49, 1.98s/it] {'loss': 0.5063, 'grad_norm': 5.698807239532471, 'learning_rate': 4.666666666666667e-05, 'logqs/rejected': -2.4756264686584473, 'logqs/chosen': -0.9614144563674927, 'logqs_logps/rejected': -0.02961517870426178, 'logps_logqs/chosen': 0.2610984444618225, 'epoch': 0.19} 9%|▉ | 700/7500 [23:29<3:44:49, 1.98s/it] 9%|▉ | 701/7500 [23:31<3:44:01, 1.98s/it] 9%|▉ | 702/7500 [23:33<3:45:12, 1.99s/it] 9%|▉ | 703/7500 [23:35<3:49:55, 2.03s/it] 9%|▉ | 704/7500 [23:37<3:46:25, 2.00s/it] 9%|▉ | 705/7500 [23:39<3:48:26, 2.02s/it] 9%|▉ | 706/7500 [23:41<3:47:58, 2.01s/it] 9%|▉ | 707/7500 [23:43<3:48:52, 2.02s/it] 9%|▉ | 708/7500 [23:45<3:50:09, 2.03s/it] 9%|▉ | 709/7500 [23:47<3:49:31, 2.03s/it] 9%|▉ | 710/7500 [23:49<3:46:24, 2.00s/it] {'loss': 0.5317, 'grad_norm': 8.41701602935791, 'learning_rate': 4.7333333333333336e-05, 'logqs/rejected': -2.5764524936676025, 'logqs/chosen': -1.0026452541351318, 'logqs_logps/rejected': -0.022800911217927933, 'logps_logqs/chosen': 0.259687215089798, 'epoch': 0.19} 9%|▉ | 710/7500 [23:49<3:46:24, 2.00s/it] 9%|▉ | 711/7500 [23:51<3:49:02, 2.02s/it] 9%|▉ | 712/7500 [23:53<3:48:33, 2.02s/it] 10%|▉ | 713/7500 [23:55<3:47:59, 2.02s/it] 10%|▉ | 714/7500 [23:57<3:50:49, 2.04s/it] 10%|▉ | 715/7500 [23:59<3:46:47, 2.01s/it] 10%|▉ | 716/7500 [24:01<3:42:44, 1.97s/it] 10%|▉ | 717/7500 [24:03<3:44:27, 1.99s/it] 10%|▉ | 718/7500 [24:05<3:44:55, 1.99s/it] 10%|▉ | 719/7500 [24:07<3:45:16, 1.99s/it] 10%|▉ | 720/7500 [24:09<3:40:21, 1.95s/it] {'loss': 0.5077, 'grad_norm': 5.242297172546387, 'learning_rate': 4.8e-05, 'logqs/rejected': -2.4789981842041016, 'logqs/chosen': -0.9662948846817017, 'logqs_logps/rejected': -0.02537614107131958, 'logps_logqs/chosen': 0.2557288110256195, 'epoch': 0.19} 10%|▉ | 720/7500 [24:09<3:40:21, 1.95s/it] 10%|▉ | 721/7500 [24:11<3:39:58, 1.95s/it] 10%|▉ | 722/7500 [24:13<3:44:18, 1.99s/it] 10%|▉ | 723/7500 [24:15<3:45:32, 2.00s/it] 10%|▉ | 724/7500 [24:17<3:47:11, 2.01s/it] 10%|▉ | 725/7500 [24:19<3:50:03, 2.04s/it] 10%|▉ | 726/7500 [24:21<3:49:30, 2.03s/it] 10%|▉ | 727/7500 [24:23<3:48:42, 2.03s/it] 10%|▉ | 728/7500 [24:25<3:47:23, 2.01s/it] 10%|▉ | 729/7500 [24:27<3:45:40, 2.00s/it] 10%|▉ | 730/7500 [24:29<3:45:07, 2.00s/it] {'loss': 0.5018, 'grad_norm': 5.256946086883545, 'learning_rate': 4.866666666666667e-05, 'logqs/rejected': -2.4821572303771973, 'logqs/chosen': -0.9727791547775269, 'logqs_logps/rejected': -0.031911205500364304, 'logps_logqs/chosen': 0.25584545731544495, 'epoch': 0.19} 10%|▉ | 730/7500 [24:29<3:45:07, 2.00s/it] 10%|▉ | 731/7500 [24:31<3:44:19, 1.99s/it] 10%|▉ | 732/7500 [24:33<3:45:07, 2.00s/it] 10%|▉ | 733/7500 [24:35<3:45:14, 2.00s/it] 10%|▉ | 734/7500 [24:37<3:45:51, 2.00s/it] 10%|▉ | 735/7500 [24:39<3:47:41, 2.02s/it] 10%|▉ | 736/7500 [24:41<3:52:47, 2.07s/it] 10%|▉ | 737/7500 [24:43<3:50:00, 2.04s/it] 10%|▉ | 738/7500 [24:45<3:47:57, 2.02s/it] 10%|▉ | 739/7500 [24:47<3:50:24, 2.04s/it] 10%|▉ | 740/7500 [24:49<3:47:55, 2.02s/it] {'loss': 0.506, 'grad_norm': 5.687158107757568, 'learning_rate': 4.933333333333334e-05, 'logqs/rejected': -2.4834725856781006, 'logqs/chosen': -0.971139132976532, 'logqs_logps/rejected': -0.025059008970856667, 'logps_logqs/chosen': 0.25084567070007324, 'epoch': 0.2} 10%|▉ | 740/7500 [24:49<3:47:55, 2.02s/it] 10%|▉ | 741/7500 [24:51<3:50:10, 2.04s/it] 10%|▉ | 742/7500 [24:53<3:47:41, 2.02s/it] 10%|▉ | 743/7500 [24:56<3:48:34, 2.03s/it] 10%|▉ | 744/7500 [24:57<3:47:22, 2.02s/it] 10%|▉ | 745/7500 [25:00<3:47:01, 2.02s/it] 10%|▉ | 746/7500 [25:02<3:47:29, 2.02s/it] 10%|▉ | 747/7500 [25:04<3:45:46, 2.01s/it] 10%|▉ | 748/7500 [25:06<3:45:44, 2.01s/it] 10%|▉ | 749/7500 [25:08<3:45:03, 2.00s/it] 10%|█ | 750/7500 [25:09<3:43:47, 1.99s/it] {'loss': 0.5471, 'grad_norm': 5.235246658325195, 'learning_rate': 5e-05, 'logqs/rejected': -2.440889596939087, 'logqs/chosen': -0.9909852743148804, 'logqs_logps/rejected': -0.015185326337814331, 'logps_logqs/chosen': 0.26256290078163147, 'epoch': 0.2} 10%|█ | 750/7500 [25:09<3:43:47, 1.99s/it] 10%|█ | 751/7500 [25:12<3:45:53, 2.01s/it] 10%|█ | 752/7500 [25:13<3:42:51, 1.98s/it] 10%|█ | 753/7500 [25:15<3:41:48, 1.97s/it] 10%|█ | 754/7500 [25:17<3:41:29, 1.97s/it] 10%|█ | 755/7500 [25:19<3:44:35, 2.00s/it] 10%|█ | 756/7500 [25:21<3:45:09, 2.00s/it] 10%|█ | 757/7500 [25:23<3:42:03, 1.98s/it] 10%|█ | 758/7500 [25:25<3:41:27, 1.97s/it] 10%|█ | 759/7500 [25:27<3:42:40, 1.98s/it] 10%|█ | 760/7500 [25:29<3:42:41, 1.98s/it] {'loss': 0.5303, 'grad_norm': 4.726527690887451, 'learning_rate': 4.9999729229448975e-05, 'logqs/rejected': -2.5554351806640625, 'logqs/chosen': -0.940621554851532, 'logqs_logps/rejected': -0.019121132791042328, 'logps_logqs/chosen': 0.25315937399864197, 'epoch': 0.2} 10%|█ | 760/7500 [25:29<3:42:41, 1.98s/it] 10%|█ | 761/7500 [25:31<3:43:48, 1.99s/it] 10%|█ | 762/7500 [25:33<3:45:15, 2.01s/it] 10%|█ | 763/7500 [25:35<3:46:42, 2.02s/it] 10%|█ | 764/7500 [25:37<3:46:31, 2.02s/it] 10%|█ | 765/7500 [25:39<3:42:55, 1.99s/it] 10%|█ | 766/7500 [25:41<3:43:03, 1.99s/it] 10%|█ | 767/7500 [25:43<3:41:25, 1.97s/it] 10%|█ | 768/7500 [25:45<3:43:00, 1.99s/it] 10%|█ | 769/7500 [25:47<3:47:37, 2.03s/it] 10%|█ | 770/7500 [25:50<3:49:52, 2.05s/it] {'loss': 0.5329, 'grad_norm': 4.940152168273926, 'learning_rate': 4.999891692366121e-05, 'logqs/rejected': -2.652617931365967, 'logqs/chosen': -0.974120020866394, 'logqs_logps/rejected': -0.026729360222816467, 'logps_logqs/chosen': 0.24348381161689758, 'epoch': 0.21} 10%|█ | 770/7500 [25:50<3:49:52, 2.05s/it] 10%|█ | 771/7500 [25:52<3:51:14, 2.06s/it] 10%|█ | 772/7500 [25:54<3:50:25, 2.05s/it] 10%|█ | 773/7500 [25:56<3:50:05, 2.05s/it] 10%|█ | 774/7500 [25:58<3:47:07, 2.03s/it] 10%|█ | 775/7500 [26:00<3:44:23, 2.00s/it] 10%|█ | 776/7500 [26:02<3:45:44, 2.01s/it] 10%|█ | 777/7500 [26:04<3:43:07, 1.99s/it] 10%|█ | 778/7500 [26:06<3:42:59, 1.99s/it] 10%|█ | 779/7500 [26:08<3:47:19, 2.03s/it] 10%|█ | 780/7500 [26:10<3:51:17, 2.07s/it] {'loss': 0.5244, 'grad_norm': 5.3875250816345215, 'learning_rate': 4.999756310023261e-05, 'logqs/rejected': -2.404883623123169, 'logqs/chosen': -0.9764770269393921, 'logqs_logps/rejected': -0.02283012866973877, 'logps_logqs/chosen': 0.2520061135292053, 'epoch': 0.21} 10%|█ | 780/7500 [26:10<3:51:17, 2.07s/it] 10%|█ | 781/7500 [26:12<3:48:53, 2.04s/it] 10%|█ | 782/7500 [26:14<3:47:57, 2.04s/it] 10%|█ | 783/7500 [26:16<3:47:25, 2.03s/it] 10%|█ | 784/7500 [26:18<3:46:19, 2.02s/it] 10%|█ | 785/7500 [26:20<3:44:59, 2.01s/it] 10%|█ | 786/7500 [26:22<3:45:27, 2.01s/it] 10%|█ | 787/7500 [26:24<3:43:49, 2.00s/it] 11%|█ | 788/7500 [26:26<3:43:46, 2.00s/it] 11%|█ | 789/7500 [26:28<3:43:29, 2.00s/it] 11%|█ | 790/7500 [26:30<3:44:23, 2.01s/it] {'loss': 0.5039, 'grad_norm': 4.74855375289917, 'learning_rate': 4.99956677884892e-05, 'logqs/rejected': -2.3586623668670654, 'logqs/chosen': -0.9597617983818054, 'logqs_logps/rejected': -0.03150682896375656, 'logps_logqs/chosen': 0.250389963388443, 'epoch': 0.21} 11%|█ | 790/7500 [26:30<3:44:23, 2.01s/it] 11%|█ | 791/7500 [26:32<3:47:02, 2.03s/it] 11%|█ | 792/7500 [26:34<3:45:52, 2.02s/it] 11%|█ | 793/7500 [26:36<3:43:33, 2.00s/it] 11%|█ | 794/7500 [26:38<3:46:01, 2.02s/it] 11%|█ | 795/7500 [26:40<3:47:47, 2.04s/it] 11%|█ | 796/7500 [26:42<3:52:10, 2.08s/it] 11%|█ | 797/7500 [26:44<3:50:30, 2.06s/it] 11%|█ | 798/7500 [26:46<3:50:40, 2.07s/it] 11%|█ | 799/7500 [26:48<3:50:51, 2.07s/it] 11%|█ | 800/7500 [26:50<3:51:56, 2.08s/it] {'loss': 0.5186, 'grad_norm': 4.728780269622803, 'learning_rate': 4.9993231029486544e-05, 'logqs/rejected': -2.5363807678222656, 'logqs/chosen': -0.9671217799186707, 'logqs_logps/rejected': -0.030751556158065796, 'logps_logqs/chosen': 0.25782355666160583, 'epoch': 0.21} 11%|█ | 800/7500 [26:51<3:51:56, 2.08s/it] 11%|█ | 801/7500 [26:53<3:50:52, 2.07s/it] 11%|█ | 802/7500 [26:55<3:49:04, 2.05s/it] 11%|█ | 803/7500 [26:57<3:48:20, 2.05s/it] 11%|█ | 804/7500 [26:59<3:47:02, 2.03s/it] 11%|█ | 805/7500 [27:01<3:44:53, 2.02s/it] 11%|█ | 806/7500 [27:03<3:45:15, 2.02s/it] 11%|█ | 807/7500 [27:05<3:43:22, 2.00s/it] 11%|█ | 808/7500 [27:07<3:42:51, 2.00s/it] 11%|█ | 809/7500 [27:09<3:45:39, 2.02s/it] 11%|█ | 810/7500 [27:11<3:46:32, 2.03s/it] {'loss': 0.5018, 'grad_norm': 4.701254844665527, 'learning_rate': 4.999025287600886e-05, 'logqs/rejected': -2.4756553173065186, 'logqs/chosen': -0.9686040878295898, 'logqs_logps/rejected': -0.028573399409651756, 'logps_logqs/chosen': 0.25182801485061646, 'epoch': 0.22} 11%|█ | 810/7500 [27:11<3:46:32, 2.03s/it] 11%|█ | 811/7500 [27:13<3:48:45, 2.05s/it] 11%|█ | 812/7500 [27:15<3:45:17, 2.02s/it] 11%|█ | 813/7500 [27:17<3:43:00, 2.00s/it] 11%|█ | 814/7500 [27:19<3:43:06, 2.00s/it] 11%|█ | 815/7500 [27:21<3:42:55, 2.00s/it] 11%|█ | 816/7500 [27:23<3:42:49, 2.00s/it] 11%|█ | 817/7500 [27:25<3:42:00, 1.99s/it] 11%|█ | 818/7500 [27:27<3:42:35, 2.00s/it] 11%|█ | 819/7500 [27:29<3:41:39, 1.99s/it] 11%|█ | 820/7500 [27:31<3:40:39, 1.98s/it] {'loss': 0.5292, 'grad_norm': 4.170595169067383, 'learning_rate': 4.9986733392567845e-05, 'logqs/rejected': -2.5270917415618896, 'logqs/chosen': -0.9572726488113403, 'logqs_logps/rejected': -0.02618049643933773, 'logps_logqs/chosen': 0.2554892897605896, 'epoch': 0.22} 11%|█ | 820/7500 [27:31<3:40:39, 1.98s/it] 11%|█ | 821/7500 [27:33<3:42:44, 2.00s/it] 11%|█ | 822/7500 [27:35<3:45:09, 2.02s/it] 11%|█ | 823/7500 [27:37<3:46:43, 2.04s/it] 11%|█ | 824/7500 [27:39<3:45:04, 2.02s/it] 11%|█ | 825/7500 [27:41<3:44:05, 2.01s/it] 11%|█ | 826/7500 [27:43<3:44:10, 2.02s/it] 11%|█ | 827/7500 [27:45<3:41:48, 1.99s/it] 11%|█ | 828/7500 [27:47<3:42:33, 2.00s/it] 11%|█ | 829/7500 [27:49<3:46:15, 2.03s/it] 11%|█ | 830/7500 [27:51<3:44:45, 2.02s/it] {'loss': 0.5127, 'grad_norm': 5.008883953094482, 'learning_rate': 4.99826726554013e-05, 'logqs/rejected': -2.3686013221740723, 'logqs/chosen': -0.945959746837616, 'logqs_logps/rejected': -0.02595512941479683, 'logps_logqs/chosen': 0.2581559121608734, 'epoch': 0.22} 11%|█ | 830/7500 [27:51<3:44:45, 2.02s/it] 11%|█ | 831/7500 [27:53<3:44:20, 2.02s/it] 11%|█ | 832/7500 [27:55<3:47:32, 2.05s/it] 11%|█ | 833/7500 [27:57<3:46:42, 2.04s/it] 11%|█ | 834/7500 [27:59<3:45:25, 2.03s/it] 11%|█ | 835/7500 [28:01<3:43:53, 2.02s/it] 11%|█ | 836/7500 [28:03<3:46:31, 2.04s/it] 11%|█ | 837/7500 [28:05<3:45:42, 2.03s/it] 11%|█ | 838/7500 [28:07<3:45:32, 2.03s/it] 11%|█ | 839/7500 [28:09<3:42:08, 2.00s/it] 11%|█ | 840/7500 [28:11<3:40:46, 1.99s/it] {'loss': 0.5408, 'grad_norm': 5.366133213043213, 'learning_rate': 4.997807075247146e-05, 'logqs/rejected': -2.7230758666992188, 'logqs/chosen': -0.987845778465271, 'logqs_logps/rejected': -0.022608840838074684, 'logps_logqs/chosen': 0.2608697712421417, 'epoch': 0.22} 11%|█ | 840/7500 [28:11<3:40:46, 1.99s/it] 11%|█ | 841/7500 [28:13<3:41:43, 2.00s/it] 11%|█ | 842/7500 [28:15<3:42:20, 2.00s/it] 11%|█ | 843/7500 [28:17<3:45:27, 2.03s/it] 11%|█▏ | 844/7500 [28:19<3:46:37, 2.04s/it] 11%|█▏ | 845/7500 [28:21<3:44:53, 2.03s/it] 11%|█▏ | 846/7500 [28:23<3:41:26, 2.00s/it] 11%|█▏ | 847/7500 [28:25<3:41:02, 1.99s/it] 11%|█▏ | 848/7500 [28:27<3:45:25, 2.03s/it] 11%|█▏ | 849/7500 [28:29<3:46:03, 2.04s/it] 11%|█▏ | 850/7500 [28:31<3:46:40, 2.05s/it] {'loss': 0.4942, 'grad_norm': 5.360482692718506, 'learning_rate': 4.997292778346312e-05, 'logqs/rejected': -2.3144824504852295, 'logqs/chosen': -0.9506081342697144, 'logqs_logps/rejected': -0.03057628870010376, 'logps_logqs/chosen': 0.25519242882728577, 'epoch': 0.23} 11%|█▏ | 850/7500 [28:31<3:46:40, 2.05s/it] 11%|█▏ | 851/7500 [28:33<3:44:42, 2.03s/it] 11%|█▏ | 852/7500 [28:35<3:46:34, 2.04s/it] 11%|█▏ | 853/7500 [28:37<3:44:24, 2.03s/it] 11%|█▏ | 854/7500 [28:39<3:45:29, 2.04s/it] 11%|█▏ | 855/7500 [28:41<3:42:41, 2.01s/it] 11%|█▏ | 856/7500 [28:43<3:44:06, 2.02s/it] 11%|█▏ | 857/7500 [28:45<3:41:43, 2.00s/it] 11%|█▏ | 858/7500 [28:47<3:42:25, 2.01s/it] 11%|█▏ | 859/7500 [28:49<3:43:07, 2.02s/it] 11%|█▏ | 860/7500 [28:52<3:45:26, 2.04s/it] {'loss': 0.525, 'grad_norm': 4.413522720336914, 'learning_rate': 4.9967243859781426e-05, 'logqs/rejected': -2.5740087032318115, 'logqs/chosen': -1.0012372732162476, 'logqs_logps/rejected': -0.026500528678297997, 'logps_logqs/chosen': 0.26457005739212036, 'epoch': 0.23} 11%|█▏ | 860/7500 [28:52<3:45:26, 2.04s/it] 11%|█▏ | 861/7500 [28:54<3:48:44, 2.07s/it] 11%|█▏ | 862/7500 [28:56<3:51:00, 2.09s/it] 12%|█▏ | 863/7500 [28:58<3:50:18, 2.08s/it] 12%|█▏ | 864/7500 [29:00<3:50:47, 2.09s/it] 12%|█▏ | 865/7500 [29:02<3:47:48, 2.06s/it] 12%|█▏ | 866/7500 [29:04<3:50:18, 2.08s/it] 12%|█▏ | 867/7500 [29:06<3:50:07, 2.08s/it] 12%|█▏ | 868/7500 [29:08<3:50:10, 2.08s/it] 12%|█▏ | 869/7500 [29:10<3:47:22, 2.06s/it] 12%|█▏ | 870/7500 [29:12<3:45:46, 2.04s/it] {'loss': 0.5352, 'grad_norm': 5.073977947235107, 'learning_rate': 4.996101910454953e-05, 'logqs/rejected': -2.472970485687256, 'logqs/chosen': -1.000656247138977, 'logqs_logps/rejected': -0.024903636425733566, 'logps_logqs/chosen': 0.26704609394073486, 'epoch': 0.23} 12%|█▏ | 870/7500 [29:12<3:45:46, 2.04s/it] 12%|█▏ | 871/7500 [29:14<3:45:52, 2.04s/it] 12%|█▏ | 872/7500 [29:16<3:46:06, 2.05s/it] 12%|█▏ | 873/7500 [29:18<3:44:59, 2.04s/it] 12%|█▏ | 874/7500 [29:20<3:41:41, 2.01s/it] 12%|█▏ | 875/7500 [29:22<3:41:49, 2.01s/it] 12%|█▏ | 876/7500 [29:24<3:38:02, 1.98s/it] 12%|█▏ | 877/7500 [29:26<3:38:47, 1.98s/it] 12%|█▏ | 878/7500 [29:28<3:40:25, 2.00s/it] 12%|█▏ | 879/7500 [29:30<3:40:56, 2.00s/it] 12%|█▏ | 880/7500 [29:32<3:44:43, 2.04s/it] {'loss': 0.5101, 'grad_norm': 4.2346930503845215, 'learning_rate': 4.995425365260585e-05, 'logqs/rejected': -2.420454502105713, 'logqs/chosen': -0.9621036648750305, 'logqs_logps/rejected': -0.031403034925460815, 'logps_logqs/chosen': 0.2556193470954895, 'epoch': 0.23} 12%|█▏ | 880/7500 [29:32<3:44:43, 2.04s/it] 12%|█▏ | 881/7500 [29:34<3:43:32, 2.03s/it] 12%|█▏ | 882/7500 [29:36<3:42:23, 2.02s/it] 12%|█▏ | 883/7500 [29:38<3:40:21, 2.00s/it] 12%|█▏ | 884/7500 [29:40<3:41:28, 2.01s/it] 12%|█▏ | 885/7500 [29:42<3:41:53, 2.01s/it] 12%|█▏ | 886/7500 [29:44<3:42:24, 2.02s/it] 12%|█▏ | 887/7500 [29:46<3:42:05, 2.02s/it] 12%|█▏ | 888/7500 [29:49<3:44:53, 2.04s/it] 12%|█▏ | 889/7500 [29:51<3:46:42, 2.06s/it] 12%|█▏ | 890/7500 [29:53<3:45:00, 2.04s/it] {'loss': 0.5239, 'grad_norm': 4.438549518585205, 'learning_rate': 4.99469476505012e-05, 'logqs/rejected': -2.3944032192230225, 'logqs/chosen': -0.9634441137313843, 'logqs_logps/rejected': -0.02586778998374939, 'logps_logqs/chosen': 0.2566954493522644, 'epoch': 0.24} 12%|█▏ | 890/7500 [29:53<3:45:00, 2.04s/it] 12%|█▏ | 891/7500 [29:55<3:42:00, 2.02s/it] 12%|█▏ | 892/7500 [29:57<3:46:22, 2.06s/it] 12%|█▏ | 893/7500 [29:59<3:51:47, 2.10s/it] 12%|█▏ | 894/7500 [30:01<3:46:19, 2.06s/it] 12%|█▏ | 895/7500 [30:03<3:46:07, 2.05s/it] 12%|█▏ | 896/7500 [30:05<3:45:17, 2.05s/it] 12%|█▏ | 897/7500 [30:07<3:45:49, 2.05s/it] 12%|█▏ | 898/7500 [30:09<3:43:44, 2.03s/it] 12%|█▏ | 899/7500 [30:11<3:44:09, 2.04s/it] 12%|█▏ | 900/7500 [30:13<3:40:56, 2.01s/it] {'loss': 0.5407, 'grad_norm': 4.183055877685547, 'learning_rate': 4.993910125649561e-05, 'logqs/rejected': -2.5792202949523926, 'logqs/chosen': -0.9902603030204773, 'logqs_logps/rejected': -0.013403053395450115, 'logps_logqs/chosen': 0.26409000158309937, 'epoch': 0.24} 12%|█▏ | 900/7500 [30:13<3:40:56, 2.01s/it] 12%|█▏ | 901/7500 [30:15<3:44:37, 2.04s/it] 12%|█▏ | 902/7500 [30:17<3:46:18, 2.06s/it] 12%|█▏ | 903/7500 [30:19<3:44:10, 2.04s/it] 12%|█▏ | 904/7500 [30:21<3:43:07, 2.03s/it] 12%|█▏ | 905/7500 [30:23<3:41:14, 2.01s/it] 12%|█▏ | 906/7500 [30:25<3:43:08, 2.03s/it] 12%|█▏ | 907/7500 [30:27<3:42:19, 2.02s/it] 12%|█▏ | 908/7500 [30:30<3:46:38, 2.06s/it] 12%|█▏ | 909/7500 [30:32<3:46:04, 2.06s/it] 12%|█▏ | 910/7500 [30:34<3:45:47, 2.06s/it] {'loss': 0.5144, 'grad_norm': 4.482913494110107, 'learning_rate': 4.993071464055486e-05, 'logqs/rejected': -2.304330587387085, 'logqs/chosen': -0.9716569185256958, 'logqs_logps/rejected': -0.03334904462099075, 'logps_logqs/chosen': 0.24799712002277374, 'epoch': 0.24} 12%|█▏ | 910/7500 [30:34<3:45:47, 2.06s/it] 12%|█▏ | 911/7500 [30:36<3:45:03, 2.05s/it] 12%|█▏ | 912/7500 [30:38<3:44:56, 2.05s/it] 12%|█▏ | 913/7500 [30:40<3:42:44, 2.03s/it] 12%|█▏ | 914/7500 [30:42<3:41:51, 2.02s/it] 12%|█▏ | 915/7500 [30:44<3:40:09, 2.01s/it] 12%|█▏ | 916/7500 [30:46<3:43:20, 2.04s/it] 12%|█▏ | 917/7500 [30:48<3:42:11, 2.03s/it] 12%|█▏ | 918/7500 [30:50<3:39:39, 2.00s/it] 12%|█▏ | 919/7500 [30:52<3:39:05, 2.00s/it] 12%|█▏ | 920/7500 [30:54<3:41:40, 2.02s/it] {'loss': 0.5241, 'grad_norm': 14.411430358886719, 'learning_rate': 4.9921787984346846e-05, 'logqs/rejected': -2.4542269706726074, 'logqs/chosen': -0.9805234670639038, 'logqs_logps/rejected': -0.02323959395289421, 'logps_logqs/chosen': 0.2593521475791931, 'epoch': 0.25} 12%|█▏ | 920/7500 [30:54<3:41:40, 2.02s/it] 12%|█▏ | 921/7500 [30:56<3:41:57, 2.02s/it] 12%|█▏ | 922/7500 [30:58<3:44:42, 2.05s/it] 12%|█▏ | 923/7500 [31:00<3:44:24, 2.05s/it] 12%|█▏ | 924/7500 [31:02<3:43:51, 2.04s/it] 12%|█▏ | 925/7500 [31:04<3:42:06, 2.03s/it] 12%|█▏ | 926/7500 [31:06<3:41:00, 2.02s/it] 12%|█▏ | 927/7500 [31:08<3:40:08, 2.01s/it] 12%|█▏ | 928/7500 [31:10<3:38:24, 1.99s/it] 12%|█▏ | 929/7500 [31:12<3:37:05, 1.98s/it] 12%|█▏ | 930/7500 [31:14<3:37:09, 1.98s/it] {'loss': 0.5183, 'grad_norm': 5.01200008392334, 'learning_rate': 4.991232148123761e-05, 'logqs/rejected': -2.3986029624938965, 'logqs/chosen': -0.9969019889831543, 'logqs_logps/rejected': -0.026479806751012802, 'logps_logqs/chosen': 0.2612362504005432, 'epoch': 0.25} 12%|█▏ | 930/7500 [31:14<3:37:09, 1.98s/it] 12%|█▏ | 931/7500 [31:16<3:36:19, 1.98s/it] 12%|█▏ | 932/7500 [31:18<3:37:22, 1.99s/it] 12%|█▏ | 933/7500 [31:20<3:35:24, 1.97s/it] 12%|█▏ | 934/7500 [31:22<3:37:57, 1.99s/it] 12%|█▏ | 935/7500 [31:24<3:35:48, 1.97s/it] 12%|█▏ | 936/7500 [31:26<3:37:16, 1.99s/it] 12%|█▏ | 937/7500 [31:28<3:39:33, 2.01s/it] 13%|█▎ | 938/7500 [31:30<3:37:40, 1.99s/it] 13%|█▎ | 939/7500 [31:32<3:38:11, 2.00s/it] 13%|█▎ | 940/7500 [31:34<3:37:13, 1.99s/it] {'loss': 0.5088, 'grad_norm': 4.572302341461182, 'learning_rate': 4.9902315336287184e-05, 'logqs/rejected': -2.5605194568634033, 'logqs/chosen': -0.9375201463699341, 'logqs_logps/rejected': -0.02546473778784275, 'logps_logqs/chosen': 0.2526441514492035, 'epoch': 0.25} 13%|█▎ | 940/7500 [31:34<3:37:13, 1.99s/it] 13%|█▎ | 941/7500 [31:36<3:37:43, 1.99s/it] 13%|█▎ | 942/7500 [31:38<3:36:02, 1.98s/it] 13%|█▎ | 943/7500 [31:40<3:39:07, 2.01s/it] 13%|█▎ | 944/7500 [31:42<3:38:48, 2.00s/it] 13%|█▎ | 945/7500 [31:44<3:39:42, 2.01s/it] 13%|█▎ | 946/7500 [31:46<3:41:38, 2.03s/it] 13%|█▎ | 947/7500 [31:48<3:42:29, 2.04s/it] 13%|█▎ | 948/7500 [31:50<3:41:12, 2.03s/it] 13%|█▎ | 949/7500 [31:52<3:46:05, 2.07s/it] 13%|█▎ | 950/7500 [31:54<3:44:07, 2.05s/it] {'loss': 0.5305, 'grad_norm': 4.215644359588623, 'learning_rate': 4.989176976624511e-05, 'logqs/rejected': -2.4783594608306885, 'logqs/chosen': -0.9846636652946472, 'logqs_logps/rejected': -0.02721923217177391, 'logps_logqs/chosen': 0.2503536641597748, 'epoch': 0.25} 13%|█▎ | 950/7500 [31:54<3:44:07, 2.05s/it] 13%|█▎ | 951/7500 [31:56<3:41:26, 2.03s/it] 13%|█▎ | 952/7500 [31:58<3:43:36, 2.05s/it] 13%|█▎ | 953/7500 [32:00<3:43:18, 2.05s/it] 13%|█▎ | 954/7500 [32:02<3:40:21, 2.02s/it] 13%|█▎ | 955/7500 [32:04<3:37:04, 1.99s/it] 13%|█▎ | 956/7500 [32:06<3:35:49, 1.98s/it] 13%|█▎ | 957/7500 [32:08<3:34:58, 1.97s/it] 13%|█▎ | 958/7500 [32:10<3:36:47, 1.99s/it] 13%|█▎ | 959/7500 [32:12<3:39:28, 2.01s/it] 13%|█▎ | 960/7500 [32:14<3:41:08, 2.03s/it] {'loss': 0.5487, 'grad_norm': 4.5285773277282715, 'learning_rate': 4.988068499954578e-05, 'logqs/rejected': -2.687187433242798, 'logqs/chosen': -1.0025092363357544, 'logqs_logps/rejected': -0.016850614920258522, 'logps_logqs/chosen': 0.25861120223999023, 'epoch': 0.26} 13%|█▎ | 960/7500 [32:14<3:41:08, 2.03s/it] 13%|█▎ | 961/7500 [32:16<3:42:54, 2.05s/it] 13%|█▎ | 962/7500 [32:18<3:38:52, 2.01s/it] 13%|█▎ | 963/7500 [32:20<3:36:33, 1.99s/it] 13%|█▎ | 964/7500 [32:22<3:36:18, 1.99s/it] 13%|█▎ | 965/7500 [32:24<3:38:32, 2.01s/it] 13%|█▎ | 966/7500 [32:26<3:39:53, 2.02s/it] 13%|█▎ | 967/7500 [32:28<3:39:51, 2.02s/it] 13%|█▎ | 968/7500 [32:30<3:39:09, 2.01s/it] 13%|█▎ | 969/7500 [32:32<3:39:02, 2.01s/it] 13%|█▎ | 970/7500 [32:34<3:39:38, 2.02s/it] {'loss': 0.4981, 'grad_norm': 4.532316207885742, 'learning_rate': 4.986906127630346e-05, 'logqs/rejected': -2.4823899269104004, 'logqs/chosen': -0.9227617979049683, 'logqs_logps/rejected': -0.03138008713722229, 'logps_logqs/chosen': 0.2465834617614746, 'epoch': 0.26} 13%|█▎ | 970/7500 [32:34<3:39:38, 2.02s/it] 13%|█▎ | 971/7500 [32:36<3:40:04, 2.02s/it] 13%|█▎ | 972/7500 [32:38<3:39:23, 2.02s/it] 13%|█▎ | 973/7500 [32:40<3:35:44, 1.98s/it] 13%|█▎ | 974/7500 [32:42<3:35:22, 1.98s/it] 13%|█▎ | 975/7500 [32:44<3:36:52, 1.99s/it] 13%|█▎ | 976/7500 [32:46<3:37:10, 2.00s/it] 13%|█▎ | 977/7500 [32:48<3:36:00, 1.99s/it] 13%|█▎ | 978/7500 [32:50<3:37:07, 2.00s/it] 13%|█▎ | 979/7500 [32:52<3:39:09, 2.02s/it] 13%|█▎ | 980/7500 [32:54<3:36:59, 2.00s/it] {'loss': 0.548, 'grad_norm': 4.927963733673096, 'learning_rate': 4.985689884830711e-05, 'logqs/rejected': -2.5769779682159424, 'logqs/chosen': -0.9907986521720886, 'logqs_logps/rejected': -0.015555794350802898, 'logps_logqs/chosen': 0.25285014510154724, 'epoch': 0.26} 13%|█▎ | 980/7500 [32:54<3:36:59, 2.00s/it] 13%|█▎ | 981/7500 [32:56<3:39:28, 2.02s/it] 13%|█▎ | 982/7500 [32:58<3:37:29, 2.00s/it] 13%|█▎ | 983/7500 [33:00<3:38:51, 2.02s/it] 13%|█▎ | 984/7500 [33:02<3:41:14, 2.04s/it] 13%|█▎ | 985/7500 [33:04<3:40:41, 2.03s/it] 13%|█▎ | 986/7500 [33:06<3:38:48, 2.02s/it] 13%|█▎ | 987/7500 [33:08<3:39:43, 2.02s/it] 13%|█▎ | 988/7500 [33:10<3:40:42, 2.03s/it] 13%|█▎ | 989/7500 [33:12<3:38:35, 2.01s/it] 13%|█▎ | 990/7500 [33:14<3:37:59, 2.01s/it] {'loss': 0.5197, 'grad_norm': 4.764709949493408, 'learning_rate': 4.984419797901491e-05, 'logqs/rejected': -2.567243814468384, 'logqs/chosen': -0.9609615206718445, 'logqs_logps/rejected': -0.021144839003682137, 'logps_logqs/chosen': 0.251412570476532, 'epoch': 0.26} 13%|█▎ | 990/7500 [33:14<3:37:59, 2.01s/it] 13%|█▎ | 991/7500 [33:16<3:38:25, 2.01s/it] 13%|█▎ | 992/7500 [33:18<3:37:14, 2.00s/it] 13%|█▎ | 993/7500 [33:20<3:39:31, 2.02s/it] 13%|█▎ | 994/7500 [33:22<3:37:27, 2.01s/it] 13%|█▎ | 995/7500 [33:24<3:35:50, 1.99s/it] 13%|█▎ | 996/7500 [33:26<3:36:28, 2.00s/it] 13%|█▎ | 997/7500 [33:28<3:36:13, 1.99s/it] 13%|█▎ | 998/7500 [33:30<3:33:59, 1.97s/it] 13%|█▎ | 999/7500 [33:32<3:35:00, 1.98s/it] 13%|█▎ | 1000/7500 [33:34<3:37:49, 2.01s/it] {'loss': 0.5137, 'grad_norm': 5.92892599105835, 'learning_rate': 4.983095894354858e-05, 'logqs/rejected': -2.50333309173584, 'logqs/chosen': -0.9540032148361206, 'logqs_logps/rejected': -0.02470395341515541, 'logps_logqs/chosen': 0.24469955265522003, 'epoch': 0.27} 13%|█▎ | 1000/7500 [33:34<3:37:49, 2.01s/it] 13%|█▎ | 1001/7500 [33:36<3:37:37, 2.01s/it] 13%|█▎ | 1002/7500 [33:38<3:36:58, 2.00s/it] 13%|█▎ | 1003/7500 [33:40<3:37:06, 2.00s/it] 13%|█▎ | 1004/7500 [33:42<3:38:16, 2.02s/it] 13%|█▎ | 1005/7500 [33:45<3:44:11, 2.07s/it] 13%|█▎ | 1006/7500 [33:47<3:44:13, 2.07s/it] 13%|█▎ | 1007/7500 [33:49<3:45:55, 2.09s/it] 13%|█▎ | 1008/7500 [33:51<3:47:50, 2.11s/it] 13%|█▎ | 1009/7500 [33:53<3:45:10, 2.08s/it] 13%|█▎ | 1010/7500 [33:55<3:45:29, 2.08s/it] {'loss': 0.5203, 'grad_norm': 5.529993534088135, 'learning_rate': 4.981718202868738e-05, 'logqs/rejected': -2.453449249267578, 'logqs/chosen': -0.9460012316703796, 'logqs_logps/rejected': -0.024447932839393616, 'logps_logqs/chosen': 0.2512446939945221, 'epoch': 0.27} 13%|█▎ | 1010/7500 [33:55<3:45:29, 2.08s/it] 13%|█▎ | 1011/7500 [33:57<3:41:39, 2.05s/it] 13%|█▎ | 1012/7500 [33:59<3:39:57, 2.03s/it] 14%|█▎ | 1013/7500 [34:01<3:40:20, 2.04s/it] 14%|█▎ | 1014/7500 [34:03<3:39:26, 2.03s/it] 14%|█▎ | 1015/7500 [34:05<3:37:27, 2.01s/it] 14%|█▎ | 1016/7500 [34:07<3:35:56, 2.00s/it] 14%|█▎ | 1017/7500 [34:09<3:37:22, 2.01s/it] 14%|█▎ | 1018/7500 [34:11<3:39:21, 2.03s/it] 14%|█▎ | 1019/7500 [34:13<3:37:26, 2.01s/it] 14%|█▎ | 1020/7500 [34:15<3:38:46, 2.03s/it] {'loss': 0.5234, 'grad_norm': 5.253599166870117, 'learning_rate': 4.980286753286195e-05, 'logqs/rejected': -2.5247695446014404, 'logqs/chosen': -0.9709250330924988, 'logqs_logps/rejected': -0.028199758380651474, 'logps_logqs/chosen': 0.2531903386116028, 'epoch': 0.27} 14%|█▎ | 1020/7500 [34:15<3:38:46, 2.03s/it] 14%|█▎ | 1021/7500 [34:17<3:37:00, 2.01s/it] 14%|█▎ | 1022/7500 [34:19<3:35:22, 1.99s/it] 14%|█▎ | 1023/7500 [34:21<3:36:37, 2.01s/it] 14%|█▎ | 1024/7500 [34:23<3:39:54, 2.04s/it] 14%|█▎ | 1025/7500 [34:25<3:38:39, 2.03s/it] 14%|█▎ | 1026/7500 [34:27<3:40:20, 2.04s/it] 14%|█▎ | 1027/7500 [34:29<3:39:45, 2.04s/it] 14%|█▎ | 1028/7500 [34:31<3:40:24, 2.04s/it] 14%|█▎ | 1029/7500 [34:33<3:40:08, 2.04s/it] 14%|█▎ | 1030/7500 [34:35<3:38:53, 2.03s/it] {'loss': 0.5306, 'grad_norm': 5.346230983734131, 'learning_rate': 4.978801576614779e-05, 'logqs/rejected': -2.607546329498291, 'logqs/chosen': -0.9797428846359253, 'logqs_logps/rejected': -0.02564539946615696, 'logps_logqs/chosen': 0.25783830881118774, 'epoch': 0.27} 14%|█▎ | 1030/7500 [34:36<3:38:53, 2.03s/it] 14%|█▎ | 1031/7500 [34:38<3:40:41, 2.05s/it] 14%|█▍ | 1032/7500 [34:40<3:40:42, 2.05s/it] 14%|█▍ | 1033/7500 [34:42<3:42:46, 2.07s/it] 14%|█▍ | 1034/7500 [34:44<3:39:07, 2.03s/it] 14%|█▍ | 1035/7500 [34:46<3:39:06, 2.03s/it] 14%|█▍ | 1036/7500 [34:48<3:35:04, 2.00s/it] 14%|█▍ | 1037/7500 [34:50<3:33:10, 1.98s/it] 14%|█▍ | 1038/7500 [34:52<3:35:58, 2.01s/it] 14%|█▍ | 1039/7500 [34:54<3:36:37, 2.01s/it] 14%|█▍ | 1040/7500 [34:56<3:36:09, 2.01s/it] {'loss': 0.5099, 'grad_norm': 4.752852439880371, 'learning_rate': 4.9772627050258604e-05, 'logqs/rejected': -2.5107645988464355, 'logqs/chosen': -0.9864835739135742, 'logqs_logps/rejected': -0.029949167743325233, 'logps_logqs/chosen': 0.2531307339668274, 'epoch': 0.28} 14%|█▍ | 1040/7500 [34:56<3:36:09, 2.01s/it] 14%|█▍ | 1041/7500 [34:58<3:38:04, 2.03s/it] 14%|█▍ | 1042/7500 [35:00<3:36:21, 2.01s/it] 14%|█▍ | 1043/7500 [35:02<3:43:19, 2.08s/it] 14%|█▍ | 1044/7500 [35:04<3:44:50, 2.09s/it] 14%|█▍ | 1045/7500 [35:06<3:43:04, 2.07s/it] 14%|█▍ | 1046/7500 [35:08<3:45:11, 2.09s/it] 14%|█▍ | 1047/7500 [35:10<3:41:49, 2.06s/it] 14%|█▍ | 1048/7500 [35:12<3:39:32, 2.04s/it] 14%|█▍ | 1049/7500 [35:14<3:38:59, 2.04s/it] 14%|█▍ | 1050/7500 [35:16<3:35:13, 2.00s/it] {'loss': 0.4914, 'grad_norm': 4.711596488952637, 'learning_rate': 4.975670171853926e-05, 'logqs/rejected': -2.3575196266174316, 'logqs/chosen': -0.9466425180435181, 'logqs_logps/rejected': -0.03228430449962616, 'logps_logqs/chosen': 0.25213682651519775, 'epoch': 0.28} 14%|█▍ | 1050/7500 [35:16<3:35:13, 2.00s/it] 14%|█▍ | 1051/7500 [35:18<3:35:54, 2.01s/it] 14%|█▍ | 1052/7500 [35:20<3:35:26, 2.00s/it] 14%|█▍ | 1053/7500 [35:22<3:36:27, 2.01s/it] 14%|█▍ | 1054/7500 [35:24<3:37:20, 2.02s/it] 14%|█▍ | 1055/7500 [35:26<3:35:41, 2.01s/it] 14%|█▍ | 1056/7500 [35:28<3:36:55, 2.02s/it] 14%|█▍ | 1057/7500 [35:30<3:35:06, 2.00s/it] 14%|█▍ | 1058/7500 [35:32<3:36:57, 2.02s/it] 14%|█▍ | 1059/7500 [35:34<3:37:17, 2.02s/it] 14%|█▍ | 1060/7500 [35:36<3:36:56, 2.02s/it] {'loss': 0.4997, 'grad_norm': 4.382611274719238, 'learning_rate': 4.974024011595864e-05, 'logqs/rejected': -2.3790011405944824, 'logqs/chosen': -0.9665301442146301, 'logqs_logps/rejected': -0.030110489577054977, 'logps_logqs/chosen': 0.2571648359298706, 'epoch': 0.28} 14%|█▍ | 1060/7500 [35:36<3:36:56, 2.02s/it] 14%|█▍ | 1061/7500 [35:38<3:38:32, 2.04s/it] 14%|█▍ | 1062/7500 [35:40<3:37:00, 2.02s/it] 14%|█▍ | 1063/7500 [35:43<3:39:44, 2.05s/it] 14%|█▍ | 1064/7500 [35:45<3:38:41, 2.04s/it] 14%|█▍ | 1065/7500 [35:47<3:39:35, 2.05s/it] 14%|█▍ | 1066/7500 [35:49<3:38:00, 2.03s/it] 14%|█▍ | 1067/7500 [35:51<3:35:51, 2.01s/it] 14%|█▍ | 1068/7500 [35:53<3:39:19, 2.05s/it] 14%|█▍ | 1069/7500 [35:55<3:41:18, 2.06s/it] 14%|█▍ | 1070/7500 [35:57<3:38:14, 2.04s/it] {'loss': 0.5089, 'grad_norm': 4.9012017250061035, 'learning_rate': 4.972324259910211e-05, 'logqs/rejected': -2.553769588470459, 'logqs/chosen': -0.9808646440505981, 'logqs_logps/rejected': -0.029565703123807907, 'logps_logqs/chosen': 0.2609332501888275, 'epoch': 0.29} 14%|█▍ | 1070/7500 [35:57<3:38:14, 2.04s/it] 14%|█▍ | 1071/7500 [35:59<3:37:22, 2.03s/it] 14%|█▍ | 1072/7500 [36:01<3:38:04, 2.04s/it] 14%|█▍ | 1073/7500 [36:03<3:37:13, 2.03s/it] 14%|█▍ | 1074/7500 [36:05<3:35:12, 2.01s/it] 14%|█▍ | 1075/7500 [36:07<3:33:47, 2.00s/it] 14%|█▍ | 1076/7500 [36:09<3:33:44, 2.00s/it] 14%|█▍ | 1077/7500 [36:11<3:34:33, 2.00s/it] 14%|█▍ | 1078/7500 [36:13<3:34:29, 2.00s/it] 14%|█▍ | 1079/7500 [36:15<3:33:39, 2.00s/it] 14%|█▍ | 1080/7500 [36:17<3:35:11, 2.01s/it] {'loss': 0.5088, 'grad_norm': 3.843731641769409, 'learning_rate': 4.9705709536163824e-05, 'logqs/rejected': -2.4854800701141357, 'logqs/chosen': -0.9487255215644836, 'logqs_logps/rejected': -0.026692980900406837, 'logps_logqs/chosen': 0.25069180130958557, 'epoch': 0.29} 14%|█▍ | 1080/7500 [36:17<3:35:11, 2.01s/it] 14%|█▍ | 1081/7500 [36:19<3:32:03, 1.98s/it] 14%|█▍ | 1082/7500 [36:21<3:39:06, 2.05s/it] 14%|█▍ | 1083/7500 [36:23<3:38:07, 2.04s/it] 14%|█▍ | 1084/7500 [36:25<3:37:33, 2.03s/it] 14%|█▍ | 1085/7500 [36:27<3:37:14, 2.03s/it] 14%|█▍ | 1086/7500 [36:29<3:35:27, 2.02s/it] 14%|█▍ | 1087/7500 [36:31<3:35:03, 2.01s/it] 15%|█▍ | 1088/7500 [36:33<3:34:23, 2.01s/it] 15%|█▍ | 1089/7500 [36:35<3:30:22, 1.97s/it] 15%|█▍ | 1090/7500 [36:37<3:32:19, 1.99s/it] {'loss': 0.5239, 'grad_norm': 4.645207405090332, 'learning_rate': 4.968764130693877e-05, 'logqs/rejected': -2.5752134323120117, 'logqs/chosen': -0.9651558995246887, 'logqs_logps/rejected': -0.0217286329716444, 'logps_logqs/chosen': 0.2509486973285675, 'epoch': 0.29} 15%|█▍ | 1090/7500 [36:37<3:32:19, 1.99s/it] 15%|█▍ | 1091/7500 [36:39<3:34:10, 2.01s/it] 15%|█▍ | 1092/7500 [36:41<3:33:16, 2.00s/it] 15%|█▍ | 1093/7500 [36:43<3:34:43, 2.01s/it] 15%|█▍ | 1094/7500 [36:45<3:33:51, 2.00s/it] 15%|█▍ | 1095/7500 [36:47<3:37:37, 2.04s/it] 15%|█▍ | 1096/7500 [36:49<3:38:09, 2.04s/it] 15%|█▍ | 1097/7500 [36:51<3:35:59, 2.02s/it] 15%|█▍ | 1098/7500 [36:53<3:33:40, 2.00s/it] 15%|█▍ | 1099/7500 [36:55<3:33:20, 2.00s/it] 15%|█▍ | 1100/7500 [36:57<3:32:48, 2.00s/it] {'loss': 0.5147, 'grad_norm': 5.441244602203369, 'learning_rate': 4.966903830281449e-05, 'logqs/rejected': -2.5781123638153076, 'logqs/chosen': -0.9522767066955566, 'logqs_logps/rejected': -0.037535861134529114, 'logps_logqs/chosen': 0.2545691430568695, 'epoch': 0.29} 15%|█▍ | 1100/7500 [36:57<3:32:48, 2.00s/it] 15%|█▍ | 1101/7500 [36:59<3:35:16, 2.02s/it] 15%|█▍ | 1102/7500 [37:01<3:33:52, 2.01s/it] 15%|█▍ | 1103/7500 [37:03<3:34:52, 2.02s/it] 15%|█▍ | 1104/7500 [37:05<3:31:50, 1.99s/it] 15%|█▍ | 1105/7500 [37:07<3:31:29, 1.98s/it] 15%|█▍ | 1106/7500 [37:09<3:36:23, 2.03s/it] 15%|█▍ | 1107/7500 [37:11<3:38:22, 2.05s/it] 15%|█▍ | 1108/7500 [37:13<3:35:46, 2.03s/it] 15%|█▍ | 1109/7500 [37:15<3:36:10, 2.03s/it] 15%|█▍ | 1110/7500 [37:17<3:34:16, 2.01s/it] {'loss': 0.5197, 'grad_norm': 5.2334208488464355, 'learning_rate': 4.964990092676263e-05, 'logqs/rejected': -2.5640769004821777, 'logqs/chosen': -0.971583366394043, 'logqs_logps/rejected': -0.023818403482437134, 'logps_logqs/chosen': 0.25017741322517395, 'epoch': 0.3} 15%|█▍ | 1110/7500 [37:17<3:34:16, 2.01s/it] 15%|█▍ | 1111/7500 [37:19<3:31:24, 1.99s/it] 15%|█▍ | 1112/7500 [37:21<3:32:52, 2.00s/it] 15%|█▍ | 1113/7500 [37:23<3:29:20, 1.97s/it] 15%|█▍ | 1114/7500 [37:25<3:30:55, 1.98s/it] 15%|█▍ | 1115/7500 [37:27<3:33:33, 2.01s/it] 15%|█▍ | 1116/7500 [37:29<3:31:53, 1.99s/it] 15%|█▍ | 1117/7500 [37:31<3:32:05, 1.99s/it] 15%|█▍ | 1118/7500 [37:33<3:30:15, 1.98s/it] 15%|█▍ | 1119/7500 [37:35<3:29:49, 1.97s/it] 15%|█▍ | 1120/7500 [37:37<3:30:15, 1.98s/it] {'loss': 0.5178, 'grad_norm': 4.966696262359619, 'learning_rate': 4.9630229593330226e-05, 'logqs/rejected': -2.7799837589263916, 'logqs/chosen': -1.0170756578445435, 'logqs_logps/rejected': -0.02998429536819458, 'logps_logqs/chosen': 0.2708713114261627, 'epoch': 0.3} 15%|█▍ | 1120/7500 [37:37<3:30:15, 1.98s/it] 15%|█▍ | 1121/7500 [37:39<3:33:17, 2.01s/it] 15%|█▍ | 1122/7500 [37:41<3:34:43, 2.02s/it] 15%|█▍ | 1123/7500 [37:43<3:38:15, 2.05s/it] 15%|█▍ | 1124/7500 [37:45<3:36:15, 2.04s/it] 15%|█▌ | 1125/7500 [37:47<3:34:33, 2.02s/it] 15%|█▌ | 1126/7500 [37:49<3:35:22, 2.03s/it] 15%|█▌ | 1127/7500 [37:51<3:36:38, 2.04s/it] 15%|█▌ | 1128/7500 [37:53<3:34:24, 2.02s/it] 15%|█▌ | 1129/7500 [37:55<3:33:21, 2.01s/it] 15%|█▌ | 1130/7500 [37:57<3:31:44, 1.99s/it] {'loss': 0.525, 'grad_norm': 5.381223201751709, 'learning_rate': 4.96100247286307e-05, 'logqs/rejected': -2.4074385166168213, 'logqs/chosen': -0.9587692022323608, 'logqs_logps/rejected': -0.022762088105082512, 'logps_logqs/chosen': 0.24844977259635925, 'epoch': 0.3} 15%|█▌ | 1130/7500 [37:57<3:31:44, 1.99s/it] 15%|█▌ | 1131/7500 [37:59<3:32:50, 2.01s/it] 15%|█▌ | 1132/7500 [38:01<3:31:21, 1.99s/it] 15%|█▌ | 1133/7500 [38:03<3:31:40, 1.99s/it] 15%|█▌ | 1134/7500 [38:05<3:33:08, 2.01s/it] 15%|█▌ | 1135/7500 [38:07<3:36:20, 2.04s/it] 15%|█▌ | 1136/7500 [38:09<3:37:02, 2.05s/it] 15%|█▌ | 1137/7500 [38:12<3:36:24, 2.04s/it] 15%|█▌ | 1138/7500 [38:13<3:34:04, 2.02s/it] 15%|█▌ | 1139/7500 [38:15<3:33:53, 2.02s/it] 15%|█▌ | 1140/7500 [38:17<3:32:31, 2.00s/it] {'loss': 0.5091, 'grad_norm': 4.021999359130859, 'learning_rate': 4.9589286770334654e-05, 'logqs/rejected': -2.4375340938568115, 'logqs/chosen': -0.9366236925125122, 'logqs_logps/rejected': -0.026224415749311447, 'logps_logqs/chosen': 0.24976427853107452, 'epoch': 0.3} 15%|█▌ | 1140/7500 [38:17<3:32:31, 2.00s/it] 15%|█▌ | 1141/7500 [38:20<3:33:55, 2.02s/it] 15%|█▌ | 1142/7500 [38:22<3:36:41, 2.04s/it] 15%|█▌ | 1143/7500 [38:24<3:34:30, 2.02s/it] 15%|█▌ | 1144/7500 [38:26<3:35:07, 2.03s/it] 15%|█▌ | 1145/7500 [38:28<3:37:59, 2.06s/it] 15%|█▌ | 1146/7500 [38:30<3:36:42, 2.05s/it] 15%|█▌ | 1147/7500 [38:32<3:37:35, 2.06s/it] 15%|█▌ | 1148/7500 [38:34<3:39:48, 2.08s/it] 15%|█▌ | 1149/7500 [38:36<3:39:20, 2.07s/it] 15%|█▌ | 1150/7500 [38:38<3:37:50, 2.06s/it] {'loss': 0.5113, 'grad_norm': 3.9338321685791016, 'learning_rate': 4.9568016167660334e-05, 'logqs/rejected': -2.454225778579712, 'logqs/chosen': -1.006645917892456, 'logqs_logps/rejected': -0.027201393619179726, 'logps_logqs/chosen': 0.2500317394733429, 'epoch': 0.31} 15%|█▌ | 1150/7500 [38:38<3:37:50, 2.06s/it] 15%|█▌ | 1151/7500 [38:40<3:35:51, 2.04s/it] 15%|█▌ | 1152/7500 [38:42<3:33:52, 2.02s/it] 15%|█▌ | 1153/7500 [38:44<3:32:58, 2.01s/it] 15%|█▌ | 1154/7500 [38:46<3:30:06, 1.99s/it] 15%|█▌ | 1155/7500 [38:48<3:30:31, 1.99s/it] 15%|█▌ | 1156/7500 [38:50<3:30:18, 1.99s/it] 15%|█▌ | 1157/7500 [38:52<3:27:43, 1.96s/it] 15%|█▌ | 1158/7500 [38:54<3:28:07, 1.97s/it] 15%|█▌ | 1159/7500 [38:56<3:27:46, 1.97s/it] 15%|█▌ | 1160/7500 [38:58<3:29:01, 1.98s/it] {'loss': 0.4973, 'grad_norm': 4.208204746246338, 'learning_rate': 4.954621338136398e-05, 'logqs/rejected': -2.449587345123291, 'logqs/chosen': -0.9380687475204468, 'logqs_logps/rejected': -0.026902899146080017, 'logps_logqs/chosen': 0.24352414906024933, 'epoch': 0.31} 15%|█▌ | 1160/7500 [38:58<3:29:01, 1.98s/it] 15%|█▌ | 1161/7500 [39:00<3:32:03, 2.01s/it] 15%|█▌ | 1162/7500 [39:02<3:29:54, 1.99s/it] 16%|█▌ | 1163/7500 [39:04<3:29:10, 1.98s/it] 16%|█▌ | 1164/7500 [39:06<3:29:46, 1.99s/it] 16%|█▌ | 1165/7500 [39:08<3:28:11, 1.97s/it] 16%|█▌ | 1166/7500 [39:10<3:31:20, 2.00s/it] 16%|█▌ | 1167/7500 [39:12<3:32:47, 2.02s/it] 16%|█▌ | 1168/7500 [39:14<3:33:32, 2.02s/it] 16%|█▌ | 1169/7500 [39:16<3:32:08, 2.01s/it] 16%|█▌ | 1170/7500 [39:18<3:29:55, 1.99s/it] {'loss': 0.5429, 'grad_norm': 5.333463668823242, 'learning_rate': 4.952387888372979e-05, 'logqs/rejected': -2.6623494625091553, 'logqs/chosen': -0.9564799070358276, 'logqs_logps/rejected': -0.020335370674729347, 'logps_logqs/chosen': 0.24730947613716125, 'epoch': 0.31} 16%|█▌ | 1170/7500 [39:18<3:29:55, 1.99s/it] 16%|█▌ | 1171/7500 [39:20<3:31:33, 2.01s/it] 16%|█▌ | 1172/7500 [39:22<3:32:49, 2.02s/it] 16%|█▌ | 1173/7500 [39:24<3:32:16, 2.01s/it] 16%|█▌ | 1174/7500 [39:26<3:29:51, 1.99s/it] 16%|█▌ | 1175/7500 [39:28<3:32:35, 2.02s/it] 16%|█▌ | 1176/7500 [39:30<3:31:35, 2.01s/it] 16%|█▌ | 1177/7500 [39:32<3:33:57, 2.03s/it] 16%|█▌ | 1178/7500 [39:34<3:35:06, 2.04s/it] 16%|█▌ | 1179/7500 [39:36<3:32:03, 2.01s/it] 16%|█▌ | 1180/7500 [39:38<3:33:38, 2.03s/it] {'loss': 0.4684, 'grad_norm': 4.206955432891846, 'learning_rate': 4.95010131585597e-05, 'logqs/rejected': -2.375499963760376, 'logqs/chosen': -0.9645260572433472, 'logqs_logps/rejected': -0.03688894584774971, 'logps_logqs/chosen': 0.25077447295188904, 'epoch': 0.31} 16%|█▌ | 1180/7500 [39:38<3:33:38, 2.03s/it] 16%|█▌ | 1181/7500 [39:40<3:31:48, 2.01s/it] 16%|█▌ | 1182/7500 [39:42<3:29:43, 1.99s/it] 16%|█▌ | 1183/7500 [39:44<3:29:53, 1.99s/it] 16%|█▌ | 1184/7500 [39:46<3:31:08, 2.01s/it] 16%|█▌ | 1185/7500 [39:48<3:30:30, 2.00s/it] 16%|█▌ | 1186/7500 [39:50<3:34:00, 2.03s/it] 16%|█▌ | 1187/7500 [39:52<3:35:39, 2.05s/it] 16%|█▌ | 1188/7500 [39:54<3:37:38, 2.07s/it] 16%|█▌ | 1189/7500 [39:56<3:37:07, 2.06s/it] 16%|█▌ | 1190/7500 [39:59<3:39:32, 2.09s/it] {'loss': 0.5079, 'grad_norm': 4.871334075927734, 'learning_rate': 4.94776167011629e-05, 'logqs/rejected': -2.238279104232788, 'logqs/chosen': -0.970722496509552, 'logqs_logps/rejected': -0.02788889780640602, 'logps_logqs/chosen': 0.24999265372753143, 'epoch': 0.32} 16%|█▌ | 1190/7500 [39:59<3:39:32, 2.09s/it] 16%|█▌ | 1191/7500 [40:01<3:40:34, 2.10s/it] 16%|█▌ | 1192/7500 [40:03<3:41:48, 2.11s/it] 16%|█▌ | 1193/7500 [40:05<3:40:16, 2.10s/it] 16%|█▌ | 1194/7500 [40:07<3:40:01, 2.09s/it] 16%|█▌ | 1195/7500 [40:09<3:38:43, 2.08s/it] 16%|█▌ | 1196/7500 [40:11<3:36:43, 2.06s/it] 16%|█▌ | 1197/7500 [40:13<3:35:37, 2.05s/it] 16%|█▌ | 1198/7500 [40:15<3:33:32, 2.03s/it] 16%|█▌ | 1199/7500 [40:17<3:30:27, 2.00s/it] 16%|█▌ | 1200/7500 [40:19<3:31:28, 2.01s/it] {'loss': 0.5004, 'grad_norm': 4.2239203453063965, 'learning_rate': 4.9453690018345144e-05, 'logqs/rejected': -2.599768877029419, 'logqs/chosen': -0.9389106035232544, 'logqs_logps/rejected': -0.025772374123334885, 'logps_logqs/chosen': 0.2514512538909912, 'epoch': 0.32} 16%|█▌ | 1200/7500 [40:19<3:31:28, 2.01s/it] 16%|█▌ | 1201/7500 [40:21<3:33:50, 2.04s/it] 16%|█▌ | 1202/7500 [40:23<3:34:35, 2.04s/it] 16%|█▌ | 1203/7500 [40:25<3:31:29, 2.02s/it] 16%|█▌ | 1204/7500 [40:27<3:32:31, 2.03s/it] 16%|█▌ | 1205/7500 [40:29<3:32:26, 2.02s/it] 16%|█▌ | 1206/7500 [40:31<3:32:26, 2.03s/it] 16%|█▌ | 1207/7500 [40:33<3:32:49, 2.03s/it] 16%|█▌ | 1208/7500 [40:35<3:33:14, 2.03s/it] 16%|█▌ | 1209/7500 [40:37<3:34:32, 2.05s/it] 16%|█▌ | 1210/7500 [40:39<3:37:00, 2.07s/it] {'loss': 0.4739, 'grad_norm': 4.956090450286865, 'learning_rate': 4.94292336283977e-05, 'logqs/rejected': -2.2926430702209473, 'logqs/chosen': -0.9333904981613159, 'logqs_logps/rejected': -0.03323855251073837, 'logps_logqs/chosen': 0.24894241988658905, 'epoch': 0.32} 16%|█▌ | 1210/7500 [40:39<3:37:00, 2.07s/it] 16%|█▌ | 1211/7500 [40:42<3:39:45, 2.10s/it] 16%|█▌ | 1212/7500 [40:44<3:37:04, 2.07s/it] 16%|█▌ | 1213/7500 [40:46<3:37:52, 2.08s/it] 16%|█▌ | 1214/7500 [40:48<3:35:18, 2.06s/it] 16%|█▌ | 1215/7500 [40:50<3:33:38, 2.04s/it] 16%|█▌ | 1216/7500 [40:52<3:36:16, 2.07s/it] 16%|█▌ | 1217/7500 [40:54<3:35:10, 2.05s/it] 16%|█▌ | 1218/7500 [40:56<3:33:19, 2.04s/it] 16%|█▋ | 1219/7500 [40:58<3:33:25, 2.04s/it] 16%|█▋ | 1220/7500 [41:00<3:30:52, 2.01s/it] {'loss': 0.4843, 'grad_norm': 4.148115634918213, 'learning_rate': 4.940424806108619e-05, 'logqs/rejected': -2.4283294677734375, 'logqs/chosen': -0.9516505002975464, 'logqs_logps/rejected': -0.029607033357024193, 'logps_logqs/chosen': 0.2426328957080841, 'epoch': 0.33} 16%|█▋ | 1220/7500 [41:00<3:30:52, 2.01s/it] 16%|█▋ | 1221/7500 [41:02<3:34:43, 2.05s/it] 16%|█▋ | 1222/7500 [41:04<3:32:36, 2.03s/it] 16%|█▋ | 1223/7500 [41:06<3:33:35, 2.04s/it] 16%|█▋ | 1224/7500 [41:08<3:31:45, 2.02s/it] 16%|█▋ | 1225/7500 [41:10<3:32:12, 2.03s/it] 16%|█▋ | 1226/7500 [41:12<3:30:33, 2.01s/it] 16%|█▋ | 1227/7500 [41:14<3:29:25, 2.00s/it] 16%|█▋ | 1228/7500 [41:16<3:29:09, 2.00s/it] 16%|█▋ | 1229/7500 [41:18<3:29:00, 2.00s/it] 16%|█▋ | 1230/7500 [41:20<3:35:14, 2.06s/it] {'loss': 0.5048, 'grad_norm': 4.773991107940674, 'learning_rate': 4.937873385763908e-05, 'logqs/rejected': -2.4467151165008545, 'logqs/chosen': -0.9673295021057129, 'logqs_logps/rejected': -0.021118521690368652, 'logps_logqs/chosen': 0.25391483306884766, 'epoch': 0.33} 16%|█▋ | 1230/7500 [41:20<3:35:14, 2.06s/it] 16%|█▋ | 1231/7500 [41:22<3:35:14, 2.06s/it] 16%|█▋ | 1232/7500 [41:24<3:35:52, 2.07s/it] 16%|█▋ | 1233/7500 [41:26<3:34:37, 2.05s/it] 16%|█▋ | 1234/7500 [41:28<3:31:13, 2.02s/it] 16%|█▋ | 1235/7500 [41:30<3:28:28, 2.00s/it] 16%|█▋ | 1236/7500 [41:32<3:30:13, 2.01s/it] 16%|█▋ | 1237/7500 [41:34<3:28:46, 2.00s/it] 17%|█▋ | 1238/7500 [41:36<3:32:53, 2.04s/it] 17%|█▋ | 1239/7500 [41:38<3:30:49, 2.02s/it] 17%|█▋ | 1240/7500 [41:40<3:27:31, 1.99s/it] {'loss': 0.4911, 'grad_norm': 3.9683241844177246, 'learning_rate': 4.9352691570735965e-05, 'logqs/rejected': -2.333076238632202, 'logqs/chosen': -0.9780262112617493, 'logqs_logps/rejected': -0.03657368943095207, 'logps_logqs/chosen': 0.2644956707954407, 'epoch': 0.33} 17%|█▋ | 1240/7500 [41:40<3:27:31, 1.99s/it] 17%|█▋ | 1241/7500 [41:42<3:27:30, 1.99s/it] 17%|█▋ | 1242/7500 [41:44<3:28:35, 2.00s/it] 17%|█▋ | 1243/7500 [41:46<3:30:02, 2.01s/it] 17%|█▋ | 1244/7500 [41:49<3:32:50, 2.04s/it] 17%|█▋ | 1245/7500 [41:51<3:33:09, 2.04s/it] 17%|█▋ | 1246/7500 [41:53<3:30:54, 2.02s/it] 17%|█▋ | 1247/7500 [41:54<3:28:51, 2.00s/it] 17%|█▋ | 1248/7500 [41:57<3:32:07, 2.04s/it] 17%|█▋ | 1249/7500 [41:59<3:29:58, 2.02s/it] 17%|█▋ | 1250/7500 [42:01<3:33:45, 2.05s/it] {'loss': 0.5079, 'grad_norm': 4.595108509063721, 'learning_rate': 4.9326121764495596e-05, 'logqs/rejected': -2.8267807960510254, 'logqs/chosen': -0.9595505595207214, 'logqs_logps/rejected': -0.023435767740011215, 'logps_logqs/chosen': 0.2527371644973755, 'epoch': 0.33} 17%|█▋ | 1250/7500 [42:01<3:33:45, 2.05s/it] 17%|█▋ | 1251/7500 [42:03<3:38:35, 2.10s/it] 17%|█▋ | 1252/7500 [42:05<3:35:27, 2.07s/it] 17%|█▋ | 1253/7500 [42:07<3:34:38, 2.06s/it] 17%|█▋ | 1254/7500 [42:09<3:35:35, 2.07s/it] 17%|█▋ | 1255/7500 [42:11<3:34:50, 2.06s/it] 17%|█▋ | 1256/7500 [42:13<3:33:55, 2.06s/it] 17%|█▋ | 1257/7500 [42:15<3:29:46, 2.02s/it] 17%|█▋ | 1258/7500 [42:17<3:32:49, 2.05s/it] 17%|█▋ | 1259/7500 [42:19<3:32:09, 2.04s/it] 17%|█▋ | 1260/7500 [42:21<3:31:09, 2.03s/it] {'loss': 0.5125, 'grad_norm': 3.883739709854126, 'learning_rate': 4.929902501446366e-05, 'logqs/rejected': -2.5895934104919434, 'logqs/chosen': -0.9860154986381531, 'logqs_logps/rejected': -0.028812900185585022, 'logps_logqs/chosen': 0.2558191418647766, 'epoch': 0.34} 17%|█▋ | 1260/7500 [42:21<3:31:09, 2.03s/it] 17%|█▋ | 1261/7500 [42:23<3:30:22, 2.02s/it] 17%|█▋ | 1262/7500 [42:25<3:34:28, 2.06s/it] 17%|█▋ | 1263/7500 [42:27<3:33:10, 2.05s/it] 17%|█▋ | 1264/7500 [42:29<3:30:42, 2.03s/it] 17%|█▋ | 1265/7500 [42:31<3:29:56, 2.02s/it] 17%|█▋ | 1266/7500 [42:33<3:28:50, 2.01s/it] 17%|█▋ | 1267/7500 [42:35<3:29:59, 2.02s/it] 17%|█▋ | 1268/7500 [42:37<3:29:36, 2.02s/it] 17%|█▋ | 1269/7500 [42:39<3:28:44, 2.01s/it] 17%|█▋ | 1270/7500 [42:41<3:29:55, 2.02s/it] {'loss': 0.5071, 'grad_norm': 4.474725246429443, 'learning_rate': 4.92714019076003e-05, 'logqs/rejected': -2.6112356185913086, 'logqs/chosen': -0.9584823846817017, 'logqs_logps/rejected': -0.02522962912917137, 'logps_logqs/chosen': 0.2466772496700287, 'epoch': 0.34} 17%|█▋ | 1270/7500 [42:41<3:29:55, 2.02s/it] 17%|█▋ | 1271/7500 [42:43<3:27:21, 2.00s/it] 17%|█▋ | 1272/7500 [42:45<3:25:55, 1.98s/it] 17%|█▋ | 1273/7500 [42:47<3:27:59, 2.00s/it] 17%|█▋ | 1274/7500 [42:49<3:29:00, 2.01s/it] 17%|█▋ | 1275/7500 [42:52<3:30:55, 2.03s/it] 17%|█▋ | 1276/7500 [42:54<3:30:34, 2.03s/it] 17%|█▋ | 1277/7500 [42:56<3:30:28, 2.03s/it] 17%|█▋ | 1278/7500 [42:58<3:29:22, 2.02s/it] 17%|█▋ | 1279/7500 [42:59<3:25:54, 1.99s/it] 17%|█▋ | 1280/7500 [43:02<3:28:26, 2.01s/it] {'loss': 0.503, 'grad_norm': 4.153003692626953, 'learning_rate': 4.924325304226745e-05, 'logqs/rejected': -2.7147507667541504, 'logqs/chosen': -0.9751055836677551, 'logqs_logps/rejected': -0.028437551110982895, 'logps_logqs/chosen': 0.2575641870498657, 'epoch': 0.34} 17%|█▋ | 1280/7500 [43:02<3:28:26, 2.01s/it] 17%|█▋ | 1281/7500 [43:03<3:26:30, 1.99s/it] 17%|█▋ | 1282/7500 [43:05<3:25:01, 1.98s/it] 17%|█▋ | 1283/7500 [43:07<3:27:32, 2.00s/it] 17%|█▋ | 1284/7500 [43:09<3:27:33, 2.00s/it] 17%|█▋ | 1285/7500 [43:11<3:26:59, 2.00s/it] 17%|█▋ | 1286/7500 [43:13<3:27:03, 2.00s/it] 17%|█▋ | 1287/7500 [43:15<3:25:00, 1.98s/it] 17%|█▋ | 1288/7500 [43:18<3:28:08, 2.01s/it] 17%|█▋ | 1289/7500 [43:20<3:28:10, 2.01s/it] 17%|█▋ | 1290/7500 [43:22<3:27:33, 2.01s/it] {'loss': 0.4827, 'grad_norm': 4.107663631439209, 'learning_rate': 4.9214579028215776e-05, 'logqs/rejected': -2.46826171875, 'logqs/chosen': -0.9350307583808899, 'logqs_logps/rejected': -0.03185320645570755, 'logps_logqs/chosen': 0.25291600823402405, 'epoch': 0.34} 17%|█▋ | 1290/7500 [43:22<3:27:33, 2.01s/it] 17%|█▋ | 1291/7500 [43:24<3:28:49, 2.02s/it] 17%|█▋ | 1292/7500 [43:26<3:27:16, 2.00s/it] 17%|█▋ | 1293/7500 [43:27<3:25:57, 1.99s/it] 17%|█▋ | 1294/7500 [43:29<3:26:25, 2.00s/it] 17%|█▋ | 1295/7500 [43:32<3:28:32, 2.02s/it] 17%|█▋ | 1296/7500 [43:34<3:27:16, 2.00s/it] 17%|█▋ | 1297/7500 [43:36<3:26:29, 2.00s/it] 17%|█▋ | 1298/7500 [43:38<3:28:01, 2.01s/it] 17%|█▋ | 1299/7500 [43:40<3:27:32, 2.01s/it] 17%|█▋ | 1300/7500 [43:42<3:29:42, 2.03s/it] {'loss': 0.491, 'grad_norm': 5.312241077423096, 'learning_rate': 4.9185380486571595e-05, 'logqs/rejected': -2.393990993499756, 'logqs/chosen': -0.9527361989021301, 'logqs_logps/rejected': -0.03194614127278328, 'logps_logqs/chosen': 0.24155378341674805, 'epoch': 0.35} 17%|█▋ | 1300/7500 [43:42<3:29:42, 2.03s/it] 17%|█▋ | 1301/7500 [43:44<3:30:36, 2.04s/it] 17%|█▋ | 1302/7500 [43:46<3:30:52, 2.04s/it] 17%|█▋ | 1303/7500 [43:48<3:31:59, 2.05s/it] 17%|█▋ | 1304/7500 [43:50<3:31:43, 2.05s/it] 17%|█▋ | 1305/7500 [43:52<3:31:04, 2.04s/it] 17%|█▋ | 1306/7500 [43:54<3:31:04, 2.04s/it] 17%|█▋ | 1307/7500 [43:56<3:30:28, 2.04s/it] 17%|█▋ | 1308/7500 [43:58<3:30:43, 2.04s/it] 17%|█▋ | 1309/7500 [44:00<3:30:13, 2.04s/it] 17%|█▋ | 1310/7500 [44:02<3:29:17, 2.03s/it] {'loss': 0.4869, 'grad_norm': 4.412881374359131, 'learning_rate': 4.9155658049823324e-05, 'logqs/rejected': -2.2317209243774414, 'logqs/chosen': -0.9892977476119995, 'logqs_logps/rejected': -0.03301815316081047, 'logps_logqs/chosen': 0.24556410312652588, 'epoch': 0.35} 17%|█▋ | 1310/7500 [44:02<3:29:17, 2.03s/it] 17%|█▋ | 1311/7500 [44:04<3:29:51, 2.03s/it] 17%|█▋ | 1312/7500 [44:06<3:29:23, 2.03s/it] 18%|█▊ | 1313/7500 [44:08<3:26:02, 2.00s/it] 18%|█▊ | 1314/7500 [44:10<3:24:59, 1.99s/it] 18%|█▊ | 1315/7500 [44:12<3:27:31, 2.01s/it] 18%|█▊ | 1316/7500 [44:14<3:28:36, 2.02s/it] 18%|█▊ | 1317/7500 [44:16<3:30:54, 2.05s/it] 18%|█▊ | 1318/7500 [44:18<3:29:45, 2.04s/it] 18%|█▊ | 1319/7500 [44:20<3:28:28, 2.02s/it] 18%|█▊ | 1320/7500 [44:22<3:26:49, 2.01s/it] {'loss': 0.4786, 'grad_norm': 5.683045387268066, 'learning_rate': 4.912541236180779e-05, 'logqs/rejected': -2.486992120742798, 'logqs/chosen': -0.9631454348564148, 'logqs_logps/rejected': -0.033156827092170715, 'logps_logqs/chosen': 0.25327420234680176, 'epoch': 0.35} 18%|█▊ | 1320/7500 [44:22<3:26:49, 2.01s/it] 18%|█▊ | 1321/7500 [44:24<3:25:22, 1.99s/it] 18%|█▊ | 1322/7500 [44:26<3:24:59, 1.99s/it] 18%|█▊ | 1323/7500 [44:28<3:23:10, 1.97s/it] 18%|█▊ | 1324/7500 [44:30<3:27:01, 2.01s/it] 18%|█▊ | 1325/7500 [44:32<3:24:42, 1.99s/it] 18%|█▊ | 1326/7500 [44:34<3:23:15, 1.98s/it] 18%|█▊ | 1327/7500 [44:36<3:22:36, 1.97s/it] 18%|█▊ | 1328/7500 [44:38<3:27:23, 2.02s/it] 18%|█▊ | 1329/7500 [44:40<3:28:59, 2.03s/it] 18%|█▊ | 1330/7500 [44:42<3:28:16, 2.03s/it] {'loss': 0.4851, 'grad_norm': 4.131486415863037, 'learning_rate': 4.909464407769633e-05, 'logqs/rejected': -2.388806104660034, 'logqs/chosen': -0.9620172381401062, 'logqs_logps/rejected': -0.03184809163212776, 'logps_logqs/chosen': 0.2481938600540161, 'epoch': 0.35} 18%|█▊ | 1330/7500 [44:42<3:28:16, 2.03s/it] 18%|█▊ | 1331/7500 [44:44<3:28:53, 2.03s/it] 18%|█▊ | 1332/7500 [44:46<3:27:45, 2.02s/it] 18%|█▊ | 1333/7500 [44:48<3:26:06, 2.01s/it] 18%|█▊ | 1334/7500 [44:50<3:24:33, 1.99s/it] 18%|█▊ | 1335/7500 [44:52<3:23:48, 1.98s/it] 18%|█▊ | 1336/7500 [44:54<3:28:08, 2.03s/it] 18%|█▊ | 1337/7500 [44:56<3:32:34, 2.07s/it] 18%|█▊ | 1338/7500 [44:58<3:31:37, 2.06s/it] 18%|█▊ | 1339/7500 [45:01<3:30:05, 2.05s/it] 18%|█▊ | 1340/7500 [45:02<3:25:06, 2.00s/it] {'loss': 0.5189, 'grad_norm': 4.454148769378662, 'learning_rate': 4.9063353863980565e-05, 'logqs/rejected': -2.549914598464966, 'logqs/chosen': -0.9649158716201782, 'logqs_logps/rejected': -0.02976357936859131, 'logps_logqs/chosen': 0.25549760460853577, 'epoch': 0.36} 18%|█▊ | 1340/7500 [45:02<3:25:06, 2.00s/it] 18%|█▊ | 1341/7500 [45:04<3:26:27, 2.01s/it] 18%|█▊ | 1342/7500 [45:06<3:25:39, 2.00s/it] 18%|█▊ | 1343/7500 [45:09<3:27:58, 2.03s/it] 18%|█▊ | 1344/7500 [45:10<3:24:33, 1.99s/it] 18%|█▊ | 1345/7500 [45:12<3:25:54, 2.01s/it] 18%|█▊ | 1346/7500 [45:14<3:23:59, 1.99s/it] 18%|█▊ | 1347/7500 [45:16<3:26:13, 2.01s/it] 18%|█▊ | 1348/7500 [45:18<3:26:38, 2.02s/it] 18%|█▊ | 1349/7500 [45:20<3:25:01, 2.00s/it] 18%|█▊ | 1350/7500 [45:22<3:24:09, 1.99s/it] {'loss': 0.4962, 'grad_norm': 4.549566745758057, 'learning_rate': 4.9031542398457974e-05, 'logqs/rejected': -2.436988592147827, 'logqs/chosen': -0.9772375822067261, 'logqs_logps/rejected': -0.030491238459944725, 'logps_logqs/chosen': 0.2532727122306824, 'epoch': 0.36} 18%|█▊ | 1350/7500 [45:22<3:24:09, 1.99s/it] 18%|█▊ | 1351/7500 [45:25<3:27:48, 2.03s/it] 18%|█▊ | 1352/7500 [45:27<3:31:03, 2.06s/it] 18%|█▊ | 1353/7500 [45:29<3:29:07, 2.04s/it] 18%|█▊ | 1354/7500 [45:31<3:26:47, 2.02s/it] 18%|█▊ | 1355/7500 [45:33<3:25:37, 2.01s/it] 18%|█▊ | 1356/7500 [45:35<3:25:07, 2.00s/it] 18%|█▊ | 1357/7500 [45:37<3:26:08, 2.01s/it] 18%|█▊ | 1358/7500 [45:39<3:29:34, 2.05s/it] 18%|█▊ | 1359/7500 [45:41<3:27:58, 2.03s/it] 18%|█▊ | 1360/7500 [45:43<3:30:05, 2.05s/it] {'loss': 0.5131, 'grad_norm': 4.4116291999816895, 'learning_rate': 4.8999210370217194e-05, 'logqs/rejected': -2.6745011806488037, 'logqs/chosen': -0.974595844745636, 'logqs_logps/rejected': -0.03050202503800392, 'logps_logqs/chosen': 0.2620945870876312, 'epoch': 0.36} 18%|█▊ | 1360/7500 [45:43<3:30:05, 2.05s/it] 18%|█▊ | 1361/7500 [45:45<3:27:28, 2.03s/it] 18%|█▊ | 1362/7500 [45:47<3:25:52, 2.01s/it] 18%|█▊ | 1363/7500 [45:49<3:24:43, 2.00s/it] 18%|█▊ | 1364/7500 [45:51<3:25:44, 2.01s/it] 18%|█▊ | 1365/7500 [45:53<3:25:07, 2.01s/it] 18%|█▊ | 1366/7500 [45:55<3:23:11, 1.99s/it] 18%|█▊ | 1367/7500 [45:57<3:24:14, 2.00s/it] 18%|█▊ | 1368/7500 [45:59<3:23:59, 2.00s/it] 18%|█▊ | 1369/7500 [46:01<3:23:22, 1.99s/it] 18%|█▊ | 1370/7500 [46:03<3:26:16, 2.02s/it] {'loss': 0.485, 'grad_norm': 4.679338455200195, 'learning_rate': 4.896635847962311e-05, 'logqs/rejected': -2.5784382820129395, 'logqs/chosen': -0.9398103952407837, 'logqs_logps/rejected': -0.029625168070197105, 'logps_logqs/chosen': 0.25168946385383606, 'epoch': 0.37} 18%|█▊ | 1370/7500 [46:03<3:26:16, 2.02s/it] 18%|█▊ | 1371/7500 [46:05<3:26:36, 2.02s/it] 18%|█▊ | 1372/7500 [46:07<3:26:47, 2.02s/it] 18%|█▊ | 1373/7500 [46:09<3:26:24, 2.02s/it] 18%|█▊ | 1374/7500 [46:11<3:24:31, 2.00s/it] 18%|█▊ | 1375/7500 [46:13<3:27:24, 2.03s/it] 18%|█▊ | 1376/7500 [46:15<3:25:50, 2.02s/it] 18%|█▊ | 1377/7500 [46:17<3:29:19, 2.05s/it] 18%|█▊ | 1378/7500 [46:19<3:30:22, 2.06s/it] 18%|█▊ | 1379/7500 [46:21<3:29:25, 2.05s/it] 18%|█▊ | 1380/7500 [46:23<3:25:30, 2.01s/it] {'loss': 0.4861, 'grad_norm': 4.123393535614014, 'learning_rate': 4.893298743830168e-05, 'logqs/rejected': -2.5496668815612793, 'logqs/chosen': -0.9443801641464233, 'logqs_logps/rejected': -0.03146081417798996, 'logps_logqs/chosen': 0.2477882355451584, 'epoch': 0.37} 18%|█▊ | 1380/7500 [46:23<3:25:30, 2.01s/it] 18%|█▊ | 1381/7500 [46:25<3:25:17, 2.01s/it] 18%|█▊ | 1382/7500 [46:27<3:25:30, 2.02s/it] 18%|█▊ | 1383/7500 [46:29<3:26:58, 2.03s/it] 18%|█▊ | 1384/7500 [46:31<3:27:22, 2.03s/it] 18%|█▊ | 1385/7500 [46:33<3:26:24, 2.03s/it] 18%|█▊ | 1386/7500 [46:35<3:26:31, 2.03s/it] 18%|█▊ | 1387/7500 [46:37<3:25:50, 2.02s/it] 19%|█▊ | 1388/7500 [46:39<3:25:09, 2.01s/it] 19%|█▊ | 1389/7500 [46:41<3:27:31, 2.04s/it] 19%|█▊ | 1390/7500 [46:43<3:27:40, 2.04s/it] {'loss': 0.4981, 'grad_norm': 4.152158737182617, 'learning_rate': 4.889909796912454e-05, 'logqs/rejected': -2.7189531326293945, 'logqs/chosen': -0.9624508023262024, 'logqs_logps/rejected': -0.030079850926995277, 'logps_logqs/chosen': 0.24932248890399933, 'epoch': 0.37} 19%|█▊ | 1390/7500 [46:43<3:27:40, 2.04s/it] 19%|█▊ | 1391/7500 [46:46<3:28:10, 2.04s/it] 19%|█▊ | 1392/7500 [46:47<3:24:44, 2.01s/it] 19%|█▊ | 1393/7500 [46:50<3:26:44, 2.03s/it] 19%|█▊ | 1394/7500 [46:52<3:27:21, 2.04s/it] 19%|█▊ | 1395/7500 [46:54<3:27:37, 2.04s/it] 19%|█▊ | 1396/7500 [46:56<3:25:21, 2.02s/it] 19%|█▊ | 1397/7500 [46:58<3:25:20, 2.02s/it] 19%|█▊ | 1398/7500 [47:00<3:27:14, 2.04s/it] 19%|█▊ | 1399/7500 [47:02<3:25:39, 2.02s/it] 19%|█▊ | 1400/7500 [47:04<3:27:11, 2.04s/it] {'loss': 0.4991, 'grad_norm': 4.955474376678467, 'learning_rate': 4.88646908061933e-05, 'logqs/rejected': -2.592194080352783, 'logqs/chosen': -0.9557517766952515, 'logqs_logps/rejected': -0.03742901235818863, 'logps_logqs/chosen': 0.2562134265899658, 'epoch': 0.37} 19%|█▊ | 1400/7500 [47:04<3:27:11, 2.04s/it] 19%|█▊ | 1401/7500 [47:06<3:28:16, 2.05s/it] 19%|█▊ | 1402/7500 [47:08<3:26:26, 2.03s/it] 19%|█▊ | 1403/7500 [47:10<3:23:13, 2.00s/it] 19%|█▊ | 1404/7500 [47:12<3:24:18, 2.01s/it] 19%|█▊ | 1405/7500 [47:14<3:23:18, 2.00s/it] 19%|█▊ | 1406/7500 [47:16<3:23:11, 2.00s/it] 19%|█▉ | 1407/7500 [47:18<3:25:03, 2.02s/it] 19%|█▉ | 1408/7500 [47:20<3:25:46, 2.03s/it] 19%|█▉ | 1409/7500 [47:22<3:27:33, 2.04s/it] 19%|█▉ | 1410/7500 [47:24<3:26:22, 2.03s/it] {'loss': 0.498, 'grad_norm': 3.889569044113159, 'learning_rate': 4.882976669482367e-05, 'logqs/rejected': -2.4371337890625, 'logqs/chosen': -0.9587616920471191, 'logqs_logps/rejected': -0.027300218120217323, 'logps_logqs/chosen': 0.24250373244285583, 'epoch': 0.38} 19%|█▉ | 1410/7500 [47:24<3:26:22, 2.03s/it] 19%|█▉ | 1411/7500 [47:26<3:28:11, 2.05s/it] 19%|█▉ | 1412/7500 [47:28<3:27:51, 2.05s/it] 19%|█▉ | 1413/7500 [47:30<3:27:55, 2.05s/it] 19%|█▉ | 1414/7500 [47:32<3:25:00, 2.02s/it] 19%|█▉ | 1415/7500 [47:34<3:23:45, 2.01s/it] 19%|█▉ | 1416/7500 [47:36<3:22:27, 2.00s/it] 19%|█▉ | 1417/7500 [47:38<3:26:15, 2.03s/it] 19%|█▉ | 1418/7500 [47:40<3:29:07, 2.06s/it] 19%|█▉ | 1419/7500 [47:42<3:28:57, 2.06s/it] 19%|█▉ | 1420/7500 [47:44<3:28:08, 2.05s/it] {'loss': 0.4838, 'grad_norm': 4.003763675689697, 'learning_rate': 4.8794326391529344e-05, 'logqs/rejected': -2.5005903244018555, 'logqs/chosen': -0.9478079676628113, 'logqs_logps/rejected': -0.02826736867427826, 'logps_logqs/chosen': 0.25135400891304016, 'epoch': 0.38} 19%|█▉ | 1420/7500 [47:44<3:28:08, 2.05s/it] 19%|█▉ | 1421/7500 [47:46<3:28:04, 2.05s/it] 19%|█▉ | 1422/7500 [47:48<3:24:15, 2.02s/it] 19%|█▉ | 1423/7500 [47:50<3:25:36, 2.03s/it] 19%|█▉ | 1424/7500 [47:52<3:26:17, 2.04s/it] 19%|█▉ | 1425/7500 [47:55<3:26:13, 2.04s/it] 19%|█▉ | 1426/7500 [47:57<3:28:47, 2.06s/it] 19%|█▉ | 1427/7500 [47:59<3:27:36, 2.05s/it] 19%|█▉ | 1428/7500 [48:01<3:24:58, 2.03s/it] 19%|█▉ | 1429/7500 [48:03<3:26:12, 2.04s/it] 19%|█▉ | 1430/7500 [48:05<3:28:13, 2.06s/it] {'loss': 0.4857, 'grad_norm': 4.91129732131958, 'learning_rate': 4.8758370664005535e-05, 'logqs/rejected': -2.5882465839385986, 'logqs/chosen': -0.930976390838623, 'logqs_logps/rejected': -0.028295446187257767, 'logps_logqs/chosen': 0.24843057990074158, 'epoch': 0.38} 19%|█▉ | 1430/7500 [48:05<3:28:13, 2.06s/it] 19%|█▉ | 1431/7500 [48:07<3:25:23, 2.03s/it] 19%|█▉ | 1432/7500 [48:09<3:25:23, 2.03s/it] 19%|█▉ | 1433/7500 [48:11<3:25:53, 2.04s/it] 19%|█▉ | 1434/7500 [48:13<3:27:35, 2.05s/it] 19%|█▉ | 1435/7500 [48:15<3:26:38, 2.04s/it] 19%|█▉ | 1436/7500 [48:17<3:24:51, 2.03s/it] 19%|█▉ | 1437/7500 [48:19<3:26:23, 2.04s/it] 19%|█▉ | 1438/7500 [48:21<3:25:22, 2.03s/it] 19%|█▉ | 1439/7500 [48:23<3:24:55, 2.03s/it] 19%|█▉ | 1440/7500 [48:25<3:28:34, 2.07s/it] {'loss': 0.4886, 'grad_norm': 4.291272163391113, 'learning_rate': 4.8721900291112415e-05, 'logqs/rejected': -2.586381435394287, 'logqs/chosen': -0.945195198059082, 'logqs_logps/rejected': -0.022369172424077988, 'logps_logqs/chosen': 0.24625983834266663, 'epoch': 0.38} 19%|█▉ | 1440/7500 [48:25<3:28:34, 2.07s/it] 19%|█▉ | 1441/7500 [48:27<3:27:15, 2.05s/it] 19%|█▉ | 1442/7500 [48:29<3:26:26, 2.04s/it] 19%|█▉ | 1443/7500 [48:31<3:27:40, 2.06s/it] 19%|█▉ | 1444/7500 [48:33<3:29:22, 2.07s/it] 19%|█▉ | 1445/7500 [48:35<3:27:29, 2.06s/it] 19%|█▉ | 1446/7500 [48:37<3:26:10, 2.04s/it] 19%|█▉ | 1447/7500 [48:40<3:29:13, 2.07s/it] 19%|█▉ | 1448/7500 [48:42<3:27:57, 2.06s/it] 19%|█▉ | 1449/7500 [48:44<3:28:30, 2.07s/it] 19%|█▉ | 1450/7500 [48:46<3:26:40, 2.05s/it] {'loss': 0.4901, 'grad_norm': 4.841124057769775, 'learning_rate': 4.868491606285823e-05, 'logqs/rejected': -2.525188684463501, 'logqs/chosen': -0.9470198750495911, 'logqs_logps/rejected': -0.030799949541687965, 'logps_logqs/chosen': 0.24772830307483673, 'epoch': 0.39} 19%|█▉ | 1450/7500 [48:46<3:26:40, 2.05s/it] 19%|█▉ | 1451/7500 [48:48<3:28:55, 2.07s/it] 19%|█▉ | 1452/7500 [48:50<3:27:09, 2.06s/it] 19%|█▉ | 1453/7500 [48:52<3:22:11, 2.01s/it] 19%|█▉ | 1454/7500 [48:54<3:21:53, 2.00s/it] 19%|█▉ | 1455/7500 [48:56<3:22:42, 2.01s/it] 19%|█▉ | 1456/7500 [48:58<3:24:42, 2.03s/it] 19%|█▉ | 1457/7500 [49:00<3:22:45, 2.01s/it] 19%|█▉ | 1458/7500 [49:02<3:21:35, 2.00s/it] 19%|█▉ | 1459/7500 [49:04<3:21:28, 2.00s/it] 19%|█▉ | 1460/7500 [49:06<3:18:35, 1.97s/it] {'loss': 0.5182, 'grad_norm': 4.3859028816223145, 'learning_rate': 4.8647418780382175e-05, 'logqs/rejected': -2.9991750717163086, 'logqs/chosen': -0.9397735595703125, 'logqs_logps/rejected': -0.017570924013853073, 'logps_logqs/chosen': 0.25387436151504517, 'epoch': 0.39} 19%|█▉ | 1460/7500 [49:06<3:18:35, 1.97s/it] 19%|█▉ | 1461/7500 [49:08<3:20:39, 1.99s/it] 19%|█▉ | 1462/7500 [49:10<3:19:09, 1.98s/it] 20%|█▉ | 1463/7500 [49:12<3:21:48, 2.01s/it] 20%|█▉ | 1464/7500 [49:14<3:20:31, 1.99s/it] 20%|█▉ | 1465/7500 [49:16<3:25:24, 2.04s/it] 20%|█▉ | 1466/7500 [49:18<3:24:07, 2.03s/it] 20%|█▉ | 1467/7500 [49:20<3:24:05, 2.03s/it] 20%|█▉ | 1468/7500 [49:22<3:23:22, 2.02s/it] 20%|█▉ | 1469/7500 [49:24<3:22:58, 2.02s/it] 20%|█▉ | 1470/7500 [49:26<3:23:32, 2.03s/it] {'loss': 0.5027, 'grad_norm': 3.926783323287964, 'learning_rate': 4.860940925593703e-05, 'logqs/rejected': -2.6529812812805176, 'logqs/chosen': -0.9462970495223999, 'logqs_logps/rejected': -0.024465885013341904, 'logps_logqs/chosen': 0.25491926074028015, 'epoch': 0.39} 20%|█▉ | 1470/7500 [49:26<3:23:32, 2.03s/it] 20%|█▉ | 1471/7500 [49:28<3:21:12, 2.00s/it] 20%|█▉ | 1472/7500 [49:30<3:23:15, 2.02s/it] 20%|█▉ | 1473/7500 [49:32<3:23:12, 2.02s/it] 20%|█▉ | 1474/7500 [49:34<3:24:21, 2.03s/it] 20%|█▉ | 1475/7500 [49:36<3:23:11, 2.02s/it] 20%|█▉ | 1476/7500 [49:38<3:26:04, 2.05s/it] 20%|█▉ | 1477/7500 [49:40<3:26:06, 2.05s/it] 20%|█▉ | 1478/7500 [49:43<3:30:47, 2.10s/it] 20%|█▉ | 1479/7500 [49:45<3:31:18, 2.11s/it] 20%|█▉ | 1480/7500 [49:47<3:28:35, 2.08s/it] {'loss': 0.4974, 'grad_norm': 4.269349575042725, 'learning_rate': 4.857088831287158e-05, 'logqs/rejected': -2.4819798469543457, 'logqs/chosen': -0.975021481513977, 'logqs_logps/rejected': -0.031825337558984756, 'logps_logqs/chosen': 0.2519763112068176, 'epoch': 0.39} 20%|█▉ | 1480/7500 [49:47<3:28:35, 2.08s/it] 20%|█▉ | 1481/7500 [49:49<3:24:50, 2.04s/it] 20%|█▉ | 1482/7500 [49:51<3:21:47, 2.01s/it] 20%|█▉ | 1483/7500 [49:52<3:19:48, 1.99s/it] 20%|█▉ | 1484/7500 [49:54<3:18:15, 1.98s/it] 20%|█▉ | 1485/7500 [49:57<3:22:17, 2.02s/it] 20%|█▉ | 1486/7500 [49:59<3:25:01, 2.05s/it] 20%|█▉ | 1487/7500 [50:01<3:26:58, 2.07s/it] 20%|█▉ | 1488/7500 [50:03<3:26:05, 2.06s/it] 20%|█▉ | 1489/7500 [50:05<3:25:35, 2.05s/it] 20%|█▉ | 1490/7500 [50:07<3:23:07, 2.03s/it] {'loss': 0.5078, 'grad_norm': 3.9204392433166504, 'learning_rate': 4.8531856785612794e-05, 'logqs/rejected': -2.650256872177124, 'logqs/chosen': -0.9547485113143921, 'logqs_logps/rejected': -0.024005107581615448, 'logps_logqs/chosen': 0.24813976883888245, 'epoch': 0.4} 20%|█▉ | 1490/7500 [50:07<3:23:07, 2.03s/it] 20%|█▉ | 1491/7500 [50:09<3:23:15, 2.03s/it] 20%|█▉ | 1492/7500 [50:11<3:22:53, 2.03s/it] 20%|█▉ | 1493/7500 [50:13<3:24:04, 2.04s/it] 20%|█▉ | 1494/7500 [50:15<3:22:56, 2.03s/it] 20%|█▉ | 1495/7500 [50:17<3:21:11, 2.01s/it] 20%|█▉ | 1496/7500 [50:19<3:21:12, 2.01s/it] 20%|█▉ | 1497/7500 [50:21<3:19:35, 1.99s/it] 20%|█▉ | 1498/7500 [50:23<3:19:33, 1.99s/it] 20%|█▉ | 1499/7500 [50:25<3:20:17, 2.00s/it] 20%|██ | 1500/7500 [50:27<3:19:52, 2.00s/it] {'loss': 0.5055, 'grad_norm': 4.346201419830322, 'learning_rate': 4.849231551964771e-05, 'logqs/rejected': -2.6900696754455566, 'logqs/chosen': -0.9369996190071106, 'logqs_logps/rejected': -0.024088691920042038, 'logps_logqs/chosen': 0.24571356177330017, 'epoch': 0.4} 20%|██ | 1500/7500 [50:27<3:19:52, 2.00s/it] 20%|██ | 1501/7500 [50:29<3:22:31, 2.03s/it] 20%|██ | 1502/7500 [50:31<3:25:00, 2.05s/it] 20%|██ | 1503/7500 [50:33<3:25:32, 2.06s/it] 20%|██ | 1504/7500 [50:35<3:24:05, 2.04s/it] 20%|██ | 1505/7500 [50:37<3:21:10, 2.01s/it] 20%|██ | 1506/7500 [50:39<3:22:56, 2.03s/it] 20%|██ | 1507/7500 [50:41<3:23:11, 2.03s/it] 20%|██ | 1508/7500 [50:43<3:22:26, 2.03s/it] 20%|██ | 1509/7500 [50:45<3:21:42, 2.02s/it] 20%|██ | 1510/7500 [50:47<3:19:38, 2.00s/it] {'loss': 0.4837, 'grad_norm': 4.450809001922607, 'learning_rate': 4.8452265371505176e-05, 'logqs/rejected': -2.6392476558685303, 'logqs/chosen': -0.9550083875656128, 'logqs_logps/rejected': -0.0311385877430439, 'logps_logqs/chosen': 0.2447023093700409, 'epoch': 0.4} 20%|██ | 1510/7500 [50:47<3:19:38, 2.00s/it] 20%|██ | 1511/7500 [50:49<3:19:59, 2.00s/it] 20%|██ | 1512/7500 [50:51<3:23:20, 2.04s/it] 20%|██ | 1513/7500 [50:53<3:23:21, 2.04s/it] 20%|██ | 1514/7500 [50:55<3:24:45, 2.05s/it] 20%|██ | 1515/7500 [50:58<3:28:57, 2.09s/it] 20%|██ | 1516/7500 [51:00<3:26:00, 2.07s/it] 20%|██ | 1517/7500 [51:02<3:26:45, 2.07s/it] 20%|██ | 1518/7500 [51:04<3:24:15, 2.05s/it] 20%|██ | 1519/7500 [51:06<3:23:58, 2.05s/it] 20%|██ | 1520/7500 [51:08<3:22:37, 2.03s/it] {'loss': 0.4945, 'grad_norm': 4.141780853271484, 'learning_rate': 4.841170720873723e-05, 'logqs/rejected': -2.606847047805786, 'logqs/chosen': -0.9473110437393188, 'logqs_logps/rejected': -0.02223214879631996, 'logps_logqs/chosen': 0.2458299696445465, 'epoch': 0.41} 20%|██ | 1520/7500 [51:08<3:22:37, 2.03s/it] 20%|██ | 1521/7500 [51:10<3:21:53, 2.03s/it] 20%|██ | 1522/7500 [51:12<3:23:25, 2.04s/it] 20%|██ | 1523/7500 [51:14<3:23:16, 2.04s/it] 20%|██ | 1524/7500 [51:16<3:23:47, 2.05s/it] 20%|██ | 1525/7500 [51:18<3:24:00, 2.05s/it] 20%|██ | 1526/7500 [51:20<3:25:11, 2.06s/it] 20%|██ | 1527/7500 [51:22<3:22:35, 2.04s/it] 20%|██ | 1528/7500 [51:24<3:22:55, 2.04s/it] 20%|██ | 1529/7500 [51:26<3:20:55, 2.02s/it] 20%|██ | 1530/7500 [51:28<3:20:35, 2.02s/it] {'loss': 0.4936, 'grad_norm': 5.18520450592041, 'learning_rate': 4.837064190990036e-05, 'logqs/rejected': -2.568042278289795, 'logqs/chosen': -0.9670203328132629, 'logqs_logps/rejected': -0.029004190117120743, 'logps_logqs/chosen': 0.23917794227600098, 'epoch': 0.41} 20%|██ | 1530/7500 [51:28<3:20:35, 2.02s/it] 20%|██ | 1531/7500 [51:30<3:18:40, 2.00s/it] 20%|██ | 1532/7500 [51:32<3:20:46, 2.02s/it] 20%|██ | 1533/7500 [51:34<3:23:58, 2.05s/it] 20%|██ | 1534/7500 [51:36<3:21:27, 2.03s/it] 20%|██ | 1535/7500 [51:38<3:21:39, 2.03s/it] 20%|██ | 1536/7500 [51:40<3:19:27, 2.01s/it] 20%|██ | 1537/7500 [51:42<3:19:36, 2.01s/it] 21%|██ | 1538/7500 [51:44<3:23:00, 2.04s/it] 21%|██ | 1539/7500 [51:47<3:28:04, 2.09s/it] 21%|██ | 1540/7500 [51:49<3:25:00, 2.06s/it] {'loss': 0.4727, 'grad_norm': 4.108314514160156, 'learning_rate': 4.832907036453647e-05, 'logqs/rejected': -2.673943042755127, 'logqs/chosen': -0.93951416015625, 'logqs_logps/rejected': -0.02835289016366005, 'logps_logqs/chosen': 0.24308772385120392, 'epoch': 0.41} 21%|██ | 1540/7500 [51:49<3:25:00, 2.06s/it] 21%|██ | 1541/7500 [51:50<3:21:37, 2.03s/it] 21%|██ | 1542/7500 [51:53<3:21:38, 2.03s/it] 21%|██ | 1543/7500 [51:55<3:22:58, 2.04s/it] 21%|██ | 1544/7500 [51:57<3:23:07, 2.05s/it] 21%|██ | 1545/7500 [51:59<3:21:38, 2.03s/it] 21%|██ | 1546/7500 [52:01<3:22:59, 2.05s/it] 21%|██ | 1547/7500 [52:03<3:24:45, 2.06s/it] 21%|██ | 1548/7500 [52:05<3:23:26, 2.05s/it] 21%|██ | 1549/7500 [52:07<3:24:35, 2.06s/it] 21%|██ | 1550/7500 [52:09<3:24:28, 2.06s/it] {'loss': 0.496, 'grad_norm': 4.550781726837158, 'learning_rate': 4.828699347315356e-05, 'logqs/rejected': -2.6437125205993652, 'logqs/chosen': -0.9556452631950378, 'logqs_logps/rejected': -0.023876164108514786, 'logps_logqs/chosen': 0.25033921003341675, 'epoch': 0.41} 21%|██ | 1550/7500 [52:09<3:24:28, 2.06s/it] 21%|██ | 1551/7500 [52:11<3:22:29, 2.04s/it] 21%|██ | 1552/7500 [52:13<3:21:20, 2.03s/it] 21%|██ | 1553/7500 [52:15<3:20:27, 2.02s/it] 21%|██ | 1554/7500 [52:17<3:19:07, 2.01s/it] 21%|██ | 1555/7500 [52:19<3:21:07, 2.03s/it] 21%|██ | 1556/7500 [52:21<3:20:46, 2.03s/it] 21%|██ | 1557/7500 [52:23<3:20:57, 2.03s/it] 21%|██ | 1558/7500 [52:25<3:19:51, 2.02s/it] 21%|██ | 1559/7500 [52:27<3:18:03, 2.00s/it] 21%|██ | 1560/7500 [52:29<3:19:01, 2.01s/it] {'loss': 0.4784, 'grad_norm': 4.983776092529297, 'learning_rate': 4.8244412147206284e-05, 'logqs/rejected': -2.6453845500946045, 'logqs/chosen': -0.9406522512435913, 'logqs_logps/rejected': -0.03915972635149956, 'logps_logqs/chosen': 0.25132226943969727, 'epoch': 0.42} 21%|██ | 1560/7500 [52:29<3:19:01, 2.01s/it] 21%|██ | 1561/7500 [52:31<3:18:24, 2.00s/it] 21%|██ | 1562/7500 [52:33<3:18:43, 2.01s/it] 21%|██ | 1563/7500 [52:35<3:21:16, 2.03s/it] 21%|██ | 1564/7500 [52:37<3:20:06, 2.02s/it] 21%|██ | 1565/7500 [52:39<3:20:17, 2.02s/it] 21%|██ | 1566/7500 [52:41<3:16:28, 1.99s/it] 21%|██ | 1567/7500 [52:43<3:15:46, 1.98s/it] 21%|██ | 1568/7500 [52:45<3:18:24, 2.01s/it] 21%|██ | 1569/7500 [52:47<3:17:06, 1.99s/it] 21%|██ | 1570/7500 [52:49<3:18:20, 2.01s/it] {'loss': 0.4833, 'grad_norm': 4.64738655090332, 'learning_rate': 4.820132730907617e-05, 'logqs/rejected': -2.53254771232605, 'logqs/chosen': -0.9758497476577759, 'logqs_logps/rejected': -0.026200344786047935, 'logps_logqs/chosen': 0.24738022685050964, 'epoch': 0.42} 21%|██ | 1570/7500 [52:49<3:18:20, 2.01s/it] 21%|██ | 1571/7500 [52:51<3:17:04, 1.99s/it] 21%|██ | 1572/7500 [52:53<3:16:52, 1.99s/it] 21%|██ | 1573/7500 [52:55<3:15:53, 1.98s/it] 21%|██ | 1574/7500 [52:57<3:21:00, 2.04s/it] 21%|██ | 1575/7500 [52:59<3:22:35, 2.05s/it] 21%|██ | 1576/7500 [53:01<3:23:34, 2.06s/it] 21%|██ | 1577/7500 [53:04<3:25:23, 2.08s/it] 21%|██ | 1578/7500 [53:05<3:20:48, 2.03s/it] 21%|██ | 1579/7500 [53:07<3:17:41, 2.00s/it] 21%|██ | 1580/7500 [53:10<3:21:00, 2.04s/it] {'loss': 0.4836, 'grad_norm': 4.955129623413086, 'learning_rate': 4.815773989205165e-05, 'logqs/rejected': -2.5931262969970703, 'logqs/chosen': -0.9559046626091003, 'logqs_logps/rejected': -0.034729938954114914, 'logps_logqs/chosen': 0.24732227623462677, 'epoch': 0.42} 21%|██ | 1580/7500 [53:10<3:21:00, 2.04s/it] 21%|██ | 1581/7500 [53:12<3:20:29, 2.03s/it] 21%|██ | 1582/7500 [53:13<3:16:17, 1.99s/it] 21%|██ | 1583/7500 [53:15<3:18:08, 2.01s/it] 21%|██ | 1584/7500 [53:18<3:19:07, 2.02s/it] 21%|██ | 1585/7500 [53:20<3:18:39, 2.02s/it] 21%|██ | 1586/7500 [53:22<3:21:33, 2.04s/it] 21%|██ | 1587/7500 [53:24<3:21:00, 2.04s/it] 21%|██ | 1588/7500 [53:26<3:21:50, 2.05s/it] 21%|██ | 1589/7500 [53:28<3:23:13, 2.06s/it] 21%|██ | 1590/7500 [53:30<3:21:30, 2.05s/it] {'loss': 0.4793, 'grad_norm': 4.41984748840332, 'learning_rate': 4.8113650840307834e-05, 'logqs/rejected': -2.421388864517212, 'logqs/chosen': -0.9566549062728882, 'logqs_logps/rejected': -0.029325256124138832, 'logps_logqs/chosen': 0.24455353617668152, 'epoch': 0.42} 21%|██ | 1590/7500 [53:30<3:21:30, 2.05s/it] 21%|██ | 1591/7500 [53:32<3:20:22, 2.03s/it] 21%|██ | 1592/7500 [53:34<3:20:44, 2.04s/it] 21%|██ | 1593/7500 [53:36<3:21:23, 2.05s/it] 21%|██▏ | 1594/7500 [53:38<3:21:42, 2.05s/it] 21%|██▏ | 1595/7500 [53:40<3:22:34, 2.06s/it] 21%|██▏ | 1596/7500 [53:42<3:22:36, 2.06s/it] 21%|██▏ | 1597/7500 [53:44<3:23:38, 2.07s/it] 21%|██▏ | 1598/7500 [53:46<3:20:11, 2.04s/it] 21%|██▏ | 1599/7500 [53:48<3:18:01, 2.01s/it] 21%|██▏ | 1600/7500 [53:50<3:18:21, 2.02s/it] {'loss': 0.5097, 'grad_norm': 5.226844787597656, 'learning_rate': 4.806906110888606e-05, 'logqs/rejected': -2.4878036975860596, 'logqs/chosen': -1.0009427070617676, 'logqs_logps/rejected': -0.01869298703968525, 'logps_logqs/chosen': 0.24945154786109924, 'epoch': 0.43} 21%|██▏ | 1600/7500 [53:50<3:18:21, 2.02s/it] 21%|██▏ | 1601/7500 [53:52<3:19:19, 2.03s/it] 21%|██▏ | 1602/7500 [53:54<3:17:16, 2.01s/it] 21%|██▏ | 1603/7500 [53:56<3:16:29, 2.00s/it] 21%|██▏ | 1604/7500 [53:58<3:20:20, 2.04s/it] 21%|██▏ | 1605/7500 [54:00<3:21:26, 2.05s/it] 21%|██▏ | 1606/7500 [54:02<3:19:35, 2.03s/it] 21%|██▏ | 1607/7500 [54:04<3:19:48, 2.03s/it] 21%|██▏ | 1608/7500 [54:06<3:19:24, 2.03s/it] 21%|██▏ | 1609/7500 [54:08<3:17:04, 2.01s/it] 21%|██▏ | 1610/7500 [54:10<3:16:23, 2.00s/it] {'loss': 0.478, 'grad_norm': 3.9513351917266846, 'learning_rate': 4.802397166367323e-05, 'logqs/rejected': -2.4997951984405518, 'logqs/chosen': -0.9463003277778625, 'logqs_logps/rejected': -0.02816052734851837, 'logps_logqs/chosen': 0.24328427016735077, 'epoch': 0.43} 21%|██▏ | 1610/7500 [54:10<3:16:23, 2.00s/it] 21%|██▏ | 1611/7500 [54:12<3:18:06, 2.02s/it] 21%|██▏ | 1612/7500 [54:14<3:18:10, 2.02s/it] 22%|██▏ | 1613/7500 [54:16<3:15:59, 2.00s/it] 22%|██▏ | 1614/7500 [54:18<3:15:54, 2.00s/it] 22%|██▏ | 1615/7500 [54:20<3:15:13, 1.99s/it] 22%|██▏ | 1616/7500 [54:22<3:16:03, 2.00s/it] 22%|██▏ | 1617/7500 [54:24<3:17:22, 2.01s/it] 22%|██▏ | 1618/7500 [54:27<3:18:59, 2.03s/it] 22%|██▏ | 1619/7500 [54:29<3:19:13, 2.03s/it] 22%|██▏ | 1620/7500 [54:31<3:19:27, 2.04s/it] {'loss': 0.4545, 'grad_norm': 4.427746295928955, 'learning_rate': 4.797838348138086e-05, 'logqs/rejected': -2.4485666751861572, 'logqs/chosen': -0.9115241765975952, 'logqs_logps/rejected': -0.036861248314380646, 'logps_logqs/chosen': 0.2495671808719635, 'epoch': 0.43} 22%|██▏ | 1620/7500 [54:31<3:19:27, 2.04s/it] 22%|██▏ | 1621/7500 [54:33<3:19:42, 2.04s/it] 22%|██▏ | 1622/7500 [54:35<3:20:18, 2.04s/it] 22%|██▏ | 1623/7500 [54:37<3:20:14, 2.04s/it] 22%|██▏ | 1624/7500 [54:39<3:18:22, 2.03s/it] 22%|██▏ | 1625/7500 [54:41<3:18:50, 2.03s/it] 22%|██▏ | 1626/7500 [54:43<3:17:30, 2.02s/it] 22%|██▏ | 1627/7500 [54:45<3:17:48, 2.02s/it] 22%|██▏ | 1628/7500 [54:47<3:17:48, 2.02s/it] 22%|██▏ | 1629/7500 [54:49<3:16:28, 2.01s/it] 22%|██▏ | 1630/7500 [54:51<3:17:27, 2.02s/it] {'loss': 0.5321, 'grad_norm': 4.228749752044678, 'learning_rate': 4.793229754952393e-05, 'logqs/rejected': -2.8605079650878906, 'logqs/chosen': -0.9782671928405762, 'logqs_logps/rejected': -0.020791638642549515, 'logps_logqs/chosen': 0.2568109631538391, 'epoch': 0.43} 22%|██▏ | 1630/7500 [54:51<3:17:27, 2.02s/it] 22%|██▏ | 1631/7500 [54:53<3:17:04, 2.01s/it] 22%|██▏ | 1632/7500 [54:55<3:16:42, 2.01s/it] 22%|██▏ | 1633/7500 [54:57<3:17:44, 2.02s/it] 22%|██▏ | 1634/7500 [54:59<3:17:39, 2.02s/it] 22%|██▏ | 1635/7500 [55:01<3:16:31, 2.01s/it] 22%|██▏ | 1636/7500 [55:03<3:15:49, 2.00s/it] 22%|██▏ | 1637/7500 [55:05<3:14:46, 1.99s/it] 22%|██▏ | 1638/7500 [55:07<3:15:20, 2.00s/it] 22%|██▏ | 1639/7500 [55:09<3:14:21, 1.99s/it] 22%|██▏ | 1640/7500 [55:11<3:18:07, 2.03s/it] {'loss': 0.4912, 'grad_norm': 4.453360557556152, 'learning_rate': 4.788571486639948e-05, 'logqs/rejected': -2.4737377166748047, 'logqs/chosen': -0.939926028251648, 'logqs_logps/rejected': -0.02154475450515747, 'logps_logqs/chosen': 0.24359416961669922, 'epoch': 0.44} 22%|██▏ | 1640/7500 [55:11<3:18:07, 2.03s/it] 22%|██▏ | 1641/7500 [55:13<3:18:53, 2.04s/it] 22%|██▏ | 1642/7500 [55:15<3:17:38, 2.02s/it] 22%|██▏ | 1643/7500 [55:17<3:16:57, 2.02s/it] 22%|██▏ | 1644/7500 [55:19<3:16:03, 2.01s/it] 22%|██▏ | 1645/7500 [55:21<3:17:17, 2.02s/it] 22%|██▏ | 1646/7500 [55:23<3:17:20, 2.02s/it] 22%|██▏ | 1647/7500 [55:25<3:18:09, 2.03s/it] 22%|██▏ | 1648/7500 [55:27<3:17:46, 2.03s/it] 22%|██▏ | 1649/7500 [55:29<3:18:20, 2.03s/it] 22%|██▏ | 1650/7500 [55:31<3:18:32, 2.04s/it] {'loss': 0.4895, 'grad_norm': 4.0961503982543945, 'learning_rate': 4.783863644106502e-05, 'logqs/rejected': -2.438183546066284, 'logqs/chosen': -0.9862468838691711, 'logqs_logps/rejected': -0.03049931488931179, 'logps_logqs/chosen': 0.244951993227005, 'epoch': 0.44} 22%|██▏ | 1650/7500 [55:31<3:18:32, 2.04s/it] 22%|██▏ | 1651/7500 [55:33<3:18:15, 2.03s/it] 22%|██▏ | 1652/7500 [55:35<3:15:03, 2.00s/it] 22%|██▏ | 1653/7500 [55:37<3:15:21, 2.00s/it] 22%|██▏ | 1654/7500 [55:39<3:12:45, 1.98s/it] 22%|██▏ | 1655/7500 [55:41<3:16:11, 2.01s/it] 22%|██▏ | 1656/7500 [55:43<3:15:11, 2.00s/it] 22%|██▏ | 1657/7500 [55:45<3:13:32, 1.99s/it] 22%|██▏ | 1658/7500 [55:47<3:13:48, 1.99s/it] 22%|██▏ | 1659/7500 [55:49<3:17:22, 2.03s/it] 22%|██▏ | 1660/7500 [55:51<3:16:51, 2.02s/it] {'loss': 0.4825, 'grad_norm': 4.330682277679443, 'learning_rate': 4.779106329331665e-05, 'logqs/rejected': -2.5161287784576416, 'logqs/chosen': -0.9447142481803894, 'logqs_logps/rejected': -0.03418523818254471, 'logps_logqs/chosen': 0.24120965600013733, 'epoch': 0.44} 22%|██▏ | 1660/7500 [55:51<3:16:51, 2.02s/it] 22%|██▏ | 1661/7500 [55:53<3:15:13, 2.01s/it] 22%|██▏ | 1662/7500 [55:55<3:11:56, 1.97s/it] 22%|██▏ | 1663/7500 [55:57<3:11:09, 1.96s/it] 22%|██▏ | 1664/7500 [55:59<3:13:37, 1.99s/it] 22%|██▏ | 1665/7500 [56:01<3:14:40, 2.00s/it] 22%|██▏ | 1666/7500 [56:03<3:16:14, 2.02s/it] 22%|██▏ | 1667/7500 [56:05<3:14:55, 2.00s/it] 22%|██▏ | 1668/7500 [56:07<3:15:01, 2.01s/it] 22%|██▏ | 1669/7500 [56:09<3:13:16, 1.99s/it] 22%|██▏ | 1670/7500 [56:11<3:12:35, 1.98s/it] {'loss': 0.4789, 'grad_norm': 4.205456256866455, 'learning_rate': 4.774299645366696e-05, 'logqs/rejected': -2.815821409225464, 'logqs/chosen': -0.9499031901359558, 'logqs_logps/rejected': -0.02583617903292179, 'logps_logqs/chosen': 0.25552719831466675, 'epoch': 0.45} 22%|██▏ | 1670/7500 [56:11<3:12:35, 1.98s/it] 22%|██▏ | 1671/7500 [56:13<3:12:07, 1.98s/it] 22%|██▏ | 1672/7500 [56:15<3:14:30, 2.00s/it] 22%|██▏ | 1673/7500 [56:17<3:13:02, 1.99s/it] 22%|██▏ | 1674/7500 [56:19<3:15:24, 2.01s/it] 22%|██▏ | 1675/7500 [56:21<3:14:18, 2.00s/it] 22%|██▏ | 1676/7500 [56:23<3:15:37, 2.02s/it] 22%|██▏ | 1677/7500 [56:25<3:14:11, 2.00s/it] 22%|██▏ | 1678/7500 [56:27<3:18:23, 2.04s/it] 22%|██▏ | 1679/7500 [56:29<3:19:11, 2.05s/it] 22%|██▏ | 1680/7500 [56:31<3:15:55, 2.02s/it] {'loss': 0.4567, 'grad_norm': 4.282940864562988, 'learning_rate': 4.769443696332272e-05, 'logqs/rejected': -2.3815298080444336, 'logqs/chosen': -0.9332104921340942, 'logqs_logps/rejected': -0.033469922840595245, 'logps_logqs/chosen': 0.23893673717975616, 'epoch': 0.45} 22%|██▏ | 1680/7500 [56:31<3:15:55, 2.02s/it] 22%|██▏ | 1681/7500 [56:33<3:18:23, 2.05s/it] 22%|██▏ | 1682/7500 [56:35<3:19:18, 2.06s/it] 22%|██▏ | 1683/7500 [56:38<3:20:12, 2.07s/it] 22%|██▏ | 1684/7500 [56:40<3:18:28, 2.05s/it] 22%|██▏ | 1685/7500 [56:42<3:21:06, 2.08s/it] 22%|██▏ | 1686/7500 [56:44<3:18:41, 2.05s/it] 22%|██▏ | 1687/7500 [56:46<3:19:02, 2.05s/it] 23%|██▎ | 1688/7500 [56:48<3:19:54, 2.06s/it] 23%|██▎ | 1689/7500 [56:50<3:18:04, 2.05s/it] 23%|██▎ | 1690/7500 [56:52<3:16:38, 2.03s/it] {'loss': 0.4737, 'grad_norm': 4.7792649269104, 'learning_rate': 4.764538587416233e-05, 'logqs/rejected': -2.642622232437134, 'logqs/chosen': -0.9480697512626648, 'logqs_logps/rejected': -0.02671549655497074, 'logps_logqs/chosen': 0.2471485584974289, 'epoch': 0.45} 23%|██▎ | 1690/7500 [56:52<3:16:38, 2.03s/it] 23%|██▎ | 1691/7500 [56:54<3:18:42, 2.05s/it] 23%|██▎ | 1692/7500 [56:56<3:16:02, 2.03s/it] 23%|██▎ | 1693/7500 [56:58<3:19:01, 2.06s/it] 23%|██▎ | 1694/7500 [57:00<3:19:02, 2.06s/it] 23%|██▎ | 1695/7500 [57:02<3:21:42, 2.08s/it] 23%|██▎ | 1696/7500 [57:04<3:23:15, 2.10s/it] 23%|██▎ | 1697/7500 [57:06<3:22:12, 2.09s/it] 23%|██▎ | 1698/7500 [57:08<3:19:07, 2.06s/it] 23%|██▎ | 1699/7500 [57:11<3:21:51, 2.09s/it] 23%|██▎ | 1700/7500 [57:13<3:20:02, 2.07s/it] {'loss': 0.4602, 'grad_norm': 4.734524726867676, 'learning_rate': 4.759584424871302e-05, 'logqs/rejected': -2.3979105949401855, 'logqs/chosen': -0.9357857704162598, 'logqs_logps/rejected': -0.03104742430150509, 'logps_logqs/chosen': 0.23813025653362274, 'epoch': 0.45} 23%|██▎ | 1700/7500 [57:13<3:20:02, 2.07s/it] 23%|██▎ | 1701/7500 [57:15<3:19:51, 2.07s/it] 23%|██▎ | 1702/7500 [57:17<3:16:15, 2.03s/it] 23%|██▎ | 1703/7500 [57:19<3:14:40, 2.01s/it] 23%|██▎ | 1704/7500 [57:21<3:17:41, 2.05s/it] 23%|██▎ | 1705/7500 [57:23<3:14:35, 2.01s/it] 23%|██▎ | 1706/7500 [57:25<3:12:44, 2.00s/it] 23%|██▎ | 1707/7500 [57:27<3:12:04, 1.99s/it] 23%|██▎ | 1708/7500 [57:29<3:10:39, 1.98s/it] 23%|██▎ | 1709/7500 [57:31<3:12:44, 2.00s/it] 23%|██▎ | 1710/7500 [57:33<3:10:39, 1.98s/it] {'loss': 0.5065, 'grad_norm': 8.005802154541016, 'learning_rate': 4.754581316012785e-05, 'logqs/rejected': -2.748656988143921, 'logqs/chosen': -0.9467251896858215, 'logqs_logps/rejected': -0.02200225368142128, 'logps_logqs/chosen': 0.24456104636192322, 'epoch': 0.46} 23%|██▎ | 1710/7500 [57:33<3:10:39, 1.98s/it] 23%|██▎ | 1711/7500 [57:35<3:12:59, 2.00s/it] 23%|██▎ | 1712/7500 [57:37<3:13:25, 2.01s/it] 23%|██▎ | 1713/7500 [57:39<3:13:38, 2.01s/it] 23%|██▎ | 1714/7500 [57:41<3:13:36, 2.01s/it] 23%|██▎ | 1715/7500 [57:43<3:13:14, 2.00s/it] 23%|██▎ | 1716/7500 [57:45<3:11:43, 1.99s/it] 23%|██▎ | 1717/7500 [57:47<3:14:36, 2.02s/it] 23%|██▎ | 1718/7500 [57:49<3:16:17, 2.04s/it] 23%|██▎ | 1719/7500 [57:51<3:14:49, 2.02s/it] 23%|██▎ | 1720/7500 [57:53<3:13:51, 2.01s/it] {'loss': 0.4715, 'grad_norm': 3.7422122955322266, 'learning_rate': 4.749529369216246e-05, 'logqs/rejected': -2.402656078338623, 'logqs/chosen': -0.9606073498725891, 'logqs_logps/rejected': -0.023937808349728584, 'logps_logqs/chosen': 0.24676954746246338, 'epoch': 0.46} 23%|██▎ | 1720/7500 [57:53<3:13:51, 2.01s/it] 23%|██▎ | 1721/7500 [57:55<3:15:37, 2.03s/it] 23%|██▎ | 1722/7500 [57:57<3:17:13, 2.05s/it] 23%|██▎ | 1723/7500 [57:59<3:13:48, 2.01s/it] 23%|██▎ | 1724/7500 [58:01<3:15:09, 2.03s/it] 23%|██▎ | 1725/7500 [58:03<3:14:35, 2.02s/it] 23%|██▎ | 1726/7500 [58:05<3:13:01, 2.01s/it] 23%|██▎ | 1727/7500 [58:07<3:15:19, 2.03s/it] 23%|██▎ | 1728/7500 [58:09<3:15:10, 2.03s/it] 23%|██▎ | 1729/7500 [58:11<3:15:56, 2.04s/it] 23%|██▎ | 1730/7500 [58:13<3:13:09, 2.01s/it] {'loss': 0.4695, 'grad_norm': 4.6967267990112305, 'learning_rate': 4.744428693915157e-05, 'logqs/rejected': -2.440857410430908, 'logqs/chosen': -0.9171786308288574, 'logqs_logps/rejected': -0.026238560676574707, 'logps_logqs/chosen': 0.2390623837709427, 'epoch': 0.46} 23%|██▎ | 1730/7500 [58:13<3:13:09, 2.01s/it] 23%|██▎ | 1731/7500 [58:15<3:14:23, 2.02s/it] 23%|██▎ | 1732/7500 [58:17<3:16:29, 2.04s/it] 23%|██▎ | 1733/7500 [58:19<3:17:58, 2.06s/it] 23%|██▎ | 1734/7500 [58:21<3:15:20, 2.03s/it] 23%|██▎ | 1735/7500 [58:23<3:18:13, 2.06s/it] 23%|██▎ | 1736/7500 [58:25<3:16:53, 2.05s/it] 23%|██▎ | 1737/7500 [58:27<3:17:50, 2.06s/it] 23%|██▎ | 1738/7500 [58:29<3:15:03, 2.03s/it] 23%|██▎ | 1739/7500 [58:31<3:16:23, 2.05s/it] 23%|██▎ | 1740/7500 [58:33<3:15:56, 2.04s/it] {'loss': 0.4723, 'grad_norm': 4.390007019042969, 'learning_rate': 4.7392794005985326e-05, 'logqs/rejected': -2.303920269012451, 'logqs/chosen': -0.9380515813827515, 'logqs_logps/rejected': -0.024816790595650673, 'logps_logqs/chosen': 0.23391950130462646, 'epoch': 0.46} 23%|██▎ | 1740/7500 [58:34<3:15:56, 2.04s/it] 23%|██▎ | 1741/7500 [58:36<3:16:05, 2.04s/it] 23%|██▎ | 1742/7500 [58:38<3:14:23, 2.03s/it] 23%|██▎ | 1743/7500 [58:39<3:13:04, 2.01s/it] 23%|██▎ | 1744/7500 [58:42<3:13:20, 2.02s/it] 23%|██▎ | 1745/7500 [58:44<3:14:53, 2.03s/it] 23%|██▎ | 1746/7500 [58:46<3:14:52, 2.03s/it] 23%|██▎ | 1747/7500 [58:48<3:14:52, 2.03s/it] 23%|██▎ | 1748/7500 [58:50<3:13:40, 2.02s/it] 23%|██▎ | 1749/7500 [58:52<3:17:45, 2.06s/it] 23%|██▎ | 1750/7500 [58:54<3:16:04, 2.05s/it] {'loss': 0.4695, 'grad_norm': 4.077282428741455, 'learning_rate': 4.734081600808531e-05, 'logqs/rejected': -2.660930871963501, 'logqs/chosen': -0.9209561347961426, 'logqs_logps/rejected': -0.029275301843881607, 'logps_logqs/chosen': 0.23960697650909424, 'epoch': 0.47} 23%|██▎ | 1750/7500 [58:54<3:16:04, 2.05s/it] 23%|██▎ | 1751/7500 [58:56<3:17:41, 2.06s/it] 23%|██▎ | 1752/7500 [58:58<3:16:30, 2.05s/it] 23%|██▎ | 1753/7500 [59:00<3:16:33, 2.05s/it] 23%|██▎ | 1754/7500 [59:02<3:14:34, 2.03s/it] 23%|██▎ | 1755/7500 [59:04<3:15:15, 2.04s/it] 23%|██▎ | 1756/7500 [59:06<3:16:29, 2.05s/it] 23%|██▎ | 1757/7500 [59:08<3:15:27, 2.04s/it] 23%|██▎ | 1758/7500 [59:10<3:13:42, 2.02s/it] 23%|██▎ | 1759/7500 [59:12<3:16:31, 2.05s/it] 23%|██▎ | 1760/7500 [59:14<3:12:29, 2.01s/it] {'loss': 0.4938, 'grad_norm': 4.200068473815918, 'learning_rate': 4.7288354071380415e-05, 'logqs/rejected': -2.525280714035034, 'logqs/chosen': -0.9390841722488403, 'logqs_logps/rejected': -0.02422340400516987, 'logps_logqs/chosen': 0.2390001267194748, 'epoch': 0.47} 23%|██▎ | 1760/7500 [59:14<3:12:29, 2.01s/it] 23%|██▎ | 1761/7500 [59:16<3:13:36, 2.02s/it] 23%|██▎ | 1762/7500 [59:18<3:14:26, 2.03s/it] 24%|██▎ | 1763/7500 [59:20<3:12:16, 2.01s/it] 24%|██▎ | 1764/7500 [59:22<3:10:37, 1.99s/it] 24%|██▎ | 1765/7500 [59:24<3:10:48, 2.00s/it] 24%|██▎ | 1766/7500 [59:26<3:12:49, 2.02s/it] 24%|██▎ | 1767/7500 [59:28<3:15:36, 2.05s/it] 24%|██▎ | 1768/7500 [59:30<3:17:37, 2.07s/it] 24%|██▎ | 1769/7500 [59:33<3:19:31, 2.09s/it] 24%|██▎ | 1770/7500 [59:35<3:18:17, 2.08s/it] {'loss': 0.4786, 'grad_norm': 4.148139953613281, 'learning_rate': 4.723540933228244e-05, 'logqs/rejected': -2.653831958770752, 'logqs/chosen': -0.9460858106613159, 'logqs_logps/rejected': -0.026024427264928818, 'logps_logqs/chosen': 0.24475157260894775, 'epoch': 0.47} 24%|██▎ | 1770/7500 [59:35<3:18:17, 2.08s/it] 24%|██▎ | 1771/7500 [59:37<3:18:14, 2.08s/it] 24%|██▎ | 1772/7500 [59:39<3:18:37, 2.08s/it] 24%|██▎ | 1773/7500 [59:41<3:18:54, 2.08s/it] 24%|██▎ | 1774/7500 [59:43<3:20:33, 2.10s/it] 24%|██▎ | 1775/7500 [59:45<3:21:38, 2.11s/it] 24%|██▎ | 1776/7500 [59:47<3:17:54, 2.07s/it] 24%|██▎ | 1777/7500 [59:49<3:15:50, 2.05s/it] 24%|██▎ | 1778/7500 [59:51<3:15:29, 2.05s/it] 24%|██▎ | 1779/7500 [59:53<3:13:22, 2.03s/it] 24%|██▎ | 1780/7500 [59:55<3:11:41, 2.01s/it] {'loss': 0.4861, 'grad_norm': 3.605621099472046, 'learning_rate': 4.718198293766148e-05, 'logqs/rejected': -2.600931406021118, 'logqs/chosen': -0.9334843754768372, 'logqs_logps/rejected': -0.027122149243950844, 'logps_logqs/chosen': 0.24502603709697723, 'epoch': 0.47} 24%|██▎ | 1780/7500 [59:55<3:11:41, 2.01s/it] 24%|██▎ | 1781/7500 [59:57<3:14:32, 2.04s/it] 24%|██▍ | 1782/7500 [59:59<3:13:13, 2.03s/it] 24%|██▍ | 1783/7500 [1:00:02<3:18:45, 2.09s/it] 24%|██▍ | 1784/7500 [1:00:03<3:16:01, 2.06s/it] 24%|██▍ | 1785/7500 [1:00:06<3:17:12, 2.07s/it] 24%|██▍ | 1786/7500 [1:00:08<3:15:44, 2.06s/it] 24%|██▍ | 1787/7500 [1:00:10<3:17:34, 2.08s/it] 24%|██▍ | 1788/7500 [1:00:12<3:15:50, 2.06s/it] 24%|██▍ | 1789/7500 [1:00:14<3:13:53, 2.04s/it] 24%|██▍ | 1790/7500 [1:00:16<3:11:00, 2.01s/it] {'loss': 0.4906, 'grad_norm': 3.129695415496826, 'learning_rate': 4.712807604482108e-05, 'logqs/rejected': -2.894099712371826, 'logqs/chosen': -0.9409432411193848, 'logqs_logps/rejected': -0.024179408326745033, 'logps_logqs/chosen': 0.24047712981700897, 'epoch': 0.48} 24%|██▍ | 1790/7500 [1:00:16<3:11:00, 2.01s/it] 24%|██▍ | 1791/7500 [1:00:18<3:09:16, 1.99s/it] 24%|██▍ | 1792/7500 [1:00:20<3:10:15, 2.00s/it] 24%|██▍ | 1793/7500 [1:00:22<3:11:17, 2.01s/it] 24%|██▍ | 1794/7500 [1:00:24<3:12:23, 2.02s/it] 24%|██▍ | 1795/7500 [1:00:26<3:13:56, 2.04s/it] 24%|██▍ | 1796/7500 [1:00:28<3:10:29, 2.00s/it] 24%|██▍ | 1797/7500 [1:00:30<3:10:13, 2.00s/it] 24%|██▍ | 1798/7500 [1:00:32<3:12:14, 2.02s/it] 24%|██▍ | 1799/7500 [1:00:34<3:12:29, 2.03s/it] 24%|██▍ | 1800/7500 [1:00:36<3:13:08, 2.03s/it] {'loss': 0.4863, 'grad_norm': 5.328497409820557, 'learning_rate': 4.707368982147318e-05, 'logqs/rejected': -2.6998579502105713, 'logqs/chosen': -0.9338487386703491, 'logqs_logps/rejected': -0.02298353984951973, 'logps_logqs/chosen': 0.23442597687244415, 'epoch': 0.48} 24%|██▍ | 1800/7500 [1:00:36<3:13:08, 2.03s/it] 24%|██▍ | 1801/7500 [1:00:38<3:16:04, 2.06s/it] 24%|██▍ | 1802/7500 [1:00:40<3:15:44, 2.06s/it] 24%|██▍ | 1803/7500 [1:00:42<3:18:41, 2.09s/it] 24%|██▍ | 1804/7500 [1:00:44<3:16:27, 2.07s/it] 24%|██▍ | 1805/7500 [1:00:46<3:16:32, 2.07s/it] 24%|██▍ | 1806/7500 [1:00:48<3:12:55, 2.03s/it] 24%|██▍ | 1807/7500 [1:00:50<3:11:26, 2.02s/it] 24%|██▍ | 1808/7500 [1:00:52<3:11:53, 2.02s/it] 24%|██▍ | 1809/7500 [1:00:54<3:12:46, 2.03s/it] 24%|██▍ | 1810/7500 [1:00:56<3:11:43, 2.02s/it] {'loss': 0.4502, 'grad_norm': 3.5935049057006836, 'learning_rate': 4.701882544571277e-05, 'logqs/rejected': -2.3221354484558105, 'logqs/chosen': -0.9028006792068481, 'logqs_logps/rejected': -0.032348595559597015, 'logps_logqs/chosen': 0.23251144587993622, 'epoch': 0.48} 24%|██▍ | 1810/7500 [1:00:56<3:11:43, 2.02s/it] 24%|██▍ | 1811/7500 [1:00:58<3:11:36, 2.02s/it] 24%|██▍ | 1812/7500 [1:01:00<3:09:43, 2.00s/it] 24%|██▍ | 1813/7500 [1:01:02<3:10:55, 2.01s/it] 24%|██▍ | 1814/7500 [1:01:04<3:10:33, 2.01s/it] 24%|██▍ | 1815/7500 [1:01:06<3:11:34, 2.02s/it] 24%|██▍ | 1816/7500 [1:01:09<3:14:42, 2.06s/it] 24%|██▍ | 1817/7500 [1:01:10<3:10:33, 2.01s/it] 24%|██▍ | 1818/7500 [1:01:12<3:08:20, 1.99s/it] 24%|██▍ | 1819/7500 [1:01:14<3:08:31, 1.99s/it] 24%|██▍ | 1820/7500 [1:01:16<3:08:38, 1.99s/it] {'loss': 0.47, 'grad_norm': 3.9710988998413086, 'learning_rate': 4.696348410599244e-05, 'logqs/rejected': -2.468906879425049, 'logqs/chosen': -0.9261810183525085, 'logqs_logps/rejected': -0.026069944724440575, 'logps_logqs/chosen': 0.23826995491981506, 'epoch': 0.49} 24%|██▍ | 1820/7500 [1:01:16<3:08:38, 1.99s/it] 24%|██▍ | 1821/7500 [1:01:18<3:08:57, 2.00s/it] 24%|██▍ | 1822/7500 [1:01:20<3:08:24, 1.99s/it] 24%|██▍ | 1823/7500 [1:01:22<3:11:51, 2.03s/it] 24%|██▍ | 1824/7500 [1:01:25<3:12:20, 2.03s/it] 24%|██▍ | 1825/7500 [1:01:26<3:09:35, 2.00s/it] 24%|██▍ | 1826/7500 [1:01:29<3:11:27, 2.02s/it] 24%|██▍ | 1827/7500 [1:01:31<3:10:35, 2.02s/it] 24%|██▍ | 1828/7500 [1:01:33<3:12:53, 2.04s/it] 24%|██▍ | 1829/7500 [1:01:35<3:12:09, 2.03s/it] 24%|██▍ | 1830/7500 [1:01:37<3:13:08, 2.04s/it] {'loss': 0.4801, 'grad_norm': 4.191669940948486, 'learning_rate': 4.690766700109659e-05, 'logqs/rejected': -2.621019124984741, 'logqs/chosen': -0.9519068002700806, 'logqs_logps/rejected': -0.024763744324445724, 'logps_logqs/chosen': 0.24174180626869202, 'epoch': 0.49} 24%|██▍ | 1830/7500 [1:01:37<3:13:08, 2.04s/it] 24%|██▍ | 1831/7500 [1:01:39<3:12:31, 2.04s/it] 24%|██▍ | 1832/7500 [1:01:41<3:13:40, 2.05s/it] 24%|██▍ | 1833/7500 [1:01:43<3:11:36, 2.03s/it] 24%|██▍ | 1834/7500 [1:01:45<3:09:01, 2.00s/it] 24%|██▍ | 1835/7500 [1:01:47<3:12:45, 2.04s/it] 24%|██▍ | 1836/7500 [1:01:49<3:13:16, 2.05s/it] 24%|██▍ | 1837/7500 [1:01:51<3:13:11, 2.05s/it] 25%|██▍ | 1838/7500 [1:01:53<3:09:24, 2.01s/it] 25%|██▍ | 1839/7500 [1:01:55<3:11:35, 2.03s/it] 25%|██▍ | 1840/7500 [1:01:57<3:10:47, 2.02s/it] {'loss': 0.485, 'grad_norm': 3.742269277572632, 'learning_rate': 4.685137534011549e-05, 'logqs/rejected': -2.5519015789031982, 'logqs/chosen': -0.9675599932670593, 'logqs_logps/rejected': -0.025324055925011635, 'logps_logqs/chosen': 0.23983824253082275, 'epoch': 0.49} 25%|██▍ | 1840/7500 [1:01:57<3:10:47, 2.02s/it] 25%|██▍ | 1841/7500 [1:01:59<3:09:52, 2.01s/it] 25%|██▍ | 1842/7500 [1:02:01<3:08:15, 2.00s/it] 25%|██▍ | 1843/7500 [1:02:03<3:11:15, 2.03s/it] 25%|██▍ | 1844/7500 [1:02:05<3:13:22, 2.05s/it] 25%|██▍ | 1845/7500 [1:02:07<3:11:29, 2.03s/it] 25%|██▍ | 1846/7500 [1:02:09<3:12:50, 2.05s/it] 25%|██▍ | 1847/7500 [1:02:11<3:12:58, 2.05s/it] 25%|██▍ | 1848/7500 [1:02:13<3:11:15, 2.03s/it] 25%|██▍ | 1849/7500 [1:02:15<3:08:55, 2.01s/it] 25%|██▍ | 1850/7500 [1:02:17<3:08:49, 2.01s/it] {'loss': 0.4826, 'grad_norm': 3.7082417011260986, 'learning_rate': 4.679461034241906e-05, 'logqs/rejected': -2.471618175506592, 'logqs/chosen': -0.9657098054885864, 'logqs_logps/rejected': -0.024114176630973816, 'logps_logqs/chosen': 0.2536889910697937, 'epoch': 0.49} 25%|██▍ | 1850/7500 [1:02:17<3:08:49, 2.01s/it] 25%|██▍ | 1851/7500 [1:02:19<3:10:51, 2.03s/it] 25%|██▍ | 1852/7500 [1:02:21<3:10:52, 2.03s/it] 25%|██▍ | 1853/7500 [1:02:23<3:09:34, 2.01s/it] 25%|██▍ | 1854/7500 [1:02:25<3:09:18, 2.01s/it] 25%|██▍ | 1855/7500 [1:02:27<3:08:44, 2.01s/it] 25%|██▍ | 1856/7500 [1:02:29<3:08:28, 2.00s/it] 25%|██▍ | 1857/7500 [1:02:31<3:12:44, 2.05s/it] 25%|██▍ | 1858/7500 [1:02:33<3:11:36, 2.04s/it] 25%|██▍ | 1859/7500 [1:02:36<3:12:41, 2.05s/it] 25%|██▍ | 1860/7500 [1:02:38<3:12:21, 2.05s/it] {'loss': 0.4949, 'grad_norm': 4.070802688598633, 'learning_rate': 4.6737373237630476e-05, 'logqs/rejected': -2.7180447578430176, 'logqs/chosen': -0.9371684789657593, 'logqs_logps/rejected': -0.02089984156191349, 'logps_logqs/chosen': 0.24269667267799377, 'epoch': 0.5} 25%|██▍ | 1860/7500 [1:02:38<3:12:21, 2.05s/it] 25%|██▍ | 1861/7500 [1:02:40<3:12:24, 2.05s/it] 25%|██▍ | 1862/7500 [1:02:42<3:11:32, 2.04s/it] 25%|██▍ | 1863/7500 [1:02:44<3:11:41, 2.04s/it] 25%|██▍ | 1864/7500 [1:02:46<3:09:25, 2.02s/it] 25%|██▍ | 1865/7500 [1:02:48<3:11:40, 2.04s/it] 25%|██▍ | 1866/7500 [1:02:50<3:10:43, 2.03s/it] 25%|██▍ | 1867/7500 [1:02:52<3:13:31, 2.06s/it] 25%|██▍ | 1868/7500 [1:02:54<3:10:39, 2.03s/it] 25%|██▍ | 1869/7500 [1:02:56<3:10:19, 2.03s/it] 25%|██▍ | 1870/7500 [1:02:58<3:10:54, 2.03s/it] {'loss': 0.4751, 'grad_norm': 4.502434730529785, 'learning_rate': 4.667966526559953e-05, 'logqs/rejected': -2.523489236831665, 'logqs/chosen': -0.9110727310180664, 'logqs_logps/rejected': -0.029919832944869995, 'logps_logqs/chosen': 0.2401018887758255, 'epoch': 0.5} 25%|██▍ | 1870/7500 [1:02:58<3:10:54, 2.03s/it] 25%|██▍ | 1871/7500 [1:03:00<3:10:48, 2.03s/it] 25%|██▍ | 1872/7500 [1:03:02<3:09:42, 2.02s/it] 25%|██▍ | 1873/7500 [1:03:04<3:09:22, 2.02s/it] 25%|██▍ | 1874/7500 [1:03:06<3:10:21, 2.03s/it] 25%|██▌ | 1875/7500 [1:03:08<3:10:25, 2.03s/it] 25%|██▌ | 1876/7500 [1:03:10<3:09:50, 2.03s/it] 25%|██▌ | 1877/7500 [1:03:12<3:09:38, 2.02s/it] 25%|██▌ | 1878/7500 [1:03:14<3:09:47, 2.03s/it] 25%|██▌ | 1879/7500 [1:03:16<3:08:27, 2.01s/it] 25%|██▌ | 1880/7500 [1:03:18<3:08:21, 2.01s/it] {'loss': 0.4724, 'grad_norm': 3.837635040283203, 'learning_rate': 4.6621487676375777e-05, 'logqs/rejected': -2.367366313934326, 'logqs/chosen': -0.9202126264572144, 'logqs_logps/rejected': -0.03792247548699379, 'logps_logqs/chosen': 0.23906859755516052, 'epoch': 0.5} 25%|██▌ | 1880/7500 [1:03:18<3:08:21, 2.01s/it] 25%|██▌ | 1881/7500 [1:03:20<3:10:51, 2.04s/it] 25%|██▌ | 1882/7500 [1:03:22<3:12:18, 2.05s/it] 25%|██▌ | 1883/7500 [1:03:24<3:11:39, 2.05s/it] 25%|██▌ | 1884/7500 [1:03:26<3:10:25, 2.03s/it] 25%|██▌ | 1885/7500 [1:03:28<3:13:01, 2.06s/it] 25%|██▌ | 1886/7500 [1:03:30<3:09:42, 2.03s/it] 25%|██▌ | 1887/7500 [1:03:32<3:11:34, 2.05s/it] 25%|██▌ | 1888/7500 [1:03:34<3:10:01, 2.03s/it] 25%|██▌ | 1889/7500 [1:03:37<3:10:45, 2.04s/it] 25%|██▌ | 1890/7500 [1:03:38<3:07:16, 2.00s/it] {'loss': 0.4761, 'grad_norm': 3.9116015434265137, 'learning_rate': 4.656284173018144e-05, 'logqs/rejected': -2.6350455284118652, 'logqs/chosen': -0.932475209236145, 'logqs_logps/rejected': -0.027783740311861038, 'logps_logqs/chosen': 0.24848926067352295, 'epoch': 0.5} 25%|██▌ | 1890/7500 [1:03:38<3:07:16, 2.00s/it] 25%|██▌ | 1891/7500 [1:03:40<3:05:55, 1.99s/it] 25%|██▌ | 1892/7500 [1:03:42<3:05:16, 1.98s/it] 25%|██▌ | 1893/7500 [1:03:44<3:05:31, 1.99s/it] 25%|██▌ | 1894/7500 [1:03:46<3:07:04, 2.00s/it] 25%|██▌ | 1895/7500 [1:03:48<3:05:33, 1.99s/it] 25%|██▌ | 1896/7500 [1:03:50<3:03:02, 1.96s/it] 25%|██▌ | 1897/7500 [1:03:52<3:08:25, 2.02s/it] 25%|██▌ | 1898/7500 [1:03:54<3:09:55, 2.03s/it] 25%|██▌ | 1899/7500 [1:03:57<3:11:51, 2.06s/it] 25%|██▌ | 1900/7500 [1:03:59<3:11:21, 2.05s/it] {'loss': 0.467, 'grad_norm': 4.293906211853027, 'learning_rate': 4.650372869738414e-05, 'logqs/rejected': -2.6063246726989746, 'logqs/chosen': -0.9449762105941772, 'logqs_logps/rejected': -0.028963839635252953, 'logps_logqs/chosen': 0.24054868519306183, 'epoch': 0.51} 25%|██▌ | 1900/7500 [1:03:59<3:11:21, 2.05s/it] 25%|██▌ | 1901/7500 [1:04:01<3:08:25, 2.02s/it] 25%|██▌ | 1902/7500 [1:04:03<3:07:14, 2.01s/it] 25%|██▌ | 1903/7500 [1:04:05<3:08:28, 2.02s/it] 25%|██▌ | 1904/7500 [1:04:07<3:08:22, 2.02s/it] 25%|██▌ | 1905/7500 [1:04:09<3:04:37, 1.98s/it] 25%|██▌ | 1906/7500 [1:04:10<3:04:29, 1.98s/it] 25%|██▌ | 1907/7500 [1:04:13<3:06:15, 2.00s/it] 25%|██▌ | 1908/7500 [1:04:15<3:08:33, 2.02s/it] 25%|██▌ | 1909/7500 [1:04:17<3:09:25, 2.03s/it] 25%|██▌ | 1910/7500 [1:04:19<3:11:08, 2.05s/it] {'loss': 0.461, 'grad_norm': 4.662118911743164, 'learning_rate': 4.644414985846934e-05, 'logqs/rejected': -2.3163299560546875, 'logqs/chosen': -0.9475654363632202, 'logqs_logps/rejected': -0.031699471175670624, 'logps_logqs/chosen': 0.24346952140331268, 'epoch': 0.51} 25%|██▌ | 1910/7500 [1:04:19<3:11:08, 2.05s/it] 25%|██▌ | 1911/7500 [1:04:21<3:11:22, 2.05s/it] 25%|██▌ | 1912/7500 [1:04:23<3:09:04, 2.03s/it] 26%|██▌ | 1913/7500 [1:04:25<3:06:40, 2.00s/it] 26%|██▌ | 1914/7500 [1:04:27<3:07:55, 2.02s/it] 26%|██▌ | 1915/7500 [1:04:29<3:10:43, 2.05s/it] 26%|██▌ | 1916/7500 [1:04:31<3:08:51, 2.03s/it] 26%|██▌ | 1917/7500 [1:04:33<3:07:17, 2.01s/it] 26%|██▌ | 1918/7500 [1:04:35<3:06:13, 2.00s/it] 26%|██▌ | 1919/7500 [1:04:37<3:10:00, 2.04s/it] 26%|██▌ | 1920/7500 [1:04:39<3:07:04, 2.01s/it] {'loss': 0.459, 'grad_norm': 3.701860189437866, 'learning_rate': 4.638410650401267e-05, 'logqs/rejected': -2.4963560104370117, 'logqs/chosen': -0.9200960397720337, 'logqs_logps/rejected': -0.029366130009293556, 'logps_logqs/chosen': 0.2330722063779831, 'epoch': 0.51} 26%|██▌ | 1920/7500 [1:04:39<3:07:04, 2.01s/it] 26%|██▌ | 1921/7500 [1:04:41<3:08:54, 2.03s/it] 26%|██▌ | 1922/7500 [1:04:43<3:09:05, 2.03s/it] 26%|██▌ | 1923/7500 [1:04:45<3:10:21, 2.05s/it] 26%|██▌ | 1924/7500 [1:04:47<3:12:52, 2.08s/it] 26%|██▌ | 1925/7500 [1:04:49<3:12:50, 2.08s/it] 26%|██▌ | 1926/7500 [1:04:51<3:10:20, 2.05s/it] 26%|██▌ | 1927/7500 [1:04:53<3:09:24, 2.04s/it] 26%|██▌ | 1928/7500 [1:04:55<3:09:02, 2.04s/it] 26%|██▌ | 1929/7500 [1:04:57<3:09:45, 2.04s/it] 26%|██▌ | 1930/7500 [1:04:59<3:07:31, 2.02s/it] {'loss': 0.4846, 'grad_norm': 4.293872356414795, 'learning_rate': 4.632359993465188e-05, 'logqs/rejected': -2.577073335647583, 'logqs/chosen': -0.9673656225204468, 'logqs_logps/rejected': -0.03085305728018284, 'logps_logqs/chosen': 0.24451501667499542, 'epoch': 0.51} 26%|██▌ | 1930/7500 [1:04:59<3:07:31, 2.02s/it] 26%|██▌ | 1931/7500 [1:05:01<3:04:10, 1.98s/it] 26%|██▌ | 1932/7500 [1:05:03<3:03:34, 1.98s/it] 26%|██▌ | 1933/7500 [1:05:05<3:05:39, 2.00s/it] 26%|██▌ | 1934/7500 [1:05:07<3:04:25, 1.99s/it] 26%|██▌ | 1935/7500 [1:05:09<3:05:06, 2.00s/it] 26%|██▌ | 1936/7500 [1:05:11<3:04:58, 1.99s/it] 26%|██▌ | 1937/7500 [1:05:13<3:05:15, 2.00s/it] 26%|██▌ | 1938/7500 [1:05:15<3:07:11, 2.02s/it] 26%|██▌ | 1939/7500 [1:05:17<3:07:29, 2.02s/it] 26%|██▌ | 1940/7500 [1:05:19<3:06:13, 2.01s/it] {'loss': 0.4411, 'grad_norm': 5.563578128814697, 'learning_rate': 4.626263146105875e-05, 'logqs/rejected': -2.3618533611297607, 'logqs/chosen': -0.8927162289619446, 'logqs_logps/rejected': -0.030728856101632118, 'logps_logqs/chosen': 0.22902338206768036, 'epoch': 0.52} 26%|██▌ | 1940/7500 [1:05:19<3:06:13, 2.01s/it] 26%|██▌ | 1941/7500 [1:05:21<3:03:53, 1.98s/it] 26%|██▌ | 1942/7500 [1:05:23<3:04:33, 1.99s/it] 26%|██▌ | 1943/7500 [1:05:25<3:04:57, 2.00s/it] 26%|██▌ | 1944/7500 [1:05:27<3:03:05, 1.98s/it] 26%|██▌ | 1945/7500 [1:05:29<3:03:29, 1.98s/it] 26%|██▌ | 1946/7500 [1:05:31<3:02:22, 1.97s/it] 26%|██▌ | 1947/7500 [1:05:33<3:03:29, 1.98s/it] 26%|██▌ | 1948/7500 [1:05:35<3:05:53, 2.01s/it] 26%|██▌ | 1949/7500 [1:05:37<3:03:41, 1.99s/it] 26%|██▌ | 1950/7500 [1:05:39<3:05:52, 2.01s/it] {'loss': 0.4824, 'grad_norm': 3.9794962406158447, 'learning_rate': 4.620120240391065e-05, 'logqs/rejected': -2.6782162189483643, 'logqs/chosen': -0.9190009832382202, 'logqs_logps/rejected': -0.018002372235059738, 'logps_logqs/chosen': 0.22804148495197296, 'epoch': 0.52} 26%|██▌ | 1950/7500 [1:05:39<3:05:52, 2.01s/it] 26%|██▌ | 1951/7500 [1:05:41<3:07:32, 2.03s/it] 26%|██▌ | 1952/7500 [1:05:43<3:08:28, 2.04s/it] 26%|██▌ | 1953/7500 [1:05:45<3:07:04, 2.02s/it] 26%|██▌ | 1954/7500 [1:05:47<3:07:18, 2.03s/it] 26%|██▌ | 1955/7500 [1:05:49<3:06:47, 2.02s/it] 26%|██▌ | 1956/7500 [1:05:51<3:07:16, 2.03s/it] 26%|██▌ | 1957/7500 [1:05:54<3:08:56, 2.05s/it] 26%|██▌ | 1958/7500 [1:05:56<3:11:07, 2.07s/it] 26%|██▌ | 1959/7500 [1:05:58<3:08:07, 2.04s/it] 26%|██▌ | 1960/7500 [1:06:00<3:07:46, 2.03s/it] {'loss': 0.4744, 'grad_norm': 3.8550708293914795, 'learning_rate': 4.613931409386196e-05, 'logqs/rejected': -2.4425511360168457, 'logqs/chosen': -0.9969555139541626, 'logqs_logps/rejected': -0.02867310121655464, 'logps_logqs/chosen': 0.24250635504722595, 'epoch': 0.52} 26%|██▌ | 1960/7500 [1:06:00<3:07:46, 2.03s/it] 26%|██▌ | 1961/7500 [1:06:02<3:06:05, 2.02s/it] 26%|██▌ | 1962/7500 [1:06:04<3:08:49, 2.05s/it] 26%|██▌ | 1963/7500 [1:06:06<3:07:05, 2.03s/it] 26%|██▌ | 1964/7500 [1:06:08<3:09:36, 2.05s/it] 26%|██▌ | 1965/7500 [1:06:10<3:07:04, 2.03s/it] 26%|██▌ | 1966/7500 [1:06:12<3:05:32, 2.01s/it] 26%|██▌ | 1967/7500 [1:06:14<3:04:49, 2.00s/it] 26%|██▌ | 1968/7500 [1:06:16<3:05:05, 2.01s/it] 26%|██▋ | 1969/7500 [1:06:18<3:03:33, 1.99s/it] 26%|██▋ | 1970/7500 [1:06:20<3:04:12, 2.00s/it] {'loss': 0.4628, 'grad_norm': 4.0095038414001465, 'learning_rate': 4.607696787151522e-05, 'logqs/rejected': -2.5462818145751953, 'logqs/chosen': -0.9412193298339844, 'logqs_logps/rejected': -0.022385014221072197, 'logps_logqs/chosen': 0.248204305768013, 'epoch': 0.53} 26%|██▋ | 1970/7500 [1:06:20<3:04:12, 2.00s/it] 26%|██▋ | 1971/7500 [1:06:22<3:04:42, 2.00s/it] 26%|██▋ | 1972/7500 [1:06:24<3:06:13, 2.02s/it] 26%|██▋ | 1973/7500 [1:06:26<3:08:35, 2.05s/it] 26%|██▋ | 1974/7500 [1:06:28<3:07:36, 2.04s/it] 26%|██▋ | 1975/7500 [1:06:30<3:05:26, 2.01s/it] 26%|██▋ | 1976/7500 [1:06:32<3:05:18, 2.01s/it] 26%|██▋ | 1977/7500 [1:06:34<3:05:37, 2.02s/it] 26%|██▋ | 1978/7500 [1:06:36<3:08:01, 2.04s/it] 26%|██▋ | 1979/7500 [1:06:38<3:05:40, 2.02s/it] 26%|██▋ | 1980/7500 [1:06:40<3:06:18, 2.03s/it] {'loss': 0.4503, 'grad_norm': 4.028712272644043, 'learning_rate': 4.601416508739211e-05, 'logqs/rejected': -2.6005256175994873, 'logqs/chosen': -0.9112085103988647, 'logqs_logps/rejected': -0.03259218484163284, 'logps_logqs/chosen': 0.2380942404270172, 'epoch': 0.53} 26%|██▋ | 1980/7500 [1:06:40<3:06:18, 2.03s/it] 26%|██▋ | 1981/7500 [1:06:42<3:06:13, 2.02s/it] 26%|██▋ | 1982/7500 [1:06:44<3:06:20, 2.03s/it] 26%|██▋ | 1983/7500 [1:06:46<3:04:25, 2.01s/it] 26%|██▋ | 1984/7500 [1:06:48<3:06:40, 2.03s/it] 26%|██▋ | 1985/7500 [1:06:50<3:03:36, 2.00s/it] 26%|██▋ | 1986/7500 [1:06:52<3:02:02, 1.98s/it] 26%|██▋ | 1987/7500 [1:06:54<3:04:06, 2.00s/it] 27%|██▋ | 1988/7500 [1:06:56<3:02:33, 1.99s/it] 27%|██▋ | 1989/7500 [1:06:58<3:01:29, 1.98s/it] 27%|██▋ | 1990/7500 [1:07:00<3:01:31, 1.98s/it] {'loss': 0.4728, 'grad_norm': 3.773196220397949, 'learning_rate': 4.595090710190418e-05, 'logqs/rejected': -2.63914155960083, 'logqs/chosen': -0.9375730752944946, 'logqs_logps/rejected': -0.020478790625929832, 'logps_logqs/chosen': 0.23608693480491638, 'epoch': 0.53} 27%|██▋ | 1990/7500 [1:07:00<3:01:31, 1.98s/it] 27%|██▋ | 1991/7500 [1:07:02<3:00:32, 1.97s/it] 27%|██▋ | 1992/7500 [1:07:04<3:01:49, 1.98s/it] 27%|██▋ | 1993/7500 [1:07:06<3:00:50, 1.97s/it] 27%|██▋ | 1994/7500 [1:07:08<3:00:50, 1.97s/it] 27%|██▋ | 1995/7500 [1:07:10<3:03:11, 2.00s/it] 27%|██▋ | 1996/7500 [1:07:12<3:05:52, 2.03s/it] 27%|██▋ | 1997/7500 [1:07:14<3:06:06, 2.03s/it] 27%|██▋ | 1998/7500 [1:07:16<3:06:53, 2.04s/it] 27%|██▋ | 1999/7500 [1:07:18<3:09:24, 2.07s/it] 27%|██▋ | 2000/7500 [1:07:20<3:06:44, 2.04s/it] {'loss': 0.4456, 'grad_norm': 3.874598979949951, 'learning_rate': 4.588719528532342e-05, 'logqs/rejected': -2.25303316116333, 'logqs/chosen': -0.9256588816642761, 'logqs_logps/rejected': -0.0306180901825428, 'logps_logqs/chosen': 0.2401019036769867, 'epoch': 0.53} 27%|██▋ | 2000/7500 [1:07:20<3:06:44, 2.04s/it] 27%|██▋ | 2001/7500 [1:07:22<3:05:02, 2.02s/it] 27%|██▋ | 2002/7500 [1:07:24<3:04:00, 2.01s/it] 27%|██▋ | 2003/7500 [1:07:26<3:05:19, 2.02s/it] 27%|██▋ | 2004/7500 [1:07:28<3:06:00, 2.03s/it] 27%|██▋ | 2005/7500 [1:07:30<3:04:02, 2.01s/it] 27%|██▋ | 2006/7500 [1:07:32<3:04:00, 2.01s/it] 27%|██▋ | 2007/7500 [1:07:34<3:07:08, 2.04s/it] 27%|██▋ | 2008/7500 [1:07:36<3:04:04, 2.01s/it] 27%|██▋ | 2009/7500 [1:07:38<3:05:30, 2.03s/it] 27%|██▋ | 2010/7500 [1:07:40<3:05:33, 2.03s/it] {'loss': 0.4748, 'grad_norm': 4.0262250900268555, 'learning_rate': 4.5823031017752485e-05, 'logqs/rejected': -2.504042148590088, 'logqs/chosen': -0.9462403059005737, 'logqs_logps/rejected': -0.030298154801130295, 'logps_logqs/chosen': 0.23690590262413025, 'epoch': 0.54} 27%|██▋ | 2010/7500 [1:07:40<3:05:33, 2.03s/it] 27%|██▋ | 2011/7500 [1:07:42<3:06:14, 2.04s/it] 27%|██▋ | 2012/7500 [1:07:45<3:08:09, 2.06s/it] 27%|██▋ | 2013/7500 [1:07:47<3:06:29, 2.04s/it] 27%|██▋ | 2014/7500 [1:07:49<3:06:36, 2.04s/it] 27%|██▋ | 2015/7500 [1:07:51<3:05:04, 2.02s/it] 27%|██▋ | 2016/7500 [1:07:53<3:04:12, 2.02s/it] 27%|██▋ | 2017/7500 [1:07:55<3:05:04, 2.03s/it] 27%|██▋ | 2018/7500 [1:07:57<3:02:15, 1.99s/it] 27%|██▋ | 2019/7500 [1:07:59<3:02:12, 1.99s/it] 27%|██▋ | 2020/7500 [1:08:01<3:01:53, 1.99s/it] {'loss': 0.4562, 'grad_norm': 3.5639238357543945, 'learning_rate': 4.575841568909494e-05, 'logqs/rejected': -2.46598219871521, 'logqs/chosen': -0.8985620737075806, 'logqs_logps/rejected': -0.03156941011548042, 'logps_logqs/chosen': 0.238674595952034, 'epoch': 0.54} 27%|██▋ | 2020/7500 [1:08:01<3:01:53, 1.99s/it] 27%|██▋ | 2021/7500 [1:08:03<3:04:03, 2.02s/it] 27%|██▋ | 2022/7500 [1:08:05<3:04:55, 2.03s/it] 27%|██▋ | 2023/7500 [1:08:07<3:02:21, 2.00s/it] 27%|██▋ | 2024/7500 [1:08:09<3:05:20, 2.03s/it] 27%|██▋ | 2025/7500 [1:08:11<3:05:17, 2.03s/it] 27%|██▋ | 2026/7500 [1:08:13<3:04:18, 2.02s/it] 27%|██▋ | 2027/7500 [1:08:15<3:06:57, 2.05s/it] 27%|██▋ | 2028/7500 [1:08:17<3:03:30, 2.01s/it] 27%|██▋ | 2029/7500 [1:08:19<3:02:58, 2.01s/it] 27%|██▋ | 2030/7500 [1:08:21<3:02:41, 2.00s/it] {'loss': 0.4791, 'grad_norm': 3.7423155307769775, 'learning_rate': 4.5693350699025014e-05, 'logqs/rejected': -2.650123119354248, 'logqs/chosen': -0.9194696545600891, 'logqs_logps/rejected': -0.026144057512283325, 'logps_logqs/chosen': 0.23782727122306824, 'epoch': 0.54} 27%|██▋ | 2030/7500 [1:08:21<3:02:41, 2.00s/it] 27%|██▋ | 2031/7500 [1:08:23<3:03:53, 2.02s/it] 27%|██▋ | 2032/7500 [1:08:25<3:04:49, 2.03s/it] 27%|██▋ | 2033/7500 [1:08:27<3:04:00, 2.02s/it] 27%|██▋ | 2034/7500 [1:08:29<3:03:52, 2.02s/it] 27%|██▋ | 2035/7500 [1:08:31<3:04:12, 2.02s/it] 27%|██▋ | 2036/7500 [1:08:33<3:03:49, 2.02s/it] 27%|██▋ | 2037/7500 [1:08:35<3:00:26, 1.98s/it] 27%|██▋ | 2038/7500 [1:08:37<2:59:21, 1.97s/it] 27%|██▋ | 2039/7500 [1:08:39<2:58:23, 1.96s/it] 27%|██▋ | 2040/7500 [1:08:41<3:01:47, 2.00s/it] {'loss': 0.4579, 'grad_norm': 10.217142105102539, 'learning_rate': 4.562783745695738e-05, 'logqs/rejected': -2.47772479057312, 'logqs/chosen': -0.9277342557907104, 'logqs_logps/rejected': -0.03161641210317612, 'logps_logqs/chosen': 0.24457938969135284, 'epoch': 0.54} 27%|██▋ | 2040/7500 [1:08:41<3:01:47, 2.00s/it] 27%|██▋ | 2041/7500 [1:08:43<3:03:32, 2.02s/it] 27%|██▋ | 2042/7500 [1:08:45<3:03:03, 2.01s/it] 27%|██▋ | 2043/7500 [1:08:47<3:04:31, 2.03s/it] 27%|██▋ | 2044/7500 [1:08:49<3:06:56, 2.06s/it] 27%|██▋ | 2045/7500 [1:08:51<3:07:35, 2.06s/it] 27%|██▋ | 2046/7500 [1:08:53<3:06:53, 2.06s/it] 27%|██▋ | 2047/7500 [1:08:55<3:06:48, 2.06s/it] 27%|██▋ | 2048/7500 [1:08:57<3:08:53, 2.08s/it] 27%|██▋ | 2049/7500 [1:08:59<3:08:47, 2.08s/it] 27%|██▋ | 2050/7500 [1:09:02<3:10:47, 2.10s/it] {'loss': 0.4596, 'grad_norm': 3.7780957221984863, 'learning_rate': 4.556187738201656e-05, 'logqs/rejected': -2.4555599689483643, 'logqs/chosen': -0.9682849645614624, 'logqs_logps/rejected': -0.039207249879837036, 'logps_logqs/chosen': 0.24924154579639435, 'epoch': 0.55} 27%|██▋ | 2050/7500 [1:09:02<3:10:47, 2.10s/it] 27%|██▋ | 2051/7500 [1:09:04<3:11:57, 2.11s/it] 27%|██▋ | 2052/7500 [1:09:06<3:08:47, 2.08s/it] 27%|██▋ | 2053/7500 [1:09:08<3:07:03, 2.06s/it] 27%|██▋ | 2054/7500 [1:09:10<3:06:28, 2.05s/it] 27%|██▋ | 2055/7500 [1:09:12<3:04:23, 2.03s/it] 27%|██▋ | 2056/7500 [1:09:14<3:03:12, 2.02s/it] 27%|██▋ | 2057/7500 [1:09:16<3:04:20, 2.03s/it] 27%|██▋ | 2058/7500 [1:09:18<3:02:12, 2.01s/it] 27%|██▋ | 2059/7500 [1:09:20<3:05:49, 2.05s/it] 27%|██▋ | 2060/7500 [1:09:22<3:05:51, 2.05s/it] {'loss': 0.4756, 'grad_norm': 3.6143600940704346, 'learning_rate': 4.549547190300622e-05, 'logqs/rejected': -2.5774097442626953, 'logqs/chosen': -0.9898718595504761, 'logqs_logps/rejected': -0.027303820475935936, 'logps_logqs/chosen': 0.2470652163028717, 'epoch': 0.55} 27%|██▋ | 2060/7500 [1:09:22<3:05:51, 2.05s/it] 27%|██▋ | 2061/7500 [1:09:24<3:02:18, 2.01s/it] 27%|██▋ | 2062/7500 [1:09:26<3:02:54, 2.02s/it] 28%|██▊ | 2063/7500 [1:09:28<3:01:16, 2.00s/it] 28%|██▊ | 2064/7500 [1:09:30<3:02:36, 2.02s/it] 28%|██▊ | 2065/7500 [1:09:32<3:03:28, 2.03s/it] 28%|██▊ | 2066/7500 [1:09:34<3:00:52, 2.00s/it] 28%|██▊ | 2067/7500 [1:09:36<3:07:48, 2.07s/it] 28%|██▊ | 2068/7500 [1:09:38<3:06:43, 2.06s/it] 28%|██▊ | 2069/7500 [1:09:40<3:08:49, 2.09s/it] 28%|██▊ | 2070/7500 [1:09:42<3:04:44, 2.04s/it] {'loss': 0.4741, 'grad_norm': 4.089685916900635, 'learning_rate': 4.542862245837821e-05, 'logqs/rejected': -2.6933882236480713, 'logqs/chosen': -0.911259651184082, 'logqs_logps/rejected': -0.023999279364943504, 'logps_logqs/chosen': 0.23494994640350342, 'epoch': 0.55} 28%|██▊ | 2070/7500 [1:09:42<3:04:44, 2.04s/it] 28%|██▊ | 2071/7500 [1:09:44<3:02:14, 2.01s/it] 28%|██▊ | 2072/7500 [1:09:46<3:05:33, 2.05s/it] 28%|██▊ | 2073/7500 [1:09:48<3:04:15, 2.04s/it] 28%|██▊ | 2074/7500 [1:09:50<3:03:08, 2.03s/it] 28%|██▊ | 2075/7500 [1:09:52<3:05:02, 2.05s/it] 28%|██▊ | 2076/7500 [1:09:55<3:08:00, 2.08s/it] 28%|██▊ | 2077/7500 [1:09:57<3:09:48, 2.10s/it] 28%|██▊ | 2078/7500 [1:09:59<3:05:50, 2.06s/it] 28%|██▊ | 2079/7500 [1:10:01<3:03:55, 2.04s/it] 28%|██▊ | 2080/7500 [1:10:03<3:04:29, 2.04s/it] {'loss': 0.4558, 'grad_norm': 3.603179454803467, 'learning_rate': 4.5361330496201425e-05, 'logqs/rejected': -2.583139419555664, 'logqs/chosen': -0.9115036725997925, 'logqs_logps/rejected': -0.025524934753775597, 'logps_logqs/chosen': 0.23592524230480194, 'epoch': 0.55} 28%|██▊ | 2080/7500 [1:10:03<3:04:29, 2.04s/it] 28%|██▊ | 2081/7500 [1:10:05<3:02:52, 2.02s/it] 28%|██▊ | 2082/7500 [1:10:07<3:02:08, 2.02s/it] 28%|██▊ | 2083/7500 [1:10:09<3:05:24, 2.05s/it] 28%|██▊ | 2084/7500 [1:10:11<3:05:26, 2.05s/it] 28%|██▊ | 2085/7500 [1:10:13<3:03:46, 2.04s/it] 28%|██▊ | 2086/7500 [1:10:15<3:04:33, 2.05s/it] 28%|██▊ | 2087/7500 [1:10:17<3:04:13, 2.04s/it] 28%|██▊ | 2088/7500 [1:10:19<3:05:26, 2.06s/it] 28%|██▊ | 2089/7500 [1:10:21<3:05:57, 2.06s/it] 28%|██▊ | 2090/7500 [1:10:23<3:05:43, 2.06s/it] {'loss': 0.4691, 'grad_norm': 4.484154224395752, 'learning_rate': 4.529359747413037e-05, 'logqs/rejected': -2.5512070655822754, 'logqs/chosen': -0.938818097114563, 'logqs_logps/rejected': -0.03003462217748165, 'logps_logqs/chosen': 0.23973941802978516, 'epoch': 0.56} 28%|██▊ | 2090/7500 [1:10:23<3:05:43, 2.06s/it] 28%|██▊ | 2091/7500 [1:10:25<3:04:01, 2.04s/it] 28%|██▊ | 2092/7500 [1:10:27<3:02:20, 2.02s/it] 28%|██▊ | 2093/7500 [1:10:29<3:01:10, 2.01s/it] 28%|██▊ | 2094/7500 [1:10:31<3:02:11, 2.02s/it] 28%|██▊ | 2095/7500 [1:10:33<2:59:57, 2.00s/it] 28%|██▊ | 2096/7500 [1:10:35<2:59:55, 2.00s/it] 28%|██▊ | 2097/7500 [1:10:37<3:02:45, 2.03s/it] 28%|██▊ | 2098/7500 [1:10:39<2:59:47, 2.00s/it] 28%|██▊ | 2099/7500 [1:10:41<3:00:57, 2.01s/it] 28%|██▊ | 2100/7500 [1:10:43<3:02:23, 2.03s/it] {'loss': 0.4546, 'grad_norm': 4.650066375732422, 'learning_rate': 4.522542485937369e-05, 'logqs/rejected': -2.4291481971740723, 'logqs/chosen': -0.9261422157287598, 'logqs_logps/rejected': -0.023656625300645828, 'logps_logqs/chosen': 0.22850532829761505, 'epoch': 0.56} 28%|██▊ | 2100/7500 [1:10:43<3:02:23, 2.03s/it] 28%|██▊ | 2101/7500 [1:10:45<3:01:13, 2.01s/it] 28%|██▊ | 2102/7500 [1:10:47<2:59:52, 2.00s/it] 28%|██▊ | 2103/7500 [1:10:49<2:59:48, 2.00s/it] 28%|██▊ | 2104/7500 [1:10:51<2:57:46, 1.98s/it] 28%|██▊ | 2105/7500 [1:10:53<2:58:57, 1.99s/it] 28%|██▊ | 2106/7500 [1:10:55<2:58:57, 1.99s/it] 28%|██▊ | 2107/7500 [1:10:57<3:01:09, 2.02s/it] 28%|██▊ | 2108/7500 [1:10:59<3:01:19, 2.02s/it] 28%|██▊ | 2109/7500 [1:11:01<2:58:38, 1.99s/it] 28%|██▊ | 2110/7500 [1:11:03<3:00:24, 2.01s/it] {'loss': 0.451, 'grad_norm': 5.850917816162109, 'learning_rate': 4.5156814128662285e-05, 'logqs/rejected': -2.463733673095703, 'logqs/chosen': -0.9450734853744507, 'logqs_logps/rejected': -0.02720732055604458, 'logps_logqs/chosen': 0.2413969486951828, 'epoch': 0.56} 28%|██▊ | 2110/7500 [1:11:03<3:00:24, 2.01s/it] 28%|██▊ | 2111/7500 [1:11:05<3:01:10, 2.02s/it] 28%|██▊ | 2112/7500 [1:11:07<3:04:27, 2.05s/it] 28%|██▊ | 2113/7500 [1:11:10<3:05:03, 2.06s/it] 28%|██▊ | 2114/7500 [1:11:12<3:03:33, 2.04s/it] 28%|██▊ | 2115/7500 [1:11:13<3:00:03, 2.01s/it] 28%|██▊ | 2116/7500 [1:11:16<3:02:52, 2.04s/it] 28%|██▊ | 2117/7500 [1:11:18<3:03:51, 2.05s/it] 28%|██▊ | 2118/7500 [1:11:20<3:03:50, 2.05s/it] 28%|██▊ | 2119/7500 [1:11:22<3:05:28, 2.07s/it] 28%|██▊ | 2120/7500 [1:11:24<3:02:52, 2.04s/it] {'loss': 0.4514, 'grad_norm': 3.520195245742798, 'learning_rate': 4.5087766768217384e-05, 'logqs/rejected': -2.3719210624694824, 'logqs/chosen': -0.9271903038024902, 'logqs_logps/rejected': -0.03346060588955879, 'logps_logqs/chosen': 0.24172616004943848, 'epoch': 0.57} 28%|██▊ | 2120/7500 [1:11:24<3:02:52, 2.04s/it] 28%|██▊ | 2121/7500 [1:11:26<3:03:19, 2.04s/it] 28%|██▊ | 2122/7500 [1:11:28<3:03:29, 2.05s/it] 28%|██▊ | 2123/7500 [1:11:30<3:02:47, 2.04s/it] 28%|██▊ | 2124/7500 [1:11:32<3:02:41, 2.04s/it] 28%|██▊ | 2125/7500 [1:11:34<3:00:31, 2.02s/it] 28%|██▊ | 2126/7500 [1:11:36<3:00:37, 2.02s/it] 28%|██▊ | 2127/7500 [1:11:38<3:00:00, 2.01s/it] 28%|██▊ | 2128/7500 [1:11:40<2:58:47, 2.00s/it] 28%|██▊ | 2129/7500 [1:11:42<3:00:50, 2.02s/it] 28%|██▊ | 2130/7500 [1:11:44<3:01:53, 2.03s/it] {'loss': 0.4877, 'grad_norm': 4.784451484680176, 'learning_rate': 4.5018284273718336e-05, 'logqs/rejected': -2.6226627826690674, 'logqs/chosen': -0.9456570744514465, 'logqs_logps/rejected': -0.029665639623999596, 'logps_logqs/chosen': 0.24511389434337616, 'epoch': 0.57} 28%|██▊ | 2130/7500 [1:11:44<3:01:53, 2.03s/it] 28%|██▊ | 2131/7500 [1:11:46<3:01:36, 2.03s/it] 28%|██▊ | 2132/7500 [1:11:48<3:03:09, 2.05s/it] 28%|██▊ | 2133/7500 [1:11:50<3:02:06, 2.04s/it] 28%|██▊ | 2134/7500 [1:11:52<2:59:32, 2.01s/it] 28%|██▊ | 2135/7500 [1:11:54<2:59:16, 2.00s/it] 28%|██▊ | 2136/7500 [1:11:56<3:01:13, 2.03s/it] 28%|██▊ | 2137/7500 [1:11:58<2:58:48, 2.00s/it] 29%|██▊ | 2138/7500 [1:12:00<3:02:12, 2.04s/it] 29%|██▊ | 2139/7500 [1:12:02<3:03:14, 2.05s/it] 29%|██▊ | 2140/7500 [1:12:04<3:06:24, 2.09s/it] {'loss': 0.4713, 'grad_norm': 4.515058517456055, 'learning_rate': 4.4948368150270215e-05, 'logqs/rejected': -2.605959177017212, 'logqs/chosen': -0.9268571734428406, 'logqs_logps/rejected': -0.017962943762540817, 'logps_logqs/chosen': 0.24195042252540588, 'epoch': 0.57} 29%|██▊ | 2140/7500 [1:12:04<3:06:24, 2.09s/it] 29%|██▊ | 2141/7500 [1:12:06<3:04:18, 2.06s/it] 29%|██▊ | 2142/7500 [1:12:08<3:02:29, 2.04s/it] 29%|██▊ | 2143/7500 [1:12:10<3:00:58, 2.03s/it] 29%|██▊ | 2144/7500 [1:12:12<3:00:58, 2.03s/it] 29%|██▊ | 2145/7500 [1:12:14<2:59:21, 2.01s/it] 29%|██▊ | 2146/7500 [1:12:16<2:56:37, 1.98s/it] 29%|██▊ | 2147/7500 [1:12:18<2:59:20, 2.01s/it] 29%|██▊ | 2148/7500 [1:12:20<2:58:59, 2.01s/it] 29%|██▊ | 2149/7500 [1:12:22<3:00:06, 2.02s/it] 29%|██▊ | 2150/7500 [1:12:25<3:01:26, 2.03s/it] {'loss': 0.4596, 'grad_norm': 3.9500832557678223, 'learning_rate': 4.48780199123712e-05, 'logqs/rejected': -2.353832960128784, 'logqs/chosen': -0.9231816530227661, 'logqs_logps/rejected': -0.0250143650919199, 'logps_logqs/chosen': 0.23584282398223877, 'epoch': 0.57} 29%|██▊ | 2150/7500 [1:12:25<3:01:26, 2.03s/it] 29%|██▊ | 2151/7500 [1:12:27<3:01:27, 2.04s/it] 29%|██▊ | 2152/7500 [1:12:29<3:02:32, 2.05s/it] 29%|██▊ | 2153/7500 [1:12:31<3:02:16, 2.05s/it] 29%|██▊ | 2154/7500 [1:12:33<3:00:09, 2.02s/it] 29%|██▊ | 2155/7500 [1:12:35<2:59:51, 2.02s/it] 29%|██▊ | 2156/7500 [1:12:37<3:00:39, 2.03s/it] 29%|██▉ | 2157/7500 [1:12:39<3:00:04, 2.02s/it] 29%|██▉ | 2158/7500 [1:12:41<2:59:03, 2.01s/it] 29%|██▉ | 2159/7500 [1:12:43<2:59:18, 2.01s/it] 29%|██▉ | 2160/7500 [1:12:45<2:59:41, 2.02s/it] {'loss': 0.4654, 'grad_norm': 4.676583290100098, 'learning_rate': 4.480724108387977e-05, 'logqs/rejected': -2.669633388519287, 'logqs/chosen': -0.9062808156013489, 'logqs_logps/rejected': -0.025928009301424026, 'logps_logqs/chosen': 0.2301710844039917, 'epoch': 0.58} 29%|██▉ | 2160/7500 [1:12:45<2:59:41, 2.02s/it] 29%|██▉ | 2161/7500 [1:12:47<3:00:32, 2.03s/it] 29%|██▉ | 2162/7500 [1:12:49<2:59:19, 2.02s/it] 29%|██▉ | 2163/7500 [1:12:51<3:00:04, 2.02s/it] 29%|██▉ | 2164/7500 [1:12:53<2:59:54, 2.02s/it] 29%|██▉ | 2165/7500 [1:12:55<3:00:07, 2.03s/it] 29%|██▉ | 2166/7500 [1:12:57<3:01:33, 2.04s/it] 29%|██▉ | 2167/7500 [1:12:59<3:03:04, 2.06s/it] 29%|██▉ | 2168/7500 [1:13:01<3:02:38, 2.06s/it] 29%|██▉ | 2169/7500 [1:13:03<3:03:26, 2.06s/it] 29%|██▉ | 2170/7500 [1:13:05<3:04:00, 2.07s/it] {'loss': 0.4575, 'grad_norm': 3.724076986312866, 'learning_rate': 4.473603319798173e-05, 'logqs/rejected': -2.5045807361602783, 'logqs/chosen': -0.8977010846138, 'logqs_logps/rejected': -0.021044667810201645, 'logps_logqs/chosen': 0.23440034687519073, 'epoch': 0.58} 29%|██▉ | 2170/7500 [1:13:05<3:04:00, 2.07s/it] 29%|██▉ | 2171/7500 [1:13:07<3:04:43, 2.08s/it] 29%|██▉ | 2172/7500 [1:13:09<3:01:47, 2.05s/it] 29%|██▉ | 2173/7500 [1:13:11<3:00:29, 2.03s/it] 29%|██▉ | 2174/7500 [1:13:13<3:00:16, 2.03s/it] 29%|██▉ | 2175/7500 [1:13:15<2:59:47, 2.03s/it] 29%|██▉ | 2176/7500 [1:13:17<2:59:03, 2.02s/it] 29%|██▉ | 2177/7500 [1:13:19<2:58:37, 2.01s/it] 29%|██▉ | 2178/7500 [1:13:21<2:58:37, 2.01s/it] 29%|██▉ | 2179/7500 [1:13:24<3:01:18, 2.04s/it] 29%|██▉ | 2180/7500 [1:13:26<2:59:28, 2.02s/it] {'loss': 0.4863, 'grad_norm': 3.9320030212402344, 'learning_rate': 4.4664397797156954e-05, 'logqs/rejected': -2.602707624435425, 'logqs/chosen': -0.9343243837356567, 'logqs_logps/rejected': -0.02190505340695381, 'logps_logqs/chosen': 0.22856926918029785, 'epoch': 0.58} 29%|██▉ | 2180/7500 [1:13:26<2:59:28, 2.02s/it] 29%|██▉ | 2181/7500 [1:13:27<2:57:41, 2.00s/it] 29%|██▉ | 2182/7500 [1:13:30<3:02:58, 2.06s/it] 29%|██▉ | 2183/7500 [1:13:32<3:02:59, 2.07s/it] 29%|██▉ | 2184/7500 [1:13:34<3:01:09, 2.04s/it] 29%|██▉ | 2185/7500 [1:13:36<2:59:57, 2.03s/it] 29%|██▉ | 2186/7500 [1:13:38<2:58:58, 2.02s/it] 29%|██▉ | 2187/7500 [1:13:40<2:56:00, 1.99s/it] 29%|██▉ | 2188/7500 [1:13:42<2:56:57, 2.00s/it] 29%|██▉ | 2189/7500 [1:13:44<2:59:43, 2.03s/it] 29%|██▉ | 2190/7500 [1:13:46<2:59:55, 2.03s/it] {'loss': 0.462, 'grad_norm': 4.0046892166137695, 'learning_rate': 4.4592336433146e-05, 'logqs/rejected': -2.6210081577301025, 'logqs/chosen': -0.920304000377655, 'logqs_logps/rejected': -0.024190466850996017, 'logps_logqs/chosen': 0.22460326552391052, 'epoch': 0.58} 29%|██▉ | 2190/7500 [1:13:46<2:59:55, 2.03s/it] 29%|██▉ | 2191/7500 [1:13:48<3:01:12, 2.05s/it] 29%|██▉ | 2192/7500 [1:13:50<3:00:52, 2.04s/it] 29%|██▉ | 2193/7500 [1:13:52<3:00:01, 2.04s/it] 29%|██▉ | 2194/7500 [1:13:54<2:58:39, 2.02s/it] 29%|██▉ | 2195/7500 [1:13:56<2:56:47, 2.00s/it] 29%|██▉ | 2196/7500 [1:13:58<3:01:14, 2.05s/it] 29%|██▉ | 2197/7500 [1:14:00<3:00:02, 2.04s/it] 29%|██▉ | 2198/7500 [1:14:02<2:57:27, 2.01s/it] 29%|██▉ | 2199/7500 [1:14:04<2:57:16, 2.01s/it] 29%|██▉ | 2200/7500 [1:14:06<2:58:54, 2.03s/it] {'loss': 0.4601, 'grad_norm': 4.108151435852051, 'learning_rate': 4.4519850666916484e-05, 'logqs/rejected': -2.5512468814849854, 'logqs/chosen': -0.9123099446296692, 'logqs_logps/rejected': -0.026909297332167625, 'logps_logqs/chosen': 0.22343388199806213, 'epoch': 0.59} 29%|██▉ | 2200/7500 [1:14:06<2:58:54, 2.03s/it] 29%|██▉ | 2201/7500 [1:14:08<3:02:34, 2.07s/it] 29%|██▉ | 2202/7500 [1:14:10<3:01:23, 2.05s/it] 29%|██▉ | 2203/7500 [1:14:12<2:59:53, 2.04s/it] 29%|██▉ | 2204/7500 [1:14:14<2:58:05, 2.02s/it] 29%|██▉ | 2205/7500 [1:14:16<2:59:32, 2.03s/it] 29%|██▉ | 2206/7500 [1:14:18<2:56:39, 2.00s/it] 29%|██▉ | 2207/7500 [1:14:20<2:58:36, 2.02s/it] 29%|██▉ | 2208/7500 [1:14:22<2:58:38, 2.03s/it] 29%|██▉ | 2209/7500 [1:14:24<2:55:56, 2.00s/it] 29%|██▉ | 2210/7500 [1:14:26<2:56:33, 2.00s/it] {'loss': 0.454, 'grad_norm': 3.857937812805176, 'learning_rate': 4.4446942068629284e-05, 'logqs/rejected': -2.4375829696655273, 'logqs/chosen': -0.9270660281181335, 'logqs_logps/rejected': -0.02998335100710392, 'logps_logqs/chosen': 0.2350584715604782, 'epoch': 0.59} 29%|██▉ | 2210/7500 [1:14:26<2:56:33, 2.00s/it] 29%|██▉ | 2211/7500 [1:14:28<2:54:08, 1.98s/it] 29%|██▉ | 2212/7500 [1:14:30<2:56:26, 2.00s/it] 30%|██▉ | 2213/7500 [1:14:32<2:57:34, 2.02s/it] 30%|██▉ | 2214/7500 [1:14:34<2:59:14, 2.03s/it] 30%|██▉ | 2215/7500 [1:14:36<3:00:14, 2.05s/it] 30%|██▉ | 2216/7500 [1:14:39<3:01:30, 2.06s/it] 30%|██▉ | 2217/7500 [1:14:41<3:00:02, 2.04s/it] 30%|██▉ | 2218/7500 [1:14:43<3:00:05, 2.05s/it] 30%|██▉ | 2219/7500 [1:14:45<2:59:20, 2.04s/it] 30%|██▉ | 2220/7500 [1:14:47<3:00:05, 2.05s/it] {'loss': 0.4777, 'grad_norm': 3.69391131401062, 'learning_rate': 4.4373612217604496e-05, 'logqs/rejected': -2.767446994781494, 'logqs/chosen': -0.9090011715888977, 'logqs_logps/rejected': -0.01686774380505085, 'logps_logqs/chosen': 0.23097233474254608, 'epoch': 0.59} 30%|██▉ | 2220/7500 [1:14:47<3:00:05, 2.05s/it] 30%|██▉ | 2221/7500 [1:14:49<2:58:21, 2.03s/it] 30%|██▉ | 2222/7500 [1:14:51<3:00:34, 2.05s/it] 30%|██▉ | 2223/7500 [1:14:53<3:00:52, 2.06s/it] 30%|██▉ | 2224/7500 [1:14:55<3:00:52, 2.06s/it] 30%|██▉ | 2225/7500 [1:14:57<3:00:45, 2.06s/it] 30%|██▉ | 2226/7500 [1:14:59<2:59:56, 2.05s/it] 30%|██▉ | 2227/7500 [1:15:01<3:00:12, 2.05s/it] 30%|██▉ | 2228/7500 [1:15:03<2:59:35, 2.04s/it] 30%|██▉ | 2229/7500 [1:15:05<2:59:42, 2.05s/it] 30%|██▉ | 2230/7500 [1:15:07<2:58:11, 2.03s/it] {'loss': 0.4869, 'grad_norm': 4.112382888793945, 'learning_rate': 4.4299862702287256e-05, 'logqs/rejected': -2.6687490940093994, 'logqs/chosen': -0.9433780908584595, 'logqs_logps/rejected': -0.024230685085058212, 'logps_logqs/chosen': 0.2397344410419464, 'epoch': 0.59} 30%|██▉ | 2230/7500 [1:15:07<2:58:11, 2.03s/it] 30%|██▉ | 2231/7500 [1:15:09<2:57:23, 2.02s/it] 30%|██▉ | 2232/7500 [1:15:11<2:56:22, 2.01s/it] 30%|██▉ | 2233/7500 [1:15:13<2:57:24, 2.02s/it] 30%|██▉ | 2234/7500 [1:15:15<2:59:43, 2.05s/it] 30%|██▉ | 2235/7500 [1:15:17<2:58:28, 2.03s/it] 30%|██▉ | 2236/7500 [1:15:19<2:57:21, 2.02s/it] 30%|██▉ | 2237/7500 [1:15:21<2:57:37, 2.02s/it] 30%|██▉ | 2238/7500 [1:15:23<2:59:40, 2.05s/it] 30%|██▉ | 2239/7500 [1:15:25<2:58:16, 2.03s/it] 30%|██▉ | 2240/7500 [1:15:28<3:00:24, 2.06s/it] {'loss': 0.4652, 'grad_norm': 3.8281824588775635, 'learning_rate': 4.422569512021332e-05, 'logqs/rejected': -2.5584487915039062, 'logqs/chosen': -0.9454306364059448, 'logqs_logps/rejected': -0.024338360875844955, 'logps_logqs/chosen': 0.2400411069393158, 'epoch': 0.6} 30%|██▉ | 2240/7500 [1:15:28<3:00:24, 2.06s/it] 30%|██▉ | 2241/7500 [1:15:30<2:59:06, 2.04s/it] 30%|██▉ | 2242/7500 [1:15:32<2:59:53, 2.05s/it] 30%|██▉ | 2243/7500 [1:15:34<2:58:47, 2.04s/it] 30%|██▉ | 2244/7500 [1:15:36<3:00:35, 2.06s/it] 30%|██▉ | 2245/7500 [1:15:38<3:01:27, 2.07s/it] 30%|██▉ | 2246/7500 [1:15:40<3:00:54, 2.07s/it] 30%|██▉ | 2247/7500 [1:15:42<3:02:13, 2.08s/it] 30%|██▉ | 2248/7500 [1:15:44<2:59:11, 2.05s/it] 30%|██▉ | 2249/7500 [1:15:46<2:58:38, 2.04s/it] 30%|███ | 2250/7500 [1:15:48<2:58:31, 2.04s/it] {'loss': 0.4442, 'grad_norm': 3.6480422019958496, 'learning_rate': 4.415111107797445e-05, 'logqs/rejected': -2.185706377029419, 'logqs/chosen': -0.9664382934570312, 'logqs_logps/rejected': -0.03453192487359047, 'logps_logqs/chosen': 0.24160993099212646, 'epoch': 0.6} 30%|███ | 2250/7500 [1:15:48<2:58:31, 2.04s/it] 30%|███ | 2251/7500 [1:15:50<2:58:37, 2.04s/it] 30%|███ | 2252/7500 [1:15:52<2:59:00, 2.05s/it] 30%|███ | 2253/7500 [1:15:54<2:58:39, 2.04s/it] 30%|███ | 2254/7500 [1:15:56<2:59:20, 2.05s/it] 30%|███ | 2255/7500 [1:15:58<2:57:13, 2.03s/it] 30%|███ | 2256/7500 [1:16:00<2:59:10, 2.05s/it] 30%|███ | 2257/7500 [1:16:02<2:58:43, 2.05s/it] 30%|███ | 2258/7500 [1:16:04<3:00:35, 2.07s/it] 30%|███ | 2259/7500 [1:16:07<3:00:24, 2.07s/it] 30%|███ | 2260/7500 [1:16:09<3:00:58, 2.07s/it] {'loss': 0.4715, 'grad_norm': 4.393789291381836, 'learning_rate': 4.4076112191183626e-05, 'logqs/rejected': -2.5013504028320312, 'logqs/chosen': -0.9265044927597046, 'logqs_logps/rejected': -0.02576862834393978, 'logps_logqs/chosen': 0.24454593658447266, 'epoch': 0.6} 30%|███ | 2260/7500 [1:16:09<3:00:58, 2.07s/it] 30%|███ | 2261/7500 [1:16:11<2:57:16, 2.03s/it] 30%|███ | 2262/7500 [1:16:13<2:59:25, 2.06s/it] 30%|███ | 2263/7500 [1:16:15<2:57:49, 2.04s/it] 30%|███ | 2264/7500 [1:16:17<2:55:56, 2.02s/it] 30%|███ | 2265/7500 [1:16:19<2:54:51, 2.00s/it] 30%|███ | 2266/7500 [1:16:21<2:54:37, 2.00s/it] 30%|███ | 2267/7500 [1:16:23<2:54:27, 2.00s/it] 30%|███ | 2268/7500 [1:16:25<2:56:46, 2.03s/it] 30%|███ | 2269/7500 [1:16:27<2:57:02, 2.03s/it] 30%|███ | 2270/7500 [1:16:29<2:56:08, 2.02s/it] {'loss': 0.4556, 'grad_norm': 3.9765419960021973, 'learning_rate': 4.4000700084440044e-05, 'logqs/rejected': -2.5144152641296387, 'logqs/chosen': -0.9138950109481812, 'logqs_logps/rejected': -0.027214020490646362, 'logps_logqs/chosen': 0.24300691485404968, 'epoch': 0.61} 30%|███ | 2270/7500 [1:16:29<2:56:08, 2.02s/it] 30%|███ | 2271/7500 [1:16:31<2:57:55, 2.04s/it] 30%|███ | 2272/7500 [1:16:33<2:57:43, 2.04s/it] 30%|███ | 2273/7500 [1:16:35<2:56:54, 2.03s/it] 30%|███ | 2274/7500 [1:16:37<2:57:55, 2.04s/it] 30%|███ | 2275/7500 [1:16:39<2:57:38, 2.04s/it] 30%|███ | 2276/7500 [1:16:41<2:59:06, 2.06s/it] 30%|███ | 2277/7500 [1:16:43<2:59:26, 2.06s/it] 30%|███ | 2278/7500 [1:16:45<2:57:51, 2.04s/it] 30%|███ | 2279/7500 [1:16:47<2:57:32, 2.04s/it] 30%|███ | 2280/7500 [1:16:49<3:00:20, 2.07s/it] {'loss': 0.4753, 'grad_norm': 6.780262470245361, 'learning_rate': 4.3924876391293915e-05, 'logqs/rejected': -2.7468008995056152, 'logqs/chosen': -0.92820805311203, 'logqs_logps/rejected': -0.021549150347709656, 'logps_logqs/chosen': 0.23495538532733917, 'epoch': 0.61} 30%|███ | 2280/7500 [1:16:49<3:00:20, 2.07s/it] 30%|███ | 2281/7500 [1:16:51<3:00:10, 2.07s/it] 30%|███ | 2282/7500 [1:16:53<2:58:17, 2.05s/it] 30%|███ | 2283/7500 [1:16:55<3:00:06, 2.07s/it] 30%|███ | 2284/7500 [1:16:58<2:59:11, 2.06s/it] 30%|███ | 2285/7500 [1:17:00<2:58:12, 2.05s/it] 30%|███ | 2286/7500 [1:17:02<2:56:25, 2.03s/it] 30%|███ | 2287/7500 [1:17:04<2:54:45, 2.01s/it] 31%|███ | 2288/7500 [1:17:06<2:55:11, 2.02s/it] 31%|███ | 2289/7500 [1:17:08<2:58:02, 2.05s/it] 31%|███ | 2290/7500 [1:17:10<2:59:21, 2.07s/it] {'loss': 0.475, 'grad_norm': 3.591085195541382, 'learning_rate': 4.3848642754211086e-05, 'logqs/rejected': -2.613060474395752, 'logqs/chosen': -0.9590749740600586, 'logqs_logps/rejected': -0.021949749439954758, 'logps_logqs/chosen': 0.23498491942882538, 'epoch': 0.61} 31%|███ | 2290/7500 [1:17:10<2:59:21, 2.07s/it] 31%|███ | 2291/7500 [1:17:12<2:58:42, 2.06s/it] 31%|███ | 2292/7500 [1:17:14<2:56:25, 2.03s/it] 31%|███ | 2293/7500 [1:17:16<2:54:41, 2.01s/it] 31%|███ | 2294/7500 [1:17:18<2:55:22, 2.02s/it] 31%|███ | 2295/7500 [1:17:20<2:54:52, 2.02s/it] 31%|███ | 2296/7500 [1:17:22<2:51:53, 1.98s/it] 31%|███ | 2297/7500 [1:17:24<2:51:44, 1.98s/it] 31%|███ | 2298/7500 [1:17:26<2:51:53, 1.98s/it] 31%|███ | 2299/7500 [1:17:28<2:53:11, 2.00s/it] 31%|███ | 2300/7500 [1:17:30<2:52:19, 1.99s/it] {'loss': 0.4585, 'grad_norm': 5.289480686187744, 'learning_rate': 4.377200082453749e-05, 'logqs/rejected': -2.465825080871582, 'logqs/chosen': -0.8932439684867859, 'logqs_logps/rejected': -0.027617260813713074, 'logps_logqs/chosen': 0.23026950657367706, 'epoch': 0.61} 31%|███ | 2300/7500 [1:17:30<2:52:19, 1.99s/it] 31%|███ | 2301/7500 [1:17:32<2:52:24, 1.99s/it] 31%|███ | 2302/7500 [1:17:34<2:54:32, 2.01s/it] 31%|███ | 2303/7500 [1:17:36<2:55:13, 2.02s/it] 31%|███ | 2304/7500 [1:17:38<2:54:59, 2.02s/it] 31%|███ | 2305/7500 [1:17:40<2:57:23, 2.05s/it] 31%|███ | 2306/7500 [1:17:42<3:00:22, 2.08s/it] 31%|███ | 2307/7500 [1:17:44<2:57:57, 2.06s/it] 31%|███ | 2308/7500 [1:17:46<2:56:57, 2.05s/it] 31%|███ | 2309/7500 [1:17:48<2:56:39, 2.04s/it] 31%|███ | 2310/7500 [1:17:50<2:54:13, 2.01s/it] {'loss': 0.4482, 'grad_norm': 3.6432478427886963, 'learning_rate': 4.36949522624633e-05, 'logqs/rejected': -2.4052960872650146, 'logqs/chosen': -0.9576358795166016, 'logqs_logps/rejected': -0.03349391371011734, 'logps_logqs/chosen': 0.24022884666919708, 'epoch': 0.62} 31%|███ | 2310/7500 [1:17:50<2:54:13, 2.01s/it] 31%|███ | 2311/7500 [1:17:52<2:55:34, 2.03s/it] 31%|███ | 2312/7500 [1:17:54<2:56:37, 2.04s/it] 31%|███ | 2313/7500 [1:17:56<2:57:09, 2.05s/it] 31%|███ | 2314/7500 [1:17:58<2:55:10, 2.03s/it] 31%|███ | 2315/7500 [1:18:00<2:53:38, 2.01s/it] 31%|███ | 2316/7500 [1:18:02<2:51:23, 1.98s/it] 31%|███ | 2317/7500 [1:18:04<2:53:42, 2.01s/it] 31%|███ | 2318/7500 [1:18:06<2:54:28, 2.02s/it] 31%|███ | 2319/7500 [1:18:08<2:55:03, 2.03s/it] 31%|███ | 2320/7500 [1:18:10<2:55:16, 2.03s/it] {'loss': 0.4516, 'grad_norm': 3.6741783618927, 'learning_rate': 4.361749873698707e-05, 'logqs/rejected': -2.5310072898864746, 'logqs/chosen': -0.9162402153015137, 'logqs_logps/rejected': -0.025990495458245277, 'logps_logqs/chosen': 0.24107691645622253, 'epoch': 0.62} 31%|███ | 2320/7500 [1:18:10<2:55:16, 2.03s/it] 31%|███ | 2321/7500 [1:18:12<2:54:05, 2.02s/it] 31%|███ | 2322/7500 [1:18:14<2:55:23, 2.03s/it] 31%|███ | 2323/7500 [1:18:16<2:54:29, 2.02s/it] 31%|███ | 2324/7500 [1:18:18<2:54:03, 2.02s/it] 31%|███ | 2325/7500 [1:18:20<2:55:16, 2.03s/it] 31%|███ | 2326/7500 [1:18:23<2:57:49, 2.06s/it] 31%|███ | 2327/7500 [1:18:25<2:56:31, 2.05s/it] 31%|███ | 2328/7500 [1:18:27<2:54:51, 2.03s/it] 31%|███ | 2329/7500 [1:18:29<2:53:56, 2.02s/it] 31%|███ | 2330/7500 [1:18:31<2:57:30, 2.06s/it] {'loss': 0.446, 'grad_norm': 4.318151950836182, 'learning_rate': 4.3539641925879495e-05, 'logqs/rejected': -2.3897643089294434, 'logqs/chosen': -0.937767505645752, 'logqs_logps/rejected': -0.025380408391356468, 'logps_logqs/chosen': 0.23599819839000702, 'epoch': 0.62} 31%|███ | 2330/7500 [1:18:31<2:57:30, 2.06s/it] 31%|███ | 2331/7500 [1:18:33<2:56:51, 2.05s/it] 31%|███ | 2332/7500 [1:18:35<2:55:02, 2.03s/it] 31%|███ | 2333/7500 [1:18:37<2:55:22, 2.04s/it] 31%|███ | 2334/7500 [1:18:39<2:54:54, 2.03s/it] 31%|███ | 2335/7500 [1:18:41<2:55:37, 2.04s/it] 31%|███ | 2336/7500 [1:18:43<2:57:47, 2.07s/it] 31%|███ | 2337/7500 [1:18:45<2:57:26, 2.06s/it] 31%|███ | 2338/7500 [1:18:47<2:54:31, 2.03s/it] 31%|███ | 2339/7500 [1:18:49<2:53:40, 2.02s/it] 31%|███ | 2340/7500 [1:18:51<2:55:53, 2.05s/it] {'loss': 0.4613, 'grad_norm': 4.9489030838012695, 'learning_rate': 4.3461383515647106e-05, 'logqs/rejected': -2.4457473754882812, 'logqs/chosen': -0.9326648712158203, 'logqs_logps/rejected': -0.0286635123193264, 'logps_logqs/chosen': 0.23893339931964874, 'epoch': 0.62} 31%|███ | 2340/7500 [1:18:51<2:55:53, 2.05s/it] 31%|███ | 2341/7500 [1:18:53<2:55:44, 2.04s/it] 31%|███ | 2342/7500 [1:18:55<2:52:57, 2.01s/it] 31%|███ | 2343/7500 [1:18:57<2:52:43, 2.01s/it] 31%|███▏ | 2344/7500 [1:18:59<2:54:14, 2.03s/it] 31%|███▏ | 2345/7500 [1:19:01<2:55:40, 2.04s/it] 31%|███▏ | 2346/7500 [1:19:03<2:56:05, 2.05s/it] 31%|███▏ | 2347/7500 [1:19:05<2:55:05, 2.04s/it] 31%|███▏ | 2348/7500 [1:19:07<2:53:13, 2.02s/it] 31%|███▏ | 2349/7500 [1:19:09<2:54:40, 2.03s/it] 31%|███▏ | 2350/7500 [1:19:11<2:52:31, 2.01s/it] {'loss': 0.4443, 'grad_norm': 4.010164260864258, 'learning_rate': 4.3382725201495723e-05, 'logqs/rejected': -2.4433159828186035, 'logqs/chosen': -0.9304143190383911, 'logqs_logps/rejected': -0.037369150668382645, 'logps_logqs/chosen': 0.24252624809741974, 'epoch': 0.63} 31%|███▏ | 2350/7500 [1:19:11<2:52:31, 2.01s/it] 31%|███▏ | 2351/7500 [1:19:13<2:52:09, 2.01s/it] 31%|███▏ | 2352/7500 [1:19:15<2:52:56, 2.02s/it] 31%|███▏ | 2353/7500 [1:19:17<2:53:40, 2.02s/it] 31%|███▏ | 2354/7500 [1:19:19<2:53:52, 2.03s/it] 31%|███▏ | 2355/7500 [1:19:21<2:52:02, 2.01s/it] 31%|███▏ | 2356/7500 [1:19:23<2:53:12, 2.02s/it] 31%|███▏ | 2357/7500 [1:19:25<2:52:25, 2.01s/it] 31%|███▏ | 2358/7500 [1:19:28<2:55:15, 2.05s/it] 31%|███▏ | 2359/7500 [1:19:30<2:55:36, 2.05s/it] 31%|███▏ | 2360/7500 [1:19:32<2:55:23, 2.05s/it] {'loss': 0.4624, 'grad_norm': 4.072958469390869, 'learning_rate': 4.330366868729376e-05, 'logqs/rejected': -2.5959010124206543, 'logqs/chosen': -0.9276024103164673, 'logqs_logps/rejected': -0.028544748201966286, 'logps_logqs/chosen': 0.2407229244709015, 'epoch': 0.63} 31%|███▏ | 2360/7500 [1:19:32<2:55:23, 2.05s/it] 31%|███▏ | 2361/7500 [1:19:34<2:53:22, 2.02s/it] 31%|███▏ | 2362/7500 [1:19:36<2:51:47, 2.01s/it] 32%|███▏ | 2363/7500 [1:19:38<2:50:51, 2.00s/it] 32%|███▏ | 2364/7500 [1:19:40<2:51:33, 2.00s/it] 32%|███▏ | 2365/7500 [1:19:42<2:52:17, 2.01s/it] 32%|███▏ | 2366/7500 [1:19:44<2:51:49, 2.01s/it] 32%|███▏ | 2367/7500 [1:19:46<2:51:19, 2.00s/it] 32%|███▏ | 2368/7500 [1:19:48<2:51:32, 2.01s/it] 32%|███▏ | 2369/7500 [1:19:50<2:50:45, 2.00s/it] 32%|███▏ | 2370/7500 [1:19:52<2:52:47, 2.02s/it] {'loss': 0.4634, 'grad_norm': 4.105278491973877, 'learning_rate': 4.3224215685535294e-05, 'logqs/rejected': -2.631322145462036, 'logqs/chosen': -0.9131301045417786, 'logqs_logps/rejected': -0.019937142729759216, 'logps_logqs/chosen': 0.23772883415222168, 'epoch': 0.63} 32%|███▏ | 2370/7500 [1:19:52<2:52:47, 2.02s/it] 32%|███▏ | 2371/7500 [1:19:54<2:51:10, 2.00s/it] 32%|███▏ | 2372/7500 [1:19:56<2:53:18, 2.03s/it] 32%|███▏ | 2373/7500 [1:19:58<2:52:08, 2.01s/it] 32%|███▏ | 2374/7500 [1:20:00<2:53:40, 2.03s/it] 32%|███▏ | 2375/7500 [1:20:02<2:53:20, 2.03s/it] 32%|███▏ | 2376/7500 [1:20:04<2:50:29, 2.00s/it] 32%|███▏ | 2377/7500 [1:20:06<2:50:06, 1.99s/it] 32%|███▏ | 2378/7500 [1:20:08<2:48:21, 1.97s/it] 32%|███▏ | 2379/7500 [1:20:10<2:48:51, 1.98s/it] 32%|███▏ | 2380/7500 [1:20:12<2:46:36, 1.95s/it] {'loss': 0.4395, 'grad_norm': 4.014285564422607, 'learning_rate': 4.3144367917302966e-05, 'logqs/rejected': -2.5824642181396484, 'logqs/chosen': -0.9080556631088257, 'logqs_logps/rejected': -0.026518037542700768, 'logps_logqs/chosen': 0.23356612026691437, 'epoch': 0.63} 32%|███▏ | 2380/7500 [1:20:12<2:46:36, 1.95s/it] 32%|███▏ | 2381/7500 [1:20:14<2:49:40, 1.99s/it] 32%|███▏ | 2382/7500 [1:20:16<2:48:30, 1.98s/it] 32%|███▏ | 2383/7500 [1:20:17<2:46:53, 1.96s/it] 32%|███▏ | 2384/7500 [1:20:19<2:47:20, 1.96s/it] 32%|███▏ | 2385/7500 [1:20:21<2:48:43, 1.98s/it] 32%|███▏ | 2386/7500 [1:20:24<2:51:45, 2.02s/it] 32%|███▏ | 2387/7500 [1:20:26<2:50:43, 2.00s/it] 32%|███▏ | 2388/7500 [1:20:27<2:49:52, 1.99s/it] 32%|███▏ | 2389/7500 [1:20:29<2:50:00, 2.00s/it] 32%|███▏ | 2390/7500 [1:20:32<2:51:18, 2.01s/it] {'loss': 0.4737, 'grad_norm': 4.531930923461914, 'learning_rate': 4.30641271122307e-05, 'logqs/rejected': -2.5403225421905518, 'logqs/chosen': -0.9150890111923218, 'logqs_logps/rejected': -0.017867986112833023, 'logps_logqs/chosen': 0.2335873544216156, 'epoch': 0.64} 32%|███▏ | 2390/7500 [1:20:32<2:51:18, 2.01s/it] 32%|███▏ | 2391/7500 [1:20:34<2:52:50, 2.03s/it] 32%|███▏ | 2392/7500 [1:20:36<2:51:29, 2.01s/it] 32%|███▏ | 2393/7500 [1:20:38<2:49:40, 1.99s/it] 32%|███▏ | 2394/7500 [1:20:40<2:50:52, 2.01s/it] 32%|███▏ | 2395/7500 [1:20:42<2:51:15, 2.01s/it] 32%|███▏ | 2396/7500 [1:20:44<2:49:24, 1.99s/it] 32%|███▏ | 2397/7500 [1:20:46<2:48:56, 1.99s/it] 32%|███▏ | 2398/7500 [1:20:47<2:48:34, 1.98s/it] 32%|███▏ | 2399/7500 [1:20:49<2:49:04, 1.99s/it] 32%|███▏ | 2400/7500 [1:20:51<2:48:45, 1.99s/it] {'loss': 0.478, 'grad_norm': 3.8545963764190674, 'learning_rate': 4.2983495008466276e-05, 'logqs/rejected': -2.7761166095733643, 'logqs/chosen': -0.9218101501464844, 'logqs_logps/rejected': -0.018778780475258827, 'logps_logqs/chosen': 0.23221969604492188, 'epoch': 0.64} 32%|███▏ | 2400/7500 [1:20:51<2:48:45, 1.99s/it] 32%|███▏ | 2401/7500 [1:20:54<2:50:12, 2.00s/it] 32%|███▏ | 2402/7500 [1:20:56<2:49:51, 2.00s/it] 32%|███▏ | 2403/7500 [1:20:58<2:50:29, 2.01s/it] 32%|███▏ | 2404/7500 [1:21:00<2:51:04, 2.01s/it] 32%|███▏ | 2405/7500 [1:21:02<2:54:04, 2.05s/it] 32%|███▏ | 2406/7500 [1:21:04<2:54:59, 2.06s/it] 32%|███▏ | 2407/7500 [1:21:06<2:54:05, 2.05s/it] 32%|███▏ | 2408/7500 [1:21:08<2:54:43, 2.06s/it] 32%|███▏ | 2409/7500 [1:21:10<2:53:38, 2.05s/it] 32%|███▏ | 2410/7500 [1:21:12<2:57:09, 2.09s/it] {'loss': 0.4234, 'grad_norm': 3.296473979949951, 'learning_rate': 4.290247335263362e-05, 'logqs/rejected': -2.1980185508728027, 'logqs/chosen': -0.9191924929618835, 'logqs_logps/rejected': -0.029995372518897057, 'logps_logqs/chosen': 0.23149733245372772, 'epoch': 0.64} 32%|███▏ | 2410/7500 [1:21:12<2:57:09, 2.09s/it] 32%|███▏ | 2411/7500 [1:21:14<2:56:05, 2.08s/it] 32%|███▏ | 2412/7500 [1:21:16<2:55:54, 2.07s/it] 32%|███▏ | 2413/7500 [1:21:18<2:54:07, 2.05s/it] 32%|███▏ | 2414/7500 [1:21:20<2:51:03, 2.02s/it] 32%|███▏ | 2415/7500 [1:21:22<2:50:55, 2.02s/it] 32%|███▏ | 2416/7500 [1:21:24<2:51:00, 2.02s/it] 32%|███▏ | 2417/7500 [1:21:26<2:54:25, 2.06s/it] 32%|███▏ | 2418/7500 [1:21:28<2:52:34, 2.04s/it] 32%|███▏ | 2419/7500 [1:21:30<2:50:34, 2.01s/it] 32%|███▏ | 2420/7500 [1:21:32<2:50:08, 2.01s/it] {'loss': 0.4455, 'grad_norm': 4.536313056945801, 'learning_rate': 4.282106389979501e-05, 'logqs/rejected': -2.4364349842071533, 'logqs/chosen': -0.9052067995071411, 'logqs_logps/rejected': -0.03193528205156326, 'logps_logqs/chosen': 0.23067347705364227, 'epoch': 0.65} 32%|███▏ | 2420/7500 [1:21:32<2:50:08, 2.01s/it] 32%|███▏ | 2421/7500 [1:21:34<2:48:36, 1.99s/it] 32%|███▏ | 2422/7500 [1:21:36<2:48:27, 1.99s/it] 32%|███▏ | 2423/7500 [1:21:38<2:46:55, 1.97s/it] 32%|███▏ | 2424/7500 [1:21:40<2:48:21, 1.99s/it] 32%|███▏ | 2425/7500 [1:21:42<2:49:37, 2.01s/it] 32%|███▏ | 2426/7500 [1:21:44<2:50:50, 2.02s/it] 32%|███▏ | 2427/7500 [1:21:46<2:50:04, 2.01s/it] 32%|███▏ | 2428/7500 [1:21:48<2:49:02, 2.00s/it] 32%|███▏ | 2429/7500 [1:21:50<2:49:12, 2.00s/it] 32%|███▏ | 2430/7500 [1:21:52<2:51:53, 2.03s/it] {'loss': 0.4501, 'grad_norm': 4.756414890289307, 'learning_rate': 4.273926841341302e-05, 'logqs/rejected': -2.574187755584717, 'logqs/chosen': -0.9083077311515808, 'logqs_logps/rejected': -0.028682496398687363, 'logps_logqs/chosen': 0.23035280406475067, 'epoch': 0.65} 32%|███▏ | 2430/7500 [1:21:52<2:51:53, 2.03s/it] 32%|███▏ | 2431/7500 [1:21:54<2:50:41, 2.02s/it] 32%|███▏ | 2432/7500 [1:21:56<2:50:06, 2.01s/it] 32%|███▏ | 2433/7500 [1:21:59<2:53:46, 2.06s/it] 32%|███▏ | 2434/7500 [1:22:01<2:52:26, 2.04s/it] 32%|███▏ | 2435/7500 [1:22:03<2:52:12, 2.04s/it] 32%|███▏ | 2436/7500 [1:22:05<2:50:56, 2.03s/it] 32%|███▏ | 2437/7500 [1:22:07<2:51:08, 2.03s/it] 33%|███▎ | 2438/7500 [1:22:09<2:48:55, 2.00s/it] 33%|███▎ | 2439/7500 [1:22:11<2:52:30, 2.05s/it] 33%|███▎ | 2440/7500 [1:22:13<2:54:19, 2.07s/it] {'loss': 0.447, 'grad_norm': 3.7499094009399414, 'learning_rate': 4.2657088665312385e-05, 'logqs/rejected': -2.433799982070923, 'logqs/chosen': -0.9356620907783508, 'logqs_logps/rejected': -0.028731312602758408, 'logps_logqs/chosen': 0.23546385765075684, 'epoch': 0.65} 33%|███▎ | 2440/7500 [1:22:13<2:54:19, 2.07s/it] 33%|███▎ | 2441/7500 [1:22:15<2:52:12, 2.04s/it] 33%|███▎ | 2442/7500 [1:22:17<2:51:56, 2.04s/it] 33%|███▎ | 2443/7500 [1:22:19<2:52:52, 2.05s/it] 33%|███▎ | 2444/7500 [1:22:21<2:50:20, 2.02s/it] 33%|███▎ | 2445/7500 [1:22:23<2:47:40, 1.99s/it] 33%|███▎ | 2446/7500 [1:22:25<2:47:41, 1.99s/it] 33%|███▎ | 2447/7500 [1:22:27<2:48:42, 2.00s/it] 33%|███▎ | 2448/7500 [1:22:29<2:49:08, 2.01s/it] 33%|███▎ | 2449/7500 [1:22:31<2:51:50, 2.04s/it] 33%|███▎ | 2450/7500 [1:22:33<2:51:14, 2.03s/it] {'loss': 0.4662, 'grad_norm': 4.392151832580566, 'learning_rate': 4.257452643564155e-05, 'logqs/rejected': -2.504117727279663, 'logqs/chosen': -0.9115661382675171, 'logqs_logps/rejected': -0.023574739694595337, 'logps_logqs/chosen': 0.2292707860469818, 'epoch': 0.65} 33%|███▎ | 2450/7500 [1:22:33<2:51:14, 2.03s/it] 33%|███▎ | 2451/7500 [1:22:35<2:50:59, 2.03s/it] 33%|███▎ | 2452/7500 [1:22:37<2:49:59, 2.02s/it] 33%|███▎ | 2453/7500 [1:22:39<2:51:47, 2.04s/it] 33%|███▎ | 2454/7500 [1:22:41<2:50:24, 2.03s/it] 33%|███▎ | 2455/7500 [1:22:43<2:51:19, 2.04s/it] 33%|███▎ | 2456/7500 [1:22:45<2:49:37, 2.02s/it] 33%|███▎ | 2457/7500 [1:22:47<2:51:21, 2.04s/it] 33%|███▎ | 2458/7500 [1:22:49<2:50:05, 2.02s/it] 33%|███▎ | 2459/7500 [1:22:51<2:49:36, 2.02s/it] 33%|███▎ | 2460/7500 [1:22:53<2:51:31, 2.04s/it] {'loss': 0.4549, 'grad_norm': 3.98689866065979, 'learning_rate': 4.249158351283414e-05, 'logqs/rejected': -2.7219436168670654, 'logqs/chosen': -0.9196375608444214, 'logqs_logps/rejected': -0.02516372874379158, 'logps_logqs/chosen': 0.23732776939868927, 'epoch': 0.66} 33%|███▎ | 2460/7500 [1:22:53<2:51:31, 2.04s/it] 33%|███▎ | 2461/7500 [1:22:55<2:49:54, 2.02s/it] 33%|███▎ | 2462/7500 [1:22:57<2:51:07, 2.04s/it] 33%|███▎ | 2463/7500 [1:22:59<2:50:37, 2.03s/it] 33%|███▎ | 2464/7500 [1:23:01<2:48:37, 2.01s/it] 33%|███▎ | 2465/7500 [1:23:03<2:47:34, 2.00s/it] 33%|███▎ | 2466/7500 [1:23:05<2:48:21, 2.01s/it] 33%|███▎ | 2467/7500 [1:23:07<2:48:03, 2.00s/it] 33%|███▎ | 2468/7500 [1:23:09<2:47:03, 1.99s/it] 33%|███▎ | 2469/7500 [1:23:11<2:46:59, 1.99s/it] 33%|███▎ | 2470/7500 [1:23:13<2:47:43, 2.00s/it] {'loss': 0.4566, 'grad_norm': 4.203855991363525, 'learning_rate': 4.240826169357024e-05, 'logqs/rejected': -2.6122536659240723, 'logqs/chosen': -0.8858240842819214, 'logqs_logps/rejected': -0.023891445249319077, 'logps_logqs/chosen': 0.2288806438446045, 'epoch': 0.66} 33%|███▎ | 2470/7500 [1:23:13<2:47:43, 2.00s/it] 33%|███▎ | 2471/7500 [1:23:15<2:49:28, 2.02s/it] 33%|███▎ | 2472/7500 [1:23:17<2:52:20, 2.06s/it] 33%|███▎ | 2473/7500 [1:23:19<2:52:12, 2.06s/it] 33%|███▎ | 2474/7500 [1:23:22<2:50:54, 2.04s/it] 33%|███▎ | 2475/7500 [1:23:24<2:50:26, 2.04s/it] 33%|███▎ | 2476/7500 [1:23:26<2:50:54, 2.04s/it] 33%|███▎ | 2477/7500 [1:23:28<2:50:40, 2.04s/it] 33%|███▎ | 2478/7500 [1:23:30<2:49:44, 2.03s/it] 33%|███▎ | 2479/7500 [1:23:32<2:48:20, 2.01s/it] 33%|███▎ | 2480/7500 [1:23:34<2:49:58, 2.03s/it] {'loss': 0.4431, 'grad_norm': 5.325348854064941, 'learning_rate': 4.232456278273743e-05, 'logqs/rejected': -2.560851573944092, 'logqs/chosen': -0.9156923294067383, 'logqs_logps/rejected': -0.02524157240986824, 'logps_logqs/chosen': 0.23398518562316895, 'epoch': 0.66} 33%|███▎ | 2480/7500 [1:23:34<2:49:58, 2.03s/it] 33%|███▎ | 2481/7500 [1:23:36<2:51:22, 2.05s/it] 33%|███▎ | 2482/7500 [1:23:38<2:48:41, 2.02s/it] 33%|███▎ | 2483/7500 [1:23:40<2:51:33, 2.05s/it] 33%|███▎ | 2484/7500 [1:23:42<2:51:47, 2.05s/it] 33%|███▎ | 2485/7500 [1:23:44<2:52:52, 2.07s/it] 33%|███▎ | 2486/7500 [1:23:46<2:52:24, 2.06s/it] 33%|███▎ | 2487/7500 [1:23:48<2:52:15, 2.06s/it] 33%|███▎ | 2488/7500 [1:23:50<2:51:19, 2.05s/it] 33%|███▎ | 2489/7500 [1:23:52<2:49:27, 2.03s/it] 33%|███▎ | 2490/7500 [1:23:54<2:50:35, 2.04s/it] {'loss': 0.4611, 'grad_norm': 3.7758450508117676, 'learning_rate': 4.224048859339175e-05, 'logqs/rejected': -2.558159351348877, 'logqs/chosen': -0.9409748315811157, 'logqs_logps/rejected': -0.027416924014687538, 'logps_logqs/chosen': 0.24029164016246796, 'epoch': 0.66} 33%|███▎ | 2490/7500 [1:23:54<2:50:35, 2.04s/it] 33%|███▎ | 2491/7500 [1:23:56<2:50:03, 2.04s/it] 33%|███▎ | 2492/7500 [1:23:58<2:49:59, 2.04s/it] 33%|███▎ | 2493/7500 [1:24:00<2:49:14, 2.03s/it] 33%|███▎ | 2494/7500 [1:24:02<2:48:15, 2.02s/it] 33%|███▎ | 2495/7500 [1:24:04<2:48:19, 2.02s/it] 33%|███▎ | 2496/7500 [1:24:06<2:47:08, 2.00s/it] 33%|███▎ | 2497/7500 [1:24:08<2:49:21, 2.03s/it] 33%|███▎ | 2498/7500 [1:24:10<2:50:12, 2.04s/it] 33%|███▎ | 2499/7500 [1:24:12<2:51:41, 2.06s/it] 33%|███▎ | 2500/7500 [1:24:15<2:50:24, 2.04s/it] {'loss': 0.4518, 'grad_norm': 3.5958621501922607, 'learning_rate': 4.215604094671835e-05, 'logqs/rejected': -2.469700813293457, 'logqs/chosen': -0.915995717048645, 'logqs_logps/rejected': -0.02534502185881138, 'logps_logqs/chosen': 0.2303992211818695, 'epoch': 0.67} 33%|███▎ | 2500/7500 [1:24:15<2:50:24, 2.04s/it] 33%|███▎ | 2501/7500 [1:24:16<2:48:57, 2.03s/it] 33%|███▎ | 2502/7500 [1:24:19<2:51:16, 2.06s/it] 33%|███▎ | 2503/7500 [1:24:21<2:52:37, 2.07s/it] 33%|███▎ | 2504/7500 [1:24:23<2:50:59, 2.05s/it] 33%|███▎ | 2505/7500 [1:24:25<2:50:16, 2.05s/it] 33%|███▎ | 2506/7500 [1:24:27<2:51:05, 2.06s/it] 33%|███▎ | 2507/7500 [1:24:29<2:49:45, 2.04s/it] 33%|███▎ | 2508/7500 [1:24:31<2:50:43, 2.05s/it] 33%|███▎ | 2509/7500 [1:24:33<2:50:12, 2.05s/it] 33%|███▎ | 2510/7500 [1:24:35<2:50:48, 2.05s/it] {'loss': 0.4199, 'grad_norm': 4.378320693969727, 'learning_rate': 4.2071221671992086e-05, 'logqs/rejected': -2.2006068229675293, 'logqs/chosen': -0.9100282788276672, 'logqs_logps/rejected': -0.03597059100866318, 'logps_logqs/chosen': 0.23127587139606476, 'epoch': 0.67} 33%|███▎ | 2510/7500 [1:24:35<2:50:48, 2.05s/it] 33%|███▎ | 2511/7500 [1:24:37<2:50:19, 2.05s/it] 33%|███▎ | 2512/7500 [1:24:39<2:50:38, 2.05s/it] 34%|███▎ | 2513/7500 [1:24:41<2:48:04, 2.02s/it] 34%|███▎ | 2514/7500 [1:24:43<2:47:43, 2.02s/it] 34%|███▎ | 2515/7500 [1:24:45<2:46:35, 2.01s/it] 34%|███▎ | 2516/7500 [1:24:47<2:47:00, 2.01s/it] 34%|███▎ | 2517/7500 [1:24:49<2:45:51, 2.00s/it] 34%|███▎ | 2518/7500 [1:24:51<2:48:25, 2.03s/it] 34%|███▎ | 2519/7500 [1:24:53<2:49:56, 2.05s/it] 34%|███▎ | 2520/7500 [1:24:55<2:47:31, 2.02s/it] {'loss': 0.4633, 'grad_norm': 3.6316418647766113, 'learning_rate': 4.198603260653792e-05, 'logqs/rejected': -2.654308795928955, 'logqs/chosen': -0.9398404955863953, 'logqs_logps/rejected': -0.021824970841407776, 'logps_logqs/chosen': 0.2340688705444336, 'epoch': 0.67} 34%|███▎ | 2520/7500 [1:24:55<2:47:31, 2.02s/it] 34%|███▎ | 2521/7500 [1:24:57<2:46:24, 2.01s/it] 34%|███▎ | 2522/7500 [1:24:59<2:47:27, 2.02s/it] 34%|███▎ | 2523/7500 [1:25:01<2:47:47, 2.02s/it] 34%|███▎ | 2524/7500 [1:25:03<2:50:08, 2.05s/it] 34%|███▎ | 2525/7500 [1:25:05<2:48:57, 2.04s/it] 34%|███▎ | 2526/7500 [1:25:07<2:47:45, 2.02s/it] 34%|███▎ | 2527/7500 [1:25:09<2:49:52, 2.05s/it] 34%|███▎ | 2528/7500 [1:25:12<2:49:46, 2.05s/it] 34%|███▎ | 2529/7500 [1:25:14<2:51:02, 2.06s/it] 34%|███▎ | 2530/7500 [1:25:16<2:49:09, 2.04s/it] {'loss': 0.4472, 'grad_norm': 3.873795747756958, 'learning_rate': 4.1900475595691046e-05, 'logqs/rejected': -2.527874231338501, 'logqs/chosen': -0.9110904932022095, 'logqs_logps/rejected': -0.022471297532320023, 'logps_logqs/chosen': 0.2318859100341797, 'epoch': 0.67} 34%|███▎ | 2530/7500 [1:25:16<2:49:09, 2.04s/it] 34%|███▎ | 2531/7500 [1:25:18<2:47:51, 2.03s/it] 34%|███▍ | 2532/7500 [1:25:20<2:47:28, 2.02s/it] 34%|███▍ | 2533/7500 [1:25:22<2:47:15, 2.02s/it] 34%|███▍ | 2534/7500 [1:25:24<2:48:58, 2.04s/it] 34%|███▍ | 2535/7500 [1:25:26<2:48:59, 2.04s/it] 34%|███▍ | 2536/7500 [1:25:28<2:49:26, 2.05s/it] 34%|███▍ | 2537/7500 [1:25:30<2:50:20, 2.06s/it] 34%|███▍ | 2538/7500 [1:25:32<2:47:15, 2.02s/it] 34%|███▍ | 2539/7500 [1:25:34<2:45:03, 2.00s/it] 34%|███▍ | 2540/7500 [1:25:36<2:49:07, 2.05s/it] {'loss': 0.4684, 'grad_norm': 4.211462020874023, 'learning_rate': 4.1814552492757006e-05, 'logqs/rejected': -2.4919540882110596, 'logqs/chosen': -0.9486827850341797, 'logqs_logps/rejected': -0.02413293533027172, 'logps_logqs/chosen': 0.2380782663822174, 'epoch': 0.68} 34%|███▍ | 2540/7500 [1:25:36<2:49:07, 2.05s/it] 34%|███▍ | 2541/7500 [1:25:38<2:49:27, 2.05s/it] 34%|███▍ | 2542/7500 [1:25:40<2:50:34, 2.06s/it] 34%|███▍ | 2543/7500 [1:25:42<2:48:05, 2.03s/it] 34%|███▍ | 2544/7500 [1:25:44<2:49:48, 2.06s/it] 34%|███▍ | 2545/7500 [1:25:46<2:47:06, 2.02s/it] 34%|███▍ | 2546/7500 [1:25:48<2:48:19, 2.04s/it] 34%|███▍ | 2547/7500 [1:25:50<2:48:04, 2.04s/it] 34%|███▍ | 2548/7500 [1:25:52<2:49:29, 2.05s/it] 34%|███▍ | 2549/7500 [1:25:54<2:47:11, 2.03s/it] 34%|███▍ | 2550/7500 [1:25:56<2:47:03, 2.02s/it] {'loss': 0.4675, 'grad_norm': 3.926809787750244, 'learning_rate': 4.172826515897146e-05, 'logqs/rejected': -2.627114772796631, 'logqs/chosen': -0.8973746299743652, 'logqs_logps/rejected': -0.021893084049224854, 'logps_logqs/chosen': 0.23856373131275177, 'epoch': 0.68} 34%|███▍ | 2550/7500 [1:25:56<2:47:03, 2.02s/it] 34%|███▍ | 2551/7500 [1:25:58<2:47:53, 2.04s/it] 34%|███▍ | 2552/7500 [1:26:00<2:46:27, 2.02s/it] 34%|███▍ | 2553/7500 [1:26:02<2:48:12, 2.04s/it] 34%|███▍ | 2554/7500 [1:26:05<2:49:05, 2.05s/it] 34%|███▍ | 2555/7500 [1:26:06<2:45:25, 2.01s/it] 34%|███▍ | 2556/7500 [1:26:08<2:45:56, 2.01s/it] 34%|███▍ | 2557/7500 [1:26:11<2:47:09, 2.03s/it] 34%|███▍ | 2558/7500 [1:26:13<2:47:26, 2.03s/it] 34%|███▍ | 2559/7500 [1:26:15<2:47:10, 2.03s/it] 34%|███▍ | 2560/7500 [1:26:17<2:48:03, 2.04s/it] {'loss': 0.46, 'grad_norm': 4.251122951507568, 'learning_rate': 4.1641615463459924e-05, 'logqs/rejected': -2.5604395866394043, 'logqs/chosen': -0.9319658279418945, 'logqs_logps/rejected': -0.02322964183986187, 'logps_logqs/chosen': 0.23658061027526855, 'epoch': 0.68} 34%|███▍ | 2560/7500 [1:26:17<2:48:03, 2.04s/it] 34%|███▍ | 2561/7500 [1:26:19<2:51:09, 2.08s/it] 34%|███▍ | 2562/7500 [1:26:21<2:49:29, 2.06s/it] 34%|███▍ | 2563/7500 [1:26:23<2:46:00, 2.02s/it] 34%|███▍ | 2564/7500 [1:26:25<2:46:48, 2.03s/it] 34%|███▍ | 2565/7500 [1:26:27<2:49:06, 2.06s/it] 34%|███▍ | 2566/7500 [1:26:29<2:49:38, 2.06s/it] 34%|███▍ | 2567/7500 [1:26:31<2:49:17, 2.06s/it] 34%|███▍ | 2568/7500 [1:26:33<2:49:37, 2.06s/it] 34%|███▍ | 2569/7500 [1:26:35<2:48:24, 2.05s/it] 34%|███▍ | 2570/7500 [1:26:37<2:46:13, 2.02s/it] {'loss': 0.4516, 'grad_norm': 3.5875561237335205, 'learning_rate': 4.155460528319726e-05, 'logqs/rejected': -2.5611279010772705, 'logqs/chosen': -0.9181677103042603, 'logqs_logps/rejected': -0.02229420468211174, 'logps_logqs/chosen': 0.2292613685131073, 'epoch': 0.69} 34%|███▍ | 2570/7500 [1:26:37<2:46:13, 2.02s/it] 34%|███▍ | 2571/7500 [1:26:39<2:45:54, 2.02s/it] 34%|███▍ | 2572/7500 [1:26:41<2:48:16, 2.05s/it] 34%|███▍ | 2573/7500 [1:26:43<2:48:22, 2.05s/it] 34%|███▍ | 2574/7500 [1:26:45<2:52:13, 2.10s/it] 34%|███▍ | 2575/7500 [1:26:48<2:53:00, 2.11s/it] 34%|███▍ | 2576/7500 [1:26:50<2:52:32, 2.10s/it] 34%|███▍ | 2577/7500 [1:26:52<2:52:41, 2.10s/it] 34%|███▍ | 2578/7500 [1:26:54<2:51:12, 2.09s/it] 34%|███▍ | 2579/7500 [1:26:56<2:48:45, 2.06s/it] 34%|███▍ | 2580/7500 [1:26:58<2:50:21, 2.08s/it] {'loss': 0.4379, 'grad_norm': 3.323455572128296, 'learning_rate': 4.146723650296701e-05, 'logqs/rejected': -2.4277119636535645, 'logqs/chosen': -0.918427586555481, 'logqs_logps/rejected': -0.03424077481031418, 'logps_logqs/chosen': 0.2344805896282196, 'epoch': 0.69} 34%|███▍ | 2580/7500 [1:26:58<2:50:21, 2.08s/it] 34%|███▍ | 2581/7500 [1:27:00<2:49:03, 2.06s/it] 34%|███▍ | 2582/7500 [1:27:02<2:47:57, 2.05s/it] 34%|███▍ | 2583/7500 [1:27:04<2:47:53, 2.05s/it] 34%|███▍ | 2584/7500 [1:27:06<2:46:04, 2.03s/it] 34%|███▍ | 2585/7500 [1:27:08<2:45:05, 2.02s/it] 34%|███▍ | 2586/7500 [1:27:10<2:45:12, 2.02s/it] 34%|███▍ | 2587/7500 [1:27:12<2:45:11, 2.02s/it] 35%|███▍ | 2588/7500 [1:27:14<2:43:21, 2.00s/it] 35%|███▍ | 2589/7500 [1:27:16<2:45:03, 2.02s/it] 35%|███▍ | 2590/7500 [1:27:18<2:44:35, 2.01s/it] {'loss': 0.4529, 'grad_norm': 4.100283622741699, 'learning_rate': 4.1379511015320626e-05, 'logqs/rejected': -2.6465394496917725, 'logqs/chosen': -0.8786660432815552, 'logqs_logps/rejected': -0.023012159392237663, 'logps_logqs/chosen': 0.22874228656291962, 'epoch': 0.69} 35%|███▍ | 2590/7500 [1:27:18<2:44:35, 2.01s/it] 35%|███▍ | 2591/7500 [1:27:20<2:44:54, 2.02s/it] 35%|███▍ | 2592/7500 [1:27:22<2:45:08, 2.02s/it] 35%|███▍ | 2593/7500 [1:27:24<2:45:53, 2.03s/it] 35%|███▍ | 2594/7500 [1:27:26<2:48:36, 2.06s/it] 35%|███▍ | 2595/7500 [1:27:28<2:47:11, 2.05s/it] 35%|███▍ | 2596/7500 [1:27:30<2:47:06, 2.04s/it] 35%|███▍ | 2597/7500 [1:27:32<2:46:04, 2.03s/it] 35%|███▍ | 2598/7500 [1:27:34<2:45:23, 2.02s/it] 35%|███▍ | 2599/7500 [1:27:36<2:44:11, 2.01s/it] 35%|███▍ | 2600/7500 [1:27:38<2:44:26, 2.01s/it] {'loss': 0.427, 'grad_norm': 3.545095682144165, 'learning_rate': 4.129143072053638e-05, 'logqs/rejected': -2.450562000274658, 'logqs/chosen': -0.8752145767211914, 'logqs_logps/rejected': -0.024268275126814842, 'logps_logqs/chosen': 0.22875818610191345, 'epoch': 0.69} 35%|███▍ | 2600/7500 [1:27:38<2:44:26, 2.01s/it] 35%|███▍ | 2601/7500 [1:27:40<2:44:04, 2.01s/it] 35%|███▍ | 2602/7500 [1:27:42<2:45:36, 2.03s/it] 35%|███▍ | 2603/7500 [1:27:44<2:42:53, 2.00s/it] 35%|███▍ | 2604/7500 [1:27:46<2:45:19, 2.03s/it] 35%|███▍ | 2605/7500 [1:27:48<2:43:55, 2.01s/it] 35%|███▍ | 2606/7500 [1:27:50<2:41:25, 1.98s/it] 35%|███▍ | 2607/7500 [1:27:52<2:40:22, 1.97s/it] 35%|███▍ | 2608/7500 [1:27:54<2:41:23, 1.98s/it] 35%|███▍ | 2609/7500 [1:27:56<2:43:29, 2.01s/it] 35%|███▍ | 2610/7500 [1:27:58<2:42:57, 2.00s/it] {'loss': 0.4562, 'grad_norm': 3.752061367034912, 'learning_rate': 4.1202997526578276e-05, 'logqs/rejected': -2.5220842361450195, 'logqs/chosen': -0.9185785055160522, 'logqs_logps/rejected': -0.024293895810842514, 'logps_logqs/chosen': 0.2280542403459549, 'epoch': 0.7} 35%|███▍ | 2610/7500 [1:27:58<2:42:57, 2.00s/it] 35%|███▍ | 2611/7500 [1:28:00<2:42:24, 1.99s/it] 35%|███▍ | 2612/7500 [1:28:02<2:43:42, 2.01s/it] 35%|███▍ | 2613/7500 [1:28:04<2:44:13, 2.02s/it] 35%|███▍ | 2614/7500 [1:28:06<2:44:49, 2.02s/it] 35%|███▍ | 2615/7500 [1:28:09<2:46:09, 2.04s/it] 35%|███▍ | 2616/7500 [1:28:11<2:45:18, 2.03s/it] 35%|███▍ | 2617/7500 [1:28:13<2:47:33, 2.06s/it] 35%|███▍ | 2618/7500 [1:28:15<2:46:52, 2.05s/it] 35%|███▍ | 2619/7500 [1:28:17<2:47:20, 2.06s/it] 35%|███▍ | 2620/7500 [1:28:19<2:47:05, 2.05s/it] {'loss': 0.4721, 'grad_norm': 3.9786922931671143, 'learning_rate': 4.111421334905468e-05, 'logqs/rejected': -2.593838930130005, 'logqs/chosen': -0.9227282404899597, 'logqs_logps/rejected': -0.0226267222315073, 'logps_logqs/chosen': 0.23061521351337433, 'epoch': 0.7} 35%|███▍ | 2620/7500 [1:28:19<2:47:05, 2.05s/it] 35%|███▍ | 2621/7500 [1:28:21<2:46:34, 2.05s/it] 35%|███▍ | 2622/7500 [1:28:23<2:45:55, 2.04s/it] 35%|███▍ | 2623/7500 [1:28:25<2:45:33, 2.04s/it] 35%|███▍ | 2624/7500 [1:28:27<2:45:56, 2.04s/it] 35%|███▌ | 2625/7500 [1:28:29<2:46:46, 2.05s/it] 35%|███▌ | 2626/7500 [1:28:31<2:45:10, 2.03s/it] 35%|███▌ | 2627/7500 [1:28:33<2:43:05, 2.01s/it] 35%|███▌ | 2628/7500 [1:28:35<2:46:41, 2.05s/it] 35%|███▌ | 2629/7500 [1:28:37<2:47:41, 2.07s/it] 35%|███▌ | 2630/7500 [1:28:39<2:48:07, 2.07s/it] {'loss': 0.4564, 'grad_norm': 3.9062352180480957, 'learning_rate': 4.102508011117684e-05, 'logqs/rejected': -2.419590473175049, 'logqs/chosen': -0.9364708065986633, 'logqs_logps/rejected': -0.024390408769249916, 'logps_logqs/chosen': 0.22926011681556702, 'epoch': 0.7} 35%|███▌ | 2630/7500 [1:28:39<2:48:07, 2.07s/it] 35%|███▌ | 2631/7500 [1:28:41<2:47:16, 2.06s/it] 35%|███▌ | 2632/7500 [1:28:43<2:46:50, 2.06s/it] 35%|███▌ | 2633/7500 [1:28:45<2:46:17, 2.05s/it] 35%|███▌ | 2634/7500 [1:28:47<2:45:00, 2.03s/it] 35%|███▌ | 2635/7500 [1:28:50<2:47:06, 2.06s/it] 35%|███▌ | 2636/7500 [1:28:51<2:44:09, 2.02s/it] 35%|███▌ | 2637/7500 [1:28:53<2:43:23, 2.02s/it] 35%|███▌ | 2638/7500 [1:28:56<2:45:37, 2.04s/it] 35%|███▌ | 2639/7500 [1:28:58<2:46:32, 2.06s/it] 35%|███▌ | 2640/7500 [1:29:00<2:46:48, 2.06s/it] {'loss': 0.4478, 'grad_norm': 4.268614292144775, 'learning_rate': 4.093559974371725e-05, 'logqs/rejected': -2.2976791858673096, 'logqs/chosen': -0.9313276410102844, 'logqs_logps/rejected': -0.03168725222349167, 'logps_logqs/chosen': 0.22238406538963318, 'epoch': 0.7} 35%|███▌ | 2640/7500 [1:29:00<2:46:48, 2.06s/it] 35%|███▌ | 2641/7500 [1:29:02<2:48:51, 2.09s/it] 35%|███▌ | 2642/7500 [1:29:04<2:48:50, 2.09s/it] 35%|███▌ | 2643/7500 [1:29:06<2:49:57, 2.10s/it] 35%|███▌ | 2644/7500 [1:29:08<2:48:42, 2.08s/it] 35%|███▌ | 2645/7500 [1:29:10<2:47:27, 2.07s/it] 35%|███▌ | 2646/7500 [1:29:12<2:48:33, 2.08s/it] 35%|███▌ | 2647/7500 [1:29:14<2:47:39, 2.07s/it] 35%|███▌ | 2648/7500 [1:29:16<2:47:19, 2.07s/it] 35%|███▌ | 2649/7500 [1:29:18<2:46:39, 2.06s/it] 35%|███▌ | 2650/7500 [1:29:20<2:44:35, 2.04s/it] {'loss': 0.4422, 'grad_norm': 3.7741663455963135, 'learning_rate': 4.0845774184967754e-05, 'logqs/rejected': -2.337155818939209, 'logqs/chosen': -0.8717969655990601, 'logqs_logps/rejected': -0.02753472328186035, 'logps_logqs/chosen': 0.22271136939525604, 'epoch': 0.71} 35%|███▌ | 2650/7500 [1:29:20<2:44:35, 2.04s/it] 35%|███▌ | 2651/7500 [1:29:23<2:45:54, 2.05s/it] 35%|███▌ | 2652/7500 [1:29:25<2:45:37, 2.05s/it] 35%|███▌ | 2653/7500 [1:29:27<2:44:00, 2.03s/it] 35%|███▌ | 2654/7500 [1:29:29<2:43:20, 2.02s/it] 35%|███▌ | 2655/7500 [1:29:31<2:45:46, 2.05s/it] 35%|███▌ | 2656/7500 [1:29:33<2:46:20, 2.06s/it] 35%|███▌ | 2657/7500 [1:29:35<2:44:17, 2.04s/it] 35%|███▌ | 2658/7500 [1:29:37<2:43:42, 2.03s/it] 35%|███▌ | 2659/7500 [1:29:39<2:41:51, 2.01s/it] 35%|███▌ | 2660/7500 [1:29:41<2:43:56, 2.03s/it] {'loss': 0.4137, 'grad_norm': 4.001570701599121, 'learning_rate': 4.075560538069767e-05, 'logqs/rejected': -2.2560982704162598, 'logqs/chosen': -0.914506733417511, 'logqs_logps/rejected': -0.03304170072078705, 'logps_logqs/chosen': 0.22424647212028503, 'epoch': 0.71} 35%|███▌ | 2660/7500 [1:29:41<2:43:56, 2.03s/it] 35%|███▌ | 2661/7500 [1:29:43<2:43:41, 2.03s/it] 35%|███▌ | 2662/7500 [1:29:45<2:43:18, 2.03s/it] 36%|███▌ | 2663/7500 [1:29:47<2:45:15, 2.05s/it] 36%|███▌ | 2664/7500 [1:29:49<2:45:36, 2.05s/it] 36%|███▌ | 2665/7500 [1:29:51<2:48:22, 2.09s/it] 36%|███▌ | 2666/7500 [1:29:53<2:47:13, 2.08s/it] 36%|███▌ | 2667/7500 [1:29:55<2:48:21, 2.09s/it] 36%|███▌ | 2668/7500 [1:29:57<2:47:59, 2.09s/it] 36%|███▌ | 2669/7500 [1:29:59<2:46:12, 2.06s/it] 36%|███▌ | 2670/7500 [1:30:01<2:45:04, 2.05s/it] {'loss': 0.4367, 'grad_norm': 4.3450493812561035, 'learning_rate': 4.066509528411152e-05, 'logqs/rejected': -2.420011043548584, 'logqs/chosen': -0.9067069888114929, 'logqs_logps/rejected': -0.02896801009774208, 'logps_logqs/chosen': 0.22913365066051483, 'epoch': 0.71} 36%|███▌ | 2670/7500 [1:30:01<2:45:04, 2.05s/it] 36%|███▌ | 2671/7500 [1:30:03<2:44:06, 2.04s/it] 36%|███▌ | 2672/7500 [1:30:06<2:44:12, 2.04s/it] 36%|███▌ | 2673/7500 [1:30:08<2:45:54, 2.06s/it] 36%|███▌ | 2674/7500 [1:30:10<2:46:27, 2.07s/it] 36%|███▌ | 2675/7500 [1:30:12<2:43:18, 2.03s/it] 36%|███▌ | 2676/7500 [1:30:14<2:43:29, 2.03s/it] 36%|███▌ | 2677/7500 [1:30:16<2:44:58, 2.05s/it] 36%|███▌ | 2678/7500 [1:30:18<2:44:43, 2.05s/it] 36%|███▌ | 2679/7500 [1:30:20<2:44:20, 2.05s/it] 36%|███▌ | 2680/7500 [1:30:22<2:48:17, 2.09s/it] {'loss': 0.4582, 'grad_norm': 3.8817756175994873, 'learning_rate': 4.0574245855806804e-05, 'logqs/rejected': -2.612907886505127, 'logqs/chosen': -0.9193437695503235, 'logqs_logps/rejected': -0.021437447518110275, 'logps_logqs/chosen': 0.2382233589887619, 'epoch': 0.71} 36%|███▌ | 2680/7500 [1:30:22<2:48:17, 2.09s/it] 36%|███▌ | 2681/7500 [1:30:24<2:46:29, 2.07s/it] 36%|███▌ | 2682/7500 [1:30:26<2:46:47, 2.08s/it] 36%|███▌ | 2683/7500 [1:30:28<2:44:00, 2.04s/it] 36%|███▌ | 2684/7500 [1:30:30<2:44:14, 2.05s/it] 36%|███▌ | 2685/7500 [1:30:32<2:44:17, 2.05s/it] 36%|███▌ | 2686/7500 [1:30:34<2:42:28, 2.03s/it] 36%|███▌ | 2687/7500 [1:30:36<2:43:46, 2.04s/it] 36%|███▌ | 2688/7500 [1:30:38<2:42:19, 2.02s/it] 36%|███▌ | 2689/7500 [1:30:40<2:41:55, 2.02s/it] 36%|███▌ | 2690/7500 [1:30:42<2:41:46, 2.02s/it] {'loss': 0.4491, 'grad_norm': 3.3902828693389893, 'learning_rate': 4.0483059063731504e-05, 'logqs/rejected': -2.6125311851501465, 'logqs/chosen': -0.9280545115470886, 'logqs_logps/rejected': -0.022795002907514572, 'logps_logqs/chosen': 0.2335093766450882, 'epoch': 0.72} 36%|███▌ | 2690/7500 [1:30:42<2:41:46, 2.02s/it] 36%|███▌ | 2691/7500 [1:30:44<2:43:55, 2.05s/it] 36%|███▌ | 2692/7500 [1:30:47<2:45:14, 2.06s/it] 36%|███▌ | 2693/7500 [1:30:48<2:43:01, 2.03s/it] 36%|███▌ | 2694/7500 [1:30:51<2:43:03, 2.04s/it] 36%|███▌ | 2695/7500 [1:30:53<2:43:54, 2.05s/it] 36%|███▌ | 2696/7500 [1:30:55<2:44:04, 2.05s/it] 36%|███▌ | 2697/7500 [1:30:57<2:42:58, 2.04s/it] 36%|███▌ | 2698/7500 [1:30:59<2:43:51, 2.05s/it] 36%|███▌ | 2699/7500 [1:31:01<2:44:24, 2.05s/it] 36%|███▌ | 2700/7500 [1:31:03<2:42:19, 2.03s/it] {'loss': 0.4567, 'grad_norm': 3.36026668548584, 'learning_rate': 4.039153688314145e-05, 'logqs/rejected': -2.5726447105407715, 'logqs/chosen': -0.895586371421814, 'logqs_logps/rejected': -0.02515656128525734, 'logps_logqs/chosen': 0.22419461607933044, 'epoch': 0.72} 36%|███▌ | 2700/7500 [1:31:03<2:42:19, 2.03s/it] 36%|███▌ | 2701/7500 [1:31:05<2:45:39, 2.07s/it] 36%|███▌ | 2702/7500 [1:31:07<2:44:33, 2.06s/it] 36%|███▌ | 2703/7500 [1:31:09<2:42:53, 2.04s/it] 36%|███▌ | 2704/7500 [1:31:11<2:42:38, 2.03s/it] 36%|███▌ | 2705/7500 [1:31:13<2:43:45, 2.05s/it] 36%|███▌ | 2706/7500 [1:31:15<2:42:46, 2.04s/it] 36%|███▌ | 2707/7500 [1:31:17<2:41:06, 2.02s/it] 36%|███▌ | 2708/7500 [1:31:19<2:45:16, 2.07s/it] 36%|███▌ | 2709/7500 [1:31:21<2:42:17, 2.03s/it] 36%|███▌ | 2710/7500 [1:31:23<2:41:02, 2.02s/it] {'loss': 0.44, 'grad_norm': 3.5836551189422607, 'learning_rate': 4.029968129655757e-05, 'logqs/rejected': -2.4707846641540527, 'logqs/chosen': -0.9169933199882507, 'logqs_logps/rejected': -0.02919134497642517, 'logps_logqs/chosen': 0.2236032485961914, 'epoch': 0.72} 36%|███▌ | 2710/7500 [1:31:23<2:41:02, 2.02s/it] 36%|███▌ | 2711/7500 [1:31:25<2:40:48, 2.01s/it] 36%|███▌ | 2712/7500 [1:31:27<2:42:23, 2.04s/it] 36%|███▌ | 2713/7500 [1:31:29<2:39:35, 2.00s/it] 36%|███▌ | 2714/7500 [1:31:31<2:39:35, 2.00s/it] 36%|███▌ | 2715/7500 [1:31:33<2:40:43, 2.02s/it] 36%|███▌ | 2716/7500 [1:31:35<2:42:40, 2.04s/it] 36%|███▌ | 2717/7500 [1:31:37<2:40:19, 2.01s/it] 36%|███▌ | 2718/7500 [1:31:39<2:41:09, 2.02s/it] 36%|███▋ | 2719/7500 [1:31:41<2:40:49, 2.02s/it] 36%|███▋ | 2720/7500 [1:31:43<2:41:48, 2.03s/it] {'loss': 0.4327, 'grad_norm': 3.6758525371551514, 'learning_rate': 4.020749429372286e-05, 'logqs/rejected': -2.4503397941589355, 'logqs/chosen': -0.8866540789604187, 'logqs_logps/rejected': -0.023165052756667137, 'logps_logqs/chosen': 0.22068867087364197, 'epoch': 0.73} 36%|███▋ | 2720/7500 [1:31:43<2:41:48, 2.03s/it] 36%|███▋ | 2721/7500 [1:31:45<2:40:39, 2.02s/it] 36%|███▋ | 2722/7500 [1:31:47<2:42:52, 2.05s/it] 36%|███▋ | 2723/7500 [1:31:50<2:42:28, 2.04s/it] 36%|███▋ | 2724/7500 [1:31:52<2:43:35, 2.06s/it] 36%|███▋ | 2725/7500 [1:31:54<2:41:32, 2.03s/it] 36%|███▋ | 2726/7500 [1:31:56<2:42:51, 2.05s/it] 36%|███▋ | 2727/7500 [1:31:58<2:42:23, 2.04s/it] 36%|███▋ | 2728/7500 [1:32:00<2:40:59, 2.02s/it] 36%|███▋ | 2729/7500 [1:32:02<2:41:04, 2.03s/it] 36%|███▋ | 2730/7500 [1:32:04<2:40:36, 2.02s/it] {'loss': 0.4344, 'grad_norm': 3.3037097454071045, 'learning_rate': 4.011497787155938e-05, 'logqs/rejected': -2.448404550552368, 'logqs/chosen': -0.9112583994865417, 'logqs_logps/rejected': -0.027999291196465492, 'logps_logqs/chosen': 0.23602528870105743, 'epoch': 0.73} 36%|███▋ | 2730/7500 [1:32:04<2:40:36, 2.02s/it] 36%|███▋ | 2731/7500 [1:32:06<2:39:42, 2.01s/it] 36%|███▋ | 2732/7500 [1:32:08<2:42:17, 2.04s/it] 36%|███▋ | 2733/7500 [1:32:10<2:40:56, 2.03s/it] 36%|███▋ | 2734/7500 [1:32:12<2:39:12, 2.00s/it] 36%|███▋ | 2735/7500 [1:32:14<2:39:53, 2.01s/it] 36%|███▋ | 2736/7500 [1:32:16<2:41:03, 2.03s/it] 36%|███▋ | 2737/7500 [1:32:18<2:42:57, 2.05s/it] 37%|███▋ | 2738/7500 [1:32:20<2:43:52, 2.06s/it] 37%|███▋ | 2739/7500 [1:32:22<2:43:58, 2.07s/it] 37%|███▋ | 2740/7500 [1:32:24<2:45:51, 2.09s/it] {'loss': 0.4745, 'grad_norm': 4.221278190612793, 'learning_rate': 4.002213403412492e-05, 'logqs/rejected': -2.6291370391845703, 'logqs/chosen': -0.9251793026924133, 'logqs_logps/rejected': -0.02388705126941204, 'logps_logqs/chosen': 0.2437409907579422, 'epoch': 0.73} 37%|███▋ | 2740/7500 [1:32:24<2:45:51, 2.09s/it] 37%|███▋ | 2741/7500 [1:32:26<2:41:41, 2.04s/it] 37%|███▋ | 2742/7500 [1:32:28<2:39:45, 2.01s/it] 37%|███▋ | 2743/7500 [1:32:30<2:40:34, 2.03s/it] 37%|███▋ | 2744/7500 [1:32:32<2:41:20, 2.04s/it] 37%|███▋ | 2745/7500 [1:32:34<2:38:05, 1.99s/it] 37%|███▋ | 2746/7500 [1:32:36<2:38:38, 2.00s/it] 37%|███▋ | 2747/7500 [1:32:38<2:39:22, 2.01s/it] 37%|███▋ | 2748/7500 [1:32:40<2:40:07, 2.02s/it] 37%|███▋ | 2749/7500 [1:32:42<2:40:10, 2.02s/it] 37%|███▋ | 2750/7500 [1:32:44<2:39:20, 2.01s/it] {'loss': 0.4746, 'grad_norm': 4.038553237915039, 'learning_rate': 3.9928964792569655e-05, 'logqs/rejected': -2.7163259983062744, 'logqs/chosen': -0.8941500782966614, 'logqs_logps/rejected': -0.019649699330329895, 'logps_logqs/chosen': 0.22925670444965363, 'epoch': 0.73} 37%|███▋ | 2750/7500 [1:32:44<2:39:20, 2.01s/it] 37%|███▋ | 2751/7500 [1:32:46<2:39:01, 2.01s/it] 37%|███▋ | 2752/7500 [1:32:48<2:39:12, 2.01s/it] 37%|███▋ | 2753/7500 [1:32:50<2:39:53, 2.02s/it] 37%|███▋ | 2754/7500 [1:32:52<2:41:55, 2.05s/it] 37%|███▋ | 2755/7500 [1:32:54<2:39:47, 2.02s/it] 37%|███▋ | 2756/7500 [1:32:56<2:39:02, 2.01s/it] 37%|███▋ | 2757/7500 [1:32:59<2:41:40, 2.05s/it] 37%|███▋ | 2758/7500 [1:33:01<2:40:59, 2.04s/it] 37%|███▋ | 2759/7500 [1:33:03<2:41:43, 2.05s/it] 37%|███▋ | 2760/7500 [1:33:05<2:42:00, 2.05s/it] {'loss': 0.4267, 'grad_norm': 3.8045756816864014, 'learning_rate': 3.983547216509254e-05, 'logqs/rejected': -2.4633288383483887, 'logqs/chosen': -0.9081869125366211, 'logqs_logps/rejected': -0.025315070524811745, 'logps_logqs/chosen': 0.23537559807300568, 'epoch': 0.74} 37%|███▋ | 2760/7500 [1:33:05<2:42:00, 2.05s/it] 37%|███▋ | 2761/7500 [1:33:07<2:40:54, 2.04s/it] 37%|███▋ | 2762/7500 [1:33:09<2:40:50, 2.04s/it] 37%|███▋ | 2763/7500 [1:33:11<2:41:26, 2.04s/it] 37%|███▋ | 2764/7500 [1:33:13<2:40:19, 2.03s/it] 37%|███▋ | 2765/7500 [1:33:15<2:41:24, 2.05s/it] 37%|███▋ | 2766/7500 [1:33:17<2:40:50, 2.04s/it] 37%|███▋ | 2767/7500 [1:33:19<2:39:23, 2.02s/it] 37%|███▋ | 2768/7500 [1:33:21<2:39:10, 2.02s/it] 37%|███▋ | 2769/7500 [1:33:23<2:38:18, 2.01s/it] 37%|███▋ | 2770/7500 [1:33:25<2:38:29, 2.01s/it] {'loss': 0.4585, 'grad_norm': 3.813082695007324, 'learning_rate': 3.9741658176897584e-05, 'logqs/rejected': -2.600801467895508, 'logqs/chosen': -0.9277113080024719, 'logqs_logps/rejected': -0.021487262099981308, 'logps_logqs/chosen': 0.2373833954334259, 'epoch': 0.74} 37%|███▋ | 2770/7500 [1:33:25<2:38:29, 2.01s/it] 37%|███▋ | 2771/7500 [1:33:27<2:37:54, 2.00s/it] 37%|███▋ | 2772/7500 [1:33:29<2:35:50, 1.98s/it] 37%|███▋ | 2773/7500 [1:33:31<2:35:36, 1.98s/it] 37%|███▋ | 2774/7500 [1:33:33<2:37:27, 2.00s/it] 37%|███▋ | 2775/7500 [1:33:35<2:39:18, 2.02s/it] 37%|███▋ | 2776/7500 [1:33:37<2:41:36, 2.05s/it] 37%|███▋ | 2777/7500 [1:33:39<2:40:24, 2.04s/it] 37%|███▋ | 2778/7500 [1:33:41<2:40:39, 2.04s/it] 37%|███▋ | 2779/7500 [1:33:43<2:42:26, 2.06s/it] 37%|███▋ | 2780/7500 [1:33:45<2:42:07, 2.06s/it] {'loss': 0.445, 'grad_norm': 3.6415016651153564, 'learning_rate': 3.964752486015001e-05, 'logqs/rejected': -2.571852684020996, 'logqs/chosen': -0.9090763926506042, 'logqs_logps/rejected': -0.021775836125016212, 'logps_logqs/chosen': 0.23091909289360046, 'epoch': 0.74} 37%|███▋ | 2780/7500 [1:33:45<2:42:07, 2.06s/it] 37%|███▋ | 2781/7500 [1:33:47<2:43:19, 2.08s/it] 37%|███▋ | 2782/7500 [1:33:49<2:44:15, 2.09s/it] 37%|███▋ | 2783/7500 [1:33:51<2:40:23, 2.04s/it] 37%|███▋ | 2784/7500 [1:33:53<2:38:59, 2.02s/it] 37%|███▋ | 2785/7500 [1:33:55<2:39:43, 2.03s/it] 37%|███▋ | 2786/7500 [1:33:57<2:39:43, 2.03s/it] 37%|███▋ | 2787/7500 [1:33:59<2:38:31, 2.02s/it] 37%|███▋ | 2788/7500 [1:34:01<2:36:20, 1.99s/it] 37%|███▋ | 2789/7500 [1:34:03<2:34:33, 1.97s/it] 37%|███▋ | 2790/7500 [1:34:05<2:36:23, 1.99s/it] {'loss': 0.4534, 'grad_norm': 3.4956541061401367, 'learning_rate': 3.955307425393224e-05, 'logqs/rejected': -2.5533900260925293, 'logqs/chosen': -0.9299807548522949, 'logqs_logps/rejected': -0.021205339580774307, 'logps_logqs/chosen': 0.2288912981748581, 'epoch': 0.74} 37%|███▋ | 2790/7500 [1:34:05<2:36:23, 1.99s/it] 37%|███▋ | 2791/7500 [1:34:07<2:37:54, 2.01s/it] 37%|███▋ | 2792/7500 [1:34:09<2:37:56, 2.01s/it] 37%|███▋ | 2793/7500 [1:34:11<2:37:33, 2.01s/it] 37%|███▋ | 2794/7500 [1:34:13<2:36:57, 2.00s/it] 37%|███▋ | 2795/7500 [1:34:15<2:35:19, 1.98s/it] 37%|███▋ | 2796/7500 [1:34:17<2:37:44, 2.01s/it] 37%|███▋ | 2797/7500 [1:34:19<2:38:05, 2.02s/it] 37%|███▋ | 2798/7500 [1:34:21<2:37:57, 2.02s/it] 37%|███▋ | 2799/7500 [1:34:24<2:39:41, 2.04s/it] 37%|███▋ | 2800/7500 [1:34:26<2:40:30, 2.05s/it] {'loss': 0.4321, 'grad_norm': 3.8775153160095215, 'learning_rate': 3.945830840419966e-05, 'logqs/rejected': -2.525649070739746, 'logqs/chosen': -0.8939350843429565, 'logqs_logps/rejected': -0.025321805849671364, 'logps_logqs/chosen': 0.22823724150657654, 'epoch': 0.75} 37%|███▋ | 2800/7500 [1:34:26<2:40:30, 2.05s/it] 37%|███▋ | 2801/7500 [1:34:28<2:41:05, 2.06s/it] 37%|███▋ | 2802/7500 [1:34:30<2:40:59, 2.06s/it] 37%|███▋ | 2803/7500 [1:34:32<2:38:31, 2.02s/it] 37%|███▋ | 2804/7500 [1:34:34<2:39:56, 2.04s/it] 37%|███▋ | 2805/7500 [1:34:36<2:40:39, 2.05s/it] 37%|███▋ | 2806/7500 [1:34:38<2:43:17, 2.09s/it] 37%|███▋ | 2807/7500 [1:34:40<2:44:19, 2.10s/it] 37%|███▋ | 2808/7500 [1:34:42<2:45:06, 2.11s/it] 37%|███▋ | 2809/7500 [1:34:44<2:43:26, 2.09s/it] 37%|███▋ | 2810/7500 [1:34:46<2:44:46, 2.11s/it] {'loss': 0.4323, 'grad_norm': 3.0220253467559814, 'learning_rate': 3.936322936373641e-05, 'logqs/rejected': -2.5337185859680176, 'logqs/chosen': -0.9122686386108398, 'logqs_logps/rejected': -0.030217796564102173, 'logps_logqs/chosen': 0.23190923035144806, 'epoch': 0.75} 37%|███▋ | 2810/7500 [1:34:47<2:44:46, 2.11s/it] 37%|███▋ | 2811/7500 [1:34:49<2:45:05, 2.11s/it] 37%|███▋ | 2812/7500 [1:34:51<2:45:26, 2.12s/it] 38%|███▊ | 2813/7500 [1:34:53<2:43:26, 2.09s/it] 38%|███▊ | 2814/7500 [1:34:55<2:40:49, 2.06s/it] 38%|███▊ | 2815/7500 [1:34:57<2:40:57, 2.06s/it] 38%|███▊ | 2816/7500 [1:34:59<2:39:21, 2.04s/it] 38%|███▊ | 2817/7500 [1:35:01<2:39:30, 2.04s/it] 38%|███▊ | 2818/7500 [1:35:03<2:38:40, 2.03s/it] 38%|███▊ | 2819/7500 [1:35:05<2:38:02, 2.03s/it] 38%|███▊ | 2820/7500 [1:35:07<2:38:21, 2.03s/it] {'loss': 0.466, 'grad_norm': 4.0266852378845215, 'learning_rate': 3.92678391921108e-05, 'logqs/rejected': -2.664973735809326, 'logqs/chosen': -0.8844863176345825, 'logqs_logps/rejected': -0.016977448016405106, 'logps_logqs/chosen': 0.22778618335723877, 'epoch': 0.75} 38%|███▊ | 2820/7500 [1:35:07<2:38:21, 2.03s/it] 38%|███▊ | 2821/7500 [1:35:09<2:38:04, 2.03s/it] 38%|███▊ | 2822/7500 [1:35:11<2:37:47, 2.02s/it] 38%|███▊ | 2823/7500 [1:35:13<2:38:54, 2.04s/it] 38%|███▊ | 2824/7500 [1:35:15<2:37:01, 2.01s/it] 38%|███▊ | 2825/7500 [1:35:17<2:38:34, 2.04s/it] 38%|███▊ | 2826/7500 [1:35:19<2:37:56, 2.03s/it] 38%|███▊ | 2827/7500 [1:35:21<2:37:36, 2.02s/it] 38%|███▊ | 2828/7500 [1:35:23<2:35:03, 1.99s/it] 38%|███▊ | 2829/7500 [1:35:25<2:36:19, 2.01s/it] 38%|███▊ | 2830/7500 [1:35:27<2:37:05, 2.02s/it] {'loss': 0.4423, 'grad_norm': 3.707828998565674, 'learning_rate': 3.9172139955630774e-05, 'logqs/rejected': -2.414626359939575, 'logqs/chosen': -0.8936433792114258, 'logqs_logps/rejected': -0.026089375838637352, 'logps_logqs/chosen': 0.23016269505023956, 'epoch': 0.75} 38%|███▊ | 2830/7500 [1:35:27<2:37:05, 2.02s/it] 38%|███▊ | 2831/7500 [1:35:29<2:34:11, 1.98s/it] 38%|███▊ | 2832/7500 [1:35:31<2:35:14, 2.00s/it] 38%|███▊ | 2833/7500 [1:35:33<2:35:35, 2.00s/it] 38%|███▊ | 2834/7500 [1:35:35<2:36:05, 2.01s/it] 38%|███▊ | 2835/7500 [1:35:37<2:36:13, 2.01s/it] 38%|███▊ | 2836/7500 [1:35:39<2:35:29, 2.00s/it] 38%|███▊ | 2837/7500 [1:35:41<2:33:59, 1.98s/it] 38%|███▊ | 2838/7500 [1:35:43<2:35:03, 2.00s/it] 38%|███▊ | 2839/7500 [1:35:45<2:35:37, 2.00s/it] 38%|███▊ | 2840/7500 [1:35:47<2:36:25, 2.01s/it] {'loss': 0.42, 'grad_norm': 4.136870861053467, 'learning_rate': 3.9076133727299155e-05, 'logqs/rejected': -2.528085231781006, 'logqs/chosen': -0.8792393803596497, 'logqs_logps/rejected': -0.02193986251950264, 'logps_logqs/chosen': 0.2247266322374344, 'epoch': 0.76} 38%|███▊ | 2840/7500 [1:35:47<2:36:25, 2.01s/it] 38%|███▊ | 2841/7500 [1:35:49<2:36:10, 2.01s/it] 38%|███▊ | 2842/7500 [1:35:51<2:37:59, 2.04s/it] 38%|███▊ | 2843/7500 [1:35:53<2:38:24, 2.04s/it] 38%|███▊ | 2844/7500 [1:35:55<2:36:02, 2.01s/it] 38%|███▊ | 2845/7500 [1:35:57<2:36:15, 2.01s/it] 38%|███▊ | 2846/7500 [1:35:59<2:36:31, 2.02s/it] 38%|███▊ | 2847/7500 [1:36:01<2:36:29, 2.02s/it] 38%|███▊ | 2848/7500 [1:36:03<2:37:19, 2.03s/it] 38%|███▊ | 2849/7500 [1:36:05<2:39:29, 2.06s/it] 38%|███▊ | 2850/7500 [1:36:07<2:39:32, 2.06s/it] {'loss': 0.4285, 'grad_norm': 3.7540361881256104, 'learning_rate': 3.897982258676867e-05, 'logqs/rejected': -2.3323235511779785, 'logqs/chosen': -0.9129384756088257, 'logqs_logps/rejected': -0.031564708799123764, 'logps_logqs/chosen': 0.22876711189746857, 'epoch': 0.76} 38%|███▊ | 2850/7500 [1:36:07<2:39:32, 2.06s/it] 38%|███▊ | 2851/7500 [1:36:09<2:38:29, 2.05s/it] 38%|███▊ | 2852/7500 [1:36:12<2:38:34, 2.05s/it] 38%|███▊ | 2853/7500 [1:36:14<2:36:55, 2.03s/it] 38%|███▊ | 2854/7500 [1:36:15<2:35:19, 2.01s/it] 38%|███▊ | 2855/7500 [1:36:18<2:37:48, 2.04s/it] 38%|███▊ | 2856/7500 [1:36:20<2:37:22, 2.03s/it] 38%|███▊ | 2857/7500 [1:36:22<2:37:15, 2.03s/it] 38%|███▊ | 2858/7500 [1:36:24<2:35:30, 2.01s/it] 38%|███▊ | 2859/7500 [1:36:26<2:34:24, 2.00s/it] 38%|███▊ | 2860/7500 [1:36:28<2:34:09, 1.99s/it] {'loss': 0.4192, 'grad_norm': 3.4880199432373047, 'learning_rate': 3.888320862029698e-05, 'logqs/rejected': -2.430290460586548, 'logqs/chosen': -0.874978244304657, 'logqs_logps/rejected': -0.030040567740797997, 'logps_logqs/chosen': 0.21918873488903046, 'epoch': 0.76} 38%|███▊ | 2860/7500 [1:36:28<2:34:09, 1.99s/it] 38%|███▊ | 2861/7500 [1:36:30<2:35:58, 2.02s/it] 38%|███▊ | 2862/7500 [1:36:32<2:36:57, 2.03s/it] 38%|███▊ | 2863/7500 [1:36:34<2:35:31, 2.01s/it] 38%|███▊ | 2864/7500 [1:36:36<2:35:32, 2.01s/it] 38%|███▊ | 2865/7500 [1:36:38<2:36:28, 2.03s/it] 38%|███▊ | 2866/7500 [1:36:40<2:37:13, 2.04s/it] 38%|███▊ | 2867/7500 [1:36:42<2:38:41, 2.06s/it] 38%|███▊ | 2868/7500 [1:36:44<2:37:28, 2.04s/it] 38%|███▊ | 2869/7500 [1:36:46<2:38:37, 2.06s/it] 38%|███▊ | 2870/7500 [1:36:48<2:37:37, 2.04s/it] {'loss': 0.4493, 'grad_norm': 3.6029040813446045, 'learning_rate': 3.8786293920701435e-05, 'logqs/rejected': -2.5938100814819336, 'logqs/chosen': -0.8797178268432617, 'logqs_logps/rejected': -0.021689143031835556, 'logps_logqs/chosen': 0.2307852804660797, 'epoch': 0.77} 38%|███▊ | 2870/7500 [1:36:48<2:37:37, 2.04s/it] 38%|███▊ | 2871/7500 [1:36:50<2:38:40, 2.06s/it] 38%|███▊ | 2872/7500 [1:36:52<2:38:46, 2.06s/it] 38%|███▊ | 2873/7500 [1:36:54<2:39:06, 2.06s/it] 38%|███▊ | 2874/7500 [1:36:56<2:38:59, 2.06s/it] 38%|███▊ | 2875/7500 [1:36:58<2:35:27, 2.02s/it] 38%|███▊ | 2876/7500 [1:37:00<2:34:41, 2.01s/it] 38%|███▊ | 2877/7500 [1:37:02<2:36:04, 2.03s/it] 38%|███▊ | 2878/7500 [1:37:04<2:33:58, 2.00s/it] 38%|███▊ | 2879/7500 [1:37:06<2:35:01, 2.01s/it] 38%|███▊ | 2880/7500 [1:37:08<2:35:35, 2.02s/it] {'loss': 0.4435, 'grad_norm': 3.7607977390289307, 'learning_rate': 3.868908058731376e-05, 'logqs/rejected': -2.4751203060150146, 'logqs/chosen': -0.9057704210281372, 'logqs_logps/rejected': -0.01967875473201275, 'logps_logqs/chosen': 0.23474808037281036, 'epoch': 0.77} 38%|███▊ | 2880/7500 [1:37:08<2:35:35, 2.02s/it] 38%|███▊ | 2881/7500 [1:37:10<2:33:52, 2.00s/it] 38%|███▊ | 2882/7500 [1:37:12<2:37:42, 2.05s/it] 38%|███▊ | 2883/7500 [1:37:14<2:36:53, 2.04s/it] 38%|███▊ | 2884/7500 [1:37:16<2:35:39, 2.02s/it] 38%|███▊ | 2885/7500 [1:37:18<2:36:04, 2.03s/it] 38%|███▊ | 2886/7500 [1:37:21<2:39:00, 2.07s/it] 38%|███▊ | 2887/7500 [1:37:23<2:36:27, 2.03s/it] 39%|███▊ | 2888/7500 [1:37:25<2:35:44, 2.03s/it] 39%|███▊ | 2889/7500 [1:37:27<2:34:37, 2.01s/it] 39%|███▊ | 2890/7500 [1:37:29<2:35:43, 2.03s/it] {'loss': 0.4411, 'grad_norm': 4.167449474334717, 'learning_rate': 3.859157072593458e-05, 'logqs/rejected': -2.4715757369995117, 'logqs/chosen': -0.8963717222213745, 'logqs_logps/rejected': -0.02230570651590824, 'logps_logqs/chosen': 0.22848200798034668, 'epoch': 0.77} 39%|███▊ | 2890/7500 [1:37:29<2:35:43, 2.03s/it] 39%|███▊ | 2891/7500 [1:37:31<2:33:42, 2.00s/it] 39%|███▊ | 2892/7500 [1:37:33<2:36:46, 2.04s/it] 39%|███▊ | 2893/7500 [1:37:35<2:40:31, 2.09s/it] 39%|███▊ | 2894/7500 [1:37:37<2:40:58, 2.10s/it] 39%|███▊ | 2895/7500 [1:37:39<2:40:18, 2.09s/it] 39%|███▊ | 2896/7500 [1:37:41<2:39:25, 2.08s/it] 39%|███▊ | 2897/7500 [1:37:43<2:36:48, 2.04s/it] 39%|███▊ | 2898/7500 [1:37:45<2:36:32, 2.04s/it] 39%|███▊ | 2899/7500 [1:37:47<2:36:04, 2.04s/it] 39%|███▊ | 2900/7500 [1:37:49<2:34:08, 2.01s/it] {'loss': 0.4383, 'grad_norm': 3.5622506141662598, 'learning_rate': 3.8493766448787825e-05, 'logqs/rejected': -2.5584020614624023, 'logqs/chosen': -0.8894013166427612, 'logqs_logps/rejected': -0.019168907776474953, 'logps_logqs/chosen': 0.2300434559583664, 'epoch': 0.77} 39%|███▊ | 2900/7500 [1:37:49<2:34:08, 2.01s/it] 39%|███▊ | 2901/7500 [1:37:51<2:32:08, 1.98s/it] 39%|███▊ | 2902/7500 [1:37:53<2:32:31, 1.99s/it] 39%|███▊ | 2903/7500 [1:37:55<2:34:23, 2.02s/it] 39%|███▊ | 2904/7500 [1:37:57<2:35:33, 2.03s/it] 39%|███▊ | 2905/7500 [1:37:59<2:35:05, 2.03s/it] 39%|███▊ | 2906/7500 [1:38:01<2:31:58, 1.98s/it] 39%|███▉ | 2907/7500 [1:38:03<2:33:09, 2.00s/it] 39%|███▉ | 2908/7500 [1:38:05<2:33:13, 2.00s/it] 39%|███▉ | 2909/7500 [1:38:07<2:32:56, 2.00s/it] 39%|███▉ | 2910/7500 [1:38:09<2:33:21, 2.00s/it] {'loss': 0.4551, 'grad_norm': 4.02085018157959, 'learning_rate': 3.8395669874474915e-05, 'logqs/rejected': -2.751371383666992, 'logqs/chosen': -0.8817911148071289, 'logqs_logps/rejected': -0.020578911527991295, 'logps_logqs/chosen': 0.2277638465166092, 'epoch': 0.78} 39%|███▉ | 2910/7500 [1:38:09<2:33:21, 2.00s/it] 39%|███▉ | 2911/7500 [1:38:11<2:35:09, 2.03s/it] 39%|███▉ | 2912/7500 [1:38:13<2:35:27, 2.03s/it] 39%|███▉ | 2913/7500 [1:38:15<2:35:12, 2.03s/it] 39%|███▉ | 2914/7500 [1:38:17<2:32:34, 2.00s/it] 39%|███▉ | 2915/7500 [1:38:19<2:35:32, 2.04s/it] 39%|███▉ | 2916/7500 [1:38:21<2:36:07, 2.04s/it] 39%|███▉ | 2917/7500 [1:38:23<2:36:28, 2.05s/it] 39%|███▉ | 2918/7500 [1:38:25<2:33:51, 2.01s/it] 39%|███▉ | 2919/7500 [1:38:27<2:34:57, 2.03s/it] 39%|███▉ | 2920/7500 [1:38:29<2:33:40, 2.01s/it] {'loss': 0.4565, 'grad_norm': 3.811269760131836, 'learning_rate': 3.829728312792895e-05, 'logqs/rejected': -2.689079999923706, 'logqs/chosen': -0.9337469935417175, 'logqs_logps/rejected': -0.022522952407598495, 'logps_logqs/chosen': 0.23483017086982727, 'epoch': 0.78} 39%|███▉ | 2920/7500 [1:38:29<2:33:40, 2.01s/it] 39%|███▉ | 2921/7500 [1:38:31<2:34:30, 2.02s/it] 39%|███▉ | 2922/7500 [1:38:33<2:33:20, 2.01s/it] 39%|███▉ | 2923/7500 [1:38:36<2:35:20, 2.04s/it] 39%|███▉ | 2924/7500 [1:38:38<2:36:41, 2.05s/it] 39%|███▉ | 2925/7500 [1:38:40<2:33:15, 2.01s/it] 39%|███▉ | 2926/7500 [1:38:42<2:34:09, 2.02s/it] 39%|███▉ | 2927/7500 [1:38:44<2:34:16, 2.02s/it] 39%|███▉ | 2928/7500 [1:38:46<2:33:18, 2.01s/it] 39%|███▉ | 2929/7500 [1:38:48<2:33:23, 2.01s/it] 39%|███▉ | 2930/7500 [1:38:50<2:34:33, 2.03s/it] {'loss': 0.4407, 'grad_norm': 3.7615015506744385, 'learning_rate': 3.819860834036859e-05, 'logqs/rejected': -2.572749614715576, 'logqs/chosen': -0.8981397747993469, 'logqs_logps/rejected': -0.02550727128982544, 'logps_logqs/chosen': 0.22608724236488342, 'epoch': 0.78} 39%|███▉ | 2930/7500 [1:38:50<2:34:33, 2.03s/it] 39%|███▉ | 2931/7500 [1:38:52<2:36:26, 2.05s/it] 39%|███▉ | 2932/7500 [1:38:54<2:35:55, 2.05s/it] 39%|███▉ | 2933/7500 [1:38:56<2:37:25, 2.07s/it] 39%|███▉ | 2934/7500 [1:38:58<2:35:59, 2.05s/it] 39%|███▉ | 2935/7500 [1:39:00<2:35:11, 2.04s/it] 39%|███▉ | 2936/7500 [1:39:02<2:34:52, 2.04s/it] 39%|███▉ | 2937/7500 [1:39:04<2:35:13, 2.04s/it] 39%|███▉ | 2938/7500 [1:39:06<2:33:22, 2.02s/it] 39%|███▉ | 2939/7500 [1:39:08<2:33:25, 2.02s/it] 39%|███▉ | 2940/7500 [1:39:10<2:33:12, 2.02s/it] {'loss': 0.4528, 'grad_norm': 4.292747497558594, 'learning_rate': 3.8099647649251986e-05, 'logqs/rejected': -2.5647823810577393, 'logqs/chosen': -0.9450405240058899, 'logqs_logps/rejected': -0.023939544335007668, 'logps_logqs/chosen': 0.23404863476753235, 'epoch': 0.78} 39%|███▉ | 2940/7500 [1:39:10<2:33:12, 2.02s/it] 39%|███▉ | 2941/7500 [1:39:12<2:31:23, 1.99s/it] 39%|███▉ | 2942/7500 [1:39:14<2:34:18, 2.03s/it] 39%|███▉ | 2943/7500 [1:39:16<2:35:39, 2.05s/it] 39%|███▉ | 2944/7500 [1:39:18<2:36:42, 2.06s/it] 39%|███▉ | 2945/7500 [1:39:20<2:35:58, 2.05s/it] 39%|███▉ | 2946/7500 [1:39:22<2:34:27, 2.04s/it] 39%|███▉ | 2947/7500 [1:39:24<2:33:22, 2.02s/it] 39%|███▉ | 2948/7500 [1:39:26<2:31:32, 2.00s/it] 39%|███▉ | 2949/7500 [1:39:28<2:30:55, 1.99s/it] 39%|███▉ | 2950/7500 [1:39:30<2:32:40, 2.01s/it] {'loss': 0.4439, 'grad_norm': 3.7484307289123535, 'learning_rate': 3.8000403198230387e-05, 'logqs/rejected': -2.5638175010681152, 'logqs/chosen': -0.9213449358940125, 'logqs_logps/rejected': -0.018000179901719093, 'logps_logqs/chosen': 0.22730064392089844, 'epoch': 0.79} 39%|███▉ | 2950/7500 [1:39:30<2:32:40, 2.01s/it] 39%|███▉ | 2951/7500 [1:39:32<2:31:21, 2.00s/it] 39%|███▉ | 2952/7500 [1:39:34<2:31:36, 2.00s/it] 39%|███▉ | 2953/7500 [1:39:36<2:33:53, 2.03s/it] 39%|███▉ | 2954/7500 [1:39:38<2:35:11, 2.05s/it] 39%|███▉ | 2955/7500 [1:39:40<2:33:33, 2.03s/it] 39%|███▉ | 2956/7500 [1:39:42<2:32:36, 2.02s/it] 39%|███▉ | 2957/7500 [1:39:44<2:32:04, 2.01s/it] 39%|███▉ | 2958/7500 [1:39:46<2:30:31, 1.99s/it] 39%|███▉ | 2959/7500 [1:39:48<2:29:37, 1.98s/it] 39%|███▉ | 2960/7500 [1:39:50<2:30:30, 1.99s/it] {'loss': 0.4263, 'grad_norm': 3.5979456901550293, 'learning_rate': 3.790087713710179e-05, 'logqs/rejected': -2.394096851348877, 'logqs/chosen': -0.8920658826828003, 'logqs_logps/rejected': -0.020563432946801186, 'logps_logqs/chosen': 0.22270694375038147, 'epoch': 0.79} 39%|███▉ | 2960/7500 [1:39:50<2:30:30, 1.99s/it] 39%|███▉ | 2961/7500 [1:39:52<2:31:01, 2.00s/it] 39%|███▉ | 2962/7500 [1:39:54<2:33:40, 2.03s/it] 40%|███▉ | 2963/7500 [1:39:56<2:34:05, 2.04s/it] 40%|███▉ | 2964/7500 [1:39:59<2:34:30, 2.04s/it] 40%|███▉ | 2965/7500 [1:40:00<2:32:17, 2.01s/it] 40%|███▉ | 2966/7500 [1:40:02<2:30:40, 1.99s/it] 40%|███▉ | 2967/7500 [1:40:04<2:30:55, 2.00s/it] 40%|███▉ | 2968/7500 [1:40:06<2:30:47, 2.00s/it] 40%|███▉ | 2969/7500 [1:40:09<2:34:48, 2.05s/it] 40%|███▉ | 2970/7500 [1:40:11<2:35:53, 2.06s/it] {'loss': 0.43, 'grad_norm': 4.3883514404296875, 'learning_rate': 3.780107162176429e-05, 'logqs/rejected': -2.6107547283172607, 'logqs/chosen': -0.9106336832046509, 'logqs_logps/rejected': -0.03301050513982773, 'logps_logqs/chosen': 0.23373842239379883, 'epoch': 0.79} 40%|███▉ | 2970/7500 [1:40:11<2:35:53, 2.06s/it] 40%|███▉ | 2971/7500 [1:40:13<2:34:42, 2.05s/it] 40%|███▉ | 2972/7500 [1:40:15<2:36:36, 2.08s/it] 40%|███▉ | 2973/7500 [1:40:17<2:35:45, 2.06s/it] 40%|███▉ | 2974/7500 [1:40:19<2:34:09, 2.04s/it] 40%|███▉ | 2975/7500 [1:40:21<2:37:23, 2.09s/it] 40%|███▉ | 2976/7500 [1:40:23<2:35:43, 2.07s/it] 40%|███▉ | 2977/7500 [1:40:25<2:33:50, 2.04s/it] 40%|███▉ | 2978/7500 [1:40:27<2:30:52, 2.00s/it] 40%|███▉ | 2979/7500 [1:40:29<2:33:29, 2.04s/it] 40%|███▉ | 2980/7500 [1:40:31<2:32:24, 2.02s/it] {'loss': 0.45, 'grad_norm': 3.9096062183380127, 'learning_rate': 3.770098881416945e-05, 'logqs/rejected': -2.650362491607666, 'logqs/chosen': -0.8972746133804321, 'logqs_logps/rejected': -0.020671453326940536, 'logps_logqs/chosen': 0.2306194305419922, 'epoch': 0.79} 40%|███▉ | 2980/7500 [1:40:31<2:32:24, 2.02s/it] 40%|███▉ | 2981/7500 [1:40:33<2:31:02, 2.01s/it] 40%|███▉ | 2982/7500 [1:40:35<2:33:18, 2.04s/it] 40%|███▉ | 2983/7500 [1:40:37<2:34:47, 2.06s/it] 40%|███▉ | 2984/7500 [1:40:39<2:33:54, 2.04s/it] 40%|███▉ | 2985/7500 [1:40:41<2:34:50, 2.06s/it] 40%|███▉ | 2986/7500 [1:40:43<2:32:31, 2.03s/it] 40%|███▉ | 2987/7500 [1:40:45<2:32:24, 2.03s/it] 40%|███▉ | 2988/7500 [1:40:47<2:32:58, 2.03s/it] 40%|███▉ | 2989/7500 [1:40:50<2:36:19, 2.08s/it] 40%|███▉ | 2990/7500 [1:40:52<2:36:06, 2.08s/it] {'loss': 0.4444, 'grad_norm': 3.9620654582977295, 'learning_rate': 3.7600630882275416e-05, 'logqs/rejected': -2.55391526222229, 'logqs/chosen': -0.9114381670951843, 'logqs_logps/rejected': -0.025081995874643326, 'logps_logqs/chosen': 0.23008975386619568, 'epoch': 0.8} 40%|███▉ | 2990/7500 [1:40:52<2:36:06, 2.08s/it] 40%|███▉ | 2991/7500 [1:40:54<2:32:59, 2.04s/it] 40%|███▉ | 2992/7500 [1:40:56<2:31:12, 2.01s/it] 40%|███▉ | 2993/7500 [1:40:58<2:33:09, 2.04s/it] 40%|███▉ | 2994/7500 [1:41:00<2:33:42, 2.05s/it] 40%|███▉ | 2995/7500 [1:41:02<2:32:36, 2.03s/it] 40%|███▉ | 2996/7500 [1:41:04<2:32:37, 2.03s/it] 40%|███▉ | 2997/7500 [1:41:06<2:35:43, 2.08s/it] 40%|███▉ | 2998/7500 [1:41:08<2:34:53, 2.06s/it] 40%|███▉ | 2999/7500 [1:41:10<2:34:53, 2.06s/it] 40%|████ | 3000/7500 [1:41:12<2:33:46, 2.05s/it] {'loss': 0.4554, 'grad_norm': 3.7428054809570312, 'learning_rate': 3.7500000000000003e-05, 'logqs/rejected': -2.5754120349884033, 'logqs/chosen': -0.9121485948562622, 'logqs_logps/rejected': -0.01907043717801571, 'logps_logqs/chosen': 0.23264841735363007, 'epoch': 0.8} 40%|████ | 3000/7500 [1:41:12<2:33:46, 2.05s/it] 40%|████ | 3001/7500 [1:41:14<2:33:24, 2.05s/it] 40%|████ | 3002/7500 [1:41:16<2:30:42, 2.01s/it] 40%|████ | 3003/7500 [1:41:18<2:35:03, 2.07s/it] 40%|████ | 3004/7500 [1:41:20<2:32:52, 2.04s/it] 40%|████ | 3005/7500 [1:41:22<2:33:06, 2.04s/it] 40%|████ | 3006/7500 [1:41:24<2:33:55, 2.06s/it] 40%|████ | 3007/7500 [1:41:26<2:30:27, 2.01s/it] 40%|████ | 3008/7500 [1:41:28<2:31:56, 2.03s/it] 40%|████ | 3009/7500 [1:41:30<2:31:18, 2.02s/it] 40%|████ | 3010/7500 [1:41:32<2:31:37, 2.03s/it] {'loss': 0.4544, 'grad_norm': 3.5803112983703613, 'learning_rate': 3.739909834717356e-05, 'logqs/rejected': -2.813368320465088, 'logqs/chosen': -0.8939613103866577, 'logqs_logps/rejected': -0.02006940171122551, 'logps_logqs/chosen': 0.2225426733493805, 'epoch': 0.8} 40%|████ | 3010/7500 [1:41:32<2:31:37, 2.03s/it] 40%|████ | 3011/7500 [1:41:34<2:30:53, 2.02s/it] 40%|████ | 3012/7500 [1:41:36<2:31:00, 2.02s/it] 40%|████ | 3013/7500 [1:41:38<2:28:57, 1.99s/it] 40%|████ | 3014/7500 [1:41:40<2:28:59, 1.99s/it] 40%|████ | 3015/7500 [1:41:42<2:30:15, 2.01s/it] 40%|████ | 3016/7500 [1:41:44<2:28:36, 1.99s/it] 40%|████ | 3017/7500 [1:41:46<2:29:49, 2.01s/it] 40%|████ | 3018/7500 [1:41:48<2:33:37, 2.06s/it] 40%|████ | 3019/7500 [1:41:50<2:31:20, 2.03s/it] 40%|████ | 3020/7500 [1:41:52<2:30:04, 2.01s/it] {'loss': 0.43, 'grad_norm': 3.903287649154663, 'learning_rate': 3.7297928109491765e-05, 'logqs/rejected': -2.5404117107391357, 'logqs/chosen': -0.9106259346008301, 'logqs_logps/rejected': -0.02044411189854145, 'logps_logqs/chosen': 0.21946486830711365, 'epoch': 0.81} 40%|████ | 3020/7500 [1:41:52<2:30:04, 2.01s/it] 40%|████ | 3021/7500 [1:41:54<2:30:19, 2.01s/it] 40%|████ | 3022/7500 [1:41:56<2:29:17, 2.00s/it] 40%|████ | 3023/7500 [1:41:58<2:28:23, 1.99s/it] 40%|████ | 3024/7500 [1:42:00<2:28:18, 1.99s/it] 40%|████ | 3025/7500 [1:42:02<2:29:33, 2.01s/it] 40%|████ | 3026/7500 [1:42:04<2:27:36, 1.98s/it] 40%|████ | 3027/7500 [1:42:06<2:30:23, 2.02s/it] 40%|████ | 3028/7500 [1:42:08<2:31:09, 2.03s/it] 40%|████ | 3029/7500 [1:42:10<2:29:42, 2.01s/it] 40%|████ | 3030/7500 [1:42:13<2:31:19, 2.03s/it] {'loss': 0.4394, 'grad_norm': 3.981236457824707, 'learning_rate': 3.719649147846832e-05, 'logqs/rejected': -2.684596061706543, 'logqs/chosen': -0.8673830032348633, 'logqs_logps/rejected': -0.025381138548254967, 'logps_logqs/chosen': 0.22386625409126282, 'epoch': 0.81} 40%|████ | 3030/7500 [1:42:13<2:31:19, 2.03s/it] 40%|████ | 3031/7500 [1:42:15<2:32:13, 2.04s/it] 40%|████ | 3032/7500 [1:42:17<2:30:30, 2.02s/it] 40%|████ | 3033/7500 [1:42:19<2:29:02, 2.00s/it] 40%|████ | 3034/7500 [1:42:21<2:28:47, 2.00s/it] 40%|████ | 3035/7500 [1:42:22<2:27:16, 1.98s/it] 40%|████ | 3036/7500 [1:42:24<2:27:29, 1.98s/it] 40%|████ | 3037/7500 [1:42:27<2:30:09, 2.02s/it] 41%|████ | 3038/7500 [1:42:29<2:31:44, 2.04s/it] 41%|████ | 3039/7500 [1:42:31<2:30:04, 2.02s/it] 41%|████ | 3040/7500 [1:42:33<2:31:04, 2.03s/it] {'loss': 0.442, 'grad_norm': 6.083058834075928, 'learning_rate': 3.7094790651387414e-05, 'logqs/rejected': -2.601163387298584, 'logqs/chosen': -0.9018617868423462, 'logqs_logps/rejected': -0.02541309967637062, 'logps_logqs/chosen': 0.22815506160259247, 'epoch': 0.81} 41%|████ | 3040/7500 [1:42:33<2:31:04, 2.03s/it] 41%|████ | 3041/7500 [1:42:35<2:31:04, 2.03s/it] 41%|████ | 3042/7500 [1:42:37<2:31:08, 2.03s/it] 41%|████ | 3043/7500 [1:42:39<2:32:43, 2.06s/it] 41%|████ | 3044/7500 [1:42:41<2:31:23, 2.04s/it] 41%|████ | 3045/7500 [1:42:43<2:31:35, 2.04s/it] 41%|████ | 3046/7500 [1:42:45<2:30:22, 2.03s/it] 41%|████ | 3047/7500 [1:42:47<2:29:03, 2.01s/it] 41%|████ | 3048/7500 [1:42:49<2:28:04, 2.00s/it] 41%|████ | 3049/7500 [1:42:51<2:27:24, 1.99s/it] 41%|████ | 3050/7500 [1:42:53<2:28:16, 2.00s/it] {'loss': 0.4234, 'grad_norm': 3.465252161026001, 'learning_rate': 3.699282783125616e-05, 'logqs/rejected': -2.402512550354004, 'logqs/chosen': -0.9234975576400757, 'logqs_logps/rejected': -0.022936712950468063, 'logps_logqs/chosen': 0.22930936515331268, 'epoch': 0.81} 41%|████ | 3050/7500 [1:42:53<2:28:16, 2.00s/it] 41%|████ | 3051/7500 [1:42:55<2:30:30, 2.03s/it] 41%|████ | 3052/7500 [1:42:57<2:31:11, 2.04s/it] 41%|████ | 3053/7500 [1:42:59<2:31:38, 2.05s/it] 41%|████ | 3054/7500 [1:43:01<2:31:48, 2.05s/it] 41%|████ | 3055/7500 [1:43:03<2:31:19, 2.04s/it] 41%|████ | 3056/7500 [1:43:05<2:29:49, 2.02s/it] 41%|████ | 3057/7500 [1:43:07<2:30:39, 2.03s/it] 41%|████ | 3058/7500 [1:43:09<2:31:09, 2.04s/it] 41%|████ | 3059/7500 [1:43:11<2:29:02, 2.01s/it] 41%|████ | 3060/7500 [1:43:13<2:29:06, 2.02s/it] {'loss': 0.4431, 'grad_norm': 3.5856401920318604, 'learning_rate': 3.689060522675689e-05, 'logqs/rejected': -2.800020217895508, 'logqs/chosen': -0.9024070501327515, 'logqs_logps/rejected': -0.017033426091074944, 'logps_logqs/chosen': 0.22129404544830322, 'epoch': 0.82} 41%|████ | 3060/7500 [1:43:13<2:29:06, 2.02s/it] 41%|████ | 3061/7500 [1:43:15<2:29:29, 2.02s/it] 41%|████ | 3062/7500 [1:43:17<2:30:28, 2.03s/it] 41%|████ | 3063/7500 [1:43:19<2:32:16, 2.06s/it] 41%|████ | 3064/7500 [1:43:21<2:32:05, 2.06s/it] 41%|████ | 3065/7500 [1:43:23<2:31:56, 2.06s/it] 41%|████ | 3066/7500 [1:43:25<2:29:23, 2.02s/it] 41%|████ | 3067/7500 [1:43:27<2:29:59, 2.03s/it] 41%|████ | 3068/7500 [1:43:30<2:30:43, 2.04s/it] 41%|████ | 3069/7500 [1:43:32<2:29:29, 2.02s/it] 41%|████ | 3070/7500 [1:43:34<2:31:58, 2.06s/it] {'loss': 0.4548, 'grad_norm': 4.440993309020996, 'learning_rate': 3.678812505219926e-05, 'logqs/rejected': -2.7330737113952637, 'logqs/chosen': -0.9434027671813965, 'logqs_logps/rejected': -0.028649646788835526, 'logps_logqs/chosen': 0.2355763465166092, 'epoch': 0.82} 41%|████ | 3070/7500 [1:43:34<2:31:58, 2.06s/it] 41%|████ | 3071/7500 [1:43:36<2:29:58, 2.03s/it] 41%|████ | 3072/7500 [1:43:38<2:30:33, 2.04s/it] 41%|████ | 3073/7500 [1:43:40<2:31:52, 2.06s/it] 41%|████ | 3074/7500 [1:43:42<2:33:06, 2.08s/it] 41%|████ | 3075/7500 [1:43:44<2:31:44, 2.06s/it] 41%|████ | 3076/7500 [1:43:46<2:29:34, 2.03s/it] 41%|████ | 3077/7500 [1:43:48<2:28:06, 2.01s/it] 41%|████ | 3078/7500 [1:43:50<2:27:10, 2.00s/it] 41%|████ | 3079/7500 [1:43:52<2:30:16, 2.04s/it] 41%|████ | 3080/7500 [1:43:54<2:30:44, 2.05s/it] {'loss': 0.4475, 'grad_norm': 3.100341796875, 'learning_rate': 3.6685389527472356e-05, 'logqs/rejected': -2.684462785720825, 'logqs/chosen': -0.8867018818855286, 'logqs_logps/rejected': -0.01362281758338213, 'logps_logqs/chosen': 0.22805781662464142, 'epoch': 0.82} 41%|████ | 3080/7500 [1:43:54<2:30:44, 2.05s/it] 41%|████ | 3081/7500 [1:43:56<2:30:48, 2.05s/it] 41%|████ | 3082/7500 [1:43:58<2:31:19, 2.06s/it] 41%|████ | 3083/7500 [1:44:00<2:28:51, 2.02s/it] 41%|████ | 3084/7500 [1:44:02<2:30:20, 2.04s/it] 41%|████ | 3085/7500 [1:44:04<2:30:48, 2.05s/it] 41%|████ | 3086/7500 [1:44:06<2:30:14, 2.04s/it] 41%|████ | 3087/7500 [1:44:08<2:29:26, 2.03s/it] 41%|████ | 3088/7500 [1:44:10<2:30:38, 2.05s/it] 41%|████ | 3089/7500 [1:44:12<2:30:46, 2.05s/it] 41%|████ | 3090/7500 [1:44:14<2:29:23, 2.03s/it] {'loss': 0.4337, 'grad_norm': 3.2097766399383545, 'learning_rate': 3.6582400877996546e-05, 'logqs/rejected': -2.498277187347412, 'logqs/chosen': -0.8945775032043457, 'logqs_logps/rejected': -0.021319231018424034, 'logps_logqs/chosen': 0.2263825386762619, 'epoch': 0.82} 41%|████ | 3090/7500 [1:44:14<2:29:23, 2.03s/it] 41%|████ | 3091/7500 [1:44:16<2:28:22, 2.02s/it] 41%|████ | 3092/7500 [1:44:18<2:28:00, 2.01s/it] 41%|████ | 3093/7500 [1:44:21<2:30:00, 2.04s/it] 41%|████▏ | 3094/7500 [1:44:23<2:29:41, 2.04s/it] 41%|████▏ | 3095/7500 [1:44:25<2:30:47, 2.05s/it] 41%|████▏ | 3096/7500 [1:44:27<2:32:25, 2.08s/it] 41%|████▏ | 3097/7500 [1:44:29<2:30:08, 2.05s/it] 41%|████▏ | 3098/7500 [1:44:31<2:30:41, 2.05s/it] 41%|████▏ | 3099/7500 [1:44:33<2:31:27, 2.06s/it] 41%|████▏ | 3100/7500 [1:44:35<2:31:26, 2.07s/it] {'loss': 0.4172, 'grad_norm': 3.7555148601531982, 'learning_rate': 3.6479161334675296e-05, 'logqs/rejected': -2.5235519409179688, 'logqs/chosen': -0.8903777003288269, 'logqs_logps/rejected': -0.028203044086694717, 'logps_logqs/chosen': 0.22523824870586395, 'epoch': 0.83} 41%|████▏ | 3100/7500 [1:44:35<2:31:26, 2.07s/it] 41%|████▏ | 3101/7500 [1:44:37<2:32:57, 2.09s/it] 41%|████▏ | 3102/7500 [1:44:39<2:32:33, 2.08s/it] 41%|████▏ | 3103/7500 [1:44:41<2:33:17, 2.09s/it] 41%|████▏ | 3104/7500 [1:44:43<2:33:33, 2.10s/it] 41%|████▏ | 3105/7500 [1:44:45<2:32:15, 2.08s/it] 41%|████▏ | 3106/7500 [1:44:48<2:32:08, 2.08s/it] 41%|████▏ | 3107/7500 [1:44:49<2:29:45, 2.05s/it] 41%|████▏ | 3108/7500 [1:44:52<2:32:56, 2.09s/it] 41%|████▏ | 3109/7500 [1:44:54<2:32:08, 2.08s/it] 41%|████▏ | 3110/7500 [1:44:56<2:32:39, 2.09s/it] {'loss': 0.4221, 'grad_norm': 3.8926656246185303, 'learning_rate': 3.637567313384684e-05, 'logqs/rejected': -2.3679661750793457, 'logqs/chosen': -0.8941890001296997, 'logqs_logps/rejected': -0.024731408804655075, 'logps_logqs/chosen': 0.21966806054115295, 'epoch': 0.83} 41%|████▏ | 3110/7500 [1:44:56<2:32:39, 2.09s/it] 41%|████▏ | 3111/7500 [1:44:58<2:34:40, 2.11s/it] 41%|████▏ | 3112/7500 [1:45:00<2:34:44, 2.12s/it] 42%|████▏ | 3113/7500 [1:45:02<2:31:10, 2.07s/it] 42%|████▏ | 3114/7500 [1:45:04<2:32:26, 2.09s/it] 42%|████▏ | 3115/7500 [1:45:06<2:30:06, 2.05s/it] 42%|████▏ | 3116/7500 [1:45:08<2:30:48, 2.06s/it] 42%|████▏ | 3117/7500 [1:45:10<2:28:35, 2.03s/it] 42%|████▏ | 3118/7500 [1:45:12<2:28:08, 2.03s/it] 42%|████▏ | 3119/7500 [1:45:14<2:29:23, 2.05s/it] 42%|████▏ | 3120/7500 [1:45:16<2:30:13, 2.06s/it] {'loss': 0.459, 'grad_norm': 4.029929161071777, 'learning_rate': 3.627193851723577e-05, 'logqs/rejected': -2.664961099624634, 'logqs/chosen': -0.9281222224235535, 'logqs_logps/rejected': -0.014861220493912697, 'logps_logqs/chosen': 0.22001048922538757, 'epoch': 0.83} 42%|████▏ | 3120/7500 [1:45:16<2:30:13, 2.06s/it] 42%|████▏ | 3121/7500 [1:45:18<2:29:27, 2.05s/it] 42%|████▏ | 3122/7500 [1:45:21<2:33:30, 2.10s/it] 42%|████▏ | 3123/7500 [1:45:23<2:34:30, 2.12s/it] 42%|████▏ | 3124/7500 [1:45:25<2:33:13, 2.10s/it] 42%|████▏ | 3125/7500 [1:45:27<2:33:01, 2.10s/it] 42%|████▏ | 3126/7500 [1:45:29<2:32:41, 2.09s/it] 42%|████▏ | 3127/7500 [1:45:31<2:30:50, 2.07s/it] 42%|████▏ | 3128/7500 [1:45:33<2:30:50, 2.07s/it] 42%|████▏ | 3129/7500 [1:45:35<2:28:58, 2.04s/it] 42%|████▏ | 3130/7500 [1:45:37<2:29:39, 2.05s/it] {'loss': 0.4322, 'grad_norm': 3.725099802017212, 'learning_rate': 3.616795973190442e-05, 'logqs/rejected': -2.4726836681365967, 'logqs/chosen': -0.9071100950241089, 'logqs_logps/rejected': -0.018072936683893204, 'logps_logqs/chosen': 0.2308134138584137, 'epoch': 0.83} 42%|████▏ | 3130/7500 [1:45:37<2:29:39, 2.05s/it] 42%|████▏ | 3131/7500 [1:45:39<2:28:47, 2.04s/it] 42%|████▏ | 3132/7500 [1:45:41<2:30:31, 2.07s/it] 42%|████▏ | 3133/7500 [1:45:43<2:29:18, 2.05s/it] 42%|████▏ | 3134/7500 [1:45:45<2:28:49, 2.05s/it] 42%|████▏ | 3135/7500 [1:45:48<2:29:56, 2.06s/it] 42%|████▏ | 3136/7500 [1:45:49<2:27:44, 2.03s/it] 42%|████▏ | 3137/7500 [1:45:51<2:27:03, 2.02s/it] 42%|████▏ | 3138/7500 [1:45:54<2:28:21, 2.04s/it] 42%|████▏ | 3139/7500 [1:45:56<2:31:09, 2.08s/it] 42%|████▏ | 3140/7500 [1:45:58<2:30:43, 2.07s/it] {'loss': 0.4311, 'grad_norm': 3.601151466369629, 'learning_rate': 3.6063739030204224e-05, 'logqs/rejected': -2.5202383995056152, 'logqs/chosen': -0.8868942260742188, 'logqs_logps/rejected': -0.024305172264575958, 'logps_logqs/chosen': 0.21824780106544495, 'epoch': 0.84} 42%|████▏ | 3140/7500 [1:45:58<2:30:43, 2.07s/it] 42%|████▏ | 3141/7500 [1:46:00<2:29:31, 2.06s/it] 42%|████▏ | 3142/7500 [1:46:02<2:29:08, 2.05s/it] 42%|████▏ | 3143/7500 [1:46:04<2:28:40, 2.05s/it] 42%|████▏ | 3144/7500 [1:46:06<2:28:19, 2.04s/it] 42%|████▏ | 3145/7500 [1:46:08<2:28:30, 2.05s/it] 42%|████▏ | 3146/7500 [1:46:10<2:28:54, 2.05s/it] 42%|████▏ | 3147/7500 [1:46:12<2:29:54, 2.07s/it] 42%|████▏ | 3148/7500 [1:46:14<2:27:58, 2.04s/it] 42%|████▏ | 3149/7500 [1:46:16<2:26:30, 2.02s/it] 42%|████▏ | 3150/7500 [1:46:18<2:27:19, 2.03s/it] {'loss': 0.4014, 'grad_norm': 3.605487108230591, 'learning_rate': 3.5959278669726935e-05, 'logqs/rejected': -2.3402600288391113, 'logqs/chosen': -0.9043596386909485, 'logqs_logps/rejected': -0.035158414393663406, 'logps_logqs/chosen': 0.23164066672325134, 'epoch': 0.84} 42%|████▏ | 3150/7500 [1:46:18<2:27:19, 2.03s/it] 42%|████▏ | 3151/7500 [1:46:20<2:27:36, 2.04s/it] 42%|████▏ | 3152/7500 [1:46:22<2:29:26, 2.06s/it] 42%|████▏ | 3153/7500 [1:46:24<2:27:55, 2.04s/it] 42%|████▏ | 3154/7500 [1:46:27<2:32:10, 2.10s/it] 42%|████▏ | 3155/7500 [1:46:29<2:28:43, 2.05s/it] 42%|████▏ | 3156/7500 [1:46:31<2:30:22, 2.08s/it] 42%|████▏ | 3157/7500 [1:46:33<2:29:47, 2.07s/it] 42%|████▏ | 3158/7500 [1:46:35<2:30:58, 2.09s/it] 42%|████▏ | 3159/7500 [1:46:37<2:29:19, 2.06s/it] 42%|████▏ | 3160/7500 [1:46:39<2:28:32, 2.05s/it] {'loss': 0.4196, 'grad_norm': 5.723771572113037, 'learning_rate': 3.5854580913255703e-05, 'logqs/rejected': -2.45889949798584, 'logqs/chosen': -0.8915220499038696, 'logqs_logps/rejected': -0.028722798451781273, 'logps_logqs/chosen': 0.2244434356689453, 'epoch': 0.84} 42%|████▏ | 3160/7500 [1:46:39<2:28:32, 2.05s/it] 42%|████▏ | 3161/7500 [1:46:41<2:26:44, 2.03s/it] 42%|████▏ | 3162/7500 [1:46:43<2:26:14, 2.02s/it] 42%|████▏ | 3163/7500 [1:46:45<2:27:08, 2.04s/it] 42%|████▏ | 3164/7500 [1:46:47<2:28:20, 2.05s/it] 42%|████▏ | 3165/7500 [1:46:49<2:28:34, 2.06s/it] 42%|████▏ | 3166/7500 [1:46:51<2:27:54, 2.05s/it] 42%|████▏ | 3167/7500 [1:46:53<2:27:48, 2.05s/it] 42%|████▏ | 3168/7500 [1:46:55<2:28:02, 2.05s/it] 42%|████▏ | 3169/7500 [1:46:57<2:28:02, 2.05s/it] 42%|████▏ | 3170/7500 [1:46:59<2:29:53, 2.08s/it] {'loss': 0.4305, 'grad_norm': 4.02019739151001, 'learning_rate': 3.574964802871607e-05, 'logqs/rejected': -2.6955912113189697, 'logqs/chosen': -0.8812238574028015, 'logqs_logps/rejected': -0.030830953270196915, 'logps_logqs/chosen': 0.23082447052001953, 'epoch': 0.85} 42%|████▏ | 3170/7500 [1:46:59<2:29:53, 2.08s/it] 42%|████▏ | 3171/7500 [1:47:01<2:29:12, 2.07s/it] 42%|████▏ | 3172/7500 [1:47:04<2:30:29, 2.09s/it] 42%|████▏ | 3173/7500 [1:47:06<2:29:46, 2.08s/it] 42%|████▏ | 3174/7500 [1:47:08<2:28:07, 2.05s/it] 42%|████▏ | 3175/7500 [1:47:10<2:27:46, 2.05s/it] 42%|████▏ | 3176/7500 [1:47:12<2:28:52, 2.07s/it] 42%|████▏ | 3177/7500 [1:47:14<2:29:12, 2.07s/it] 42%|████▏ | 3178/7500 [1:47:16<2:29:13, 2.07s/it] 42%|████▏ | 3179/7500 [1:47:18<2:29:41, 2.08s/it] 42%|████▏ | 3180/7500 [1:47:20<2:30:04, 2.08s/it] {'loss': 0.4567, 'grad_norm': 4.102969169616699, 'learning_rate': 3.564448228912682e-05, 'logqs/rejected': -2.609161138534546, 'logqs/chosen': -0.9088441133499146, 'logqs_logps/rejected': -0.018410250544548035, 'logps_logqs/chosen': 0.22711165249347687, 'epoch': 0.85} 42%|████▏ | 3180/7500 [1:47:20<2:30:04, 2.08s/it] 42%|████▏ | 3181/7500 [1:47:22<2:28:20, 2.06s/it] 42%|████▏ | 3182/7500 [1:47:24<2:28:39, 2.07s/it] 42%|████▏ | 3183/7500 [1:47:26<2:29:50, 2.08s/it] 42%|████▏ | 3184/7500 [1:47:28<2:28:10, 2.06s/it] 42%|████▏ | 3185/7500 [1:47:30<2:29:34, 2.08s/it] 42%|████▏ | 3186/7500 [1:47:33<2:29:41, 2.08s/it] 42%|████▏ | 3187/7500 [1:47:35<2:28:01, 2.06s/it] 43%|████▎ | 3188/7500 [1:47:37<2:28:13, 2.06s/it] 43%|████▎ | 3189/7500 [1:47:39<2:27:53, 2.06s/it] 43%|████▎ | 3190/7500 [1:47:41<2:26:18, 2.04s/it] {'loss': 0.41, 'grad_norm': 4.064980983734131, 'learning_rate': 3.553908597255078e-05, 'logqs/rejected': -2.467982769012451, 'logqs/chosen': -0.8650034666061401, 'logqs_logps/rejected': -0.022132698446512222, 'logps_logqs/chosen': 0.21745768189430237, 'epoch': 0.85} 43%|████▎ | 3190/7500 [1:47:41<2:26:18, 2.04s/it] 43%|████▎ | 3191/7500 [1:47:43<2:28:08, 2.06s/it] 43%|████▎ | 3192/7500 [1:47:45<2:30:55, 2.10s/it] 43%|████▎ | 3193/7500 [1:47:47<2:28:04, 2.06s/it] 43%|████▎ | 3194/7500 [1:47:49<2:25:24, 2.03s/it] 43%|████▎ | 3195/7500 [1:47:51<2:26:44, 2.05s/it] 43%|████▎ | 3196/7500 [1:47:53<2:29:22, 2.08s/it] 43%|████▎ | 3197/7500 [1:47:55<2:28:36, 2.07s/it] 43%|████▎ | 3198/7500 [1:47:57<2:27:29, 2.06s/it] 43%|████▎ | 3199/7500 [1:47:59<2:27:14, 2.05s/it] 43%|████▎ | 3200/7500 [1:48:01<2:25:53, 2.04s/it] {'loss': 0.4392, 'grad_norm': 3.4080312252044678, 'learning_rate': 3.543346136204545e-05, 'logqs/rejected': -2.621488571166992, 'logqs/chosen': -0.925550103187561, 'logqs_logps/rejected': -0.02999820187687874, 'logps_logqs/chosen': 0.2318381369113922, 'epoch': 0.85} 43%|████▎ | 3200/7500 [1:48:01<2:25:53, 2.04s/it] 43%|████▎ | 3201/7500 [1:48:03<2:26:29, 2.04s/it] 43%|████▎ | 3202/7500 [1:48:05<2:25:35, 2.03s/it] 43%|████▎ | 3203/7500 [1:48:07<2:25:34, 2.03s/it] 43%|████▎ | 3204/7500 [1:48:09<2:26:20, 2.04s/it] 43%|████▎ | 3205/7500 [1:48:11<2:27:16, 2.06s/it] 43%|████▎ | 3206/7500 [1:48:14<2:26:53, 2.05s/it] 43%|████▎ | 3207/7500 [1:48:16<2:26:19, 2.04s/it] 43%|████▎ | 3208/7500 [1:48:18<2:27:11, 2.06s/it] 43%|████▎ | 3209/7500 [1:48:20<2:25:06, 2.03s/it] 43%|████▎ | 3210/7500 [1:48:22<2:27:49, 2.07s/it] {'loss': 0.4471, 'grad_norm': 3.674257755279541, 'learning_rate': 3.532761074561355e-05, 'logqs/rejected': -2.521242618560791, 'logqs/chosen': -0.948661208152771, 'logqs_logps/rejected': -0.022133037447929382, 'logps_logqs/chosen': 0.23230786621570587, 'epoch': 0.86} 43%|████▎ | 3210/7500 [1:48:22<2:27:49, 2.07s/it] 43%|████▎ | 3211/7500 [1:48:24<2:28:25, 2.08s/it] 43%|████▎ | 3212/7500 [1:48:26<2:27:30, 2.06s/it] 43%|████▎ | 3213/7500 [1:48:28<2:26:46, 2.05s/it] 43%|████▎ | 3214/7500 [1:48:30<2:28:02, 2.07s/it] 43%|████▎ | 3215/7500 [1:48:32<2:28:23, 2.08s/it] 43%|████▎ | 3216/7500 [1:48:34<2:27:17, 2.06s/it] 43%|████▎ | 3217/7500 [1:48:36<2:27:11, 2.06s/it] 43%|████▎ | 3218/7500 [1:48:38<2:29:01, 2.09s/it] 43%|████▎ | 3219/7500 [1:48:40<2:27:15, 2.06s/it] 43%|████▎ | 3220/7500 [1:48:43<2:28:36, 2.08s/it] {'loss': 0.4481, 'grad_norm': 3.823709487915039, 'learning_rate': 3.522153641615345e-05, 'logqs/rejected': -2.5989625453948975, 'logqs/chosen': -0.9304300546646118, 'logqs_logps/rejected': -0.022061355412006378, 'logps_logqs/chosen': 0.23011544346809387, 'epoch': 0.86} 43%|████▎ | 3220/7500 [1:48:43<2:28:36, 2.08s/it] 43%|████▎ | 3221/7500 [1:48:45<2:26:33, 2.05s/it] 43%|████▎ | 3222/7500 [1:48:47<2:25:40, 2.04s/it] 43%|████▎ | 3223/7500 [1:48:49<2:27:06, 2.06s/it] 43%|████▎ | 3224/7500 [1:48:51<2:26:46, 2.06s/it] 43%|████▎ | 3225/7500 [1:48:53<2:25:29, 2.04s/it] 43%|████▎ | 3226/7500 [1:48:55<2:25:04, 2.04s/it] 43%|████▎ | 3227/7500 [1:48:57<2:22:55, 2.01s/it] 43%|████▎ | 3228/7500 [1:48:59<2:24:24, 2.03s/it] 43%|████▎ | 3229/7500 [1:49:01<2:24:11, 2.03s/it] 43%|████▎ | 3230/7500 [1:49:03<2:23:47, 2.02s/it] {'loss': 0.4276, 'grad_norm': 3.605323076248169, 'learning_rate': 3.511524067140953e-05, 'logqs/rejected': -2.6055967807769775, 'logqs/chosen': -0.8766711354255676, 'logqs_logps/rejected': -0.024007007479667664, 'logps_logqs/chosen': 0.21867573261260986, 'epoch': 0.86} 43%|████▎ | 3230/7500 [1:49:03<2:23:47, 2.02s/it] 43%|████▎ | 3231/7500 [1:49:05<2:21:53, 1.99s/it] 43%|████▎ | 3232/7500 [1:49:07<2:20:19, 1.97s/it] 43%|████▎ | 3233/7500 [1:49:09<2:22:13, 2.00s/it] 43%|████▎ | 3234/7500 [1:49:11<2:24:09, 2.03s/it] 43%|████▎ | 3235/7500 [1:49:13<2:27:04, 2.07s/it] 43%|████▎ | 3236/7500 [1:49:15<2:26:53, 2.07s/it] 43%|████▎ | 3237/7500 [1:49:17<2:25:48, 2.05s/it] 43%|████▎ | 3238/7500 [1:49:19<2:26:11, 2.06s/it] 43%|████▎ | 3239/7500 [1:49:21<2:27:06, 2.07s/it] 43%|████▎ | 3240/7500 [1:49:23<2:24:34, 2.04s/it] {'loss': 0.4086, 'grad_norm': 3.6988658905029297, 'learning_rate': 3.5008725813922386e-05, 'logqs/rejected': -2.371478796005249, 'logqs/chosen': -0.8979811668395996, 'logqs_logps/rejected': -0.023869607597589493, 'logps_logqs/chosen': 0.22148974239826202, 'epoch': 0.86} 43%|████▎ | 3240/7500 [1:49:23<2:24:34, 2.04s/it] 43%|████▎ | 3241/7500 [1:49:25<2:26:32, 2.06s/it] 43%|████▎ | 3242/7500 [1:49:27<2:24:57, 2.04s/it] 43%|████▎ | 3243/7500 [1:49:29<2:25:19, 2.05s/it] 43%|████▎ | 3244/7500 [1:49:31<2:26:40, 2.07s/it] 43%|████▎ | 3245/7500 [1:49:34<2:26:53, 2.07s/it] 43%|████▎ | 3246/7500 [1:49:36<2:27:20, 2.08s/it] 43%|████▎ | 3247/7500 [1:49:38<2:25:31, 2.05s/it] 43%|████▎ | 3248/7500 [1:49:40<2:25:43, 2.06s/it] 43%|████▎ | 3249/7500 [1:49:42<2:25:48, 2.06s/it] 43%|████▎ | 3250/7500 [1:49:44<2:24:45, 2.04s/it] {'loss': 0.4202, 'grad_norm': 3.6390926837921143, 'learning_rate': 3.490199415097892e-05, 'logqs/rejected': -2.339326858520508, 'logqs/chosen': -0.9045270085334778, 'logqs_logps/rejected': -0.0223246943205595, 'logps_logqs/chosen': 0.22306868433952332, 'epoch': 0.87} 43%|████▎ | 3250/7500 [1:49:44<2:24:45, 2.04s/it] 43%|████▎ | 3251/7500 [1:49:46<2:25:59, 2.06s/it] 43%|████▎ | 3252/7500 [1:49:48<2:25:16, 2.05s/it] 43%|████▎ | 3253/7500 [1:49:50<2:25:01, 2.05s/it] 43%|████▎ | 3254/7500 [1:49:52<2:26:13, 2.07s/it] 43%|████▎ | 3255/7500 [1:49:54<2:27:05, 2.08s/it] 43%|████▎ | 3256/7500 [1:49:56<2:26:33, 2.07s/it] 43%|████▎ | 3257/7500 [1:49:58<2:25:56, 2.06s/it] 43%|████▎ | 3258/7500 [1:50:00<2:25:05, 2.05s/it] 43%|████▎ | 3259/7500 [1:50:02<2:26:11, 2.07s/it] 43%|████▎ | 3260/7500 [1:50:04<2:25:27, 2.06s/it] {'loss': 0.4099, 'grad_norm': 3.1895551681518555, 'learning_rate': 3.479504799456246e-05, 'logqs/rejected': -2.3461251258850098, 'logqs/chosen': -0.8772614598274231, 'logqs_logps/rejected': -0.022001493722200394, 'logps_logqs/chosen': 0.22071118652820587, 'epoch': 0.87} 43%|████▎ | 3260/7500 [1:50:04<2:25:27, 2.06s/it] 43%|████▎ | 3261/7500 [1:50:06<2:25:48, 2.06s/it] 43%|████▎ | 3262/7500 [1:50:09<2:25:11, 2.06s/it] 44%|████▎ | 3263/7500 [1:50:10<2:23:39, 2.03s/it] 44%|████▎ | 3264/7500 [1:50:13<2:24:43, 2.05s/it] 44%|████▎ | 3265/7500 [1:50:15<2:26:24, 2.07s/it] 44%|████▎ | 3266/7500 [1:50:17<2:24:42, 2.05s/it] 44%|████▎ | 3267/7500 [1:50:19<2:26:33, 2.08s/it] 44%|████▎ | 3268/7500 [1:50:21<2:30:15, 2.13s/it] 44%|████▎ | 3269/7500 [1:50:23<2:31:24, 2.15s/it] 44%|████▎ | 3270/7500 [1:50:25<2:30:20, 2.13s/it] {'loss': 0.4339, 'grad_norm': 4.197684288024902, 'learning_rate': 3.4687889661302576e-05, 'logqs/rejected': -2.452388048171997, 'logqs/chosen': -0.9084236025810242, 'logqs_logps/rejected': -0.02213212661445141, 'logps_logqs/chosen': 0.2202318012714386, 'epoch': 0.87} 44%|████▎ | 3270/7500 [1:50:25<2:30:20, 2.13s/it] 44%|████▎ | 3271/7500 [1:50:27<2:29:55, 2.13s/it] 44%|████▎ | 3272/7500 [1:50:30<2:29:03, 2.12s/it] 44%|████▎ | 3273/7500 [1:50:32<2:30:49, 2.14s/it] 44%|████▎ | 3274/7500 [1:50:34<2:28:55, 2.11s/it] 44%|████▎ | 3275/7500 [1:50:36<2:28:03, 2.10s/it] 44%|████▎ | 3276/7500 [1:50:38<2:28:28, 2.11s/it] 44%|████▎ | 3277/7500 [1:50:40<2:27:25, 2.09s/it] 44%|████▎ | 3278/7500 [1:50:42<2:25:00, 2.06s/it] 44%|████▎ | 3279/7500 [1:50:44<2:27:14, 2.09s/it] 44%|████▎ | 3280/7500 [1:50:46<2:28:18, 2.11s/it] {'loss': 0.4418, 'grad_norm': 4.178828239440918, 'learning_rate': 3.458052147242494e-05, 'logqs/rejected': -2.595640182495117, 'logqs/chosen': -0.925810694694519, 'logqs_logps/rejected': -0.03167225047945976, 'logps_logqs/chosen': 0.22507640719413757, 'epoch': 0.87} 44%|████▎ | 3280/7500 [1:50:46<2:28:18, 2.11s/it] 44%|████▎ | 3281/7500 [1:50:48<2:25:39, 2.07s/it] 44%|████▍ | 3282/7500 [1:50:50<2:25:17, 2.07s/it] 44%|████▍ | 3283/7500 [1:50:53<2:25:54, 2.08s/it] 44%|████▍ | 3284/7500 [1:50:55<2:23:59, 2.05s/it] 44%|████▍ | 3285/7500 [1:50:57<2:23:16, 2.04s/it] 44%|████▍ | 3286/7500 [1:50:59<2:22:07, 2.02s/it] 44%|████▍ | 3287/7500 [1:51:00<2:19:42, 1.99s/it] 44%|████▍ | 3288/7500 [1:51:02<2:20:22, 2.00s/it] 44%|████▍ | 3289/7500 [1:51:05<2:23:28, 2.04s/it] 44%|████▍ | 3290/7500 [1:51:07<2:21:56, 2.02s/it] {'loss': 0.4134, 'grad_norm': 3.2250077724456787, 'learning_rate': 3.447294575370103e-05, 'logqs/rejected': -2.6835389137268066, 'logqs/chosen': -0.874750018119812, 'logqs_logps/rejected': -0.02608180046081543, 'logps_logqs/chosen': 0.22266988456249237, 'epoch': 0.88} 44%|████▍ | 3290/7500 [1:51:07<2:21:56, 2.02s/it] 44%|████▍ | 3291/7500 [1:51:09<2:22:02, 2.02s/it] 44%|████▍ | 3292/7500 [1:51:11<2:21:39, 2.02s/it] 44%|████▍ | 3293/7500 [1:51:13<2:23:48, 2.05s/it] 44%|████▍ | 3294/7500 [1:51:15<2:25:04, 2.07s/it] 44%|████▍ | 3295/7500 [1:51:17<2:26:14, 2.09s/it] 44%|████▍ | 3296/7500 [1:51:19<2:25:41, 2.08s/it] 44%|████▍ | 3297/7500 [1:51:21<2:24:16, 2.06s/it] 44%|████▍ | 3298/7500 [1:51:23<2:22:45, 2.04s/it] 44%|████▍ | 3299/7500 [1:51:25<2:23:03, 2.04s/it] 44%|████▍ | 3300/7500 [1:51:27<2:21:03, 2.02s/it] {'loss': 0.4464, 'grad_norm': 3.5658011436462402, 'learning_rate': 3.436516483539781e-05, 'logqs/rejected': -2.7148921489715576, 'logqs/chosen': -0.8894504308700562, 'logqs_logps/rejected': -0.020480923354625702, 'logps_logqs/chosen': 0.22774776816368103, 'epoch': 0.88} 44%|████▍ | 3300/7500 [1:51:27<2:21:03, 2.02s/it] 44%|████▍ | 3301/7500 [1:51:29<2:19:39, 2.00s/it] 44%|████▍ | 3302/7500 [1:51:31<2:19:58, 2.00s/it] 44%|████▍ | 3303/7500 [1:51:33<2:23:08, 2.05s/it] 44%|████▍ | 3304/7500 [1:51:35<2:23:37, 2.05s/it] 44%|████▍ | 3305/7500 [1:51:37<2:22:07, 2.03s/it] 44%|████▍ | 3306/7500 [1:51:39<2:22:02, 2.03s/it] 44%|████▍ | 3307/7500 [1:51:41<2:23:17, 2.05s/it] 44%|████▍ | 3308/7500 [1:51:43<2:23:33, 2.05s/it] 44%|████▍ | 3309/7500 [1:51:45<2:23:13, 2.05s/it] 44%|████▍ | 3310/7500 [1:51:48<2:23:45, 2.06s/it] {'loss': 0.4169, 'grad_norm': 3.5241622924804688, 'learning_rate': 3.425718105222713e-05, 'logqs/rejected': -2.259570837020874, 'logqs/chosen': -0.8724038004875183, 'logqs_logps/rejected': -0.029804710298776627, 'logps_logqs/chosen': 0.22416460514068604, 'epoch': 0.88} 44%|████▍ | 3310/7500 [1:51:48<2:23:45, 2.06s/it] 44%|████▍ | 3311/7500 [1:51:50<2:23:44, 2.06s/it] 44%|████▍ | 3312/7500 [1:51:52<2:23:18, 2.05s/it] 44%|████▍ | 3313/7500 [1:51:54<2:23:08, 2.05s/it] 44%|████▍ | 3314/7500 [1:51:56<2:24:49, 2.08s/it] 44%|████▍ | 3315/7500 [1:51:58<2:23:49, 2.06s/it] 44%|████▍ | 3316/7500 [1:52:00<2:23:47, 2.06s/it] 44%|████▍ | 3317/7500 [1:52:02<2:22:02, 2.04s/it] 44%|████▍ | 3318/7500 [1:52:04<2:24:21, 2.07s/it] 44%|████▍ | 3319/7500 [1:52:06<2:22:40, 2.05s/it] 44%|████▍ | 3320/7500 [1:52:08<2:22:12, 2.04s/it] {'loss': 0.4331, 'grad_norm': 4.147331237792969, 'learning_rate': 3.41489967432953e-05, 'logqs/rejected': -2.2954249382019043, 'logqs/chosen': -0.937643826007843, 'logqs_logps/rejected': -0.03160037472844124, 'logps_logqs/chosen': 0.22775574028491974, 'epoch': 0.89} 44%|████▍ | 3320/7500 [1:52:08<2:22:12, 2.04s/it] 44%|████▍ | 3321/7500 [1:52:10<2:22:09, 2.04s/it] 44%|████▍ | 3322/7500 [1:52:12<2:21:51, 2.04s/it] 44%|████▍ | 3323/7500 [1:52:14<2:23:56, 2.07s/it] 44%|████▍ | 3324/7500 [1:52:16<2:21:51, 2.04s/it] 44%|████▍ | 3325/7500 [1:52:18<2:23:10, 2.06s/it] 44%|████▍ | 3326/7500 [1:52:20<2:21:16, 2.03s/it] 44%|████▍ | 3327/7500 [1:52:22<2:21:41, 2.04s/it] 44%|████▍ | 3328/7500 [1:52:24<2:22:07, 2.04s/it] 44%|████▍ | 3329/7500 [1:52:27<2:25:16, 2.09s/it] 44%|████▍ | 3330/7500 [1:52:29<2:24:40, 2.08s/it] {'loss': 0.4447, 'grad_norm': 3.5774307250976562, 'learning_rate': 3.4040614252052305e-05, 'logqs/rejected': -2.8093833923339844, 'logqs/chosen': -0.8997332453727722, 'logqs_logps/rejected': -0.01922680251300335, 'logps_logqs/chosen': 0.224041149020195, 'epoch': 0.89} 44%|████▍ | 3330/7500 [1:52:29<2:24:40, 2.08s/it] 44%|████▍ | 3331/7500 [1:52:31<2:23:02, 2.06s/it] 44%|████▍ | 3332/7500 [1:52:33<2:22:23, 2.05s/it] 44%|████▍ | 3333/7500 [1:52:35<2:22:37, 2.05s/it] 44%|████▍ | 3334/7500 [1:52:37<2:22:43, 2.06s/it] 44%|████▍ | 3335/7500 [1:52:39<2:22:32, 2.05s/it] 44%|████▍ | 3336/7500 [1:52:41<2:24:00, 2.08s/it] 44%|████▍ | 3337/7500 [1:52:43<2:22:18, 2.05s/it] 45%|████▍ | 3338/7500 [1:52:45<2:20:44, 2.03s/it] 45%|████▍ | 3339/7500 [1:52:47<2:21:43, 2.04s/it] 45%|████▍ | 3340/7500 [1:52:49<2:20:14, 2.02s/it] {'loss': 0.4152, 'grad_norm': 3.177783250808716, 'learning_rate': 3.39320359262411e-05, 'logqs/rejected': -2.5596017837524414, 'logqs/chosen': -0.8568077087402344, 'logqs_logps/rejected': -0.020897887647151947, 'logps_logqs/chosen': 0.2217048704624176, 'epoch': 0.89} 45%|████▍ | 3340/7500 [1:52:49<2:20:14, 2.02s/it] 45%|████▍ | 3341/7500 [1:52:51<2:20:20, 2.02s/it] 45%|████▍ | 3342/7500 [1:52:53<2:19:27, 2.01s/it] 45%|████▍ | 3343/7500 [1:52:55<2:19:36, 2.01s/it] 45%|████▍ | 3344/7500 [1:52:57<2:21:22, 2.04s/it] 45%|████▍ | 3345/7500 [1:52:59<2:22:49, 2.06s/it] 45%|████▍ | 3346/7500 [1:53:01<2:23:04, 2.07s/it] 45%|████▍ | 3347/7500 [1:53:03<2:23:47, 2.08s/it] 45%|████▍ | 3348/7500 [1:53:05<2:23:00, 2.07s/it] 45%|████▍ | 3349/7500 [1:53:08<2:22:36, 2.06s/it] 45%|████▍ | 3350/7500 [1:53:10<2:23:41, 2.08s/it] {'loss': 0.4216, 'grad_norm': 3.5332038402557373, 'learning_rate': 3.382326411784672e-05, 'logqs/rejected': -2.560192346572876, 'logqs/chosen': -0.878656268119812, 'logqs_logps/rejected': -0.023911921307444572, 'logps_logqs/chosen': 0.21609263122081757, 'epoch': 0.89} 45%|████▍ | 3350/7500 [1:53:10<2:23:41, 2.08s/it] 45%|████▍ | 3351/7500 [1:53:12<2:22:04, 2.05s/it] 45%|████▍ | 3352/7500 [1:53:14<2:22:22, 2.06s/it] 45%|████▍ | 3353/7500 [1:53:16<2:20:29, 2.03s/it] 45%|████▍ | 3354/7500 [1:53:18<2:20:53, 2.04s/it] 45%|████▍ | 3355/7500 [1:53:20<2:22:42, 2.07s/it] 45%|████▍ | 3356/7500 [1:53:22<2:22:56, 2.07s/it] 45%|████▍ | 3357/7500 [1:53:24<2:22:51, 2.07s/it] 45%|████▍ | 3358/7500 [1:53:26<2:21:58, 2.06s/it] 45%|████▍ | 3359/7500 [1:53:28<2:21:25, 2.05s/it] 45%|████▍ | 3360/7500 [1:53:30<2:20:40, 2.04s/it] {'loss': 0.4372, 'grad_norm': 3.3470516204833984, 'learning_rate': 3.3714301183045385e-05, 'logqs/rejected': -2.8323898315429688, 'logqs/chosen': -0.8972234725952148, 'logqs_logps/rejected': -0.018382394686341286, 'logps_logqs/chosen': 0.2228221446275711, 'epoch': 0.9} 45%|████▍ | 3360/7500 [1:53:30<2:20:40, 2.04s/it] 45%|████▍ | 3361/7500 [1:53:32<2:19:19, 2.02s/it] 45%|████▍ | 3362/7500 [1:53:34<2:19:36, 2.02s/it] 45%|████▍ | 3363/7500 [1:53:36<2:19:32, 2.02s/it] 45%|████▍ | 3364/7500 [1:53:38<2:20:16, 2.03s/it] 45%|████▍ | 3365/7500 [1:53:40<2:21:46, 2.06s/it] 45%|████▍ | 3366/7500 [1:53:42<2:22:27, 2.07s/it] 45%|████▍ | 3367/7500 [1:53:44<2:20:22, 2.04s/it] 45%|████▍ | 3368/7500 [1:53:46<2:18:45, 2.01s/it] 45%|████▍ | 3369/7500 [1:53:48<2:20:36, 2.04s/it] 45%|████▍ | 3370/7500 [1:53:51<2:22:24, 2.07s/it] {'loss': 0.4107, 'grad_norm': 3.8933074474334717, 'learning_rate': 3.360514948215339e-05, 'logqs/rejected': -2.5291202068328857, 'logqs/chosen': -0.8829663991928101, 'logqs_logps/rejected': -0.032430365681648254, 'logps_logqs/chosen': 0.218277245759964, 'epoch': 0.9} 45%|████▍ | 3370/7500 [1:53:51<2:22:24, 2.07s/it] 45%|████▍ | 3371/7500 [1:53:53<2:22:59, 2.08s/it] 45%|████▍ | 3372/7500 [1:53:55<2:21:19, 2.05s/it] 45%|████▍ | 3373/7500 [1:53:57<2:22:23, 2.07s/it] 45%|████▍ | 3374/7500 [1:53:59<2:20:44, 2.05s/it] 45%|████▌ | 3375/7500 [1:54:01<2:18:01, 2.01s/it] 45%|████▌ | 3376/7500 [1:54:03<2:19:39, 2.03s/it] 45%|████▌ | 3377/7500 [1:54:05<2:18:58, 2.02s/it] 45%|████▌ | 3378/7500 [1:54:07<2:18:28, 2.02s/it] 45%|████▌ | 3379/7500 [1:54:09<2:17:52, 2.01s/it] 45%|████▌ | 3380/7500 [1:54:11<2:17:04, 2.00s/it] {'loss': 0.431, 'grad_norm': 3.7159595489501953, 'learning_rate': 3.349581137957604e-05, 'logqs/rejected': -2.630235195159912, 'logqs/chosen': -0.8931095004081726, 'logqs_logps/rejected': -0.02211085893213749, 'logps_logqs/chosen': 0.2223953902721405, 'epoch': 0.9} 45%|████▌ | 3380/7500 [1:54:11<2:17:04, 2.00s/it] 45%|████▌ | 3381/7500 [1:54:13<2:18:38, 2.02s/it] 45%|████▌ | 3382/7500 [1:54:15<2:19:28, 2.03s/it] 45%|████▌ | 3383/7500 [1:54:17<2:21:01, 2.06s/it] 45%|████▌ | 3384/7500 [1:54:19<2:20:36, 2.05s/it] 45%|████▌ | 3385/7500 [1:54:21<2:20:08, 2.04s/it] 45%|████▌ | 3386/7500 [1:54:23<2:19:41, 2.04s/it] 45%|████▌ | 3387/7500 [1:54:25<2:20:08, 2.04s/it] 45%|████▌ | 3388/7500 [1:54:27<2:19:53, 2.04s/it] 45%|████▌ | 3389/7500 [1:54:29<2:19:44, 2.04s/it] 45%|████▌ | 3390/7500 [1:54:31<2:20:13, 2.05s/it] {'loss': 0.413, 'grad_norm': 3.5519957542419434, 'learning_rate': 3.338628924375638e-05, 'logqs/rejected': -2.512956380844116, 'logqs/chosen': -0.8813234567642212, 'logqs_logps/rejected': -0.023714840412139893, 'logps_logqs/chosen': 0.21945448219776154, 'epoch': 0.9} 45%|████▌ | 3390/7500 [1:54:31<2:20:13, 2.05s/it] 45%|████▌ | 3391/7500 [1:54:33<2:20:33, 2.05s/it] 45%|████▌ | 3392/7500 [1:54:35<2:21:27, 2.07s/it] 45%|████▌ | 3393/7500 [1:54:37<2:18:14, 2.02s/it] 45%|████▌ | 3394/7500 [1:54:39<2:19:18, 2.04s/it] 45%|████▌ | 3395/7500 [1:54:41<2:20:38, 2.06s/it] 45%|████▌ | 3396/7500 [1:54:44<2:20:59, 2.06s/it] 45%|████▌ | 3397/7500 [1:54:46<2:20:22, 2.05s/it] 45%|████▌ | 3398/7500 [1:54:48<2:19:47, 2.04s/it] 45%|████▌ | 3399/7500 [1:54:50<2:21:27, 2.07s/it] 45%|████▌ | 3400/7500 [1:54:52<2:23:33, 2.10s/it] {'loss': 0.4276, 'grad_norm': 3.6265549659729004, 'learning_rate': 3.327658544712395e-05, 'logqs/rejected': -2.5679309368133545, 'logqs/chosen': -0.8797392845153809, 'logqs_logps/rejected': -0.015139536932110786, 'logps_logqs/chosen': 0.22066664695739746, 'epoch': 0.91} 45%|████▌ | 3400/7500 [1:54:52<2:23:33, 2.10s/it] 45%|████▌ | 3401/7500 [1:54:54<2:23:19, 2.10s/it] 45%|████▌ | 3402/7500 [1:54:56<2:22:26, 2.09s/it] 45%|████▌ | 3403/7500 [1:54:58<2:20:38, 2.06s/it] 45%|████▌ | 3404/7500 [1:55:00<2:20:20, 2.06s/it] 45%|████▌ | 3405/7500 [1:55:02<2:20:54, 2.06s/it] 45%|████▌ | 3406/7500 [1:55:04<2:19:41, 2.05s/it] 45%|████▌ | 3407/7500 [1:55:06<2:18:54, 2.04s/it] 45%|████▌ | 3408/7500 [1:55:08<2:18:11, 2.03s/it] 45%|████▌ | 3409/7500 [1:55:10<2:18:21, 2.03s/it] 45%|████▌ | 3410/7500 [1:55:12<2:20:34, 2.06s/it] {'loss': 0.4357, 'grad_norm': 4.186779975891113, 'learning_rate': 3.3166702366043365e-05, 'logqs/rejected': -2.5315470695495605, 'logqs/chosen': -0.8821337819099426, 'logqs_logps/rejected': -0.019968751817941666, 'logps_logqs/chosen': 0.2317989319562912, 'epoch': 0.91} 45%|████▌ | 3410/7500 [1:55:12<2:20:34, 2.06s/it] 45%|████▌ | 3411/7500 [1:55:14<2:20:28, 2.06s/it] 45%|████▌ | 3412/7500 [1:55:17<2:22:26, 2.09s/it] 46%|████▌ | 3413/7500 [1:55:19<2:23:03, 2.10s/it] 46%|████▌ | 3414/7500 [1:55:21<2:23:42, 2.11s/it] 46%|████▌ | 3415/7500 [1:55:23<2:24:16, 2.12s/it] 46%|████▌ | 3416/7500 [1:55:25<2:23:26, 2.11s/it] 46%|████▌ | 3417/7500 [1:55:27<2:24:02, 2.12s/it] 46%|████▌ | 3418/7500 [1:55:29<2:21:31, 2.08s/it] 46%|████▌ | 3419/7500 [1:55:31<2:20:58, 2.07s/it] 46%|████▌ | 3420/7500 [1:55:33<2:18:51, 2.04s/it] {'loss': 0.4238, 'grad_norm': 3.548952102661133, 'learning_rate': 3.305664238076278e-05, 'logqs/rejected': -2.5717594623565674, 'logqs/chosen': -0.9065338373184204, 'logqs_logps/rejected': -0.025950079783797264, 'logps_logqs/chosen': 0.2269265204668045, 'epoch': 0.91} 46%|████▌ | 3420/7500 [1:55:33<2:18:51, 2.04s/it] 46%|████▌ | 3421/7500 [1:55:35<2:20:20, 2.06s/it] 46%|████▌ | 3422/7500 [1:55:37<2:18:24, 2.04s/it] 46%|████▌ | 3423/7500 [1:55:39<2:17:41, 2.03s/it] 46%|████▌ | 3424/7500 [1:55:41<2:17:17, 2.02s/it] 46%|████▌ | 3425/7500 [1:55:43<2:17:55, 2.03s/it] 46%|████▌ | 3426/7500 [1:55:45<2:16:34, 2.01s/it] 46%|████▌ | 3427/7500 [1:55:47<2:16:51, 2.02s/it] 46%|████▌ | 3428/7500 [1:55:50<2:19:14, 2.05s/it] 46%|████▌ | 3429/7500 [1:55:52<2:20:04, 2.06s/it] 46%|████▌ | 3430/7500 [1:55:54<2:20:37, 2.07s/it] {'loss': 0.4286, 'grad_norm': 5.501941204071045, 'learning_rate': 3.294640787536245e-05, 'logqs/rejected': -2.4402661323547363, 'logqs/chosen': -0.9162972569465637, 'logqs_logps/rejected': -0.02936469577252865, 'logps_logqs/chosen': 0.22750887274742126, 'epoch': 0.91} 46%|████▌ | 3430/7500 [1:55:54<2:20:37, 2.07s/it] 46%|████▌ | 3431/7500 [1:55:56<2:20:34, 2.07s/it] 46%|████▌ | 3432/7500 [1:55:58<2:20:13, 2.07s/it] 46%|████▌ | 3433/7500 [1:56:00<2:19:41, 2.06s/it] 46%|████▌ | 3434/7500 [1:56:02<2:18:27, 2.04s/it] 46%|████▌ | 3435/7500 [1:56:04<2:17:07, 2.02s/it] 46%|████▌ | 3436/7500 [1:56:06<2:19:52, 2.07s/it] 46%|████▌ | 3437/7500 [1:56:08<2:18:03, 2.04s/it] 46%|████▌ | 3438/7500 [1:56:10<2:18:56, 2.05s/it] 46%|████▌ | 3439/7500 [1:56:12<2:19:38, 2.06s/it] 46%|████▌ | 3440/7500 [1:56:14<2:21:24, 2.09s/it] {'loss': 0.4441, 'grad_norm': 3.561145305633545, 'learning_rate': 3.283600123770299e-05, 'logqs/rejected': -2.594874143600464, 'logqs/chosen': -0.8761631846427917, 'logqs_logps/rejected': -0.024466179311275482, 'logps_logqs/chosen': 0.22627386450767517, 'epoch': 0.92} 46%|████▌ | 3440/7500 [1:56:14<2:21:24, 2.09s/it] 46%|████▌ | 3441/7500 [1:56:16<2:20:52, 2.08s/it] 46%|████▌ | 3442/7500 [1:56:18<2:18:09, 2.04s/it] 46%|████▌ | 3443/7500 [1:56:20<2:16:28, 2.02s/it] 46%|████▌ | 3444/7500 [1:56:22<2:17:24, 2.03s/it] 46%|████▌ | 3445/7500 [1:56:25<2:19:35, 2.07s/it] 46%|████▌ | 3446/7500 [1:56:27<2:20:22, 2.08s/it] 46%|████▌ | 3447/7500 [1:56:29<2:20:48, 2.08s/it] 46%|████▌ | 3448/7500 [1:56:31<2:21:59, 2.10s/it] 46%|████▌ | 3449/7500 [1:56:33<2:22:12, 2.11s/it] 46%|████▌ | 3450/7500 [1:56:35<2:23:18, 2.12s/it] {'loss': 0.4413, 'grad_norm': 3.9934945106506348, 'learning_rate': 3.272542485937369e-05, 'logqs/rejected': -2.6063427925109863, 'logqs/chosen': -0.9195873141288757, 'logqs_logps/rejected': -0.02616659365594387, 'logps_logqs/chosen': 0.22766348719596863, 'epoch': 0.92} 46%|████▌ | 3450/7500 [1:56:35<2:23:18, 2.12s/it] 46%|████▌ | 3451/7500 [1:56:37<2:22:41, 2.11s/it] 46%|████▌ | 3452/7500 [1:56:39<2:20:43, 2.09s/it] 46%|████▌ | 3453/7500 [1:56:41<2:21:27, 2.10s/it] 46%|████▌ | 3454/7500 [1:56:43<2:18:57, 2.06s/it] 46%|████▌ | 3455/7500 [1:56:45<2:16:23, 2.02s/it] 46%|████▌ | 3456/7500 [1:56:47<2:17:49, 2.04s/it] 46%|████▌ | 3457/7500 [1:56:49<2:18:33, 2.06s/it] 46%|████▌ | 3458/7500 [1:56:52<2:18:42, 2.06s/it] 46%|████▌ | 3459/7500 [1:56:54<2:18:21, 2.05s/it] 46%|████▌ | 3460/7500 [1:56:56<2:16:54, 2.03s/it] {'loss': 0.4348, 'grad_norm': 3.5600390434265137, 'learning_rate': 3.26146811356407e-05, 'logqs/rejected': -2.547572374343872, 'logqs/chosen': -0.8829684257507324, 'logqs_logps/rejected': -0.017770571634173393, 'logps_logqs/chosen': 0.22550395131111145, 'epoch': 0.92} 46%|████▌ | 3460/7500 [1:56:56<2:16:54, 2.03s/it] 46%|████▌ | 3461/7500 [1:56:58<2:17:44, 2.05s/it] 46%|████▌ | 3462/7500 [1:57:00<2:16:06, 2.02s/it] 46%|████▌ | 3463/7500 [1:57:02<2:16:01, 2.02s/it] 46%|████▌ | 3464/7500 [1:57:04<2:19:03, 2.07s/it] 46%|████▌ | 3465/7500 [1:57:06<2:21:16, 2.10s/it] 46%|████▌ | 3466/7500 [1:57:08<2:19:05, 2.07s/it] 46%|████▌ | 3467/7500 [1:57:10<2:18:30, 2.06s/it] 46%|████▌ | 3468/7500 [1:57:12<2:19:20, 2.07s/it] 46%|████▋ | 3469/7500 [1:57:14<2:20:01, 2.08s/it] 46%|████▋ | 3470/7500 [1:57:16<2:18:56, 2.07s/it] {'loss': 0.4168, 'grad_norm': 3.4643774032592773, 'learning_rate': 3.250377246539515e-05, 'logqs/rejected': -2.461912155151367, 'logqs/chosen': -0.9033958315849304, 'logqs_logps/rejected': -0.02282852865755558, 'logps_logqs/chosen': 0.2297479212284088, 'epoch': 0.93} 46%|████▋ | 3470/7500 [1:57:16<2:18:56, 2.07s/it] 46%|████▋ | 3471/7500 [1:57:18<2:17:25, 2.05s/it] 46%|████▋ | 3472/7500 [1:57:20<2:16:12, 2.03s/it] 46%|████▋ | 3473/7500 [1:57:22<2:16:27, 2.03s/it] 46%|████▋ | 3474/7500 [1:57:24<2:17:52, 2.05s/it] 46%|████▋ | 3475/7500 [1:57:26<2:17:45, 2.05s/it] 46%|████▋ | 3476/7500 [1:57:29<2:18:57, 2.07s/it] 46%|████▋ | 3477/7500 [1:57:31<2:18:39, 2.07s/it] 46%|████▋ | 3478/7500 [1:57:33<2:19:27, 2.08s/it] 46%|████▋ | 3479/7500 [1:57:35<2:18:59, 2.07s/it] 46%|████▋ | 3480/7500 [1:57:37<2:16:51, 2.04s/it] {'loss': 0.4327, 'grad_norm': 3.365015983581543, 'learning_rate': 3.239270125110117e-05, 'logqs/rejected': -2.703099250793457, 'logqs/chosen': -0.8812240362167358, 'logqs_logps/rejected': -0.024948596954345703, 'logps_logqs/chosen': 0.22374741733074188, 'epoch': 0.93} 46%|████▋ | 3480/7500 [1:57:37<2:16:51, 2.04s/it] 46%|████▋ | 3481/7500 [1:57:39<2:15:21, 2.02s/it] 46%|████▋ | 3482/7500 [1:57:41<2:16:08, 2.03s/it] 46%|████▋ | 3483/7500 [1:57:43<2:17:18, 2.05s/it] 46%|████▋ | 3484/7500 [1:57:45<2:19:46, 2.09s/it] 46%|████▋ | 3485/7500 [1:57:47<2:17:52, 2.06s/it] 46%|████▋ | 3486/7500 [1:57:49<2:18:26, 2.07s/it] 46%|████▋ | 3487/7500 [1:57:51<2:18:52, 2.08s/it] 47%|████▋ | 3488/7500 [1:57:53<2:19:37, 2.09s/it] 47%|████▋ | 3489/7500 [1:57:55<2:18:32, 2.07s/it] 47%|████▋ | 3490/7500 [1:57:57<2:17:10, 2.05s/it] {'loss': 0.4209, 'grad_norm': 3.9495158195495605, 'learning_rate': 3.228146989874389e-05, 'logqs/rejected': -2.5191497802734375, 'logqs/chosen': -0.8771330118179321, 'logqs_logps/rejected': -0.01908065564930439, 'logps_logqs/chosen': 0.22150151431560516, 'epoch': 0.93} 47%|████▋ | 3490/7500 [1:57:57<2:17:10, 2.05s/it] 47%|████▋ | 3491/7500 [1:57:59<2:17:34, 2.06s/it] 47%|████▋ | 3492/7500 [1:58:02<2:18:54, 2.08s/it] 47%|████▋ | 3493/7500 [1:58:04<2:19:37, 2.09s/it] 47%|████▋ | 3494/7500 [1:58:06<2:17:11, 2.05s/it] 47%|████▋ | 3495/7500 [1:58:08<2:15:27, 2.03s/it] 47%|████▋ | 3496/7500 [1:58:10<2:17:16, 2.06s/it] 47%|████▋ | 3497/7500 [1:58:12<2:15:19, 2.03s/it] 47%|████▋ | 3498/7500 [1:58:14<2:15:17, 2.03s/it] 47%|████▋ | 3499/7500 [1:58:16<2:15:12, 2.03s/it] 47%|████▋ | 3500/7500 [1:58:18<2:15:30, 2.03s/it] {'loss': 0.4206, 'grad_norm': 3.905801773071289, 'learning_rate': 3.217008081777726e-05, 'logqs/rejected': -2.4441826343536377, 'logqs/chosen': -0.8657708168029785, 'logqs_logps/rejected': -0.025864705443382263, 'logps_logqs/chosen': 0.22913160920143127, 'epoch': 0.93} 47%|████▋ | 3500/7500 [1:58:18<2:15:30, 2.03s/it] 47%|████▋ | 3501/7500 [1:58:20<2:18:12, 2.07s/it] 47%|████▋ | 3502/7500 [1:58:22<2:17:52, 2.07s/it] 47%|████▋ | 3503/7500 [1:58:24<2:18:10, 2.07s/it] 47%|████▋ | 3504/7500 [1:58:26<2:20:34, 2.11s/it] 47%|████▋ | 3505/7500 [1:58:28<2:18:23, 2.08s/it] 47%|████▋ | 3506/7500 [1:58:30<2:18:37, 2.08s/it] 47%|████▋ | 3507/7500 [1:58:32<2:17:23, 2.06s/it] 47%|████▋ | 3508/7500 [1:58:35<2:16:45, 2.06s/it] 47%|████▋ | 3509/7500 [1:58:37<2:17:23, 2.07s/it] 47%|████▋ | 3510/7500 [1:58:39<2:17:48, 2.07s/it] {'loss': 0.3898, 'grad_norm': 3.940756320953369, 'learning_rate': 3.205853642107192e-05, 'logqs/rejected': -2.292236328125, 'logqs/chosen': -0.8908727765083313, 'logqs_logps/rejected': -0.03239910304546356, 'logps_logqs/chosen': 0.2202000916004181, 'epoch': 0.94} 47%|████▋ | 3510/7500 [1:58:39<2:17:48, 2.07s/it] 47%|████▋ | 3511/7500 [1:58:41<2:17:00, 2.06s/it] 47%|████▋ | 3512/7500 [1:58:43<2:15:06, 2.03s/it] 47%|████▋ | 3513/7500 [1:58:45<2:13:21, 2.01s/it] 47%|████▋ | 3514/7500 [1:58:47<2:14:32, 2.03s/it] 47%|████▋ | 3515/7500 [1:58:49<2:16:23, 2.05s/it] 47%|████▋ | 3516/7500 [1:58:51<2:14:33, 2.03s/it] 47%|████▋ | 3517/7500 [1:58:53<2:14:53, 2.03s/it] 47%|████▋ | 3518/7500 [1:58:55<2:14:26, 2.03s/it] 47%|████▋ | 3519/7500 [1:58:57<2:13:48, 2.02s/it] 47%|████▋ | 3520/7500 [1:58:59<2:13:53, 2.02s/it] {'loss': 0.4114, 'grad_norm': 4.276479721069336, 'learning_rate': 3.194683912486287e-05, 'logqs/rejected': -2.6802289485931396, 'logqs/chosen': -0.880527138710022, 'logqs_logps/rejected': -0.0339897982776165, 'logps_logqs/chosen': 0.22338518500328064, 'epoch': 0.94} 47%|████▋ | 3520/7500 [1:58:59<2:13:53, 2.02s/it] 47%|████▋ | 3521/7500 [1:59:01<2:12:47, 2.00s/it] 47%|████▋ | 3522/7500 [1:59:03<2:13:41, 2.02s/it] 47%|████▋ | 3523/7500 [1:59:05<2:15:04, 2.04s/it] 47%|████▋ | 3524/7500 [1:59:07<2:16:32, 2.06s/it] 47%|████▋ | 3525/7500 [1:59:09<2:17:28, 2.08s/it] 47%|████▋ | 3526/7500 [1:59:11<2:18:39, 2.09s/it] 47%|████▋ | 3527/7500 [1:59:13<2:15:51, 2.05s/it] 47%|████▋ | 3528/7500 [1:59:15<2:14:27, 2.03s/it] 47%|████▋ | 3529/7500 [1:59:17<2:14:50, 2.04s/it] 47%|████▋ | 3530/7500 [1:59:19<2:14:24, 2.03s/it] {'loss': 0.4323, 'grad_norm': 3.7861716747283936, 'learning_rate': 3.1834991348697204e-05, 'logqs/rejected': -2.654794216156006, 'logqs/chosen': -0.8840656280517578, 'logqs_logps/rejected': -0.016306722536683083, 'logps_logqs/chosen': 0.21939215064048767, 'epoch': 0.94} 47%|████▋ | 3530/7500 [1:59:19<2:14:24, 2.03s/it] 47%|████▋ | 3531/7500 [1:59:21<2:13:43, 2.02s/it] 47%|████▋ | 3532/7500 [1:59:23<2:14:00, 2.03s/it] 47%|████▋ | 3533/7500 [1:59:25<2:14:48, 2.04s/it] 47%|████▋ | 3534/7500 [1:59:28<2:16:13, 2.06s/it] 47%|████▋ | 3535/7500 [1:59:30<2:15:06, 2.04s/it] 47%|████▋ | 3536/7500 [1:59:32<2:15:39, 2.05s/it] 47%|████▋ | 3537/7500 [1:59:34<2:17:32, 2.08s/it] 47%|████▋ | 3538/7500 [1:59:36<2:16:52, 2.07s/it] 47%|████▋ | 3539/7500 [1:59:38<2:16:37, 2.07s/it] 47%|████▋ | 3540/7500 [1:59:40<2:18:32, 2.10s/it] {'loss': 0.4335, 'grad_norm': 3.4301376342773438, 'learning_rate': 3.172299551538164e-05, 'logqs/rejected': -2.588444948196411, 'logqs/chosen': -0.8804572820663452, 'logqs_logps/rejected': -0.018371878191828728, 'logps_logqs/chosen': 0.22361591458320618, 'epoch': 0.94} 47%|████▋ | 3540/7500 [1:59:40<2:18:32, 2.10s/it] 47%|████▋ | 3541/7500 [1:59:42<2:17:37, 2.09s/it] 47%|████▋ | 3542/7500 [1:59:44<2:16:07, 2.06s/it] 47%|████▋ | 3543/7500 [1:59:46<2:17:03, 2.08s/it] 47%|████▋ | 3544/7500 [1:59:48<2:17:51, 2.09s/it] 47%|████▋ | 3545/7500 [1:59:50<2:15:57, 2.06s/it] 47%|████▋ | 3546/7500 [1:59:52<2:16:07, 2.07s/it] 47%|████▋ | 3547/7500 [1:59:54<2:15:54, 2.06s/it] 47%|████▋ | 3548/7500 [1:59:57<2:15:56, 2.06s/it] 47%|████▋ | 3549/7500 [1:59:59<2:18:14, 2.10s/it] 47%|████▋ | 3550/7500 [2:00:01<2:15:59, 2.07s/it] {'loss': 0.4079, 'grad_norm': 3.6965513229370117, 'learning_rate': 3.161085405093006e-05, 'logqs/rejected': -2.400285482406616, 'logqs/chosen': -0.8871909976005554, 'logqs_logps/rejected': -0.022856757044792175, 'logps_logqs/chosen': 0.22336752712726593, 'epoch': 0.95} 47%|████▋ | 3550/7500 [2:00:01<2:15:59, 2.07s/it] 47%|████▋ | 3551/7500 [2:00:03<2:15:23, 2.06s/it] 47%|████▋ | 3552/7500 [2:00:05<2:13:24, 2.03s/it] 47%|████▋ | 3553/7500 [2:00:07<2:13:47, 2.03s/it] 47%|████▋ | 3554/7500 [2:00:09<2:13:39, 2.03s/it] 47%|████▋ | 3555/7500 [2:00:11<2:14:27, 2.04s/it] 47%|████▋ | 3556/7500 [2:00:13<2:15:33, 2.06s/it] 47%|████▋ | 3557/7500 [2:00:15<2:17:44, 2.10s/it] 47%|████▋ | 3558/7500 [2:00:17<2:17:47, 2.10s/it] 47%|████▋ | 3559/7500 [2:00:19<2:16:43, 2.08s/it] 47%|████▋ | 3560/7500 [2:00:21<2:15:04, 2.06s/it] {'loss': 0.4197, 'grad_norm': 3.761596202850342, 'learning_rate': 3.149856938451094e-05, 'logqs/rejected': -2.4458231925964355, 'logqs/chosen': -0.898980438709259, 'logqs_logps/rejected': -0.02761838771402836, 'logps_logqs/chosen': 0.22297139465808868, 'epoch': 0.95} 47%|████▋ | 3560/7500 [2:00:21<2:15:04, 2.06s/it] 47%|████▋ | 3561/7500 [2:00:23<2:15:03, 2.06s/it] 47%|████▋ | 3562/7500 [2:00:25<2:16:58, 2.09s/it] 48%|████▊ | 3563/7500 [2:00:27<2:14:28, 2.05s/it] 48%|████▊ | 3564/7500 [2:00:30<2:14:32, 2.05s/it] 48%|████▊ | 3565/7500 [2:00:32<2:15:15, 2.06s/it] 48%|████▊ | 3566/7500 [2:00:34<2:14:59, 2.06s/it] 48%|████▊ | 3567/7500 [2:00:36<2:14:20, 2.05s/it] 48%|████▊ | 3568/7500 [2:00:38<2:15:43, 2.07s/it] 48%|████▊ | 3569/7500 [2:00:40<2:15:27, 2.07s/it] 48%|████▊ | 3570/7500 [2:00:42<2:16:39, 2.09s/it] {'loss': 0.433, 'grad_norm': 4.219820499420166, 'learning_rate': 3.138614394839476e-05, 'logqs/rejected': -2.53969144821167, 'logqs/chosen': -0.867659866809845, 'logqs_logps/rejected': -0.019513795152306557, 'logps_logqs/chosen': 0.22125104069709778, 'epoch': 0.95} 48%|████▊ | 3570/7500 [2:00:42<2:16:39, 2.09s/it] 48%|████▊ | 3571/7500 [2:00:44<2:15:17, 2.07s/it] 48%|████▊ | 3572/7500 [2:00:46<2:14:01, 2.05s/it] 48%|████▊ | 3573/7500 [2:00:48<2:13:00, 2.03s/it] 48%|████▊ | 3574/7500 [2:00:50<2:13:45, 2.04s/it] 48%|████▊ | 3575/7500 [2:00:52<2:14:38, 2.06s/it] 48%|████▊ | 3576/7500 [2:00:54<2:15:24, 2.07s/it] 48%|████▊ | 3577/7500 [2:00:56<2:14:54, 2.06s/it] 48%|████▊ | 3578/7500 [2:00:58<2:13:35, 2.04s/it] 48%|████▊ | 3579/7500 [2:01:00<2:12:06, 2.02s/it] 48%|████▊ | 3580/7500 [2:01:02<2:11:47, 2.02s/it] {'loss': 0.414, 'grad_norm': 3.4089102745056152, 'learning_rate': 3.127358017790132e-05, 'logqs/rejected': -2.5043511390686035, 'logqs/chosen': -0.8684635162353516, 'logqs_logps/rejected': -0.02522372640669346, 'logps_logqs/chosen': 0.21823176741600037, 'epoch': 0.95} 48%|████▊ | 3580/7500 [2:01:02<2:11:47, 2.02s/it] 48%|████▊ | 3581/7500 [2:01:04<2:12:00, 2.02s/it] 48%|████▊ | 3582/7500 [2:01:06<2:12:08, 2.02s/it] 48%|████▊ | 3583/7500 [2:01:09<2:14:41, 2.06s/it] 48%|████▊ | 3584/7500 [2:01:11<2:13:39, 2.05s/it] 48%|████▊ | 3585/7500 [2:01:13<2:14:57, 2.07s/it] 48%|████▊ | 3586/7500 [2:01:15<2:16:16, 2.09s/it] 48%|████▊ | 3587/7500 [2:01:17<2:15:45, 2.08s/it] 48%|████▊ | 3588/7500 [2:01:19<2:16:12, 2.09s/it] 48%|████▊ | 3589/7500 [2:01:21<2:16:34, 2.10s/it] 48%|████▊ | 3590/7500 [2:01:23<2:15:09, 2.07s/it] {'loss': 0.4338, 'grad_norm': 4.3271613121032715, 'learning_rate': 3.116088051134695e-05, 'logqs/rejected': -2.4979615211486816, 'logqs/chosen': -0.9657538533210754, 'logqs_logps/rejected': -0.02601180598139763, 'logps_logqs/chosen': 0.23324653506278992, 'epoch': 0.96} 48%|████▊ | 3590/7500 [2:01:23<2:15:09, 2.07s/it] 48%|████▊ | 3591/7500 [2:01:25<2:12:54, 2.04s/it] 48%|████▊ | 3592/7500 [2:01:27<2:12:14, 2.03s/it] 48%|████▊ | 3593/7500 [2:01:29<2:10:42, 2.01s/it] 48%|████▊ | 3594/7500 [2:01:31<2:10:44, 2.01s/it] 48%|████▊ | 3595/7500 [2:01:33<2:12:49, 2.04s/it] 48%|████▊ | 3596/7500 [2:01:35<2:12:40, 2.04s/it] 48%|████▊ | 3597/7500 [2:01:37<2:10:49, 2.01s/it] 48%|████▊ | 3598/7500 [2:01:39<2:11:45, 2.03s/it] 48%|████▊ | 3599/7500 [2:01:41<2:12:06, 2.03s/it] 48%|████▊ | 3600/7500 [2:01:43<2:13:07, 2.05s/it] {'loss': 0.4213, 'grad_norm': 3.907493829727173, 'learning_rate': 3.104804738999169e-05, 'logqs/rejected': -2.5985963344573975, 'logqs/chosen': -0.8793473243713379, 'logqs_logps/rejected': -0.02678755857050419, 'logps_logqs/chosen': 0.22581148147583008, 'epoch': 0.96} 48%|████▊ | 3600/7500 [2:01:43<2:13:07, 2.05s/it] 48%|████▊ | 3601/7500 [2:01:45<2:11:41, 2.03s/it] 48%|████▊ | 3602/7500 [2:01:47<2:10:49, 2.01s/it] 48%|████▊ | 3603/7500 [2:01:49<2:12:42, 2.04s/it] 48%|████▊ | 3604/7500 [2:01:52<2:15:11, 2.08s/it] 48%|████▊ | 3605/7500 [2:01:54<2:15:23, 2.09s/it] 48%|████▊ | 3606/7500 [2:01:56<2:12:48, 2.05s/it] 48%|████▊ | 3607/7500 [2:01:58<2:13:46, 2.06s/it] 48%|████▊ | 3608/7500 [2:02:00<2:12:38, 2.04s/it] 48%|████▊ | 3609/7500 [2:02:02<2:11:27, 2.03s/it] 48%|████▊ | 3610/7500 [2:02:04<2:11:16, 2.02s/it] {'loss': 0.4176, 'grad_norm': 4.142851829528809, 'learning_rate': 3.093508325798649e-05, 'logqs/rejected': -2.5330288410186768, 'logqs/chosen': -0.8835688829421997, 'logqs_logps/rejected': -0.020455973222851753, 'logps_logqs/chosen': 0.22018925845623016, 'epoch': 0.96} 48%|████▊ | 3610/7500 [2:02:04<2:11:16, 2.02s/it] 48%|████▊ | 3611/7500 [2:02:06<2:13:26, 2.06s/it] 48%|████▊ | 3612/7500 [2:02:08<2:12:42, 2.05s/it] 48%|████▊ | 3613/7500 [2:02:10<2:12:30, 2.05s/it] 48%|████▊ | 3614/7500 [2:02:12<2:11:35, 2.03s/it] 48%|████▊ | 3615/7500 [2:02:14<2:14:07, 2.07s/it] 48%|████▊ | 3616/7500 [2:02:16<2:13:21, 2.06s/it] 48%|████▊ | 3617/7500 [2:02:18<2:11:33, 2.03s/it] 48%|████▊ | 3618/7500 [2:02:20<2:12:33, 2.05s/it] 48%|████▊ | 3619/7500 [2:02:22<2:12:43, 2.05s/it] 48%|████▊ | 3620/7500 [2:02:24<2:13:52, 2.07s/it] {'loss': 0.4227, 'grad_norm': 3.949652910232544, 'learning_rate': 3.0821990562320146e-05, 'logqs/rejected': -2.432400941848755, 'logqs/chosen': -0.8874961137771606, 'logqs_logps/rejected': -0.020329786464571953, 'logps_logqs/chosen': 0.2233799695968628, 'epoch': 0.97} 48%|████▊ | 3620/7500 [2:02:24<2:13:52, 2.07s/it] 48%|████▊ | 3621/7500 [2:02:26<2:13:10, 2.06s/it] 48%|████▊ | 3622/7500 [2:02:28<2:11:10, 2.03s/it] 48%|████▊ | 3623/7500 [2:02:30<2:13:32, 2.07s/it] 48%|████▊ | 3624/7500 [2:02:32<2:11:52, 2.04s/it] 48%|████▊ | 3625/7500 [2:02:34<2:11:42, 2.04s/it] 48%|████▊ | 3626/7500 [2:02:37<2:12:29, 2.05s/it] 48%|████▊ | 3627/7500 [2:02:39<2:13:15, 2.06s/it] 48%|████▊ | 3628/7500 [2:02:41<2:12:16, 2.05s/it] 48%|████▊ | 3629/7500 [2:02:43<2:12:31, 2.05s/it] 48%|████▊ | 3630/7500 [2:02:45<2:12:28, 2.05s/it] {'loss': 0.4172, 'grad_norm': 7.067124843597412, 'learning_rate': 3.0708771752766394e-05, 'logqs/rejected': -2.4121389389038086, 'logqs/chosen': -0.8587659597396851, 'logqs_logps/rejected': -0.02387268655002117, 'logps_logqs/chosen': 0.22029569745063782, 'epoch': 0.97} 48%|████▊ | 3630/7500 [2:02:45<2:12:28, 2.05s/it] 48%|████▊ | 3631/7500 [2:02:47<2:14:34, 2.09s/it] 48%|████▊ | 3632/7500 [2:02:49<2:14:35, 2.09s/it] 48%|████▊ | 3633/7500 [2:02:51<2:13:55, 2.08s/it] 48%|████▊ | 3634/7500 [2:02:53<2:11:44, 2.04s/it] 48%|████▊ | 3635/7500 [2:02:55<2:10:21, 2.02s/it] 48%|████▊ | 3636/7500 [2:02:57<2:11:26, 2.04s/it] 48%|████▊ | 3637/7500 [2:02:59<2:11:59, 2.05s/it] 49%|████▊ | 3638/7500 [2:03:01<2:10:26, 2.03s/it] 49%|████▊ | 3639/7500 [2:03:03<2:11:35, 2.04s/it] 49%|████▊ | 3640/7500 [2:03:05<2:12:06, 2.05s/it] {'loss': 0.4144, 'grad_norm': 3.8708722591400146, 'learning_rate': 3.059542928183079e-05, 'logqs/rejected': -2.4322304725646973, 'logqs/chosen': -0.8854069709777832, 'logqs_logps/rejected': -0.028030887246131897, 'logps_logqs/chosen': 0.21880574524402618, 'epoch': 0.97} 49%|████▊ | 3640/7500 [2:03:05<2:12:06, 2.05s/it] 49%|████▊ | 3641/7500 [2:03:07<2:12:02, 2.05s/it] 49%|████▊ | 3642/7500 [2:03:09<2:12:45, 2.06s/it] 49%|████▊ | 3643/7500 [2:03:12<2:13:01, 2.07s/it] 49%|████▊ | 3644/7500 [2:03:14<2:12:30, 2.06s/it] 49%|████▊ | 3645/7500 [2:03:16<2:12:28, 2.06s/it] 49%|████▊ | 3646/7500 [2:03:18<2:12:58, 2.07s/it] 49%|████▊ | 3647/7500 [2:03:20<2:12:17, 2.06s/it] 49%|████▊ | 3648/7500 [2:03:22<2:12:54, 2.07s/it] 49%|████▊ | 3649/7500 [2:03:24<2:09:50, 2.02s/it] 49%|████▊ | 3650/7500 [2:03:26<2:13:31, 2.08s/it] {'loss': 0.4381, 'grad_norm': 3.797109365463257, 'learning_rate': 3.048196560469758e-05, 'logqs/rejected': -2.7148005962371826, 'logqs/chosen': -0.8881216049194336, 'logqs_logps/rejected': -0.014678200706839561, 'logps_logqs/chosen': 0.2195882350206375, 'epoch': 0.97} 49%|████▊ | 3650/7500 [2:03:26<2:13:31, 2.08s/it] 49%|████▊ | 3651/7500 [2:03:28<2:13:42, 2.08s/it] 49%|████▊ | 3652/7500 [2:03:30<2:13:51, 2.09s/it] 49%|████▊ | 3653/7500 [2:03:32<2:12:47, 2.07s/it] 49%|████▊ | 3654/7500 [2:03:34<2:13:33, 2.08s/it] 49%|████▊ | 3655/7500 [2:03:36<2:12:16, 2.06s/it] 49%|████▊ | 3656/7500 [2:03:38<2:12:32, 2.07s/it] 49%|████▉ | 3657/7500 [2:03:40<2:11:15, 2.05s/it] 49%|████▉ | 3658/7500 [2:03:43<2:11:24, 2.05s/it] 49%|████▉ | 3659/7500 [2:03:45<2:12:01, 2.06s/it] 49%|████▉ | 3660/7500 [2:03:47<2:10:19, 2.04s/it] {'loss': 0.4066, 'grad_norm': 3.6761577129364014, 'learning_rate': 3.0368383179176585e-05, 'logqs/rejected': -2.323202610015869, 'logqs/chosen': -0.8823652267456055, 'logqs_logps/rejected': -0.021309545263648033, 'logps_logqs/chosen': 0.21649804711341858, 'epoch': 0.98} 49%|████▉ | 3660/7500 [2:03:47<2:10:19, 2.04s/it] 49%|████▉ | 3661/7500 [2:03:49<2:12:50, 2.08s/it] 49%|████▉ | 3662/7500 [2:03:51<2:11:30, 2.06s/it] 49%|████▉ | 3663/7500 [2:03:53<2:10:27, 2.04s/it] 49%|████▉ | 3664/7500 [2:03:55<2:11:53, 2.06s/it] 49%|████▉ | 3665/7500 [2:03:57<2:12:34, 2.07s/it] 49%|████▉ | 3666/7500 [2:03:59<2:14:05, 2.10s/it] 49%|████▉ | 3667/7500 [2:04:01<2:12:10, 2.07s/it] 49%|████▉ | 3668/7500 [2:04:03<2:12:45, 2.08s/it] 49%|████▉ | 3669/7500 [2:04:05<2:12:12, 2.07s/it] 49%|████▉ | 3670/7500 [2:04:07<2:10:38, 2.05s/it] {'loss': 0.4364, 'grad_norm': 3.588770866394043, 'learning_rate': 3.025468446564985e-05, 'logqs/rejected': -2.5458903312683105, 'logqs/chosen': -0.9042037725448608, 'logqs_logps/rejected': -0.020956292748451233, 'logps_logqs/chosen': 0.22491364181041718, 'epoch': 0.98} 49%|████▉ | 3670/7500 [2:04:07<2:10:38, 2.05s/it] 49%|████▉ | 3671/7500 [2:04:09<2:10:39, 2.05s/it] 49%|████▉ | 3672/7500 [2:04:11<2:11:31, 2.06s/it] 49%|████▉ | 3673/7500 [2:04:13<2:11:42, 2.06s/it] 49%|████▉ | 3674/7500 [2:04:16<2:11:55, 2.07s/it] 49%|████▉ | 3675/7500 [2:04:18<2:13:11, 2.09s/it] 49%|████▉ | 3676/7500 [2:04:20<2:13:51, 2.10s/it] 49%|████▉ | 3677/7500 [2:04:22<2:11:45, 2.07s/it] 49%|████▉ | 3678/7500 [2:04:24<2:10:54, 2.06s/it] 49%|████▉ | 3679/7500 [2:04:26<2:11:19, 2.06s/it] 49%|████▉ | 3680/7500 [2:04:28<2:11:32, 2.07s/it] {'loss': 0.4269, 'grad_norm': 3.382664203643799, 'learning_rate': 3.0140871927018467e-05, 'logqs/rejected': -2.4124581813812256, 'logqs/chosen': -0.9094220399856567, 'logqs_logps/rejected': -0.027191516011953354, 'logps_logqs/chosen': 0.21751996874809265, 'epoch': 0.98} 49%|████▉ | 3680/7500 [2:04:28<2:11:32, 2.07s/it] 49%|████▉ | 3681/7500 [2:04:30<2:10:31, 2.05s/it] 49%|████▉ | 3682/7500 [2:04:32<2:09:43, 2.04s/it] 49%|████▉ | 3683/7500 [2:04:34<2:10:59, 2.06s/it] 49%|████▉ | 3684/7500 [2:04:36<2:10:34, 2.05s/it] 49%|████▉ | 3685/7500 [2:04:38<2:10:04, 2.05s/it] 49%|████▉ | 3686/7500 [2:04:40<2:11:10, 2.06s/it] 49%|████▉ | 3687/7500 [2:04:42<2:11:20, 2.07s/it] 49%|████▉ | 3688/7500 [2:04:44<2:11:46, 2.07s/it] 49%|████▉ | 3689/7500 [2:04:46<2:10:31, 2.06s/it] 49%|████▉ | 3690/7500 [2:04:49<2:10:22, 2.05s/it] {'loss': 0.3939, 'grad_norm': 3.4487051963806152, 'learning_rate': 3.002694802864912e-05, 'logqs/rejected': -2.4276037216186523, 'logqs/chosen': -0.8684017062187195, 'logqs_logps/rejected': -0.022442784160375595, 'logps_logqs/chosen': 0.21575114130973816, 'epoch': 0.98} 49%|████▉ | 3690/7500 [2:04:49<2:10:22, 2.05s/it] 49%|████▉ | 3691/7500 [2:04:51<2:10:53, 2.06s/it] 49%|████▉ | 3692/7500 [2:04:53<2:10:44, 2.06s/it] 49%|████▉ | 3693/7500 [2:04:55<2:11:26, 2.07s/it] 49%|████▉ | 3694/7500 [2:04:57<2:10:31, 2.06s/it] 49%|████▉ | 3695/7500 [2:04:59<2:12:01, 2.08s/it] 49%|████▉ | 3696/7500 [2:05:01<2:11:23, 2.07s/it] 49%|████▉ | 3697/7500 [2:05:03<2:10:37, 2.06s/it] 49%|████▉ | 3698/7500 [2:05:05<2:11:35, 2.08s/it] 49%|████▉ | 3699/7500 [2:05:07<2:10:32, 2.06s/it] 49%|████▉ | 3700/7500 [2:05:09<2:10:37, 2.06s/it] {'loss': 0.4521, 'grad_norm': 3.992398500442505, 'learning_rate': 2.9912915238320754e-05, 'logqs/rejected': -2.663504123687744, 'logqs/chosen': -0.8989904522895813, 'logqs_logps/rejected': -0.01561348419636488, 'logps_logqs/chosen': 0.21963195502758026, 'epoch': 0.99} 49%|████▉ | 3700/7500 [2:05:09<2:10:37, 2.06s/it] 49%|████▉ | 3701/7500 [2:05:11<2:10:29, 2.06s/it] 49%|████▉ | 3702/7500 [2:05:13<2:08:49, 2.04s/it] 49%|████▉ | 3703/7500 [2:05:15<2:07:38, 2.02s/it] 49%|████▉ | 3704/7500 [2:05:17<2:06:55, 2.01s/it] 49%|████▉ | 3705/7500 [2:05:19<2:07:07, 2.01s/it] 49%|████▉ | 3706/7500 [2:05:21<2:07:33, 2.02s/it] 49%|████▉ | 3707/7500 [2:05:23<2:10:21, 2.06s/it] 49%|████▉ | 3708/7500 [2:05:25<2:10:06, 2.06s/it] 49%|████▉ | 3709/7500 [2:05:27<2:08:42, 2.04s/it] 49%|████▉ | 3710/7500 [2:05:29<2:06:53, 2.01s/it] {'loss': 0.421, 'grad_norm': 3.727295398712158, 'learning_rate': 2.9798776026171086e-05, 'logqs/rejected': -2.541208505630493, 'logqs/chosen': -0.8715192079544067, 'logqs_logps/rejected': -0.0237265694886446, 'logps_logqs/chosen': 0.21562299132347107, 'epoch': 0.99} 49%|████▉ | 3710/7500 [2:05:29<2:06:53, 2.01s/it] 49%|████▉ | 3711/7500 [2:05:31<2:07:24, 2.02s/it] 49%|████▉ | 3712/7500 [2:05:34<2:08:57, 2.04s/it] 50%|████▉ | 3713/7500 [2:05:36<2:08:48, 2.04s/it] 50%|████▉ | 3714/7500 [2:05:38<2:08:37, 2.04s/it] 50%|████▉ | 3715/7500 [2:05:40<2:11:48, 2.09s/it] 50%|████▉ | 3716/7500 [2:05:42<2:09:45, 2.06s/it] 50%|████▉ | 3717/7500 [2:05:44<2:10:08, 2.06s/it] 50%|████▉ | 3718/7500 [2:05:46<2:09:05, 2.05s/it] 50%|████▉ | 3719/7500 [2:05:48<2:09:23, 2.05s/it] 50%|████▉ | 3720/7500 [2:05:50<2:09:51, 2.06s/it] {'loss': 0.4528, 'grad_norm': 3.540454149246216, 'learning_rate': 2.9684532864643122e-05, 'logqs/rejected': -2.654181480407715, 'logqs/chosen': -0.89490807056427, 'logqs_logps/rejected': -0.02075108513236046, 'logps_logqs/chosen': 0.223428413271904, 'epoch': 0.99} 50%|████▉ | 3720/7500 [2:05:50<2:09:51, 2.06s/it] 50%|████▉ | 3721/7500 [2:05:52<2:09:50, 2.06s/it] 50%|████▉ | 3722/7500 [2:05:54<2:08:26, 2.04s/it] 50%|████▉ | 3723/7500 [2:05:56<2:07:51, 2.03s/it] 50%|████▉ | 3724/7500 [2:05:58<2:08:30, 2.04s/it] 50%|████▉ | 3725/7500 [2:06:00<2:05:53, 2.00s/it] 50%|████▉ | 3726/7500 [2:06:02<2:07:08, 2.02s/it] 50%|████▉ | 3727/7500 [2:06:04<2:05:54, 2.00s/it] 50%|████▉ | 3728/7500 [2:06:06<2:07:34, 2.03s/it] 50%|████▉ | 3729/7500 [2:06:08<2:07:11, 2.02s/it] 50%|████▉ | 3730/7500 [2:06:10<2:06:46, 2.02s/it] {'loss': 0.4252, 'grad_norm': 3.052727699279785, 'learning_rate': 2.957018822843154e-05, 'logqs/rejected': -2.8100619316101074, 'logqs/chosen': -0.8746237754821777, 'logqs_logps/rejected': -0.024813393130898476, 'logps_logqs/chosen': 0.2240094244480133, 'epoch': 0.99} 50%|████▉ | 3730/7500 [2:06:10<2:06:46, 2.02s/it] 50%|████▉ | 3731/7500 [2:06:12<2:08:16, 2.04s/it] 50%|████▉ | 3732/7500 [2:06:14<2:08:32, 2.05s/it] 50%|████▉ | 3733/7500 [2:06:16<2:09:22, 2.06s/it] 50%|████▉ | 3734/7500 [2:06:19<2:09:42, 2.07s/it] 50%|████▉ | 3735/7500 [2:06:21<2:09:08, 2.06s/it] 50%|████▉ | 3736/7500 [2:06:23<2:08:41, 2.05s/it] 50%|████▉ | 3737/7500 [2:06:25<2:08:47, 2.05s/it] 50%|████▉ | 3738/7500 [2:06:27<2:08:50, 2.05s/it] 50%|████▉ | 3739/7500 [2:06:29<2:08:47, 2.05s/it] 50%|████▉ | 3740/7500 [2:06:31<2:09:46, 2.07s/it] {'loss': 0.4245, 'grad_norm': 4.108197212219238, 'learning_rate': 2.945574459442917e-05, 'logqs/rejected': -2.6481635570526123, 'logqs/chosen': -0.9141990542411804, 'logqs_logps/rejected': -0.023473404347896576, 'logps_logqs/chosen': 0.22640585899353027, 'epoch': 1.0} 50%|████▉ | 3740/7500 [2:06:31<2:09:46, 2.07s/it] 50%|████▉ | 3741/7500 [2:06:33<2:09:34, 2.07s/it] 50%|████▉ | 3742/7500 [2:06:35<2:11:05, 2.09s/it] 50%|████▉ | 3743/7500 [2:06:37<2:10:13, 2.08s/it] 50%|████▉ | 3744/7500 [2:06:39<2:09:53, 2.07s/it] 50%|████▉ | 3745/7500 [2:06:41<2:08:57, 2.06s/it] 50%|████▉ | 3746/7500 [2:06:43<2:09:22, 2.07s/it] 50%|████▉ | 3747/7500 [2:06:45<2:09:20, 2.07s/it] 50%|████▉ | 3748/7500 [2:06:47<2:09:38, 2.07s/it] 50%|████▉ | 3749/7500 [2:06:50<2:09:10, 2.07s/it] 50%|█████ | 3750/7500 [2:06:52<2:08:48, 2.06s/it] {'loss': 0.4257, 'grad_norm': 3.2118029594421387, 'learning_rate': 2.9341204441673266e-05, 'logqs/rejected': -2.480656385421753, 'logqs/chosen': -0.9002636671066284, 'logqs_logps/rejected': -0.026851970702409744, 'logps_logqs/chosen': 0.22440561652183533, 'epoch': 1.0} 50%|█████ | 3750/7500 [2:06:52<2:08:48, 2.06s/it][INFO|trainer.py:3503] 2025-12-21 15:15:40,425 >> Saving model checkpoint to ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-3750 [INFO|configuration_utils.py:472] 2025-12-21 15:15:40,429 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-3750/config.json [INFO|configuration_utils.py:807] 2025-12-21 15:15:40,429 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-3750/generation_config.json [INFO|modeling_utils.py:2755] 2025-12-21 15:15:42,384 >> Model weights saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-3750/model.safetensors [INFO|tokenization_utils_base.py:2702] 2025-12-21 15:15:42,385 >> tokenizer config file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-3750/tokenizer_config.json [INFO|tokenization_utils_base.py:2711] 2025-12-21 15:15:42,385 >> Special tokens file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-3750/special_tokens_map.json 50%|█████ | 3751/7500 [2:06:56<2:47:48, 2.69s/it] 50%|█████ | 3752/7500 [2:06:58<2:35:55, 2.50s/it] 50%|█████ | 3753/7500 [2:07:00<2:26:58, 2.35s/it] 50%|█████ | 3754/7500 [2:07:02<2:23:01, 2.29s/it] 50%|█████ | 3755/7500 [2:07:04<2:20:55, 2.26s/it] 50%|█████ | 3756/7500 [2:07:06<2:14:53, 2.16s/it] 50%|█████ | 3757/7500 [2:07:08<2:15:03, 2.16s/it] 50%|█████ | 3758/7500 [2:07:10<2:11:59, 2.12s/it] 50%|█████ | 3759/7500 [2:07:12<2:12:02, 2.12s/it] 50%|█████ | 3760/7500 [2:07:14<2:09:33, 2.08s/it] {'loss': 0.3696, 'grad_norm': 3.4118478298187256, 'learning_rate': 2.922657025129185e-05, 'logqs/rejected': -2.573075532913208, 'logqs/chosen': -0.8573020696640015, 'logqs_logps/rejected': -0.025045379996299744, 'logps_logqs/chosen': 0.20706097781658173, 'epoch': 1.0} 50%|█████ | 3760/7500 [2:07:14<2:09:33, 2.08s/it] 50%|█████ | 3761/7500 [2:07:16<2:09:27, 2.08s/it] 50%|█████ | 3762/7500 [2:07:18<2:07:53, 2.05s/it] 50%|█████ | 3763/7500 [2:07:20<2:08:15, 2.06s/it] 50%|█████ | 3764/7500 [2:07:22<2:07:03, 2.04s/it] 50%|█████ | 3765/7500 [2:07:25<2:06:59, 2.04s/it] 50%|█████ | 3766/7500 [2:07:27<2:06:53, 2.04s/it] 50%|█████ | 3767/7500 [2:07:29<2:06:54, 2.04s/it] 50%|█████ | 3768/7500 [2:07:31<2:07:06, 2.04s/it] 50%|█████ | 3769/7500 [2:07:33<2:06:33, 2.04s/it] 50%|█████ | 3770/7500 [2:07:35<2:06:07, 2.03s/it] {'loss': 0.366, 'grad_norm': 2.8177638053894043, 'learning_rate': 2.911184450644997e-05, 'logqs/rejected': -2.546245574951172, 'logqs/chosen': -0.8285255432128906, 'logqs_logps/rejected': -0.026740044355392456, 'logps_logqs/chosen': 0.21397237479686737, 'epoch': 1.01} 50%|█████ | 3770/7500 [2:07:35<2:06:07, 2.03s/it] 50%|█████ | 3771/7500 [2:07:37<2:06:03, 2.03s/it] 50%|█████ | 3772/7500 [2:07:39<2:06:32, 2.04s/it] 50%|█████ | 3773/7500 [2:07:41<2:06:37, 2.04s/it] 50%|█████ | 3774/7500 [2:07:43<2:08:17, 2.07s/it] 50%|█████ | 3775/7500 [2:07:45<2:09:37, 2.09s/it] 50%|█████ | 3776/7500 [2:07:47<2:09:38, 2.09s/it] 50%|█████ | 3777/7500 [2:07:49<2:09:13, 2.08s/it] 50%|█████ | 3778/7500 [2:07:51<2:08:34, 2.07s/it] 50%|█████ | 3779/7500 [2:07:53<2:06:06, 2.03s/it] 50%|█████ | 3780/7500 [2:07:55<2:04:39, 2.01s/it] {'loss': 0.3583, 'grad_norm': 3.0272610187530518, 'learning_rate': 2.8997029692295874e-05, 'logqs/rejected': -2.3936374187469482, 'logqs/chosen': -0.8580888509750366, 'logqs_logps/rejected': -0.027756880968809128, 'logps_logqs/chosen': 0.20979544520378113, 'epoch': 1.01} 50%|█████ | 3780/7500 [2:07:55<2:04:39, 2.01s/it] 50%|█████ | 3781/7500 [2:07:57<2:05:53, 2.03s/it] 50%|█████ | 3782/7500 [2:07:59<2:06:52, 2.05s/it] 50%|█████ | 3783/7500 [2:08:01<2:05:18, 2.02s/it] 50%|█████ | 3784/7500 [2:08:03<2:04:25, 2.01s/it] 50%|█████ | 3785/7500 [2:08:05<2:04:21, 2.01s/it] 50%|█████ | 3786/7500 [2:08:07<2:05:04, 2.02s/it] 50%|█████ | 3787/7500 [2:08:09<2:06:58, 2.05s/it] 51%|█████ | 3788/7500 [2:08:11<2:06:22, 2.04s/it] 51%|█████ | 3789/7500 [2:08:14<2:05:57, 2.04s/it] 51%|█████ | 3790/7500 [2:08:15<2:04:52, 2.02s/it] {'loss': 0.3607, 'grad_norm': 2.7352941036224365, 'learning_rate': 2.888212829590719e-05, 'logqs/rejected': -2.5094265937805176, 'logqs/chosen': -0.822465717792511, 'logqs_logps/rejected': -0.02889898419380188, 'logps_logqs/chosen': 0.20814833045005798, 'epoch': 1.01} 51%|█████ | 3790/7500 [2:08:16<2:04:52, 2.02s/it] 51%|█████ | 3791/7500 [2:08:17<2:03:28, 2.00s/it] 51%|█████ | 3792/7500 [2:08:20<2:07:25, 2.06s/it] 51%|█████ | 3793/7500 [2:08:22<2:07:04, 2.06s/it] 51%|█████ | 3794/7500 [2:08:24<2:05:15, 2.03s/it] 51%|█████ | 3795/7500 [2:08:26<2:07:01, 2.06s/it] 51%|█████ | 3796/7500 [2:08:28<2:06:46, 2.05s/it] 51%|█████ | 3797/7500 [2:08:30<2:06:42, 2.05s/it] 51%|█████ | 3798/7500 [2:08:32<2:06:40, 2.05s/it] 51%|█████ | 3799/7500 [2:08:34<2:07:22, 2.06s/it] 51%|█████ | 3800/7500 [2:08:36<2:09:51, 2.11s/it] {'loss': 0.3559, 'grad_norm': 3.2978217601776123, 'learning_rate': 2.876714280623708e-05, 'logqs/rejected': -2.568937063217163, 'logqs/chosen': -0.8652032613754272, 'logqs_logps/rejected': -0.025779832154512405, 'logps_logqs/chosen': 0.20318134129047394, 'epoch': 1.01} 51%|█████ | 3800/7500 [2:08:36<2:09:51, 2.11s/it] 51%|█████ | 3801/7500 [2:08:38<2:09:57, 2.11s/it] 51%|█████ | 3802/7500 [2:08:40<2:09:13, 2.10s/it] 51%|█████ | 3803/7500 [2:08:43<2:09:54, 2.11s/it] 51%|█████ | 3804/7500 [2:08:45<2:09:27, 2.10s/it] 51%|█████ | 3805/7500 [2:08:47<2:08:46, 2.09s/it] 51%|█████ | 3806/7500 [2:08:49<2:07:54, 2.08s/it] 51%|█████ | 3807/7500 [2:08:51<2:08:14, 2.08s/it] 51%|█████ | 3808/7500 [2:08:53<2:09:59, 2.11s/it] 51%|█████ | 3809/7500 [2:08:55<2:08:44, 2.09s/it] 51%|█████ | 3810/7500 [2:08:57<2:07:26, 2.07s/it] {'loss': 0.3866, 'grad_norm': 3.3046298027038574, 'learning_rate': 2.8652075714060295e-05, 'logqs/rejected': -2.628119945526123, 'logqs/chosen': -0.8731848001480103, 'logqs_logps/rejected': -0.02408771775662899, 'logps_logqs/chosen': 0.2128746211528778, 'epoch': 1.02} 51%|█████ | 3810/7500 [2:08:57<2:07:26, 2.07s/it] 51%|█████ | 3811/7500 [2:08:59<2:08:17, 2.09s/it] 51%|█████ | 3812/7500 [2:09:01<2:06:22, 2.06s/it] 51%|█████ | 3813/7500 [2:09:03<2:08:26, 2.09s/it] 51%|█████ | 3814/7500 [2:09:05<2:06:57, 2.07s/it] 51%|█████ | 3815/7500 [2:09:07<2:05:43, 2.05s/it] 51%|█████ | 3816/7500 [2:09:09<2:05:12, 2.04s/it] 51%|█████ | 3817/7500 [2:09:12<2:07:50, 2.08s/it] 51%|█████ | 3818/7500 [2:09:14<2:06:24, 2.06s/it] 51%|█████ | 3819/7500 [2:09:16<2:04:19, 2.03s/it] 51%|█████ | 3820/7500 [2:09:18<2:05:46, 2.05s/it] {'loss': 0.3658, 'grad_norm': 3.332134246826172, 'learning_rate': 2.8536929511919226e-05, 'logqs/rejected': -2.6642885208129883, 'logqs/chosen': -0.8549314737319946, 'logqs_logps/rejected': -0.02769007720053196, 'logps_logqs/chosen': 0.21163764595985413, 'epoch': 1.02} 51%|█████ | 3820/7500 [2:09:18<2:05:46, 2.05s/it] 51%|█████ | 3821/7500 [2:09:20<2:03:33, 2.02s/it] 51%|█████ | 3822/7500 [2:09:22<2:04:42, 2.03s/it] 51%|█████ | 3823/7500 [2:09:24<2:04:57, 2.04s/it] 51%|█████ | 3824/7500 [2:09:26<2:04:08, 2.03s/it] 51%|█████ | 3825/7500 [2:09:28<2:05:48, 2.05s/it] 51%|█████ | 3826/7500 [2:09:30<2:05:05, 2.04s/it] 51%|█████ | 3827/7500 [2:09:32<2:04:47, 2.04s/it] 51%|█████ | 3828/7500 [2:09:34<2:05:56, 2.06s/it] 51%|█████ | 3829/7500 [2:09:36<2:06:21, 2.07s/it] 51%|█████ | 3830/7500 [2:09:38<2:06:11, 2.06s/it] {'loss': 0.3515, 'grad_norm': 2.234992742538452, 'learning_rate': 2.8421706694069926e-05, 'logqs/rejected': -2.553647518157959, 'logqs/chosen': -0.8357205390930176, 'logqs_logps/rejected': -0.024310428649187088, 'logps_logqs/chosen': 0.2041347473859787, 'epoch': 1.02} 51%|█████ | 3830/7500 [2:09:38<2:06:11, 2.06s/it] 51%|█████ | 3831/7500 [2:09:40<2:06:49, 2.07s/it] 51%|█████ | 3832/7500 [2:09:42<2:06:56, 2.08s/it] 51%|█████ | 3833/7500 [2:09:44<2:05:46, 2.06s/it] 51%|█████ | 3834/7500 [2:09:46<2:05:33, 2.05s/it] 51%|█████ | 3835/7500 [2:09:48<2:05:29, 2.05s/it] 51%|█████ | 3836/7500 [2:09:50<2:04:52, 2.04s/it] 51%|█████ | 3837/7500 [2:09:53<2:05:20, 2.05s/it] 51%|█████ | 3838/7500 [2:09:55<2:04:53, 2.05s/it] 51%|█████ | 3839/7500 [2:09:57<2:04:59, 2.05s/it] 51%|█████ | 3840/7500 [2:09:59<2:05:10, 2.05s/it] {'loss': 0.3694, 'grad_norm': 2.455482006072998, 'learning_rate': 2.8306409756428064e-05, 'logqs/rejected': -2.5598766803741455, 'logqs/chosen': -0.8107408285140991, 'logqs_logps/rejected': -0.02223910018801689, 'logps_logqs/chosen': 0.2060507982969284, 'epoch': 1.02} 51%|█████ | 3840/7500 [2:09:59<2:05:10, 2.05s/it] 51%|█████ | 3841/7500 [2:10:01<2:05:58, 2.07s/it] 51%|█████ | 3842/7500 [2:10:03<2:05:05, 2.05s/it] 51%|█████ | 3843/7500 [2:10:05<2:05:46, 2.06s/it] 51%|█████▏ | 3844/7500 [2:10:07<2:06:18, 2.07s/it] 51%|█████▏ | 3845/7500 [2:10:09<2:06:05, 2.07s/it] 51%|█████▏ | 3846/7500 [2:10:11<2:08:20, 2.11s/it] 51%|█████▏ | 3847/7500 [2:10:13<2:08:34, 2.11s/it] 51%|█████▏ | 3848/7500 [2:10:15<2:07:10, 2.09s/it] 51%|█████▏ | 3849/7500 [2:10:17<2:07:02, 2.09s/it] 51%|█████▏ | 3850/7500 [2:10:20<2:09:07, 2.12s/it] {'loss': 0.3772, 'grad_norm': 3.2678844928741455, 'learning_rate': 2.8191041196514873e-05, 'logqs/rejected': -2.6692113876342773, 'logqs/chosen': -0.8774774670600891, 'logqs_logps/rejected': -0.02850523218512535, 'logps_logqs/chosen': 0.2065679132938385, 'epoch': 1.03} 51%|█████▏ | 3850/7500 [2:10:20<2:09:07, 2.12s/it] 51%|█████▏ | 3851/7500 [2:10:22<2:07:56, 2.10s/it] 51%|█████▏ | 3852/7500 [2:10:24<2:06:05, 2.07s/it] 51%|█████▏ | 3853/7500 [2:10:26<2:05:27, 2.06s/it] 51%|█████▏ | 3854/7500 [2:10:28<2:06:25, 2.08s/it] 51%|█████▏ | 3855/7500 [2:10:30<2:05:20, 2.06s/it] 51%|█████▏ | 3856/7500 [2:10:32<2:05:40, 2.07s/it] 51%|█████▏ | 3857/7500 [2:10:34<2:04:30, 2.05s/it] 51%|█████▏ | 3858/7500 [2:10:36<2:03:22, 2.03s/it] 51%|█████▏ | 3859/7500 [2:10:38<2:02:58, 2.03s/it] 51%|█████▏ | 3860/7500 [2:10:40<2:03:59, 2.04s/it] {'loss': 0.361, 'grad_norm': 3.0699076652526855, 'learning_rate': 2.807560351340302e-05, 'logqs/rejected': -2.495346784591675, 'logqs/chosen': -0.8632195591926575, 'logqs_logps/rejected': -0.030787089839577675, 'logps_logqs/chosen': 0.21163272857666016, 'epoch': 1.03} 51%|█████▏ | 3860/7500 [2:10:40<2:03:59, 2.04s/it] 51%|█████▏ | 3861/7500 [2:10:42<2:04:45, 2.06s/it] 51%|█████▏ | 3862/7500 [2:10:44<2:05:00, 2.06s/it] 52%|█████▏ | 3863/7500 [2:10:46<2:05:03, 2.06s/it] 52%|█████▏ | 3864/7500 [2:10:48<2:04:11, 2.05s/it] 52%|█████▏ | 3865/7500 [2:10:50<2:04:49, 2.06s/it] 52%|█████▏ | 3866/7500 [2:10:53<2:07:20, 2.10s/it] 52%|█████▏ | 3867/7500 [2:10:55<2:06:04, 2.08s/it] 52%|█████▏ | 3868/7500 [2:10:57<2:05:58, 2.08s/it] 52%|█████▏ | 3869/7500 [2:10:59<2:05:00, 2.07s/it] 52%|█████▏ | 3870/7500 [2:11:01<2:04:23, 2.06s/it] {'loss': 0.3652, 'grad_norm': 2.6701107025146484, 'learning_rate': 2.7960099207662532e-05, 'logqs/rejected': -2.5266778469085693, 'logqs/chosen': -0.8574451208114624, 'logqs_logps/rejected': -0.023384204134345055, 'logps_logqs/chosen': 0.20978060364723206, 'epoch': 1.03} 52%|█████▏ | 3870/7500 [2:11:01<2:04:23, 2.06s/it] 52%|█████▏ | 3871/7500 [2:11:03<2:05:24, 2.07s/it] 52%|█████▏ | 3872/7500 [2:11:05<2:05:14, 2.07s/it] 52%|█████▏ | 3873/7500 [2:11:07<2:04:05, 2.05s/it] 52%|█████▏ | 3874/7500 [2:11:09<2:04:49, 2.07s/it] 52%|█████▏ | 3875/7500 [2:11:11<2:05:41, 2.08s/it] 52%|█████▏ | 3876/7500 [2:11:13<2:06:17, 2.09s/it] 52%|█████▏ | 3877/7500 [2:11:15<2:04:28, 2.06s/it] 52%|█████▏ | 3878/7500 [2:11:17<2:02:37, 2.03s/it] 52%|█████▏ | 3879/7500 [2:11:19<2:02:46, 2.03s/it] 52%|█████▏ | 3880/7500 [2:11:21<2:03:42, 2.05s/it] {'loss': 0.3588, 'grad_norm': 3.041146755218506, 'learning_rate': 2.7844530781306544e-05, 'logqs/rejected': -2.550330638885498, 'logqs/chosen': -0.8517144918441772, 'logqs_logps/rejected': -0.0280965156853199, 'logps_logqs/chosen': 0.20782849192619324, 'epoch': 1.03} 52%|█████▏ | 3880/7500 [2:11:21<2:03:42, 2.05s/it] 52%|█████▏ | 3881/7500 [2:11:24<2:05:41, 2.08s/it] 52%|█████▏ | 3882/7500 [2:11:26<2:05:00, 2.07s/it] 52%|█████▏ | 3883/7500 [2:11:28<2:04:29, 2.07s/it] 52%|█████▏ | 3884/7500 [2:11:30<2:03:39, 2.05s/it] 52%|█████▏ | 3885/7500 [2:11:32<2:04:41, 2.07s/it] 52%|█████▏ | 3886/7500 [2:11:34<2:03:32, 2.05s/it] 52%|█████▏ | 3887/7500 [2:11:36<2:02:27, 2.03s/it] 52%|█████▏ | 3888/7500 [2:11:38<2:02:05, 2.03s/it] 52%|█████▏ | 3889/7500 [2:11:40<2:01:39, 2.02s/it] 52%|█████▏ | 3890/7500 [2:11:42<2:02:52, 2.04s/it] {'loss': 0.3544, 'grad_norm': 2.820336103439331, 'learning_rate': 2.77289007377372e-05, 'logqs/rejected': -2.463157892227173, 'logqs/chosen': -0.8101438283920288, 'logqs_logps/rejected': -0.023234792053699493, 'logps_logqs/chosen': 0.20112869143486023, 'epoch': 1.04} 52%|█████▏ | 3890/7500 [2:11:42<2:02:52, 2.04s/it] 52%|█████▏ | 3891/7500 [2:11:44<2:01:46, 2.02s/it] 52%|█████▏ | 3892/7500 [2:11:46<2:03:51, 2.06s/it] 52%|█████▏ | 3893/7500 [2:11:48<2:04:03, 2.06s/it] 52%|█████▏ | 3894/7500 [2:11:50<2:02:59, 2.05s/it] 52%|█████▏ | 3895/7500 [2:11:52<2:02:38, 2.04s/it] 52%|█████▏ | 3896/7500 [2:11:54<2:03:12, 2.05s/it] 52%|█████▏ | 3897/7500 [2:11:56<2:03:24, 2.06s/it] 52%|█████▏ | 3898/7500 [2:11:58<2:01:48, 2.03s/it] 52%|█████▏ | 3899/7500 [2:12:00<2:01:20, 2.02s/it] 52%|█████▏ | 3900/7500 [2:12:02<2:02:37, 2.04s/it] {'loss': 0.3775, 'grad_norm': 2.703286647796631, 'learning_rate': 2.761321158169134e-05, 'logqs/rejected': -2.709794521331787, 'logqs/chosen': -0.8376859426498413, 'logqs_logps/rejected': -0.024188725277781487, 'logps_logqs/chosen': 0.2079714834690094, 'epoch': 1.04} 52%|█████▏ | 3900/7500 [2:12:02<2:02:37, 2.04s/it] 52%|█████▏ | 3901/7500 [2:12:04<2:02:13, 2.04s/it] 52%|█████▏ | 3902/7500 [2:12:06<2:01:09, 2.02s/it] 52%|█████▏ | 3903/7500 [2:12:09<2:04:24, 2.08s/it] 52%|█████▏ | 3904/7500 [2:12:11<2:04:19, 2.07s/it] 52%|█████▏ | 3905/7500 [2:12:13<2:04:35, 2.08s/it] 52%|█████▏ | 3906/7500 [2:12:15<2:04:48, 2.08s/it] 52%|█████▏ | 3907/7500 [2:12:17<2:03:17, 2.06s/it] 52%|█████▏ | 3908/7500 [2:12:19<2:03:03, 2.06s/it] 52%|█████▏ | 3909/7500 [2:12:21<2:04:05, 2.07s/it] 52%|█████▏ | 3910/7500 [2:12:23<2:03:21, 2.06s/it] {'loss': 0.3805, 'grad_norm': 3.190605640411377, 'learning_rate': 2.7497465819186285e-05, 'logqs/rejected': -2.784821033477783, 'logqs/chosen': -0.8548698425292969, 'logqs_logps/rejected': -0.026128489524126053, 'logps_logqs/chosen': 0.21104547381401062, 'epoch': 1.04} 52%|█████▏ | 3910/7500 [2:12:23<2:03:21, 2.06s/it] 52%|█████▏ | 3911/7500 [2:12:25<2:03:49, 2.07s/it] 52%|█████▏ | 3912/7500 [2:12:27<2:02:51, 2.05s/it] 52%|█████▏ | 3913/7500 [2:12:29<2:03:26, 2.06s/it] 52%|█████▏ | 3914/7500 [2:12:31<2:02:59, 2.06s/it] 52%|█████▏ | 3915/7500 [2:12:33<2:02:31, 2.05s/it] 52%|█████▏ | 3916/7500 [2:12:35<2:01:55, 2.04s/it] 52%|█████▏ | 3917/7500 [2:12:37<2:01:35, 2.04s/it] 52%|█████▏ | 3918/7500 [2:12:39<2:01:55, 2.04s/it] 52%|█████▏ | 3919/7500 [2:12:41<2:02:47, 2.06s/it] 52%|█████▏ | 3920/7500 [2:12:43<2:02:17, 2.05s/it] {'loss': 0.3663, 'grad_norm': 2.9252302646636963, 'learning_rate': 2.7381665957465542e-05, 'logqs/rejected': -2.618222951889038, 'logqs/chosen': -0.8333565592765808, 'logqs_logps/rejected': -0.028155062347650528, 'logps_logqs/chosen': 0.2055988311767578, 'epoch': 1.05} 52%|█████▏ | 3920/7500 [2:12:44<2:02:17, 2.05s/it] 52%|█████▏ | 3921/7500 [2:12:46<2:02:09, 2.05s/it] 52%|█████▏ | 3922/7500 [2:12:48<2:04:22, 2.09s/it] 52%|█████▏ | 3923/7500 [2:12:50<2:04:17, 2.08s/it] 52%|█████▏ | 3924/7500 [2:12:52<2:03:53, 2.08s/it] 52%|█████▏ | 3925/7500 [2:12:54<2:00:52, 2.03s/it] 52%|█████▏ | 3926/7500 [2:12:56<2:02:14, 2.05s/it] 52%|█████▏ | 3927/7500 [2:12:58<2:01:46, 2.04s/it] 52%|█████▏ | 3928/7500 [2:13:00<2:01:06, 2.03s/it] 52%|█████▏ | 3929/7500 [2:13:02<2:02:18, 2.05s/it] 52%|█████▏ | 3930/7500 [2:13:04<2:01:25, 2.04s/it] {'loss': 0.3737, 'grad_norm': 2.8367724418640137, 'learning_rate': 2.726581450494451e-05, 'logqs/rejected': -2.511260986328125, 'logqs/chosen': -0.8477511405944824, 'logqs_logps/rejected': -0.028704609721899033, 'logps_logqs/chosen': 0.20871035754680634, 'epoch': 1.05} 52%|█████▏ | 3930/7500 [2:13:04<2:01:25, 2.04s/it] 52%|█████▏ | 3931/7500 [2:13:06<2:00:26, 2.02s/it] 52%|█████▏ | 3932/7500 [2:13:08<1:59:49, 2.01s/it] 52%|█████▏ | 3933/7500 [2:13:10<1:59:25, 2.01s/it] 52%|█████▏ | 3934/7500 [2:13:12<2:00:48, 2.03s/it] 52%|█████▏ | 3935/7500 [2:13:14<2:00:51, 2.03s/it] 52%|█████▏ | 3936/7500 [2:13:16<1:59:35, 2.01s/it] 52%|█████▏ | 3937/7500 [2:13:18<1:58:51, 2.00s/it] 53%|█████▎ | 3938/7500 [2:13:20<1:59:45, 2.02s/it] 53%|█████▎ | 3939/7500 [2:13:22<1:59:33, 2.01s/it] 53%|█████▎ | 3940/7500 [2:13:24<2:01:03, 2.04s/it] {'loss': 0.363, 'grad_norm': 2.5240511894226074, 'learning_rate': 2.7149913971156103e-05, 'logqs/rejected': -2.544522762298584, 'logqs/chosen': -0.845871090888977, 'logqs_logps/rejected': -0.03313130885362625, 'logps_logqs/chosen': 0.2091343104839325, 'epoch': 1.05} 53%|█████▎ | 3940/7500 [2:13:24<2:01:03, 2.04s/it] 53%|█████▎ | 3941/7500 [2:13:26<2:00:45, 2.04s/it] 53%|█████▎ | 3942/7500 [2:13:28<2:00:56, 2.04s/it] 53%|█████▎ | 3943/7500 [2:13:30<2:00:29, 2.03s/it] 53%|█████▎ | 3944/7500 [2:13:32<2:01:58, 2.06s/it] 53%|█████▎ | 3945/7500 [2:13:35<2:03:18, 2.08s/it] 53%|█████▎ | 3946/7500 [2:13:37<2:00:45, 2.04s/it] 53%|█████▎ | 3947/7500 [2:13:39<2:00:59, 2.04s/it] 53%|█████▎ | 3948/7500 [2:13:41<1:59:28, 2.02s/it] 53%|█████▎ | 3949/7500 [2:13:43<1:59:32, 2.02s/it] 53%|█████▎ | 3950/7500 [2:13:44<1:58:19, 2.00s/it] {'loss': 0.3682, 'grad_norm': 2.918085813522339, 'learning_rate': 2.7033966866696457e-05, 'logqs/rejected': -2.574988603591919, 'logqs/chosen': -0.8562361001968384, 'logqs_logps/rejected': -0.02945718728005886, 'logps_logqs/chosen': 0.2022906243801117, 'epoch': 1.05} 53%|█████▎ | 3950/7500 [2:13:45<1:58:19, 2.00s/it] 53%|█████▎ | 3951/7500 [2:13:47<1:59:38, 2.02s/it] 53%|█████▎ | 3952/7500 [2:13:49<1:59:06, 2.01s/it] 53%|█████▎ | 3953/7500 [2:13:51<1:58:47, 2.01s/it] 53%|█████▎ | 3954/7500 [2:13:53<2:00:34, 2.04s/it] 53%|█████▎ | 3955/7500 [2:13:55<1:59:55, 2.03s/it] 53%|█████▎ | 3956/7500 [2:13:57<1:59:31, 2.02s/it] 53%|█████▎ | 3957/7500 [2:13:59<2:00:02, 2.03s/it] 53%|█████▎ | 3958/7500 [2:14:01<1:59:17, 2.02s/it] 53%|█████▎ | 3959/7500 [2:14:03<1:58:23, 2.01s/it] 53%|█████▎ | 3960/7500 [2:14:05<1:59:24, 2.02s/it] {'loss': 0.3806, 'grad_norm': 3.134615898132324, 'learning_rate': 2.6917975703170466e-05, 'logqs/rejected': -2.6317994594573975, 'logqs/chosen': -0.8444727063179016, 'logqs_logps/rejected': -0.01830771006643772, 'logps_logqs/chosen': 0.2029609978199005, 'epoch': 1.06} 53%|█████▎ | 3960/7500 [2:14:05<1:59:24, 2.02s/it] 53%|█████▎ | 3961/7500 [2:14:07<2:00:12, 2.04s/it] 53%|█████▎ | 3962/7500 [2:14:09<1:59:13, 2.02s/it] 53%|█████▎ | 3963/7500 [2:14:11<2:00:01, 2.04s/it] 53%|█████▎ | 3964/7500 [2:14:13<1:59:19, 2.02s/it] 53%|█████▎ | 3965/7500 [2:14:15<2:01:35, 2.06s/it] 53%|█████▎ | 3966/7500 [2:14:17<2:01:18, 2.06s/it] 53%|█████▎ | 3967/7500 [2:14:19<2:00:57, 2.05s/it] 53%|█████▎ | 3968/7500 [2:14:21<2:00:02, 2.04s/it] 53%|█████▎ | 3969/7500 [2:14:23<1:59:09, 2.02s/it] 53%|█████▎ | 3970/7500 [2:14:25<2:00:13, 2.04s/it] {'loss': 0.3787, 'grad_norm': 2.674680709838867, 'learning_rate': 2.6801942993137437e-05, 'logqs/rejected': -2.772082805633545, 'logqs/chosen': -0.8271814584732056, 'logqs_logps/rejected': -0.028408724814653397, 'logps_logqs/chosen': 0.20594987273216248, 'epoch': 1.06} 53%|█████▎ | 3970/7500 [2:14:25<2:00:13, 2.04s/it] 53%|█████▎ | 3971/7500 [2:14:27<2:00:34, 2.05s/it] 53%|█████▎ | 3972/7500 [2:14:29<1:58:43, 2.02s/it] 53%|█████▎ | 3973/7500 [2:14:31<1:59:52, 2.04s/it] 53%|█████▎ | 3974/7500 [2:14:33<1:59:21, 2.03s/it] 53%|█████▎ | 3975/7500 [2:14:35<1:58:37, 2.02s/it] 53%|█████▎ | 3976/7500 [2:14:37<1:58:01, 2.01s/it] 53%|█████▎ | 3977/7500 [2:14:39<1:59:21, 2.03s/it] 53%|█████▎ | 3978/7500 [2:14:41<1:57:58, 2.01s/it] 53%|█████▎ | 3979/7500 [2:14:43<1:59:09, 2.03s/it] 53%|█████▎ | 3980/7500 [2:14:46<2:00:35, 2.06s/it] {'loss': 0.361, 'grad_norm': 5.707002639770508, 'learning_rate': 2.6685871250056634e-05, 'logqs/rejected': -2.6108345985412598, 'logqs/chosen': -0.8607810139656067, 'logqs_logps/rejected': -0.02287008985877037, 'logps_logqs/chosen': 0.21050700545310974, 'epoch': 1.06} 53%|█████▎ | 3980/7500 [2:14:46<2:00:35, 2.06s/it] 53%|█████▎ | 3981/7500 [2:14:48<1:59:42, 2.04s/it] 53%|█████▎ | 3982/7500 [2:14:50<1:58:21, 2.02s/it] 53%|█████▎ | 3983/7500 [2:14:52<2:00:04, 2.05s/it] 53%|█████▎ | 3984/7500 [2:14:54<1:59:41, 2.04s/it] 53%|█████▎ | 3985/7500 [2:14:56<1:58:40, 2.03s/it] 53%|█████▎ | 3986/7500 [2:14:58<1:59:40, 2.04s/it] 53%|█████▎ | 3987/7500 [2:15:00<2:00:00, 2.05s/it] 53%|█████▎ | 3988/7500 [2:15:02<1:59:02, 2.03s/it] 53%|█████▎ | 3989/7500 [2:15:04<2:00:14, 2.05s/it] 53%|█████▎ | 3990/7500 [2:15:06<1:59:22, 2.04s/it] {'loss': 0.3813, 'grad_norm': 2.782740831375122, 'learning_rate': 2.656976298823284e-05, 'logqs/rejected': -2.7851169109344482, 'logqs/chosen': -0.840066134929657, 'logqs_logps/rejected': -0.02101622335612774, 'logps_logqs/chosen': 0.20960919559001923, 'epoch': 1.06} 53%|█████▎ | 3990/7500 [2:15:06<1:59:22, 2.04s/it] 53%|█████▎ | 3991/7500 [2:15:08<1:58:29, 2.03s/it] 53%|█████▎ | 3992/7500 [2:15:10<1:57:18, 2.01s/it] 53%|█████▎ | 3993/7500 [2:15:12<1:58:07, 2.02s/it] 53%|█████▎ | 3994/7500 [2:15:14<1:58:44, 2.03s/it] 53%|█████▎ | 3995/7500 [2:15:16<1:57:10, 2.01s/it] 53%|█████▎ | 3996/7500 [2:15:18<1:56:29, 1.99s/it] 53%|█████▎ | 3997/7500 [2:15:20<1:56:50, 2.00s/it] 53%|█████▎ | 3998/7500 [2:15:22<1:55:43, 1.98s/it] 53%|█████▎ | 3999/7500 [2:15:24<1:56:52, 2.00s/it] 53%|█████▎ | 4000/7500 [2:15:26<1:59:05, 2.04s/it] {'loss': 0.3845, 'grad_norm': 3.190962791442871, 'learning_rate': 2.6453620722761896e-05, 'logqs/rejected': -2.662350654602051, 'logqs/chosen': -0.8507592082023621, 'logqs_logps/rejected': -0.018914369866251945, 'logps_logqs/chosen': 0.2086441069841385, 'epoch': 1.07} 53%|█████▎ | 4000/7500 [2:15:26<1:59:05, 2.04s/it] 53%|█████▎ | 4001/7500 [2:15:28<2:00:26, 2.07s/it] 53%|█████▎ | 4002/7500 [2:15:30<1:59:39, 2.05s/it] 53%|█████▎ | 4003/7500 [2:15:32<1:58:47, 2.04s/it] 53%|█████▎ | 4004/7500 [2:15:34<1:59:24, 2.05s/it] 53%|█████▎ | 4005/7500 [2:15:36<1:59:33, 2.05s/it] 53%|█████▎ | 4006/7500 [2:15:38<1:59:04, 2.04s/it] 53%|█████▎ | 4007/7500 [2:15:40<1:58:02, 2.03s/it] 53%|█████▎ | 4008/7500 [2:15:42<1:58:16, 2.03s/it] 53%|█████▎ | 4009/7500 [2:15:44<1:58:27, 2.04s/it] 53%|█████▎ | 4010/7500 [2:15:46<1:58:59, 2.05s/it] {'loss': 0.3609, 'grad_norm': 2.883150815963745, 'learning_rate': 2.633744696947623e-05, 'logqs/rejected': -2.570610523223877, 'logqs/chosen': -0.8628002405166626, 'logqs_logps/rejected': -0.026743415743112564, 'logps_logqs/chosen': 0.2046932429075241, 'epoch': 1.07} 53%|█████▎ | 4010/7500 [2:15:47<1:58:59, 2.05s/it] 53%|█████▎ | 4011/7500 [2:15:49<2:00:02, 2.06s/it] 53%|█████▎ | 4012/7500 [2:15:51<1:59:09, 2.05s/it] 54%|█████▎ | 4013/7500 [2:15:53<1:58:39, 2.04s/it] 54%|█████▎ | 4014/7500 [2:15:55<1:59:48, 2.06s/it] 54%|█████▎ | 4015/7500 [2:15:57<2:01:23, 2.09s/it] 54%|█████▎ | 4016/7500 [2:15:59<2:01:28, 2.09s/it] 54%|█████▎ | 4017/7500 [2:16:01<2:00:20, 2.07s/it] 54%|█████▎ | 4018/7500 [2:16:03<1:58:09, 2.04s/it] 54%|█████▎ | 4019/7500 [2:16:05<1:58:39, 2.05s/it] 54%|█████▎ | 4020/7500 [2:16:07<1:59:20, 2.06s/it] {'loss': 0.3645, 'grad_norm': 2.4731409549713135, 'learning_rate': 2.6221244244890336e-05, 'logqs/rejected': -2.5094587802886963, 'logqs/chosen': -0.8715581893920898, 'logqs_logps/rejected': -0.029337015002965927, 'logps_logqs/chosen': 0.21219225227832794, 'epoch': 1.07} 54%|█████▎ | 4020/7500 [2:16:07<1:59:20, 2.06s/it] 54%|█████▎ | 4021/7500 [2:16:09<1:58:28, 2.04s/it] 54%|█████▎ | 4022/7500 [2:16:11<1:57:05, 2.02s/it] 54%|█████▎ | 4023/7500 [2:16:13<1:56:22, 2.01s/it] 54%|█████▎ | 4024/7500 [2:16:15<1:57:14, 2.02s/it] 54%|█████▎ | 4025/7500 [2:16:17<1:57:26, 2.03s/it] 54%|█████▎ | 4026/7500 [2:16:19<1:59:37, 2.07s/it] 54%|█████▎ | 4027/7500 [2:16:21<2:00:10, 2.08s/it] 54%|█████▎ | 4028/7500 [2:16:23<1:59:56, 2.07s/it] 54%|█████▎ | 4029/7500 [2:16:26<2:00:21, 2.08s/it] 54%|█████▎ | 4030/7500 [2:16:28<2:00:07, 2.08s/it] {'loss': 0.3581, 'grad_norm': 2.875173568725586, 'learning_rate': 2.6105015066146266e-05, 'logqs/rejected': -2.4210667610168457, 'logqs/chosen': -0.8440338969230652, 'logqs_logps/rejected': -0.030177485197782516, 'logps_logqs/chosen': 0.20370249450206757, 'epoch': 1.07} 54%|█████▎ | 4030/7500 [2:16:28<2:00:07, 2.08s/it] 54%|█████▎ | 4031/7500 [2:16:30<2:01:16, 2.10s/it] 54%|█████▍ | 4032/7500 [2:16:32<2:00:57, 2.09s/it] 54%|█████▍ | 4033/7500 [2:16:34<1:59:25, 2.07s/it] 54%|█████▍ | 4034/7500 [2:16:36<2:00:10, 2.08s/it] 54%|█████▍ | 4035/7500 [2:16:38<2:00:34, 2.09s/it] 54%|█████▍ | 4036/7500 [2:16:40<1:59:46, 2.07s/it] 54%|█████▍ | 4037/7500 [2:16:42<1:58:23, 2.05s/it] 54%|█████▍ | 4038/7500 [2:16:44<1:58:05, 2.05s/it] 54%|█████▍ | 4039/7500 [2:16:46<1:57:43, 2.04s/it] 54%|█████▍ | 4040/7500 [2:16:48<1:59:47, 2.08s/it] {'loss': 0.3612, 'grad_norm': 2.993206739425659, 'learning_rate': 2.5988761950959134e-05, 'logqs/rejected': -2.487520933151245, 'logqs/chosen': -0.8566810488700867, 'logqs_logps/rejected': -0.028558794409036636, 'logps_logqs/chosen': 0.2072642594575882, 'epoch': 1.08} 54%|█████▍ | 4040/7500 [2:16:48<1:59:47, 2.08s/it] 54%|█████▍ | 4041/7500 [2:16:50<1:57:21, 2.04s/it] 54%|█████▍ | 4042/7500 [2:16:52<1:55:58, 2.01s/it] 54%|█████▍ | 4043/7500 [2:16:54<1:57:41, 2.04s/it] 54%|█████▍ | 4044/7500 [2:16:56<1:58:48, 2.06s/it] 54%|█████▍ | 4045/7500 [2:16:59<1:58:56, 2.07s/it] 54%|█████▍ | 4046/7500 [2:17:01<1:59:02, 2.07s/it] 54%|█████▍ | 4047/7500 [2:17:03<1:58:58, 2.07s/it] 54%|█████▍ | 4048/7500 [2:17:05<1:59:42, 2.08s/it] 54%|█████▍ | 4049/7500 [2:17:07<1:59:35, 2.08s/it] 54%|█████▍ | 4050/7500 [2:17:09<1:59:29, 2.08s/it] {'loss': 0.3427, 'grad_norm': 2.916696786880493, 'learning_rate': 2.587248741756253e-05, 'logqs/rejected': -2.491004467010498, 'logqs/chosen': -0.850887656211853, 'logqs_logps/rejected': -0.03319762274622917, 'logps_logqs/chosen': 0.21086671948432922, 'epoch': 1.08} 54%|█████▍ | 4050/7500 [2:17:09<1:59:29, 2.08s/it] 54%|█████▍ | 4051/7500 [2:17:11<2:00:15, 2.09s/it] 54%|█████▍ | 4052/7500 [2:17:13<2:00:17, 2.09s/it] 54%|█████▍ | 4053/7500 [2:17:15<1:58:31, 2.06s/it] 54%|█████▍ | 4054/7500 [2:17:17<1:59:06, 2.07s/it] 54%|█████▍ | 4055/7500 [2:17:19<1:58:58, 2.07s/it] 54%|█████▍ | 4056/7500 [2:17:21<2:00:01, 2.09s/it] 54%|█████▍ | 4057/7500 [2:17:24<2:01:05, 2.11s/it] 54%|█████▍ | 4058/7500 [2:17:26<2:01:11, 2.11s/it] 54%|█████▍ | 4059/7500 [2:17:28<2:01:09, 2.11s/it] 54%|█████▍ | 4060/7500 [2:17:30<2:00:10, 2.10s/it] {'loss': 0.3645, 'grad_norm': 2.845365285873413, 'learning_rate': 2.575619398465402e-05, 'logqs/rejected': -2.4312045574188232, 'logqs/chosen': -0.8579082489013672, 'logqs_logps/rejected': -0.02329608052968979, 'logps_logqs/chosen': 0.20758327841758728, 'epoch': 1.08} 54%|█████▍ | 4060/7500 [2:17:30<2:00:10, 2.10s/it] 54%|█████▍ | 4061/7500 [2:17:32<2:00:05, 2.10s/it] 54%|█████▍ | 4062/7500 [2:17:34<1:58:37, 2.07s/it] 54%|█████▍ | 4063/7500 [2:17:36<1:58:02, 2.06s/it] 54%|█████▍ | 4064/7500 [2:17:38<1:57:40, 2.05s/it] 54%|█████▍ | 4065/7500 [2:17:40<1:57:47, 2.06s/it] 54%|█████▍ | 4066/7500 [2:17:42<1:57:42, 2.06s/it] 54%|█████▍ | 4067/7500 [2:17:44<1:58:08, 2.06s/it] 54%|█████▍ | 4068/7500 [2:17:46<1:58:48, 2.08s/it] 54%|█████▍ | 4069/7500 [2:17:49<1:58:58, 2.08s/it] 54%|█████▍ | 4070/7500 [2:17:51<1:59:09, 2.08s/it] {'loss': 0.3661, 'grad_norm': 2.9155449867248535, 'learning_rate': 2.5639884171340557e-05, 'logqs/rejected': -2.517112970352173, 'logqs/chosen': -0.8682069778442383, 'logqs_logps/rejected': -0.025507360696792603, 'logps_logqs/chosen': 0.21335406601428986, 'epoch': 1.09} 54%|█████▍ | 4070/7500 [2:17:51<1:59:09, 2.08s/it] 54%|█████▍ | 4071/7500 [2:17:53<1:58:26, 2.07s/it] 54%|█████▍ | 4072/7500 [2:17:55<1:59:37, 2.09s/it] 54%|█████▍ | 4073/7500 [2:17:57<1:59:04, 2.08s/it] 54%|█████▍ | 4074/7500 [2:17:59<1:58:41, 2.08s/it] 54%|█████▍ | 4075/7500 [2:18:01<1:57:46, 2.06s/it] 54%|█████▍ | 4076/7500 [2:18:03<1:58:53, 2.08s/it] 54%|█████▍ | 4077/7500 [2:18:05<1:59:00, 2.09s/it] 54%|█████▍ | 4078/7500 [2:18:07<1:58:04, 2.07s/it] 54%|█████▍ | 4079/7500 [2:18:09<1:57:57, 2.07s/it] 54%|█████▍ | 4080/7500 [2:18:11<1:57:49, 2.07s/it] {'loss': 0.3904, 'grad_norm': 2.9903130531311035, 'learning_rate': 2.5523560497083926e-05, 'logqs/rejected': -2.734626293182373, 'logqs/chosen': -0.8740688562393188, 'logqs_logps/rejected': -0.01822221465408802, 'logps_logqs/chosen': 0.20803864300251007, 'epoch': 1.09} 54%|█████▍ | 4080/7500 [2:18:11<1:57:49, 2.07s/it] 54%|█████▍ | 4081/7500 [2:18:13<1:58:46, 2.08s/it] 54%|█████▍ | 4082/7500 [2:18:16<1:58:08, 2.07s/it] 54%|█████▍ | 4083/7500 [2:18:17<1:55:39, 2.03s/it] 54%|█████▍ | 4084/7500 [2:18:19<1:55:48, 2.03s/it] 54%|█████▍ | 4085/7500 [2:18:22<1:56:26, 2.05s/it] 54%|█████▍ | 4086/7500 [2:18:24<1:57:19, 2.06s/it] 54%|█████▍ | 4087/7500 [2:18:26<1:56:07, 2.04s/it] 55%|█████▍ | 4088/7500 [2:18:28<1:56:48, 2.05s/it] 55%|█████▍ | 4089/7500 [2:18:30<1:56:07, 2.04s/it] 55%|█████▍ | 4090/7500 [2:18:32<1:55:09, 2.03s/it] {'loss': 0.3609, 'grad_norm': 2.628713846206665, 'learning_rate': 2.5407225481646147e-05, 'logqs/rejected': -2.4777095317840576, 'logqs/chosen': -0.8267927169799805, 'logqs_logps/rejected': -0.03412676602602005, 'logps_logqs/chosen': 0.20014849305152893, 'epoch': 1.09} 55%|█████▍ | 4090/7500 [2:18:32<1:55:09, 2.03s/it] 55%|█████▍ | 4091/7500 [2:18:34<1:56:54, 2.06s/it] 55%|█████▍ | 4092/7500 [2:18:36<1:57:19, 2.07s/it] 55%|█████▍ | 4093/7500 [2:18:38<1:56:30, 2.05s/it] 55%|█████▍ | 4094/7500 [2:18:40<1:56:57, 2.06s/it] 55%|█████▍ | 4095/7500 [2:18:42<1:57:02, 2.06s/it] 55%|█████▍ | 4096/7500 [2:18:44<1:57:40, 2.07s/it] 55%|█████▍ | 4097/7500 [2:18:46<1:57:38, 2.07s/it] 55%|█████▍ | 4098/7500 [2:18:48<1:56:17, 2.05s/it] 55%|█████▍ | 4099/7500 [2:18:50<1:56:59, 2.06s/it] 55%|█████▍ | 4100/7500 [2:18:52<1:55:36, 2.04s/it] {'loss': 0.356, 'grad_norm': 2.673854351043701, 'learning_rate': 2.5290881645034932e-05, 'logqs/rejected': -2.627275228500366, 'logqs/chosen': -0.8410700559616089, 'logqs_logps/rejected': -0.031404756009578705, 'logps_logqs/chosen': 0.20714505016803741, 'epoch': 1.09} 55%|█████▍ | 4100/7500 [2:18:52<1:55:36, 2.04s/it] 55%|█████▍ | 4101/7500 [2:18:54<1:56:10, 2.05s/it] 55%|█████▍ | 4102/7500 [2:18:57<1:57:08, 2.07s/it] 55%|█████▍ | 4103/7500 [2:18:59<1:57:07, 2.07s/it] 55%|█████▍ | 4104/7500 [2:19:01<1:57:26, 2.07s/it] 55%|█████▍ | 4105/7500 [2:19:03<1:57:09, 2.07s/it] 55%|█████▍ | 4106/7500 [2:19:05<1:58:01, 2.09s/it] 55%|█████▍ | 4107/7500 [2:19:07<1:57:07, 2.07s/it] 55%|█████▍ | 4108/7500 [2:19:09<1:57:53, 2.09s/it] 55%|█████▍ | 4109/7500 [2:19:11<1:57:04, 2.07s/it] 55%|█████▍ | 4110/7500 [2:19:13<1:55:27, 2.04s/it] {'loss': 0.3613, 'grad_norm': 2.6553804874420166, 'learning_rate': 2.517453150744904e-05, 'logqs/rejected': -2.445103168487549, 'logqs/chosen': -0.8547525405883789, 'logqs_logps/rejected': -0.023164546117186546, 'logps_logqs/chosen': 0.20959539711475372, 'epoch': 1.1} 55%|█████▍ | 4110/7500 [2:19:13<1:55:27, 2.04s/it] 55%|█████▍ | 4111/7500 [2:19:15<1:55:15, 2.04s/it] 55%|█████▍ | 4112/7500 [2:19:17<1:55:05, 2.04s/it] 55%|█████▍ | 4113/7500 [2:19:19<1:57:08, 2.08s/it] 55%|█████▍ | 4114/7500 [2:19:21<1:57:11, 2.08s/it] 55%|█████▍ | 4115/7500 [2:19:24<1:58:16, 2.10s/it] 55%|█████▍ | 4116/7500 [2:19:25<1:56:15, 2.06s/it] 55%|█████▍ | 4117/7500 [2:19:28<1:55:36, 2.05s/it] 55%|█████▍ | 4118/7500 [2:19:30<1:54:43, 2.04s/it] 55%|█████▍ | 4119/7500 [2:19:32<1:54:34, 2.03s/it] 55%|█████▍ | 4120/7500 [2:19:34<1:54:03, 2.02s/it] {'loss': 0.3896, 'grad_norm': 2.702571153640747, 'learning_rate': 2.5058177589223762e-05, 'logqs/rejected': -2.6693408489227295, 'logqs/chosen': -0.8456821441650391, 'logqs_logps/rejected': -0.019039150327444077, 'logps_logqs/chosen': 0.20820410549640656, 'epoch': 1.1} 55%|█████▍ | 4120/7500 [2:19:34<1:54:03, 2.02s/it] 55%|█████▍ | 4121/7500 [2:19:36<1:55:00, 2.04s/it] 55%|█████▍ | 4122/7500 [2:19:38<1:57:05, 2.08s/it] 55%|█████▍ | 4123/7500 [2:19:40<1:55:13, 2.05s/it] 55%|█████▍ | 4124/7500 [2:19:42<1:54:44, 2.04s/it] 55%|█████▌ | 4125/7500 [2:19:44<1:53:41, 2.02s/it] 55%|█████▌ | 4126/7500 [2:19:46<1:54:31, 2.04s/it] 55%|█████▌ | 4127/7500 [2:19:48<1:54:30, 2.04s/it] 55%|█████▌ | 4128/7500 [2:19:50<1:55:45, 2.06s/it] 55%|█████▌ | 4129/7500 [2:19:52<1:55:26, 2.05s/it] 55%|█████▌ | 4130/7500 [2:19:54<1:54:58, 2.05s/it] {'loss': 0.36, 'grad_norm': 2.295849561691284, 'learning_rate': 2.4941822410776247e-05, 'logqs/rejected': -2.422525644302368, 'logqs/chosen': -0.8498603105545044, 'logqs_logps/rejected': -0.02487950399518013, 'logps_logqs/chosen': 0.20281198620796204, 'epoch': 1.1} 55%|█████▌ | 4130/7500 [2:19:54<1:54:58, 2.05s/it] 55%|█████▌ | 4131/7500 [2:19:56<1:54:15, 2.03s/it] 55%|█████▌ | 4132/7500 [2:19:58<1:53:03, 2.01s/it] 55%|█████▌ | 4133/7500 [2:20:00<1:56:28, 2.08s/it] 55%|█████▌ | 4134/7500 [2:20:02<1:55:40, 2.06s/it] 55%|█████▌ | 4135/7500 [2:20:04<1:56:52, 2.08s/it] 55%|█████▌ | 4136/7500 [2:20:07<1:56:51, 2.08s/it] 55%|█████▌ | 4137/7500 [2:20:09<1:55:58, 2.07s/it] 55%|█████▌ | 4138/7500 [2:20:11<1:56:01, 2.07s/it] 55%|█████▌ | 4139/7500 [2:20:13<1:57:15, 2.09s/it] 55%|█████▌ | 4140/7500 [2:20:15<1:57:20, 2.10s/it] {'loss': 0.3429, 'grad_norm': 3.321754217147827, 'learning_rate': 2.4825468492550964e-05, 'logqs/rejected': -2.3669514656066895, 'logqs/chosen': -0.8431553840637207, 'logqs_logps/rejected': -0.03141530230641365, 'logps_logqs/chosen': 0.20586314797401428, 'epoch': 1.1} 55%|█████▌ | 4140/7500 [2:20:15<1:57:20, 2.10s/it] 55%|█████▌ | 4141/7500 [2:20:17<1:56:22, 2.08s/it] 55%|█████▌ | 4142/7500 [2:20:19<1:55:18, 2.06s/it] 55%|█████▌ | 4143/7500 [2:20:21<1:53:55, 2.04s/it] 55%|█████▌ | 4144/7500 [2:20:23<1:53:45, 2.03s/it] 55%|█████▌ | 4145/7500 [2:20:25<1:54:49, 2.05s/it] 55%|█████▌ | 4146/7500 [2:20:27<1:55:12, 2.06s/it] 55%|█████▌ | 4147/7500 [2:20:29<1:55:09, 2.06s/it] 55%|█████▌ | 4148/7500 [2:20:31<1:55:06, 2.06s/it] 55%|█████▌ | 4149/7500 [2:20:33<1:54:46, 2.06s/it] 55%|█████▌ | 4150/7500 [2:20:35<1:54:26, 2.05s/it] {'loss': 0.3519, 'grad_norm': 2.9446637630462646, 'learning_rate': 2.470911835496508e-05, 'logqs/rejected': -2.282219409942627, 'logqs/chosen': -0.8478663563728333, 'logqs_logps/rejected': -0.026678001508116722, 'logps_logqs/chosen': 0.20605453848838806, 'epoch': 1.11} 55%|█████▌ | 4150/7500 [2:20:35<1:54:26, 2.05s/it] 55%|█████▌ | 4151/7500 [2:20:37<1:55:28, 2.07s/it] 55%|█████▌ | 4152/7500 [2:20:39<1:55:18, 2.07s/it] 55%|█████▌ | 4153/7500 [2:20:41<1:54:11, 2.05s/it] 55%|█████▌ | 4154/7500 [2:20:44<1:54:38, 2.06s/it] 55%|█████▌ | 4155/7500 [2:20:46<1:55:47, 2.08s/it] 55%|█████▌ | 4156/7500 [2:20:48<1:55:14, 2.07s/it] 55%|█████▌ | 4157/7500 [2:20:50<1:54:29, 2.05s/it] 55%|█████▌ | 4158/7500 [2:20:52<1:55:00, 2.06s/it] 55%|█████▌ | 4159/7500 [2:20:54<1:56:02, 2.08s/it] 55%|█████▌ | 4160/7500 [2:20:56<1:55:28, 2.07s/it] {'loss': 0.357, 'grad_norm': 2.5828866958618164, 'learning_rate': 2.4592774518353855e-05, 'logqs/rejected': -2.5369508266448975, 'logqs/chosen': -0.8382646441459656, 'logqs_logps/rejected': -0.022074002772569656, 'logps_logqs/chosen': 0.19247713685035706, 'epoch': 1.11} 55%|█████▌ | 4160/7500 [2:20:56<1:55:28, 2.07s/it] 55%|█████▌ | 4161/7500 [2:20:58<1:56:14, 2.09s/it] 55%|█████▌ | 4162/7500 [2:21:00<1:55:15, 2.07s/it] 56%|█████▌ | 4163/7500 [2:21:02<1:54:33, 2.06s/it] 56%|█████▌ | 4164/7500 [2:21:04<1:52:51, 2.03s/it] 56%|█████▌ | 4165/7500 [2:21:06<1:51:59, 2.01s/it] 56%|█████▌ | 4166/7500 [2:21:08<1:51:41, 2.01s/it] 56%|█████▌ | 4167/7500 [2:21:10<1:52:49, 2.03s/it] 56%|█████▌ | 4168/7500 [2:21:12<1:53:05, 2.04s/it] 56%|█████▌ | 4169/7500 [2:21:14<1:52:53, 2.03s/it] 56%|█████▌ | 4170/7500 [2:21:16<1:53:43, 2.05s/it] {'loss': 0.3543, 'grad_norm': 3.488054037094116, 'learning_rate': 2.447643950291608e-05, 'logqs/rejected': -2.362257719039917, 'logqs/chosen': -0.8560121655464172, 'logqs_logps/rejected': -0.032238077372312546, 'logps_logqs/chosen': 0.20311732590198517, 'epoch': 1.11} 56%|█████▌ | 4170/7500 [2:21:16<1:53:43, 2.05s/it] 56%|█████▌ | 4171/7500 [2:21:19<1:54:43, 2.07s/it] 56%|█████▌ | 4172/7500 [2:21:21<1:54:31, 2.06s/it] 56%|█████▌ | 4173/7500 [2:21:23<1:52:54, 2.04s/it] 56%|█████▌ | 4174/7500 [2:21:25<1:57:40, 2.12s/it] 56%|█████▌ | 4175/7500 [2:21:27<1:54:57, 2.07s/it] 56%|█████▌ | 4176/7500 [2:21:29<1:55:05, 2.08s/it] 56%|█████▌ | 4177/7500 [2:21:31<1:56:19, 2.10s/it] 56%|█████▌ | 4178/7500 [2:21:33<1:55:03, 2.08s/it] 56%|█████▌ | 4179/7500 [2:21:35<1:55:57, 2.10s/it] 56%|█████▌ | 4180/7500 [2:21:37<1:55:58, 2.10s/it] {'loss': 0.3762, 'grad_norm': 2.5100884437561035, 'learning_rate': 2.436011582865945e-05, 'logqs/rejected': -2.599730968475342, 'logqs/chosen': -0.8692420721054077, 'logqs_logps/rejected': -0.020131399855017662, 'logps_logqs/chosen': 0.20861807465553284, 'epoch': 1.11} 56%|█████▌ | 4180/7500 [2:21:37<1:55:58, 2.10s/it] 56%|█████▌ | 4181/7500 [2:21:39<1:56:35, 2.11s/it] 56%|█████▌ | 4182/7500 [2:21:42<1:55:34, 2.09s/it] 56%|█████▌ | 4183/7500 [2:21:44<1:55:14, 2.08s/it] 56%|█████▌ | 4184/7500 [2:21:46<1:53:38, 2.06s/it] 56%|█████▌ | 4185/7500 [2:21:48<1:54:43, 2.08s/it] 56%|█████▌ | 4186/7500 [2:21:50<1:56:30, 2.11s/it] 56%|█████▌ | 4187/7500 [2:21:52<1:56:16, 2.11s/it] 56%|█████▌ | 4188/7500 [2:21:54<1:53:51, 2.06s/it] 56%|█████▌ | 4189/7500 [2:21:56<1:53:46, 2.06s/it] 56%|█████▌ | 4190/7500 [2:21:58<1:52:53, 2.05s/it] {'loss': 0.3566, 'grad_norm': 2.8051578998565674, 'learning_rate': 2.4243806015345984e-05, 'logqs/rejected': -2.710526943206787, 'logqs/chosen': -0.8392660021781921, 'logqs_logps/rejected': -0.02790920063853264, 'logps_logqs/chosen': 0.2059510201215744, 'epoch': 1.12} 56%|█████▌ | 4190/7500 [2:21:58<1:52:53, 2.05s/it] 56%|█████▌ | 4191/7500 [2:22:00<1:53:36, 2.06s/it] 56%|█████▌ | 4192/7500 [2:22:02<1:54:51, 2.08s/it] 56%|█████▌ | 4193/7500 [2:22:04<1:54:57, 2.09s/it] 56%|█████▌ | 4194/7500 [2:22:06<1:54:32, 2.08s/it] 56%|█████▌ | 4195/7500 [2:22:09<1:55:18, 2.09s/it] 56%|█████▌ | 4196/7500 [2:22:11<1:54:24, 2.08s/it] 56%|█████▌ | 4197/7500 [2:22:13<1:53:56, 2.07s/it] 56%|█████▌ | 4198/7500 [2:22:15<1:52:48, 2.05s/it] 56%|█████▌ | 4199/7500 [2:22:17<1:53:24, 2.06s/it] 56%|█████▌ | 4200/7500 [2:22:19<1:55:38, 2.10s/it] {'loss': 0.3468, 'grad_norm': 4.039936065673828, 'learning_rate': 2.4127512582437485e-05, 'logqs/rejected': -2.3544516563415527, 'logqs/chosen': -0.8604367971420288, 'logqs_logps/rejected': -0.030704518780112267, 'logps_logqs/chosen': 0.20466163754463196, 'epoch': 1.12} 56%|█████▌ | 4200/7500 [2:22:19<1:55:38, 2.10s/it] 56%|█████▌ | 4201/7500 [2:22:21<1:54:07, 2.08s/it] 56%|█████▌ | 4202/7500 [2:22:23<1:53:32, 2.07s/it] 56%|█████▌ | 4203/7500 [2:22:25<1:53:36, 2.07s/it] 56%|█████▌ | 4204/7500 [2:22:27<1:53:02, 2.06s/it] 56%|█████▌ | 4205/7500 [2:22:29<1:54:26, 2.08s/it] 56%|█████▌ | 4206/7500 [2:22:31<1:55:06, 2.10s/it] 56%|█████▌ | 4207/7500 [2:22:33<1:55:05, 2.10s/it] 56%|█████▌ | 4208/7500 [2:22:36<1:56:37, 2.13s/it] 56%|█████▌ | 4209/7500 [2:22:38<1:54:43, 2.09s/it] 56%|█████▌ | 4210/7500 [2:22:40<1:53:53, 2.08s/it] {'loss': 0.3415, 'grad_norm': 2.8107306957244873, 'learning_rate': 2.4011238049040875e-05, 'logqs/rejected': -2.2819089889526367, 'logqs/chosen': -0.8180556297302246, 'logqs_logps/rejected': -0.033988066017627716, 'logps_logqs/chosen': 0.20351238548755646, 'epoch': 1.12} 56%|█████▌ | 4210/7500 [2:22:40<1:53:53, 2.08s/it] 56%|█████▌ | 4211/7500 [2:22:42<1:53:45, 2.08s/it] 56%|█████▌ | 4212/7500 [2:22:44<1:53:44, 2.08s/it] 56%|█████▌ | 4213/7500 [2:22:46<1:52:17, 2.05s/it] 56%|█████▌ | 4214/7500 [2:22:48<1:52:07, 2.05s/it] 56%|█████▌ | 4215/7500 [2:22:50<1:53:09, 2.07s/it] 56%|█████▌ | 4216/7500 [2:22:52<1:53:07, 2.07s/it] 56%|█████▌ | 4217/7500 [2:22:54<1:52:12, 2.05s/it] 56%|█████▌ | 4218/7500 [2:22:56<1:50:40, 2.02s/it] 56%|█████▋ | 4219/7500 [2:22:58<1:51:45, 2.04s/it] 56%|█████▋ | 4220/7500 [2:23:00<1:53:31, 2.08s/it] {'loss': 0.3644, 'grad_norm': 2.6679794788360596, 'learning_rate': 2.3894984933853733e-05, 'logqs/rejected': -2.4556376934051514, 'logqs/chosen': -0.8515406847000122, 'logqs_logps/rejected': -0.027990704402327538, 'logps_logqs/chosen': 0.2032768279314041, 'epoch': 1.13} 56%|█████▋ | 4220/7500 [2:23:00<1:53:31, 2.08s/it] 56%|█████▋ | 4221/7500 [2:23:02<1:53:37, 2.08s/it] 56%|█████▋ | 4222/7500 [2:23:04<1:52:43, 2.06s/it] 56%|█████▋ | 4223/7500 [2:23:06<1:52:21, 2.06s/it] 56%|█████▋ | 4224/7500 [2:23:08<1:52:54, 2.07s/it] 56%|█████▋ | 4225/7500 [2:23:11<1:53:27, 2.08s/it] 56%|█████▋ | 4226/7500 [2:23:13<1:50:31, 2.03s/it] 56%|█████▋ | 4227/7500 [2:23:15<1:51:06, 2.04s/it] 56%|█████▋ | 4228/7500 [2:23:17<1:52:03, 2.05s/it] 56%|█████▋ | 4229/7500 [2:23:19<1:53:08, 2.08s/it] 56%|█████▋ | 4230/7500 [2:23:21<1:51:27, 2.05s/it] {'loss': 0.3776, 'grad_norm': 2.631009817123413, 'learning_rate': 2.377875575510967e-05, 'logqs/rejected': -2.750755786895752, 'logqs/chosen': -0.8385472297668457, 'logqs_logps/rejected': -0.030243435874581337, 'logps_logqs/chosen': 0.20582862198352814, 'epoch': 1.13} 56%|█████▋ | 4230/7500 [2:23:21<1:51:27, 2.05s/it] 56%|█████▋ | 4231/7500 [2:23:23<1:53:08, 2.08s/it] 56%|█████▋ | 4232/7500 [2:23:25<1:54:40, 2.11s/it] 56%|█████▋ | 4233/7500 [2:23:27<1:53:57, 2.09s/it] 56%|█████▋ | 4234/7500 [2:23:29<1:53:29, 2.08s/it] 56%|█████▋ | 4235/7500 [2:23:31<1:53:12, 2.08s/it] 56%|█████▋ | 4236/7500 [2:23:33<1:52:56, 2.08s/it] 56%|█████▋ | 4237/7500 [2:23:35<1:53:18, 2.08s/it] 57%|█████▋ | 4238/7500 [2:23:38<1:53:11, 2.08s/it] 57%|█████▋ | 4239/7500 [2:23:40<1:54:18, 2.10s/it] 57%|█████▋ | 4240/7500 [2:23:42<1:53:36, 2.09s/it] {'loss': 0.3893, 'grad_norm': 2.9636993408203125, 'learning_rate': 2.366255303052377e-05, 'logqs/rejected': -2.750750780105591, 'logqs/chosen': -0.8526908159255981, 'logqs_logps/rejected': -0.020222807303071022, 'logps_logqs/chosen': 0.2153116911649704, 'epoch': 1.13} 57%|█████▋ | 4240/7500 [2:23:42<1:53:36, 2.09s/it] 57%|█████▋ | 4241/7500 [2:23:44<1:53:17, 2.09s/it] 57%|█████▋ | 4242/7500 [2:23:46<1:51:59, 2.06s/it] 57%|█████▋ | 4243/7500 [2:23:48<1:50:26, 2.03s/it] 57%|█████▋ | 4244/7500 [2:23:50<1:51:42, 2.06s/it] 57%|█████▋ | 4245/7500 [2:23:52<1:52:10, 2.07s/it] 57%|█████▋ | 4246/7500 [2:23:54<1:53:20, 2.09s/it] 57%|█████▋ | 4247/7500 [2:23:56<1:52:48, 2.08s/it] 57%|█████▋ | 4248/7500 [2:23:58<1:50:27, 2.04s/it] 57%|█████▋ | 4249/7500 [2:24:00<1:52:16, 2.07s/it] 57%|█████▋ | 4250/7500 [2:24:02<1:51:48, 2.06s/it] {'loss': 0.3542, 'grad_norm': 2.4904987812042236, 'learning_rate': 2.3546379277238107e-05, 'logqs/rejected': -2.5869805812835693, 'logqs/chosen': -0.8299957513809204, 'logqs_logps/rejected': -0.026803534477949142, 'logps_logqs/chosen': 0.2012680470943451, 'epoch': 1.13} 57%|█████▋ | 4250/7500 [2:24:02<1:51:48, 2.06s/it] 57%|█████▋ | 4251/7500 [2:24:04<1:51:18, 2.06s/it] 57%|█████▋ | 4252/7500 [2:24:06<1:50:27, 2.04s/it] 57%|█████▋ | 4253/7500 [2:24:08<1:50:39, 2.04s/it] 57%|█████▋ | 4254/7500 [2:24:11<1:51:25, 2.06s/it] 57%|█████▋ | 4255/7500 [2:24:13<1:52:57, 2.09s/it] 57%|█████▋ | 4256/7500 [2:24:15<1:53:52, 2.11s/it] 57%|█████▋ | 4257/7500 [2:24:17<1:51:16, 2.06s/it] 57%|█████▋ | 4258/7500 [2:24:19<1:51:38, 2.07s/it] 57%|█████▋ | 4259/7500 [2:24:21<1:51:53, 2.07s/it] 57%|█████▋ | 4260/7500 [2:24:23<1:50:54, 2.05s/it] {'loss': 0.349, 'grad_norm': 2.4978561401367188, 'learning_rate': 2.3430237011767167e-05, 'logqs/rejected': -2.473461151123047, 'logqs/chosen': -0.8309664726257324, 'logqs_logps/rejected': -0.02328023687005043, 'logps_logqs/chosen': 0.20363450050354004, 'epoch': 1.14} 57%|█████▋ | 4260/7500 [2:24:23<1:50:54, 2.05s/it] 57%|█████▋ | 4261/7500 [2:24:25<1:52:30, 2.08s/it] 57%|█████▋ | 4262/7500 [2:24:27<1:51:42, 2.07s/it] 57%|█████▋ | 4263/7500 [2:24:29<1:52:02, 2.08s/it] 57%|█████▋ | 4264/7500 [2:24:31<1:50:28, 2.05s/it] 57%|█████▋ | 4265/7500 [2:24:33<1:51:27, 2.07s/it] 57%|█████▋ | 4266/7500 [2:24:35<1:51:23, 2.07s/it] 57%|█████▋ | 4267/7500 [2:24:37<1:50:44, 2.06s/it] 57%|█████▋ | 4268/7500 [2:24:39<1:50:35, 2.05s/it] 57%|█████▋ | 4269/7500 [2:24:42<1:50:14, 2.05s/it] 57%|█████▋ | 4270/7500 [2:24:44<1:51:11, 2.07s/it] {'loss': 0.3654, 'grad_norm': 3.1942992210388184, 'learning_rate': 2.3314128749943375e-05, 'logqs/rejected': -2.4662857055664062, 'logqs/chosen': -0.8581123352050781, 'logqs_logps/rejected': -0.028675079345703125, 'logps_logqs/chosen': 0.2127944678068161, 'epoch': 1.14} 57%|█████▋ | 4270/7500 [2:24:44<1:51:11, 2.07s/it] 57%|█████▋ | 4271/7500 [2:24:46<1:52:02, 2.08s/it] 57%|█████▋ | 4272/7500 [2:24:48<1:51:36, 2.07s/it] 57%|█████▋ | 4273/7500 [2:24:50<1:50:53, 2.06s/it] 57%|█████▋ | 4274/7500 [2:24:52<1:50:25, 2.05s/it] 57%|█████▋ | 4275/7500 [2:24:54<1:50:51, 2.06s/it] 57%|█████▋ | 4276/7500 [2:24:56<1:51:00, 2.07s/it] 57%|█████▋ | 4277/7500 [2:24:58<1:50:55, 2.06s/it] 57%|█████▋ | 4278/7500 [2:25:00<1:51:11, 2.07s/it] 57%|█████▋ | 4279/7500 [2:25:02<1:50:37, 2.06s/it] 57%|█████▋ | 4280/7500 [2:25:04<1:51:15, 2.07s/it] {'loss': 0.3498, 'grad_norm': 3.552103042602539, 'learning_rate': 2.3198057006862566e-05, 'logqs/rejected': -2.3315396308898926, 'logqs/chosen': -0.8580049276351929, 'logqs_logps/rejected': -0.0316372886300087, 'logps_logqs/chosen': 0.2057034969329834, 'epoch': 1.14} 57%|█████▋ | 4280/7500 [2:25:04<1:51:15, 2.07s/it] 57%|█████▋ | 4281/7500 [2:25:06<1:50:30, 2.06s/it] 57%|█████▋ | 4282/7500 [2:25:08<1:51:04, 2.07s/it] 57%|█████▋ | 4283/7500 [2:25:11<1:51:46, 2.08s/it] 57%|█████▋ | 4284/7500 [2:25:13<1:52:46, 2.10s/it] 57%|█████▋ | 4285/7500 [2:25:15<1:50:37, 2.06s/it] 57%|█████▋ | 4286/7500 [2:25:17<1:49:14, 2.04s/it] 57%|█████▋ | 4287/7500 [2:25:19<1:48:49, 2.03s/it] 57%|█████▋ | 4288/7500 [2:25:21<1:49:33, 2.05s/it] 57%|█████▋ | 4289/7500 [2:25:23<1:49:44, 2.05s/it] 57%|█████▋ | 4290/7500 [2:25:25<1:50:12, 2.06s/it] {'loss': 0.3616, 'grad_norm': 2.89274263381958, 'learning_rate': 2.3082024296829536e-05, 'logqs/rejected': -2.5190975666046143, 'logqs/chosen': -0.8476068377494812, 'logqs_logps/rejected': -0.022162217646837234, 'logps_logqs/chosen': 0.21175961196422577, 'epoch': 1.14} 57%|█████▋ | 4290/7500 [2:25:25<1:50:12, 2.06s/it] 57%|█████▋ | 4291/7500 [2:25:27<1:50:15, 2.06s/it] 57%|█████▋ | 4292/7500 [2:25:29<1:49:38, 2.05s/it] 57%|█████▋ | 4293/7500 [2:25:31<1:49:36, 2.05s/it] 57%|█████▋ | 4294/7500 [2:25:33<1:51:30, 2.09s/it] 57%|█████▋ | 4295/7500 [2:25:35<1:51:39, 2.09s/it] 57%|█████▋ | 4296/7500 [2:25:37<1:50:43, 2.07s/it] 57%|█████▋ | 4297/7500 [2:25:39<1:48:49, 2.04s/it] 57%|█████▋ | 4298/7500 [2:25:41<1:47:54, 2.02s/it] 57%|█████▋ | 4299/7500 [2:25:43<1:49:34, 2.05s/it] 57%|█████▋ | 4300/7500 [2:25:45<1:48:18, 2.03s/it] {'loss': 0.357, 'grad_norm': 2.9313604831695557, 'learning_rate': 2.2966033133303545e-05, 'logqs/rejected': -2.5685932636260986, 'logqs/chosen': -0.8483695983886719, 'logqs_logps/rejected': -0.028376510366797447, 'logps_logqs/chosen': 0.2080857753753662, 'epoch': 1.15} 57%|█████▋ | 4300/7500 [2:25:45<1:48:18, 2.03s/it] 57%|█████▋ | 4301/7500 [2:25:48<1:50:34, 2.07s/it] 57%|█████▋ | 4302/7500 [2:25:50<1:50:53, 2.08s/it] 57%|█████▋ | 4303/7500 [2:25:52<1:51:09, 2.09s/it] 57%|█████▋ | 4304/7500 [2:25:54<1:49:49, 2.06s/it] 57%|█████▋ | 4305/7500 [2:25:56<1:50:51, 2.08s/it] 57%|█████▋ | 4306/7500 [2:25:58<1:50:05, 2.07s/it] 57%|█████▋ | 4307/7500 [2:26:00<1:49:27, 2.06s/it] 57%|█████▋ | 4308/7500 [2:26:02<1:50:27, 2.08s/it] 57%|█████▋ | 4309/7500 [2:26:04<1:51:11, 2.09s/it] 57%|█████▋ | 4310/7500 [2:26:06<1:50:39, 2.08s/it] {'loss': 0.3299, 'grad_norm': 2.8768973350524902, 'learning_rate': 2.2850086028843896e-05, 'logqs/rejected': -2.176409959793091, 'logqs/chosen': -0.862419605255127, 'logqs_logps/rejected': -0.03481452539563179, 'logps_logqs/chosen': 0.20150423049926758, 'epoch': 1.15} 57%|█████▋ | 4310/7500 [2:26:06<1:50:39, 2.08s/it] 57%|█████▋ | 4311/7500 [2:26:08<1:49:11, 2.05s/it] 57%|█████▋ | 4312/7500 [2:26:10<1:47:45, 2.03s/it] 58%|█████▊ | 4313/7500 [2:26:12<1:47:09, 2.02s/it] 58%|█████▊ | 4314/7500 [2:26:14<1:47:46, 2.03s/it] 58%|█████▊ | 4315/7500 [2:26:16<1:48:13, 2.04s/it] 58%|█████▊ | 4316/7500 [2:26:18<1:48:58, 2.05s/it] 58%|█████▊ | 4317/7500 [2:26:20<1:47:43, 2.03s/it] 58%|█████▊ | 4318/7500 [2:26:22<1:46:39, 2.01s/it] 58%|█████▊ | 4319/7500 [2:26:24<1:46:44, 2.01s/it] 58%|█████▊ | 4320/7500 [2:26:26<1:47:01, 2.02s/it] {'loss': 0.3791, 'grad_norm': 3.3856186866760254, 'learning_rate': 2.2734185495055503e-05, 'logqs/rejected': -2.8036770820617676, 'logqs/chosen': -0.8492264747619629, 'logqs_logps/rejected': -0.01985337771475315, 'logps_logqs/chosen': 0.20886778831481934, 'epoch': 1.15} 58%|█████▊ | 4320/7500 [2:26:26<1:47:01, 2.02s/it] 58%|█████▊ | 4321/7500 [2:26:28<1:46:05, 2.00s/it] 58%|█████▊ | 4322/7500 [2:26:30<1:45:47, 2.00s/it] 58%|█████▊ | 4323/7500 [2:26:32<1:47:59, 2.04s/it] 58%|█████▊ | 4324/7500 [2:26:35<1:48:59, 2.06s/it] 58%|█████▊ | 4325/7500 [2:26:37<1:48:22, 2.05s/it] 58%|█████▊ | 4326/7500 [2:26:39<1:46:56, 2.02s/it] 58%|█████▊ | 4327/7500 [2:26:41<1:47:42, 2.04s/it] 58%|█████▊ | 4328/7500 [2:26:43<1:47:12, 2.03s/it] 58%|█████▊ | 4329/7500 [2:26:45<1:48:29, 2.05s/it] 58%|█████▊ | 4330/7500 [2:26:47<1:50:18, 2.09s/it] {'loss': 0.3555, 'grad_norm': 3.0400986671447754, 'learning_rate': 2.2618334042534464e-05, 'logqs/rejected': -2.4165091514587402, 'logqs/chosen': -0.8427149057388306, 'logqs_logps/rejected': -0.023617666214704514, 'logps_logqs/chosen': 0.20640650391578674, 'epoch': 1.15} 58%|█████▊ | 4330/7500 [2:26:47<1:50:18, 2.09s/it] 58%|█████▊ | 4331/7500 [2:26:49<1:50:44, 2.10s/it] 58%|█████▊ | 4332/7500 [2:26:51<1:50:52, 2.10s/it] 58%|█████▊ | 4333/7500 [2:26:53<1:51:36, 2.11s/it] 58%|█████▊ | 4334/7500 [2:26:55<1:50:54, 2.10s/it] 58%|█████▊ | 4335/7500 [2:26:57<1:50:58, 2.10s/it] 58%|█████▊ | 4336/7500 [2:27:00<1:50:41, 2.10s/it] 58%|█████▊ | 4337/7500 [2:27:02<1:49:33, 2.08s/it] 58%|█████▊ | 4338/7500 [2:27:04<1:49:19, 2.07s/it] 58%|█████▊ | 4339/7500 [2:27:06<1:47:44, 2.04s/it] 58%|█████▊ | 4340/7500 [2:27:08<1:47:45, 2.05s/it] {'loss': 0.3531, 'grad_norm': 2.7024707794189453, 'learning_rate': 2.2502534180813728e-05, 'logqs/rejected': -2.5483641624450684, 'logqs/chosen': -0.8263174295425415, 'logqs_logps/rejected': -0.027322569862008095, 'logps_logqs/chosen': 0.2106703221797943, 'epoch': 1.16} 58%|█████▊ | 4340/7500 [2:27:08<1:47:45, 2.05s/it] 58%|█████▊ | 4341/7500 [2:27:10<1:49:17, 2.08s/it] 58%|█████▊ | 4342/7500 [2:27:12<1:49:56, 2.09s/it] 58%|█████▊ | 4343/7500 [2:27:14<1:50:12, 2.09s/it] 58%|█████▊ | 4344/7500 [2:27:16<1:50:06, 2.09s/it] 58%|█████▊ | 4345/7500 [2:27:18<1:49:26, 2.08s/it] 58%|█████▊ | 4346/7500 [2:27:20<1:48:01, 2.05s/it] 58%|█████▊ | 4347/7500 [2:27:22<1:48:14, 2.06s/it] 58%|█████▊ | 4348/7500 [2:27:24<1:47:35, 2.05s/it] 58%|█████▊ | 4349/7500 [2:27:26<1:47:07, 2.04s/it] 58%|█████▊ | 4350/7500 [2:27:28<1:47:52, 2.05s/it] {'loss': 0.3476, 'grad_norm': 2.835550308227539, 'learning_rate': 2.238678841830867e-05, 'logqs/rejected': -2.445991039276123, 'logqs/chosen': -0.8462057113647461, 'logqs_logps/rejected': -0.03237290307879448, 'logps_logqs/chosen': 0.21018430590629578, 'epoch': 1.16} 58%|█████▊ | 4350/7500 [2:27:28<1:47:52, 2.05s/it] 58%|█████▊ | 4351/7500 [2:27:30<1:47:40, 2.05s/it] 58%|█████▊ | 4352/7500 [2:27:33<1:48:21, 2.07s/it] 58%|█████▊ | 4353/7500 [2:27:35<1:47:47, 2.06s/it] 58%|█████▊ | 4354/7500 [2:27:37<1:47:24, 2.05s/it] 58%|█████▊ | 4355/7500 [2:27:39<1:47:01, 2.04s/it] 58%|█████▊ | 4356/7500 [2:27:41<1:46:39, 2.04s/it] 58%|█████▊ | 4357/7500 [2:27:43<1:46:12, 2.03s/it] 58%|█████▊ | 4358/7500 [2:27:45<1:47:12, 2.05s/it] 58%|█████▊ | 4359/7500 [2:27:47<1:45:31, 2.02s/it] 58%|█████▊ | 4360/7500 [2:27:49<1:45:58, 2.02s/it] {'loss': 0.3562, 'grad_norm': 2.624605417251587, 'learning_rate': 2.2271099262262798e-05, 'logqs/rejected': -2.5971920490264893, 'logqs/chosen': -0.8455503582954407, 'logqs_logps/rejected': -0.026847172528505325, 'logps_logqs/chosen': 0.2061176300048828, 'epoch': 1.16} 58%|█████▊ | 4360/7500 [2:27:49<1:45:58, 2.02s/it] 58%|█████▊ | 4361/7500 [2:27:51<1:45:21, 2.01s/it] 58%|█████▊ | 4362/7500 [2:27:53<1:44:48, 2.00s/it] 58%|█████▊ | 4363/7500 [2:27:55<1:46:15, 2.03s/it] 58%|█████▊ | 4364/7500 [2:27:57<1:47:00, 2.05s/it] 58%|█████▊ | 4365/7500 [2:27:59<1:48:42, 2.08s/it] 58%|█████▊ | 4366/7500 [2:28:01<1:47:38, 2.06s/it] 58%|█████▊ | 4367/7500 [2:28:03<1:47:55, 2.07s/it] 58%|█████▊ | 4368/7500 [2:28:05<1:47:52, 2.07s/it] 58%|█████▊ | 4369/7500 [2:28:07<1:47:59, 2.07s/it] 58%|█████▊ | 4370/7500 [2:28:09<1:47:56, 2.07s/it] {'loss': 0.3449, 'grad_norm': 2.415839672088623, 'learning_rate': 2.215546921869346e-05, 'logqs/rejected': -2.4781241416931152, 'logqs/chosen': -0.8527339100837708, 'logqs_logps/rejected': -0.033451974391937256, 'logps_logqs/chosen': 0.20801086723804474, 'epoch': 1.17} 58%|█████▊ | 4370/7500 [2:28:09<1:47:56, 2.07s/it] 58%|█████▊ | 4371/7500 [2:28:11<1:47:48, 2.07s/it] 58%|█████▊ | 4372/7500 [2:28:13<1:47:30, 2.06s/it] 58%|█████▊ | 4373/7500 [2:28:16<1:48:58, 2.09s/it] 58%|█████▊ | 4374/7500 [2:28:18<1:48:46, 2.09s/it] 58%|█████▊ | 4375/7500 [2:28:20<1:47:45, 2.07s/it] 58%|█████▊ | 4376/7500 [2:28:22<1:46:42, 2.05s/it] 58%|█████▊ | 4377/7500 [2:28:24<1:45:32, 2.03s/it] 58%|█████▊ | 4378/7500 [2:28:26<1:46:57, 2.06s/it] 58%|█████▊ | 4379/7500 [2:28:28<1:46:52, 2.05s/it] 58%|█████▊ | 4380/7500 [2:28:30<1:46:55, 2.06s/it] {'loss': 0.3792, 'grad_norm': 3.318103313446045, 'learning_rate': 2.2039900792337474e-05, 'logqs/rejected': -2.6390738487243652, 'logqs/chosen': -0.8761832118034363, 'logqs_logps/rejected': -0.027105208486318588, 'logps_logqs/chosen': 0.20978975296020508, 'epoch': 1.17} 58%|█████▊ | 4380/7500 [2:28:30<1:46:55, 2.06s/it] 58%|█████▊ | 4381/7500 [2:28:32<1:46:38, 2.05s/it] 58%|█████▊ | 4382/7500 [2:28:34<1:46:32, 2.05s/it] 58%|█████▊ | 4383/7500 [2:28:36<1:46:27, 2.05s/it] 58%|█████▊ | 4384/7500 [2:28:38<1:47:18, 2.07s/it] 58%|█████▊ | 4385/7500 [2:28:40<1:47:13, 2.07s/it] 58%|█████▊ | 4386/7500 [2:28:42<1:46:05, 2.04s/it] 58%|█████▊ | 4387/7500 [2:28:44<1:45:21, 2.03s/it] 59%|█████▊ | 4388/7500 [2:28:46<1:46:40, 2.06s/it] 59%|█████▊ | 4389/7500 [2:28:48<1:45:41, 2.04s/it] 59%|█████▊ | 4390/7500 [2:28:50<1:45:54, 2.04s/it] {'loss': 0.3583, 'grad_norm': 3.1058390140533447, 'learning_rate': 2.192439648659699e-05, 'logqs/rejected': -2.557248115539551, 'logqs/chosen': -0.8386688232421875, 'logqs_logps/rejected': -0.025870025157928467, 'logps_logqs/chosen': 0.2074214667081833, 'epoch': 1.17} 59%|█████▊ | 4390/7500 [2:28:50<1:45:54, 2.04s/it] 59%|█████▊ | 4391/7500 [2:28:53<1:46:44, 2.06s/it] 59%|█████▊ | 4392/7500 [2:28:55<1:48:18, 2.09s/it] 59%|█████▊ | 4393/7500 [2:28:57<1:47:17, 2.07s/it] 59%|█████▊ | 4394/7500 [2:28:59<1:46:39, 2.06s/it] 59%|█████▊ | 4395/7500 [2:29:01<1:46:08, 2.05s/it] 59%|█████▊ | 4396/7500 [2:29:03<1:45:26, 2.04s/it] 59%|█████▊ | 4397/7500 [2:29:05<1:45:54, 2.05s/it] 59%|█████▊ | 4398/7500 [2:29:07<1:46:43, 2.06s/it] 59%|█████▊ | 4399/7500 [2:29:09<1:47:42, 2.08s/it] 59%|█████▊ | 4400/7500 [2:29:11<1:48:31, 2.10s/it] {'loss': 0.3571, 'grad_norm': 2.6526145935058594, 'learning_rate': 2.1808958803485136e-05, 'logqs/rejected': -2.5251896381378174, 'logqs/chosen': -0.8471916317939758, 'logqs_logps/rejected': -0.028565606102347374, 'logps_logqs/chosen': 0.2121257781982422, 'epoch': 1.17} 59%|█████▊ | 4400/7500 [2:29:11<1:48:31, 2.10s/it] 59%|█████▊ | 4401/7500 [2:29:13<1:47:18, 2.08s/it] 59%|█████▊ | 4402/7500 [2:29:15<1:45:32, 2.04s/it] 59%|█████▊ | 4403/7500 [2:29:17<1:45:47, 2.05s/it] 59%|█████▊ | 4404/7500 [2:29:19<1:46:21, 2.06s/it] 59%|█████▊ | 4405/7500 [2:29:21<1:45:35, 2.05s/it] 59%|█████▊ | 4406/7500 [2:29:23<1:44:46, 2.03s/it] 59%|█████▉ | 4407/7500 [2:29:25<1:44:11, 2.02s/it] 59%|█████▉ | 4408/7500 [2:29:27<1:44:43, 2.03s/it] 59%|█████▉ | 4409/7500 [2:29:30<1:45:57, 2.06s/it] 59%|█████▉ | 4410/7500 [2:29:32<1:45:22, 2.05s/it] {'loss': 0.3776, 'grad_norm': 3.099666118621826, 'learning_rate': 2.1693590243571938e-05, 'logqs/rejected': -2.544349431991577, 'logqs/chosen': -0.853014349937439, 'logqs_logps/rejected': -0.027130495756864548, 'logps_logqs/chosen': 0.2103491723537445, 'epoch': 1.18} 59%|█████▉ | 4410/7500 [2:29:32<1:45:22, 2.05s/it] 59%|█████▉ | 4411/7500 [2:29:34<1:47:31, 2.09s/it] 59%|█████▉ | 4412/7500 [2:29:36<1:47:29, 2.09s/it] 59%|█████▉ | 4413/7500 [2:29:38<1:46:41, 2.07s/it] 59%|█████▉ | 4414/7500 [2:29:40<1:47:22, 2.09s/it] 59%|█████▉ | 4415/7500 [2:29:42<1:49:14, 2.12s/it] 59%|█████▉ | 4416/7500 [2:29:44<1:48:26, 2.11s/it] 59%|█████▉ | 4417/7500 [2:29:46<1:47:47, 2.10s/it] 59%|█████▉ | 4418/7500 [2:29:48<1:47:49, 2.10s/it] 59%|█████▉ | 4419/7500 [2:29:50<1:46:44, 2.08s/it] 59%|█████▉ | 4420/7500 [2:29:53<1:46:03, 2.07s/it] {'loss': 0.3563, 'grad_norm': 2.83815598487854, 'learning_rate': 2.157829330593008e-05, 'logqs/rejected': -2.543855667114258, 'logqs/chosen': -0.8360154032707214, 'logqs_logps/rejected': -0.02464757300913334, 'logps_logqs/chosen': 0.2077595740556717, 'epoch': 1.18} 59%|█████▉ | 4420/7500 [2:29:53<1:46:03, 2.07s/it] 59%|█████▉ | 4421/7500 [2:29:55<1:45:02, 2.05s/it] 59%|█████▉ | 4422/7500 [2:29:57<1:46:51, 2.08s/it] 59%|█████▉ | 4423/7500 [2:29:59<1:47:01, 2.09s/it] 59%|█████▉ | 4424/7500 [2:30:01<1:45:08, 2.05s/it] 59%|█████▉ | 4425/7500 [2:30:03<1:45:23, 2.06s/it] 59%|█████▉ | 4426/7500 [2:30:05<1:46:34, 2.08s/it] 59%|█████▉ | 4427/7500 [2:30:07<1:44:50, 2.05s/it] 59%|█████▉ | 4428/7500 [2:30:09<1:45:19, 2.06s/it] 59%|█████▉ | 4429/7500 [2:30:11<1:46:13, 2.08s/it] 59%|█████▉ | 4430/7500 [2:30:13<1:46:46, 2.09s/it] {'loss': 0.3655, 'grad_norm': 2.5162534713745117, 'learning_rate': 2.1463070488080777e-05, 'logqs/rejected': -2.5694355964660645, 'logqs/chosen': -0.8594890832901001, 'logqs_logps/rejected': -0.027891745790839195, 'logps_logqs/chosen': 0.20180204510688782, 'epoch': 1.18} 59%|█████▉ | 4430/7500 [2:30:13<1:46:46, 2.09s/it] 59%|█████▉ | 4431/7500 [2:30:15<1:46:42, 2.09s/it] 59%|█████▉ | 4432/7500 [2:30:17<1:45:35, 2.07s/it] 59%|█████▉ | 4433/7500 [2:30:19<1:45:02, 2.05s/it] 59%|█████▉ | 4434/7500 [2:30:21<1:43:30, 2.03s/it] 59%|█████▉ | 4435/7500 [2:30:23<1:44:56, 2.05s/it] 59%|█████▉ | 4436/7500 [2:30:26<1:45:30, 2.07s/it] 59%|█████▉ | 4437/7500 [2:30:28<1:45:32, 2.07s/it] 59%|█████▉ | 4438/7500 [2:30:30<1:45:08, 2.06s/it] 59%|█████▉ | 4439/7500 [2:30:32<1:45:18, 2.06s/it] 59%|█████▉ | 4440/7500 [2:30:34<1:44:22, 2.05s/it] {'loss': 0.3514, 'grad_norm': 2.926835536956787, 'learning_rate': 2.1347924285939714e-05, 'logqs/rejected': -2.366899013519287, 'logqs/chosen': -0.8647640347480774, 'logqs_logps/rejected': -0.026451457291841507, 'logps_logqs/chosen': 0.21216890215873718, 'epoch': 1.18} 59%|█████▉ | 4440/7500 [2:30:34<1:44:22, 2.05s/it] 59%|█████▉ | 4441/7500 [2:30:36<1:42:52, 2.02s/it] 59%|█████▉ | 4442/7500 [2:30:38<1:43:21, 2.03s/it] 59%|█████▉ | 4443/7500 [2:30:40<1:45:21, 2.07s/it] 59%|█████▉ | 4444/7500 [2:30:42<1:45:32, 2.07s/it] 59%|█████▉ | 4445/7500 [2:30:44<1:44:36, 2.05s/it] 59%|█████▉ | 4446/7500 [2:30:46<1:45:22, 2.07s/it] 59%|█████▉ | 4447/7500 [2:30:48<1:46:49, 2.10s/it] 59%|█████▉ | 4448/7500 [2:30:50<1:46:23, 2.09s/it] 59%|█████▉ | 4449/7500 [2:30:52<1:45:01, 2.07s/it] 59%|█████▉ | 4450/7500 [2:30:54<1:44:30, 2.06s/it] {'loss': 0.3417, 'grad_norm': 2.7801246643066406, 'learning_rate': 2.1232857193762924e-05, 'logqs/rejected': -2.408831834793091, 'logqs/chosen': -0.8418442010879517, 'logqs_logps/rejected': -0.025310304015874863, 'logps_logqs/chosen': 0.20265159010887146, 'epoch': 1.19} 59%|█████▉ | 4450/7500 [2:30:54<1:44:30, 2.06s/it] 59%|█████▉ | 4451/7500 [2:30:57<1:45:29, 2.08s/it] 59%|█████▉ | 4452/7500 [2:30:59<1:46:13, 2.09s/it] 59%|█████▉ | 4453/7500 [2:31:01<1:46:13, 2.09s/it] 59%|█████▉ | 4454/7500 [2:31:03<1:43:28, 2.04s/it] 59%|█████▉ | 4455/7500 [2:31:05<1:44:50, 2.07s/it] 59%|█████▉ | 4456/7500 [2:31:07<1:45:09, 2.07s/it] 59%|█████▉ | 4457/7500 [2:31:09<1:43:58, 2.05s/it] 59%|█████▉ | 4458/7500 [2:31:11<1:43:41, 2.05s/it] 59%|█████▉ | 4459/7500 [2:31:13<1:44:32, 2.06s/it] 59%|█████▉ | 4460/7500 [2:31:15<1:44:57, 2.07s/it] {'loss': 0.3588, 'grad_norm': 3.4913620948791504, 'learning_rate': 2.1117871704092814e-05, 'logqs/rejected': -2.3872029781341553, 'logqs/chosen': -0.8500860929489136, 'logqs_logps/rejected': -0.025613104924559593, 'logps_logqs/chosen': 0.20735454559326172, 'epoch': 1.19} 59%|█████▉ | 4460/7500 [2:31:15<1:44:57, 2.07s/it] 59%|█████▉ | 4461/7500 [2:31:17<1:46:00, 2.09s/it] 59%|█████▉ | 4462/7500 [2:31:19<1:44:51, 2.07s/it] 60%|█████▉ | 4463/7500 [2:31:21<1:44:24, 2.06s/it] 60%|█████▉ | 4464/7500 [2:31:23<1:44:00, 2.06s/it] 60%|█████▉ | 4465/7500 [2:31:25<1:44:53, 2.07s/it] 60%|█████▉ | 4466/7500 [2:31:27<1:44:17, 2.06s/it] 60%|█████▉ | 4467/7500 [2:31:30<1:44:06, 2.06s/it] 60%|█████▉ | 4468/7500 [2:31:32<1:44:50, 2.07s/it] 60%|█████▉ | 4469/7500 [2:31:34<1:44:10, 2.06s/it] 60%|█████▉ | 4470/7500 [2:31:36<1:44:06, 2.06s/it] {'loss': 0.3713, 'grad_norm': 2.9441704750061035, 'learning_rate': 2.1002970307704132e-05, 'logqs/rejected': -2.4790806770324707, 'logqs/chosen': -0.8705891370773315, 'logqs_logps/rejected': -0.022256311029195786, 'logps_logqs/chosen': 0.20303232967853546, 'epoch': 1.19} 60%|█████▉ | 4470/7500 [2:31:36<1:44:06, 2.06s/it] 60%|█████▉ | 4471/7500 [2:31:38<1:44:00, 2.06s/it] 60%|█████▉ | 4472/7500 [2:31:40<1:43:33, 2.05s/it] 60%|█████▉ | 4473/7500 [2:31:42<1:43:13, 2.05s/it] 60%|█████▉ | 4474/7500 [2:31:44<1:44:03, 2.06s/it] 60%|█████▉ | 4475/7500 [2:31:46<1:43:59, 2.06s/it] 60%|█████▉ | 4476/7500 [2:31:48<1:43:33, 2.05s/it] 60%|█████▉ | 4477/7500 [2:31:50<1:43:40, 2.06s/it] 60%|█████▉ | 4478/7500 [2:31:52<1:43:47, 2.06s/it] 60%|█████▉ | 4479/7500 [2:31:54<1:43:30, 2.06s/it] 60%|█████▉ | 4480/7500 [2:31:56<1:43:24, 2.05s/it] {'loss': 0.3606, 'grad_norm': 2.676698923110962, 'learning_rate': 2.0888155493550026e-05, 'logqs/rejected': -2.500425338745117, 'logqs/chosen': -0.8457938432693481, 'logqs_logps/rejected': -0.01982049085199833, 'logps_logqs/chosen': 0.20308467745780945, 'epoch': 1.19} 60%|█████▉ | 4480/7500 [2:31:56<1:43:24, 2.05s/it] 60%|█████▉ | 4481/7500 [2:31:58<1:43:26, 2.06s/it] 60%|█████▉ | 4482/7500 [2:32:00<1:44:21, 2.07s/it] 60%|█████▉ | 4483/7500 [2:32:02<1:42:46, 2.04s/it] 60%|█████▉ | 4484/7500 [2:32:05<1:43:19, 2.06s/it] 60%|█████▉ | 4485/7500 [2:32:07<1:42:42, 2.04s/it] 60%|█████▉ | 4486/7500 [2:32:09<1:42:30, 2.04s/it] 60%|█████▉ | 4487/7500 [2:32:11<1:41:56, 2.03s/it] 60%|█████▉ | 4488/7500 [2:32:13<1:43:06, 2.05s/it] 60%|█████▉ | 4489/7500 [2:32:15<1:42:06, 2.03s/it] 60%|█████▉ | 4490/7500 [2:32:17<1:41:55, 2.03s/it] {'loss': 0.3568, 'grad_norm': 3.1362619400024414, 'learning_rate': 2.0773429748708153e-05, 'logqs/rejected': -2.5691139698028564, 'logqs/chosen': -0.8258768916130066, 'logqs_logps/rejected': -0.028701314702630043, 'logps_logqs/chosen': 0.20271103084087372, 'epoch': 1.2} 60%|█████▉ | 4490/7500 [2:32:17<1:41:55, 2.03s/it] 60%|█████▉ | 4491/7500 [2:32:19<1:42:33, 2.05s/it] 60%|█████▉ | 4492/7500 [2:32:21<1:42:52, 2.05s/it] 60%|█████▉ | 4493/7500 [2:32:23<1:42:46, 2.05s/it] 60%|█████▉ | 4494/7500 [2:32:25<1:42:02, 2.04s/it] 60%|█████▉ | 4495/7500 [2:32:27<1:42:35, 2.05s/it] 60%|█████▉ | 4496/7500 [2:32:29<1:42:59, 2.06s/it] 60%|█████▉ | 4497/7500 [2:32:31<1:41:50, 2.03s/it] 60%|█████▉ | 4498/7500 [2:32:33<1:42:29, 2.05s/it] 60%|█████▉ | 4499/7500 [2:32:35<1:42:21, 2.05s/it] 60%|██████ | 4500/7500 [2:32:37<1:43:38, 2.07s/it] {'loss': 0.3573, 'grad_norm': 3.226891279220581, 'learning_rate': 2.0658795558326743e-05, 'logqs/rejected': -2.3630764484405518, 'logqs/chosen': -0.8788013458251953, 'logqs_logps/rejected': -0.02901347540318966, 'logps_logqs/chosen': 0.20262618362903595, 'epoch': 1.2} 60%|██████ | 4500/7500 [2:32:37<1:43:38, 2.07s/it] 60%|██████ | 4501/7500 [2:32:39<1:41:50, 2.04s/it] 60%|██████ | 4502/7500 [2:32:41<1:40:57, 2.02s/it] 60%|██████ | 4503/7500 [2:32:43<1:41:48, 2.04s/it] 60%|██████ | 4504/7500 [2:32:45<1:43:14, 2.07s/it] 60%|██████ | 4505/7500 [2:32:47<1:41:44, 2.04s/it] 60%|██████ | 4506/7500 [2:32:50<1:43:02, 2.07s/it] 60%|██████ | 4507/7500 [2:32:52<1:42:37, 2.06s/it] 60%|██████ | 4508/7500 [2:32:54<1:41:03, 2.03s/it] 60%|██████ | 4509/7500 [2:32:56<1:40:05, 2.01s/it] 60%|██████ | 4510/7500 [2:32:58<1:39:55, 2.01s/it] {'loss': 0.3458, 'grad_norm': 3.4995532035827637, 'learning_rate': 2.0544255405570843e-05, 'logqs/rejected': -2.5788097381591797, 'logqs/chosen': -0.8459285497665405, 'logqs_logps/rejected': -0.029563486576080322, 'logps_logqs/chosen': 0.20397984981536865, 'epoch': 1.2} 60%|██████ | 4510/7500 [2:32:58<1:39:55, 2.01s/it] 60%|██████ | 4511/7500 [2:33:00<1:41:49, 2.04s/it] 60%|██████ | 4512/7500 [2:33:02<1:42:42, 2.06s/it] 60%|██████ | 4513/7500 [2:33:04<1:43:09, 2.07s/it] 60%|██████ | 4514/7500 [2:33:06<1:42:28, 2.06s/it] 60%|██████ | 4515/7500 [2:33:08<1:43:20, 2.08s/it] 60%|██████ | 4516/7500 [2:33:10<1:44:05, 2.09s/it] 60%|██████ | 4517/7500 [2:33:12<1:43:14, 2.08s/it] 60%|██████ | 4518/7500 [2:33:14<1:43:01, 2.07s/it] 60%|██████ | 4519/7500 [2:33:16<1:41:58, 2.05s/it] 60%|██████ | 4520/7500 [2:33:18<1:41:27, 2.04s/it] {'loss': 0.3582, 'grad_norm': 3.3431143760681152, 'learning_rate': 2.042981177156847e-05, 'logqs/rejected': -2.478410243988037, 'logqs/chosen': -0.8305307626724243, 'logqs_logps/rejected': -0.02199743129312992, 'logps_logqs/chosen': 0.21185505390167236, 'epoch': 1.21} 60%|██████ | 4520/7500 [2:33:18<1:41:27, 2.04s/it] 60%|██████ | 4521/7500 [2:33:20<1:40:41, 2.03s/it] 60%|██████ | 4522/7500 [2:33:22<1:43:26, 2.08s/it] 60%|██████ | 4523/7500 [2:33:25<1:43:28, 2.09s/it] 60%|██████ | 4524/7500 [2:33:27<1:43:33, 2.09s/it] 60%|██████ | 4525/7500 [2:33:29<1:41:40, 2.05s/it] 60%|██████ | 4526/7500 [2:33:31<1:41:26, 2.05s/it] 60%|██████ | 4527/7500 [2:33:33<1:40:51, 2.04s/it] 60%|██████ | 4528/7500 [2:33:35<1:39:39, 2.01s/it] 60%|██████ | 4529/7500 [2:33:37<1:39:40, 2.01s/it] 60%|██████ | 4530/7500 [2:33:39<1:38:44, 1.99s/it] {'loss': 0.3528, 'grad_norm': 3.094303846359253, 'learning_rate': 2.031546713535688e-05, 'logqs/rejected': -2.4901022911071777, 'logqs/chosen': -0.8228713870048523, 'logqs_logps/rejected': -0.02482057921588421, 'logps_logqs/chosen': 0.20120441913604736, 'epoch': 1.21} 60%|██████ | 4530/7500 [2:33:39<1:38:44, 1.99s/it] 60%|██████ | 4531/7500 [2:33:41<1:39:56, 2.02s/it] 60%|██████ | 4532/7500 [2:33:43<1:40:17, 2.03s/it] 60%|██████ | 4533/7500 [2:33:45<1:41:54, 2.06s/it] 60%|██████ | 4534/7500 [2:33:47<1:41:08, 2.05s/it] 60%|██████ | 4535/7500 [2:33:49<1:40:24, 2.03s/it] 60%|██████ | 4536/7500 [2:33:51<1:42:23, 2.07s/it] 60%|██████ | 4537/7500 [2:33:53<1:40:28, 2.03s/it] 61%|██████ | 4538/7500 [2:33:55<1:39:53, 2.02s/it] 61%|██████ | 4539/7500 [2:33:57<1:41:16, 2.05s/it] 61%|██████ | 4540/7500 [2:33:59<1:42:20, 2.07s/it] {'loss': 0.3536, 'grad_norm': 2.968362808227539, 'learning_rate': 2.0201223973828916e-05, 'logqs/rejected': -2.45265531539917, 'logqs/chosen': -0.845894992351532, 'logqs_logps/rejected': -0.029519757255911827, 'logps_logqs/chosen': 0.21019594371318817, 'epoch': 1.21} 61%|██████ | 4540/7500 [2:33:59<1:42:20, 2.07s/it] 61%|██████ | 4541/7500 [2:34:01<1:42:21, 2.08s/it] 61%|██████ | 4542/7500 [2:34:03<1:41:13, 2.05s/it] 61%|██████ | 4543/7500 [2:34:05<1:40:44, 2.04s/it] 61%|██████ | 4544/7500 [2:34:07<1:39:35, 2.02s/it] 61%|██████ | 4545/7500 [2:34:09<1:39:47, 2.03s/it] 61%|██████ | 4546/7500 [2:34:11<1:39:40, 2.02s/it] 61%|██████ | 4547/7500 [2:34:13<1:41:00, 2.05s/it] 61%|██████ | 4548/7500 [2:34:15<1:40:15, 2.04s/it] 61%|██████ | 4549/7500 [2:34:18<1:41:27, 2.06s/it] 61%|██████ | 4550/7500 [2:34:20<1:41:23, 2.06s/it] {'loss': 0.3647, 'grad_norm': 3.2679598331451416, 'learning_rate': 2.0087084761679245e-05, 'logqs/rejected': -2.7649121284484863, 'logqs/chosen': -0.8164501190185547, 'logqs_logps/rejected': -0.020207490772008896, 'logps_logqs/chosen': 0.20383234322071075, 'epoch': 1.21} 61%|██████ | 4550/7500 [2:34:20<1:41:23, 2.06s/it] 61%|██████ | 4551/7500 [2:34:22<1:42:10, 2.08s/it] 61%|██████ | 4552/7500 [2:34:24<1:41:38, 2.07s/it] 61%|██████ | 4553/7500 [2:34:26<1:40:19, 2.04s/it] 61%|██████ | 4554/7500 [2:34:28<1:39:55, 2.04s/it] 61%|██████ | 4555/7500 [2:34:30<1:38:32, 2.01s/it] 61%|██████ | 4556/7500 [2:34:32<1:39:02, 2.02s/it] 61%|██████ | 4557/7500 [2:34:34<1:39:53, 2.04s/it] 61%|██████ | 4558/7500 [2:34:36<1:39:31, 2.03s/it] 61%|██████ | 4559/7500 [2:34:38<1:39:26, 2.03s/it] 61%|██████ | 4560/7500 [2:34:40<1:39:10, 2.02s/it] {'loss': 0.343, 'grad_norm': 3.0390267372131348, 'learning_rate': 1.9973051971350888e-05, 'logqs/rejected': -2.3777120113372803, 'logqs/chosen': -0.8374263644218445, 'logqs_logps/rejected': -0.03056817129254341, 'logps_logqs/chosen': 0.2040044516324997, 'epoch': 1.22} 61%|██████ | 4560/7500 [2:34:40<1:39:10, 2.02s/it] 61%|██████ | 4561/7500 [2:34:42<1:39:28, 2.03s/it] 61%|██████ | 4562/7500 [2:34:44<1:39:12, 2.03s/it] 61%|██████ | 4563/7500 [2:34:46<1:40:18, 2.05s/it] 61%|██████ | 4564/7500 [2:34:48<1:41:21, 2.07s/it] 61%|██████ | 4565/7500 [2:34:50<1:39:59, 2.04s/it] 61%|██████ | 4566/7500 [2:34:52<1:40:02, 2.05s/it] 61%|██████ | 4567/7500 [2:34:54<1:39:00, 2.03s/it] 61%|██████ | 4568/7500 [2:34:56<1:39:08, 2.03s/it] 61%|██████ | 4569/7500 [2:34:58<1:40:19, 2.05s/it] 61%|██████ | 4570/7500 [2:35:01<1:41:39, 2.08s/it] {'loss': 0.3714, 'grad_norm': 2.7564475536346436, 'learning_rate': 1.9859128072981535e-05, 'logqs/rejected': -2.55263090133667, 'logqs/chosen': -0.8624430894851685, 'logqs_logps/rejected': -0.018204031512141228, 'logps_logqs/chosen': 0.20656263828277588, 'epoch': 1.22} 61%|██████ | 4570/7500 [2:35:01<1:41:39, 2.08s/it] 61%|██████ | 4571/7500 [2:35:03<1:42:21, 2.10s/it] 61%|██████ | 4572/7500 [2:35:05<1:42:40, 2.10s/it] 61%|██████ | 4573/7500 [2:35:07<1:41:55, 2.09s/it] 61%|██████ | 4574/7500 [2:35:09<1:40:10, 2.05s/it] 61%|██████ | 4575/7500 [2:35:11<1:38:44, 2.03s/it] 61%|██████ | 4576/7500 [2:35:13<1:38:43, 2.03s/it] 61%|██████ | 4577/7500 [2:35:15<1:38:18, 2.02s/it] 61%|██████ | 4578/7500 [2:35:17<1:37:01, 1.99s/it] 61%|██████ | 4579/7500 [2:35:19<1:37:19, 2.00s/it] 61%|██████ | 4580/7500 [2:35:21<1:38:46, 2.03s/it] {'loss': 0.3723, 'grad_norm': 2.842633008956909, 'learning_rate': 1.9745315534350155e-05, 'logqs/rejected': -2.5534212589263916, 'logqs/chosen': -0.8530095219612122, 'logqs_logps/rejected': -0.01871670037508011, 'logps_logqs/chosen': 0.20640626549720764, 'epoch': 1.22} 61%|██████ | 4580/7500 [2:35:21<1:38:46, 2.03s/it] 61%|██████ | 4581/7500 [2:35:23<1:39:29, 2.05s/it] 61%|██████ | 4582/7500 [2:35:25<1:39:16, 2.04s/it] 61%|██████ | 4583/7500 [2:35:27<1:39:02, 2.04s/it] 61%|██████ | 4584/7500 [2:35:29<1:38:57, 2.04s/it] 61%|██████ | 4585/7500 [2:35:31<1:38:50, 2.03s/it] 61%|██████ | 4586/7500 [2:35:33<1:38:53, 2.04s/it] 61%|██████ | 4587/7500 [2:35:35<1:38:43, 2.03s/it] 61%|██████ | 4588/7500 [2:35:37<1:38:42, 2.03s/it] 61%|██████ | 4589/7500 [2:35:39<1:38:47, 2.04s/it] 61%|██████ | 4590/7500 [2:35:41<1:39:18, 2.05s/it] {'loss': 0.3695, 'grad_norm': 2.3007431030273438, 'learning_rate': 1.963161682082342e-05, 'logqs/rejected': -2.837968111038208, 'logqs/chosen': -0.8511551022529602, 'logqs_logps/rejected': -0.021652977913618088, 'logps_logqs/chosen': 0.20420995354652405, 'epoch': 1.22} 61%|██████ | 4590/7500 [2:35:41<1:39:18, 2.05s/it] 61%|██████ | 4591/7500 [2:35:43<1:40:04, 2.06s/it] 61%|██████ | 4592/7500 [2:35:45<1:39:45, 2.06s/it] 61%|██████ | 4593/7500 [2:35:48<1:40:30, 2.07s/it] 61%|██████▏ | 4594/7500 [2:35:50<1:39:58, 2.06s/it] 61%|██████▏ | 4595/7500 [2:35:52<1:39:10, 2.05s/it] 61%|██████▏ | 4596/7500 [2:35:54<1:38:19, 2.03s/it] 61%|██████▏ | 4597/7500 [2:35:56<1:38:20, 2.03s/it] 61%|██████▏ | 4598/7500 [2:35:58<1:38:34, 2.04s/it] 61%|██████▏ | 4599/7500 [2:36:00<1:39:42, 2.06s/it] 61%|██████▏ | 4600/7500 [2:36:02<1:39:45, 2.06s/it] {'loss': 0.3333, 'grad_norm': 2.812959909439087, 'learning_rate': 1.9518034395302414e-05, 'logqs/rejected': -2.2326340675354004, 'logqs/chosen': -0.8436763882637024, 'logqs_logps/rejected': -0.03132075071334839, 'logps_logqs/chosen': 0.20946398377418518, 'epoch': 1.23} 61%|██████▏ | 4600/7500 [2:36:02<1:39:45, 2.06s/it] 61%|██████▏ | 4601/7500 [2:36:04<1:39:08, 2.05s/it] 61%|██████▏ | 4602/7500 [2:36:06<1:39:08, 2.05s/it] 61%|██████▏ | 4603/7500 [2:36:08<1:40:35, 2.08s/it] 61%|██████▏ | 4604/7500 [2:36:10<1:39:40, 2.06s/it] 61%|██████▏ | 4605/7500 [2:36:12<1:41:27, 2.10s/it] 61%|██████▏ | 4606/7500 [2:36:14<1:41:42, 2.11s/it] 61%|██████▏ | 4607/7500 [2:36:16<1:39:49, 2.07s/it] 61%|██████▏ | 4608/7500 [2:36:18<1:39:05, 2.06s/it] 61%|██████▏ | 4609/7500 [2:36:20<1:39:12, 2.06s/it] 61%|██████▏ | 4610/7500 [2:36:22<1:38:41, 2.05s/it] {'loss': 0.3601, 'grad_norm': 2.989708423614502, 'learning_rate': 1.940457071816922e-05, 'logqs/rejected': -2.5609335899353027, 'logqs/chosen': -0.8413921594619751, 'logqs_logps/rejected': -0.02890120819211006, 'logps_logqs/chosen': 0.21349482238292694, 'epoch': 1.23} 61%|██████▏ | 4610/7500 [2:36:23<1:38:41, 2.05s/it] 61%|██████▏ | 4611/7500 [2:36:25<1:38:51, 2.05s/it] 61%|██████▏ | 4612/7500 [2:36:27<1:37:52, 2.03s/it] 62%|██████▏ | 4613/7500 [2:36:29<1:39:04, 2.06s/it] 62%|██████▏ | 4614/7500 [2:36:31<1:39:37, 2.07s/it] 62%|██████▏ | 4615/7500 [2:36:33<1:38:49, 2.06s/it] 62%|██████▏ | 4616/7500 [2:36:35<1:38:43, 2.05s/it] 62%|██████▏ | 4617/7500 [2:36:37<1:37:38, 2.03s/it] 62%|██████▏ | 4618/7500 [2:36:39<1:37:28, 2.03s/it] 62%|██████▏ | 4619/7500 [2:36:41<1:37:25, 2.03s/it] 62%|██████▏ | 4620/7500 [2:36:43<1:37:17, 2.03s/it] {'loss': 0.3545, 'grad_norm': 2.682227373123169, 'learning_rate': 1.9291228247233605e-05, 'logqs/rejected': -2.531440258026123, 'logqs/chosen': -0.8391246795654297, 'logqs_logps/rejected': -0.028880596160888672, 'logps_logqs/chosen': 0.2035110890865326, 'epoch': 1.23} 62%|██████▏ | 4620/7500 [2:36:43<1:37:17, 2.03s/it] 62%|██████▏ | 4621/7500 [2:36:45<1:37:14, 2.03s/it] 62%|██████▏ | 4622/7500 [2:36:47<1:38:01, 2.04s/it] 62%|██████▏ | 4623/7500 [2:36:49<1:38:00, 2.04s/it] 62%|██████▏ | 4624/7500 [2:36:51<1:37:02, 2.02s/it] 62%|██████▏ | 4625/7500 [2:36:53<1:37:02, 2.03s/it] 62%|██████▏ | 4626/7500 [2:36:55<1:36:52, 2.02s/it] 62%|██████▏ | 4627/7500 [2:36:57<1:37:12, 2.03s/it] 62%|██████▏ | 4628/7500 [2:36:59<1:38:39, 2.06s/it] 62%|██████▏ | 4629/7500 [2:37:01<1:38:57, 2.07s/it] 62%|██████▏ | 4630/7500 [2:37:03<1:39:35, 2.08s/it] {'loss': 0.3565, 'grad_norm': 2.7575628757476807, 'learning_rate': 1.9178009437679857e-05, 'logqs/rejected': -2.540386199951172, 'logqs/chosen': -0.8321666717529297, 'logqs_logps/rejected': -0.024292971938848495, 'logps_logqs/chosen': 0.20647093653678894, 'epoch': 1.23} 62%|██████▏ | 4630/7500 [2:37:03<1:39:35, 2.08s/it] 62%|██████▏ | 4631/7500 [2:37:05<1:39:07, 2.07s/it] 62%|██████▏ | 4632/7500 [2:37:08<1:38:40, 2.06s/it] 62%|██████▏ | 4633/7500 [2:37:10<1:38:10, 2.05s/it] 62%|██████▏ | 4634/7500 [2:37:12<1:37:51, 2.05s/it] 62%|██████▏ | 4635/7500 [2:37:14<1:36:35, 2.02s/it] 62%|██████▏ | 4636/7500 [2:37:16<1:37:37, 2.05s/it] 62%|██████▏ | 4637/7500 [2:37:18<1:36:32, 2.02s/it] 62%|██████▏ | 4638/7500 [2:37:20<1:37:36, 2.05s/it] 62%|██████▏ | 4639/7500 [2:37:22<1:37:54, 2.05s/it] 62%|██████▏ | 4640/7500 [2:37:24<1:36:52, 2.03s/it] {'loss': 0.3871, 'grad_norm': 2.6322176456451416, 'learning_rate': 1.9064916742013515e-05, 'logqs/rejected': -2.5782530307769775, 'logqs/chosen': -0.8776620626449585, 'logqs_logps/rejected': -0.018019353970885277, 'logps_logqs/chosen': 0.20997297763824463, 'epoch': 1.24} 62%|██████▏ | 4640/7500 [2:37:24<1:36:52, 2.03s/it] 62%|██████▏ | 4641/7500 [2:37:26<1:37:16, 2.04s/it] 62%|██████▏ | 4642/7500 [2:37:28<1:37:45, 2.05s/it] 62%|██████▏ | 4643/7500 [2:37:30<1:37:54, 2.06s/it] 62%|██████▏ | 4644/7500 [2:37:32<1:38:14, 2.06s/it] 62%|██████▏ | 4645/7500 [2:37:34<1:37:22, 2.05s/it] 62%|██████▏ | 4646/7500 [2:37:36<1:37:35, 2.05s/it] 62%|██████▏ | 4647/7500 [2:37:38<1:38:03, 2.06s/it] 62%|██████▏ | 4648/7500 [2:37:40<1:37:45, 2.06s/it] 62%|██████▏ | 4649/7500 [2:37:42<1:37:03, 2.04s/it] 62%|██████▏ | 4650/7500 [2:37:44<1:36:15, 2.03s/it] {'loss': 0.3702, 'grad_norm': 3.358405590057373, 'learning_rate': 1.895195261000831e-05, 'logqs/rejected': -2.5810272693634033, 'logqs/chosen': -0.8504071235656738, 'logqs_logps/rejected': -0.017572689801454544, 'logps_logqs/chosen': 0.20950698852539062, 'epoch': 1.24} 62%|██████▏ | 4650/7500 [2:37:44<1:36:15, 2.03s/it] 62%|██████▏ | 4651/7500 [2:37:46<1:35:34, 2.01s/it] 62%|██████▏ | 4652/7500 [2:37:48<1:35:07, 2.00s/it] 62%|██████▏ | 4653/7500 [2:37:50<1:35:29, 2.01s/it] 62%|██████▏ | 4654/7500 [2:37:52<1:35:44, 2.02s/it] 62%|██████▏ | 4655/7500 [2:37:54<1:36:40, 2.04s/it] 62%|██████▏ | 4656/7500 [2:37:56<1:37:13, 2.05s/it] 62%|██████▏ | 4657/7500 [2:37:58<1:36:25, 2.03s/it] 62%|██████▏ | 4658/7500 [2:38:00<1:35:22, 2.01s/it] 62%|██████▏ | 4659/7500 [2:38:02<1:35:52, 2.02s/it] 62%|██████▏ | 4660/7500 [2:38:05<1:36:22, 2.04s/it] {'loss': 0.3542, 'grad_norm': 3.237199544906616, 'learning_rate': 1.883911948865306e-05, 'logqs/rejected': -2.5201728343963623, 'logqs/chosen': -0.8430054783821106, 'logqs_logps/rejected': -0.022433793172240257, 'logps_logqs/chosen': 0.19975678622722626, 'epoch': 1.24} 62%|██████▏ | 4660/7500 [2:38:05<1:36:22, 2.04s/it] 62%|██████▏ | 4661/7500 [2:38:07<1:36:37, 2.04s/it] 62%|██████▏ | 4662/7500 [2:38:09<1:36:40, 2.04s/it] 62%|██████▏ | 4663/7500 [2:38:11<1:35:43, 2.02s/it] 62%|██████▏ | 4664/7500 [2:38:13<1:37:17, 2.06s/it] 62%|██████▏ | 4665/7500 [2:38:15<1:37:16, 2.06s/it] 62%|██████▏ | 4666/7500 [2:38:17<1:36:47, 2.05s/it] 62%|██████▏ | 4667/7500 [2:38:19<1:37:37, 2.07s/it] 62%|██████▏ | 4668/7500 [2:38:21<1:36:10, 2.04s/it] 62%|██████▏ | 4669/7500 [2:38:23<1:36:52, 2.05s/it] 62%|██████▏ | 4670/7500 [2:38:25<1:35:35, 2.03s/it] {'loss': 0.3528, 'grad_norm': 2.7006943225860596, 'learning_rate': 1.872641982209868e-05, 'logqs/rejected': -2.652398109436035, 'logqs/chosen': -0.8488935232162476, 'logqs_logps/rejected': -0.02415482886135578, 'logps_logqs/chosen': 0.20654813945293427, 'epoch': 1.25} 62%|██████▏ | 4670/7500 [2:38:25<1:35:35, 2.03s/it] 62%|██████▏ | 4671/7500 [2:38:27<1:35:47, 2.03s/it] 62%|██████▏ | 4672/7500 [2:38:29<1:35:17, 2.02s/it] 62%|██████▏ | 4673/7500 [2:38:31<1:35:30, 2.03s/it] 62%|██████▏ | 4674/7500 [2:38:33<1:35:33, 2.03s/it] 62%|██████▏ | 4675/7500 [2:38:35<1:34:55, 2.02s/it] 62%|██████▏ | 4676/7500 [2:38:37<1:34:44, 2.01s/it] 62%|██████▏ | 4677/7500 [2:38:39<1:35:13, 2.02s/it] 62%|██████▏ | 4678/7500 [2:38:41<1:35:25, 2.03s/it] 62%|██████▏ | 4679/7500 [2:38:43<1:36:59, 2.06s/it] 62%|██████▏ | 4680/7500 [2:38:45<1:36:24, 2.05s/it] {'loss': 0.3439, 'grad_norm': 2.635676383972168, 'learning_rate': 1.8613856051605243e-05, 'logqs/rejected': -2.4081177711486816, 'logqs/chosen': -0.8253768682479858, 'logqs_logps/rejected': -0.03118940070271492, 'logps_logqs/chosen': 0.2012127935886383, 'epoch': 1.25} 62%|██████▏ | 4680/7500 [2:38:45<1:36:24, 2.05s/it] 62%|██████▏ | 4681/7500 [2:38:47<1:36:03, 2.04s/it] 62%|██████▏ | 4682/7500 [2:38:49<1:34:49, 2.02s/it] 62%|██████▏ | 4683/7500 [2:38:51<1:35:39, 2.04s/it] 62%|██████▏ | 4684/7500 [2:38:54<1:36:30, 2.06s/it] 62%|██████▏ | 4685/7500 [2:38:56<1:35:46, 2.04s/it] 62%|██████▏ | 4686/7500 [2:38:58<1:38:01, 2.09s/it] 62%|██████▏ | 4687/7500 [2:39:00<1:38:07, 2.09s/it] 63%|██████▎ | 4688/7500 [2:39:02<1:35:44, 2.04s/it] 63%|██████▎ | 4689/7500 [2:39:04<1:36:55, 2.07s/it] 63%|██████▎ | 4690/7500 [2:39:06<1:36:49, 2.07s/it] {'loss': 0.3559, 'grad_norm': 2.7971982955932617, 'learning_rate': 1.850143061548907e-05, 'logqs/rejected': -2.5391435623168945, 'logqs/chosen': -0.8556163907051086, 'logqs_logps/rejected': -0.02435336634516716, 'logps_logqs/chosen': 0.2086412012577057, 'epoch': 1.25} 63%|██████▎ | 4690/7500 [2:39:06<1:36:49, 2.07s/it] 63%|██████▎ | 4691/7500 [2:39:08<1:36:29, 2.06s/it] 63%|██████▎ | 4692/7500 [2:39:10<1:34:59, 2.03s/it] 63%|██████▎ | 4693/7500 [2:39:12<1:35:34, 2.04s/it] 63%|██████▎ | 4694/7500 [2:39:14<1:34:30, 2.02s/it] 63%|██████▎ | 4695/7500 [2:39:16<1:36:02, 2.05s/it] 63%|██████▎ | 4696/7500 [2:39:18<1:35:31, 2.04s/it] 63%|██████▎ | 4697/7500 [2:39:21<1:40:02, 2.14s/it] 63%|██████▎ | 4698/7500 [2:39:23<1:40:25, 2.15s/it] 63%|██████▎ | 4699/7500 [2:39:25<1:39:48, 2.14s/it] 63%|██████▎ | 4700/7500 [2:39:27<1:39:18, 2.13s/it] {'loss': 0.3479, 'grad_norm': 3.0552854537963867, 'learning_rate': 1.838914594906995e-05, 'logqs/rejected': -2.394540786743164, 'logqs/chosen': -0.8448970913887024, 'logqs_logps/rejected': -0.028393950313329697, 'logps_logqs/chosen': 0.20929913222789764, 'epoch': 1.25} 63%|██████▎ | 4700/7500 [2:39:27<1:39:18, 2.13s/it] 63%|██████▎ | 4701/7500 [2:39:29<1:39:31, 2.13s/it] 63%|██████▎ | 4702/7500 [2:39:31<1:38:35, 2.11s/it] 63%|██████▎ | 4703/7500 [2:39:33<1:39:01, 2.12s/it] 63%|██████▎ | 4704/7500 [2:39:35<1:38:16, 2.11s/it] 63%|██████▎ | 4705/7500 [2:39:37<1:37:35, 2.09s/it] 63%|██████▎ | 4706/7500 [2:39:39<1:37:23, 2.09s/it] 63%|██████▎ | 4707/7500 [2:39:42<1:36:45, 2.08s/it] 63%|██████▎ | 4708/7500 [2:39:44<1:38:34, 2.12s/it] 63%|██████▎ | 4709/7500 [2:39:46<1:38:14, 2.11s/it] 63%|██████▎ | 4710/7500 [2:39:48<1:37:47, 2.10s/it] {'loss': 0.3678, 'grad_norm': 3.3588955402374268, 'learning_rate': 1.827700448461836e-05, 'logqs/rejected': -2.5400149822235107, 'logqs/chosen': -0.8650927543640137, 'logqs_logps/rejected': -0.023807961493730545, 'logps_logqs/chosen': 0.2112056314945221, 'epoch': 1.26} 63%|██████▎ | 4710/7500 [2:39:48<1:37:47, 2.10s/it] 63%|██████▎ | 4711/7500 [2:39:50<1:36:57, 2.09s/it] 63%|██████▎ | 4712/7500 [2:39:52<1:37:19, 2.09s/it] 63%|██████▎ | 4713/7500 [2:39:54<1:37:55, 2.11s/it] 63%|██████▎ | 4714/7500 [2:39:56<1:37:36, 2.10s/it] 63%|██████▎ | 4715/7500 [2:39:58<1:36:27, 2.08s/it] 63%|██████▎ | 4716/7500 [2:40:00<1:35:54, 2.07s/it] 63%|██████▎ | 4717/7500 [2:40:02<1:36:43, 2.09s/it] 63%|██████▎ | 4718/7500 [2:40:05<1:36:40, 2.08s/it] 63%|██████▎ | 4719/7500 [2:40:07<1:36:18, 2.08s/it] 63%|██████▎ | 4720/7500 [2:40:09<1:35:58, 2.07s/it] {'loss': 0.3588, 'grad_norm': 2.842529535293579, 'learning_rate': 1.8165008651302788e-05, 'logqs/rejected': -2.56204891204834, 'logqs/chosen': -0.8511019945144653, 'logqs_logps/rejected': -0.018048657104372978, 'logps_logqs/chosen': 0.20788399875164032, 'epoch': 1.26} 63%|██████▎ | 4720/7500 [2:40:09<1:35:58, 2.07s/it] 63%|██████▎ | 4721/7500 [2:40:11<1:37:27, 2.10s/it] 63%|██████▎ | 4722/7500 [2:40:13<1:37:03, 2.10s/it] 63%|██████▎ | 4723/7500 [2:40:15<1:36:09, 2.08s/it] 63%|██████▎ | 4724/7500 [2:40:17<1:36:22, 2.08s/it] 63%|██████▎ | 4725/7500 [2:40:19<1:36:08, 2.08s/it] 63%|██████▎ | 4726/7500 [2:40:21<1:36:50, 2.09s/it] 63%|██████▎ | 4727/7500 [2:40:23<1:37:01, 2.10s/it] 63%|██████▎ | 4728/7500 [2:40:26<1:37:16, 2.11s/it] 63%|██████▎ | 4729/7500 [2:40:28<1:37:22, 2.11s/it] 63%|██████▎ | 4730/7500 [2:40:30<1:37:25, 2.11s/it] {'loss': 0.3634, 'grad_norm': 3.401381731033325, 'learning_rate': 1.8053160875137137e-05, 'logqs/rejected': -2.5453643798828125, 'logqs/chosen': -0.8480016589164734, 'logqs_logps/rejected': -0.02631036378443241, 'logps_logqs/chosen': 0.20598192512989044, 'epoch': 1.26} 63%|██████▎ | 4730/7500 [2:40:30<1:37:25, 2.11s/it] 63%|██████▎ | 4731/7500 [2:40:32<1:36:31, 2.09s/it] 63%|██████▎ | 4732/7500 [2:40:34<1:36:15, 2.09s/it] 63%|██████▎ | 4733/7500 [2:40:36<1:36:55, 2.10s/it] 63%|██████▎ | 4734/7500 [2:40:38<1:36:22, 2.09s/it] 63%|██████▎ | 4735/7500 [2:40:40<1:36:42, 2.10s/it] 63%|██████▎ | 4736/7500 [2:40:42<1:36:58, 2.11s/it] 63%|██████▎ | 4737/7500 [2:40:44<1:36:43, 2.10s/it] 63%|██████▎ | 4738/7500 [2:40:46<1:36:07, 2.09s/it] 63%|██████▎ | 4739/7500 [2:40:49<1:37:20, 2.12s/it] 63%|██████▎ | 4740/7500 [2:40:51<1:36:40, 2.10s/it] {'loss': 0.341, 'grad_norm': 2.559401750564575, 'learning_rate': 1.7941463578928086e-05, 'logqs/rejected': -2.3539066314697266, 'logqs/chosen': -0.855099081993103, 'logqs_logps/rejected': -0.03666319698095322, 'logps_logqs/chosen': 0.20269985496997833, 'epoch': 1.26} 63%|██████▎ | 4740/7500 [2:40:51<1:36:40, 2.10s/it] 63%|██████▎ | 4741/7500 [2:40:53<1:35:55, 2.09s/it] 63%|██████▎ | 4742/7500 [2:40:55<1:35:12, 2.07s/it] 63%|██████▎ | 4743/7500 [2:40:57<1:34:35, 2.06s/it] 63%|██████▎ | 4744/7500 [2:40:59<1:35:48, 2.09s/it] 63%|██████▎ | 4745/7500 [2:41:01<1:35:08, 2.07s/it] 63%|██████▎ | 4746/7500 [2:41:03<1:34:38, 2.06s/it] 63%|██████▎ | 4747/7500 [2:41:05<1:33:45, 2.04s/it] 63%|██████▎ | 4748/7500 [2:41:07<1:33:43, 2.04s/it] 63%|██████▎ | 4749/7500 [2:41:09<1:35:59, 2.09s/it] 63%|██████▎ | 4750/7500 [2:41:11<1:35:48, 2.09s/it] {'loss': 0.3529, 'grad_norm': 2.7052295207977295, 'learning_rate': 1.7829919182222752e-05, 'logqs/rejected': -2.5079801082611084, 'logqs/chosen': -0.8560906648635864, 'logqs_logps/rejected': -0.03388064354658127, 'logps_logqs/chosen': 0.20770876109600067, 'epoch': 1.27} 63%|██████▎ | 4750/7500 [2:41:11<1:35:48, 2.09s/it] 63%|██████▎ | 4751/7500 [2:41:13<1:34:44, 2.07s/it] 63%|██████▎ | 4752/7500 [2:41:15<1:34:01, 2.05s/it] 63%|██████▎ | 4753/7500 [2:41:17<1:34:19, 2.06s/it] 63%|██████▎ | 4754/7500 [2:41:20<1:34:28, 2.06s/it] 63%|██████▎ | 4755/7500 [2:41:22<1:33:29, 2.04s/it] 63%|██████▎ | 4756/7500 [2:41:24<1:33:29, 2.04s/it] 63%|██████▎ | 4757/7500 [2:41:26<1:33:48, 2.05s/it] 63%|██████▎ | 4758/7500 [2:41:28<1:34:40, 2.07s/it] 63%|██████▎ | 4759/7500 [2:41:30<1:33:05, 2.04s/it] 63%|██████▎ | 4760/7500 [2:41:32<1:33:04, 2.04s/it] {'loss': 0.3356, 'grad_norm': 2.515921115875244, 'learning_rate': 1.7718530101256113e-05, 'logqs/rejected': -2.4959826469421387, 'logqs/chosen': -0.8233832120895386, 'logqs_logps/rejected': -0.026719694957137108, 'logps_logqs/chosen': 0.20034965872764587, 'epoch': 1.27} 63%|██████▎ | 4760/7500 [2:41:32<1:33:04, 2.04s/it] 63%|██████▎ | 4761/7500 [2:41:34<1:34:30, 2.07s/it] 63%|██████▎ | 4762/7500 [2:41:36<1:33:37, 2.05s/it] 64%|██████▎ | 4763/7500 [2:41:38<1:33:18, 2.05s/it] 64%|██████▎ | 4764/7500 [2:41:40<1:34:15, 2.07s/it] 64%|██████▎ | 4765/7500 [2:41:42<1:36:12, 2.11s/it] 64%|██████▎ | 4766/7500 [2:41:44<1:35:21, 2.09s/it] 64%|██████▎ | 4767/7500 [2:41:46<1:34:17, 2.07s/it] 64%|██████▎ | 4768/7500 [2:41:49<1:35:31, 2.10s/it] 64%|██████▎ | 4769/7500 [2:41:51<1:35:20, 2.09s/it] 64%|██████▎ | 4770/7500 [2:41:53<1:36:20, 2.12s/it] {'loss': 0.3789, 'grad_norm': 3.5247557163238525, 'learning_rate': 1.7607298748898842e-05, 'logqs/rejected': -2.5469918251037598, 'logqs/chosen': -0.8672428131103516, 'logqs_logps/rejected': -0.02534819208085537, 'logps_logqs/chosen': 0.20677772164344788, 'epoch': 1.27} 64%|██████▎ | 4770/7500 [2:41:53<1:36:20, 2.12s/it] 64%|██████▎ | 4771/7500 [2:41:55<1:36:52, 2.13s/it] 64%|██████▎ | 4772/7500 [2:41:57<1:36:27, 2.12s/it] 64%|██████▎ | 4773/7500 [2:41:59<1:35:45, 2.11s/it] 64%|██████▎ | 4774/7500 [2:42:01<1:36:25, 2.12s/it] 64%|██████▎ | 4775/7500 [2:42:03<1:37:00, 2.14s/it] 64%|██████▎ | 4776/7500 [2:42:06<1:38:29, 2.17s/it] 64%|██████▎ | 4777/7500 [2:42:08<1:37:11, 2.14s/it] 64%|██████▎ | 4778/7500 [2:42:10<1:36:58, 2.14s/it] 64%|██████▎ | 4779/7500 [2:42:12<1:36:19, 2.12s/it] 64%|██████▎ | 4780/7500 [2:42:14<1:36:30, 2.13s/it] {'loss': 0.3667, 'grad_norm': 3.0045576095581055, 'learning_rate': 1.749622753460486e-05, 'logqs/rejected': -2.4781196117401123, 'logqs/chosen': -0.8668379783630371, 'logqs_logps/rejected': -0.024072017520666122, 'logps_logqs/chosen': 0.20787568390369415, 'epoch': 1.27} 64%|██████▎ | 4780/7500 [2:42:14<1:36:30, 2.13s/it] 64%|██████▎ | 4781/7500 [2:42:16<1:36:22, 2.13s/it] 64%|██████▍ | 4782/7500 [2:42:18<1:37:21, 2.15s/it] 64%|██████▍ | 4783/7500 [2:42:21<1:36:21, 2.13s/it] 64%|██████▍ | 4784/7500 [2:42:23<1:35:36, 2.11s/it] 64%|██████▍ | 4785/7500 [2:42:25<1:35:03, 2.10s/it] 64%|██████▍ | 4786/7500 [2:42:27<1:34:26, 2.09s/it] 64%|██████▍ | 4787/7500 [2:42:29<1:34:53, 2.10s/it] 64%|██████▍ | 4788/7500 [2:42:31<1:34:46, 2.10s/it] 64%|██████▍ | 4789/7500 [2:42:33<1:34:55, 2.10s/it] 64%|██████▍ | 4790/7500 [2:42:35<1:33:54, 2.08s/it] {'loss': 0.3608, 'grad_norm': 3.084813117980957, 'learning_rate': 1.7385318864359305e-05, 'logqs/rejected': -2.4780375957489014, 'logqs/chosen': -0.828381359577179, 'logqs_logps/rejected': -0.03307093307375908, 'logps_logqs/chosen': 0.20938006043434143, 'epoch': 1.28} 64%|██████▍ | 4790/7500 [2:42:35<1:33:54, 2.08s/it] 64%|██████▍ | 4791/7500 [2:42:37<1:34:25, 2.09s/it] 64%|██████▍ | 4792/7500 [2:42:39<1:33:13, 2.07s/it] 64%|██████▍ | 4793/7500 [2:42:41<1:34:00, 2.08s/it] 64%|██████▍ | 4794/7500 [2:42:43<1:34:08, 2.09s/it] 64%|██████▍ | 4795/7500 [2:42:45<1:31:59, 2.04s/it] 64%|██████▍ | 4796/7500 [2:42:47<1:31:07, 2.02s/it] 64%|██████▍ | 4797/7500 [2:42:49<1:31:44, 2.04s/it] 64%|██████▍ | 4798/7500 [2:42:52<1:33:49, 2.08s/it] 64%|██████▍ | 4799/7500 [2:42:54<1:33:10, 2.07s/it] 64%|██████▍ | 4800/7500 [2:42:56<1:32:16, 2.05s/it] {'loss': 0.3661, 'grad_norm': 2.3913590908050537, 'learning_rate': 1.7274575140626318e-05, 'logqs/rejected': -2.6529476642608643, 'logqs/chosen': -0.8378537893295288, 'logqs_logps/rejected': -0.01969287171959877, 'logps_logqs/chosen': 0.20344150066375732, 'epoch': 1.28} 64%|██████▍ | 4800/7500 [2:42:56<1:32:16, 2.05s/it] 64%|██████▍ | 4801/7500 [2:42:58<1:33:48, 2.09s/it] 64%|██████▍ | 4802/7500 [2:43:00<1:32:58, 2.07s/it] 64%|██████▍ | 4803/7500 [2:43:02<1:32:01, 2.05s/it] 64%|██████▍ | 4804/7500 [2:43:04<1:31:00, 2.03s/it] 64%|██████▍ | 4805/7500 [2:43:06<1:32:10, 2.05s/it] 64%|██████▍ | 4806/7500 [2:43:08<1:31:53, 2.05s/it] 64%|██████▍ | 4807/7500 [2:43:10<1:32:09, 2.05s/it] 64%|██████▍ | 4808/7500 [2:43:12<1:33:18, 2.08s/it] 64%|██████▍ | 4809/7500 [2:43:14<1:33:58, 2.10s/it] 64%|██████▍ | 4810/7500 [2:43:16<1:33:12, 2.08s/it] {'loss': 0.3738, 'grad_norm': 2.9829487800598145, 'learning_rate': 1.7163998762297013e-05, 'logqs/rejected': -2.613546848297119, 'logqs/chosen': -0.8519772291183472, 'logqs_logps/rejected': -0.02075525000691414, 'logps_logqs/chosen': 0.2117188274860382, 'epoch': 1.28} 64%|██████▍ | 4810/7500 [2:43:16<1:33:12, 2.08s/it] 64%|██████▍ | 4811/7500 [2:43:18<1:32:31, 2.06s/it] 64%|██████▍ | 4812/7500 [2:43:20<1:32:07, 2.06s/it] 64%|██████▍ | 4813/7500 [2:43:22<1:32:01, 2.06s/it] 64%|██████▍ | 4814/7500 [2:43:24<1:31:29, 2.04s/it] 64%|██████▍ | 4815/7500 [2:43:27<1:32:40, 2.07s/it] 64%|██████▍ | 4816/7500 [2:43:29<1:33:11, 2.08s/it] 64%|██████▍ | 4817/7500 [2:43:31<1:32:40, 2.07s/it] 64%|██████▍ | 4818/7500 [2:43:33<1:32:19, 2.07s/it] 64%|██████▍ | 4819/7500 [2:43:35<1:33:06, 2.08s/it] 64%|██████▍ | 4820/7500 [2:43:37<1:33:55, 2.10s/it] {'loss': 0.3766, 'grad_norm': 3.0824129581451416, 'learning_rate': 1.7053592124637558e-05, 'logqs/rejected': -2.5981202125549316, 'logqs/chosen': -0.8919575810432434, 'logqs_logps/rejected': -0.02605597674846649, 'logps_logqs/chosen': 0.20144446194171906, 'epoch': 1.29} 64%|██████▍ | 4820/7500 [2:43:37<1:33:55, 2.10s/it] 64%|██████▍ | 4821/7500 [2:43:39<1:32:42, 2.08s/it] 64%|██████▍ | 4822/7500 [2:43:41<1:31:46, 2.06s/it] 64%|██████▍ | 4823/7500 [2:43:43<1:33:13, 2.09s/it] 64%|██████▍ | 4824/7500 [2:43:45<1:31:40, 2.06s/it] 64%|██████▍ | 4825/7500 [2:43:47<1:31:02, 2.04s/it] 64%|██████▍ | 4826/7500 [2:43:49<1:31:14, 2.05s/it] 64%|██████▍ | 4827/7500 [2:43:51<1:30:07, 2.02s/it] 64%|██████▍ | 4828/7500 [2:43:53<1:30:24, 2.03s/it] 64%|██████▍ | 4829/7500 [2:43:55<1:31:09, 2.05s/it] 64%|██████▍ | 4830/7500 [2:43:58<1:31:29, 2.06s/it] {'loss': 0.3662, 'grad_norm': 2.4721457958221436, 'learning_rate': 1.6943357619237226e-05, 'logqs/rejected': -2.734330177307129, 'logqs/chosen': -0.8327099680900574, 'logqs_logps/rejected': -0.01851014606654644, 'logps_logqs/chosen': 0.20607523620128632, 'epoch': 1.29} 64%|██████▍ | 4830/7500 [2:43:58<1:31:29, 2.06s/it] 64%|██████▍ | 4831/7500 [2:44:00<1:31:47, 2.06s/it] 64%|██████▍ | 4832/7500 [2:44:02<1:32:46, 2.09s/it] 64%|██████▍ | 4833/7500 [2:44:04<1:32:39, 2.08s/it] 64%|██████▍ | 4834/7500 [2:44:06<1:32:56, 2.09s/it] 64%|██████▍ | 4835/7500 [2:44:08<1:31:54, 2.07s/it] 64%|██████▍ | 4836/7500 [2:44:10<1:33:20, 2.10s/it] 64%|██████▍ | 4837/7500 [2:44:12<1:33:35, 2.11s/it] 65%|██████▍ | 4838/7500 [2:44:14<1:33:16, 2.10s/it] 65%|██████▍ | 4839/7500 [2:44:16<1:32:16, 2.08s/it] 65%|██████▍ | 4840/7500 [2:44:18<1:31:48, 2.07s/it] {'loss': 0.3701, 'grad_norm': 3.0255496501922607, 'learning_rate': 1.6833297633956645e-05, 'logqs/rejected': -2.5337860584259033, 'logqs/chosen': -0.8562291860580444, 'logqs_logps/rejected': -0.02512858249247074, 'logps_logqs/chosen': 0.20897427201271057, 'epoch': 1.29} 65%|██████▍ | 4840/7500 [2:44:18<1:31:48, 2.07s/it] 65%|██████▍ | 4841/7500 [2:44:20<1:31:32, 2.07s/it] 65%|██████▍ | 4842/7500 [2:44:23<1:31:56, 2.08s/it] 65%|██████▍ | 4843/7500 [2:44:25<1:31:54, 2.08s/it] 65%|██████▍ | 4844/7500 [2:44:27<1:30:34, 2.05s/it] 65%|██████▍ | 4845/7500 [2:44:29<1:29:48, 2.03s/it] 65%|██████▍ | 4846/7500 [2:44:31<1:29:45, 2.03s/it] 65%|██████▍ | 4847/7500 [2:44:33<1:29:38, 2.03s/it] 65%|██████▍ | 4848/7500 [2:44:35<1:32:13, 2.09s/it] 65%|██████▍ | 4849/7500 [2:44:37<1:33:16, 2.11s/it] 65%|██████▍ | 4850/7500 [2:44:39<1:31:59, 2.08s/it] {'loss': 0.3611, 'grad_norm': 2.868919610977173, 'learning_rate': 1.672341455287605e-05, 'logqs/rejected': -2.555896043777466, 'logqs/chosen': -0.8277832269668579, 'logqs_logps/rejected': -0.021665159612894058, 'logps_logqs/chosen': 0.19909803569316864, 'epoch': 1.29} 65%|██████▍ | 4850/7500 [2:44:39<1:31:59, 2.08s/it] 65%|██████▍ | 4851/7500 [2:44:41<1:31:27, 2.07s/it] 65%|██████▍ | 4852/7500 [2:44:43<1:32:24, 2.09s/it] 65%|██████▍ | 4853/7500 [2:44:45<1:31:24, 2.07s/it] 65%|██████▍ | 4854/7500 [2:44:47<1:31:11, 2.07s/it] 65%|██████▍ | 4855/7500 [2:44:49<1:30:03, 2.04s/it] 65%|██████▍ | 4856/7500 [2:44:51<1:31:05, 2.07s/it] 65%|██████▍ | 4857/7500 [2:44:54<1:32:08, 2.09s/it] 65%|██████▍ | 4858/7500 [2:44:56<1:31:02, 2.07s/it] 65%|██████▍ | 4859/7500 [2:44:58<1:30:51, 2.06s/it] 65%|██████▍ | 4860/7500 [2:45:00<1:30:55, 2.07s/it] {'loss': 0.3452, 'grad_norm': 2.327688217163086, 'learning_rate': 1.6613710756243626e-05, 'logqs/rejected': -2.380448818206787, 'logqs/chosen': -0.8536176681518555, 'logqs_logps/rejected': -0.025930682197213173, 'logps_logqs/chosen': 0.2076340913772583, 'epoch': 1.3} 65%|██████▍ | 4860/7500 [2:45:00<1:30:55, 2.07s/it] 65%|██████▍ | 4861/7500 [2:45:02<1:30:25, 2.06s/it] 65%|██████▍ | 4862/7500 [2:45:04<1:32:44, 2.11s/it] 65%|██████▍ | 4863/7500 [2:45:06<1:31:09, 2.07s/it] 65%|██████▍ | 4864/7500 [2:45:08<1:30:03, 2.05s/it] 65%|██████▍ | 4865/7500 [2:45:10<1:30:42, 2.07s/it] 65%|██████▍ | 4866/7500 [2:45:12<1:29:59, 2.05s/it] 65%|██████▍ | 4867/7500 [2:45:14<1:29:37, 2.04s/it] 65%|██████▍ | 4868/7500 [2:45:16<1:30:15, 2.06s/it] 65%|██████▍ | 4869/7500 [2:45:18<1:30:21, 2.06s/it] 65%|██████▍ | 4870/7500 [2:45:20<1:29:25, 2.04s/it] {'loss': 0.3441, 'grad_norm': 2.790463447570801, 'learning_rate': 1.6504188620423977e-05, 'logqs/rejected': -2.567035436630249, 'logqs/chosen': -0.8106153607368469, 'logqs_logps/rejected': -0.024446865543723106, 'logps_logqs/chosen': 0.20164763927459717, 'epoch': 1.3} 65%|██████▍ | 4870/7500 [2:45:20<1:29:25, 2.04s/it] 65%|██████▍ | 4871/7500 [2:45:22<1:29:23, 2.04s/it] 65%|██████▍ | 4872/7500 [2:45:24<1:28:21, 2.02s/it] 65%|██████▍ | 4873/7500 [2:45:26<1:27:55, 2.01s/it] 65%|██████▍ | 4874/7500 [2:45:28<1:29:20, 2.04s/it] 65%|██████▌ | 4875/7500 [2:45:30<1:29:13, 2.04s/it] 65%|██████▌ | 4876/7500 [2:45:33<1:30:40, 2.07s/it] 65%|██████▌ | 4877/7500 [2:45:35<1:28:57, 2.03s/it] 65%|██████▌ | 4878/7500 [2:45:37<1:29:08, 2.04s/it] 65%|██████▌ | 4879/7500 [2:45:39<1:30:21, 2.07s/it] 65%|██████▌ | 4880/7500 [2:45:41<1:29:57, 2.06s/it] {'loss': 0.3661, 'grad_norm': 3.0498578548431396, 'learning_rate': 1.639485051784662e-05, 'logqs/rejected': -2.6218249797821045, 'logqs/chosen': -0.8338747024536133, 'logqs_logps/rejected': -0.023995094001293182, 'logps_logqs/chosen': 0.20476186275482178, 'epoch': 1.3} 65%|██████▌ | 4880/7500 [2:45:41<1:29:57, 2.06s/it] 65%|██████▌ | 4881/7500 [2:45:43<1:30:27, 2.07s/it] 65%|██████▌ | 4882/7500 [2:45:45<1:29:34, 2.05s/it] 65%|██████▌ | 4883/7500 [2:45:47<1:29:20, 2.05s/it] 65%|██████▌ | 4884/7500 [2:45:49<1:30:08, 2.07s/it] 65%|██████▌ | 4885/7500 [2:45:51<1:29:40, 2.06s/it] 65%|██████▌ | 4886/7500 [2:45:53<1:30:36, 2.08s/it] 65%|██████▌ | 4887/7500 [2:45:55<1:30:43, 2.08s/it] 65%|██████▌ | 4888/7500 [2:45:57<1:30:01, 2.07s/it] 65%|██████▌ | 4889/7500 [2:45:59<1:29:00, 2.05s/it] 65%|██████▌ | 4890/7500 [2:46:01<1:27:38, 2.01s/it] {'loss': 0.3705, 'grad_norm': 3.0892510414123535, 'learning_rate': 1.6285698816954624e-05, 'logqs/rejected': -2.856790065765381, 'logqs/chosen': -0.8402729034423828, 'logqs_logps/rejected': -0.019713737070560455, 'logps_logqs/chosen': 0.20744562149047852, 'epoch': 1.3} 65%|██████▌ | 4890/7500 [2:46:01<1:27:38, 2.01s/it] 65%|██████▌ | 4891/7500 [2:46:03<1:27:55, 2.02s/it] 65%|██████▌ | 4892/7500 [2:46:05<1:28:29, 2.04s/it] 65%|██████▌ | 4893/7500 [2:46:07<1:28:32, 2.04s/it] 65%|██████▌ | 4894/7500 [2:46:09<1:29:13, 2.05s/it] 65%|██████▌ | 4895/7500 [2:46:12<1:29:05, 2.05s/it] 65%|██████▌ | 4896/7500 [2:46:14<1:28:52, 2.05s/it] 65%|██████▌ | 4897/7500 [2:46:16<1:28:55, 2.05s/it] 65%|██████▌ | 4898/7500 [2:46:18<1:28:30, 2.04s/it] 65%|██████▌ | 4899/7500 [2:46:20<1:29:51, 2.07s/it] 65%|██████▌ | 4900/7500 [2:46:22<1:29:39, 2.07s/it] {'loss': 0.369, 'grad_norm': 3.079681396484375, 'learning_rate': 1.617673588215328e-05, 'logqs/rejected': -2.8536088466644287, 'logqs/chosen': -0.83617103099823, 'logqs_logps/rejected': -0.01940852589905262, 'logps_logqs/chosen': 0.20402535796165466, 'epoch': 1.31} 65%|██████▌ | 4900/7500 [2:46:22<1:29:39, 2.07s/it] 65%|██████▌ | 4901/7500 [2:46:24<1:30:11, 2.08s/it] 65%|██████▌ | 4902/7500 [2:46:26<1:28:59, 2.06s/it] 65%|██████▌ | 4903/7500 [2:46:28<1:29:20, 2.06s/it] 65%|██████▌ | 4904/7500 [2:46:30<1:28:01, 2.03s/it] 65%|██████▌ | 4905/7500 [2:46:32<1:29:00, 2.06s/it] 65%|██████▌ | 4906/7500 [2:46:34<1:28:40, 2.05s/it] 65%|██████▌ | 4907/7500 [2:46:36<1:28:26, 2.05s/it] 65%|██████▌ | 4908/7500 [2:46:38<1:28:56, 2.06s/it] 65%|██████▌ | 4909/7500 [2:46:40<1:28:43, 2.05s/it] 65%|██████▌ | 4910/7500 [2:46:42<1:29:11, 2.07s/it] {'loss': 0.3629, 'grad_norm': 2.8316445350646973, 'learning_rate': 1.60679640737589e-05, 'logqs/rejected': -2.629330635070801, 'logqs/chosen': -0.8280022740364075, 'logqs_logps/rejected': -0.02535988762974739, 'logps_logqs/chosen': 0.20201309025287628, 'epoch': 1.31} 65%|██████▌ | 4910/7500 [2:46:42<1:29:11, 2.07s/it] 65%|██████▌ | 4911/7500 [2:46:45<1:29:48, 2.08s/it] 65%|██████▌ | 4912/7500 [2:46:47<1:30:20, 2.09s/it] 66%|██████▌ | 4913/7500 [2:46:49<1:29:13, 2.07s/it] 66%|██████▌ | 4914/7500 [2:46:51<1:29:03, 2.07s/it] 66%|██████▌ | 4915/7500 [2:46:53<1:30:25, 2.10s/it] 66%|██████▌ | 4916/7500 [2:46:55<1:30:31, 2.10s/it] 66%|██████▌ | 4917/7500 [2:46:57<1:30:40, 2.11s/it] 66%|██████▌ | 4918/7500 [2:46:59<1:30:17, 2.10s/it] 66%|██████▌ | 4919/7500 [2:47:01<1:30:43, 2.11s/it] 66%|██████▌ | 4920/7500 [2:47:03<1:30:31, 2.11s/it] {'loss': 0.3359, 'grad_norm': 2.9911906719207764, 'learning_rate': 1.5959385747947698e-05, 'logqs/rejected': -2.3573062419891357, 'logqs/chosen': -0.842551589012146, 'logqs_logps/rejected': -0.02818717062473297, 'logps_logqs/chosen': 0.20296785235404968, 'epoch': 1.31} 66%|██████▌ | 4920/7500 [2:47:03<1:30:31, 2.11s/it] 66%|██████▌ | 4921/7500 [2:47:06<1:30:08, 2.10s/it] 66%|██████▌ | 4922/7500 [2:47:08<1:31:41, 2.13s/it] 66%|██████▌ | 4923/7500 [2:47:10<1:29:59, 2.10s/it] 66%|██████▌ | 4924/7500 [2:47:12<1:29:42, 2.09s/it] 66%|██████▌ | 4925/7500 [2:47:14<1:30:18, 2.10s/it] 66%|██████▌ | 4926/7500 [2:47:16<1:30:22, 2.11s/it] 66%|██████▌ | 4927/7500 [2:47:18<1:29:38, 2.09s/it] 66%|██████▌ | 4928/7500 [2:47:20<1:28:31, 2.07s/it] 66%|██████▌ | 4929/7500 [2:47:22<1:28:28, 2.06s/it] 66%|██████▌ | 4930/7500 [2:47:24<1:27:06, 2.03s/it] {'loss': 0.3574, 'grad_norm': 2.7046561241149902, 'learning_rate': 1.5851003256704698e-05, 'logqs/rejected': -2.6126410961151123, 'logqs/chosen': -0.8304583430290222, 'logqs_logps/rejected': -0.017925169318914413, 'logps_logqs/chosen': 0.2021595984697342, 'epoch': 1.31} 66%|██████▌ | 4930/7500 [2:47:24<1:27:06, 2.03s/it] 66%|██████▌ | 4931/7500 [2:47:26<1:27:19, 2.04s/it] 66%|██████▌ | 4932/7500 [2:47:28<1:27:26, 2.04s/it] 66%|██████▌ | 4933/7500 [2:47:30<1:27:32, 2.05s/it] 66%|██████▌ | 4934/7500 [2:47:32<1:27:17, 2.04s/it] 66%|██████▌ | 4935/7500 [2:47:34<1:28:06, 2.06s/it] 66%|██████▌ | 4936/7500 [2:47:37<1:29:24, 2.09s/it] 66%|██████▌ | 4937/7500 [2:47:39<1:27:54, 2.06s/it] 66%|██████▌ | 4938/7500 [2:47:41<1:27:41, 2.05s/it] 66%|██████▌ | 4939/7500 [2:47:43<1:27:23, 2.05s/it] 66%|██████▌ | 4940/7500 [2:47:45<1:27:20, 2.05s/it] {'loss': 0.3444, 'grad_norm': 2.474177837371826, 'learning_rate': 1.5742818947772877e-05, 'logqs/rejected': -2.484210729598999, 'logqs/chosen': -0.8241927027702332, 'logqs_logps/rejected': -0.02184728905558586, 'logps_logqs/chosen': 0.20711222290992737, 'epoch': 1.32} 66%|██████▌ | 4940/7500 [2:47:45<1:27:20, 2.05s/it] 66%|██████▌ | 4941/7500 [2:47:47<1:26:25, 2.03s/it] 66%|██████▌ | 4942/7500 [2:47:49<1:27:27, 2.05s/it] 66%|██████▌ | 4943/7500 [2:47:51<1:27:33, 2.05s/it] 66%|██████▌ | 4944/7500 [2:47:53<1:27:11, 2.05s/it] 66%|██████▌ | 4945/7500 [2:47:55<1:27:25, 2.05s/it] 66%|██████▌ | 4946/7500 [2:47:57<1:27:30, 2.06s/it] 66%|██████▌ | 4947/7500 [2:47:59<1:27:08, 2.05s/it] 66%|██████▌ | 4948/7500 [2:48:01<1:27:44, 2.06s/it] 66%|██████▌ | 4949/7500 [2:48:03<1:27:35, 2.06s/it] 66%|██████▌ | 4950/7500 [2:48:05<1:28:32, 2.08s/it] {'loss': 0.3625, 'grad_norm': 2.876878261566162, 'learning_rate': 1.56348351646022e-05, 'logqs/rejected': -2.4272541999816895, 'logqs/chosen': -0.8608362078666687, 'logqs_logps/rejected': -0.024381713941693306, 'logps_logqs/chosen': 0.2074965238571167, 'epoch': 1.32} 66%|██████▌ | 4950/7500 [2:48:05<1:28:32, 2.08s/it] 66%|██████▌ | 4951/7500 [2:48:07<1:27:56, 2.07s/it] 66%|██████▌ | 4952/7500 [2:48:09<1:27:13, 2.05s/it] 66%|██████▌ | 4953/7500 [2:48:12<1:28:00, 2.07s/it] 66%|██████▌ | 4954/7500 [2:48:14<1:27:15, 2.06s/it] 66%|██████▌ | 4955/7500 [2:48:16<1:28:00, 2.07s/it] 66%|██████▌ | 4956/7500 [2:48:18<1:25:59, 2.03s/it] 66%|██████▌ | 4957/7500 [2:48:20<1:26:34, 2.04s/it] 66%|██████▌ | 4958/7500 [2:48:22<1:26:48, 2.05s/it] 66%|██████▌ | 4959/7500 [2:48:24<1:26:41, 2.05s/it] 66%|██████▌ | 4960/7500 [2:48:26<1:26:58, 2.05s/it] {'loss': 0.3468, 'grad_norm': 2.8050220012664795, 'learning_rate': 1.5527054246298977e-05, 'logqs/rejected': -2.4503848552703857, 'logqs/chosen': -0.8389137387275696, 'logqs_logps/rejected': -0.02674154005944729, 'logps_logqs/chosen': 0.20547504723072052, 'epoch': 1.32} 66%|██████▌ | 4960/7500 [2:48:26<1:26:58, 2.05s/it] 66%|██████▌ | 4961/7500 [2:48:28<1:28:11, 2.08s/it] 66%|██████▌ | 4962/7500 [2:48:30<1:28:44, 2.10s/it] 66%|██████▌ | 4963/7500 [2:48:32<1:27:58, 2.08s/it] 66%|██████▌ | 4964/7500 [2:48:34<1:27:18, 2.07s/it] 66%|██████▌ | 4965/7500 [2:48:36<1:27:57, 2.08s/it] 66%|██████▌ | 4966/7500 [2:48:38<1:26:46, 2.05s/it] 66%|██████▌ | 4967/7500 [2:48:40<1:27:36, 2.08s/it] 66%|██████▌ | 4968/7500 [2:48:42<1:27:22, 2.07s/it] 66%|██████▋ | 4969/7500 [2:48:45<1:27:06, 2.07s/it] 66%|██████▋ | 4970/7500 [2:48:47<1:28:01, 2.09s/it] {'loss': 0.332, 'grad_norm': 2.6672170162200928, 'learning_rate': 1.5419478527575068e-05, 'logqs/rejected': -2.250277042388916, 'logqs/chosen': -0.8474068641662598, 'logqs_logps/rejected': -0.02792198583483696, 'logps_logqs/chosen': 0.20891675353050232, 'epoch': 1.33} 66%|██████▋ | 4970/7500 [2:48:47<1:28:01, 2.09s/it] 66%|██████▋ | 4971/7500 [2:48:49<1:28:27, 2.10s/it] 66%|██████▋ | 4972/7500 [2:48:51<1:27:50, 2.08s/it] 66%|██████▋ | 4973/7500 [2:48:53<1:28:28, 2.10s/it] 66%|██████▋ | 4974/7500 [2:48:55<1:27:49, 2.09s/it] 66%|██████▋ | 4975/7500 [2:48:57<1:26:06, 2.05s/it] 66%|██████▋ | 4976/7500 [2:48:59<1:26:20, 2.05s/it] 66%|██████▋ | 4977/7500 [2:49:01<1:26:49, 2.06s/it] 66%|██████▋ | 4978/7500 [2:49:03<1:27:46, 2.09s/it] 66%|██████▋ | 4979/7500 [2:49:05<1:28:25, 2.10s/it] 66%|██████▋ | 4980/7500 [2:49:08<1:29:50, 2.14s/it] {'loss': 0.3497, 'grad_norm': 2.8878443241119385, 'learning_rate': 1.5312110338697426e-05, 'logqs/rejected': -2.342054843902588, 'logqs/chosen': -0.8545166850090027, 'logqs_logps/rejected': -0.026702165603637695, 'logps_logqs/chosen': 0.20442573726177216, 'epoch': 1.33} 66%|██████▋ | 4980/7500 [2:49:08<1:29:50, 2.14s/it] 66%|██████▋ | 4981/7500 [2:49:10<1:29:30, 2.13s/it] 66%|██████▋ | 4982/7500 [2:49:12<1:30:04, 2.15s/it] 66%|██████▋ | 4983/7500 [2:49:14<1:29:21, 2.13s/it] 66%|██████▋ | 4984/7500 [2:49:16<1:27:04, 2.08s/it] 66%|██████▋ | 4985/7500 [2:49:18<1:26:23, 2.06s/it] 66%|██████▋ | 4986/7500 [2:49:20<1:26:44, 2.07s/it] 66%|██████▋ | 4987/7500 [2:49:22<1:26:09, 2.06s/it] 67%|██████▋ | 4988/7500 [2:49:24<1:26:08, 2.06s/it] 67%|██████▋ | 4989/7500 [2:49:26<1:26:19, 2.06s/it] 67%|██████▋ | 4990/7500 [2:49:28<1:25:57, 2.05s/it] {'loss': 0.3506, 'grad_norm': 2.736358165740967, 'learning_rate': 1.520495200543754e-05, 'logqs/rejected': -2.4573075771331787, 'logqs/chosen': -0.8631631731987, 'logqs_logps/rejected': -0.025508489459753036, 'logps_logqs/chosen': 0.2098219394683838, 'epoch': 1.33} 67%|██████▋ | 4990/7500 [2:49:28<1:25:57, 2.05s/it] 67%|██████▋ | 4991/7500 [2:49:30<1:26:18, 2.06s/it] 67%|██████▋ | 4992/7500 [2:49:32<1:25:49, 2.05s/it] 67%|██████▋ | 4993/7500 [2:49:34<1:26:07, 2.06s/it] 67%|██████▋ | 4994/7500 [2:49:37<1:26:46, 2.08s/it] 67%|██████▋ | 4995/7500 [2:49:39<1:26:49, 2.08s/it] 67%|██████▋ | 4996/7500 [2:49:41<1:27:05, 2.09s/it] 67%|██████▋ | 4997/7500 [2:49:43<1:26:15, 2.07s/it] 67%|██████▋ | 4998/7500 [2:49:45<1:26:41, 2.08s/it] 67%|██████▋ | 4999/7500 [2:49:47<1:25:52, 2.06s/it] 67%|██████▋ | 5000/7500 [2:49:49<1:25:42, 2.06s/it] {'loss': 0.3422, 'grad_norm': 3.229323625564575, 'learning_rate': 1.509800584902108e-05, 'logqs/rejected': -2.361201524734497, 'logqs/chosen': -0.8591989278793335, 'logqs_logps/rejected': -0.026161735877394676, 'logps_logqs/chosen': 0.21303525567054749, 'epoch': 1.33} 67%|██████▋ | 5000/7500 [2:49:49<1:25:42, 2.06s/it] 67%|██████▋ | 5001/7500 [2:49:51<1:25:12, 2.05s/it] 67%|██████▋ | 5002/7500 [2:49:53<1:26:14, 2.07s/it] 67%|██████▋ | 5003/7500 [2:49:55<1:26:27, 2.08s/it] 67%|██████▋ | 5004/7500 [2:49:57<1:27:07, 2.09s/it] 67%|██████▋ | 5005/7500 [2:49:59<1:26:37, 2.08s/it] 67%|██████▋ | 5006/7500 [2:50:02<1:26:37, 2.08s/it] 67%|██████▋ | 5007/7500 [2:50:04<1:26:21, 2.08s/it] 67%|██████▋ | 5008/7500 [2:50:06<1:26:23, 2.08s/it] 67%|██████▋ | 5009/7500 [2:50:08<1:26:41, 2.09s/it] 67%|██████▋ | 5010/7500 [2:50:10<1:26:19, 2.08s/it] {'loss': 0.3779, 'grad_norm': 2.669079542160034, 'learning_rate': 1.4991274186077632e-05, 'logqs/rejected': -2.7332584857940674, 'logqs/chosen': -0.8626919984817505, 'logqs_logps/rejected': -0.01827774941921234, 'logps_logqs/chosen': 0.19895343482494354, 'epoch': 1.34} 67%|██████▋ | 5010/7500 [2:50:10<1:26:19, 2.08s/it] 67%|██████▋ | 5011/7500 [2:50:12<1:25:58, 2.07s/it] 67%|██████▋ | 5012/7500 [2:50:14<1:26:10, 2.08s/it] 67%|██████▋ | 5013/7500 [2:50:16<1:25:11, 2.06s/it] 67%|██████▋ | 5014/7500 [2:50:18<1:25:26, 2.06s/it] 67%|██████▋ | 5015/7500 [2:50:20<1:24:55, 2.05s/it] 67%|██████▋ | 5016/7500 [2:50:22<1:24:14, 2.03s/it] 67%|██████▋ | 5017/7500 [2:50:24<1:24:24, 2.04s/it] 67%|██████▋ | 5018/7500 [2:50:26<1:25:24, 2.06s/it] 67%|██████▋ | 5019/7500 [2:50:28<1:25:59, 2.08s/it] 67%|██████▋ | 5020/7500 [2:50:30<1:25:52, 2.08s/it] {'loss': 0.3568, 'grad_norm': 3.6868770122528076, 'learning_rate': 1.4884759328590475e-05, 'logqs/rejected': -2.6353588104248047, 'logqs/chosen': -0.8308342099189758, 'logqs_logps/rejected': -0.02702728472650051, 'logps_logqs/chosen': 0.20689983665943146, 'epoch': 1.34} 67%|██████▋ | 5020/7500 [2:50:30<1:25:52, 2.08s/it] 67%|██████▋ | 5021/7500 [2:50:32<1:25:20, 2.07s/it] 67%|██████▋ | 5022/7500 [2:50:35<1:24:56, 2.06s/it] 67%|██████▋ | 5023/7500 [2:50:37<1:24:51, 2.06s/it] 67%|██████▋ | 5024/7500 [2:50:39<1:23:51, 2.03s/it] 67%|██████▋ | 5025/7500 [2:50:41<1:24:29, 2.05s/it] 67%|██████▋ | 5026/7500 [2:50:43<1:24:05, 2.04s/it] 67%|██████▋ | 5027/7500 [2:50:45<1:24:31, 2.05s/it] 67%|██████▋ | 5028/7500 [2:50:47<1:24:19, 2.05s/it] 67%|██████▋ | 5029/7500 [2:50:49<1:23:58, 2.04s/it] 67%|██████▋ | 5030/7500 [2:50:51<1:24:16, 2.05s/it] {'loss': 0.3578, 'grad_norm': 3.8822975158691406, 'learning_rate': 1.4778463583846552e-05, 'logqs/rejected': -2.5191593170166016, 'logqs/chosen': -0.8428597450256348, 'logqs_logps/rejected': -0.021615484729409218, 'logps_logqs/chosen': 0.201696515083313, 'epoch': 1.34} 67%|██████▋ | 5030/7500 [2:50:51<1:24:16, 2.05s/it] 67%|██████▋ | 5031/7500 [2:50:53<1:24:32, 2.05s/it] 67%|██████▋ | 5032/7500 [2:50:55<1:24:08, 2.05s/it] 67%|██████▋ | 5033/7500 [2:50:57<1:24:13, 2.05s/it] 67%|██████▋ | 5034/7500 [2:50:59<1:24:38, 2.06s/it] 67%|██████▋ | 5035/7500 [2:51:01<1:25:09, 2.07s/it] 67%|██████▋ | 5036/7500 [2:51:03<1:24:48, 2.07s/it] 67%|██████▋ | 5037/7500 [2:51:05<1:25:03, 2.07s/it] 67%|██████▋ | 5038/7500 [2:51:07<1:25:16, 2.08s/it] 67%|██████▋ | 5039/7500 [2:51:09<1:24:28, 2.06s/it] 67%|██████▋ | 5040/7500 [2:51:12<1:25:18, 2.08s/it] {'loss': 0.3488, 'grad_norm': 3.1574695110321045, 'learning_rate': 1.467238925438646e-05, 'logqs/rejected': -2.4643003940582275, 'logqs/chosen': -0.8579459190368652, 'logqs_logps/rejected': -0.026327019557356834, 'logps_logqs/chosen': 0.20423392951488495, 'epoch': 1.34} 67%|██████▋ | 5040/7500 [2:51:12<1:25:18, 2.08s/it] 67%|██████▋ | 5041/7500 [2:51:14<1:25:16, 2.08s/it] 67%|██████▋ | 5042/7500 [2:51:16<1:24:03, 2.05s/it] 67%|██████▋ | 5043/7500 [2:51:18<1:23:59, 2.05s/it] 67%|██████▋ | 5044/7500 [2:51:20<1:24:37, 2.07s/it] 67%|██████▋ | 5045/7500 [2:51:22<1:23:50, 2.05s/it] 67%|██████▋ | 5046/7500 [2:51:24<1:25:04, 2.08s/it] 67%|██████▋ | 5047/7500 [2:51:26<1:25:10, 2.08s/it] 67%|██████▋ | 5048/7500 [2:51:28<1:23:43, 2.05s/it] 67%|██████▋ | 5049/7500 [2:51:30<1:23:10, 2.04s/it] 67%|██████▋ | 5050/7500 [2:51:32<1:25:32, 2.09s/it] {'loss': 0.3592, 'grad_norm': 3.603691577911377, 'learning_rate': 1.4566538637954554e-05, 'logqs/rejected': -2.7716805934906006, 'logqs/chosen': -0.838468074798584, 'logqs_logps/rejected': -0.02315216325223446, 'logps_logqs/chosen': 0.20356139540672302, 'epoch': 1.35} 67%|██████▋ | 5050/7500 [2:51:32<1:25:32, 2.09s/it] 67%|██████▋ | 5051/7500 [2:51:34<1:24:38, 2.07s/it] 67%|██████▋ | 5052/7500 [2:51:36<1:24:12, 2.06s/it] 67%|██████▋ | 5053/7500 [2:51:38<1:24:18, 2.07s/it] 67%|██████▋ | 5054/7500 [2:51:40<1:23:36, 2.05s/it] 67%|██████▋ | 5055/7500 [2:51:42<1:23:14, 2.04s/it] 67%|██████▋ | 5056/7500 [2:51:44<1:22:50, 2.03s/it] 67%|██████▋ | 5057/7500 [2:51:46<1:23:11, 2.04s/it] 67%|██████▋ | 5058/7500 [2:51:49<1:24:19, 2.07s/it] 67%|██████▋ | 5059/7500 [2:51:51<1:23:38, 2.06s/it] 67%|██████▋ | 5060/7500 [2:51:53<1:23:25, 2.05s/it] {'loss': 0.3549, 'grad_norm': 3.0498664379119873, 'learning_rate': 1.446091402744923e-05, 'logqs/rejected': -2.5364716053009033, 'logqs/chosen': -0.8244959115982056, 'logqs_logps/rejected': -0.022154266014695168, 'logps_logqs/chosen': 0.2050856053829193, 'epoch': 1.35} 67%|██████▋ | 5060/7500 [2:51:53<1:23:25, 2.05s/it] 67%|██████▋ | 5061/7500 [2:51:55<1:23:28, 2.05s/it] 67%|██████▋ | 5062/7500 [2:51:57<1:23:29, 2.05s/it] 68%|██████▊ | 5063/7500 [2:51:59<1:22:37, 2.03s/it] 68%|██████▊ | 5064/7500 [2:52:01<1:22:50, 2.04s/it] 68%|██████▊ | 5065/7500 [2:52:03<1:23:16, 2.05s/it] 68%|██████▊ | 5066/7500 [2:52:05<1:23:33, 2.06s/it] 68%|██████▊ | 5067/7500 [2:52:07<1:24:01, 2.07s/it] 68%|██████▊ | 5068/7500 [2:52:09<1:24:36, 2.09s/it] 68%|██████▊ | 5069/7500 [2:52:11<1:23:42, 2.07s/it] 68%|██████▊ | 5070/7500 [2:52:13<1:24:03, 2.08s/it] {'loss': 0.3625, 'grad_norm': 2.94558048248291, 'learning_rate': 1.4355517710873184e-05, 'logqs/rejected': -2.6858339309692383, 'logqs/chosen': -0.8389512896537781, 'logqs_logps/rejected': -0.028869956731796265, 'logps_logqs/chosen': 0.2060094177722931, 'epoch': 1.35} 68%|██████▊ | 5070/7500 [2:52:13<1:24:03, 2.08s/it] 68%|██████▊ | 5071/7500 [2:52:16<1:25:07, 2.10s/it] 68%|██████▊ | 5072/7500 [2:52:18<1:24:36, 2.09s/it] 68%|██████▊ | 5073/7500 [2:52:20<1:26:56, 2.15s/it] 68%|██████▊ | 5074/7500 [2:52:22<1:27:04, 2.15s/it] 68%|██████▊ | 5075/7500 [2:52:24<1:26:35, 2.14s/it] 68%|██████▊ | 5076/7500 [2:52:26<1:27:28, 2.17s/it] 68%|██████▊ | 5077/7500 [2:52:29<1:27:41, 2.17s/it] 68%|██████▊ | 5078/7500 [2:52:31<1:27:55, 2.18s/it] 68%|██████▊ | 5079/7500 [2:52:33<1:27:12, 2.16s/it] 68%|██████▊ | 5080/7500 [2:52:35<1:27:41, 2.17s/it] {'loss': 0.3619, 'grad_norm': 2.6805765628814697, 'learning_rate': 1.4250351971283937e-05, 'logqs/rejected': -2.5657713413238525, 'logqs/chosen': -0.8649271130561829, 'logqs_logps/rejected': -0.029224559664726257, 'logps_logqs/chosen': 0.202935129404068, 'epoch': 1.35} 68%|██████▊ | 5080/7500 [2:52:35<1:27:41, 2.17s/it] 68%|██████▊ | 5081/7500 [2:52:37<1:26:06, 2.14s/it] 68%|██████▊ | 5082/7500 [2:52:39<1:26:15, 2.14s/it] 68%|██████▊ | 5083/7500 [2:52:41<1:26:52, 2.16s/it] 68%|██████▊ | 5084/7500 [2:52:44<1:26:46, 2.15s/it] 68%|██████▊ | 5085/7500 [2:52:46<1:25:41, 2.13s/it] 68%|██████▊ | 5086/7500 [2:52:48<1:24:28, 2.10s/it] 68%|██████▊ | 5087/7500 [2:52:50<1:23:58, 2.09s/it] 68%|██████▊ | 5088/7500 [2:52:52<1:24:54, 2.11s/it] 68%|██████▊ | 5089/7500 [2:52:54<1:23:26, 2.08s/it] 68%|██████▊ | 5090/7500 [2:52:56<1:23:54, 2.09s/it] {'loss': 0.3417, 'grad_norm': 2.9679391384124756, 'learning_rate': 1.4145419086744299e-05, 'logqs/rejected': -2.315514087677002, 'logqs/chosen': -0.8505709767341614, 'logqs_logps/rejected': -0.028808975592255592, 'logps_logqs/chosen': 0.20197471976280212, 'epoch': 1.36} 68%|██████▊ | 5090/7500 [2:52:56<1:23:54, 2.09s/it] 68%|██████▊ | 5091/7500 [2:52:58<1:22:58, 2.07s/it] 68%|██████▊ | 5092/7500 [2:53:00<1:22:00, 2.04s/it] 68%|██████▊ | 5093/7500 [2:53:02<1:21:51, 2.04s/it] 68%|██████▊ | 5094/7500 [2:53:04<1:22:52, 2.07s/it] 68%|██████▊ | 5095/7500 [2:53:06<1:24:18, 2.10s/it] 68%|██████▊ | 5096/7500 [2:53:09<1:26:07, 2.15s/it] 68%|██████▊ | 5097/7500 [2:53:11<1:26:14, 2.15s/it] 68%|██████▊ | 5098/7500 [2:53:13<1:26:15, 2.15s/it] 68%|██████▊ | 5099/7500 [2:53:15<1:24:58, 2.12s/it] 68%|██████▊ | 5100/7500 [2:53:17<1:25:41, 2.14s/it] {'loss': 0.3396, 'grad_norm': 2.686574935913086, 'learning_rate': 1.4040721330273062e-05, 'logqs/rejected': -2.498471736907959, 'logqs/chosen': -0.8372131586074829, 'logqs_logps/rejected': -0.02440991997718811, 'logps_logqs/chosen': 0.19919699430465698, 'epoch': 1.36} 68%|██████▊ | 5100/7500 [2:53:17<1:25:41, 2.14s/it] 68%|██████▊ | 5101/7500 [2:53:19<1:25:22, 2.14s/it] 68%|██████▊ | 5102/7500 [2:53:21<1:25:05, 2.13s/it] 68%|██████▊ | 5103/7500 [2:53:24<1:25:21, 2.14s/it] 68%|██████▊ | 5104/7500 [2:53:26<1:25:41, 2.15s/it] 68%|██████▊ | 5105/7500 [2:53:28<1:24:44, 2.12s/it] 68%|██████▊ | 5106/7500 [2:53:30<1:24:34, 2.12s/it] 68%|██████▊ | 5107/7500 [2:53:32<1:23:26, 2.09s/it] 68%|██████▊ | 5108/7500 [2:53:34<1:23:50, 2.10s/it] 68%|██████▊ | 5109/7500 [2:53:36<1:23:17, 2.09s/it] 68%|██████▊ | 5110/7500 [2:53:38<1:23:18, 2.09s/it] {'loss': 0.3554, 'grad_norm': 3.021228313446045, 'learning_rate': 1.3936260969795778e-05, 'logqs/rejected': -2.44895076751709, 'logqs/chosen': -0.85847407579422, 'logqs_logps/rejected': -0.02724059857428074, 'logps_logqs/chosen': 0.20854751765727997, 'epoch': 1.36} 68%|██████▊ | 5110/7500 [2:53:38<1:23:18, 2.09s/it] 68%|██████▊ | 5111/7500 [2:53:40<1:22:54, 2.08s/it] 68%|██████▊ | 5112/7500 [2:53:42<1:22:47, 2.08s/it] 68%|██████▊ | 5113/7500 [2:53:45<1:23:28, 2.10s/it] 68%|██████▊ | 5114/7500 [2:53:47<1:23:21, 2.10s/it] 68%|██████▊ | 5115/7500 [2:53:49<1:22:32, 2.08s/it] 68%|██████▊ | 5116/7500 [2:53:51<1:22:19, 2.07s/it] 68%|██████▊ | 5117/7500 [2:53:53<1:23:34, 2.10s/it] 68%|██████▊ | 5118/7500 [2:53:55<1:22:28, 2.08s/it] 68%|██████▊ | 5119/7500 [2:53:57<1:23:38, 2.11s/it] 68%|██████▊ | 5120/7500 [2:53:59<1:24:36, 2.13s/it] {'loss': 0.3381, 'grad_norm': 2.781031847000122, 'learning_rate': 1.383204026809559e-05, 'logqs/rejected': -2.2488999366760254, 'logqs/chosen': -0.8622024655342102, 'logqs_logps/rejected': -0.03476809710264206, 'logps_logqs/chosen': 0.20300018787384033, 'epoch': 1.37} 68%|██████▊ | 5120/7500 [2:53:59<1:24:36, 2.13s/it] 68%|██████▊ | 5121/7500 [2:54:01<1:24:26, 2.13s/it] 68%|██████▊ | 5122/7500 [2:54:03<1:23:26, 2.11s/it] 68%|██████▊ | 5123/7500 [2:54:06<1:23:03, 2.10s/it] 68%|██████▊ | 5124/7500 [2:54:08<1:24:09, 2.13s/it] 68%|██████▊ | 5125/7500 [2:54:10<1:24:09, 2.13s/it] 68%|██████▊ | 5126/7500 [2:54:12<1:22:56, 2.10s/it] 68%|██████▊ | 5127/7500 [2:54:14<1:23:21, 2.11s/it] 68%|██████▊ | 5128/7500 [2:54:16<1:24:30, 2.14s/it] 68%|██████▊ | 5129/7500 [2:54:18<1:24:34, 2.14s/it] 68%|██████▊ | 5130/7500 [2:54:21<1:25:24, 2.16s/it] {'loss': 0.34, 'grad_norm': 3.0824854373931885, 'learning_rate': 1.3728061482764238e-05, 'logqs/rejected': -2.287252187728882, 'logqs/chosen': -0.8139087557792664, 'logqs_logps/rejected': -0.02946271374821663, 'logps_logqs/chosen': 0.20045645534992218, 'epoch': 1.37} 68%|██████▊ | 5130/7500 [2:54:21<1:25:24, 2.16s/it] 68%|██████▊ | 5131/7500 [2:54:23<1:23:47, 2.12s/it] 68%|██████▊ | 5132/7500 [2:54:25<1:22:17, 2.08s/it] 68%|██████▊ | 5133/7500 [2:54:27<1:24:43, 2.15s/it] 68%|██████▊ | 5134/7500 [2:54:29<1:25:19, 2.16s/it] 68%|██████▊ | 5135/7500 [2:54:31<1:24:06, 2.13s/it] 68%|██████▊ | 5136/7500 [2:54:33<1:24:12, 2.14s/it] 68%|██████▊ | 5137/7500 [2:54:35<1:24:38, 2.15s/it] 69%|██████▊ | 5138/7500 [2:54:38<1:24:55, 2.16s/it] 69%|██████▊ | 5139/7500 [2:54:40<1:24:48, 2.16s/it] 69%|██████▊ | 5140/7500 [2:54:42<1:23:47, 2.13s/it] {'loss': 0.3322, 'grad_norm': 2.668611764907837, 'learning_rate': 1.3624326866153158e-05, 'logqs/rejected': -2.4446969032287598, 'logqs/chosen': -0.8057644963264465, 'logqs_logps/rejected': -0.02681758441030979, 'logps_logqs/chosen': 0.19881384074687958, 'epoch': 1.37} 69%|██████▊ | 5140/7500 [2:54:42<1:23:47, 2.13s/it] 69%|██████▊ | 5141/7500 [2:54:44<1:24:23, 2.15s/it] 69%|██████▊ | 5142/7500 [2:54:46<1:24:01, 2.14s/it] 69%|██████▊ | 5143/7500 [2:54:48<1:25:35, 2.18s/it] 69%|██████▊ | 5144/7500 [2:54:51<1:25:37, 2.18s/it] 69%|██████▊ | 5145/7500 [2:54:53<1:24:12, 2.15s/it] 69%|██████▊ | 5146/7500 [2:54:55<1:24:30, 2.15s/it] 69%|██████▊ | 5147/7500 [2:54:57<1:23:58, 2.14s/it] 69%|██████▊ | 5148/7500 [2:54:59<1:23:04, 2.12s/it] 69%|██████▊ | 5149/7500 [2:55:01<1:22:40, 2.11s/it] 69%|██████▊ | 5150/7500 [2:55:03<1:22:25, 2.10s/it] {'loss': 0.3434, 'grad_norm': 2.7600622177124023, 'learning_rate': 1.3520838665324703e-05, 'logqs/rejected': -2.3223586082458496, 'logqs/chosen': -0.8506800532341003, 'logqs_logps/rejected': -0.027146482840180397, 'logps_logqs/chosen': 0.20685556530952454, 'epoch': 1.37} 69%|██████▊ | 5150/7500 [2:55:03<1:22:25, 2.10s/it] 69%|██████▊ | 5151/7500 [2:55:05<1:21:43, 2.09s/it] 69%|██████▊ | 5152/7500 [2:55:07<1:20:34, 2.06s/it] 69%|██████▊ | 5153/7500 [2:55:09<1:20:10, 2.05s/it] 69%|██████▊ | 5154/7500 [2:55:12<1:21:48, 2.09s/it] 69%|██████▊ | 5155/7500 [2:55:14<1:20:34, 2.06s/it] 69%|██████▊ | 5156/7500 [2:55:16<1:20:28, 2.06s/it] 69%|██████▉ | 5157/7500 [2:55:18<1:20:46, 2.07s/it] 69%|██████▉ | 5158/7500 [2:55:20<1:21:06, 2.08s/it] 69%|██████▉ | 5159/7500 [2:55:22<1:20:41, 2.07s/it] 69%|██████▉ | 5160/7500 [2:55:24<1:21:13, 2.08s/it] {'loss': 0.3553, 'grad_norm': 3.030883312225342, 'learning_rate': 1.3417599122003464e-05, 'logqs/rejected': -2.549666166305542, 'logqs/chosen': -0.8394668698310852, 'logqs_logps/rejected': -0.017158418893814087, 'logps_logqs/chosen': 0.202271968126297, 'epoch': 1.38} 69%|██████▉ | 5160/7500 [2:55:24<1:21:13, 2.08s/it] 69%|██████▉ | 5161/7500 [2:55:26<1:21:12, 2.08s/it] 69%|██████▉ | 5162/7500 [2:55:28<1:22:05, 2.11s/it] 69%|██████▉ | 5163/7500 [2:55:30<1:21:57, 2.10s/it] 69%|██████▉ | 5164/7500 [2:55:32<1:21:05, 2.08s/it] 69%|██████▉ | 5165/7500 [2:55:34<1:20:54, 2.08s/it] 69%|██████▉ | 5166/7500 [2:55:36<1:21:12, 2.09s/it] 69%|██████▉ | 5167/7500 [2:55:39<1:20:50, 2.08s/it] 69%|██████▉ | 5168/7500 [2:55:41<1:20:54, 2.08s/it] 69%|██████▉ | 5169/7500 [2:55:43<1:19:48, 2.05s/it] 69%|██████▉ | 5170/7500 [2:55:45<1:19:49, 2.06s/it] {'loss': 0.3548, 'grad_norm': 2.826030969619751, 'learning_rate': 1.3314610472527645e-05, 'logqs/rejected': -2.6083009243011475, 'logqs/chosen': -0.8211549520492554, 'logqs_logps/rejected': -0.029617836698889732, 'logps_logqs/chosen': 0.2025901824235916, 'epoch': 1.38} 69%|██████▉ | 5170/7500 [2:55:45<1:19:49, 2.06s/it] 69%|██████▉ | 5171/7500 [2:55:47<1:19:47, 2.06s/it] 69%|██████▉ | 5172/7500 [2:55:49<1:18:58, 2.04s/it] 69%|██████▉ | 5173/7500 [2:55:51<1:19:46, 2.06s/it] 69%|██████▉ | 5174/7500 [2:55:53<1:19:26, 2.05s/it] 69%|██████▉ | 5175/7500 [2:55:55<1:20:24, 2.08s/it] 69%|██████▉ | 5176/7500 [2:55:57<1:19:36, 2.06s/it] 69%|██████▉ | 5177/7500 [2:55:59<1:20:13, 2.07s/it] 69%|██████▉ | 5178/7500 [2:56:01<1:20:27, 2.08s/it] 69%|██████▉ | 5179/7500 [2:56:03<1:20:48, 2.09s/it] 69%|██████▉ | 5180/7500 [2:56:05<1:20:28, 2.08s/it] {'loss': 0.3461, 'grad_norm': 3.23331618309021, 'learning_rate': 1.3211874947800746e-05, 'logqs/rejected': -2.478942394256592, 'logqs/chosen': -0.8389862179756165, 'logqs_logps/rejected': -0.026245778426527977, 'logps_logqs/chosen': 0.2010921686887741, 'epoch': 1.38} 69%|██████▉ | 5180/7500 [2:56:05<1:20:28, 2.08s/it] 69%|██████▉ | 5181/7500 [2:56:07<1:19:52, 2.07s/it] 69%|██████▉ | 5182/7500 [2:56:09<1:19:14, 2.05s/it] 69%|██████▉ | 5183/7500 [2:56:11<1:19:16, 2.05s/it] 69%|██████▉ | 5184/7500 [2:56:14<1:18:56, 2.05s/it] 69%|██████▉ | 5185/7500 [2:56:16<1:18:54, 2.05s/it] 69%|██████▉ | 5186/7500 [2:56:18<1:19:26, 2.06s/it] 69%|██████▉ | 5187/7500 [2:56:20<1:19:53, 2.07s/it] 69%|██████▉ | 5188/7500 [2:56:22<1:19:15, 2.06s/it] 69%|██████▉ | 5189/7500 [2:56:24<1:19:01, 2.05s/it] 69%|██████▉ | 5190/7500 [2:56:26<1:19:44, 2.07s/it] {'loss': 0.3547, 'grad_norm': 2.6887712478637695, 'learning_rate': 1.3109394773243117e-05, 'logqs/rejected': -2.510784864425659, 'logqs/chosen': -0.8685073852539062, 'logqs_logps/rejected': -0.027989987283945084, 'logps_logqs/chosen': 0.20216719806194305, 'epoch': 1.38} 69%|██████▉ | 5190/7500 [2:56:26<1:19:44, 2.07s/it] 69%|██████▉ | 5191/7500 [2:56:28<1:19:30, 2.07s/it] 69%|██████▉ | 5192/7500 [2:56:30<1:19:05, 2.06s/it] 69%|██████▉ | 5193/7500 [2:56:32<1:20:23, 2.09s/it] 69%|██████▉ | 5194/7500 [2:56:34<1:21:28, 2.12s/it] 69%|██████▉ | 5195/7500 [2:56:37<1:21:52, 2.13s/it] 69%|██████▉ | 5196/7500 [2:56:39<1:21:34, 2.12s/it] 69%|██████▉ | 5197/7500 [2:56:41<1:21:37, 2.13s/it] 69%|██████▉ | 5198/7500 [2:56:43<1:20:30, 2.10s/it] 69%|██████▉ | 5199/7500 [2:56:45<1:21:35, 2.13s/it] 69%|██████▉ | 5200/7500 [2:56:47<1:20:33, 2.10s/it] {'loss': 0.3566, 'grad_norm': 3.0980772972106934, 'learning_rate': 1.3007172168743854e-05, 'logqs/rejected': -2.404528856277466, 'logqs/chosen': -0.8493932485580444, 'logqs_logps/rejected': -0.04071410000324249, 'logps_logqs/chosen': 0.20743879675865173, 'epoch': 1.39} 69%|██████▉ | 5200/7500 [2:56:47<1:20:33, 2.10s/it] 69%|██████▉ | 5201/7500 [2:56:49<1:20:36, 2.10s/it] 69%|██████▉ | 5202/7500 [2:56:51<1:20:14, 2.09s/it] 69%|██████▉ | 5203/7500 [2:56:53<1:20:42, 2.11s/it] 69%|██████▉ | 5204/7500 [2:56:55<1:20:28, 2.10s/it] 69%|██████▉ | 5205/7500 [2:56:57<1:19:13, 2.07s/it] 69%|██████▉ | 5206/7500 [2:57:00<1:19:44, 2.09s/it] 69%|██████▉ | 5207/7500 [2:57:02<1:19:51, 2.09s/it] 69%|██████▉ | 5208/7500 [2:57:04<1:19:14, 2.07s/it] 69%|██████▉ | 5209/7500 [2:57:06<1:18:04, 2.04s/it] 69%|██████▉ | 5210/7500 [2:57:08<1:17:43, 2.04s/it] {'loss': 0.36, 'grad_norm': 2.5993967056274414, 'learning_rate': 1.2905209348612596e-05, 'logqs/rejected': -2.533586025238037, 'logqs/chosen': -0.8425617218017578, 'logqs_logps/rejected': -0.020232515409588814, 'logps_logqs/chosen': 0.20274671912193298, 'epoch': 1.39} 69%|██████▉ | 5210/7500 [2:57:08<1:17:43, 2.04s/it] 69%|██████▉ | 5211/7500 [2:57:10<1:17:52, 2.04s/it] 69%|██████▉ | 5212/7500 [2:57:12<1:18:19, 2.05s/it] 70%|██████▉ | 5213/7500 [2:57:14<1:18:29, 2.06s/it] 70%|██████▉ | 5214/7500 [2:57:16<1:18:15, 2.05s/it] 70%|██████▉ | 5215/7500 [2:57:18<1:18:38, 2.06s/it] 70%|██████▉ | 5216/7500 [2:57:20<1:18:45, 2.07s/it] 70%|██████▉ | 5217/7500 [2:57:22<1:18:32, 2.06s/it] 70%|██████▉ | 5218/7500 [2:57:24<1:18:43, 2.07s/it] 70%|██████▉ | 5219/7500 [2:57:26<1:18:48, 2.07s/it] 70%|██████▉ | 5220/7500 [2:57:28<1:19:41, 2.10s/it] {'loss': 0.374, 'grad_norm': 3.0589401721954346, 'learning_rate': 1.280350852153168e-05, 'logqs/rejected': -2.6498045921325684, 'logqs/chosen': -0.8369390368461609, 'logqs_logps/rejected': -0.025897327810525894, 'logps_logqs/chosen': 0.20771677792072296, 'epoch': 1.39} 70%|██████▉ | 5220/7500 [2:57:29<1:19:41, 2.10s/it] 70%|██████▉ | 5221/7500 [2:57:30<1:18:39, 2.07s/it] 70%|██████▉ | 5222/7500 [2:57:33<1:18:15, 2.06s/it] 70%|██████▉ | 5223/7500 [2:57:35<1:18:43, 2.07s/it] 70%|██████▉ | 5224/7500 [2:57:37<1:18:25, 2.07s/it] 70%|██████▉ | 5225/7500 [2:57:39<1:19:27, 2.10s/it] 70%|██████▉ | 5226/7500 [2:57:41<1:18:48, 2.08s/it] 70%|██████▉ | 5227/7500 [2:57:43<1:18:54, 2.08s/it] 70%|██████▉ | 5228/7500 [2:57:45<1:18:57, 2.09s/it] 70%|██████▉ | 5229/7500 [2:57:47<1:18:43, 2.08s/it] 70%|██████▉ | 5230/7500 [2:57:49<1:17:38, 2.05s/it] {'loss': 0.3446, 'grad_norm': 2.801726818084717, 'learning_rate': 1.2702071890508235e-05, 'logqs/rejected': -2.5364925861358643, 'logqs/chosen': -0.8234828114509583, 'logqs_logps/rejected': -0.027336787432432175, 'logps_logqs/chosen': 0.19949501752853394, 'epoch': 1.39} 70%|██████▉ | 5230/7500 [2:57:49<1:17:38, 2.05s/it] 70%|██████▉ | 5231/7500 [2:57:51<1:20:34, 2.13s/it] 70%|██████▉ | 5232/7500 [2:57:53<1:19:35, 2.11s/it] 70%|██████▉ | 5233/7500 [2:57:55<1:18:27, 2.08s/it] 70%|██████▉ | 5234/7500 [2:57:58<1:18:09, 2.07s/it] 70%|██████▉ | 5235/7500 [2:58:00<1:18:47, 2.09s/it] 70%|██████▉ | 5236/7500 [2:58:02<1:17:55, 2.07s/it] 70%|██████▉ | 5237/7500 [2:58:04<1:18:34, 2.08s/it] 70%|██████▉ | 5238/7500 [2:58:06<1:17:18, 2.05s/it] 70%|██████▉ | 5239/7500 [2:58:08<1:17:17, 2.05s/it] 70%|██████▉ | 5240/7500 [2:58:10<1:17:23, 2.05s/it] {'loss': 0.3458, 'grad_norm': 3.1226999759674072, 'learning_rate': 1.2600901652826449e-05, 'logqs/rejected': -2.341060161590576, 'logqs/chosen': -0.8353041410446167, 'logqs_logps/rejected': -0.02987058088183403, 'logps_logqs/chosen': 0.20672956109046936, 'epoch': 1.4} 70%|██████▉ | 5240/7500 [2:58:10<1:17:23, 2.05s/it] 70%|██████▉ | 5241/7500 [2:58:12<1:17:23, 2.06s/it] 70%|██████▉ | 5242/7500 [2:58:14<1:17:37, 2.06s/it] 70%|██████▉ | 5243/7500 [2:58:16<1:17:03, 2.05s/it] 70%|██████▉ | 5244/7500 [2:58:18<1:16:44, 2.04s/it] 70%|██████▉ | 5245/7500 [2:58:20<1:17:29, 2.06s/it] 70%|██████▉ | 5246/7500 [2:58:22<1:17:38, 2.07s/it] 70%|██████▉ | 5247/7500 [2:58:24<1:17:56, 2.08s/it] 70%|██████▉ | 5248/7500 [2:58:27<1:19:12, 2.11s/it] 70%|██████▉ | 5249/7500 [2:58:29<1:18:47, 2.10s/it] 70%|███████ | 5250/7500 [2:58:31<1:18:01, 2.08s/it] {'loss': 0.3539, 'grad_norm': 3.3996264934539795, 'learning_rate': 1.2500000000000006e-05, 'logqs/rejected': -2.67035174369812, 'logqs/chosen': -0.8305028080940247, 'logqs_logps/rejected': -0.023318089544773102, 'logps_logqs/chosen': 0.19734282791614532, 'epoch': 1.4} 70%|███████ | 5250/7500 [2:58:31<1:18:01, 2.08s/it] 70%|███████ | 5251/7500 [2:58:33<1:17:14, 2.06s/it] 70%|███████ | 5252/7500 [2:58:35<1:17:09, 2.06s/it] 70%|███████ | 5253/7500 [2:58:37<1:18:32, 2.10s/it] 70%|███████ | 5254/7500 [2:58:39<1:18:56, 2.11s/it] 70%|███████ | 5255/7500 [2:58:41<1:19:25, 2.12s/it] 70%|███████ | 5256/7500 [2:58:43<1:18:52, 2.11s/it] 70%|███████ | 5257/7500 [2:58:45<1:18:54, 2.11s/it] 70%|███████ | 5258/7500 [2:58:47<1:17:59, 2.09s/it] 70%|███████ | 5259/7500 [2:58:50<1:17:59, 2.09s/it] 70%|███████ | 5260/7500 [2:58:52<1:17:49, 2.08s/it] {'loss': 0.3538, 'grad_norm': 2.85140323638916, 'learning_rate': 1.2399369117724583e-05, 'logqs/rejected': -2.458585739135742, 'logqs/chosen': -0.8405072093009949, 'logqs_logps/rejected': -0.02432854101061821, 'logps_logqs/chosen': 0.20197102427482605, 'epoch': 1.4} 70%|███████ | 5260/7500 [2:58:52<1:17:49, 2.08s/it] 70%|███████ | 5261/7500 [2:58:54<1:18:30, 2.10s/it] 70%|███████ | 5262/7500 [2:58:56<1:19:28, 2.13s/it] 70%|███████ | 5263/7500 [2:58:58<1:20:01, 2.15s/it] 70%|███████ | 5264/7500 [2:59:00<1:19:13, 2.13s/it] 70%|███████ | 5265/7500 [2:59:02<1:18:27, 2.11s/it] 70%|███████ | 5266/7500 [2:59:04<1:18:29, 2.11s/it] 70%|███████ | 5267/7500 [2:59:06<1:17:16, 2.08s/it] 70%|███████ | 5268/7500 [2:59:09<1:18:28, 2.11s/it] 70%|███████ | 5269/7500 [2:59:11<1:18:05, 2.10s/it] 70%|███████ | 5270/7500 [2:59:13<1:17:50, 2.09s/it] {'loss': 0.3621, 'grad_norm': 2.7420129776000977, 'learning_rate': 1.2299011185830555e-05, 'logqs/rejected': -2.527421236038208, 'logqs/chosen': -0.8507786989212036, 'logqs_logps/rejected': -0.025431090965867043, 'logps_logqs/chosen': 0.20220620930194855, 'epoch': 1.41} 70%|███████ | 5270/7500 [2:59:13<1:17:50, 2.09s/it] 70%|███████ | 5271/7500 [2:59:15<1:17:46, 2.09s/it] 70%|███████ | 5272/7500 [2:59:17<1:17:36, 2.09s/it] 70%|███████ | 5273/7500 [2:59:19<1:18:25, 2.11s/it] 70%|███████ | 5274/7500 [2:59:21<1:18:51, 2.13s/it] 70%|███████ | 5275/7500 [2:59:23<1:17:52, 2.10s/it] 70%|███████ | 5276/7500 [2:59:25<1:18:53, 2.13s/it] 70%|███████ | 5277/7500 [2:59:28<1:19:22, 2.14s/it] 70%|███████ | 5278/7500 [2:59:30<1:19:00, 2.13s/it] 70%|███████ | 5279/7500 [2:59:32<1:19:54, 2.16s/it] 70%|███████ | 5280/7500 [2:59:34<1:18:32, 2.12s/it] {'loss': 0.3683, 'grad_norm': 3.0378458499908447, 'learning_rate': 1.2198928378235716e-05, 'logqs/rejected': -2.580916166305542, 'logqs/chosen': -0.8394020795822144, 'logqs_logps/rejected': -0.023015037178993225, 'logps_logqs/chosen': 0.2091611623764038, 'epoch': 1.41} 70%|███████ | 5280/7500 [2:59:34<1:18:32, 2.12s/it] 70%|███████ | 5281/7500 [2:59:36<1:18:31, 2.12s/it] 70%|███████ | 5282/7500 [2:59:38<1:18:05, 2.11s/it] 70%|███████ | 5283/7500 [2:59:40<1:18:35, 2.13s/it] 70%|███████ | 5284/7500 [2:59:42<1:17:50, 2.11s/it] 70%|███████ | 5285/7500 [2:59:45<1:17:54, 2.11s/it] 70%|███████ | 5286/7500 [2:59:47<1:16:46, 2.08s/it] 70%|███████ | 5287/7500 [2:59:49<1:16:46, 2.08s/it] 71%|███████ | 5288/7500 [2:59:51<1:16:47, 2.08s/it] 71%|███████ | 5289/7500 [2:59:53<1:17:05, 2.09s/it] 71%|███████ | 5290/7500 [2:59:55<1:15:50, 2.06s/it] {'loss': 0.3527, 'grad_norm': 2.6408746242523193, 'learning_rate': 1.2099122862898212e-05, 'logqs/rejected': -2.453601360321045, 'logqs/chosen': -0.8338179588317871, 'logqs_logps/rejected': -0.028936531394720078, 'logps_logqs/chosen': 0.20441563427448273, 'epoch': 1.41} 71%|███████ | 5290/7500 [2:59:55<1:15:50, 2.06s/it] 71%|███████ | 5291/7500 [2:59:57<1:15:47, 2.06s/it] 71%|███████ | 5292/7500 [2:59:59<1:15:38, 2.06s/it] 71%|███████ | 5293/7500 [3:00:01<1:16:01, 2.07s/it] 71%|███████ | 5294/7500 [3:00:03<1:16:22, 2.08s/it] 71%|███████ | 5295/7500 [3:00:05<1:16:56, 2.09s/it] 71%|███████ | 5296/7500 [3:00:07<1:16:07, 2.07s/it] 71%|███████ | 5297/7500 [3:00:09<1:16:55, 2.10s/it] 71%|███████ | 5298/7500 [3:00:12<1:17:50, 2.12s/it] 71%|███████ | 5299/7500 [3:00:14<1:17:30, 2.11s/it] 71%|███████ | 5300/7500 [3:00:16<1:17:06, 2.10s/it] {'loss': 0.3596, 'grad_norm': 2.465662717819214, 'learning_rate': 1.1999596801769616e-05, 'logqs/rejected': -2.648977279663086, 'logqs/chosen': -0.8543775677680969, 'logqs_logps/rejected': -0.02111494168639183, 'logps_logqs/chosen': 0.20367877185344696, 'epoch': 1.41} 71%|███████ | 5300/7500 [3:00:16<1:17:06, 2.10s/it] 71%|███████ | 5301/7500 [3:00:18<1:16:56, 2.10s/it] 71%|███████ | 5302/7500 [3:00:20<1:17:04, 2.10s/it] 71%|███████ | 5303/7500 [3:00:22<1:17:24, 2.11s/it] 71%|███████ | 5304/7500 [3:00:24<1:16:34, 2.09s/it] 71%|███████ | 5305/7500 [3:00:26<1:15:57, 2.08s/it] 71%|███████ | 5306/7500 [3:00:28<1:15:10, 2.06s/it] 71%|███████ | 5307/7500 [3:00:30<1:16:01, 2.08s/it] 71%|███████ | 5308/7500 [3:00:32<1:16:05, 2.08s/it] 71%|███████ | 5309/7500 [3:00:35<1:17:11, 2.11s/it] 71%|███████ | 5310/7500 [3:00:37<1:17:01, 2.11s/it] {'loss': 0.3595, 'grad_norm': 3.4041099548339844, 'learning_rate': 1.1900352350748026e-05, 'logqs/rejected': -2.576845645904541, 'logqs/chosen': -0.8680912256240845, 'logqs_logps/rejected': -0.0235548447817564, 'logps_logqs/chosen': 0.20993945002555847, 'epoch': 1.42} 71%|███████ | 5310/7500 [3:00:37<1:17:01, 2.11s/it] 71%|███████ | 5311/7500 [3:00:39<1:16:30, 2.10s/it] 71%|███████ | 5312/7500 [3:00:41<1:15:47, 2.08s/it] 71%|███████ | 5313/7500 [3:00:43<1:15:54, 2.08s/it] 71%|███████ | 5314/7500 [3:00:45<1:14:43, 2.05s/it] 71%|███████ | 5315/7500 [3:00:47<1:15:11, 2.06s/it] 71%|███████ | 5316/7500 [3:00:49<1:15:46, 2.08s/it] 71%|███████ | 5317/7500 [3:00:51<1:16:19, 2.10s/it] 71%|███████ | 5318/7500 [3:00:53<1:15:07, 2.07s/it] 71%|███████ | 5319/7500 [3:00:55<1:15:49, 2.09s/it] 71%|███████ | 5320/7500 [3:00:57<1:15:16, 2.07s/it] {'loss': 0.3559, 'grad_norm': 2.670797348022461, 'learning_rate': 1.1801391659631423e-05, 'logqs/rejected': -2.580636501312256, 'logqs/chosen': -0.8367868661880493, 'logqs_logps/rejected': -0.020837435498833656, 'logps_logqs/chosen': 0.20796163380146027, 'epoch': 1.42} 71%|███████ | 5320/7500 [3:00:57<1:15:16, 2.07s/it] 71%|███████ | 5321/7500 [3:01:00<1:15:50, 2.09s/it] 71%|███████ | 5322/7500 [3:01:02<1:17:03, 2.12s/it] 71%|███████ | 5323/7500 [3:01:04<1:16:36, 2.11s/it] 71%|███████ | 5324/7500 [3:01:06<1:16:31, 2.11s/it] 71%|███████ | 5325/7500 [3:01:08<1:16:02, 2.10s/it] 71%|███████ | 5326/7500 [3:01:10<1:15:44, 2.09s/it] 71%|███████ | 5327/7500 [3:01:12<1:16:20, 2.11s/it] 71%|███████ | 5328/7500 [3:01:14<1:16:22, 2.11s/it] 71%|███████ | 5329/7500 [3:01:17<1:17:16, 2.14s/it] 71%|███████ | 5330/7500 [3:01:19<1:16:47, 2.12s/it] {'loss': 0.3758, 'grad_norm': 2.6813554763793945, 'learning_rate': 1.170271687207106e-05, 'logqs/rejected': -2.6190149784088135, 'logqs/chosen': -0.8810839653015137, 'logqs_logps/rejected': -0.02210645191371441, 'logps_logqs/chosen': 0.2051519900560379, 'epoch': 1.42} 71%|███████ | 5330/7500 [3:01:19<1:16:47, 2.12s/it] 71%|███████ | 5331/7500 [3:01:21<1:15:50, 2.10s/it] 71%|███████ | 5332/7500 [3:01:23<1:15:41, 2.09s/it] 71%|███████ | 5333/7500 [3:01:25<1:16:23, 2.11s/it] 71%|███████ | 5334/7500 [3:01:27<1:15:50, 2.10s/it] 71%|███████ | 5335/7500 [3:01:29<1:15:45, 2.10s/it] 71%|███████ | 5336/7500 [3:01:31<1:16:17, 2.12s/it] 71%|███████ | 5337/7500 [3:01:33<1:15:39, 2.10s/it] 71%|███████ | 5338/7500 [3:01:35<1:14:27, 2.07s/it] 71%|███████ | 5339/7500 [3:01:37<1:13:28, 2.04s/it] 71%|███████ | 5340/7500 [3:01:39<1:13:17, 2.04s/it] {'loss': 0.3448, 'grad_norm': 2.547996997833252, 'learning_rate': 1.1604330125525079e-05, 'logqs/rejected': -2.358055591583252, 'logqs/chosen': -0.8588920831680298, 'logqs_logps/rejected': -0.025470290333032608, 'logps_logqs/chosen': 0.20651695132255554, 'epoch': 1.42} 71%|███████ | 5340/7500 [3:01:39<1:13:17, 2.04s/it] 71%|███████ | 5341/7500 [3:01:41<1:13:49, 2.05s/it] 71%|███████ | 5342/7500 [3:01:44<1:14:47, 2.08s/it] 71%|███████ | 5343/7500 [3:01:46<1:14:28, 2.07s/it] 71%|███████▏ | 5344/7500 [3:01:48<1:15:39, 2.11s/it] 71%|███████▏ | 5345/7500 [3:01:50<1:15:12, 2.09s/it] 71%|███████▏ | 5346/7500 [3:01:52<1:14:47, 2.08s/it] 71%|███████▏ | 5347/7500 [3:01:54<1:14:26, 2.07s/it] 71%|███████▏ | 5348/7500 [3:01:56<1:14:16, 2.07s/it] 71%|███████▏ | 5349/7500 [3:01:58<1:13:47, 2.06s/it] 71%|███████▏ | 5350/7500 [3:02:00<1:13:06, 2.04s/it] {'loss': 0.3354, 'grad_norm': 2.7659785747528076, 'learning_rate': 1.1506233551212186e-05, 'logqs/rejected': -2.3491358757019043, 'logqs/chosen': -0.8451981544494629, 'logqs_logps/rejected': -0.027206826955080032, 'logps_logqs/chosen': 0.20814481377601624, 'epoch': 1.43} 71%|███████▏ | 5350/7500 [3:02:00<1:13:06, 2.04s/it] 71%|███████▏ | 5351/7500 [3:02:02<1:14:32, 2.08s/it] 71%|███████▏ | 5352/7500 [3:02:04<1:15:06, 2.10s/it] 71%|███████▏ | 5353/7500 [3:02:06<1:15:17, 2.10s/it] 71%|███████▏ | 5354/7500 [3:02:09<1:15:00, 2.10s/it] 71%|███████▏ | 5355/7500 [3:02:11<1:14:36, 2.09s/it] 71%|███████▏ | 5356/7500 [3:02:13<1:14:38, 2.09s/it] 71%|███████▏ | 5357/7500 [3:02:15<1:15:17, 2.11s/it] 71%|███████▏ | 5358/7500 [3:02:17<1:14:59, 2.10s/it] 71%|███████▏ | 5359/7500 [3:02:19<1:14:45, 2.10s/it] 71%|███████▏ | 5360/7500 [3:02:21<1:13:55, 2.07s/it] {'loss': 0.3599, 'grad_norm': 2.6204206943511963, 'learning_rate': 1.1408429274065418e-05, 'logqs/rejected': -2.5670764446258545, 'logqs/chosen': -0.828438937664032, 'logqs_logps/rejected': -0.024392224848270416, 'logps_logqs/chosen': 0.19995437562465668, 'epoch': 1.43} 71%|███████▏ | 5360/7500 [3:02:21<1:13:55, 2.07s/it] 71%|███████▏ | 5361/7500 [3:02:23<1:13:38, 2.07s/it] 71%|███████▏ | 5362/7500 [3:02:25<1:13:21, 2.06s/it] 72%|███████▏ | 5363/7500 [3:02:27<1:13:09, 2.05s/it] 72%|███████▏ | 5364/7500 [3:02:29<1:13:54, 2.08s/it] 72%|███████▏ | 5365/7500 [3:02:31<1:13:42, 2.07s/it] 72%|███████▏ | 5366/7500 [3:02:33<1:13:33, 2.07s/it] 72%|███████▏ | 5367/7500 [3:02:35<1:12:46, 2.05s/it] 72%|███████▏ | 5368/7500 [3:02:38<1:12:58, 2.05s/it] 72%|███████▏ | 5369/7500 [3:02:40<1:13:03, 2.06s/it] 72%|███████▏ | 5370/7500 [3:02:42<1:13:25, 2.07s/it] {'loss': 0.3585, 'grad_norm': 3.021162509918213, 'learning_rate': 1.1310919412686247e-05, 'logqs/rejected': -2.5128610134124756, 'logqs/chosen': -0.8290001153945923, 'logqs_logps/rejected': -0.024168586358428, 'logps_logqs/chosen': 0.20018692314624786, 'epoch': 1.43} 72%|███████▏ | 5370/7500 [3:02:42<1:13:25, 2.07s/it] 72%|███████▏ | 5371/7500 [3:02:44<1:13:49, 2.08s/it] 72%|███████▏ | 5372/7500 [3:02:46<1:14:22, 2.10s/it] 72%|███████▏ | 5373/7500 [3:02:48<1:14:05, 2.09s/it] 72%|███████▏ | 5374/7500 [3:02:50<1:13:45, 2.08s/it] 72%|███████▏ | 5375/7500 [3:02:52<1:14:07, 2.09s/it] 72%|███████▏ | 5376/7500 [3:02:54<1:13:18, 2.07s/it] 72%|███████▏ | 5377/7500 [3:02:56<1:12:40, 2.05s/it] 72%|███████▏ | 5378/7500 [3:02:58<1:12:58, 2.06s/it] 72%|███████▏ | 5379/7500 [3:03:00<1:11:54, 2.03s/it] 72%|███████▏ | 5380/7500 [3:03:02<1:12:48, 2.06s/it] {'loss': 0.3576, 'grad_norm': 2.789613723754883, 'learning_rate': 1.1213706079298566e-05, 'logqs/rejected': -2.481140375137329, 'logqs/chosen': -0.830984890460968, 'logqs_logps/rejected': -0.02280903235077858, 'logps_logqs/chosen': 0.20895984768867493, 'epoch': 1.43} 72%|███████▏ | 5380/7500 [3:03:02<1:12:48, 2.06s/it] 72%|███████▏ | 5381/7500 [3:03:04<1:12:23, 2.05s/it] 72%|███████▏ | 5382/7500 [3:03:06<1:11:55, 2.04s/it] 72%|███████▏ | 5383/7500 [3:03:08<1:12:09, 2.05s/it] 72%|███████▏ | 5384/7500 [3:03:11<1:13:08, 2.07s/it] 72%|███████▏ | 5385/7500 [3:03:13<1:13:42, 2.09s/it] 72%|███████▏ | 5386/7500 [3:03:15<1:13:08, 2.08s/it] 72%|███████▏ | 5387/7500 [3:03:17<1:13:24, 2.08s/it] 72%|███████▏ | 5388/7500 [3:03:19<1:13:21, 2.08s/it] 72%|███████▏ | 5389/7500 [3:03:21<1:14:01, 2.10s/it] 72%|███████▏ | 5390/7500 [3:03:23<1:12:34, 2.06s/it] {'loss': 0.3376, 'grad_norm': 2.516679525375366, 'learning_rate': 1.111679137970303e-05, 'logqs/rejected': -2.3426671028137207, 'logqs/chosen': -0.8357677459716797, 'logqs_logps/rejected': -0.030763551592826843, 'logps_logqs/chosen': 0.2102503478527069, 'epoch': 1.44} 72%|███████▏ | 5390/7500 [3:03:23<1:12:34, 2.06s/it] 72%|███████▏ | 5391/7500 [3:03:25<1:12:32, 2.06s/it] 72%|███████▏ | 5392/7500 [3:03:27<1:11:44, 2.04s/it] 72%|███████▏ | 5393/7500 [3:03:29<1:11:28, 2.04s/it] 72%|███████▏ | 5394/7500 [3:03:31<1:11:49, 2.05s/it] 72%|███████▏ | 5395/7500 [3:03:33<1:11:31, 2.04s/it] 72%|███████▏ | 5396/7500 [3:03:35<1:10:54, 2.02s/it] 72%|███████▏ | 5397/7500 [3:03:37<1:12:00, 2.05s/it] 72%|███████▏ | 5398/7500 [3:03:39<1:11:29, 2.04s/it] 72%|███████▏ | 5399/7500 [3:03:42<1:12:35, 2.07s/it] 72%|███████▏ | 5400/7500 [3:03:44<1:13:27, 2.10s/it] {'loss': 0.3381, 'grad_norm': 2.345278739929199, 'learning_rate': 1.1020177413231334e-05, 'logqs/rejected': -2.384469985961914, 'logqs/chosen': -0.8337333798408508, 'logqs_logps/rejected': -0.026987582445144653, 'logps_logqs/chosen': 0.20539884269237518, 'epoch': 1.44} 72%|███████▏ | 5400/7500 [3:03:44<1:13:27, 2.10s/it] 72%|███████▏ | 5401/7500 [3:03:46<1:12:25, 2.07s/it] 72%|███████▏ | 5402/7500 [3:03:48<1:12:01, 2.06s/it] 72%|███████▏ | 5403/7500 [3:03:50<1:13:34, 2.11s/it] 72%|███████▏ | 5404/7500 [3:03:52<1:12:10, 2.07s/it] 72%|███████▏ | 5405/7500 [3:03:54<1:12:36, 2.08s/it] 72%|███████▏ | 5406/7500 [3:03:56<1:11:46, 2.06s/it] 72%|███████▏ | 5407/7500 [3:03:58<1:11:30, 2.05s/it] 72%|███████▏ | 5408/7500 [3:04:00<1:12:04, 2.07s/it] 72%|███████▏ | 5409/7500 [3:04:02<1:12:01, 2.07s/it] 72%|███████▏ | 5410/7500 [3:04:04<1:11:09, 2.04s/it] {'loss': 0.3473, 'grad_norm': 2.609163761138916, 'learning_rate': 1.0923866272700844e-05, 'logqs/rejected': -2.4529623985290527, 'logqs/chosen': -0.8127182126045227, 'logqs_logps/rejected': -0.028649333864450455, 'logps_logqs/chosen': 0.20264574885368347, 'epoch': 1.44} 72%|███████▏ | 5410/7500 [3:04:04<1:11:09, 2.04s/it] 72%|███████▏ | 5411/7500 [3:04:06<1:10:52, 2.04s/it] 72%|███████▏ | 5412/7500 [3:04:08<1:11:22, 2.05s/it] 72%|███████▏ | 5413/7500 [3:04:10<1:11:29, 2.06s/it] 72%|███████▏ | 5414/7500 [3:04:12<1:11:45, 2.06s/it] 72%|███████▏ | 5415/7500 [3:04:15<1:11:52, 2.07s/it] 72%|███████▏ | 5416/7500 [3:04:17<1:12:35, 2.09s/it] 72%|███████▏ | 5417/7500 [3:04:19<1:11:19, 2.05s/it] 72%|███████▏ | 5418/7500 [3:04:21<1:11:41, 2.07s/it] 72%|███████▏ | 5419/7500 [3:04:23<1:11:32, 2.06s/it] 72%|███████▏ | 5420/7500 [3:04:25<1:12:47, 2.10s/it] {'loss': 0.3586, 'grad_norm': 3.719137668609619, 'learning_rate': 1.0827860044369226e-05, 'logqs/rejected': -2.5734190940856934, 'logqs/chosen': -0.8614774942398071, 'logqs_logps/rejected': -0.02041609212756157, 'logps_logqs/chosen': 0.2100989818572998, 'epoch': 1.45} 72%|███████▏ | 5420/7500 [3:04:25<1:12:47, 2.10s/it] 72%|███████▏ | 5421/7500 [3:04:27<1:12:25, 2.09s/it] 72%|███████▏ | 5422/7500 [3:04:29<1:12:18, 2.09s/it] 72%|███████▏ | 5423/7500 [3:04:31<1:11:48, 2.07s/it] 72%|███████▏ | 5424/7500 [3:04:33<1:11:34, 2.07s/it] 72%|███████▏ | 5425/7500 [3:04:35<1:11:55, 2.08s/it] 72%|███████▏ | 5426/7500 [3:04:37<1:12:09, 2.09s/it] 72%|███████▏ | 5427/7500 [3:04:39<1:11:27, 2.07s/it] 72%|███████▏ | 5428/7500 [3:04:42<1:11:31, 2.07s/it] 72%|███████▏ | 5429/7500 [3:04:44<1:10:35, 2.04s/it] 72%|███████▏ | 5430/7500 [3:04:46<1:10:57, 2.06s/it] {'loss': 0.3538, 'grad_norm': 2.463705062866211, 'learning_rate': 1.0732160807889211e-05, 'logqs/rejected': -2.497473955154419, 'logqs/chosen': -0.8479315042495728, 'logqs_logps/rejected': -0.028434714302420616, 'logps_logqs/chosen': 0.205860897898674, 'epoch': 1.45} 72%|███████▏ | 5430/7500 [3:04:46<1:10:57, 2.06s/it] 72%|███████▏ | 5431/7500 [3:04:48<1:11:20, 2.07s/it] 72%|███████▏ | 5432/7500 [3:04:50<1:10:42, 2.05s/it] 72%|███████▏ | 5433/7500 [3:04:52<1:11:05, 2.06s/it] 72%|███████▏ | 5434/7500 [3:04:54<1:11:50, 2.09s/it] 72%|███████▏ | 5435/7500 [3:04:56<1:11:31, 2.08s/it] 72%|███████▏ | 5436/7500 [3:04:58<1:10:30, 2.05s/it] 72%|███████▏ | 5437/7500 [3:05:00<1:11:26, 2.08s/it] 73%|███████▎ | 5438/7500 [3:05:02<1:11:04, 2.07s/it] 73%|███████▎ | 5439/7500 [3:05:04<1:10:43, 2.06s/it] 73%|███████▎ | 5440/7500 [3:05:06<1:09:52, 2.03s/it] {'loss': 0.3459, 'grad_norm': 2.8609962463378906, 'learning_rate': 1.0636770636263602e-05, 'logqs/rejected': -2.395002603530884, 'logqs/chosen': -0.8456621170043945, 'logqs_logps/rejected': -0.02507338486611843, 'logps_logqs/chosen': 0.2022821456193924, 'epoch': 1.45} 73%|███████▎ | 5440/7500 [3:05:06<1:09:52, 2.03s/it] 73%|███████▎ | 5441/7500 [3:05:08<1:09:43, 2.03s/it] 73%|███████▎ | 5442/7500 [3:05:10<1:09:47, 2.03s/it] 73%|███████▎ | 5443/7500 [3:05:12<1:09:58, 2.04s/it] 73%|███████▎ | 5444/7500 [3:05:14<1:09:49, 2.04s/it] 73%|███████▎ | 5445/7500 [3:05:16<1:09:47, 2.04s/it] 73%|███████▎ | 5446/7500 [3:05:18<1:10:11, 2.05s/it] 73%|███████▎ | 5447/7500 [3:05:21<1:10:22, 2.06s/it] 73%|███████▎ | 5448/7500 [3:05:23<1:09:59, 2.05s/it] 73%|███████▎ | 5449/7500 [3:05:25<1:11:07, 2.08s/it] 73%|███████▎ | 5450/7500 [3:05:27<1:11:22, 2.09s/it] {'loss': 0.3642, 'grad_norm': 2.886995553970337, 'learning_rate': 1.0541691595800337e-05, 'logqs/rejected': -2.6417064666748047, 'logqs/chosen': -0.8557147979736328, 'logqs_logps/rejected': -0.0197830181568861, 'logps_logqs/chosen': 0.20323574542999268, 'epoch': 1.45} 73%|███████▎ | 5450/7500 [3:05:27<1:11:22, 2.09s/it] 73%|███████▎ | 5451/7500 [3:05:29<1:09:57, 2.05s/it] 73%|███████▎ | 5452/7500 [3:05:31<1:10:13, 2.06s/it] 73%|███████▎ | 5453/7500 [3:05:33<1:10:34, 2.07s/it] 73%|███████▎ | 5454/7500 [3:05:35<1:10:15, 2.06s/it] 73%|███████▎ | 5455/7500 [3:05:37<1:10:44, 2.08s/it] 73%|███████▎ | 5456/7500 [3:05:39<1:11:01, 2.08s/it] 73%|███████▎ | 5457/7500 [3:05:41<1:10:51, 2.08s/it] 73%|███████▎ | 5458/7500 [3:05:43<1:11:28, 2.10s/it] 73%|███████▎ | 5459/7500 [3:05:45<1:10:41, 2.08s/it] 73%|███████▎ | 5460/7500 [3:05:48<1:11:35, 2.11s/it] {'loss': 0.3597, 'grad_norm': 2.513469934463501, 'learning_rate': 1.0446925746067768e-05, 'logqs/rejected': -2.61346435546875, 'logqs/chosen': -0.8455215692520142, 'logqs_logps/rejected': -0.02353062853217125, 'logps_logqs/chosen': 0.20413246750831604, 'epoch': 1.46} 73%|███████▎ | 5460/7500 [3:05:48<1:11:35, 2.11s/it] 73%|███████▎ | 5461/7500 [3:05:50<1:11:37, 2.11s/it] 73%|███████▎ | 5462/7500 [3:05:52<1:11:38, 2.11s/it] 73%|███████▎ | 5463/7500 [3:05:54<1:11:34, 2.11s/it] 73%|███████▎ | 5464/7500 [3:05:56<1:11:13, 2.10s/it] 73%|███████▎ | 5465/7500 [3:05:58<1:11:54, 2.12s/it] 73%|███████▎ | 5466/7500 [3:06:00<1:11:40, 2.11s/it] 73%|███████▎ | 5467/7500 [3:06:02<1:10:34, 2.08s/it] 73%|███████▎ | 5468/7500 [3:06:04<1:09:46, 2.06s/it] 73%|███████▎ | 5469/7500 [3:06:06<1:10:30, 2.08s/it] 73%|███████▎ | 5470/7500 [3:06:09<1:09:59, 2.07s/it] {'loss': 0.3649, 'grad_norm': 2.8013153076171875, 'learning_rate': 1.0352475139849993e-05, 'logqs/rejected': -2.7092413902282715, 'logqs/chosen': -0.8428744077682495, 'logqs_logps/rejected': -0.02426614984869957, 'logps_logqs/chosen': 0.20310382544994354, 'epoch': 1.46} 73%|███████▎ | 5470/7500 [3:06:09<1:09:59, 2.07s/it] 73%|███████▎ | 5471/7500 [3:06:11<1:09:43, 2.06s/it] 73%|███████▎ | 5472/7500 [3:06:13<1:09:46, 2.06s/it] 73%|███████▎ | 5473/7500 [3:06:15<1:09:36, 2.06s/it] 73%|███████▎ | 5474/7500 [3:06:17<1:09:30, 2.06s/it] 73%|███████▎ | 5475/7500 [3:06:19<1:09:53, 2.07s/it] 73%|███████▎ | 5476/7500 [3:06:21<1:10:36, 2.09s/it] 73%|███████▎ | 5477/7500 [3:06:23<1:10:23, 2.09s/it] 73%|███████▎ | 5478/7500 [3:06:25<1:10:27, 2.09s/it] 73%|███████▎ | 5479/7500 [3:06:27<1:11:09, 2.11s/it] 73%|███████▎ | 5480/7500 [3:06:29<1:11:34, 2.13s/it] {'loss': 0.3453, 'grad_norm': 3.346184492111206, 'learning_rate': 1.0258341823102418e-05, 'logqs/rejected': -2.2771356105804443, 'logqs/chosen': -0.8554939031600952, 'logqs_logps/rejected': -0.024432769045233727, 'logps_logqs/chosen': 0.2107125073671341, 'epoch': 1.46} 73%|███████▎ | 5480/7500 [3:06:30<1:11:34, 2.13s/it] 73%|███████▎ | 5481/7500 [3:06:32<1:11:52, 2.14s/it] 73%|███████▎ | 5482/7500 [3:06:34<1:11:29, 2.13s/it] 73%|███████▎ | 5483/7500 [3:06:36<1:10:43, 2.10s/it] 73%|███████▎ | 5484/7500 [3:06:38<1:09:47, 2.08s/it] 73%|███████▎ | 5485/7500 [3:06:40<1:10:24, 2.10s/it] 73%|███████▎ | 5486/7500 [3:06:42<1:09:37, 2.07s/it] 73%|███████▎ | 5487/7500 [3:06:44<1:09:39, 2.08s/it] 73%|███████▎ | 5488/7500 [3:06:46<1:09:31, 2.07s/it] 73%|███████▎ | 5489/7500 [3:06:48<1:10:13, 2.10s/it] 73%|███████▎ | 5490/7500 [3:06:50<1:09:55, 2.09s/it] {'loss': 0.36, 'grad_norm': 2.96410870552063, 'learning_rate': 1.0164527834907467e-05, 'logqs/rejected': -2.6240692138671875, 'logqs/chosen': -0.8475914001464844, 'logqs_logps/rejected': -0.02709081768989563, 'logps_logqs/chosen': 0.19511902332305908, 'epoch': 1.46} 73%|███████▎ | 5490/7500 [3:06:50<1:09:55, 2.09s/it] 73%|███████▎ | 5491/7500 [3:06:52<1:10:01, 2.09s/it] 73%|███████▎ | 5492/7500 [3:06:55<1:10:21, 2.10s/it] 73%|███████▎ | 5493/7500 [3:06:57<1:10:06, 2.10s/it] 73%|███████▎ | 5494/7500 [3:06:59<1:09:54, 2.09s/it] 73%|███████▎ | 5495/7500 [3:07:01<1:09:09, 2.07s/it] 73%|███████▎ | 5496/7500 [3:07:03<1:09:22, 2.08s/it] 73%|███████▎ | 5497/7500 [3:07:05<1:09:49, 2.09s/it] 73%|███████▎ | 5498/7500 [3:07:07<1:10:00, 2.10s/it] 73%|███████▎ | 5499/7500 [3:07:09<1:09:48, 2.09s/it] 73%|███████▎ | 5500/7500 [3:07:11<1:10:00, 2.10s/it] {'loss': 0.3536, 'grad_norm': 2.693864345550537, 'learning_rate': 1.0071035207430352e-05, 'logqs/rejected': -2.4743735790252686, 'logqs/chosen': -0.8234676122665405, 'logqs_logps/rejected': -0.024782579392194748, 'logps_logqs/chosen': 0.20434340834617615, 'epoch': 1.47} 73%|███████▎ | 5500/7500 [3:07:11<1:10:00, 2.10s/it] 73%|███████▎ | 5501/7500 [3:07:13<1:09:35, 2.09s/it] 73%|███████▎ | 5502/7500 [3:07:16<1:10:22, 2.11s/it] 73%|███████▎ | 5503/7500 [3:07:18<1:09:37, 2.09s/it] 73%|███████▎ | 5504/7500 [3:07:20<1:08:42, 2.07s/it] 73%|███████▎ | 5505/7500 [3:07:22<1:09:17, 2.08s/it] 73%|███████▎ | 5506/7500 [3:07:24<1:09:27, 2.09s/it] 73%|███████▎ | 5507/7500 [3:07:26<1:10:01, 2.11s/it] 73%|███████▎ | 5508/7500 [3:07:28<1:10:40, 2.13s/it] 73%|███████▎ | 5509/7500 [3:07:30<1:09:31, 2.09s/it] 73%|███████▎ | 5510/7500 [3:07:32<1:09:27, 2.09s/it] {'loss': 0.3441, 'grad_norm': 2.865217447280884, 'learning_rate': 9.97786596587509e-06, 'logqs/rejected': -2.3440849781036377, 'logqs/chosen': -0.8691399693489075, 'logqs_logps/rejected': -0.027565401047468185, 'logps_logqs/chosen': 0.2012350857257843, 'epoch': 1.47} 73%|███████▎ | 5510/7500 [3:07:32<1:09:27, 2.09s/it] 73%|███████▎ | 5511/7500 [3:07:34<1:09:32, 2.10s/it] 73%|███████▎ | 5512/7500 [3:07:36<1:08:54, 2.08s/it] 74%|███████▎ | 5513/7500 [3:07:38<1:08:59, 2.08s/it] 74%|███████▎ | 5514/7500 [3:07:40<1:08:37, 2.07s/it] 74%|███████▎ | 5515/7500 [3:07:43<1:08:05, 2.06s/it] 74%|███████▎ | 5516/7500 [3:07:45<1:09:20, 2.10s/it] 74%|███████▎ | 5517/7500 [3:07:47<1:08:43, 2.08s/it] 74%|███████▎ | 5518/7500 [3:07:49<1:08:27, 2.07s/it] 74%|███████▎ | 5519/7500 [3:07:51<1:08:38, 2.08s/it] 74%|███████▎ | 5520/7500 [3:07:53<1:07:55, 2.06s/it] {'loss': 0.3657, 'grad_norm': 2.9242560863494873, 'learning_rate': 9.88502212844063e-06, 'logqs/rejected': -2.550509452819824, 'logqs/chosen': -0.8485096096992493, 'logqs_logps/rejected': -0.018335167318582535, 'logps_logqs/chosen': 0.203611820936203, 'epoch': 1.47} 74%|███████▎ | 5520/7500 [3:07:53<1:07:55, 2.06s/it] 74%|███████▎ | 5521/7500 [3:07:55<1:07:46, 2.05s/it] 74%|███████▎ | 5522/7500 [3:07:57<1:07:56, 2.06s/it] 74%|███████▎ | 5523/7500 [3:07:59<1:08:05, 2.07s/it] 74%|███████▎ | 5524/7500 [3:08:01<1:08:05, 2.07s/it] 74%|███████▎ | 5525/7500 [3:08:03<1:08:36, 2.08s/it] 74%|███████▎ | 5526/7500 [3:08:05<1:09:19, 2.11s/it] 74%|███████▎ | 5527/7500 [3:08:08<1:09:02, 2.10s/it] 74%|███████▎ | 5528/7500 [3:08:10<1:08:34, 2.09s/it] 74%|███████▎ | 5529/7500 [3:08:12<1:09:05, 2.10s/it] 74%|███████▎ | 5530/7500 [3:08:14<1:09:08, 2.11s/it] {'loss': 0.3513, 'grad_norm': 4.035285949707031, 'learning_rate': 9.792505706277136e-06, 'logqs/rejected': -2.3103625774383545, 'logqs/chosen': -0.8652924299240112, 'logqs_logps/rejected': -0.02176799066364765, 'logps_logqs/chosen': 0.20509441196918488, 'epoch': 1.47} 74%|███████▎ | 5530/7500 [3:08:14<1:09:08, 2.11s/it] 74%|███████▎ | 5531/7500 [3:08:16<1:07:48, 2.07s/it] 74%|███████▍ | 5532/7500 [3:08:18<1:07:21, 2.05s/it] 74%|███████▍ | 5533/7500 [3:08:20<1:06:58, 2.04s/it] 74%|███████▍ | 5534/7500 [3:08:22<1:08:45, 2.10s/it] 74%|███████▍ | 5535/7500 [3:08:24<1:09:01, 2.11s/it] 74%|███████▍ | 5536/7500 [3:08:26<1:09:00, 2.11s/it] 74%|███████▍ | 5537/7500 [3:08:28<1:09:01, 2.11s/it] 74%|███████▍ | 5538/7500 [3:08:30<1:08:13, 2.09s/it] 74%|███████▍ | 5539/7500 [3:08:32<1:07:11, 2.06s/it] 74%|███████▍ | 5540/7500 [3:08:35<1:06:54, 2.05s/it] {'loss': 0.3711, 'grad_norm': 3.336141586303711, 'learning_rate': 9.700318703442438e-06, 'logqs/rejected': -2.595444679260254, 'logqs/chosen': -0.8401853442192078, 'logqs_logps/rejected': -0.023520754650235176, 'logps_logqs/chosen': 0.211019828915596, 'epoch': 1.48} 74%|███████▍ | 5540/7500 [3:08:35<1:06:54, 2.05s/it] 74%|███████▍ | 5541/7500 [3:08:37<1:07:37, 2.07s/it] 74%|███████▍ | 5542/7500 [3:08:39<1:07:10, 2.06s/it] 74%|███████▍ | 5543/7500 [3:08:41<1:07:32, 2.07s/it] 74%|███████▍ | 5544/7500 [3:08:43<1:07:34, 2.07s/it] 74%|███████▍ | 5545/7500 [3:08:45<1:07:21, 2.07s/it] 74%|███████▍ | 5546/7500 [3:08:47<1:07:36, 2.08s/it] 74%|███████▍ | 5547/7500 [3:08:49<1:07:02, 2.06s/it] 74%|███████▍ | 5548/7500 [3:08:51<1:06:19, 2.04s/it] 74%|███████▍ | 5549/7500 [3:08:53<1:06:53, 2.06s/it] 74%|███████▍ | 5550/7500 [3:08:55<1:06:11, 2.04s/it] {'loss': 0.3573, 'grad_norm': 2.6360511779785156, 'learning_rate': 9.608463116858542e-06, 'logqs/rejected': -2.6977620124816895, 'logqs/chosen': -0.8378075361251831, 'logqs_logps/rejected': -0.02746693231165409, 'logps_logqs/chosen': 0.19935670495033264, 'epoch': 1.48} 74%|███████▍ | 5550/7500 [3:08:55<1:06:11, 2.04s/it] 74%|███████▍ | 5551/7500 [3:08:57<1:07:22, 2.07s/it] 74%|███████▍ | 5552/7500 [3:08:59<1:07:09, 2.07s/it] 74%|███████▍ | 5553/7500 [3:09:01<1:06:12, 2.04s/it] 74%|███████▍ | 5554/7500 [3:09:03<1:05:22, 2.02s/it] 74%|███████▍ | 5555/7500 [3:09:05<1:05:35, 2.02s/it] 74%|███████▍ | 5556/7500 [3:09:07<1:04:53, 2.00s/it] 74%|███████▍ | 5557/7500 [3:09:09<1:05:18, 2.02s/it] 74%|███████▍ | 5558/7500 [3:09:11<1:05:30, 2.02s/it] 74%|███████▍ | 5559/7500 [3:09:13<1:05:16, 2.02s/it] 74%|███████▍ | 5560/7500 [3:09:15<1:05:40, 2.03s/it] {'loss': 0.362, 'grad_norm': 2.590141534805298, 'learning_rate': 9.516940936268503e-06, 'logqs/rejected': -2.6762571334838867, 'logqs/chosen': -0.8323318362236023, 'logqs_logps/rejected': -0.019475528970360756, 'logps_logqs/chosen': 0.20908010005950928, 'epoch': 1.48} 74%|███████▍ | 5560/7500 [3:09:15<1:05:40, 2.03s/it] 74%|███████▍ | 5561/7500 [3:09:17<1:06:19, 2.05s/it] 74%|███████▍ | 5562/7500 [3:09:19<1:05:53, 2.04s/it] 74%|███████▍ | 5563/7500 [3:09:22<1:05:55, 2.04s/it] 74%|███████▍ | 5564/7500 [3:09:24<1:06:07, 2.05s/it] 74%|███████▍ | 5565/7500 [3:09:26<1:06:33, 2.06s/it] 74%|███████▍ | 5566/7500 [3:09:28<1:06:30, 2.06s/it] 74%|███████▍ | 5567/7500 [3:09:30<1:05:59, 2.05s/it] 74%|███████▍ | 5568/7500 [3:09:32<1:05:22, 2.03s/it] 74%|███████▍ | 5569/7500 [3:09:34<1:05:16, 2.03s/it] 74%|███████▍ | 5570/7500 [3:09:36<1:06:43, 2.07s/it] {'loss': 0.3521, 'grad_norm': 2.7784805297851562, 'learning_rate': 9.4257541441932e-06, 'logqs/rejected': -2.4001376628875732, 'logqs/chosen': -0.8308441042900085, 'logqs_logps/rejected': -0.023768160492181778, 'logps_logqs/chosen': 0.2030165195465088, 'epoch': 1.49} 74%|███████▍ | 5570/7500 [3:09:36<1:06:43, 2.07s/it] 74%|███████▍ | 5571/7500 [3:09:38<1:06:38, 2.07s/it] 74%|███████▍ | 5572/7500 [3:09:40<1:06:21, 2.06s/it] 74%|███████▍ | 5573/7500 [3:09:42<1:07:35, 2.10s/it] 74%|███████▍ | 5574/7500 [3:09:44<1:06:53, 2.08s/it] 74%|███████▍ | 5575/7500 [3:09:46<1:07:09, 2.09s/it] 74%|███████▍ | 5576/7500 [3:09:48<1:06:46, 2.08s/it] 74%|███████▍ | 5577/7500 [3:09:51<1:07:05, 2.09s/it] 74%|███████▍ | 5578/7500 [3:09:53<1:06:59, 2.09s/it] 74%|███████▍ | 5579/7500 [3:09:55<1:06:02, 2.06s/it] 74%|███████▍ | 5580/7500 [3:09:57<1:05:41, 2.05s/it] {'loss': 0.3551, 'grad_norm': 2.7078018188476562, 'learning_rate': 9.334904715888495e-06, 'logqs/rejected': -2.39911150932312, 'logqs/chosen': -0.8493253588676453, 'logqs_logps/rejected': -0.029755080118775368, 'logps_logqs/chosen': 0.20576930046081543, 'epoch': 1.49} 74%|███████▍ | 5580/7500 [3:09:57<1:05:41, 2.05s/it] 74%|███████▍ | 5581/7500 [3:09:59<1:05:20, 2.04s/it] 74%|███████▍ | 5582/7500 [3:10:01<1:06:07, 2.07s/it] 74%|███████▍ | 5583/7500 [3:10:03<1:06:18, 2.08s/it] 74%|███████▍ | 5584/7500 [3:10:05<1:05:52, 2.06s/it] 74%|███████▍ | 5585/7500 [3:10:07<1:05:58, 2.07s/it] 74%|███████▍ | 5586/7500 [3:10:09<1:05:49, 2.06s/it] 74%|███████▍ | 5587/7500 [3:10:11<1:05:45, 2.06s/it] 75%|███████▍ | 5588/7500 [3:10:13<1:04:48, 2.03s/it] 75%|███████▍ | 5589/7500 [3:10:15<1:05:08, 2.05s/it] 75%|███████▍ | 5590/7500 [3:10:17<1:04:51, 2.04s/it] {'loss': 0.3302, 'grad_norm': 3.460520029067993, 'learning_rate': 9.244394619302338e-06, 'logqs/rejected': -2.2791497707366943, 'logqs/chosen': -0.834926962852478, 'logqs_logps/rejected': -0.03166760876774788, 'logps_logqs/chosen': 0.20277781784534454, 'epoch': 1.49} 75%|███████▍ | 5590/7500 [3:10:17<1:04:51, 2.04s/it] 75%|███████▍ | 5591/7500 [3:10:19<1:04:29, 2.03s/it] 75%|███████▍ | 5592/7500 [3:10:21<1:05:32, 2.06s/it] 75%|███████▍ | 5593/7500 [3:10:23<1:05:14, 2.05s/it] 75%|███████▍ | 5594/7500 [3:10:25<1:05:20, 2.06s/it] 75%|███████▍ | 5595/7500 [3:10:28<1:05:12, 2.05s/it] 75%|███████▍ | 5596/7500 [3:10:30<1:04:43, 2.04s/it] 75%|███████▍ | 5597/7500 [3:10:32<1:05:10, 2.05s/it] 75%|███████▍ | 5598/7500 [3:10:34<1:04:26, 2.03s/it] 75%|███████▍ | 5599/7500 [3:10:36<1:03:33, 2.01s/it] 75%|███████▍ | 5600/7500 [3:10:38<1:03:20, 2.00s/it] {'loss': 0.3693, 'grad_norm': 2.570904493331909, 'learning_rate': 9.154225815032242e-06, 'logqs/rejected': -2.7143023014068604, 'logqs/chosen': -0.8429845571517944, 'logqs_logps/rejected': -0.01583649218082428, 'logps_logqs/chosen': 0.20226123929023743, 'epoch': 1.49} 75%|███████▍ | 5600/7500 [3:10:38<1:03:20, 2.00s/it] 75%|███████▍ | 5601/7500 [3:10:40<1:03:21, 2.00s/it] 75%|███████▍ | 5602/7500 [3:10:42<1:03:24, 2.00s/it] 75%|███████▍ | 5603/7500 [3:10:44<1:04:09, 2.03s/it] 75%|███████▍ | 5604/7500 [3:10:46<1:03:38, 2.01s/it] 75%|███████▍ | 5605/7500 [3:10:48<1:04:09, 2.03s/it] 75%|███████▍ | 5606/7500 [3:10:50<1:04:41, 2.05s/it] 75%|███████▍ | 5607/7500 [3:10:52<1:04:20, 2.04s/it] 75%|███████▍ | 5608/7500 [3:10:54<1:04:28, 2.04s/it] 75%|███████▍ | 5609/7500 [3:10:56<1:04:51, 2.06s/it] 75%|███████▍ | 5610/7500 [3:10:58<1:05:02, 2.07s/it] {'loss': 0.357, 'grad_norm': 2.650660991668701, 'learning_rate': 9.064400256282757e-06, 'logqs/rejected': -2.7145514488220215, 'logqs/chosen': -0.8410978317260742, 'logqs_logps/rejected': -0.022019432857632637, 'logps_logqs/chosen': 0.20218594372272491, 'epoch': 1.5} 75%|███████▍ | 5610/7500 [3:10:58<1:05:02, 2.07s/it] 75%|███████▍ | 5611/7500 [3:11:00<1:04:52, 2.06s/it] 75%|███████▍ | 5612/7500 [3:11:02<1:05:36, 2.09s/it] 75%|███████▍ | 5613/7500 [3:11:04<1:04:43, 2.06s/it] 75%|███████▍ | 5614/7500 [3:11:06<1:04:43, 2.06s/it] 75%|███████▍ | 5615/7500 [3:11:08<1:04:58, 2.07s/it] 75%|███████▍ | 5616/7500 [3:11:11<1:05:33, 2.09s/it] 75%|███████▍ | 5617/7500 [3:11:13<1:04:52, 2.07s/it] 75%|███████▍ | 5618/7500 [3:11:15<1:04:15, 2.05s/it] 75%|███████▍ | 5619/7500 [3:11:17<1:04:46, 2.07s/it] 75%|███████▍ | 5620/7500 [3:11:19<1:05:57, 2.10s/it] {'loss': 0.3449, 'grad_norm': 2.9957337379455566, 'learning_rate': 8.974919888823164e-06, 'logqs/rejected': -2.407392978668213, 'logqs/chosen': -0.8365524411201477, 'logqs_logps/rejected': -0.0270084235817194, 'logps_logqs/chosen': 0.20027661323547363, 'epoch': 1.5} 75%|███████▍ | 5620/7500 [3:11:19<1:05:57, 2.10s/it] 75%|███████▍ | 5621/7500 [3:11:21<1:05:46, 2.10s/it] 75%|███████▍ | 5622/7500 [3:11:23<1:05:18, 2.09s/it] 75%|███████▍ | 5623/7500 [3:11:25<1:05:10, 2.08s/it] 75%|███████▍ | 5624/7500 [3:11:27<1:06:00, 2.11s/it] 75%|███████▌ | 5625/7500 [3:11:29<1:05:05, 2.08s/it] 75%|███████▌ | 5626/7500 [3:11:31<1:05:31, 2.10s/it] 75%|███████▌ | 5627/7500 [3:11:34<1:05:53, 2.11s/it] 75%|███████▌ | 5628/7500 [3:11:36<1:05:30, 2.10s/it] 75%|███████▌ | 5629/7500 [3:11:38<1:05:26, 2.10s/it] 75%|███████▌ | 5630/7500 [3:11:40<1:04:47, 2.08s/it] {'loss': 0.3305, 'grad_norm': 2.7820165157318115, 'learning_rate': 8.885786650945332e-06, 'logqs/rejected': -2.241445779800415, 'logqs/chosen': -0.8343561887741089, 'logqs_logps/rejected': -0.02487107180058956, 'logps_logqs/chosen': 0.20443376898765564, 'epoch': 1.5} 75%|███████▌ | 5630/7500 [3:11:40<1:04:47, 2.08s/it] 75%|███████▌ | 5631/7500 [3:11:42<1:05:32, 2.10s/it] 75%|███████▌ | 5632/7500 [3:11:44<1:05:37, 2.11s/it] 75%|███████▌ | 5633/7500 [3:11:46<1:05:34, 2.11s/it] 75%|███████▌ | 5634/7500 [3:11:48<1:04:47, 2.08s/it] 75%|███████▌ | 5635/7500 [3:11:50<1:04:32, 2.08s/it] 75%|███████▌ | 5636/7500 [3:11:52<1:03:59, 2.06s/it] 75%|███████▌ | 5637/7500 [3:11:54<1:03:43, 2.05s/it] 75%|███████▌ | 5638/7500 [3:11:56<1:02:56, 2.03s/it] 75%|███████▌ | 5639/7500 [3:11:58<1:03:27, 2.05s/it] 75%|███████▌ | 5640/7500 [3:12:00<1:04:32, 2.08s/it] {'loss': 0.362, 'grad_norm': 2.4927027225494385, 'learning_rate': 8.797002473421728e-06, 'logqs/rejected': -2.5568079948425293, 'logqs/chosen': -0.8357383608818054, 'logqs_logps/rejected': -0.019808229058980942, 'logps_logqs/chosen': 0.20321345329284668, 'epoch': 1.5} 75%|███████▌ | 5640/7500 [3:12:01<1:04:32, 2.08s/it] 75%|███████▌ | 5641/7500 [3:12:03<1:05:00, 2.10s/it] 75%|███████▌ | 5642/7500 [3:12:05<1:06:13, 2.14s/it] 75%|███████▌ | 5643/7500 [3:12:07<1:05:49, 2.13s/it] 75%|███████▌ | 5644/7500 [3:12:09<1:05:32, 2.12s/it] 75%|███████▌ | 5645/7500 [3:12:11<1:04:38, 2.09s/it] 75%|███████▌ | 5646/7500 [3:12:13<1:04:03, 2.07s/it] 75%|███████▌ | 5647/7500 [3:12:15<1:03:02, 2.04s/it] 75%|███████▌ | 5648/7500 [3:12:17<1:03:38, 2.06s/it] 75%|███████▌ | 5649/7500 [3:12:19<1:04:58, 2.11s/it] 75%|███████▌ | 5650/7500 [3:12:21<1:04:35, 2.09s/it] {'loss': 0.3353, 'grad_norm': 2.9931204319000244, 'learning_rate': 8.708569279463622e-06, 'logqs/rejected': -2.38944935798645, 'logqs/chosen': -0.8284313082695007, 'logqs_logps/rejected': -0.025501925498247147, 'logps_logqs/chosen': 0.20888292789459229, 'epoch': 1.51} 75%|███████▌ | 5650/7500 [3:12:21<1:04:35, 2.09s/it] 75%|███████▌ | 5651/7500 [3:12:23<1:03:34, 2.06s/it] 75%|███████▌ | 5652/7500 [3:12:25<1:03:20, 2.06s/it] 75%|███████▌ | 5653/7500 [3:12:28<1:04:03, 2.08s/it] 75%|███████▌ | 5654/7500 [3:12:30<1:03:55, 2.08s/it] 75%|███████▌ | 5655/7500 [3:12:32<1:03:41, 2.07s/it] 75%|███████▌ | 5656/7500 [3:12:34<1:03:52, 2.08s/it] 75%|███████▌ | 5657/7500 [3:12:36<1:03:41, 2.07s/it] 75%|███████▌ | 5658/7500 [3:12:38<1:03:35, 2.07s/it] 75%|███████▌ | 5659/7500 [3:12:40<1:03:32, 2.07s/it] 75%|███████▌ | 5660/7500 [3:12:42<1:04:35, 2.11s/it] {'loss': 0.3582, 'grad_norm': 3.4665167331695557, 'learning_rate': 8.620488984679378e-06, 'logqs/rejected': -2.6488733291625977, 'logqs/chosen': -0.8349672555923462, 'logqs_logps/rejected': -0.02379489503800869, 'logps_logqs/chosen': 0.20117321610450745, 'epoch': 1.51} 75%|███████▌ | 5660/7500 [3:12:42<1:04:35, 2.11s/it] 75%|███████▌ | 5661/7500 [3:12:44<1:04:59, 2.12s/it] 75%|███████▌ | 5662/7500 [3:12:46<1:04:34, 2.11s/it] 76%|███████▌ | 5663/7500 [3:12:49<1:04:29, 2.11s/it] 76%|███████▌ | 5664/7500 [3:12:51<1:03:26, 2.07s/it] 76%|███████▌ | 5665/7500 [3:12:53<1:03:24, 2.07s/it] 76%|███████▌ | 5666/7500 [3:12:55<1:02:31, 2.05s/it] 76%|███████▌ | 5667/7500 [3:12:57<1:03:34, 2.08s/it] 76%|███████▌ | 5668/7500 [3:12:59<1:03:12, 2.07s/it] 76%|███████▌ | 5669/7500 [3:13:01<1:03:31, 2.08s/it] 76%|███████▌ | 5670/7500 [3:13:03<1:04:27, 2.11s/it] {'loss': 0.3516, 'grad_norm': 2.7176129817962646, 'learning_rate': 8.532763497032987e-06, 'logqs/rejected': -2.497450113296509, 'logqs/chosen': -0.852881133556366, 'logqs_logps/rejected': -0.027820218354463577, 'logps_logqs/chosen': 0.20689454674720764, 'epoch': 1.51} 76%|███████▌ | 5670/7500 [3:13:03<1:04:27, 2.11s/it] 76%|███████▌ | 5671/7500 [3:13:05<1:02:48, 2.06s/it] 76%|███████▌ | 5672/7500 [3:13:07<1:02:42, 2.06s/it] 76%|███████▌ | 5673/7500 [3:13:09<1:02:27, 2.05s/it] 76%|███████▌ | 5674/7500 [3:13:11<1:02:55, 2.07s/it] 76%|███████▌ | 5675/7500 [3:13:13<1:02:03, 2.04s/it] 76%|███████▌ | 5676/7500 [3:13:15<1:01:57, 2.04s/it] 76%|███████▌ | 5677/7500 [3:13:17<1:01:56, 2.04s/it] 76%|███████▌ | 5678/7500 [3:13:20<1:03:40, 2.10s/it] 76%|███████▌ | 5679/7500 [3:13:22<1:03:28, 2.09s/it] 76%|███████▌ | 5680/7500 [3:13:24<1:03:18, 2.09s/it] {'loss': 0.363, 'grad_norm': 2.4728260040283203, 'learning_rate': 8.445394716802754e-06, 'logqs/rejected': -2.540576457977295, 'logqs/chosen': -0.8237944841384888, 'logqs_logps/rejected': -0.020840667188167572, 'logps_logqs/chosen': 0.20181965827941895, 'epoch': 1.51} 76%|███████▌ | 5680/7500 [3:13:24<1:03:18, 2.09s/it] 76%|███████▌ | 5681/7500 [3:13:26<1:03:06, 2.08s/it] 76%|███████▌ | 5682/7500 [3:13:28<1:02:34, 2.06s/it] 76%|███████▌ | 5683/7500 [3:13:30<1:02:33, 2.07s/it] 76%|███████▌ | 5684/7500 [3:13:32<1:03:03, 2.08s/it] 76%|███████▌ | 5685/7500 [3:13:34<1:03:00, 2.08s/it] 76%|███████▌ | 5686/7500 [3:13:36<1:03:43, 2.11s/it] 76%|███████▌ | 5687/7500 [3:13:38<1:03:17, 2.09s/it] 76%|███████▌ | 5688/7500 [3:13:40<1:03:15, 2.09s/it] 76%|███████▌ | 5689/7500 [3:13:43<1:03:27, 2.10s/it] 76%|███████▌ | 5690/7500 [3:13:45<1:02:28, 2.07s/it] {'loss': 0.33, 'grad_norm': 2.672478199005127, 'learning_rate': 8.35838453654009e-06, 'logqs/rejected': -2.2582054138183594, 'logqs/chosen': -0.8440686464309692, 'logqs_logps/rejected': -0.026798348873853683, 'logps_logqs/chosen': 0.20002540946006775, 'epoch': 1.52} 76%|███████▌ | 5690/7500 [3:13:45<1:02:28, 2.07s/it] 76%|███████▌ | 5691/7500 [3:13:47<1:03:01, 2.09s/it] 76%|███████▌ | 5692/7500 [3:13:49<1:02:39, 2.08s/it] 76%|███████▌ | 5693/7500 [3:13:51<1:02:11, 2.07s/it] 76%|███████▌ | 5694/7500 [3:13:53<1:02:39, 2.08s/it] 76%|███████▌ | 5695/7500 [3:13:55<1:02:17, 2.07s/it] 76%|███████▌ | 5696/7500 [3:13:57<1:01:16, 2.04s/it] 76%|███████▌ | 5697/7500 [3:13:59<1:01:54, 2.06s/it] 76%|███████▌ | 5698/7500 [3:14:01<1:02:06, 2.07s/it] 76%|███████▌ | 5699/7500 [3:14:03<1:02:11, 2.07s/it] 76%|███████▌ | 5700/7500 [3:14:05<1:01:48, 2.06s/it] {'loss': 0.3446, 'grad_norm': 2.798917770385742, 'learning_rate': 8.271734841028553e-06, 'logqs/rejected': -2.4495580196380615, 'logqs/chosen': -0.8486621975898743, 'logqs_logps/rejected': -0.028571179136633873, 'logps_logqs/chosen': 0.2045561522245407, 'epoch': 1.52} 76%|███████▌ | 5700/7500 [3:14:05<1:01:48, 2.06s/it] 76%|███████▌ | 5701/7500 [3:14:07<1:02:02, 2.07s/it] 76%|███████▌ | 5702/7500 [3:14:09<1:01:54, 2.07s/it] 76%|███████▌ | 5703/7500 [3:14:11<1:01:53, 2.07s/it] 76%|███████▌ | 5704/7500 [3:14:13<1:02:06, 2.08s/it] 76%|███████▌ | 5705/7500 [3:14:16<1:01:58, 2.07s/it] 76%|███████▌ | 5706/7500 [3:14:18<1:01:50, 2.07s/it] 76%|███████▌ | 5707/7500 [3:14:20<1:01:32, 2.06s/it] 76%|███████▌ | 5708/7500 [3:14:22<1:01:49, 2.07s/it] 76%|███████▌ | 5709/7500 [3:14:24<1:01:10, 2.05s/it] 76%|███████▌ | 5710/7500 [3:14:26<1:00:23, 2.02s/it] {'loss': 0.3632, 'grad_norm': 3.1029868125915527, 'learning_rate': 8.185447507242999e-06, 'logqs/rejected': -2.5709164142608643, 'logqs/chosen': -0.8673877716064453, 'logqs_logps/rejected': -0.019021138548851013, 'logps_logqs/chosen': 0.20369529724121094, 'epoch': 1.52} 76%|███████▌ | 5710/7500 [3:14:26<1:00:23, 2.02s/it] 76%|███████▌ | 5711/7500 [3:14:28<1:01:09, 2.05s/it] 76%|███████▌ | 5712/7500 [3:14:30<1:01:15, 2.06s/it] 76%|███████▌ | 5713/7500 [3:14:32<1:01:26, 2.06s/it] 76%|███████▌ | 5714/7500 [3:14:34<1:02:30, 2.10s/it] 76%|███████▌ | 5715/7500 [3:14:36<1:01:32, 2.07s/it] 76%|███████▌ | 5716/7500 [3:14:38<1:01:21, 2.06s/it] 76%|███████▌ | 5717/7500 [3:14:40<1:01:24, 2.07s/it] 76%|███████▌ | 5718/7500 [3:14:42<1:01:27, 2.07s/it] 76%|███████▋ | 5719/7500 [3:14:44<1:01:18, 2.07s/it] 76%|███████▋ | 5720/7500 [3:14:46<1:00:54, 2.05s/it] {'loss': 0.3559, 'grad_norm': 2.7148547172546387, 'learning_rate': 8.099524404308948e-06, 'logqs/rejected': -2.6082966327667236, 'logqs/chosen': -0.8272854685783386, 'logqs_logps/rejected': -0.02368364855647087, 'logps_logqs/chosen': 0.202113077044487, 'epoch': 1.53} 76%|███████▋ | 5720/7500 [3:14:46<1:00:54, 2.05s/it] 76%|███████▋ | 5721/7500 [3:14:48<1:00:25, 2.04s/it] 76%|███████▋ | 5722/7500 [3:14:51<1:00:46, 2.05s/it] 76%|███████▋ | 5723/7500 [3:14:53<1:01:02, 2.06s/it] 76%|███████▋ | 5724/7500 [3:14:55<1:00:55, 2.06s/it] 76%|███████▋ | 5725/7500 [3:14:57<1:01:35, 2.08s/it] 76%|███████▋ | 5726/7500 [3:14:59<1:01:32, 2.08s/it] 76%|███████▋ | 5727/7500 [3:15:01<1:01:08, 2.07s/it] 76%|███████▋ | 5728/7500 [3:15:03<1:01:31, 2.08s/it] 76%|███████▋ | 5729/7500 [3:15:05<1:00:52, 2.06s/it] 76%|███████▋ | 5730/7500 [3:15:07<1:00:15, 2.04s/it] {'loss': 0.3467, 'grad_norm': 3.056844472885132, 'learning_rate': 8.013967393462094e-06, 'logqs/rejected': -2.6606366634368896, 'logqs/chosen': -0.8291158676147461, 'logqs_logps/rejected': -0.02373509667813778, 'logps_logqs/chosen': 0.2037593424320221, 'epoch': 1.53} 76%|███████▋ | 5730/7500 [3:15:07<1:00:15, 2.04s/it] 76%|███████▋ | 5731/7500 [3:15:09<1:00:32, 2.05s/it] 76%|███████▋ | 5732/7500 [3:15:11<1:00:53, 2.07s/it] 76%|███████▋ | 5733/7500 [3:15:13<1:01:37, 2.09s/it] 76%|███████▋ | 5734/7500 [3:15:15<1:01:10, 2.08s/it] 76%|███████▋ | 5735/7500 [3:15:17<1:01:05, 2.08s/it] 76%|███████▋ | 5736/7500 [3:15:20<1:00:45, 2.07s/it] 76%|███████▋ | 5737/7500 [3:15:22<1:00:53, 2.07s/it] 77%|███████▋ | 5738/7500 [3:15:24<1:00:10, 2.05s/it] 77%|███████▋ | 5739/7500 [3:15:26<1:00:14, 2.05s/it] 77%|███████▋ | 5740/7500 [3:15:28<1:00:44, 2.07s/it] {'loss': 0.3604, 'grad_norm': 2.6766693592071533, 'learning_rate': 7.92877832800792e-06, 'logqs/rejected': -2.4480507373809814, 'logqs/chosen': -0.850960910320282, 'logqs_logps/rejected': -0.018168658018112183, 'logps_logqs/chosen': 0.20779171586036682, 'epoch': 1.53} 77%|███████▋ | 5740/7500 [3:15:28<1:00:44, 2.07s/it] 77%|███████▋ | 5741/7500 [3:15:30<1:01:02, 2.08s/it] 77%|███████▋ | 5742/7500 [3:15:32<1:01:16, 2.09s/it] 77%|███████▋ | 5743/7500 [3:15:34<1:01:10, 2.09s/it] 77%|███████▋ | 5744/7500 [3:15:36<1:00:17, 2.06s/it] 77%|███████▋ | 5745/7500 [3:15:38<59:57, 2.05s/it] 77%|███████▋ | 5746/7500 [3:15:40<1:00:54, 2.08s/it] 77%|███████▋ | 5747/7500 [3:15:42<1:00:23, 2.07s/it] 77%|███████▋ | 5748/7500 [3:15:44<1:00:08, 2.06s/it] 77%|███████▋ | 5749/7500 [3:15:46<59:39, 2.04s/it] 77%|███████▋ | 5750/7500 [3:15:48<59:20, 2.03s/it] {'loss': 0.3649, 'grad_norm': 2.9520819187164307, 'learning_rate': 7.843959053281663e-06, 'logqs/rejected': -2.5411787033081055, 'logqs/chosen': -0.8538929224014282, 'logqs_logps/rejected': -0.019727537408471107, 'logps_logqs/chosen': 0.20206956565380096, 'epoch': 1.53} 77%|███████▋ | 5750/7500 [3:15:48<59:20, 2.03s/it] 77%|███████▋ | 5751/7500 [3:15:50<59:27, 2.04s/it] 77%|███████▋ | 5752/7500 [3:15:53<1:00:04, 2.06s/it] 77%|███████▋ | 5753/7500 [3:15:55<59:21, 2.04s/it] 77%|███████▋ | 5754/7500 [3:15:57<58:57, 2.03s/it] 77%|███████▋ | 5755/7500 [3:15:59<1:00:34, 2.08s/it] 77%|███████▋ | 5756/7500 [3:16:01<59:40, 2.05s/it] 77%|███████▋ | 5757/7500 [3:16:03<59:44, 2.06s/it] 77%|███████▋ | 5758/7500 [3:16:05<1:00:33, 2.09s/it] 77%|███████▋ | 5759/7500 [3:16:07<1:00:44, 2.09s/it] 77%|███████▋ | 5760/7500 [3:16:09<59:54, 2.07s/it] {'loss': 0.3584, 'grad_norm': 2.554075241088867, 'learning_rate': 7.759511406608255e-06, 'logqs/rejected': -2.61044979095459, 'logqs/chosen': -0.8533557653427124, 'logqs_logps/rejected': -0.021291252225637436, 'logps_logqs/chosen': 0.20334652066230774, 'epoch': 1.54} 77%|███████▋ | 5760/7500 [3:16:09<59:54, 2.07s/it] 77%|███████▋ | 5761/7500 [3:16:11<1:00:10, 2.08s/it] 77%|███████▋ | 5762/7500 [3:16:13<1:00:10, 2.08s/it] 77%|███████▋ | 5763/7500 [3:16:15<1:00:57, 2.11s/it] 77%|███████▋ | 5764/7500 [3:16:18<1:01:07, 2.11s/it] 77%|███████▋ | 5765/7500 [3:16:19<59:57, 2.07s/it] 77%|███████▋ | 5766/7500 [3:16:22<59:54, 2.07s/it] 77%|███████▋ | 5767/7500 [3:16:24<59:12, 2.05s/it] 77%|███████▋ | 5768/7500 [3:16:26<59:33, 2.06s/it] 77%|███████▋ | 5769/7500 [3:16:28<59:24, 2.06s/it] 77%|███████▋ | 5770/7500 [3:16:30<59:55, 2.08s/it] {'loss': 0.3602, 'grad_norm': 3.701611042022705, 'learning_rate': 7.675437217262571e-06, 'logqs/rejected': -2.460231304168701, 'logqs/chosen': -0.8542235493659973, 'logqs_logps/rejected': -0.02403974160552025, 'logps_logqs/chosen': 0.2007196694612503, 'epoch': 1.54} 77%|███████▋ | 5770/7500 [3:16:30<59:55, 2.08s/it] 77%|███████▋ | 5771/7500 [3:16:32<59:00, 2.05s/it] 77%|███████▋ | 5772/7500 [3:16:34<58:58, 2.05s/it] 77%|███████▋ | 5773/7500 [3:16:36<59:05, 2.05s/it] 77%|███████▋ | 5774/7500 [3:16:38<58:54, 2.05s/it] 77%|███████▋ | 5775/7500 [3:16:40<59:57, 2.09s/it] 77%|███████▋ | 5776/7500 [3:16:42<1:00:22, 2.10s/it] 77%|███████▋ | 5777/7500 [3:16:44<59:46, 2.08s/it] 77%|███████▋ | 5778/7500 [3:16:46<59:50, 2.09s/it] 77%|███████▋ | 5779/7500 [3:16:49<1:00:26, 2.11s/it] 77%|███████▋ | 5780/7500 [3:16:51<1:00:05, 2.10s/it] {'loss': 0.3384, 'grad_norm': 3.281830310821533, 'learning_rate': 7.591738306429769e-06, 'logqs/rejected': -2.453014373779297, 'logqs/chosen': -0.8207336664199829, 'logqs_logps/rejected': -0.022208575159311295, 'logps_logqs/chosen': 0.19951839745044708, 'epoch': 1.54} 77%|███████▋ | 5780/7500 [3:16:51<1:00:05, 2.10s/it] 77%|███████▋ | 5781/7500 [3:16:53<59:49, 2.09s/it] 77%|███████▋ | 5782/7500 [3:16:55<1:00:17, 2.11s/it] 77%|███████▋ | 5783/7500 [3:16:57<1:00:00, 2.10s/it] 77%|███████▋ | 5784/7500 [3:16:59<1:00:10, 2.10s/it] 77%|███████▋ | 5785/7500 [3:17:01<59:28, 2.08s/it] 77%|███████▋ | 5786/7500 [3:17:03<59:03, 2.07s/it] 77%|███████▋ | 5787/7500 [3:17:05<58:12, 2.04s/it] 77%|███████▋ | 5788/7500 [3:17:07<58:04, 2.04s/it] 77%|███████▋ | 5789/7500 [3:17:09<59:16, 2.08s/it] 77%|███████▋ | 5790/7500 [3:17:11<59:26, 2.09s/it] {'loss': 0.347, 'grad_norm': 3.8202080726623535, 'learning_rate': 7.508416487165862e-06, 'logqs/rejected': -2.5340259075164795, 'logqs/chosen': -0.8502823710441589, 'logqs_logps/rejected': -0.02761649526655674, 'logps_logqs/chosen': 0.21136395633220673, 'epoch': 1.54} 77%|███████▋ | 5790/7500 [3:17:11<59:26, 2.09s/it] 77%|███████▋ | 5791/7500 [3:17:13<58:54, 2.07s/it] 77%|███████▋ | 5792/7500 [3:17:16<59:07, 2.08s/it] 77%|███████▋ | 5793/7500 [3:17:18<58:51, 2.07s/it] 77%|███████▋ | 5794/7500 [3:17:20<59:31, 2.09s/it] 77%|███████▋ | 5795/7500 [3:17:22<59:05, 2.08s/it] 77%|███████▋ | 5796/7500 [3:17:24<59:07, 2.08s/it] 77%|███████▋ | 5797/7500 [3:17:26<59:24, 2.09s/it] 77%|███████▋ | 5798/7500 [3:17:28<59:34, 2.10s/it] 77%|███████▋ | 5799/7500 [3:17:30<59:16, 2.09s/it] 77%|███████▋ | 5800/7500 [3:17:32<59:08, 2.09s/it] {'loss': 0.3486, 'grad_norm': 2.974259853363037, 'learning_rate': 7.4254735643584564e-06, 'logqs/rejected': -2.3696725368499756, 'logqs/chosen': -0.860556423664093, 'logqs_logps/rejected': -0.026585351675748825, 'logps_logqs/chosen': 0.2016081064939499, 'epoch': 1.55} 77%|███████▋ | 5800/7500 [3:17:32<59:08, 2.09s/it] 77%|███████▋ | 5801/7500 [3:17:34<59:02, 2.09s/it] 77%|███████▋ | 5802/7500 [3:17:36<59:02, 2.09s/it] 77%|███████▋ | 5803/7500 [3:17:38<58:08, 2.06s/it] 77%|███████▋ | 5804/7500 [3:17:40<57:48, 2.05s/it] 77%|███████▋ | 5805/7500 [3:17:42<57:57, 2.05s/it] 77%|███████▋ | 5806/7500 [3:17:45<58:11, 2.06s/it] 77%|███████▋ | 5807/7500 [3:17:47<58:20, 2.07s/it] 77%|███████▋ | 5808/7500 [3:17:49<58:57, 2.09s/it] 77%|███████▋ | 5809/7500 [3:17:51<59:23, 2.11s/it] 77%|███████▋ | 5810/7500 [3:17:53<59:20, 2.11s/it] {'loss': 0.3496, 'grad_norm': 3.0943210124969482, 'learning_rate': 7.342911334687619e-06, 'logqs/rejected': -2.532510757446289, 'logqs/chosen': -0.8391607999801636, 'logqs_logps/rejected': -0.02022623084485531, 'logps_logqs/chosen': 0.2047225534915924, 'epoch': 1.55} 77%|███████▋ | 5810/7500 [3:17:53<59:20, 2.11s/it] 77%|███████▋ | 5811/7500 [3:17:55<59:04, 2.10s/it] 77%|███████▋ | 5812/7500 [3:17:57<59:02, 2.10s/it] 78%|███████▊ | 5813/7500 [3:17:59<1:00:00, 2.13s/it] 78%|███████▊ | 5814/7500 [3:18:01<58:34, 2.08s/it] 78%|███████▊ | 5815/7500 [3:18:04<59:10, 2.11s/it] 78%|███████▊ | 5816/7500 [3:18:06<59:29, 2.12s/it] 78%|███████▊ | 5817/7500 [3:18:08<58:56, 2.10s/it] 78%|███████▊ | 5818/7500 [3:18:10<57:52, 2.06s/it] 78%|███████▊ | 5819/7500 [3:18:12<58:21, 2.08s/it] 78%|███████▊ | 5820/7500 [3:18:14<57:35, 2.06s/it] {'loss': 0.3546, 'grad_norm': 2.8120625019073486, 'learning_rate': 7.260731586586983e-06, 'logqs/rejected': -2.505751371383667, 'logqs/chosen': -0.8247493505477905, 'logqs_logps/rejected': -0.02267501689493656, 'logps_logqs/chosen': 0.19661854207515717, 'epoch': 1.55} 78%|███████▊ | 5820/7500 [3:18:14<57:35, 2.06s/it] 78%|███████▊ | 5821/7500 [3:18:16<57:25, 2.05s/it] 78%|███████▊ | 5822/7500 [3:18:18<57:20, 2.05s/it] 78%|███████▊ | 5823/7500 [3:18:20<57:51, 2.07s/it] 78%|███████▊ | 5824/7500 [3:18:22<58:06, 2.08s/it] 78%|███████▊ | 5825/7500 [3:18:24<56:55, 2.04s/it] 78%|███████▊ | 5826/7500 [3:18:26<56:45, 2.03s/it] 78%|███████▊ | 5827/7500 [3:18:28<56:38, 2.03s/it] 78%|███████▊ | 5828/7500 [3:18:30<56:44, 2.04s/it] 78%|███████▊ | 5829/7500 [3:18:32<57:26, 2.06s/it] 78%|███████▊ | 5830/7500 [3:18:34<57:46, 2.08s/it] {'loss': 0.3671, 'grad_norm': 2.559566020965576, 'learning_rate': 7.178936100204994e-06, 'logqs/rejected': -2.667365789413452, 'logqs/chosen': -0.8294084668159485, 'logqs_logps/rejected': -0.022877100855112076, 'logps_logqs/chosen': 0.20213189721107483, 'epoch': 1.55} 78%|███████▊ | 5830/7500 [3:18:34<57:46, 2.08s/it] 78%|███████▊ | 5831/7500 [3:18:37<58:04, 2.09s/it] 78%|███████▊ | 5832/7500 [3:18:39<58:14, 2.09s/it] 78%|███████▊ | 5833/7500 [3:18:41<58:03, 2.09s/it] 78%|███████▊ | 5834/7500 [3:18:43<57:50, 2.08s/it] 78%|███████▊ | 5835/7500 [3:18:45<57:40, 2.08s/it] 78%|███████▊ | 5836/7500 [3:18:47<58:33, 2.11s/it] 78%|███████▊ | 5837/7500 [3:18:49<58:56, 2.13s/it] 78%|███████▊ | 5838/7500 [3:18:51<58:31, 2.11s/it] 78%|███████▊ | 5839/7500 [3:18:54<59:41, 2.16s/it] 78%|███████▊ | 5840/7500 [3:18:56<59:08, 2.14s/it] {'loss': 0.336, 'grad_norm': 3.2442660331726074, 'learning_rate': 7.097526647366379e-06, 'logqs/rejected': -2.206770658493042, 'logqs/chosen': -0.8618705868721008, 'logqs_logps/rejected': -0.026197034865617752, 'logps_logqs/chosen': 0.20550088584423065, 'epoch': 1.56} 78%|███████▊ | 5840/7500 [3:18:56<59:08, 2.14s/it] 78%|███████▊ | 5841/7500 [3:18:58<58:04, 2.10s/it] 78%|███████▊ | 5842/7500 [3:19:00<57:37, 2.09s/it] 78%|███████▊ | 5843/7500 [3:19:02<57:02, 2.07s/it] 78%|███████▊ | 5844/7500 [3:19:04<56:42, 2.05s/it] 78%|███████▊ | 5845/7500 [3:19:06<57:44, 2.09s/it] 78%|███████▊ | 5846/7500 [3:19:08<57:02, 2.07s/it] 78%|███████▊ | 5847/7500 [3:19:10<56:27, 2.05s/it] 78%|███████▊ | 5848/7500 [3:19:12<56:45, 2.06s/it] 78%|███████▊ | 5849/7500 [3:19:14<56:15, 2.04s/it] 78%|███████▊ | 5850/7500 [3:19:16<57:17, 2.08s/it] {'loss': 0.3439, 'grad_norm': 3.982052803039551, 'learning_rate': 7.016504991533726e-06, 'logqs/rejected': -2.4991183280944824, 'logqs/chosen': -0.84912109375, 'logqs_logps/rejected': -0.026729637756943703, 'logps_logqs/chosen': 0.2081836462020874, 'epoch': 1.56} 78%|███████▊ | 5850/7500 [3:19:16<57:17, 2.08s/it] 78%|███████▊ | 5851/7500 [3:19:18<57:08, 2.08s/it] 78%|███████▊ | 5852/7500 [3:19:20<56:29, 2.06s/it] 78%|███████▊ | 5853/7500 [3:19:22<56:46, 2.07s/it] 78%|███████▊ | 5854/7500 [3:19:25<57:34, 2.10s/it] 78%|███████▊ | 5855/7500 [3:19:27<57:41, 2.10s/it] 78%|███████▊ | 5856/7500 [3:19:29<57:46, 2.11s/it] 78%|███████▊ | 5857/7500 [3:19:31<56:54, 2.08s/it] 78%|███████▊ | 5858/7500 [3:19:33<57:04, 2.09s/it] 78%|███████▊ | 5859/7500 [3:19:35<57:04, 2.09s/it] 78%|███████▊ | 5860/7500 [3:19:37<58:25, 2.14s/it] {'loss': 0.3542, 'grad_norm': 2.8086562156677246, 'learning_rate': 6.9358728877692986e-06, 'logqs/rejected': -2.371915102005005, 'logqs/chosen': -0.8678433299064636, 'logqs_logps/rejected': -0.023416221141815186, 'logps_logqs/chosen': 0.20276841521263123, 'epoch': 1.56} 78%|███████▊ | 5860/7500 [3:19:37<58:25, 2.14s/it] 78%|███████▊ | 5861/7500 [3:19:39<58:27, 2.14s/it] 78%|███████▊ | 5862/7500 [3:19:41<57:32, 2.11s/it] 78%|███████▊ | 5863/7500 [3:19:44<57:14, 2.10s/it] 78%|███████▊ | 5864/7500 [3:19:46<56:35, 2.08s/it] 78%|███████▊ | 5865/7500 [3:19:48<57:23, 2.11s/it] 78%|███████▊ | 5866/7500 [3:19:50<56:56, 2.09s/it] 78%|███████▊ | 5867/7500 [3:19:52<56:55, 2.09s/it] 78%|███████▊ | 5868/7500 [3:19:54<57:08, 2.10s/it] 78%|███████▊ | 5869/7500 [3:19:56<56:23, 2.07s/it] 78%|███████▊ | 5870/7500 [3:19:58<56:22, 2.07s/it] {'loss': 0.3416, 'grad_norm': 2.7078638076782227, 'learning_rate': 6.855632082697045e-06, 'logqs/rejected': -2.6392669677734375, 'logqs/chosen': -0.8405951261520386, 'logqs_logps/rejected': -0.030777398496866226, 'logps_logqs/chosen': 0.20735923945903778, 'epoch': 1.57} 78%|███████▊ | 5870/7500 [3:19:58<56:22, 2.07s/it] 78%|███████▊ | 5871/7500 [3:20:00<56:50, 2.09s/it] 78%|███████▊ | 5872/7500 [3:20:02<56:09, 2.07s/it] 78%|███████▊ | 5873/7500 [3:20:04<57:34, 2.12s/it] 78%|███████▊ | 5874/7500 [3:20:07<57:23, 2.12s/it] 78%|███████▊ | 5875/7500 [3:20:09<56:31, 2.09s/it] 78%|███████▊ | 5876/7500 [3:20:11<57:24, 2.12s/it] 78%|███████▊ | 5877/7500 [3:20:13<57:43, 2.13s/it] 78%|███████▊ | 5878/7500 [3:20:15<57:23, 2.12s/it] 78%|███████▊ | 5879/7500 [3:20:17<58:20, 2.16s/it] 78%|███████▊ | 5880/7500 [3:20:19<57:23, 2.13s/it] {'loss': 0.3445, 'grad_norm': 2.968248128890991, 'learning_rate': 6.775784314464717e-06, 'logqs/rejected': -2.3198883533477783, 'logqs/chosen': -0.8346685171127319, 'logqs_logps/rejected': -0.033934272825717926, 'logps_logqs/chosen': 0.20074817538261414, 'epoch': 1.57} 78%|███████▊ | 5880/7500 [3:20:19<57:23, 2.13s/it] 78%|███████▊ | 5881/7500 [3:20:21<56:29, 2.09s/it] 78%|███████▊ | 5882/7500 [3:20:23<55:42, 2.07s/it] 78%|███████▊ | 5883/7500 [3:20:25<55:50, 2.07s/it] 78%|███████▊ | 5884/7500 [3:20:28<56:43, 2.11s/it] 78%|███████▊ | 5885/7500 [3:20:30<56:27, 2.10s/it] 78%|███████▊ | 5886/7500 [3:20:32<56:36, 2.10s/it] 78%|███████▊ | 5887/7500 [3:20:34<55:57, 2.08s/it] 79%|███████▊ | 5888/7500 [3:20:36<55:50, 2.08s/it] 79%|███████▊ | 5889/7500 [3:20:38<56:30, 2.10s/it] 79%|███████▊ | 5890/7500 [3:20:40<57:02, 2.13s/it] {'loss': 0.3608, 'grad_norm': 2.9871156215667725, 'learning_rate': 6.696331312706244e-06, 'logqs/rejected': -2.6377973556518555, 'logqs/chosen': -0.8361560702323914, 'logqs_logps/rejected': -0.029904022812843323, 'logps_logqs/chosen': 0.20485492050647736, 'epoch': 1.57} 79%|███████▊ | 5890/7500 [3:20:40<57:02, 2.13s/it] 79%|███████▊ | 5891/7500 [3:20:42<56:42, 2.11s/it] 79%|███████▊ | 5892/7500 [3:20:44<55:51, 2.08s/it] 79%|███████▊ | 5893/7500 [3:20:47<56:24, 2.11s/it] 79%|███████▊ | 5894/7500 [3:20:49<56:10, 2.10s/it] 79%|███████▊ | 5895/7500 [3:20:51<56:53, 2.13s/it] 79%|███████▊ | 5896/7500 [3:20:53<56:09, 2.10s/it] 79%|███████▊ | 5897/7500 [3:20:55<55:50, 2.09s/it] 79%|███████▊ | 5898/7500 [3:20:57<55:29, 2.08s/it] 79%|███████▊ | 5899/7500 [3:20:59<55:14, 2.07s/it] 79%|███████▊ | 5900/7500 [3:21:01<55:30, 2.08s/it] {'loss': 0.3552, 'grad_norm': 2.985283374786377, 'learning_rate': 6.617274798504286e-06, 'logqs/rejected': -2.5942907333374023, 'logqs/chosen': -0.849206268787384, 'logqs_logps/rejected': -0.02135365642607212, 'logps_logqs/chosen': 0.20906808972358704, 'epoch': 1.57} 79%|███████▊ | 5900/7500 [3:21:01<55:30, 2.08s/it] 79%|███████▊ | 5901/7500 [3:21:03<56:23, 2.12s/it] 79%|███████▊ | 5902/7500 [3:21:05<55:47, 2.09s/it] 79%|███████▊ | 5903/7500 [3:21:08<56:29, 2.12s/it] 79%|███████▊ | 5904/7500 [3:21:10<55:03, 2.07s/it] 79%|███████▊ | 5905/7500 [3:21:12<55:02, 2.07s/it] 79%|███████▊ | 5906/7500 [3:21:14<55:42, 2.10s/it] 79%|███████▉ | 5907/7500 [3:21:16<56:15, 2.12s/it] 79%|███████▉ | 5908/7500 [3:21:18<56:30, 2.13s/it] 79%|███████▉ | 5909/7500 [3:21:20<56:47, 2.14s/it] 79%|███████▉ | 5910/7500 [3:21:22<56:40, 2.14s/it] {'loss': 0.3384, 'grad_norm': 3.0448672771453857, 'learning_rate': 6.538616484352902e-06, 'logqs/rejected': -2.285767078399658, 'logqs/chosen': -0.8317879438400269, 'logqs_logps/rejected': -0.029744554311037064, 'logps_logqs/chosen': 0.20169024169445038, 'epoch': 1.58} 79%|███████▉ | 5910/7500 [3:21:22<56:40, 2.14s/it] 79%|███████▉ | 5911/7500 [3:21:24<56:33, 2.14s/it] 79%|███████▉ | 5912/7500 [3:21:27<56:04, 2.12s/it] 79%|███████▉ | 5913/7500 [3:21:29<56:31, 2.14s/it] 79%|███████▉ | 5914/7500 [3:21:31<56:04, 2.12s/it] 79%|███████▉ | 5915/7500 [3:21:33<56:21, 2.13s/it] 79%|███████▉ | 5916/7500 [3:21:35<55:19, 2.10s/it] 79%|███████▉ | 5917/7500 [3:21:37<55:22, 2.10s/it] 79%|███████▉ | 5918/7500 [3:21:39<54:34, 2.07s/it] 79%|███████▉ | 5919/7500 [3:21:41<54:59, 2.09s/it] 79%|███████▉ | 5920/7500 [3:21:43<55:41, 2.11s/it] {'loss': 0.3501, 'grad_norm': 2.697298526763916, 'learning_rate': 6.460358074120518e-06, 'logqs/rejected': -2.5959572792053223, 'logqs/chosen': -0.8285811543464661, 'logqs_logps/rejected': -0.020691480487585068, 'logps_logqs/chosen': 0.20209333300590515, 'epoch': 1.58} 79%|███████▉ | 5920/7500 [3:21:43<55:41, 2.11s/it] 79%|███████▉ | 5921/7500 [3:21:46<56:00, 2.13s/it] 79%|███████▉ | 5922/7500 [3:21:48<55:39, 2.12s/it] 79%|███████▉ | 5923/7500 [3:21:50<55:50, 2.12s/it] 79%|███████▉ | 5924/7500 [3:21:52<56:06, 2.14s/it] 79%|███████▉ | 5925/7500 [3:21:54<57:09, 2.18s/it] 79%|███████▉ | 5926/7500 [3:21:56<56:30, 2.15s/it] 79%|███████▉ | 5927/7500 [3:21:58<55:28, 2.12s/it] 79%|███████▉ | 5928/7500 [3:22:00<55:13, 2.11s/it] 79%|███████▉ | 5929/7500 [3:22:03<55:20, 2.11s/it] 79%|███████▉ | 5930/7500 [3:22:05<54:12, 2.07s/it] {'loss': 0.3366, 'grad_norm': 2.6575868129730225, 'learning_rate': 6.382501263012935e-06, 'logqs/rejected': -2.4227161407470703, 'logqs/chosen': -0.8588000535964966, 'logqs_logps/rejected': -0.0286735650151968, 'logps_logqs/chosen': 0.20581527054309845, 'epoch': 1.58} 79%|███████▉ | 5930/7500 [3:22:05<54:12, 2.07s/it] 79%|███████▉ | 5931/7500 [3:22:07<54:20, 2.08s/it] 79%|███████▉ | 5932/7500 [3:22:09<54:24, 2.08s/it] 79%|███████▉ | 5933/7500 [3:22:11<53:30, 2.05s/it] 79%|███████▉ | 5934/7500 [3:22:13<53:10, 2.04s/it] 79%|███████▉ | 5935/7500 [3:22:15<53:57, 2.07s/it] 79%|███████▉ | 5936/7500 [3:22:17<53:32, 2.05s/it] 79%|███████▉ | 5937/7500 [3:22:19<53:56, 2.07s/it] 79%|███████▉ | 5938/7500 [3:22:21<54:08, 2.08s/it] 79%|███████▉ | 5939/7500 [3:22:23<53:51, 2.07s/it] 79%|███████▉ | 5940/7500 [3:22:25<53:53, 2.07s/it] {'loss': 0.3509, 'grad_norm': 2.682849407196045, 'learning_rate': 6.305047737536707e-06, 'logqs/rejected': -2.564943552017212, 'logqs/chosen': -0.8548133969306946, 'logqs_logps/rejected': -0.023925671353936195, 'logps_logqs/chosen': 0.20245662331581116, 'epoch': 1.58} 79%|███████▉ | 5940/7500 [3:22:25<53:53, 2.07s/it] 79%|███████▉ | 5941/7500 [3:22:27<54:12, 2.09s/it] 79%|███████▉ | 5942/7500 [3:22:29<54:16, 2.09s/it] 79%|███████▉ | 5943/7500 [3:22:32<53:59, 2.08s/it] 79%|███████▉ | 5944/7500 [3:22:34<53:58, 2.08s/it] 79%|███████▉ | 5945/7500 [3:22:36<54:53, 2.12s/it] 79%|███████▉ | 5946/7500 [3:22:38<54:33, 2.11s/it] 79%|███████▉ | 5947/7500 [3:22:40<53:43, 2.08s/it] 79%|███████▉ | 5948/7500 [3:22:42<54:20, 2.10s/it] 79%|███████▉ | 5949/7500 [3:22:44<54:07, 2.09s/it] 79%|███████▉ | 5950/7500 [3:22:46<54:26, 2.11s/it] {'loss': 0.3482, 'grad_norm': 2.8588640689849854, 'learning_rate': 6.22799917546252e-06, 'logqs/rejected': -2.5013318061828613, 'logqs/chosen': -0.8443573117256165, 'logqs_logps/rejected': -0.025652030482888222, 'logps_logqs/chosen': 0.20114299654960632, 'epoch': 1.59} 79%|███████▉ | 5950/7500 [3:22:46<54:26, 2.11s/it] 79%|███████▉ | 5951/7500 [3:22:48<54:42, 2.12s/it] 79%|███████▉ | 5952/7500 [3:22:50<54:08, 2.10s/it] 79%|███████▉ | 5953/7500 [3:22:53<54:12, 2.10s/it] 79%|███████▉ | 5954/7500 [3:22:55<54:43, 2.12s/it] 79%|███████▉ | 5955/7500 [3:22:57<54:10, 2.10s/it] 79%|███████▉ | 5956/7500 [3:22:59<54:02, 2.10s/it] 79%|███████▉ | 5957/7500 [3:23:01<53:43, 2.09s/it] 79%|███████▉ | 5958/7500 [3:23:03<53:02, 2.06s/it] 79%|███████▉ | 5959/7500 [3:23:05<52:49, 2.06s/it] 79%|███████▉ | 5960/7500 [3:23:07<52:30, 2.05s/it] {'loss': 0.3624, 'grad_norm': 47.48930358886719, 'learning_rate': 6.151357245788916e-06, 'logqs/rejected': -2.5240511894226074, 'logqs/chosen': -0.8601707220077515, 'logqs_logps/rejected': -0.028933633118867874, 'logps_logqs/chosen': 0.20663245022296906, 'epoch': 1.59} 79%|███████▉ | 5960/7500 [3:23:07<52:30, 2.05s/it] 79%|███████▉ | 5961/7500 [3:23:09<52:52, 2.06s/it] 79%|███████▉ | 5962/7500 [3:23:11<53:08, 2.07s/it] 80%|███████▉ | 5963/7500 [3:23:13<53:08, 2.07s/it] 80%|███████▉ | 5964/7500 [3:23:15<53:05, 2.07s/it] 80%|███████▉ | 5965/7500 [3:23:17<52:52, 2.07s/it] 80%|███████▉ | 5966/7500 [3:23:19<52:38, 2.06s/it] 80%|███████▉ | 5967/7500 [3:23:22<53:07, 2.08s/it] 80%|███████▉ | 5968/7500 [3:23:24<53:29, 2.09s/it] 80%|███████▉ | 5969/7500 [3:23:26<53:13, 2.09s/it] 80%|███████▉ | 5970/7500 [3:23:28<53:18, 2.09s/it] {'loss': 0.3576, 'grad_norm': 3.0998129844665527, 'learning_rate': 6.075123608706093e-06, 'logqs/rejected': -2.710658550262451, 'logqs/chosen': -0.8463001251220703, 'logqs_logps/rejected': -0.024119718000292778, 'logps_logqs/chosen': 0.20310774445533752, 'epoch': 1.59} 80%|███████▉ | 5970/7500 [3:23:28<53:18, 2.09s/it] 80%|███████▉ | 5971/7500 [3:23:30<54:16, 2.13s/it] 80%|███████▉ | 5972/7500 [3:23:32<53:27, 2.10s/it] 80%|███████▉ | 5973/7500 [3:23:34<53:03, 2.08s/it] 80%|███████▉ | 5974/7500 [3:23:36<53:33, 2.11s/it] 80%|███████▉ | 5975/7500 [3:23:38<53:51, 2.12s/it] 80%|███████▉ | 5976/7500 [3:23:41<53:22, 2.10s/it] 80%|███████▉ | 5977/7500 [3:23:43<52:35, 2.07s/it] 80%|███████▉ | 5978/7500 [3:23:45<52:38, 2.08s/it] 80%|███████▉ | 5979/7500 [3:23:47<52:16, 2.06s/it] 80%|███████▉ | 5980/7500 [3:23:49<52:15, 2.06s/it] {'loss': 0.3805, 'grad_norm': 2.3408255577087402, 'learning_rate': 5.999299915559956e-06, 'logqs/rejected': -2.6398632526397705, 'logqs/chosen': -0.8594229817390442, 'logqs_logps/rejected': -0.01721104606986046, 'logps_logqs/chosen': 0.2066006362438202, 'epoch': 1.59} 80%|███████▉ | 5980/7500 [3:23:49<52:15, 2.06s/it] 80%|███████▉ | 5981/7500 [3:23:51<51:47, 2.05s/it] 80%|███████▉ | 5982/7500 [3:23:53<51:50, 2.05s/it] 80%|███████▉ | 5983/7500 [3:23:55<51:43, 2.05s/it] 80%|███████▉ | 5984/7500 [3:23:57<51:25, 2.04s/it] 80%|███████▉ | 5985/7500 [3:23:59<51:37, 2.04s/it] 80%|███████▉ | 5986/7500 [3:24:01<51:28, 2.04s/it] 80%|███████▉ | 5987/7500 [3:24:03<51:43, 2.05s/it] 80%|███████▉ | 5988/7500 [3:24:05<52:11, 2.07s/it] 80%|███████▉ | 5989/7500 [3:24:07<51:48, 2.06s/it] 80%|███████▉ | 5990/7500 [3:24:09<51:13, 2.04s/it] {'loss': 0.366, 'grad_norm': 2.6636013984680176, 'learning_rate': 5.923887808816372e-06, 'logqs/rejected': -2.7223551273345947, 'logqs/chosen': -0.8511573672294617, 'logqs_logps/rejected': -0.015890777111053467, 'logps_logqs/chosen': 0.20608767867088318, 'epoch': 1.6} 80%|███████▉ | 5990/7500 [3:24:09<51:13, 2.04s/it] 80%|███████▉ | 5991/7500 [3:24:11<51:11, 2.04s/it] 80%|███████▉ | 5992/7500 [3:24:13<51:21, 2.04s/it] 80%|███████▉ | 5993/7500 [3:24:15<52:06, 2.07s/it] 80%|███████▉ | 5994/7500 [3:24:17<51:26, 2.05s/it] 80%|███████▉ | 5995/7500 [3:24:19<51:50, 2.07s/it] 80%|███████▉ | 5996/7500 [3:24:21<51:19, 2.05s/it] 80%|███████▉ | 5997/7500 [3:24:24<51:18, 2.05s/it] 80%|███████▉ | 5998/7500 [3:24:26<51:48, 2.07s/it] 80%|███████▉ | 5999/7500 [3:24:28<51:58, 2.08s/it] 80%|████████ | 6000/7500 [3:24:30<51:41, 2.07s/it] {'loss': 0.3551, 'grad_norm': 3.0818698406219482, 'learning_rate': 5.848888922025553e-06, 'logqs/rejected': -2.5741372108459473, 'logqs/chosen': -0.8325112462043762, 'logqs_logps/rejected': -0.025827834382653236, 'logps_logqs/chosen': 0.20447711646556854, 'epoch': 1.6} 80%|████████ | 6000/7500 [3:24:30<51:41, 2.07s/it] 80%|████████ | 6001/7500 [3:24:32<51:16, 2.05s/it] 80%|████████ | 6002/7500 [3:24:34<51:18, 2.06s/it] 80%|████████ | 6003/7500 [3:24:36<51:11, 2.05s/it] 80%|████████ | 6004/7500 [3:24:38<51:15, 2.06s/it] 80%|████████ | 6005/7500 [3:24:40<51:08, 2.05s/it] 80%|████████ | 6006/7500 [3:24:42<51:21, 2.06s/it] 80%|████████ | 6007/7500 [3:24:44<51:51, 2.08s/it] 80%|████████ | 6008/7500 [3:24:46<51:32, 2.07s/it] 80%|████████ | 6009/7500 [3:24:48<51:32, 2.07s/it] 80%|████████ | 6010/7500 [3:24:51<52:04, 2.10s/it] {'loss': 0.371, 'grad_norm': 2.900057792663574, 'learning_rate': 5.774304879786688e-06, 'logqs/rejected': -2.7011685371398926, 'logqs/chosen': -0.8413955569267273, 'logqs_logps/rejected': -0.0213993638753891, 'logps_logqs/chosen': 0.20719707012176514, 'epoch': 1.6} 80%|████████ | 6010/7500 [3:24:51<52:04, 2.10s/it] 80%|████████ | 6011/7500 [3:24:53<51:11, 2.06s/it] 80%|████████ | 6012/7500 [3:24:55<51:41, 2.08s/it] 80%|████████ | 6013/7500 [3:24:57<51:54, 2.09s/it] 80%|████████ | 6014/7500 [3:24:59<51:56, 2.10s/it] 80%|████████ | 6015/7500 [3:25:01<51:38, 2.09s/it] 80%|████████ | 6016/7500 [3:25:03<51:13, 2.07s/it] 80%|████████ | 6017/7500 [3:25:05<51:11, 2.07s/it] 80%|████████ | 6018/7500 [3:25:07<50:47, 2.06s/it] 80%|████████ | 6019/7500 [3:25:09<51:29, 2.09s/it] 80%|████████ | 6020/7500 [3:25:11<51:50, 2.10s/it] {'loss': 0.3401, 'grad_norm': 2.813662528991699, 'learning_rate': 5.7001372977127485e-06, 'logqs/rejected': -2.3336598873138428, 'logqs/chosen': -0.8522003889083862, 'logqs_logps/rejected': -0.02353980578482151, 'logps_logqs/chosen': 0.2052021473646164, 'epoch': 1.61} 80%|████████ | 6020/7500 [3:25:11<51:50, 2.10s/it] 80%|████████ | 6021/7500 [3:25:13<51:04, 2.07s/it] 80%|████████ | 6022/7500 [3:25:15<50:49, 2.06s/it] 80%|████████ | 6023/7500 [3:25:17<50:29, 2.05s/it] 80%|████████ | 6024/7500 [3:25:19<49:40, 2.02s/it] 80%|████████ | 6025/7500 [3:25:21<50:06, 2.04s/it] 80%|████████ | 6026/7500 [3:25:24<50:24, 2.05s/it] 80%|████████ | 6027/7500 [3:25:26<50:49, 2.07s/it] 80%|████████ | 6028/7500 [3:25:28<50:31, 2.06s/it] 80%|████████ | 6029/7500 [3:25:30<50:30, 2.06s/it] 80%|████████ | 6030/7500 [3:25:32<49:57, 2.04s/it] {'loss': 0.3522, 'grad_norm': 3.2330901622772217, 'learning_rate': 5.626387782395512e-06, 'logqs/rejected': -2.516920566558838, 'logqs/chosen': -0.8646548986434937, 'logqs_logps/rejected': -0.02678055502474308, 'logps_logqs/chosen': 0.2050189971923828, 'epoch': 1.61} 80%|████████ | 6030/7500 [3:25:32<49:57, 2.04s/it] 80%|████████ | 6031/7500 [3:25:34<50:24, 2.06s/it] 80%|████████ | 6032/7500 [3:25:36<51:02, 2.09s/it] 80%|████████ | 6033/7500 [3:25:38<50:50, 2.08s/it] 80%|████████ | 6034/7500 [3:25:40<50:38, 2.07s/it] 80%|████████ | 6035/7500 [3:25:42<50:50, 2.08s/it] 80%|████████ | 6036/7500 [3:25:44<50:24, 2.07s/it] 80%|████████ | 6037/7500 [3:25:46<50:42, 2.08s/it] 81%|████████ | 6038/7500 [3:25:48<50:07, 2.06s/it] 81%|████████ | 6039/7500 [3:25:50<49:47, 2.04s/it] 81%|████████ | 6040/7500 [3:25:52<49:50, 2.05s/it] {'loss': 0.3592, 'grad_norm': 2.845196008682251, 'learning_rate': 5.553057931370728e-06, 'logqs/rejected': -2.7715752124786377, 'logqs/chosen': -0.8540598154067993, 'logqs_logps/rejected': -0.023146888241171837, 'logps_logqs/chosen': 0.20971941947937012, 'epoch': 1.61} 81%|████████ | 6040/7500 [3:25:52<49:50, 2.05s/it] 81%|████████ | 6041/7500 [3:25:55<50:45, 2.09s/it] 81%|████████ | 6042/7500 [3:25:57<50:40, 2.09s/it] 81%|████████ | 6043/7500 [3:25:59<50:08, 2.06s/it] 81%|████████ | 6044/7500 [3:26:01<50:01, 2.06s/it] 81%|████████ | 6045/7500 [3:26:03<50:18, 2.07s/it] 81%|████████ | 6046/7500 [3:26:05<51:11, 2.11s/it] 81%|████████ | 6047/7500 [3:26:07<51:13, 2.12s/it] 81%|████████ | 6048/7500 [3:26:09<50:28, 2.09s/it] 81%|████████ | 6049/7500 [3:26:11<50:20, 2.08s/it] 81%|████████ | 6050/7500 [3:26:13<50:25, 2.09s/it] {'loss': 0.3507, 'grad_norm': 3.4207746982574463, 'learning_rate': 5.48014933308352e-06, 'logqs/rejected': -2.3523144721984863, 'logqs/chosen': -0.8376720547676086, 'logqs_logps/rejected': -0.027146419510245323, 'logps_logqs/chosen': 0.20108743011951447, 'epoch': 1.61} 81%|████████ | 6050/7500 [3:26:13<50:25, 2.09s/it] 81%|████████ | 6051/7500 [3:26:15<50:08, 2.08s/it] 81%|████████ | 6052/7500 [3:26:17<50:00, 2.07s/it] 81%|████████ | 6053/7500 [3:26:19<49:34, 2.06s/it] 81%|████████ | 6054/7500 [3:26:22<49:42, 2.06s/it] 81%|████████ | 6055/7500 [3:26:24<49:38, 2.06s/it] 81%|████████ | 6056/7500 [3:26:26<49:11, 2.04s/it] 81%|████████ | 6057/7500 [3:26:28<49:35, 2.06s/it] 81%|████████ | 6058/7500 [3:26:30<49:56, 2.08s/it] 81%|████████ | 6059/7500 [3:26:32<49:57, 2.08s/it] 81%|████████ | 6060/7500 [3:26:34<49:32, 2.06s/it] {'loss': 0.3312, 'grad_norm': 2.556411027908325, 'learning_rate': 5.4076635668540075e-06, 'logqs/rejected': -2.208439588546753, 'logqs/chosen': -0.8429640531539917, 'logqs_logps/rejected': -0.02918125130236149, 'logps_logqs/chosen': 0.2049146145582199, 'epoch': 1.62} 81%|████████ | 6060/7500 [3:26:34<49:32, 2.06s/it] 81%|████████ | 6061/7500 [3:26:36<49:23, 2.06s/it] 81%|████████ | 6062/7500 [3:26:38<49:52, 2.08s/it] 81%|████████ | 6063/7500 [3:26:40<49:23, 2.06s/it] 81%|████████ | 6064/7500 [3:26:42<48:52, 2.04s/it] 81%|████████ | 6065/7500 [3:26:44<49:37, 2.08s/it] 81%|████████ | 6066/7500 [3:26:46<49:56, 2.09s/it] 81%|████████ | 6067/7500 [3:26:49<49:50, 2.09s/it] 81%|████████ | 6068/7500 [3:26:51<49:34, 2.08s/it] 81%|████████ | 6069/7500 [3:26:53<48:40, 2.04s/it] 81%|████████ | 6070/7500 [3:26:55<48:33, 2.04s/it] {'loss': 0.349, 'grad_norm': 2.591247081756592, 'learning_rate': 5.335602202843054e-06, 'logqs/rejected': -2.4644556045532227, 'logqs/chosen': -0.8463153839111328, 'logqs_logps/rejected': -0.02464006468653679, 'logps_logqs/chosen': 0.20469801127910614, 'epoch': 1.62} 81%|████████ | 6070/7500 [3:26:55<48:33, 2.04s/it] 81%|████████ | 6071/7500 [3:26:57<48:57, 2.06s/it] 81%|████████ | 6072/7500 [3:26:59<48:36, 2.04s/it] 81%|████████ | 6073/7500 [3:27:01<48:49, 2.05s/it] 81%|████████ | 6074/7500 [3:27:03<49:27, 2.08s/it] 81%|████████ | 6075/7500 [3:27:05<49:44, 2.09s/it] 81%|████████ | 6076/7500 [3:27:07<49:46, 2.10s/it] 81%|████████ | 6077/7500 [3:27:09<48:57, 2.06s/it] 81%|████████ | 6078/7500 [3:27:11<49:21, 2.08s/it] 81%|████████ | 6079/7500 [3:27:13<48:38, 2.05s/it] 81%|████████ | 6080/7500 [3:27:15<48:30, 2.05s/it] {'loss': 0.3485, 'grad_norm': 2.852473735809326, 'learning_rate': 5.263966802018275e-06, 'logqs/rejected': -2.4708166122436523, 'logqs/chosen': -0.8502756953239441, 'logqs_logps/rejected': -0.020804543048143387, 'logps_logqs/chosen': 0.2039601057767868, 'epoch': 1.62} 81%|████████ | 6080/7500 [3:27:15<48:30, 2.05s/it] 81%|████████ | 6081/7500 [3:27:17<48:28, 2.05s/it] 81%|████████ | 6082/7500 [3:27:19<48:36, 2.06s/it] 81%|████████ | 6083/7500 [3:27:22<49:22, 2.09s/it] 81%|████████ | 6084/7500 [3:27:24<49:18, 2.09s/it] 81%|████████ | 6085/7500 [3:27:26<49:15, 2.09s/it] 81%|████████ | 6086/7500 [3:27:28<49:28, 2.10s/it] 81%|████████ | 6087/7500 [3:27:30<49:03, 2.08s/it] 81%|████████ | 6088/7500 [3:27:32<48:39, 2.07s/it] 81%|████████ | 6089/7500 [3:27:34<48:05, 2.05s/it] 81%|████████ | 6090/7500 [3:27:36<48:20, 2.06s/it] {'loss': 0.3569, 'grad_norm': 2.5540831089019775, 'learning_rate': 5.192758916120236e-06, 'logqs/rejected': -2.43878436088562, 'logqs/chosen': -0.8490233421325684, 'logqs_logps/rejected': -0.026596438139677048, 'logps_logqs/chosen': 0.21021375060081482, 'epoch': 1.62} 81%|████████ | 6090/7500 [3:27:36<48:20, 2.06s/it] 81%|████████ | 6091/7500 [3:27:38<48:14, 2.05s/it] 81%|████████ | 6092/7500 [3:27:40<47:46, 2.04s/it] 81%|████████ | 6093/7500 [3:27:42<47:46, 2.04s/it] 81%|████████▏ | 6094/7500 [3:27:44<47:34, 2.03s/it] 81%|████████▏ | 6095/7500 [3:27:46<48:01, 2.05s/it] 81%|████████▏ | 6096/7500 [3:27:48<48:08, 2.06s/it] 81%|████████▏ | 6097/7500 [3:27:50<47:32, 2.03s/it] 81%|████████▏ | 6098/7500 [3:27:52<47:44, 2.04s/it] 81%|████████▏ | 6099/7500 [3:27:54<47:58, 2.05s/it] 81%|████████▏ | 6100/7500 [3:27:56<48:00, 2.06s/it] {'loss': 0.3558, 'grad_norm': 2.7938737869262695, 'learning_rate': 5.121980087628803e-06, 'logqs/rejected': -2.573974847793579, 'logqs/chosen': -0.8418834805488586, 'logqs_logps/rejected': -0.024269619956612587, 'logps_logqs/chosen': 0.20333370566368103, 'epoch': 1.63} 81%|████████▏ | 6100/7500 [3:27:56<48:00, 2.06s/it] 81%|████████▏ | 6101/7500 [3:27:59<47:51, 2.05s/it] 81%|████████▏ | 6102/7500 [3:28:01<48:38, 2.09s/it] 81%|████████▏ | 6103/7500 [3:28:03<48:05, 2.07s/it] 81%|████████▏ | 6104/7500 [3:28:05<48:05, 2.07s/it] 81%|████████▏ | 6105/7500 [3:28:07<48:28, 2.09s/it] 81%|████████▏ | 6106/7500 [3:28:09<48:48, 2.10s/it] 81%|████████▏ | 6107/7500 [3:28:11<48:16, 2.08s/it] 81%|████████▏ | 6108/7500 [3:28:13<48:17, 2.08s/it] 81%|████████▏ | 6109/7500 [3:28:15<47:39, 2.06s/it] 81%|████████▏ | 6110/7500 [3:28:17<47:28, 2.05s/it] {'loss': 0.3485, 'grad_norm': 2.6486475467681885, 'learning_rate': 5.051631849729785e-06, 'logqs/rejected': -2.3829691410064697, 'logqs/chosen': -0.8497799038887024, 'logqs_logps/rejected': -0.022240877151489258, 'logps_logqs/chosen': 0.19895091652870178, 'epoch': 1.63} 81%|████████▏ | 6110/7500 [3:28:17<47:28, 2.05s/it] 81%|████████▏ | 6111/7500 [3:28:19<47:58, 2.07s/it] 81%|████████▏ | 6112/7500 [3:28:21<46:54, 2.03s/it] 82%|████████▏ | 6113/7500 [3:28:23<46:51, 2.03s/it] 82%|████████▏ | 6114/7500 [3:28:25<47:16, 2.05s/it] 82%|████████▏ | 6115/7500 [3:28:28<48:14, 2.09s/it] 82%|████████▏ | 6116/7500 [3:28:30<48:13, 2.09s/it] 82%|████████▏ | 6117/7500 [3:28:32<47:57, 2.08s/it] 82%|████████▏ | 6118/7500 [3:28:34<48:43, 2.12s/it] 82%|████████▏ | 6119/7500 [3:28:36<47:39, 2.07s/it] 82%|████████▏ | 6120/7500 [3:28:38<48:08, 2.09s/it] {'loss': 0.3447, 'grad_norm': 2.900522232055664, 'learning_rate': 4.981715726281666e-06, 'logqs/rejected': -2.5950660705566406, 'logqs/chosen': -0.8632844686508179, 'logqs_logps/rejected': -0.02746790088713169, 'logps_logqs/chosen': 0.2014615833759308, 'epoch': 1.63} 82%|████████▏ | 6120/7500 [3:28:38<48:08, 2.09s/it] 82%|████████▏ | 6121/7500 [3:28:40<48:19, 2.10s/it] 82%|████████▏ | 6122/7500 [3:28:42<48:27, 2.11s/it] 82%|████████▏ | 6123/7500 [3:28:44<48:22, 2.11s/it] 82%|████████▏ | 6124/7500 [3:28:46<48:06, 2.10s/it] 82%|████████▏ | 6125/7500 [3:28:49<48:08, 2.10s/it] 82%|████████▏ | 6126/7500 [3:28:51<47:34, 2.08s/it] 82%|████████▏ | 6127/7500 [3:28:53<47:59, 2.10s/it] 82%|████████▏ | 6128/7500 [3:28:55<48:08, 2.11s/it] 82%|████████▏ | 6129/7500 [3:28:57<47:22, 2.07s/it] 82%|████████▏ | 6130/7500 [3:28:59<47:25, 2.08s/it] {'loss': 0.3498, 'grad_norm': 2.9039251804351807, 'learning_rate': 4.912233231782623e-06, 'logqs/rejected': -2.4179844856262207, 'logqs/chosen': -0.8527978658676147, 'logqs_logps/rejected': -0.026752904057502747, 'logps_logqs/chosen': 0.2014215737581253, 'epoch': 1.63} 82%|████████▏ | 6130/7500 [3:28:59<47:25, 2.08s/it] 82%|████████▏ | 6131/7500 [3:29:01<47:38, 2.09s/it] 82%|████████▏ | 6132/7500 [3:29:03<47:38, 2.09s/it] 82%|████████▏ | 6133/7500 [3:29:05<47:18, 2.08s/it] 82%|████████▏ | 6134/7500 [3:29:07<47:02, 2.07s/it] 82%|████████▏ | 6135/7500 [3:29:09<46:37, 2.05s/it] 82%|████████▏ | 6136/7500 [3:29:11<47:18, 2.08s/it] 82%|████████▏ | 6137/7500 [3:29:13<47:25, 2.09s/it] 82%|████████▏ | 6138/7500 [3:29:15<46:54, 2.07s/it] 82%|████████▏ | 6139/7500 [3:29:18<47:04, 2.08s/it] 82%|████████▏ | 6140/7500 [3:29:20<46:56, 2.07s/it] {'loss': 0.3485, 'grad_norm': 2.5338006019592285, 'learning_rate': 4.843185871337721e-06, 'logqs/rejected': -2.359182834625244, 'logqs/chosen': -0.8434022068977356, 'logqs_logps/rejected': -0.028207868337631226, 'logps_logqs/chosen': 0.20931577682495117, 'epoch': 1.64} 82%|████████▏ | 6140/7500 [3:29:20<46:56, 2.07s/it] 82%|████████▏ | 6141/7500 [3:29:22<46:41, 2.06s/it] 82%|████████▏ | 6142/7500 [3:29:24<46:17, 2.05s/it] 82%|████████▏ | 6143/7500 [3:29:26<46:04, 2.04s/it] 82%|████████▏ | 6144/7500 [3:29:28<45:57, 2.03s/it] 82%|████████▏ | 6145/7500 [3:29:30<46:19, 2.05s/it] 82%|████████▏ | 6146/7500 [3:29:32<47:03, 2.09s/it] 82%|████████▏ | 6147/7500 [3:29:34<47:47, 2.12s/it] 82%|████████▏ | 6148/7500 [3:29:36<47:41, 2.12s/it] 82%|████████▏ | 6149/7500 [3:29:38<47:03, 2.09s/it] 82%|████████▏ | 6150/7500 [3:29:40<47:01, 2.09s/it] {'loss': 0.368, 'grad_norm': 2.858891725540161, 'learning_rate': 4.7745751406263165e-06, 'logqs/rejected': -2.71364688873291, 'logqs/chosen': -0.8431018590927124, 'logqs_logps/rejected': -0.017999127507209778, 'logps_logqs/chosen': 0.19791002571582794, 'epoch': 1.64} 82%|████████▏ | 6150/7500 [3:29:40<47:01, 2.09s/it] 82%|████████▏ | 6151/7500 [3:29:43<47:12, 2.10s/it] 82%|████████▏ | 6152/7500 [3:29:45<47:17, 2.10s/it] 82%|████████▏ | 6153/7500 [3:29:47<47:09, 2.10s/it] 82%|████████▏ | 6154/7500 [3:29:49<47:23, 2.11s/it] 82%|████████▏ | 6155/7500 [3:29:51<46:47, 2.09s/it] 82%|████████▏ | 6156/7500 [3:29:53<46:56, 2.10s/it] 82%|████████▏ | 6157/7500 [3:29:55<47:02, 2.10s/it] 82%|████████▏ | 6158/7500 [3:29:57<46:44, 2.09s/it] 82%|████████▏ | 6159/7500 [3:29:59<46:47, 2.09s/it] 82%|████████▏ | 6160/7500 [3:30:01<46:41, 2.09s/it] {'loss': 0.3499, 'grad_norm': 2.7502059936523438, 'learning_rate': 4.706402525869633e-06, 'logqs/rejected': -2.496717929840088, 'logqs/chosen': -0.8492307662963867, 'logqs_logps/rejected': -0.022657472640275955, 'logps_logqs/chosen': 0.20806019008159637, 'epoch': 1.64} 82%|████████▏ | 6160/7500 [3:30:01<46:41, 2.09s/it] 82%|████████▏ | 6161/7500 [3:30:03<46:14, 2.07s/it] 82%|████████▏ | 6162/7500 [3:30:06<46:58, 2.11s/it] 82%|████████▏ | 6163/7500 [3:30:08<46:54, 2.11s/it] 82%|████████▏ | 6164/7500 [3:30:10<46:44, 2.10s/it] 82%|████████▏ | 6165/7500 [3:30:12<46:10, 2.08s/it] 82%|████████▏ | 6166/7500 [3:30:14<46:43, 2.10s/it] 82%|████████▏ | 6167/7500 [3:30:16<46:36, 2.10s/it] 82%|████████▏ | 6168/7500 [3:30:18<46:36, 2.10s/it] 82%|████████▏ | 6169/7500 [3:30:20<46:32, 2.10s/it] 82%|████████▏ | 6170/7500 [3:30:22<46:05, 2.08s/it] {'loss': 0.3663, 'grad_norm': 2.9954030513763428, 'learning_rate': 4.638669503798579e-06, 'logqs/rejected': -2.6761107444763184, 'logqs/chosen': -0.8438793420791626, 'logqs_logps/rejected': -0.01840701699256897, 'logps_logqs/chosen': 0.2043849676847458, 'epoch': 1.65} 82%|████████▏ | 6170/7500 [3:30:22<46:05, 2.08s/it] 82%|████████▏ | 6171/7500 [3:30:24<45:47, 2.07s/it] 82%|████████▏ | 6172/7500 [3:30:26<45:23, 2.05s/it] 82%|████████▏ | 6173/7500 [3:30:28<45:38, 2.06s/it] 82%|████████▏ | 6174/7500 [3:30:31<46:04, 2.08s/it] 82%|████████▏ | 6175/7500 [3:30:33<45:36, 2.07s/it] 82%|████████▏ | 6176/7500 [3:30:35<45:34, 2.07s/it] 82%|████████▏ | 6177/7500 [3:30:37<45:34, 2.07s/it] 82%|████████▏ | 6178/7500 [3:30:39<45:13, 2.05s/it] 82%|████████▏ | 6179/7500 [3:30:41<45:37, 2.07s/it] 82%|████████▏ | 6180/7500 [3:30:43<45:15, 2.06s/it] {'loss': 0.3455, 'grad_norm': 2.6600091457366943, 'learning_rate': 4.571377541621788e-06, 'logqs/rejected': -2.4545600414276123, 'logqs/chosen': -0.853366494178772, 'logqs_logps/rejected': -0.031175460666418076, 'logps_logqs/chosen': 0.20324774086475372, 'epoch': 1.65} 82%|████████▏ | 6180/7500 [3:30:43<45:15, 2.06s/it] 82%|████████▏ | 6181/7500 [3:30:45<44:59, 2.05s/it] 82%|████████▏ | 6182/7500 [3:30:47<45:32, 2.07s/it] 82%|████████▏ | 6183/7500 [3:30:49<45:32, 2.08s/it] 82%|████████▏ | 6184/7500 [3:30:51<45:54, 2.09s/it] 82%|████████▏ | 6185/7500 [3:30:53<45:35, 2.08s/it] 82%|████████▏ | 6186/7500 [3:30:55<45:48, 2.09s/it] 82%|████████▏ | 6187/7500 [3:30:58<46:34, 2.13s/it] 83%|████████▎ | 6188/7500 [3:31:00<46:29, 2.13s/it] 83%|████████▎ | 6189/7500 [3:31:02<45:44, 2.09s/it] 83%|████████▎ | 6190/7500 [3:31:04<45:31, 2.08s/it] {'loss': 0.3532, 'grad_norm': 2.92103910446167, 'learning_rate': 4.504528096993785e-06, 'logqs/rejected': -2.600402355194092, 'logqs/chosen': -0.8529601097106934, 'logqs_logps/rejected': -0.03109394945204258, 'logps_logqs/chosen': 0.20813444256782532, 'epoch': 1.65} 83%|████████▎ | 6190/7500 [3:31:04<45:31, 2.08s/it] 83%|████████▎ | 6191/7500 [3:31:06<44:54, 2.06s/it] 83%|████████▎ | 6192/7500 [3:31:08<45:06, 2.07s/it] 83%|████████▎ | 6193/7500 [3:31:10<45:08, 2.07s/it] 83%|████████▎ | 6194/7500 [3:31:12<44:44, 2.06s/it] 83%|████████▎ | 6195/7500 [3:31:14<45:11, 2.08s/it] 83%|████████▎ | 6196/7500 [3:31:16<44:59, 2.07s/it] 83%|████████▎ | 6197/7500 [3:31:18<45:15, 2.08s/it] 83%|████████▎ | 6198/7500 [3:31:21<45:47, 2.11s/it] 83%|████████▎ | 6199/7500 [3:31:23<46:12, 2.13s/it] 83%|████████▎ | 6200/7500 [3:31:25<46:23, 2.14s/it] {'loss': 0.3569, 'grad_norm': 3.1617419719696045, 'learning_rate': 4.438122617983443e-06, 'logqs/rejected': -2.3981518745422363, 'logqs/chosen': -0.8487162590026855, 'logqs_logps/rejected': -0.02499077096581459, 'logps_logqs/chosen': 0.20532390475273132, 'epoch': 1.65} 83%|████████▎ | 6200/7500 [3:31:25<46:23, 2.14s/it] 83%|████████▎ | 6201/7500 [3:31:27<46:05, 2.13s/it] 83%|████████▎ | 6202/7500 [3:31:29<45:57, 2.12s/it] 83%|████████▎ | 6203/7500 [3:31:31<45:21, 2.10s/it] 83%|████████▎ | 6204/7500 [3:31:33<45:27, 2.10s/it] 83%|████████▎ | 6205/7500 [3:31:35<45:01, 2.09s/it] 83%|████████▎ | 6206/7500 [3:31:37<44:41, 2.07s/it] 83%|████████▎ | 6207/7500 [3:31:39<44:05, 2.05s/it] 83%|████████▎ | 6208/7500 [3:31:41<43:35, 2.02s/it] 83%|████████▎ | 6209/7500 [3:31:43<43:43, 2.03s/it] 83%|████████▎ | 6210/7500 [3:31:45<43:20, 2.02s/it] {'loss': 0.3735, 'grad_norm': 2.5572667121887207, 'learning_rate': 4.372162543042624e-06, 'logqs/rejected': -2.725722551345825, 'logqs/chosen': -0.8481962084770203, 'logqs_logps/rejected': -0.020126691088080406, 'logps_logqs/chosen': 0.20648548007011414, 'epoch': 1.66} 83%|████████▎ | 6210/7500 [3:31:45<43:20, 2.02s/it] 83%|████████▎ | 6211/7500 [3:31:47<43:52, 2.04s/it] 83%|████████▎ | 6212/7500 [3:31:50<44:31, 2.07s/it] 83%|████████▎ | 6213/7500 [3:31:52<44:35, 2.08s/it] 83%|████████▎ | 6214/7500 [3:31:54<45:17, 2.11s/it] 83%|████████▎ | 6215/7500 [3:31:56<45:28, 2.12s/it] 83%|████████▎ | 6216/7500 [3:31:58<44:28, 2.08s/it] 83%|████████▎ | 6217/7500 [3:32:00<43:48, 2.05s/it] 83%|████████▎ | 6218/7500 [3:32:02<43:44, 2.05s/it] 83%|████████▎ | 6219/7500 [3:32:04<43:49, 2.05s/it] 83%|████████▎ | 6220/7500 [3:32:06<43:51, 2.06s/it] {'loss': 0.3527, 'grad_norm': 2.530162811279297, 'learning_rate': 4.306649300974985e-06, 'logqs/rejected': -2.7275538444519043, 'logqs/chosen': -0.8475677371025085, 'logqs_logps/rejected': -0.024966290220618248, 'logps_logqs/chosen': 0.2042393982410431, 'epoch': 1.66} 83%|████████▎ | 6220/7500 [3:32:06<43:51, 2.06s/it] 83%|████████▎ | 6221/7500 [3:32:08<44:25, 2.08s/it] 83%|████████▎ | 6222/7500 [3:32:10<44:25, 2.09s/it] 83%|████████▎ | 6223/7500 [3:32:12<43:58, 2.07s/it] 83%|████████▎ | 6224/7500 [3:32:14<43:48, 2.06s/it] 83%|████████▎ | 6225/7500 [3:32:16<43:33, 2.05s/it] 83%|████████▎ | 6226/7500 [3:32:18<43:20, 2.04s/it] 83%|████████▎ | 6227/7500 [3:32:21<43:46, 2.06s/it] 83%|████████▎ | 6228/7500 [3:32:23<44:32, 2.10s/it] 83%|████████▎ | 6229/7500 [3:32:25<45:05, 2.13s/it] 83%|████████▎ | 6230/7500 [3:32:27<44:28, 2.10s/it] {'loss': 0.3511, 'grad_norm': 2.8517634868621826, 'learning_rate': 4.241584310905067e-06, 'logqs/rejected': -2.370884656906128, 'logqs/chosen': -0.856482982635498, 'logqs_logps/rejected': -0.021006206050515175, 'logps_logqs/chosen': 0.2095051258802414, 'epoch': 1.66} 83%|████████▎ | 6230/7500 [3:32:27<44:28, 2.10s/it] 83%|████████▎ | 6231/7500 [3:32:29<44:56, 2.12s/it] 83%|████████▎ | 6232/7500 [3:32:31<44:30, 2.11s/it] 83%|████████▎ | 6233/7500 [3:32:33<44:10, 2.09s/it] 83%|████████▎ | 6234/7500 [3:32:35<44:08, 2.09s/it] 83%|████████▎ | 6235/7500 [3:32:37<43:40, 2.07s/it] 83%|████████▎ | 6236/7500 [3:32:39<43:34, 2.07s/it] 83%|████████▎ | 6237/7500 [3:32:42<43:55, 2.09s/it] 83%|████████▎ | 6238/7500 [3:32:44<43:54, 2.09s/it] 83%|████████▎ | 6239/7500 [3:32:46<43:53, 2.09s/it] 83%|████████▎ | 6240/7500 [3:32:48<43:47, 2.09s/it] {'loss': 0.3622, 'grad_norm': 2.816859722137451, 'learning_rate': 4.176968982247514e-06, 'logqs/rejected': -2.504260540008545, 'logqs/chosen': -0.8408690690994263, 'logqs_logps/rejected': -0.02277597226202488, 'logps_logqs/chosen': 0.20485320687294006, 'epoch': 1.66} 83%|████████▎ | 6240/7500 [3:32:48<43:47, 2.09s/it] 83%|████████▎ | 6241/7500 [3:32:50<43:16, 2.06s/it] 83%|████████▎ | 6242/7500 [3:32:52<42:55, 2.05s/it] 83%|████████▎ | 6243/7500 [3:32:54<42:56, 2.05s/it] 83%|████████▎ | 6244/7500 [3:32:56<42:39, 2.04s/it] 83%|████████▎ | 6245/7500 [3:32:58<42:50, 2.05s/it] 83%|████████▎ | 6246/7500 [3:33:00<43:08, 2.06s/it] 83%|████████▎ | 6247/7500 [3:33:02<43:21, 2.08s/it] 83%|████████▎ | 6248/7500 [3:33:04<43:27, 2.08s/it] 83%|████████▎ | 6249/7500 [3:33:06<43:49, 2.10s/it] 83%|████████▎ | 6250/7500 [3:33:09<43:50, 2.10s/it] {'loss': 0.3562, 'grad_norm': 3.0845694541931152, 'learning_rate': 4.112804714676594e-06, 'logqs/rejected': -2.4802539348602295, 'logqs/chosen': -0.8640007972717285, 'logqs_logps/rejected': -0.023807678371667862, 'logps_logqs/chosen': 0.20326869189739227, 'epoch': 1.67} 83%|████████▎ | 6250/7500 [3:33:09<43:50, 2.10s/it] 83%|████████▎ | 6251/7500 [3:33:11<43:35, 2.09s/it] 83%|████████▎ | 6252/7500 [3:33:13<43:29, 2.09s/it] 83%|████████▎ | 6253/7500 [3:33:15<43:32, 2.10s/it] 83%|████████▎ | 6254/7500 [3:33:17<43:11, 2.08s/it] 83%|████████▎ | 6255/7500 [3:33:19<42:48, 2.06s/it] 83%|████████▎ | 6256/7500 [3:33:21<42:44, 2.06s/it] 83%|████████▎ | 6257/7500 [3:33:23<42:57, 2.07s/it] 83%|████████▎ | 6258/7500 [3:33:25<42:57, 2.08s/it] 83%|████████▎ | 6259/7500 [3:33:27<42:37, 2.06s/it] 83%|████████▎ | 6260/7500 [3:33:29<42:22, 2.05s/it] {'loss': 0.353, 'grad_norm': 2.92760968208313, 'learning_rate': 4.049092898095816e-06, 'logqs/rejected': -2.661320447921753, 'logqs/chosen': -0.8512943983078003, 'logqs_logps/rejected': -0.02556971088051796, 'logps_logqs/chosen': 0.1983298510313034, 'epoch': 1.67} 83%|████████▎ | 6260/7500 [3:33:29<42:22, 2.05s/it] 83%|████████▎ | 6261/7500 [3:33:31<42:23, 2.05s/it] 83%|████████▎ | 6262/7500 [3:33:33<42:11, 2.05s/it] 84%|████████▎ | 6263/7500 [3:33:35<42:01, 2.04s/it] 84%|████████▎ | 6264/7500 [3:33:37<42:00, 2.04s/it] 84%|████████▎ | 6265/7500 [3:33:39<42:07, 2.05s/it] 84%|████████▎ | 6266/7500 [3:33:42<42:52, 2.08s/it] 84%|████████▎ | 6267/7500 [3:33:44<43:15, 2.11s/it] 84%|████████▎ | 6268/7500 [3:33:46<43:26, 2.12s/it] 84%|████████▎ | 6269/7500 [3:33:48<42:41, 2.08s/it] 84%|████████▎ | 6270/7500 [3:33:50<42:20, 2.07s/it] {'loss': 0.3427, 'grad_norm': 2.511902332305908, 'learning_rate': 3.985834912607894e-06, 'logqs/rejected': -2.517529010772705, 'logqs/chosen': -0.8170323371887207, 'logqs_logps/rejected': -0.02715241350233555, 'logps_logqs/chosen': 0.20362310111522675, 'epoch': 1.67} 84%|████████▎ | 6270/7500 [3:33:50<42:20, 2.07s/it] 84%|████████▎ | 6271/7500 [3:33:52<42:22, 2.07s/it] 84%|████████▎ | 6272/7500 [3:33:54<42:26, 2.07s/it] 84%|████████▎ | 6273/7500 [3:33:56<42:50, 2.10s/it] 84%|████████▎ | 6274/7500 [3:33:58<43:09, 2.11s/it] 84%|████████▎ | 6275/7500 [3:34:00<42:57, 2.10s/it] 84%|████████▎ | 6276/7500 [3:34:02<42:38, 2.09s/it] 84%|████████▎ | 6277/7500 [3:34:05<42:56, 2.11s/it] 84%|████████▎ | 6278/7500 [3:34:07<42:19, 2.08s/it] 84%|████████▎ | 6279/7500 [3:34:09<42:16, 2.08s/it] 84%|████████▎ | 6280/7500 [3:34:11<43:53, 2.16s/it] {'loss': 0.3365, 'grad_norm': 2.6995489597320557, 'learning_rate': 3.923032128484786e-06, 'logqs/rejected': -2.354093551635742, 'logqs/chosen': -0.8254247903823853, 'logqs_logps/rejected': -0.027248214930295944, 'logps_logqs/chosen': 0.20849713683128357, 'epoch': 1.67} 84%|████████▎ | 6280/7500 [3:34:11<43:53, 2.16s/it] 84%|████████▎ | 6281/7500 [3:34:13<43:23, 2.14s/it] 84%|████████▍ | 6282/7500 [3:34:15<43:29, 2.14s/it] 84%|████████▍ | 6283/7500 [3:34:17<43:30, 2.14s/it] 84%|████████▍ | 6284/7500 [3:34:20<43:21, 2.14s/it] 84%|████████▍ | 6285/7500 [3:34:22<43:02, 2.13s/it] 84%|████████▍ | 6286/7500 [3:34:24<42:51, 2.12s/it] 84%|████████▍ | 6287/7500 [3:34:26<42:16, 2.09s/it] 84%|████████▍ | 6288/7500 [3:34:28<42:19, 2.10s/it] 84%|████████▍ | 6289/7500 [3:34:30<42:18, 2.10s/it] 84%|████████▍ | 6290/7500 [3:34:32<42:31, 2.11s/it] {'loss': 0.3407, 'grad_norm': 2.8925838470458984, 'learning_rate': 3.86068590613804e-06, 'logqs/rejected': -2.3178577423095703, 'logqs/chosen': -0.8658015131950378, 'logqs_logps/rejected': -0.032448213547468185, 'logps_logqs/chosen': 0.20782169699668884, 'epoch': 1.68} 84%|████████▍ | 6290/7500 [3:34:32<42:31, 2.11s/it] 84%|████████▍ | 6291/7500 [3:34:34<42:36, 2.11s/it] 84%|████████▍ | 6292/7500 [3:34:36<42:43, 2.12s/it] 84%|████████▍ | 6293/7500 [3:34:39<42:28, 2.11s/it] 84%|████████▍ | 6294/7500 [3:34:41<42:24, 2.11s/it] 84%|████████▍ | 6295/7500 [3:34:43<41:59, 2.09s/it] 84%|████████▍ | 6296/7500 [3:34:45<42:21, 2.11s/it] 84%|████████▍ | 6297/7500 [3:34:47<41:34, 2.07s/it] 84%|████████▍ | 6298/7500 [3:34:49<41:14, 2.06s/it] 84%|████████▍ | 6299/7500 [3:34:51<40:49, 2.04s/it] 84%|████████▍ | 6300/7500 [3:34:53<40:35, 2.03s/it] {'loss': 0.3615, 'grad_norm': 3.7125661373138428, 'learning_rate': 3.798797596089351e-06, 'logqs/rejected': -2.5606849193573, 'logqs/chosen': -0.8513924479484558, 'logqs_logps/rejected': -0.024420997127890587, 'logps_logqs/chosen': 0.20735469460487366, 'epoch': 1.68} 84%|████████▍ | 6300/7500 [3:34:53<40:35, 2.03s/it] 84%|████████▍ | 6301/7500 [3:34:55<40:23, 2.02s/it] 84%|████████▍ | 6302/7500 [3:34:57<40:20, 2.02s/it] 84%|████████▍ | 6303/7500 [3:34:59<40:24, 2.03s/it] 84%|████████▍ | 6304/7500 [3:35:01<40:58, 2.06s/it] 84%|████████▍ | 6305/7500 [3:35:03<41:44, 2.10s/it] 84%|████████▍ | 6306/7500 [3:35:05<41:47, 2.10s/it] 84%|████████▍ | 6307/7500 [3:35:07<41:31, 2.09s/it] 84%|████████▍ | 6308/7500 [3:35:10<41:46, 2.10s/it] 84%|████████▍ | 6309/7500 [3:35:12<41:44, 2.10s/it] 84%|████████▍ | 6310/7500 [3:35:14<41:21, 2.09s/it] {'loss': 0.3553, 'grad_norm': 3.938570022583008, 'learning_rate': 3.7373685389412548e-06, 'logqs/rejected': -2.419829845428467, 'logqs/chosen': -0.8379001617431641, 'logqs_logps/rejected': -0.03267356753349304, 'logps_logqs/chosen': 0.20669157803058624, 'epoch': 1.68} 84%|████████▍ | 6310/7500 [3:35:14<41:21, 2.09s/it] 84%|████████▍ | 6311/7500 [3:35:16<41:25, 2.09s/it] 84%|████████▍ | 6312/7500 [3:35:18<41:15, 2.08s/it] 84%|████████▍ | 6313/7500 [3:35:20<41:14, 2.08s/it] 84%|████████▍ | 6314/7500 [3:35:22<41:09, 2.08s/it] 84%|████████▍ | 6315/7500 [3:35:24<40:44, 2.06s/it] 84%|████████▍ | 6316/7500 [3:35:26<40:44, 2.06s/it] 84%|████████▍ | 6317/7500 [3:35:28<40:31, 2.06s/it] 84%|████████▍ | 6318/7500 [3:35:30<40:59, 2.08s/it] 84%|████████▍ | 6319/7500 [3:35:32<41:16, 2.10s/it] 84%|████████▍ | 6320/7500 [3:35:35<41:21, 2.10s/it] {'loss': 0.3268, 'grad_norm': 2.5401415824890137, 'learning_rate': 3.676400065348126e-06, 'logqs/rejected': -2.1444849967956543, 'logqs/chosen': -0.850202202796936, 'logqs_logps/rejected': -0.034930869936943054, 'logps_logqs/chosen': 0.20253749191761017, 'epoch': 1.69} 84%|████████▍ | 6320/7500 [3:35:35<41:21, 2.10s/it] 84%|████████▍ | 6321/7500 [3:35:37<41:24, 2.11s/it] 84%|████████▍ | 6322/7500 [3:35:39<40:58, 2.09s/it] 84%|████████▍ | 6323/7500 [3:35:41<41:06, 2.10s/it] 84%|████████▍ | 6324/7500 [3:35:43<41:31, 2.12s/it] 84%|████████▍ | 6325/7500 [3:35:45<42:13, 2.16s/it] 84%|████████▍ | 6326/7500 [3:35:47<41:23, 2.12s/it] 84%|████████▍ | 6327/7500 [3:35:49<40:50, 2.09s/it] 84%|████████▍ | 6328/7500 [3:35:51<40:53, 2.09s/it] 84%|████████▍ | 6329/7500 [3:35:53<40:45, 2.09s/it] 84%|████████▍ | 6330/7500 [3:35:55<40:12, 2.06s/it] {'loss': 0.3485, 'grad_norm': 2.6395134925842285, 'learning_rate': 3.6158934959873353e-06, 'logqs/rejected': -2.345452308654785, 'logqs/chosen': -0.8576388359069824, 'logqs_logps/rejected': -0.031658392399549484, 'logps_logqs/chosen': 0.20288999378681183, 'epoch': 1.69} 84%|████████▍ | 6330/7500 [3:35:55<40:12, 2.06s/it] 84%|████████▍ | 6331/7500 [3:35:57<39:39, 2.04s/it] 84%|████████▍ | 6332/7500 [3:36:00<40:10, 2.06s/it] 84%|████████▍ | 6333/7500 [3:36:02<40:41, 2.09s/it] 84%|████████▍ | 6334/7500 [3:36:04<40:42, 2.10s/it] 84%|████████▍ | 6335/7500 [3:36:06<40:41, 2.10s/it] 84%|████████▍ | 6336/7500 [3:36:08<40:05, 2.07s/it] 84%|████████▍ | 6337/7500 [3:36:10<39:35, 2.04s/it] 85%|████████▍ | 6338/7500 [3:36:12<39:12, 2.02s/it] 85%|████████▍ | 6339/7500 [3:36:14<39:27, 2.04s/it] 85%|████████▍ | 6340/7500 [3:36:16<39:19, 2.03s/it] {'loss': 0.3538, 'grad_norm': 2.526381731033325, 'learning_rate': 3.555850141530659e-06, 'logqs/rejected': -2.5720131397247314, 'logqs/chosen': -0.8409544825553894, 'logqs_logps/rejected': -0.027843335643410683, 'logps_logqs/chosen': 0.2074325531721115, 'epoch': 1.69} 85%|████████▍ | 6340/7500 [3:36:16<39:19, 2.03s/it] 85%|████████▍ | 6341/7500 [3:36:18<39:23, 2.04s/it] 85%|████████▍ | 6342/7500 [3:36:20<39:26, 2.04s/it] 85%|████████▍ | 6343/7500 [3:36:22<39:56, 2.07s/it] 85%|████████▍ | 6344/7500 [3:36:24<40:24, 2.10s/it] 85%|████████▍ | 6345/7500 [3:36:26<40:17, 2.09s/it] 85%|████████▍ | 6346/7500 [3:36:29<40:21, 2.10s/it] 85%|████████▍ | 6347/7500 [3:36:31<40:13, 2.09s/it] 85%|████████▍ | 6348/7500 [3:36:33<39:51, 2.08s/it] 85%|████████▍ | 6349/7500 [3:36:35<39:57, 2.08s/it] 85%|████████▍ | 6350/7500 [3:36:37<40:26, 2.11s/it] {'loss': 0.3723, 'grad_norm': 3.0649876594543457, 'learning_rate': 3.4962713026158694e-06, 'logqs/rejected': -2.5820634365081787, 'logqs/chosen': -0.8348864316940308, 'logqs_logps/rejected': -0.018544495105743408, 'logps_logqs/chosen': 0.2051878273487091, 'epoch': 1.69} 85%|████████▍ | 6350/7500 [3:36:37<40:26, 2.11s/it] 85%|████████▍ | 6351/7500 [3:36:39<39:45, 2.08s/it] 85%|████████▍ | 6352/7500 [3:36:41<39:25, 2.06s/it] 85%|████████▍ | 6353/7500 [3:36:43<39:22, 2.06s/it] 85%|████████▍ | 6354/7500 [3:36:45<39:08, 2.05s/it] 85%|████████▍ | 6355/7500 [3:36:47<39:12, 2.05s/it] 85%|████████▍ | 6356/7500 [3:36:49<39:17, 2.06s/it] 85%|████████▍ | 6357/7500 [3:36:51<39:03, 2.05s/it] 85%|████████▍ | 6358/7500 [3:36:53<39:34, 2.08s/it] 85%|████████▍ | 6359/7500 [3:36:55<39:28, 2.08s/it] 85%|████████▍ | 6360/7500 [3:36:58<39:36, 2.08s/it] {'loss': 0.3723, 'grad_norm': 2.877859592437744, 'learning_rate': 3.4371582698185633e-06, 'logqs/rejected': -2.758948802947998, 'logqs/chosen': -0.8444474339485168, 'logqs_logps/rejected': -0.016387738287448883, 'logps_logqs/chosen': 0.20207376778125763, 'epoch': 1.7} 85%|████████▍ | 6360/7500 [3:36:58<39:36, 2.08s/it] 85%|████████▍ | 6361/7500 [3:37:00<39:31, 2.08s/it] 85%|████████▍ | 6362/7500 [3:37:02<39:27, 2.08s/it] 85%|████████▍ | 6363/7500 [3:37:04<39:10, 2.07s/it] 85%|████████▍ | 6364/7500 [3:37:06<40:04, 2.12s/it] 85%|████████▍ | 6365/7500 [3:37:08<39:44, 2.10s/it] 85%|████████▍ | 6366/7500 [3:37:10<39:49, 2.11s/it] 85%|████████▍ | 6367/7500 [3:37:12<40:00, 2.12s/it] 85%|████████▍ | 6368/7500 [3:37:14<40:29, 2.15s/it] 85%|████████▍ | 6369/7500 [3:37:17<40:06, 2.13s/it] 85%|████████▍ | 6370/7500 [3:37:19<39:45, 2.11s/it] {'loss': 0.3383, 'grad_norm': 3.117283582687378, 'learning_rate': 3.378512323624228e-06, 'logqs/rejected': -2.237076997756958, 'logqs/chosen': -0.8655673861503601, 'logqs_logps/rejected': -0.025605883449316025, 'logps_logqs/chosen': 0.19969689846038818, 'epoch': 1.7} 85%|████████▍ | 6370/7500 [3:37:19<39:45, 2.11s/it] 85%|████████▍ | 6371/7500 [3:37:21<39:38, 2.11s/it] 85%|████████▍ | 6372/7500 [3:37:23<39:25, 2.10s/it] 85%|████████▍ | 6373/7500 [3:37:25<38:48, 2.07s/it] 85%|████████▍ | 6374/7500 [3:37:27<39:11, 2.09s/it] 85%|████████▌ | 6375/7500 [3:37:29<39:14, 2.09s/it] 85%|████████▌ | 6376/7500 [3:37:31<39:26, 2.11s/it] 85%|████████▌ | 6377/7500 [3:37:33<38:50, 2.07s/it] 85%|████████▌ | 6378/7500 [3:37:35<39:10, 2.10s/it] 85%|████████▌ | 6379/7500 [3:37:37<39:08, 2.09s/it] 85%|████████▌ | 6380/7500 [3:37:40<39:11, 2.10s/it] {'loss': 0.3508, 'grad_norm': 2.894808769226074, 'learning_rate': 3.3203347344004736e-06, 'logqs/rejected': -2.471254587173462, 'logqs/chosen': -0.8418119549751282, 'logqs_logps/rejected': -0.024734806269407272, 'logps_logqs/chosen': 0.20403413474559784, 'epoch': 1.7} 85%|████████▌ | 6380/7500 [3:37:40<39:11, 2.10s/it] 85%|████████▌ | 6381/7500 [3:37:42<38:31, 2.07s/it] 85%|████████▌ | 6382/7500 [3:37:44<38:04, 2.04s/it] 85%|████████▌ | 6383/7500 [3:37:46<38:18, 2.06s/it] 85%|████████▌ | 6384/7500 [3:37:48<38:41, 2.08s/it] 85%|████████▌ | 6385/7500 [3:37:50<38:42, 2.08s/it] 85%|████████▌ | 6386/7500 [3:37:52<38:59, 2.10s/it] 85%|████████▌ | 6387/7500 [3:37:54<39:07, 2.11s/it] 85%|████████▌ | 6388/7500 [3:37:56<39:10, 2.11s/it] 85%|████████▌ | 6389/7500 [3:37:58<38:50, 2.10s/it] 85%|████████▌ | 6390/7500 [3:38:00<38:27, 2.08s/it] {'loss': 0.3532, 'grad_norm': 2.902867317199707, 'learning_rate': 3.262626762369525e-06, 'logqs/rejected': -2.516970634460449, 'logqs/chosen': -0.8469941020011902, 'logqs_logps/rejected': -0.02526179328560829, 'logps_logqs/chosen': 0.2044038027524948, 'epoch': 1.7} 85%|████████▌ | 6390/7500 [3:38:00<38:27, 2.08s/it] 85%|████████▌ | 6391/7500 [3:38:02<37:46, 2.04s/it] 85%|████████▌ | 6392/7500 [3:38:04<37:38, 2.04s/it] 85%|████████▌ | 6393/7500 [3:38:06<37:44, 2.05s/it] 85%|████████▌ | 6394/7500 [3:38:08<37:51, 2.05s/it] 85%|████████▌ | 6395/7500 [3:38:10<37:48, 2.05s/it] 85%|████████▌ | 6396/7500 [3:38:12<37:24, 2.03s/it] 85%|████████▌ | 6397/7500 [3:38:15<37:38, 2.05s/it] 85%|████████▌ | 6398/7500 [3:38:17<37:43, 2.05s/it] 85%|████████▌ | 6399/7500 [3:38:19<37:34, 2.05s/it] 85%|████████▌ | 6400/7500 [3:38:21<37:55, 2.07s/it] {'loss': 0.3421, 'grad_norm': 2.956287384033203, 'learning_rate': 3.205389657580943e-06, 'logqs/rejected': -2.579280138015747, 'logqs/chosen': -0.8295291662216187, 'logqs_logps/rejected': -0.028887033462524414, 'logps_logqs/chosen': 0.2012813836336136, 'epoch': 1.71} 85%|████████▌ | 6400/7500 [3:38:21<37:55, 2.07s/it] 85%|████████▌ | 6401/7500 [3:38:23<37:46, 2.06s/it] 85%|████████▌ | 6402/7500 [3:38:25<37:44, 2.06s/it] 85%|████████▌ | 6403/7500 [3:38:27<37:27, 2.05s/it] 85%|████████▌ | 6404/7500 [3:38:29<37:54, 2.08s/it] 85%|████████▌ | 6405/7500 [3:38:31<37:53, 2.08s/it] 85%|████████▌ | 6406/7500 [3:38:33<37:41, 2.07s/it] 85%|████████▌ | 6407/7500 [3:38:35<37:26, 2.06s/it] 85%|████████▌ | 6408/7500 [3:38:37<37:29, 2.06s/it] 85%|████████▌ | 6409/7500 [3:38:39<37:29, 2.06s/it] 85%|████████▌ | 6410/7500 [3:38:41<37:36, 2.07s/it] {'loss': 0.3534, 'grad_norm': 3.1207594871520996, 'learning_rate': 3.1486246598845082e-06, 'logqs/rejected': -2.6054952144622803, 'logqs/chosen': -0.8489745855331421, 'logqs_logps/rejected': -0.024988744407892227, 'logps_logqs/chosen': 0.19959837198257446, 'epoch': 1.71} 85%|████████▌ | 6410/7500 [3:38:41<37:36, 2.07s/it] 85%|████████▌ | 6411/7500 [3:38:43<37:26, 2.06s/it] 85%|████████▌ | 6412/7500 [3:38:46<37:47, 2.08s/it] 86%|████████▌ | 6413/7500 [3:38:48<37:13, 2.05s/it] 86%|████████▌ | 6414/7500 [3:38:50<37:11, 2.05s/it] 86%|████████▌ | 6415/7500 [3:38:52<38:15, 2.12s/it] 86%|████████▌ | 6416/7500 [3:38:54<37:49, 2.09s/it] 86%|████████▌ | 6417/7500 [3:38:56<37:38, 2.09s/it] 86%|████████▌ | 6418/7500 [3:38:58<37:46, 2.09s/it] 86%|████████▌ | 6419/7500 [3:39:00<37:43, 2.09s/it] 86%|████████▌ | 6420/7500 [3:39:02<37:33, 2.09s/it] {'loss': 0.3533, 'grad_norm': 3.0548126697540283, 'learning_rate': 3.092332998903416e-06, 'logqs/rejected': -2.484527111053467, 'logqs/chosen': -0.8657293319702148, 'logqs_logps/rejected': -0.02594761922955513, 'logps_logqs/chosen': 0.21430988609790802, 'epoch': 1.71} 86%|████████▌ | 6420/7500 [3:39:02<37:33, 2.09s/it] 86%|████████▌ | 6421/7500 [3:39:04<36:49, 2.05s/it] 86%|████████▌ | 6422/7500 [3:39:06<36:46, 2.05s/it] 86%|████████▌ | 6423/7500 [3:39:08<36:51, 2.05s/it] 86%|████████▌ | 6424/7500 [3:39:10<36:55, 2.06s/it] 86%|████████▌ | 6425/7500 [3:39:12<36:40, 2.05s/it] 86%|████████▌ | 6426/7500 [3:39:15<36:59, 2.07s/it] 86%|████████▌ | 6427/7500 [3:39:17<37:21, 2.09s/it] 86%|████████▌ | 6428/7500 [3:39:19<36:44, 2.06s/it] 86%|████████▌ | 6429/7500 [3:39:21<36:52, 2.07s/it] 86%|████████▌ | 6430/7500 [3:39:23<36:34, 2.05s/it] {'loss': 0.3513, 'grad_norm': 2.471992015838623, 'learning_rate': 3.036515894007566e-06, 'logqs/rejected': -2.663177967071533, 'logqs/chosen': -0.8389245867729187, 'logqs_logps/rejected': -0.01803390495479107, 'logps_logqs/chosen': 0.2010050117969513, 'epoch': 1.71} 86%|████████▌ | 6430/7500 [3:39:23<36:34, 2.05s/it] 86%|████████▌ | 6431/7500 [3:39:25<36:49, 2.07s/it] 86%|████████▌ | 6432/7500 [3:39:27<36:55, 2.07s/it] 86%|████████▌ | 6433/7500 [3:39:29<36:48, 2.07s/it] 86%|████████▌ | 6434/7500 [3:39:31<36:48, 2.07s/it] 86%|████████▌ | 6435/7500 [3:39:33<36:52, 2.08s/it] 86%|████████▌ | 6436/7500 [3:39:35<36:41, 2.07s/it] 86%|████████▌ | 6437/7500 [3:39:37<36:41, 2.07s/it] 86%|████████▌ | 6438/7500 [3:39:39<36:20, 2.05s/it] 86%|████████▌ | 6439/7500 [3:39:41<36:01, 2.04s/it] 86%|████████▌ | 6440/7500 [3:39:43<35:50, 2.03s/it] {'loss': 0.3579, 'grad_norm': 3.0036160945892334, 'learning_rate': 2.9811745542872388e-06, 'logqs/rejected': -2.5176174640655518, 'logqs/chosen': -0.8326548337936401, 'logqs_logps/rejected': -0.018581558018922806, 'logps_logqs/chosen': 0.19954535365104675, 'epoch': 1.72} 86%|████████▌ | 6440/7500 [3:39:43<35:50, 2.03s/it] 86%|████████▌ | 6441/7500 [3:39:46<36:38, 2.08s/it] 86%|████████▌ | 6442/7500 [3:39:48<36:36, 2.08s/it] 86%|████████▌ | 6443/7500 [3:39:50<36:31, 2.07s/it] 86%|████████▌ | 6444/7500 [3:39:52<36:57, 2.10s/it] 86%|████████▌ | 6445/7500 [3:39:54<36:53, 2.10s/it] 86%|████████▌ | 6446/7500 [3:39:56<36:37, 2.08s/it] 86%|████████▌ | 6447/7500 [3:39:58<36:39, 2.09s/it] 86%|████████▌ | 6448/7500 [3:40:00<36:28, 2.08s/it] 86%|████████▌ | 6449/7500 [3:40:02<36:10, 2.07s/it] 86%|████████▌ | 6450/7500 [3:40:04<36:16, 2.07s/it] {'loss': 0.3326, 'grad_norm': 2.558889150619507, 'learning_rate': 2.9263101785268254e-06, 'logqs/rejected': -2.505153179168701, 'logqs/chosen': -0.8381830453872681, 'logqs_logps/rejected': -0.03336915746331215, 'logps_logqs/chosen': 0.20322470366954803, 'epoch': 1.72} 86%|████████▌ | 6450/7500 [3:40:04<36:16, 2.07s/it] 86%|████████▌ | 6451/7500 [3:40:06<36:00, 2.06s/it] 86%|████████▌ | 6452/7500 [3:40:08<35:49, 2.05s/it] 86%|████████▌ | 6453/7500 [3:40:10<35:24, 2.03s/it] 86%|████████▌ | 6454/7500 [3:40:12<35:05, 2.01s/it] 86%|████████▌ | 6455/7500 [3:40:14<35:10, 2.02s/it] 86%|████████▌ | 6456/7500 [3:40:16<35:13, 2.02s/it] 86%|████████▌ | 6457/7500 [3:40:18<35:37, 2.05s/it] 86%|████████▌ | 6458/7500 [3:40:21<35:41, 2.06s/it] 86%|████████▌ | 6459/7500 [3:40:23<35:38, 2.05s/it] 86%|████████▌ | 6460/7500 [3:40:25<35:42, 2.06s/it] {'loss': 0.339, 'grad_norm': 2.537736177444458, 'learning_rate': 2.871923955178918e-06, 'logqs/rejected': -2.392148494720459, 'logqs/chosen': -0.856471061706543, 'logqs_logps/rejected': -0.023124810308218002, 'logps_logqs/chosen': 0.19886082410812378, 'epoch': 1.72} 86%|████████▌ | 6460/7500 [3:40:25<35:42, 2.06s/it] 86%|████████▌ | 6461/7500 [3:40:27<35:39, 2.06s/it] 86%|████████▌ | 6462/7500 [3:40:29<36:05, 2.09s/it] 86%|████████▌ | 6463/7500 [3:40:31<35:45, 2.07s/it] 86%|████████▌ | 6464/7500 [3:40:33<36:01, 2.09s/it] 86%|████████▌ | 6465/7500 [3:40:35<35:48, 2.08s/it] 86%|████████▌ | 6466/7500 [3:40:37<35:32, 2.06s/it] 86%|████████▌ | 6467/7500 [3:40:39<35:36, 2.07s/it] 86%|████████▌ | 6468/7500 [3:40:41<35:35, 2.07s/it] 86%|████████▋ | 6469/7500 [3:40:43<35:43, 2.08s/it] 86%|████████▋ | 6470/7500 [3:40:45<35:44, 2.08s/it] {'loss': 0.3514, 'grad_norm': 3.3233096599578857, 'learning_rate': 2.818017062338521e-06, 'logqs/rejected': -2.4506945610046387, 'logqs/chosen': -0.8186792135238647, 'logqs_logps/rejected': -0.02464723214507103, 'logps_logqs/chosen': 0.20760312676429749, 'epoch': 1.73} 86%|████████▋ | 6470/7500 [3:40:45<35:44, 2.08s/it] 86%|████████▋ | 6471/7500 [3:40:48<35:59, 2.10s/it] 86%|████████▋ | 6472/7500 [3:40:50<35:57, 2.10s/it] 86%|████████▋ | 6473/7500 [3:40:52<35:55, 2.10s/it] 86%|████████▋ | 6474/7500 [3:40:54<35:31, 2.08s/it] 86%|████████▋ | 6475/7500 [3:40:56<35:17, 2.07s/it] 86%|████████▋ | 6476/7500 [3:40:58<35:19, 2.07s/it] 86%|████████▋ | 6477/7500 [3:41:00<35:02, 2.06s/it] 86%|████████▋ | 6478/7500 [3:41:02<34:49, 2.04s/it] 86%|████████▋ | 6479/7500 [3:41:04<35:01, 2.06s/it] 86%|████████▋ | 6480/7500 [3:41:06<35:03, 2.06s/it] {'loss': 0.3466, 'grad_norm': 2.483530044555664, 'learning_rate': 2.764590667717562e-06, 'logqs/rejected': -2.4012222290039062, 'logqs/chosen': -0.8393797874450684, 'logqs_logps/rejected': -0.02844349667429924, 'logps_logqs/chosen': 0.20589280128479004, 'epoch': 1.73} 86%|████████▋ | 6480/7500 [3:41:06<35:03, 2.06s/it] 86%|████████▋ | 6481/7500 [3:41:08<34:50, 2.05s/it] 86%|████████▋ | 6482/7500 [3:41:10<35:14, 2.08s/it] 86%|████████▋ | 6483/7500 [3:41:12<35:13, 2.08s/it] 86%|████████▋ | 6484/7500 [3:41:14<35:01, 2.07s/it] 86%|████████▋ | 6485/7500 [3:41:16<34:34, 2.04s/it] 86%|████████▋ | 6486/7500 [3:41:18<34:35, 2.05s/it] 86%|████████▋ | 6487/7500 [3:41:20<34:28, 2.04s/it] 87%|████████▋ | 6488/7500 [3:41:23<34:53, 2.07s/it] 87%|████████▋ | 6489/7500 [3:41:25<34:32, 2.05s/it] 87%|████████▋ | 6490/7500 [3:41:27<34:32, 2.05s/it] {'loss': 0.3609, 'grad_norm': 2.7622294425964355, 'learning_rate': 2.7116459286195883e-06, 'logqs/rejected': -2.598630905151367, 'logqs/chosen': -0.8577197194099426, 'logqs_logps/rejected': -0.02108720690011978, 'logps_logqs/chosen': 0.2048567533493042, 'epoch': 1.73} 87%|████████▋ | 6490/7500 [3:41:27<34:32, 2.05s/it] 87%|████████▋ | 6491/7500 [3:41:29<34:55, 2.08s/it] 87%|████████▋ | 6492/7500 [3:41:31<34:54, 2.08s/it] 87%|████████▋ | 6493/7500 [3:41:33<35:02, 2.09s/it] 87%|████████▋ | 6494/7500 [3:41:35<35:04, 2.09s/it] 87%|████████▋ | 6495/7500 [3:41:37<34:25, 2.05s/it] 87%|████████▋ | 6496/7500 [3:41:39<34:35, 2.07s/it] 87%|████████▋ | 6497/7500 [3:41:41<34:29, 2.06s/it] 87%|████████▋ | 6498/7500 [3:41:43<34:39, 2.08s/it] 87%|████████▋ | 6499/7500 [3:41:45<34:07, 2.05s/it] 87%|████████▋ | 6500/7500 [3:41:47<34:43, 2.08s/it] {'loss': 0.3353, 'grad_norm': 2.5105628967285156, 'learning_rate': 2.659183991914696e-06, 'logqs/rejected': -2.3988099098205566, 'logqs/chosen': -0.8190563917160034, 'logqs_logps/rejected': -0.026603329926729202, 'logps_logqs/chosen': 0.2024223357439041, 'epoch': 1.73} 87%|████████▋ | 6500/7500 [3:41:48<34:43, 2.08s/it] 87%|████████▋ | 6501/7500 [3:41:49<34:03, 2.05s/it] 87%|████████▋ | 6502/7500 [3:41:51<33:43, 2.03s/it] 87%|████████▋ | 6503/7500 [3:41:53<33:48, 2.03s/it] 87%|████████▋ | 6504/7500 [3:41:56<34:21, 2.07s/it] 87%|████████▋ | 6505/7500 [3:41:58<33:55, 2.05s/it] 87%|████████▋ | 6506/7500 [3:42:00<33:29, 2.02s/it] 87%|████████▋ | 6507/7500 [3:42:02<33:39, 2.03s/it] 87%|████████▋ | 6508/7500 [3:42:04<33:29, 2.03s/it] 87%|████████▋ | 6509/7500 [3:42:06<33:30, 2.03s/it] 87%|████████▋ | 6510/7500 [3:42:08<33:47, 2.05s/it] {'loss': 0.3685, 'grad_norm': 2.951155185699463, 'learning_rate': 2.6072059940146775e-06, 'logqs/rejected': -2.6062979698181152, 'logqs/chosen': -0.8447366952896118, 'logqs_logps/rejected': -0.018369238823652267, 'logps_logqs/chosen': 0.20933851599693298, 'epoch': 1.74} 87%|████████▋ | 6510/7500 [3:42:08<33:47, 2.05s/it] 87%|████████▋ | 6511/7500 [3:42:10<34:14, 2.08s/it] 87%|████████▋ | 6512/7500 [3:42:12<34:19, 2.08s/it] 87%|████████▋ | 6513/7500 [3:42:14<33:42, 2.05s/it] 87%|████████▋ | 6514/7500 [3:42:16<34:00, 2.07s/it] 87%|████████▋ | 6515/7500 [3:42:18<33:57, 2.07s/it] 87%|████████▋ | 6516/7500 [3:42:20<34:16, 2.09s/it] 87%|████████▋ | 6517/7500 [3:42:23<34:54, 2.13s/it] 87%|████████▋ | 6518/7500 [3:42:25<35:03, 2.14s/it] 87%|████████▋ | 6519/7500 [3:42:27<35:01, 2.14s/it] 87%|████████▋ | 6520/7500 [3:42:29<34:45, 2.13s/it] {'loss': 0.3594, 'grad_norm': 2.82486629486084, 'learning_rate': 2.555713060848433e-06, 'logqs/rejected': -2.3264853954315186, 'logqs/chosen': -0.8660510182380676, 'logqs_logps/rejected': -0.024402687326073647, 'logps_logqs/chosen': 0.2069472372531891, 'epoch': 1.74} 87%|████████▋ | 6520/7500 [3:42:29<34:45, 2.13s/it] 87%|████████▋ | 6521/7500 [3:42:31<34:27, 2.11s/it] 87%|████████▋ | 6522/7500 [3:42:33<33:57, 2.08s/it] 87%|████████▋ | 6523/7500 [3:42:35<34:22, 2.11s/it] 87%|████████▋ | 6524/7500 [3:42:37<34:07, 2.10s/it] 87%|████████▋ | 6525/7500 [3:42:39<34:08, 2.10s/it] 87%|████████▋ | 6526/7500 [3:42:41<33:28, 2.06s/it] 87%|████████▋ | 6527/7500 [3:42:43<33:21, 2.06s/it] 87%|████████▋ | 6528/7500 [3:42:46<33:45, 2.08s/it] 87%|████████▋ | 6529/7500 [3:42:48<33:15, 2.05s/it] 87%|████████▋ | 6530/7500 [3:42:49<32:43, 2.02s/it] {'loss': 0.3439, 'grad_norm': 2.704381227493286, 'learning_rate': 2.504706307837551e-06, 'logqs/rejected': -2.573864698410034, 'logqs/chosen': -0.8508548736572266, 'logqs_logps/rejected': -0.02678755484521389, 'logps_logqs/chosen': 0.20324751734733582, 'epoch': 1.74} 87%|████████▋ | 6530/7500 [3:42:50<32:43, 2.02s/it] 87%|████████▋ | 6531/7500 [3:42:52<32:35, 2.02s/it] 87%|████████▋ | 6532/7500 [3:42:53<32:07, 1.99s/it] 87%|████████▋ | 6533/7500 [3:42:55<32:09, 2.00s/it] 87%|████████▋ | 6534/7500 [3:42:58<32:38, 2.03s/it] 87%|████████▋ | 6535/7500 [3:43:00<33:00, 2.05s/it] 87%|████████▋ | 6536/7500 [3:43:02<32:46, 2.04s/it] 87%|████████▋ | 6537/7500 [3:43:04<33:01, 2.06s/it] 87%|████████▋ | 6538/7500 [3:43:06<32:51, 2.05s/it] 87%|████████▋ | 6539/7500 [3:43:08<32:58, 2.06s/it] 87%|████████▋ | 6540/7500 [3:43:10<33:10, 2.07s/it] {'loss': 0.3573, 'grad_norm': 2.865234375, 'learning_rate': 2.454186839872158e-06, 'logqs/rejected': -2.5992302894592285, 'logqs/chosen': -0.8404319882392883, 'logqs_logps/rejected': -0.02418920025229454, 'logps_logqs/chosen': 0.2050396203994751, 'epoch': 1.74} 87%|████████▋ | 6540/7500 [3:43:10<33:10, 2.07s/it] 87%|████████▋ | 6541/7500 [3:43:12<32:58, 2.06s/it] 87%|████████▋ | 6542/7500 [3:43:14<33:20, 2.09s/it] 87%|████████▋ | 6543/7500 [3:43:16<33:19, 2.09s/it] 87%|████████▋ | 6544/7500 [3:43:18<33:04, 2.08s/it] 87%|████████▋ | 6545/7500 [3:43:20<32:54, 2.07s/it] 87%|████████▋ | 6546/7500 [3:43:22<32:55, 2.07s/it] 87%|████████▋ | 6547/7500 [3:43:24<32:31, 2.05s/it] 87%|████████▋ | 6548/7500 [3:43:26<32:16, 2.03s/it] 87%|████████▋ | 6549/7500 [3:43:29<32:29, 2.05s/it] 87%|████████▋ | 6550/7500 [3:43:31<33:01, 2.09s/it] {'loss': 0.3769, 'grad_norm': 3.685107707977295, 'learning_rate': 2.4041557512869878e-06, 'logqs/rejected': -2.845912456512451, 'logqs/chosen': -0.8613595962524414, 'logqs_logps/rejected': -0.022436099126935005, 'logps_logqs/chosen': 0.2019067108631134, 'epoch': 1.75} 87%|████████▋ | 6550/7500 [3:43:31<33:01, 2.09s/it] 87%|████████▋ | 6551/7500 [3:43:33<32:45, 2.07s/it] 87%|████████▋ | 6552/7500 [3:43:35<32:21, 2.05s/it] 87%|████████▋ | 6553/7500 [3:43:37<32:03, 2.03s/it] 87%|████████▋ | 6554/7500 [3:43:39<32:30, 2.06s/it] 87%|████████▋ | 6555/7500 [3:43:41<32:32, 2.07s/it] 87%|████████▋ | 6556/7500 [3:43:43<32:26, 2.06s/it] 87%|████████▋ | 6557/7500 [3:43:45<32:23, 2.06s/it] 87%|████████▋ | 6558/7500 [3:43:47<32:27, 2.07s/it] 87%|████████▋ | 6559/7500 [3:43:49<32:06, 2.05s/it] 87%|████████▋ | 6560/7500 [3:43:51<32:21, 2.07s/it] {'loss': 0.3658, 'grad_norm': 2.6645727157592773, 'learning_rate': 2.3546141258376785e-06, 'logqs/rejected': -2.56270694732666, 'logqs/chosen': -0.8379380106925964, 'logqs_logps/rejected': -0.02330847829580307, 'logps_logqs/chosen': 0.20398974418640137, 'epoch': 1.75} 87%|████████▋ | 6560/7500 [3:43:51<32:21, 2.07s/it] 87%|████████▋ | 6561/7500 [3:43:53<32:36, 2.08s/it] 87%|████████▋ | 6562/7500 [3:43:55<32:23, 2.07s/it] 88%|████████▊ | 6563/7500 [3:43:57<32:04, 2.05s/it] 88%|████████▊ | 6564/7500 [3:43:59<31:45, 2.04s/it] 88%|████████▊ | 6565/7500 [3:44:01<31:46, 2.04s/it] 88%|████████▊ | 6566/7500 [3:44:03<31:41, 2.04s/it] 88%|████████▊ | 6567/7500 [3:44:06<32:19, 2.08s/it] 88%|████████▊ | 6568/7500 [3:44:08<32:14, 2.08s/it] 88%|████████▊ | 6569/7500 [3:44:10<32:00, 2.06s/it] 88%|████████▊ | 6570/7500 [3:44:12<32:12, 2.08s/it] {'loss': 0.3528, 'grad_norm': 2.8056259155273438, 'learning_rate': 2.3055630366772856e-06, 'logqs/rejected': -2.699709415435791, 'logqs/chosen': -0.830964207649231, 'logqs_logps/rejected': -0.022664109244942665, 'logps_logqs/chosen': 0.2040327489376068, 'epoch': 1.75} 88%|████████▊ | 6570/7500 [3:44:12<32:12, 2.08s/it] 88%|████████▊ | 6571/7500 [3:44:14<32:08, 2.08s/it] 88%|████████▊ | 6572/7500 [3:44:16<31:52, 2.06s/it] 88%|████████▊ | 6573/7500 [3:44:18<32:09, 2.08s/it] 88%|████████▊ | 6574/7500 [3:44:20<32:09, 2.08s/it] 88%|████████▊ | 6575/7500 [3:44:22<32:23, 2.10s/it] 88%|████████▊ | 6576/7500 [3:44:24<32:37, 2.12s/it] 88%|████████▊ | 6577/7500 [3:44:27<33:27, 2.17s/it] 88%|████████▊ | 6578/7500 [3:44:29<33:15, 2.16s/it] 88%|████████▊ | 6579/7500 [3:44:31<32:30, 2.12s/it] 88%|████████▊ | 6580/7500 [3:44:33<32:10, 2.10s/it] {'loss': 0.3614, 'grad_norm': 2.9842638969421387, 'learning_rate': 2.257003546333042e-06, 'logqs/rejected': -2.7182564735412598, 'logqs/chosen': -0.8396220207214355, 'logqs_logps/rejected': -0.022881392389535904, 'logps_logqs/chosen': 0.20369529724121094, 'epoch': 1.75} 88%|████████▊ | 6580/7500 [3:44:33<32:10, 2.10s/it] 88%|████████▊ | 6581/7500 [3:44:35<32:03, 2.09s/it] 88%|████████▊ | 6582/7500 [3:44:37<32:05, 2.10s/it] 88%|████████▊ | 6583/7500 [3:44:39<31:54, 2.09s/it] 88%|████████▊ | 6584/7500 [3:44:41<31:28, 2.06s/it] 88%|████████▊ | 6585/7500 [3:44:43<31:31, 2.07s/it] 88%|████████▊ | 6586/7500 [3:44:45<31:46, 2.09s/it] 88%|████████▊ | 6587/7500 [3:44:47<31:30, 2.07s/it] 88%|████████▊ | 6588/7500 [3:44:50<31:21, 2.06s/it] 88%|████████▊ | 6589/7500 [3:44:51<30:53, 2.04s/it] 88%|████████▊ | 6590/7500 [3:44:54<30:50, 2.03s/it] {'loss': 0.344, 'grad_norm': 2.7786521911621094, 'learning_rate': 2.208936706683351e-06, 'logqs/rejected': -2.351536989212036, 'logqs/chosen': -0.8587857484817505, 'logqs_logps/rejected': -0.0281527079641819, 'logps_logqs/chosen': 0.20112721621990204, 'epoch': 1.76} 88%|████████▊ | 6590/7500 [3:44:54<30:50, 2.03s/it] 88%|████████▊ | 6591/7500 [3:44:56<30:35, 2.02s/it] 88%|████████▊ | 6592/7500 [3:44:58<30:36, 2.02s/it] 88%|████████▊ | 6593/7500 [3:45:00<30:31, 2.02s/it] 88%|████████▊ | 6594/7500 [3:45:02<30:44, 2.04s/it] 88%|████████▊ | 6595/7500 [3:45:04<31:02, 2.06s/it] 88%|████████▊ | 6596/7500 [3:45:06<30:59, 2.06s/it] 88%|████████▊ | 6597/7500 [3:45:08<30:31, 2.03s/it] 88%|████████▊ | 6598/7500 [3:45:10<30:33, 2.03s/it] 88%|████████▊ | 6599/7500 [3:45:12<31:06, 2.07s/it] 88%|████████▊ | 6600/7500 [3:45:14<31:10, 2.08s/it] {'loss': 0.3483, 'grad_norm': 3.0171091556549072, 'learning_rate': 2.1613635589349756e-06, 'logqs/rejected': -2.4215164184570312, 'logqs/chosen': -0.8718080520629883, 'logqs_logps/rejected': -0.026405606418848038, 'logps_logqs/chosen': 0.20489656925201416, 'epoch': 1.76} 88%|████████▊ | 6600/7500 [3:45:14<31:10, 2.08s/it] 88%|████████▊ | 6601/7500 [3:45:16<30:49, 2.06s/it] 88%|████████▊ | 6602/7500 [3:45:18<30:51, 2.06s/it] 88%|████████▊ | 6603/7500 [3:45:20<30:53, 2.07s/it] 88%|████████▊ | 6604/7500 [3:45:22<30:59, 2.08s/it] 88%|████████▊ | 6605/7500 [3:45:24<30:46, 2.06s/it] 88%|████████▊ | 6606/7500 [3:45:26<30:32, 2.05s/it] 88%|████████▊ | 6607/7500 [3:45:28<30:30, 2.05s/it] 88%|████████▊ | 6608/7500 [3:45:31<30:52, 2.08s/it] 88%|████████▊ | 6609/7500 [3:45:32<30:14, 2.04s/it] 88%|████████▊ | 6610/7500 [3:45:34<30:03, 2.03s/it] {'loss': 0.3565, 'grad_norm': 2.7923197746276855, 'learning_rate': 2.1142851336005243e-06, 'logqs/rejected': -2.535633087158203, 'logqs/chosen': -0.8267834782600403, 'logqs_logps/rejected': -0.019648101180791855, 'logps_logqs/chosen': 0.19790998101234436, 'epoch': 1.76} 88%|████████▊ | 6610/7500 [3:45:35<30:03, 2.03s/it] 88%|████████▊ | 6611/7500 [3:45:37<29:56, 2.02s/it] 88%|████████▊ | 6612/7500 [3:45:39<30:21, 2.05s/it] 88%|████████▊ | 6613/7500 [3:45:41<30:09, 2.04s/it] 88%|████████▊ | 6614/7500 [3:45:43<30:04, 2.04s/it] 88%|████████▊ | 6615/7500 [3:45:45<30:17, 2.05s/it] 88%|████████▊ | 6616/7500 [3:45:47<30:12, 2.05s/it] 88%|████████▊ | 6617/7500 [3:45:49<29:57, 2.04s/it] 88%|████████▊ | 6618/7500 [3:45:51<30:32, 2.08s/it] 88%|████████▊ | 6619/7500 [3:45:53<30:21, 2.07s/it] 88%|████████▊ | 6620/7500 [3:45:55<30:10, 2.06s/it] {'loss': 0.3736, 'grad_norm': 3.2186079025268555, 'learning_rate': 2.067702450476075e-06, 'logqs/rejected': -2.863408327102661, 'logqs/chosen': -0.8308460116386414, 'logqs_logps/rejected': -0.023759888485074043, 'logps_logqs/chosen': 0.20169202983379364, 'epoch': 1.77} 88%|████████▊ | 6620/7500 [3:45:55<30:10, 2.06s/it] 88%|████████▊ | 6621/7500 [3:45:57<29:51, 2.04s/it] 88%|████████▊ | 6622/7500 [3:45:59<29:48, 2.04s/it] 88%|████████▊ | 6623/7500 [3:46:01<29:42, 2.03s/it] 88%|████████▊ | 6624/7500 [3:46:03<29:52, 2.05s/it] 88%|████████▊ | 6625/7500 [3:46:05<29:40, 2.04s/it] 88%|████████▊ | 6626/7500 [3:46:07<29:37, 2.03s/it] 88%|████████▊ | 6627/7500 [3:46:09<30:01, 2.06s/it] 88%|████████▊ | 6628/7500 [3:46:11<30:01, 2.07s/it] 88%|████████▊ | 6629/7500 [3:46:14<30:11, 2.08s/it] 88%|████████▊ | 6630/7500 [3:46:16<29:44, 2.05s/it] {'loss': 0.3582, 'grad_norm': 2.648358106613159, 'learning_rate': 2.0216165186191407e-06, 'logqs/rejected': -2.6703364849090576, 'logqs/chosen': -0.834446132183075, 'logqs_logps/rejected': -0.017131637781858444, 'logps_logqs/chosen': 0.2041604071855545, 'epoch': 1.77} 88%|████████▊ | 6630/7500 [3:46:16<29:44, 2.05s/it] 88%|████████▊ | 6631/7500 [3:46:18<29:33, 2.04s/it] 88%|████████▊ | 6632/7500 [3:46:20<29:38, 2.05s/it] 88%|████████▊ | 6633/7500 [3:46:22<29:23, 2.03s/it] 88%|████████▊ | 6634/7500 [3:46:24<29:41, 2.06s/it] 88%|████████▊ | 6635/7500 [3:46:26<29:55, 2.08s/it] 88%|████████▊ | 6636/7500 [3:46:28<29:42, 2.06s/it] 88%|████████▊ | 6637/7500 [3:46:30<29:46, 2.07s/it] 89%|████████▊ | 6638/7500 [3:46:32<29:32, 2.06s/it] 89%|████████▊ | 6639/7500 [3:46:34<29:07, 2.03s/it] 89%|████████▊ | 6640/7500 [3:46:36<28:58, 2.02s/it] {'loss': 0.3553, 'grad_norm': 2.706773519515991, 'learning_rate': 1.9760283363267685e-06, 'logqs/rejected': -2.466472864151001, 'logqs/chosen': -0.8335142135620117, 'logqs_logps/rejected': -0.021768979728221893, 'logps_logqs/chosen': 0.19976136088371277, 'epoch': 1.77} 89%|████████▊ | 6640/7500 [3:46:36<28:58, 2.02s/it] 89%|████████▊ | 6641/7500 [3:46:38<29:16, 2.05s/it] 89%|████████▊ | 6642/7500 [3:46:40<29:16, 2.05s/it] 89%|████████▊ | 6643/7500 [3:46:42<29:23, 2.06s/it] 89%|████████▊ | 6644/7500 [3:46:44<29:24, 2.06s/it] 89%|████████▊ | 6645/7500 [3:46:46<29:22, 2.06s/it] 89%|████████▊ | 6646/7500 [3:46:48<29:36, 2.08s/it] 89%|████████▊ | 6647/7500 [3:46:50<29:24, 2.07s/it] 89%|████████▊ | 6648/7500 [3:46:52<29:00, 2.04s/it] 89%|████████▊ | 6649/7500 [3:46:55<29:15, 2.06s/it] 89%|████████▊ | 6650/7500 [3:46:57<29:13, 2.06s/it] {'loss': 0.3627, 'grad_norm': 2.6825010776519775, 'learning_rate': 1.9309388911139426e-06, 'logqs/rejected': -2.550105333328247, 'logqs/chosen': -0.8622382879257202, 'logqs_logps/rejected': -0.025242934003472328, 'logps_logqs/chosen': 0.20835967361927032, 'epoch': 1.77} 89%|████████▊ | 6650/7500 [3:46:57<29:13, 2.06s/it] 89%|████████▊ | 6651/7500 [3:46:59<29:18, 2.07s/it] 89%|████████▊ | 6652/7500 [3:47:01<29:34, 2.09s/it] 89%|████████▊ | 6653/7500 [3:47:03<29:11, 2.07s/it] 89%|████████▊ | 6654/7500 [3:47:05<28:50, 2.05s/it] 89%|████████▊ | 6655/7500 [3:47:07<28:38, 2.03s/it] 89%|████████▊ | 6656/7500 [3:47:09<28:56, 2.06s/it] 89%|████████▉ | 6657/7500 [3:47:11<28:33, 2.03s/it] 89%|████████▉ | 6658/7500 [3:47:13<28:46, 2.05s/it] 89%|████████▉ | 6659/7500 [3:47:15<28:32, 2.04s/it] 89%|████████▉ | 6660/7500 [3:47:17<28:55, 2.07s/it] {'loss': 0.329, 'grad_norm': 3.629014730453491, 'learning_rate': 1.8863491596921745e-06, 'logqs/rejected': -2.307041883468628, 'logqs/chosen': -0.8494784235954285, 'logqs_logps/rejected': -0.028533685952425003, 'logps_logqs/chosen': 0.2030322104692459, 'epoch': 1.78} 89%|████████▉ | 6660/7500 [3:47:17<28:55, 2.07s/it] 89%|████████▉ | 6661/7500 [3:47:19<28:53, 2.07s/it] 89%|████████▉ | 6662/7500 [3:47:21<28:52, 2.07s/it] 89%|████████▉ | 6663/7500 [3:47:23<28:40, 2.06s/it] 89%|████████▉ | 6664/7500 [3:47:25<28:46, 2.07s/it] 89%|████████▉ | 6665/7500 [3:47:28<28:46, 2.07s/it] 89%|████████▉ | 6666/7500 [3:47:30<28:49, 2.07s/it] 89%|████████▉ | 6667/7500 [3:47:32<28:39, 2.06s/it] 89%|████████▉ | 6668/7500 [3:47:34<28:41, 2.07s/it] 89%|████████▉ | 6669/7500 [3:47:36<28:30, 2.06s/it] 89%|████████▉ | 6670/7500 [3:47:38<28:39, 2.07s/it] {'loss': 0.3415, 'grad_norm': 2.971815347671509, 'learning_rate': 1.8422601079483514e-06, 'logqs/rejected': -2.347790002822876, 'logqs/chosen': -0.8428101539611816, 'logqs_logps/rejected': -0.02609463967382908, 'logps_logqs/chosen': 0.20624549686908722, 'epoch': 1.78} 89%|████████▉ | 6670/7500 [3:47:38<28:39, 2.07s/it] 89%|████████▉ | 6671/7500 [3:47:40<28:36, 2.07s/it] 89%|████████▉ | 6672/7500 [3:47:42<28:28, 2.06s/it] 89%|████████▉ | 6673/7500 [3:47:44<28:42, 2.08s/it] 89%|████████▉ | 6674/7500 [3:47:46<28:37, 2.08s/it] 89%|████████▉ | 6675/7500 [3:47:48<28:14, 2.05s/it] 89%|████████▉ | 6676/7500 [3:47:50<28:40, 2.09s/it] 89%|████████▉ | 6677/7500 [3:47:52<28:25, 2.07s/it] 89%|████████▉ | 6678/7500 [3:47:54<28:10, 2.06s/it] 89%|████████▉ | 6679/7500 [3:47:56<28:01, 2.05s/it] 89%|████████▉ | 6680/7500 [3:47:59<28:07, 2.06s/it] {'loss': 0.3474, 'grad_norm': 3.328947067260742, 'learning_rate': 1.7986726909238278e-06, 'logqs/rejected': -2.532921552658081, 'logqs/chosen': -0.8425439596176147, 'logqs_logps/rejected': -0.02589184045791626, 'logps_logqs/chosen': 0.20632080733776093, 'epoch': 1.78} 89%|████████▉ | 6680/7500 [3:47:59<28:07, 2.06s/it] 89%|████████▉ | 6681/7500 [3:48:01<27:50, 2.04s/it] 89%|████████▉ | 6682/7500 [3:48:03<28:01, 2.06s/it] 89%|████████▉ | 6683/7500 [3:48:05<27:43, 2.04s/it] 89%|████████▉ | 6684/7500 [3:48:07<27:40, 2.04s/it] 89%|████████▉ | 6685/7500 [3:48:09<28:03, 2.07s/it] 89%|████████▉ | 6686/7500 [3:48:11<28:18, 2.09s/it] 89%|████████▉ | 6687/7500 [3:48:13<28:26, 2.10s/it] 89%|████████▉ | 6688/7500 [3:48:15<28:27, 2.10s/it] 89%|████████▉ | 6689/7500 [3:48:17<28:28, 2.11s/it] 89%|████████▉ | 6690/7500 [3:48:19<28:03, 2.08s/it] {'loss': 0.3576, 'grad_norm': 2.901440382003784, 'learning_rate': 1.7555878527937164e-06, 'logqs/rejected': -2.717027187347412, 'logqs/chosen': -0.8674246668815613, 'logqs_logps/rejected': -0.021497488021850586, 'logps_logqs/chosen': 0.20310115814208984, 'epoch': 1.78} 89%|████████▉ | 6690/7500 [3:48:19<28:03, 2.08s/it] 89%|████████▉ | 6691/7500 [3:48:21<28:18, 2.10s/it] 89%|████████▉ | 6692/7500 [3:48:23<27:55, 2.07s/it] 89%|████████▉ | 6693/7500 [3:48:26<27:53, 2.07s/it] 89%|████████▉ | 6694/7500 [3:48:28<28:03, 2.09s/it] 89%|████████▉ | 6695/7500 [3:48:30<28:01, 2.09s/it] 89%|████████▉ | 6696/7500 [3:48:32<27:39, 2.06s/it] 89%|████████▉ | 6697/7500 [3:48:34<27:23, 2.05s/it] 89%|████████▉ | 6698/7500 [3:48:36<27:25, 2.05s/it] 89%|████████▉ | 6699/7500 [3:48:38<27:08, 2.03s/it] 89%|████████▉ | 6700/7500 [3:48:40<27:18, 2.05s/it] {'loss': 0.353, 'grad_norm': 2.4785561561584473, 'learning_rate': 1.713006526846439e-06, 'logqs/rejected': -2.482844591140747, 'logqs/chosen': -0.8432254791259766, 'logqs_logps/rejected': -0.02506417967379093, 'logps_logqs/chosen': 0.2026190310716629, 'epoch': 1.79} 89%|████████▉ | 6700/7500 [3:48:40<27:18, 2.05s/it] 89%|████████▉ | 6701/7500 [3:48:42<27:34, 2.07s/it] 89%|████████▉ | 6702/7500 [3:48:44<27:00, 2.03s/it] 89%|████████▉ | 6703/7500 [3:48:46<27:07, 2.04s/it] 89%|████████▉ | 6704/7500 [3:48:48<27:23, 2.06s/it] 89%|████████▉ | 6705/7500 [3:48:50<27:10, 2.05s/it] 89%|████████▉ | 6706/7500 [3:48:52<26:50, 2.03s/it] 89%|████████▉ | 6707/7500 [3:48:54<26:27, 2.00s/it] 89%|████████▉ | 6708/7500 [3:48:56<27:03, 2.05s/it] 89%|████████▉ | 6709/7500 [3:48:58<27:26, 2.08s/it] 89%|████████▉ | 6710/7500 [3:49:00<27:14, 2.07s/it] {'loss': 0.3602, 'grad_norm': 2.4371581077575684, 'learning_rate': 1.6709296354635335e-06, 'logqs/rejected': -2.574871063232422, 'logqs/chosen': -0.8532410860061646, 'logqs_logps/rejected': -0.02774166688323021, 'logps_logqs/chosen': 0.20821061730384827, 'epoch': 1.79} 89%|████████▉ | 6710/7500 [3:49:00<27:14, 2.07s/it] 89%|████████▉ | 6711/7500 [3:49:03<27:24, 2.08s/it] 89%|████████▉ | 6712/7500 [3:49:05<27:25, 2.09s/it] 90%|████████▉ | 6713/7500 [3:49:07<27:08, 2.07s/it] 90%|████████▉ | 6714/7500 [3:49:09<27:45, 2.12s/it] 90%|████████▉ | 6715/7500 [3:49:11<27:45, 2.12s/it] 90%|████████▉ | 6716/7500 [3:49:13<27:38, 2.12s/it] 90%|████████▉ | 6717/7500 [3:49:15<27:35, 2.11s/it] 90%|████████▉ | 6718/7500 [3:49:17<27:42, 2.13s/it] 90%|████████▉ | 6719/7500 [3:49:19<27:15, 2.09s/it] 90%|████████▉ | 6720/7500 [3:49:21<27:09, 2.09s/it] {'loss': 0.352, 'grad_norm': 2.7873427867889404, 'learning_rate': 1.629358090099639e-06, 'logqs/rejected': -2.437338352203369, 'logqs/chosen': -0.8537731170654297, 'logqs_logps/rejected': -0.024842670187354088, 'logps_logqs/chosen': 0.2100890576839447, 'epoch': 1.79} 90%|████████▉ | 6720/7500 [3:49:22<27:09, 2.09s/it] 90%|████████▉ | 6721/7500 [3:49:24<27:12, 2.10s/it] 90%|████████▉ | 6722/7500 [3:49:26<27:13, 2.10s/it] 90%|████████▉ | 6723/7500 [3:49:28<26:54, 2.08s/it] 90%|████████▉ | 6724/7500 [3:49:30<27:08, 2.10s/it] 90%|████████▉ | 6725/7500 [3:49:32<26:57, 2.09s/it] 90%|████████▉ | 6726/7500 [3:49:34<26:54, 2.09s/it] 90%|████████▉ | 6727/7500 [3:49:36<26:50, 2.08s/it] 90%|████████▉ | 6728/7500 [3:49:38<26:36, 2.07s/it] 90%|████████▉ | 6729/7500 [3:49:40<26:26, 2.06s/it] 90%|████████▉ | 6730/7500 [3:49:42<26:26, 2.06s/it] {'loss': 0.3423, 'grad_norm': 2.7191500663757324, 'learning_rate': 1.588292791262777e-06, 'logqs/rejected': -2.4281516075134277, 'logqs/chosen': -0.8279264569282532, 'logqs_logps/rejected': -0.02731684409081936, 'logps_logqs/chosen': 0.2046407163143158, 'epoch': 1.79} 90%|████████▉ | 6730/7500 [3:49:42<26:26, 2.06s/it] 90%|████████▉ | 6731/7500 [3:49:44<26:35, 2.08s/it] 90%|████████▉ | 6732/7500 [3:49:46<26:32, 2.07s/it] 90%|████████▉ | 6733/7500 [3:49:48<26:33, 2.08s/it] 90%|████████▉ | 6734/7500 [3:49:50<26:15, 2.06s/it] 90%|████████▉ | 6735/7500 [3:49:52<25:55, 2.03s/it] 90%|████████▉ | 6736/7500 [3:49:55<25:52, 2.03s/it] 90%|████████▉ | 6737/7500 [3:49:57<25:44, 2.02s/it] 90%|████████▉ | 6738/7500 [3:49:59<26:12, 2.06s/it] 90%|████████▉ | 6739/7500 [3:50:01<26:13, 2.07s/it] 90%|████████▉ | 6740/7500 [3:50:03<25:52, 2.04s/it] {'loss': 0.3556, 'grad_norm': 2.8797779083251953, 'learning_rate': 1.547734628494829e-06, 'logqs/rejected': -2.7079272270202637, 'logqs/chosen': -0.8499747514724731, 'logqs_logps/rejected': -0.02203221246600151, 'logps_logqs/chosen': 0.20315881073474884, 'epoch': 1.8} 90%|████████▉ | 6740/7500 [3:50:03<25:52, 2.04s/it] 90%|████████▉ | 6741/7500 [3:50:05<25:48, 2.04s/it] 90%|████████▉ | 6742/7500 [3:50:07<25:49, 2.04s/it] 90%|████████▉ | 6743/7500 [3:50:09<25:36, 2.03s/it] 90%|████████▉ | 6744/7500 [3:50:11<25:38, 2.03s/it] 90%|████████▉ | 6745/7500 [3:50:13<25:21, 2.02s/it] 90%|████████▉ | 6746/7500 [3:50:15<25:51, 2.06s/it] 90%|████████▉ | 6747/7500 [3:50:17<25:43, 2.05s/it] 90%|████████▉ | 6748/7500 [3:50:19<26:12, 2.09s/it] 90%|████████▉ | 6749/7500 [3:50:21<26:08, 2.09s/it] 90%|█████████ | 6750/7500 [3:50:23<25:46, 2.06s/it] {'loss': 0.3479, 'grad_norm': 2.613455295562744, 'learning_rate': 1.5076844803522922e-06, 'logqs/rejected': -2.6339640617370605, 'logqs/chosen': -0.8278962969779968, 'logqs_logps/rejected': -0.02896052598953247, 'logps_logqs/chosen': 0.20317165553569794, 'epoch': 1.8} 90%|█████████ | 6750/7500 [3:50:23<25:46, 2.06s/it] 90%|█████████ | 6751/7500 [3:50:25<25:31, 2.04s/it] 90%|█████████ | 6752/7500 [3:50:27<25:30, 2.05s/it] 90%|█████████ | 6753/7500 [3:50:30<25:53, 2.08s/it] 90%|█████████ | 6754/7500 [3:50:32<26:07, 2.10s/it] 90%|█████████ | 6755/7500 [3:50:34<26:17, 2.12s/it] 90%|█████████ | 6756/7500 [3:50:36<26:04, 2.10s/it] 90%|█████████ | 6757/7500 [3:50:38<25:24, 2.05s/it] 90%|█████████ | 6758/7500 [3:50:40<25:30, 2.06s/it] 90%|█████████ | 6759/7500 [3:50:42<25:30, 2.07s/it] 90%|█████████ | 6760/7500 [3:50:44<25:26, 2.06s/it] {'loss': 0.3502, 'grad_norm': 2.8815457820892334, 'learning_rate': 1.468143214387213e-06, 'logqs/rejected': -2.6065919399261475, 'logqs/chosen': -0.8606058955192566, 'logqs_logps/rejected': -0.027034860104322433, 'logps_logqs/chosen': 0.2025764435529709, 'epoch': 1.8} 90%|█████████ | 6760/7500 [3:50:44<25:26, 2.06s/it] 90%|█████████ | 6761/7500 [3:50:46<25:16, 2.05s/it] 90%|█████████ | 6762/7500 [3:50:48<25:13, 2.05s/it] 90%|█████████ | 6763/7500 [3:50:50<25:29, 2.07s/it] 90%|█████████ | 6764/7500 [3:50:52<25:38, 2.09s/it] 90%|█████████ | 6765/7500 [3:50:54<25:33, 2.09s/it] 90%|█████████ | 6766/7500 [3:50:56<25:15, 2.06s/it] 90%|█████████ | 6767/7500 [3:50:59<25:33, 2.09s/it] 90%|█████████ | 6768/7500 [3:51:01<25:23, 2.08s/it] 90%|█████████ | 6769/7500 [3:51:03<25:43, 2.11s/it] 90%|█████████ | 6770/7500 [3:51:05<25:16, 2.08s/it] {'loss': 0.3434, 'grad_norm': 3.2658324241638184, 'learning_rate': 1.4291116871284205e-06, 'logqs/rejected': -2.6768128871917725, 'logqs/chosen': -0.8271886706352234, 'logqs_logps/rejected': -0.027186071500182152, 'logps_logqs/chosen': 0.20464280247688293, 'epoch': 1.81} 90%|█████████ | 6770/7500 [3:51:05<25:16, 2.08s/it] 90%|█████████ | 6771/7500 [3:51:07<25:09, 2.07s/it] 90%|█████████ | 6772/7500 [3:51:09<24:54, 2.05s/it] 90%|█████████ | 6773/7500 [3:51:11<24:57, 2.06s/it] 90%|█████████ | 6774/7500 [3:51:13<24:54, 2.06s/it] 90%|█████████ | 6775/7500 [3:51:15<24:49, 2.05s/it] 90%|█████████ | 6776/7500 [3:51:17<25:00, 2.07s/it] 90%|█████████ | 6777/7500 [3:51:19<24:38, 2.04s/it] 90%|█████████ | 6778/7500 [3:51:21<24:24, 2.03s/it] 90%|█████████ | 6779/7500 [3:51:23<24:38, 2.05s/it] 90%|█████████ | 6780/7500 [3:51:25<24:46, 2.07s/it] {'loss': 0.3465, 'grad_norm': 3.0872087478637695, 'learning_rate': 1.3905907440629752e-06, 'logqs/rejected': -2.4638760089874268, 'logqs/chosen': -0.8328338861465454, 'logqs_logps/rejected': -0.025052016600966454, 'logps_logqs/chosen': 0.1959652155637741, 'epoch': 1.81} 90%|█████████ | 6780/7500 [3:51:25<24:46, 2.07s/it] 90%|█████████ | 6781/7500 [3:51:27<24:32, 2.05s/it] 90%|█████████ | 6782/7500 [3:51:30<24:52, 2.08s/it] 90%|█████████ | 6783/7500 [3:51:32<24:52, 2.08s/it] 90%|█████████ | 6784/7500 [3:51:34<24:23, 2.04s/it] 90%|█████████ | 6785/7500 [3:51:36<24:22, 2.05s/it] 90%|█████████ | 6786/7500 [3:51:38<24:51, 2.09s/it] 90%|█████████ | 6787/7500 [3:51:40<24:47, 2.09s/it] 91%|█████████ | 6788/7500 [3:51:42<24:35, 2.07s/it] 91%|█████████ | 6789/7500 [3:51:44<24:28, 2.06s/it] 91%|█████████ | 6790/7500 [3:51:46<24:29, 2.07s/it] {'loss': 0.3313, 'grad_norm': 3.2183592319488525, 'learning_rate': 1.3525812196178239e-06, 'logqs/rejected': -2.3509891033172607, 'logqs/chosen': -0.8384313583374023, 'logqs_logps/rejected': -0.026378944516181946, 'logps_logqs/chosen': 0.20292067527770996, 'epoch': 1.81} 91%|█████████ | 6790/7500 [3:51:46<24:29, 2.07s/it] 91%|█████████ | 6791/7500 [3:51:48<24:39, 2.09s/it] 91%|█████████ | 6792/7500 [3:51:50<24:28, 2.07s/it] 91%|█████████ | 6793/7500 [3:51:52<24:19, 2.06s/it] 91%|█████████ | 6794/7500 [3:51:54<24:14, 2.06s/it] 91%|█████████ | 6795/7500 [3:51:56<24:16, 2.07s/it] 91%|█████████ | 6796/7500 [3:51:58<24:08, 2.06s/it] 91%|█████████ | 6797/7500 [3:52:00<24:01, 2.05s/it] 91%|█████████ | 6798/7500 [3:52:03<24:06, 2.06s/it] 91%|█████████ | 6799/7500 [3:52:05<24:13, 2.07s/it] 91%|█████████ | 6800/7500 [3:52:07<24:12, 2.07s/it] {'loss': 0.3676, 'grad_norm': 2.4869384765625, 'learning_rate': 1.31508393714177e-06, 'logqs/rejected': -2.866652011871338, 'logqs/chosen': -0.8283501863479614, 'logqs_logps/rejected': -0.018966907635331154, 'logps_logqs/chosen': 0.20688596367835999, 'epoch': 1.81} 91%|█████████ | 6800/7500 [3:52:07<24:12, 2.07s/it] 91%|█████████ | 6801/7500 [3:52:09<23:52, 2.05s/it] 91%|█████████ | 6802/7500 [3:52:11<24:05, 2.07s/it] 91%|█████████ | 6803/7500 [3:52:13<23:53, 2.06s/it] 91%|█████████ | 6804/7500 [3:52:15<23:56, 2.06s/it] 91%|█████████ | 6805/7500 [3:52:17<24:03, 2.08s/it] 91%|█████████ | 6806/7500 [3:52:19<23:40, 2.05s/it] 91%|█████████ | 6807/7500 [3:52:21<23:57, 2.07s/it] 91%|█████████ | 6808/7500 [3:52:23<24:42, 2.14s/it] 91%|█████████ | 6809/7500 [3:52:26<24:32, 2.13s/it] 91%|█████████ | 6810/7500 [3:52:28<24:14, 2.11s/it] {'loss': 0.3567, 'grad_norm': 2.939582347869873, 'learning_rate': 1.2780997088875869e-06, 'logqs/rejected': -2.6436400413513184, 'logqs/chosen': -0.8262968063354492, 'logqs_logps/rejected': -0.024531947448849678, 'logps_logqs/chosen': 0.20314598083496094, 'epoch': 1.82} 91%|█████████ | 6810/7500 [3:52:28<24:14, 2.11s/it] 91%|█████████ | 6811/7500 [3:52:30<24:17, 2.12s/it] 91%|█████████ | 6812/7500 [3:52:32<24:00, 2.09s/it] 91%|█████████ | 6813/7500 [3:52:34<23:49, 2.08s/it] 91%|█████████ | 6814/7500 [3:52:36<23:29, 2.06s/it] 91%|█████████ | 6815/7500 [3:52:38<23:38, 2.07s/it] 91%|█████████ | 6816/7500 [3:52:40<23:39, 2.08s/it] 91%|█████████ | 6817/7500 [3:52:42<23:30, 2.07s/it] 91%|█████████ | 6818/7500 [3:52:44<23:34, 2.07s/it] 91%|█████████ | 6819/7500 [3:52:46<23:28, 2.07s/it] 91%|█████████ | 6820/7500 [3:52:48<23:36, 2.08s/it] {'loss': 0.3677, 'grad_norm': 5.178951740264893, 'learning_rate': 1.2416293359944708e-06, 'logqs/rejected': -2.6159167289733887, 'logqs/chosen': -0.8146454691886902, 'logqs_logps/rejected': -0.01737937144935131, 'logps_logqs/chosen': 0.20432715117931366, 'epoch': 1.82} 91%|█████████ | 6820/7500 [3:52:48<23:36, 2.08s/it] 91%|█████████ | 6821/7500 [3:52:50<23:40, 2.09s/it] 91%|█████████ | 6822/7500 [3:52:52<23:17, 2.06s/it] 91%|█████████ | 6823/7500 [3:52:55<23:17, 2.06s/it] 91%|█████████ | 6824/7500 [3:52:57<23:12, 2.06s/it] 91%|█████████ | 6825/7500 [3:52:59<22:54, 2.04s/it] 91%|█████████ | 6826/7500 [3:53:01<23:08, 2.06s/it] 91%|█████████ | 6827/7500 [3:53:03<23:20, 2.08s/it] 91%|█████████ | 6828/7500 [3:53:05<23:35, 2.11s/it] 91%|█████████ | 6829/7500 [3:53:07<23:40, 2.12s/it] 91%|█████████ | 6830/7500 [3:53:09<23:45, 2.13s/it] {'loss': 0.3604, 'grad_norm': 2.8342125415802, 'learning_rate': 1.2056736084706589e-06, 'logqs/rejected': -2.5386996269226074, 'logqs/chosen': -0.8577585220336914, 'logqs_logps/rejected': -0.02381417527794838, 'logps_logqs/chosen': 0.2025373876094818, 'epoch': 1.82} 91%|█████████ | 6830/7500 [3:53:09<23:45, 2.13s/it] 91%|█████████ | 6831/7500 [3:53:11<23:34, 2.11s/it] 91%|█████████ | 6832/7500 [3:53:14<23:51, 2.14s/it] 91%|█████████ | 6833/7500 [3:53:16<23:14, 2.09s/it] 91%|█████████ | 6834/7500 [3:53:18<23:07, 2.08s/it] 91%|█████████ | 6835/7500 [3:53:20<23:02, 2.08s/it] 91%|█████████ | 6836/7500 [3:53:22<23:02, 2.08s/it] 91%|█████████ | 6837/7500 [3:53:24<23:03, 2.09s/it] 91%|█████████ | 6838/7500 [3:53:26<22:40, 2.06s/it] 91%|█████████ | 6839/7500 [3:53:28<22:28, 2.04s/it] 91%|█████████ | 6840/7500 [3:53:30<22:28, 2.04s/it] {'loss': 0.3605, 'grad_norm': 2.6661477088928223, 'learning_rate': 1.170233305176327e-06, 'logqs/rejected': -2.589921712875366, 'logqs/chosen': -0.868582546710968, 'logqs_logps/rejected': -0.026638850569725037, 'logps_logqs/chosen': 0.20350608229637146, 'epoch': 1.82} 91%|█████████ | 6840/7500 [3:53:30<22:28, 2.04s/it] 91%|█████████ | 6841/7500 [3:53:32<22:18, 2.03s/it] 91%|█████████ | 6842/7500 [3:53:34<22:02, 2.01s/it] 91%|█████████ | 6843/7500 [3:53:36<22:25, 2.05s/it] 91%|█████████▏| 6844/7500 [3:53:38<22:45, 2.08s/it] 91%|█████████▏| 6845/7500 [3:53:40<22:44, 2.08s/it] 91%|█████████▏| 6846/7500 [3:53:42<22:47, 2.09s/it] 91%|█████████▏| 6847/7500 [3:53:44<22:36, 2.08s/it] 91%|█████████▏| 6848/7500 [3:53:47<22:58, 2.11s/it] 91%|█████████▏| 6849/7500 [3:53:49<22:42, 2.09s/it] 91%|█████████▏| 6850/7500 [3:53:51<22:33, 2.08s/it] {'loss': 0.35, 'grad_norm': 2.555414915084839, 'learning_rate': 1.1353091938067023e-06, 'logqs/rejected': -2.7864630222320557, 'logqs/chosen': -0.8407818675041199, 'logqs_logps/rejected': -0.024999845772981644, 'logps_logqs/chosen': 0.2044728696346283, 'epoch': 1.83} 91%|█████████▏| 6850/7500 [3:53:51<22:33, 2.08s/it] 91%|█████████▏| 6851/7500 [3:53:53<22:38, 2.09s/it] 91%|█████████▏| 6852/7500 [3:53:55<22:16, 2.06s/it] 91%|█████████▏| 6853/7500 [3:53:57<22:26, 2.08s/it] 91%|█████████▏| 6854/7500 [3:53:59<22:41, 2.11s/it] 91%|█████████▏| 6855/7500 [3:54:01<22:36, 2.10s/it] 91%|█████████▏| 6856/7500 [3:54:03<22:27, 2.09s/it] 91%|█████████▏| 6857/7500 [3:54:05<22:27, 2.10s/it] 91%|█████████▏| 6858/7500 [3:54:07<22:26, 2.10s/it] 91%|█████████▏| 6859/7500 [3:54:09<22:11, 2.08s/it] 91%|█████████▏| 6860/7500 [3:54:12<22:16, 2.09s/it] {'loss': 0.3478, 'grad_norm': 3.0573787689208984, 'learning_rate': 1.1009020308754586e-06, 'logqs/rejected': -2.5505356788635254, 'logqs/chosen': -0.8626365661621094, 'logqs_logps/rejected': -0.03159504383802414, 'logps_logqs/chosen': 0.20364543795585632, 'epoch': 1.83} 91%|█████████▏| 6860/7500 [3:54:12<22:16, 2.09s/it] 91%|█████████▏| 6861/7500 [3:54:14<22:00, 2.07s/it] 91%|█████████▏| 6862/7500 [3:54:16<22:03, 2.07s/it] 92%|█████████▏| 6863/7500 [3:54:18<22:09, 2.09s/it] 92%|█████████▏| 6864/7500 [3:54:20<22:08, 2.09s/it] 92%|█████████▏| 6865/7500 [3:54:22<22:20, 2.11s/it] 92%|█████████▏| 6866/7500 [3:54:24<22:23, 2.12s/it] 92%|█████████▏| 6867/7500 [3:54:26<22:00, 2.09s/it] 92%|█████████▏| 6868/7500 [3:54:28<22:04, 2.10s/it] 92%|█████████▏| 6869/7500 [3:54:30<21:50, 2.08s/it] 92%|█████████▏| 6870/7500 [3:54:33<21:54, 2.09s/it] {'loss': 0.3633, 'grad_norm': 3.4648826122283936, 'learning_rate': 1.067012561698319e-06, 'logqs/rejected': -2.6931374073028564, 'logqs/chosen': -0.8529256582260132, 'logqs_logps/rejected': -0.02062794752418995, 'logps_logqs/chosen': 0.20467190444469452, 'epoch': 1.83} 92%|█████████▏| 6870/7500 [3:54:33<21:54, 2.09s/it] 92%|█████████▏| 6871/7500 [3:54:35<21:53, 2.09s/it] 92%|█████████▏| 6872/7500 [3:54:37<21:47, 2.08s/it] 92%|█████████▏| 6873/7500 [3:54:39<21:38, 2.07s/it] 92%|█████████▏| 6874/7500 [3:54:41<21:40, 2.08s/it] 92%|█████████▏| 6875/7500 [3:54:43<21:31, 2.07s/it] 92%|█████████▏| 6876/7500 [3:54:45<21:20, 2.05s/it] 92%|█████████▏| 6877/7500 [3:54:47<21:41, 2.09s/it] 92%|█████████▏| 6878/7500 [3:54:49<22:00, 2.12s/it] 92%|█████████▏| 6879/7500 [3:54:51<21:55, 2.12s/it] 92%|█████████▏| 6880/7500 [3:54:53<21:49, 2.11s/it] {'loss': 0.3613, 'grad_norm': 2.8400352001190186, 'learning_rate': 1.0336415203768962e-06, 'logqs/rejected': -2.4821479320526123, 'logqs/chosen': -0.876547634601593, 'logqs_logps/rejected': -0.022601881995797157, 'logps_logqs/chosen': 0.20654375851154327, 'epoch': 1.83} 92%|█████████▏| 6880/7500 [3:54:53<21:49, 2.11s/it] 92%|█████████▏| 6881/7500 [3:54:55<21:27, 2.08s/it] 92%|█████████▏| 6882/7500 [3:54:58<21:41, 2.11s/it] 92%|█████████▏| 6883/7500 [3:55:00<21:31, 2.09s/it] 92%|█████████▏| 6884/7500 [3:55:02<21:15, 2.07s/it] 92%|█████████▏| 6885/7500 [3:55:04<21:17, 2.08s/it] 92%|█████████▏| 6886/7500 [3:55:06<21:24, 2.09s/it] 92%|█████████▏| 6887/7500 [3:55:08<21:16, 2.08s/it] 92%|█████████▏| 6888/7500 [3:55:10<21:17, 2.09s/it] 92%|█████████▏| 6889/7500 [3:55:12<21:06, 2.07s/it] 92%|█████████▏| 6890/7500 [3:55:14<21:01, 2.07s/it] {'loss': 0.3556, 'grad_norm': 2.9712986946105957, 'learning_rate': 1.0007896297828113e-06, 'logqs/rejected': -2.5530600547790527, 'logqs/chosen': -0.8493191003799438, 'logqs_logps/rejected': -0.030283451080322266, 'logps_logqs/chosen': 0.2059970647096634, 'epoch': 1.84} 92%|█████████▏| 6890/7500 [3:55:14<21:01, 2.07s/it] 92%|█████████▏| 6891/7500 [3:55:16<21:11, 2.09s/it] 92%|█████████▏| 6892/7500 [3:55:18<21:00, 2.07s/it] 92%|█████████▏| 6893/7500 [3:55:20<20:48, 2.06s/it] 92%|█████████▏| 6894/7500 [3:55:22<20:46, 2.06s/it] 92%|█████████▏| 6895/7500 [3:55:25<20:54, 2.07s/it] 92%|█████████▏| 6896/7500 [3:55:27<20:43, 2.06s/it] 92%|█████████▏| 6897/7500 [3:55:29<20:43, 2.06s/it] 92%|█████████▏| 6898/7500 [3:55:31<20:40, 2.06s/it] 92%|█████████▏| 6899/7500 [3:55:33<20:54, 2.09s/it] 92%|█████████▏| 6900/7500 [3:55:35<20:56, 2.09s/it] {'loss': 0.3457, 'grad_norm': 2.500746488571167, 'learning_rate': 9.684576015420278e-07, 'logqs/rejected': -2.3746840953826904, 'logqs/chosen': -0.8418115377426147, 'logqs_logps/rejected': -0.027400946244597435, 'logps_logqs/chosen': 0.2059028446674347, 'epoch': 1.84} 92%|█████████▏| 6900/7500 [3:55:35<20:56, 2.09s/it] 92%|█████████▏| 6901/7500 [3:55:37<21:05, 2.11s/it] 92%|█████████▏| 6902/7500 [3:55:39<20:56, 2.10s/it] 92%|█████████▏| 6903/7500 [3:55:41<20:47, 2.09s/it] 92%|█████████▏| 6904/7500 [3:55:43<20:48, 2.10s/it] 92%|█████████▏| 6905/7500 [3:55:45<20:33, 2.07s/it] 92%|█████████▏| 6906/7500 [3:55:47<20:38, 2.08s/it] 92%|█████████▏| 6907/7500 [3:55:50<20:27, 2.07s/it] 92%|█████████▏| 6908/7500 [3:55:52<20:25, 2.07s/it] 92%|█████████▏| 6909/7500 [3:55:54<20:19, 2.06s/it] 92%|█████████▏| 6910/7500 [3:55:56<20:12, 2.05s/it] {'loss': 0.345, 'grad_norm': 2.92647647857666, 'learning_rate': 9.366461360194339e-07, 'logqs/rejected': -2.438178777694702, 'logqs/chosen': -0.8468382954597473, 'logqs_logps/rejected': -0.026197051629424095, 'logps_logqs/chosen': 0.1989055573940277, 'epoch': 1.84} 92%|█████████▏| 6910/7500 [3:55:56<20:12, 2.05s/it] 92%|█████████▏| 6911/7500 [3:55:58<19:56, 2.03s/it] 92%|█████████▏| 6912/7500 [3:56:00<20:11, 2.06s/it] 92%|█████████▏| 6913/7500 [3:56:02<20:14, 2.07s/it] 92%|█████████▏| 6914/7500 [3:56:04<20:02, 2.05s/it] 92%|█████████▏| 6915/7500 [3:56:06<20:25, 2.10s/it] 92%|█████████▏| 6916/7500 [3:56:08<20:17, 2.08s/it] 92%|█████████▏| 6917/7500 [3:56:10<20:14, 2.08s/it] 92%|█████████▏| 6918/7500 [3:56:12<20:16, 2.09s/it] 92%|█████████▏| 6919/7500 [3:56:14<20:05, 2.08s/it] 92%|█████████▏| 6920/7500 [3:56:16<20:02, 2.07s/it] {'loss': 0.3525, 'grad_norm': 3.7343180179595947, 'learning_rate': 9.053559223036745e-07, 'logqs/rejected': -2.3617682456970215, 'logqs/chosen': -0.8379664421081543, 'logqs_logps/rejected': -0.026347452774643898, 'logps_logqs/chosen': 0.21094179153442383, 'epoch': 1.85} 92%|█████████▏| 6920/7500 [3:56:16<20:02, 2.07s/it] 92%|█████████▏| 6921/7500 [3:56:19<20:06, 2.08s/it] 92%|█████████▏| 6922/7500 [3:56:21<20:03, 2.08s/it] 92%|█████████▏| 6923/7500 [3:56:23<19:48, 2.06s/it] 92%|█████████▏| 6924/7500 [3:56:25<19:39, 2.05s/it] 92%|█████████▏| 6925/7500 [3:56:27<19:28, 2.03s/it] 92%|█████████▏| 6926/7500 [3:56:29<20:06, 2.10s/it] 92%|█████████▏| 6927/7500 [3:56:31<20:10, 2.11s/it] 92%|█████████▏| 6928/7500 [3:56:33<19:48, 2.08s/it] 92%|█████████▏| 6929/7500 [3:56:35<19:31, 2.05s/it] 92%|█████████▏| 6930/7500 [3:56:37<19:32, 2.06s/it] {'loss': 0.3481, 'grad_norm': 2.922255754470825, 'learning_rate': 8.745876381922147e-07, 'logqs/rejected': -2.420527458190918, 'logqs/chosen': -0.8419563174247742, 'logqs_logps/rejected': -0.020985368639230728, 'logps_logqs/chosen': 0.2046816647052765, 'epoch': 1.85} 92%|█████████▏| 6930/7500 [3:56:37<19:32, 2.06s/it] 92%|█████████▏| 6931/7500 [3:56:39<19:32, 2.06s/it] 92%|█████████▏| 6932/7500 [3:56:41<19:35, 2.07s/it] 92%|█████████▏| 6933/7500 [3:56:43<19:45, 2.09s/it] 92%|█████████▏| 6934/7500 [3:56:45<19:39, 2.08s/it] 92%|█████████▏| 6935/7500 [3:56:48<19:37, 2.08s/it] 92%|█████████▏| 6936/7500 [3:56:50<19:32, 2.08s/it] 92%|█████████▏| 6937/7500 [3:56:52<19:33, 2.08s/it] 93%|█████████▎| 6938/7500 [3:56:54<19:25, 2.07s/it] 93%|█████████▎| 6939/7500 [3:56:56<19:12, 2.05s/it] 93%|█████████▎| 6940/7500 [3:56:58<19:06, 2.05s/it] {'loss': 0.359, 'grad_norm': 2.891068696975708, 'learning_rate': 8.443419501766831e-07, 'logqs/rejected': -2.448302745819092, 'logqs/chosen': -0.8356693387031555, 'logqs_logps/rejected': -0.022235233336687088, 'logps_logqs/chosen': 0.20369437336921692, 'epoch': 1.85} 93%|█████████▎| 6940/7500 [3:56:58<19:06, 2.05s/it] 93%|█████████▎| 6941/7500 [3:57:00<19:04, 2.05s/it] 93%|█████████▎| 6942/7500 [3:57:02<19:16, 2.07s/it] 93%|█████████▎| 6943/7500 [3:57:04<19:34, 2.11s/it] 93%|█████████▎| 6944/7500 [3:57:06<19:16, 2.08s/it] 93%|█████████▎| 6945/7500 [3:57:08<19:13, 2.08s/it] 93%|█████████▎| 6946/7500 [3:57:10<19:07, 2.07s/it] 93%|█████████▎| 6947/7500 [3:57:12<18:56, 2.05s/it] 93%|█████████▎| 6948/7500 [3:57:14<18:54, 2.05s/it] 93%|█████████▎| 6949/7500 [3:57:16<18:48, 2.05s/it] 93%|█████████▎| 6950/7500 [3:57:19<19:12, 2.10s/it] {'loss': 0.367, 'grad_norm': 3.020768642425537, 'learning_rate': 8.146195134284052e-07, 'logqs/rejected': -2.384185791015625, 'logqs/chosen': -0.8642687797546387, 'logqs_logps/rejected': -0.018643399700522423, 'logps_logqs/chosen': 0.21098439395427704, 'epoch': 1.85} 93%|█████████▎| 6950/7500 [3:57:19<19:12, 2.10s/it] 93%|█████████▎| 6951/7500 [3:57:21<19:27, 2.13s/it] 93%|█████████▎| 6952/7500 [3:57:23<19:14, 2.11s/it] 93%|█████████▎| 6953/7500 [3:57:25<19:10, 2.10s/it] 93%|█████████▎| 6954/7500 [3:57:27<19:19, 2.12s/it] 93%|█████████▎| 6955/7500 [3:57:29<19:17, 2.12s/it] 93%|█████████▎| 6956/7500 [3:57:31<18:59, 2.10s/it] 93%|█████████▎| 6957/7500 [3:57:33<18:59, 2.10s/it] 93%|█████████▎| 6958/7500 [3:57:36<19:00, 2.10s/it] 93%|█████████▎| 6959/7500 [3:57:38<18:47, 2.08s/it] 93%|█████████▎| 6960/7500 [3:57:40<18:35, 2.07s/it] {'loss': 0.3556, 'grad_norm': 3.4292781352996826, 'learning_rate': 7.854209717842231e-07, 'logqs/rejected': -2.4963557720184326, 'logqs/chosen': -0.8804244995117188, 'logqs_logps/rejected': -0.02347567118704319, 'logps_logqs/chosen': 0.20616130530834198, 'epoch': 1.86} 93%|█████████▎| 6960/7500 [3:57:40<18:35, 2.07s/it] 93%|█████████▎| 6961/7500 [3:57:42<18:25, 2.05s/it] 93%|█████████▎| 6962/7500 [3:57:44<18:27, 2.06s/it] 93%|█████████▎| 6963/7500 [3:57:46<18:32, 2.07s/it] 93%|█████████▎| 6964/7500 [3:57:48<18:20, 2.05s/it] 93%|█████████▎| 6965/7500 [3:57:50<18:35, 2.09s/it] 93%|█████████▎| 6966/7500 [3:57:52<18:33, 2.09s/it] 93%|█████████▎| 6967/7500 [3:57:54<18:33, 2.09s/it] 93%|█████████▎| 6968/7500 [3:57:56<18:41, 2.11s/it] 93%|█████████▎| 6969/7500 [3:57:58<18:29, 2.09s/it] 93%|█████████▎| 6970/7500 [3:58:00<18:24, 2.08s/it] {'loss': 0.3493, 'grad_norm': 3.385817050933838, 'learning_rate': 7.567469577325598e-07, 'logqs/rejected': -2.496750593185425, 'logqs/chosen': -0.83271723985672, 'logqs_logps/rejected': -0.026345735415816307, 'logps_logqs/chosen': 0.20923829078674316, 'epoch': 1.86} 93%|█████████▎| 6970/7500 [3:58:00<18:24, 2.08s/it] 93%|█████████▎| 6971/7500 [3:58:02<18:23, 2.09s/it] 93%|█████████▎| 6972/7500 [3:58:04<18:07, 2.06s/it] 93%|█████████▎| 6973/7500 [3:58:07<18:05, 2.06s/it] 93%|█████████▎| 6974/7500 [3:58:09<18:01, 2.06s/it] 93%|█████████▎| 6975/7500 [3:58:11<17:52, 2.04s/it] 93%|█████████▎| 6976/7500 [3:58:13<17:59, 2.06s/it] 93%|█████████▎| 6977/7500 [3:58:15<17:56, 2.06s/it] 93%|█████████▎| 6978/7500 [3:58:17<18:05, 2.08s/it] 93%|█████████▎| 6979/7500 [3:58:19<18:01, 2.08s/it] 93%|█████████▎| 6980/7500 [3:58:21<17:59, 2.08s/it] {'loss': 0.3555, 'grad_norm': 2.6685311794281006, 'learning_rate': 7.285980923996988e-07, 'logqs/rejected': -2.588716983795166, 'logqs/chosen': -0.8471843600273132, 'logqs_logps/rejected': -0.026189183816313744, 'logps_logqs/chosen': 0.20602980256080627, 'epoch': 1.86} 93%|█████████▎| 6980/7500 [3:58:21<17:59, 2.08s/it] 93%|█████████▎| 6981/7500 [3:58:23<17:58, 2.08s/it] 93%|█████████▎| 6982/7500 [3:58:25<17:50, 2.07s/it] 93%|█████████▎| 6983/7500 [3:58:27<17:49, 2.07s/it] 93%|█████████▎| 6984/7500 [3:58:29<17:47, 2.07s/it] 93%|█████████▎| 6985/7500 [3:58:31<17:49, 2.08s/it] 93%|█████████▎| 6986/7500 [3:58:34<18:13, 2.13s/it] 93%|█████████▎| 6987/7500 [3:58:36<17:50, 2.09s/it] 93%|█████████▎| 6988/7500 [3:58:38<17:41, 2.07s/it] 93%|█████████▎| 6989/7500 [3:58:40<17:32, 2.06s/it] 93%|█████████▎| 6990/7500 [3:58:42<17:48, 2.09s/it] {'loss': 0.3589, 'grad_norm': 2.4147095680236816, 'learning_rate': 7.009749855363456e-07, 'logqs/rejected': -2.7124595642089844, 'logqs/chosen': -0.8272632360458374, 'logqs_logps/rejected': -0.023257693275809288, 'logps_logqs/chosen': 0.20262520015239716, 'epoch': 1.86} 93%|█████████▎| 6990/7500 [3:58:42<17:48, 2.09s/it] 93%|█████████▎| 6991/7500 [3:58:44<17:45, 2.09s/it] 93%|█████████▎| 6992/7500 [3:58:46<17:37, 2.08s/it] 93%|█████████▎| 6993/7500 [3:58:48<17:19, 2.05s/it] 93%|█████████▎| 6994/7500 [3:58:50<17:24, 2.06s/it] 93%|█████████▎| 6995/7500 [3:58:52<17:32, 2.08s/it] 93%|█████████▎| 6996/7500 [3:58:54<17:27, 2.08s/it] 93%|█████████▎| 6997/7500 [3:58:56<17:14, 2.06s/it] 93%|█████████▎| 6998/7500 [3:58:58<17:18, 2.07s/it] 93%|█████████▎| 6999/7500 [3:59:01<17:27, 2.09s/it] 93%|█████████▎| 7000/7500 [3:59:03<17:24, 2.09s/it] {'loss': 0.3528, 'grad_norm': 2.800353527069092, 'learning_rate': 6.738782355044049e-07, 'logqs/rejected': -2.602264881134033, 'logqs/chosen': -0.8515856862068176, 'logqs_logps/rejected': -0.02294999361038208, 'logps_logqs/chosen': 0.20015983283519745, 'epoch': 1.87} 93%|█████████▎| 7000/7500 [3:59:03<17:24, 2.09s/it] 93%|█████████▎| 7001/7500 [3:59:05<17:20, 2.09s/it] 93%|█████████▎| 7002/7500 [3:59:07<17:16, 2.08s/it] 93%|█████████▎| 7003/7500 [3:59:09<17:06, 2.06s/it] 93%|█████████▎| 7004/7500 [3:59:11<17:08, 2.07s/it] 93%|█████████▎| 7005/7500 [3:59:13<17:13, 2.09s/it] 93%|█████████▎| 7006/7500 [3:59:15<17:04, 2.07s/it] 93%|█████████▎| 7007/7500 [3:59:17<17:13, 2.10s/it] 93%|█████████▎| 7008/7500 [3:59:19<17:28, 2.13s/it] 93%|█████████▎| 7009/7500 [3:59:22<17:23, 2.12s/it] 93%|█████████▎| 7010/7500 [3:59:24<17:12, 2.11s/it] {'loss': 0.376, 'grad_norm': 2.5693321228027344, 'learning_rate': 6.473084292640319e-07, 'logqs/rejected': -2.8773794174194336, 'logqs/chosen': -0.8404903411865234, 'logqs_logps/rejected': -0.01939675770699978, 'logps_logqs/chosen': 0.20756319165229797, 'epoch': 1.87} 93%|█████████▎| 7010/7500 [3:59:24<17:12, 2.11s/it] 93%|█████████▎| 7011/7500 [3:59:26<17:03, 2.09s/it] 93%|█████████▎| 7012/7500 [3:59:28<17:07, 2.11s/it] 94%|█████████▎| 7013/7500 [3:59:30<17:06, 2.11s/it] 94%|█████████▎| 7014/7500 [3:59:32<16:58, 2.10s/it] 94%|█████████▎| 7015/7500 [3:59:34<16:52, 2.09s/it] 94%|█████████▎| 7016/7500 [3:59:36<16:51, 2.09s/it] 94%|█████████▎| 7017/7500 [3:59:38<16:46, 2.08s/it] 94%|█████████▎| 7018/7500 [3:59:40<16:34, 2.06s/it] 94%|█████████▎| 7019/7500 [3:59:42<16:30, 2.06s/it] 94%|█████████▎| 7020/7500 [3:59:44<16:45, 2.09s/it] {'loss': 0.3666, 'grad_norm': 2.87178897857666, 'learning_rate': 6.212661423609184e-07, 'logqs/rejected': -2.6238274574279785, 'logqs/chosen': -0.8488869667053223, 'logqs_logps/rejected': -0.0250246524810791, 'logps_logqs/chosen': 0.20585255324840546, 'epoch': 1.87} 94%|█████████▎| 7020/7500 [3:59:44<16:45, 2.09s/it] 94%|█████████▎| 7021/7500 [3:59:46<16:31, 2.07s/it] 94%|█████████▎| 7022/7500 [3:59:49<16:26, 2.06s/it] 94%|█████████▎| 7023/7500 [3:59:51<16:21, 2.06s/it] 94%|█████████▎| 7024/7500 [3:59:53<16:15, 2.05s/it] 94%|█████████▎| 7025/7500 [3:59:55<16:32, 2.09s/it] 94%|█████████▎| 7026/7500 [3:59:57<16:33, 2.10s/it] 94%|█████████▎| 7027/7500 [3:59:59<16:22, 2.08s/it] 94%|█████████▎| 7028/7500 [4:00:01<16:25, 2.09s/it] 94%|█████████▎| 7029/7500 [4:00:03<16:37, 2.12s/it] 94%|█████████▎| 7030/7500 [4:00:05<16:35, 2.12s/it] {'loss': 0.3654, 'grad_norm': 2.847666025161743, 'learning_rate': 5.957519389138105e-07, 'logqs/rejected': -2.717665672302246, 'logqs/chosen': -0.8207191228866577, 'logqs_logps/rejected': -0.02098185382783413, 'logps_logqs/chosen': 0.20261533558368683, 'epoch': 1.87} 94%|█████████▎| 7030/7500 [4:00:05<16:35, 2.12s/it] 94%|█████████▎| 7031/7500 [4:00:07<16:14, 2.08s/it] 94%|█████████▍| 7032/7500 [4:00:09<16:09, 2.07s/it] 94%|█████████▍| 7033/7500 [4:00:11<16:04, 2.06s/it] 94%|█████████▍| 7034/7500 [4:00:13<15:58, 2.06s/it] 94%|█████████▍| 7035/7500 [4:00:15<15:45, 2.03s/it] 94%|█████████▍| 7036/7500 [4:00:17<15:45, 2.04s/it] 94%|█████████▍| 7037/7500 [4:00:20<16:06, 2.09s/it] 94%|█████████▍| 7038/7500 [4:00:22<15:54, 2.07s/it] 94%|█████████▍| 7039/7500 [4:00:24<15:49, 2.06s/it] 94%|█████████▍| 7040/7500 [4:00:26<15:51, 2.07s/it] {'loss': 0.3746, 'grad_norm': 3.089846611022949, 'learning_rate': 5.70766371602302e-07, 'logqs/rejected': -2.7546181678771973, 'logqs/chosen': -0.842376708984375, 'logqs_logps/rejected': -0.016002826392650604, 'logps_logqs/chosen': 0.20790711045265198, 'epoch': 1.88} 94%|█████████▍| 7040/7500 [4:00:26<15:51, 2.07s/it] 94%|█████████▍| 7041/7500 [4:00:28<15:42, 2.05s/it] 94%|█████████▍| 7042/7500 [4:00:30<15:42, 2.06s/it] 94%|█████████▍| 7043/7500 [4:00:32<15:48, 2.07s/it] 94%|█████████▍| 7044/7500 [4:00:34<15:41, 2.06s/it] 94%|█████████▍| 7045/7500 [4:00:36<15:58, 2.11s/it] 94%|█████████▍| 7046/7500 [4:00:38<15:53, 2.10s/it] 94%|█████████▍| 7047/7500 [4:00:40<15:41, 2.08s/it] 94%|█████████▍| 7048/7500 [4:00:42<15:31, 2.06s/it] 94%|█████████▍| 7049/7500 [4:00:45<15:41, 2.09s/it] 94%|█████████▍| 7050/7500 [4:00:47<15:28, 2.06s/it] {'loss': 0.3396, 'grad_norm': 2.3100244998931885, 'learning_rate': 5.463099816548579e-07, 'logqs/rejected': -2.5782198905944824, 'logqs/chosen': -0.8188560605049133, 'logqs_logps/rejected': -0.02394760027527809, 'logps_logqs/chosen': 0.19783897697925568, 'epoch': 1.88} 94%|█████████▍| 7050/7500 [4:00:47<15:28, 2.06s/it] 94%|█████████▍| 7051/7500 [4:00:49<15:35, 2.08s/it] 94%|█████████▍| 7052/7500 [4:00:51<15:36, 2.09s/it] 94%|█████████▍| 7053/7500 [4:00:53<15:45, 2.11s/it] 94%|█████████▍| 7054/7500 [4:00:55<15:47, 2.12s/it] 94%|█████████▍| 7055/7500 [4:00:57<15:40, 2.11s/it] 94%|█████████▍| 7056/7500 [4:00:59<15:28, 2.09s/it] 94%|█████████▍| 7057/7500 [4:01:01<15:20, 2.08s/it] 94%|█████████▍| 7058/7500 [4:01:03<15:24, 2.09s/it] 94%|█████████▍| 7059/7500 [4:01:05<15:14, 2.07s/it] 94%|█████████▍| 7060/7500 [4:01:07<15:02, 2.05s/it] {'loss': 0.3493, 'grad_norm': 3.046771764755249, 'learning_rate': 5.22383298837098e-07, 'logqs/rejected': -2.575200080871582, 'logqs/chosen': -0.8357488512992859, 'logqs_logps/rejected': -0.023725058883428574, 'logps_logqs/chosen': 0.20407584309577942, 'epoch': 1.88} 94%|█████████▍| 7060/7500 [4:01:07<15:02, 2.05s/it] 94%|█████████▍| 7061/7500 [4:01:09<14:59, 2.05s/it] 94%|█████████▍| 7062/7500 [4:01:12<14:53, 2.04s/it] 94%|█████████▍| 7063/7500 [4:01:14<14:53, 2.05s/it] 94%|█████████▍| 7064/7500 [4:01:16<14:55, 2.05s/it] 94%|█████████▍| 7065/7500 [4:01:18<14:53, 2.06s/it] 94%|█████████▍| 7066/7500 [4:01:20<14:54, 2.06s/it] 94%|█████████▍| 7067/7500 [4:01:22<14:50, 2.06s/it] 94%|█████████▍| 7068/7500 [4:01:24<14:52, 2.07s/it] 94%|█████████▍| 7069/7500 [4:01:26<14:50, 2.07s/it] 94%|█████████▍| 7070/7500 [4:01:28<14:53, 2.08s/it] {'loss': 0.3523, 'grad_norm': 2.8003876209259033, 'learning_rate': 4.989868414403048e-07, 'logqs/rejected': -2.546311855316162, 'logqs/chosen': -0.8440299034118652, 'logqs_logps/rejected': -0.023457959294319153, 'logps_logqs/chosen': 0.20464248955249786, 'epoch': 1.89} 94%|█████████▍| 7070/7500 [4:01:28<14:53, 2.08s/it] 94%|█████████▍| 7071/7500 [4:01:30<14:53, 2.08s/it] 94%|█████████▍| 7072/7500 [4:01:32<14:45, 2.07s/it] 94%|█████████▍| 7073/7500 [4:01:34<14:36, 2.05s/it] 94%|█████████▍| 7074/7500 [4:01:36<14:45, 2.08s/it] 94%|█████████▍| 7075/7500 [4:01:38<14:36, 2.06s/it] 94%|█████████▍| 7076/7500 [4:01:40<14:34, 2.06s/it] 94%|█████████▍| 7077/7500 [4:01:42<14:29, 2.06s/it] 94%|█████████▍| 7078/7500 [4:01:45<14:25, 2.05s/it] 94%|█████████▍| 7079/7500 [4:01:47<14:20, 2.04s/it] 94%|█████████▍| 7080/7500 [4:01:49<14:23, 2.06s/it] {'loss': 0.3601, 'grad_norm': 2.96708607673645, 'learning_rate': 4.7612111627021175e-07, 'logqs/rejected': -2.7156615257263184, 'logqs/chosen': -0.841122031211853, 'logqs_logps/rejected': -0.01881686970591545, 'logps_logqs/chosen': 0.20244666934013367, 'epoch': 1.89} 94%|█████████▍| 7080/7500 [4:01:49<14:23, 2.06s/it] 94%|█████████▍| 7081/7500 [4:01:51<14:26, 2.07s/it] 94%|█████████▍| 7082/7500 [4:01:53<14:25, 2.07s/it] 94%|█████████▍| 7083/7500 [4:01:55<14:23, 2.07s/it] 94%|█████████▍| 7084/7500 [4:01:57<14:14, 2.05s/it] 94%|█████████▍| 7085/7500 [4:01:59<14:19, 2.07s/it] 94%|█████████▍| 7086/7500 [4:02:01<14:13, 2.06s/it] 94%|█████████▍| 7087/7500 [4:02:03<14:12, 2.07s/it] 95%|█████████▍| 7088/7500 [4:02:05<14:05, 2.05s/it] 95%|█████████▍| 7089/7500 [4:02:07<13:55, 2.03s/it] 95%|█████████▍| 7090/7500 [4:02:09<13:54, 2.04s/it] {'loss': 0.3592, 'grad_norm': 2.6991002559661865, 'learning_rate': 4.537866186360207e-07, 'logqs/rejected': -2.6062099933624268, 'logqs/chosen': -0.835049033164978, 'logqs_logps/rejected': -0.019591491669416428, 'logps_logqs/chosen': 0.20873203873634338, 'epoch': 1.89} 95%|█████████▍| 7090/7500 [4:02:09<13:54, 2.04s/it] 95%|█████████▍| 7091/7500 [4:02:11<13:46, 2.02s/it] 95%|█████████▍| 7092/7500 [4:02:13<13:45, 2.02s/it] 95%|█████████▍| 7093/7500 [4:02:15<14:02, 2.07s/it] 95%|█████████▍| 7094/7500 [4:02:18<14:22, 2.12s/it] 95%|█████████▍| 7095/7500 [4:02:20<14:25, 2.14s/it] 95%|█████████▍| 7096/7500 [4:02:22<14:22, 2.14s/it] 95%|█████████▍| 7097/7500 [4:02:24<14:09, 2.11s/it] 95%|█████████▍| 7098/7500 [4:02:26<13:58, 2.08s/it] 95%|█████████▍| 7099/7500 [4:02:28<13:51, 2.07s/it] 95%|█████████▍| 7100/7500 [4:02:30<13:44, 2.06s/it] {'loss': 0.3574, 'grad_norm': 3.2839934825897217, 'learning_rate': 4.319838323396691e-07, 'logqs/rejected': -2.5980961322784424, 'logqs/chosen': -0.8660918474197388, 'logqs_logps/rejected': -0.029916837811470032, 'logps_logqs/chosen': 0.21170537173748016, 'epoch': 1.89} 95%|█████████▍| 7100/7500 [4:02:30<13:44, 2.06s/it] 95%|█████████▍| 7101/7500 [4:02:32<13:38, 2.05s/it] 95%|█████████▍| 7102/7500 [4:02:34<13:33, 2.05s/it] 95%|█████████▍| 7103/7500 [4:02:36<13:44, 2.08s/it] 95%|█████████▍| 7104/7500 [4:02:38<13:43, 2.08s/it] 95%|█████████▍| 7105/7500 [4:02:40<13:44, 2.09s/it] 95%|█████████▍| 7106/7500 [4:02:42<13:34, 2.07s/it] 95%|█████████▍| 7107/7500 [4:02:45<13:30, 2.06s/it] 95%|█████████▍| 7108/7500 [4:02:47<13:22, 2.05s/it] 95%|█████████▍| 7109/7500 [4:02:49<13:26, 2.06s/it] 95%|█████████▍| 7110/7500 [4:02:51<13:28, 2.07s/it] {'loss': 0.3549, 'grad_norm': 3.1481378078460693, 'learning_rate': 4.107132296653549e-07, 'logqs/rejected': -2.586765766143799, 'logqs/chosen': -0.85736083984375, 'logqs_logps/rejected': -0.027681168168783188, 'logps_logqs/chosen': 0.2044748067855835, 'epoch': 1.9} 95%|█████████▍| 7110/7500 [4:02:51<13:28, 2.07s/it] 95%|█████████▍| 7111/7500 [4:02:53<13:28, 2.08s/it] 95%|█████████▍| 7112/7500 [4:02:55<13:24, 2.07s/it] 95%|█████████▍| 7113/7500 [4:02:57<13:19, 2.07s/it] 95%|█████████▍| 7114/7500 [4:02:59<13:08, 2.04s/it] 95%|█████████▍| 7115/7500 [4:03:01<13:11, 2.05s/it] 95%|█████████▍| 7116/7500 [4:03:03<13:08, 2.05s/it] 95%|█████████▍| 7117/7500 [4:03:05<13:16, 2.08s/it] 95%|█████████▍| 7118/7500 [4:03:07<13:06, 2.06s/it] 95%|█████████▍| 7119/7500 [4:03:09<13:00, 2.05s/it] 95%|█████████▍| 7120/7500 [4:03:11<13:00, 2.05s/it] {'loss': 0.3444, 'grad_norm': 3.251033067703247, 'learning_rate': 3.899752713693e-07, 'logqs/rejected': -2.591370105743408, 'logqs/chosen': -0.82594233751297, 'logqs_logps/rejected': -0.02562524378299713, 'logps_logqs/chosen': 0.20112302899360657, 'epoch': 1.9} 95%|█████████▍| 7120/7500 [4:03:11<13:00, 2.05s/it] 95%|█████████▍| 7121/7500 [4:03:13<13:01, 2.06s/it] 95%|█████████▍| 7122/7500 [4:03:15<12:50, 2.04s/it] 95%|█████████▍| 7123/7500 [4:03:17<12:41, 2.02s/it] 95%|█████████▍| 7124/7500 [4:03:19<12:43, 2.03s/it] 95%|█████████▌| 7125/7500 [4:03:21<12:37, 2.02s/it] 95%|█████████▌| 7126/7500 [4:03:23<12:37, 2.02s/it] 95%|█████████▌| 7127/7500 [4:03:25<12:35, 2.02s/it] 95%|█████████▌| 7128/7500 [4:03:28<12:33, 2.03s/it] 95%|█████████▌| 7129/7500 [4:03:30<12:36, 2.04s/it] 95%|█████████▌| 7130/7500 [4:03:32<12:42, 2.06s/it] {'loss': 0.3658, 'grad_norm': 2.7394702434539795, 'learning_rate': 3.6977040666977547e-07, 'logqs/rejected': -2.636199474334717, 'logqs/chosen': -0.818893551826477, 'logqs_logps/rejected': -0.023332759737968445, 'logps_logqs/chosen': 0.20004022121429443, 'epoch': 1.9} 95%|█████████▌| 7130/7500 [4:03:32<12:42, 2.06s/it] 95%|█████████▌| 7131/7500 [4:03:34<12:48, 2.08s/it] 95%|█████████▌| 7132/7500 [4:03:36<12:49, 2.09s/it] 95%|█████████▌| 7133/7500 [4:03:38<12:49, 2.10s/it] 95%|█████████▌| 7134/7500 [4:03:40<12:44, 2.09s/it] 95%|█████████▌| 7135/7500 [4:03:42<12:52, 2.12s/it] 95%|█████████▌| 7136/7500 [4:03:44<12:44, 2.10s/it] 95%|█████████▌| 7137/7500 [4:03:46<12:40, 2.10s/it] 95%|█████████▌| 7138/7500 [4:03:49<12:43, 2.11s/it] 95%|█████████▌| 7139/7500 [4:03:51<12:37, 2.10s/it] 95%|█████████▌| 7140/7500 [4:03:53<12:28, 2.08s/it] {'loss': 0.3498, 'grad_norm': 2.6289329528808594, 'learning_rate': 3.5009907323737825e-07, 'logqs/rejected': -2.4020347595214844, 'logqs/chosen': -0.8296673893928528, 'logqs_logps/rejected': -0.028335874900221825, 'logps_logqs/chosen': 0.20929677784442902, 'epoch': 1.9} 95%|█████████▌| 7140/7500 [4:03:53<12:28, 2.08s/it] 95%|█████████▌| 7141/7500 [4:03:55<12:23, 2.07s/it] 95%|█████████▌| 7142/7500 [4:03:57<12:20, 2.07s/it] 95%|█████████▌| 7143/7500 [4:03:59<12:10, 2.05s/it] 95%|█████████▌| 7144/7500 [4:04:01<12:14, 2.06s/it] 95%|█████████▌| 7145/7500 [4:04:03<12:13, 2.07s/it] 95%|█████████▌| 7146/7500 [4:04:05<12:11, 2.07s/it] 95%|█████████▌| 7147/7500 [4:04:07<12:10, 2.07s/it] 95%|█████████▌| 7148/7500 [4:04:09<12:04, 2.06s/it] 95%|█████████▌| 7149/7500 [4:04:11<12:05, 2.07s/it] 95%|█████████▌| 7150/7500 [4:04:13<12:02, 2.07s/it] {'loss': 0.3518, 'grad_norm': 2.907613754272461, 'learning_rate': 3.309616971855195e-07, 'logqs/rejected': -2.6365270614624023, 'logqs/chosen': -0.8508655428886414, 'logqs_logps/rejected': -0.02503938041627407, 'logps_logqs/chosen': 0.20684058964252472, 'epoch': 1.91} 95%|█████████▌| 7150/7500 [4:04:13<12:02, 2.07s/it] 95%|█████████▌| 7151/7500 [4:04:15<12:02, 2.07s/it] 95%|█████████▌| 7152/7500 [4:04:17<11:51, 2.04s/it] 95%|█████████▌| 7153/7500 [4:04:19<11:56, 2.07s/it] 95%|█████████▌| 7154/7500 [4:04:22<11:57, 2.07s/it] 95%|█████████▌| 7155/7500 [4:04:24<12:01, 2.09s/it] 95%|█████████▌| 7156/7500 [4:04:26<11:57, 2.09s/it] 95%|█████████▌| 7157/7500 [4:04:28<11:55, 2.08s/it] 95%|█████████▌| 7158/7500 [4:04:30<12:00, 2.11s/it] 95%|█████████▌| 7159/7500 [4:04:32<11:50, 2.08s/it] 95%|█████████▌| 7160/7500 [4:04:34<11:43, 2.07s/it] {'loss': 0.3407, 'grad_norm': 3.338545322418213, 'learning_rate': 3.1235869306123767e-07, 'logqs/rejected': -2.5299324989318848, 'logqs/chosen': -0.8286073803901672, 'logqs_logps/rejected': -0.027269786223769188, 'logps_logqs/chosen': 0.20164187252521515, 'epoch': 1.91} 95%|█████████▌| 7160/7500 [4:04:34<11:43, 2.07s/it] 95%|█████████▌| 7161/7500 [4:04:36<11:35, 2.05s/it] 95%|█████████▌| 7162/7500 [4:04:38<11:39, 2.07s/it] 96%|█████████▌| 7163/7500 [4:04:40<11:36, 2.07s/it] 96%|█████████▌| 7164/7500 [4:04:42<11:32, 2.06s/it] 96%|█████████▌| 7165/7500 [4:04:44<11:30, 2.06s/it] 96%|█████████▌| 7166/7500 [4:04:46<11:22, 2.04s/it] 96%|█████████▌| 7167/7500 [4:04:48<11:17, 2.03s/it] 96%|█████████▌| 7168/7500 [4:04:50<11:17, 2.04s/it] 96%|█████████▌| 7169/7500 [4:04:52<11:17, 2.05s/it] 96%|█████████▌| 7170/7500 [4:04:55<11:20, 2.06s/it] {'loss': 0.3413, 'grad_norm': 2.584813117980957, 'learning_rate': 2.942904638361804e-07, 'logqs/rejected': -2.5809826850891113, 'logqs/chosen': -0.8256394267082214, 'logqs_logps/rejected': -0.026604752987623215, 'logps_logqs/chosen': 0.20325715839862823, 'epoch': 1.91} 96%|█████████▌| 7170/7500 [4:04:55<11:20, 2.06s/it] 96%|█████████▌| 7171/7500 [4:04:57<11:24, 2.08s/it] 96%|█████████▌| 7172/7500 [4:04:59<11:17, 2.07s/it] 96%|█████████▌| 7173/7500 [4:05:01<11:09, 2.05s/it] 96%|█████████▌| 7174/7500 [4:05:03<11:15, 2.07s/it] 96%|█████████▌| 7175/7500 [4:05:05<11:20, 2.09s/it] 96%|█████████▌| 7176/7500 [4:05:07<11:18, 2.10s/it] 96%|█████████▌| 7177/7500 [4:05:09<11:25, 2.12s/it] 96%|█████████▌| 7178/7500 [4:05:11<11:22, 2.12s/it] 96%|█████████▌| 7179/7500 [4:05:14<11:18, 2.11s/it] 96%|█████████▌| 7180/7500 [4:05:16<11:10, 2.10s/it] {'loss': 0.361, 'grad_norm': 2.652116060256958, 'learning_rate': 2.767574008979007e-07, 'logqs/rejected': -2.6018099784851074, 'logqs/chosen': -0.8561531901359558, 'logqs_logps/rejected': -0.021664796397089958, 'logps_logqs/chosen': 0.20912079513072968, 'epoch': 1.91} 96%|█████████▌| 7180/7500 [4:05:16<11:10, 2.10s/it] 96%|█████████▌| 7181/7500 [4:05:18<11:08, 2.10s/it] 96%|█████████▌| 7182/7500 [4:05:20<11:00, 2.08s/it] 96%|█████████▌| 7183/7500 [4:05:22<11:04, 2.10s/it] 96%|█████████▌| 7184/7500 [4:05:24<10:59, 2.09s/it] 96%|█████████▌| 7185/7500 [4:05:26<10:53, 2.07s/it] 96%|█████████▌| 7186/7500 [4:05:28<10:47, 2.06s/it] 96%|█████████▌| 7187/7500 [4:05:30<10:40, 2.04s/it] 96%|█████████▌| 7188/7500 [4:05:32<10:38, 2.05s/it] 96%|█████████▌| 7189/7500 [4:05:34<10:37, 2.05s/it] 96%|█████████▌| 7190/7500 [4:05:36<10:44, 2.08s/it] {'loss': 0.3492, 'grad_norm': 2.619626045227051, 'learning_rate': 2.59759884041369e-07, 'logqs/rejected': -2.3645012378692627, 'logqs/chosen': -0.8328851461410522, 'logqs_logps/rejected': -0.023886162787675858, 'logps_logqs/chosen': 0.20007523894309998, 'epoch': 1.92} 96%|█████████▌| 7190/7500 [4:05:36<10:44, 2.08s/it] 96%|█████████▌| 7191/7500 [4:05:38<10:37, 2.06s/it] 96%|█████████▌| 7192/7500 [4:05:40<10:32, 2.05s/it] 96%|█████████▌| 7193/7500 [4:05:42<10:27, 2.04s/it] 96%|█████████▌| 7194/7500 [4:05:44<10:29, 2.06s/it] 96%|█████████▌| 7195/7500 [4:05:46<10:20, 2.03s/it] 96%|█████████▌| 7196/7500 [4:05:48<10:19, 2.04s/it] 96%|█████████▌| 7197/7500 [4:05:50<10:13, 2.03s/it] 96%|█████████▌| 7198/7500 [4:05:53<10:15, 2.04s/it] 96%|█████████▌| 7199/7500 [4:05:55<10:24, 2.07s/it] 96%|█████████▌| 7200/7500 [4:05:57<10:35, 2.12s/it] {'loss': 0.3476, 'grad_norm': 3.318495273590088, 'learning_rate': 2.4329828146074095e-07, 'logqs/rejected': -2.4613444805145264, 'logqs/chosen': -0.8286117315292358, 'logqs_logps/rejected': -0.02172081358730793, 'logps_logqs/chosen': 0.20828139781951904, 'epoch': 1.92} 96%|█████████▌| 7200/7500 [4:05:57<10:35, 2.12s/it] 96%|█████████▌| 7201/7500 [4:05:59<10:24, 2.09s/it] 96%|█████████▌| 7202/7500 [4:06:01<10:16, 2.07s/it] 96%|█████████▌| 7203/7500 [4:06:03<10:11, 2.06s/it] 96%|█████████▌| 7204/7500 [4:06:05<10:17, 2.09s/it] 96%|█████████▌| 7205/7500 [4:06:07<10:13, 2.08s/it] 96%|█████████▌| 7206/7500 [4:06:09<10:15, 2.10s/it] 96%|█████████▌| 7207/7500 [4:06:11<10:13, 2.10s/it] 96%|█████████▌| 7208/7500 [4:06:13<10:10, 2.09s/it] 96%|█████████▌| 7209/7500 [4:06:16<10:02, 2.07s/it] 96%|█████████▌| 7210/7500 [4:06:18<09:58, 2.06s/it] {'loss': 0.3721, 'grad_norm': 2.597769021987915, 'learning_rate': 2.273729497414001e-07, 'logqs/rejected': -2.571840763092041, 'logqs/chosen': -0.8503929376602173, 'logqs_logps/rejected': -0.020224621519446373, 'logps_logqs/chosen': 0.19991858303546906, 'epoch': 1.92} 96%|█████████▌| 7210/7500 [4:06:18<09:58, 2.06s/it] 96%|█████████▌| 7211/7500 [4:06:20<09:51, 2.05s/it] 96%|█████████▌| 7212/7500 [4:06:22<09:53, 2.06s/it] 96%|█████████▌| 7213/7500 [4:06:24<09:54, 2.07s/it] 96%|█████████▌| 7214/7500 [4:06:26<09:50, 2.07s/it] 96%|█████████▌| 7215/7500 [4:06:28<09:52, 2.08s/it] 96%|█████████▌| 7216/7500 [4:06:30<09:54, 2.09s/it] 96%|█████████▌| 7217/7500 [4:06:32<09:48, 2.08s/it] 96%|█████████▌| 7218/7500 [4:06:34<09:44, 2.07s/it] 96%|█████████▋| 7219/7500 [4:06:36<09:34, 2.05s/it] 96%|█████████▋| 7220/7500 [4:06:38<09:30, 2.04s/it] {'loss': 0.3646, 'grad_norm': 3.2398462295532227, 'learning_rate': 2.1198423385220824e-07, 'logqs/rejected': -2.5456244945526123, 'logqs/chosen': -0.8215928077697754, 'logqs_logps/rejected': -0.023322874680161476, 'logps_logqs/chosen': 0.20126767456531525, 'epoch': 1.93} 96%|█████████▋| 7220/7500 [4:06:38<09:30, 2.04s/it] 96%|█████████▋| 7221/7500 [4:06:40<09:36, 2.07s/it] 96%|█████████▋| 7222/7500 [4:06:42<09:37, 2.08s/it] 96%|█████████▋| 7223/7500 [4:06:44<09:34, 2.07s/it] 96%|█████████▋| 7224/7500 [4:06:47<09:33, 2.08s/it] 96%|█████████▋| 7225/7500 [4:06:49<09:27, 2.07s/it] 96%|█████████▋| 7226/7500 [4:06:51<09:19, 2.04s/it] 96%|█████████▋| 7227/7500 [4:06:53<09:16, 2.04s/it] 96%|█████████▋| 7228/7500 [4:06:55<09:22, 2.07s/it] 96%|█████████▋| 7229/7500 [4:06:57<09:11, 2.03s/it] 96%|█████████▋| 7230/7500 [4:06:59<09:19, 2.07s/it] {'loss': 0.3515, 'grad_norm': 2.362633466720581, 'learning_rate': 1.9713246713805588e-07, 'logqs/rejected': -2.6171834468841553, 'logqs/chosen': -0.8534365892410278, 'logqs_logps/rejected': -0.026874834671616554, 'logps_logqs/chosen': 0.204123854637146, 'epoch': 1.93} 96%|█████████▋| 7230/7500 [4:06:59<09:19, 2.07s/it] 96%|█████████▋| 7231/7500 [4:07:01<09:14, 2.06s/it] 96%|█████████▋| 7232/7500 [4:07:03<09:13, 2.06s/it] 96%|█████████▋| 7233/7500 [4:07:05<09:04, 2.04s/it] 96%|█████████▋| 7234/7500 [4:07:07<09:12, 2.08s/it] 96%|█████████▋| 7235/7500 [4:07:09<09:09, 2.07s/it] 96%|█████████▋| 7236/7500 [4:07:11<09:12, 2.09s/it] 96%|█████████▋| 7237/7500 [4:07:13<09:11, 2.10s/it] 97%|█████████▋| 7238/7500 [4:07:16<09:15, 2.12s/it] 97%|█████████▋| 7239/7500 [4:07:18<09:12, 2.12s/it] 97%|█████████▋| 7240/7500 [4:07:20<09:15, 2.14s/it] {'loss': 0.3642, 'grad_norm': 3.2491393089294434, 'learning_rate': 1.82817971312621e-07, 'logqs/rejected': -2.610335350036621, 'logqs/chosen': -0.8472406268119812, 'logqs_logps/rejected': -0.02477775700390339, 'logps_logqs/chosen': 0.20479103922843933, 'epoch': 1.93} 97%|█████████▋| 7240/7500 [4:07:20<09:15, 2.14s/it] 97%|█████████▋| 7241/7500 [4:07:22<09:06, 2.11s/it] 97%|█████████▋| 7242/7500 [4:07:24<09:07, 2.12s/it] 97%|█████████▋| 7243/7500 [4:07:26<09:10, 2.14s/it] 97%|█████████▋| 7244/7500 [4:07:28<09:08, 2.14s/it] 97%|█████████▋| 7245/7500 [4:07:30<09:01, 2.12s/it] 97%|█████████▋| 7246/7500 [4:07:33<08:55, 2.11s/it] 97%|█████████▋| 7247/7500 [4:07:35<08:44, 2.07s/it] 97%|█████████▋| 7248/7500 [4:07:37<08:41, 2.07s/it] 97%|█████████▋| 7249/7500 [4:07:39<08:34, 2.05s/it] 97%|█████████▋| 7250/7500 [4:07:41<08:40, 2.08s/it] {'loss': 0.3432, 'grad_norm': 2.6661369800567627, 'learning_rate': 1.6904105645142444e-07, 'logqs/rejected': -2.517651319503784, 'logqs/chosen': -0.8413822054862976, 'logqs_logps/rejected': -0.02934342250227928, 'logps_logqs/chosen': 0.19630104303359985, 'epoch': 1.93} 97%|█████████▋| 7250/7500 [4:07:41<08:40, 2.08s/it] 97%|█████████▋| 7251/7500 [4:07:43<08:37, 2.08s/it] 97%|█████████▋| 7252/7500 [4:07:45<08:35, 2.08s/it] 97%|█████████▋| 7253/7500 [4:07:47<08:31, 2.07s/it] 97%|█████████▋| 7254/7500 [4:07:49<08:34, 2.09s/it] 97%|█████████▋| 7255/7500 [4:07:51<08:31, 2.09s/it] 97%|█████████▋| 7256/7500 [4:07:53<08:26, 2.08s/it] 97%|█████████▋| 7257/7500 [4:07:55<08:20, 2.06s/it] 97%|█████████▋| 7258/7500 [4:07:57<08:16, 2.05s/it] 97%|█████████▋| 7259/7500 [4:07:59<08:12, 2.05s/it] 97%|█████████▋| 7260/7500 [4:08:01<08:13, 2.06s/it] {'loss': 0.3631, 'grad_norm': 2.7273757457733154, 'learning_rate': 1.5580202098509077e-07, 'logqs/rejected': -2.6096720695495605, 'logqs/chosen': -0.8592588305473328, 'logqs_logps/rejected': -0.023503649979829788, 'logps_logqs/chosen': 0.20289091765880585, 'epoch': 1.94} 97%|█████████▋| 7260/7500 [4:08:01<08:13, 2.06s/it] 97%|█████████▋| 7261/7500 [4:08:03<08:10, 2.05s/it] 97%|█████████▋| 7262/7500 [4:08:05<08:06, 2.04s/it] 97%|█████████▋| 7263/7500 [4:08:07<08:00, 2.03s/it] 97%|█████████▋| 7264/7500 [4:08:10<08:10, 2.08s/it] 97%|█████████▋| 7265/7500 [4:08:12<08:05, 2.06s/it] 97%|█████████▋| 7266/7500 [4:08:14<08:00, 2.05s/it] 97%|█████████▋| 7267/7500 [4:08:16<07:55, 2.04s/it] 97%|█████████▋| 7268/7500 [4:08:18<08:03, 2.08s/it] 97%|█████████▋| 7269/7500 [4:08:20<08:04, 2.10s/it] 97%|█████████▋| 7270/7500 [4:08:22<08:06, 2.12s/it] {'loss': 0.348, 'grad_norm': 2.663740634918213, 'learning_rate': 1.4310115169289262e-07, 'logqs/rejected': -2.5790345668792725, 'logqs/chosen': -0.8187144994735718, 'logqs_logps/rejected': -0.024687010794878006, 'logps_logqs/chosen': 0.20058664679527283, 'epoch': 1.94} 97%|█████████▋| 7270/7500 [4:08:22<08:06, 2.12s/it] 97%|█████████▋| 7271/7500 [4:08:24<08:02, 2.11s/it] 97%|█████████▋| 7272/7500 [4:08:26<08:03, 2.12s/it] 97%|█████████▋| 7273/7500 [4:08:29<08:02, 2.13s/it] 97%|█████████▋| 7274/7500 [4:08:31<07:52, 2.09s/it] 97%|█████████▋| 7275/7500 [4:08:33<07:48, 2.08s/it] 97%|█████████▋| 7276/7500 [4:08:35<07:52, 2.11s/it] 97%|█████████▋| 7277/7500 [4:08:37<07:50, 2.11s/it] 97%|█████████▋| 7278/7500 [4:08:39<07:49, 2.12s/it] 97%|█████████▋| 7279/7500 [4:08:41<07:45, 2.11s/it] 97%|█████████▋| 7280/7500 [4:08:43<07:41, 2.10s/it] {'loss': 0.3637, 'grad_norm': 2.4142415523529053, 'learning_rate': 1.3093872369654148e-07, 'logqs/rejected': -2.663745880126953, 'logqs/chosen': -0.8501392602920532, 'logqs_logps/rejected': -0.023850474506616592, 'logps_logqs/chosen': 0.20732983946800232, 'epoch': 1.94} 97%|█████████▋| 7280/7500 [4:08:43<07:41, 2.10s/it] 97%|█████████▋| 7281/7500 [4:08:45<07:31, 2.06s/it] 97%|█████████▋| 7282/7500 [4:08:47<07:32, 2.07s/it] 97%|█████████▋| 7283/7500 [4:08:49<07:27, 2.06s/it] 97%|█████████▋| 7284/7500 [4:08:51<07:21, 2.04s/it] 97%|█████████▋| 7285/7500 [4:08:53<07:21, 2.05s/it] 97%|█████████▋| 7286/7500 [4:08:56<07:22, 2.07s/it] 97%|█████████▋| 7287/7500 [4:08:58<07:20, 2.07s/it] 97%|█████████▋| 7288/7500 [4:09:00<07:19, 2.07s/it] 97%|█████████▋| 7289/7500 [4:09:02<07:15, 2.06s/it] 97%|█████████▋| 7290/7500 [4:09:04<07:16, 2.08s/it] {'loss': 0.3527, 'grad_norm': 2.8647565841674805, 'learning_rate': 1.193150004542204e-07, 'logqs/rejected': -2.618847370147705, 'logqs/chosen': -0.8404722213745117, 'logqs_logps/rejected': -0.026462476700544357, 'logps_logqs/chosen': 0.2071470469236374, 'epoch': 1.94} 97%|█████████▋| 7290/7500 [4:09:04<07:16, 2.08s/it] 97%|█████████▋| 7291/7500 [4:09:06<07:13, 2.08s/it] 97%|█████████▋| 7292/7500 [4:09:08<07:13, 2.08s/it] 97%|█████████▋| 7293/7500 [4:09:10<07:10, 2.08s/it] 97%|█████████▋| 7294/7500 [4:09:12<07:06, 2.07s/it] 97%|█████████▋| 7295/7500 [4:09:14<06:59, 2.05s/it] 97%|█████████▋| 7296/7500 [4:09:16<06:56, 2.04s/it] 97%|█████████▋| 7297/7500 [4:09:18<06:52, 2.03s/it] 97%|█████████▋| 7298/7500 [4:09:20<06:57, 2.07s/it] 97%|█████████▋| 7299/7500 [4:09:22<06:58, 2.08s/it] 97%|█████████▋| 7300/7500 [4:09:24<06:54, 2.07s/it] {'loss': 0.3548, 'grad_norm': 2.5057456493377686, 'learning_rate': 1.0823023375489127e-07, 'logqs/rejected': -2.444093704223633, 'logqs/chosen': -0.832266628742218, 'logqs_logps/rejected': -0.02650131843984127, 'logps_logqs/chosen': 0.20066168904304504, 'epoch': 1.95} 97%|█████████▋| 7300/7500 [4:09:24<06:54, 2.07s/it] 97%|█████████▋| 7301/7500 [4:09:27<06:53, 2.08s/it] 97%|█████████▋| 7302/7500 [4:09:29<06:49, 2.07s/it] 97%|█████████▋| 7303/7500 [4:09:31<06:49, 2.08s/it] 97%|█████████▋| 7304/7500 [4:09:33<06:43, 2.06s/it] 97%|█████████▋| 7305/7500 [4:09:35<06:44, 2.07s/it] 97%|█████████▋| 7306/7500 [4:09:37<06:42, 2.07s/it] 97%|█████████▋| 7307/7500 [4:09:39<06:42, 2.09s/it] 97%|█████████▋| 7308/7500 [4:09:41<06:45, 2.11s/it] 97%|█████████▋| 7309/7500 [4:09:43<06:46, 2.13s/it] 97%|█████████▋| 7310/7500 [4:09:45<06:43, 2.13s/it] {'loss': 0.3488, 'grad_norm': 2.6776676177978516, 'learning_rate': 9.76846637128187e-08, 'logqs/rejected': -2.425741672515869, 'logqs/chosen': -0.8658267259597778, 'logqs_logps/rejected': -0.031788475811481476, 'logps_logqs/chosen': 0.2068479061126709, 'epoch': 1.95} 97%|█████████▋| 7310/7500 [4:09:45<06:43, 2.13s/it] 97%|█████████▋| 7311/7500 [4:09:48<06:40, 2.12s/it] 97%|█████████▋| 7312/7500 [4:09:50<06:39, 2.12s/it] 98%|█████████▊| 7313/7500 [4:09:52<06:33, 2.10s/it] 98%|█████████▊| 7314/7500 [4:09:54<06:29, 2.10s/it] 98%|█████████▊| 7315/7500 [4:09:56<06:23, 2.07s/it] 98%|█████████▊| 7316/7500 [4:09:58<06:20, 2.07s/it] 98%|█████████▊| 7317/7500 [4:10:00<06:16, 2.06s/it] 98%|█████████▊| 7318/7500 [4:10:02<06:10, 2.04s/it] 98%|█████████▊| 7319/7500 [4:10:04<06:09, 2.04s/it] 98%|█████████▊| 7320/7500 [4:10:06<06:03, 2.02s/it] {'loss': 0.3568, 'grad_norm': 3.730656862258911, 'learning_rate': 8.767851876239074e-08, 'logqs/rejected': -2.6169238090515137, 'logqs/chosen': -0.8420189023017883, 'logqs_logps/rejected': -0.026789048686623573, 'logps_logqs/chosen': 0.20565783977508545, 'epoch': 1.95} 98%|█████████▊| 7320/7500 [4:10:06<06:03, 2.02s/it] 98%|█████████▊| 7321/7500 [4:10:08<06:04, 2.03s/it] 98%|█████████▊| 7322/7500 [4:10:10<06:10, 2.08s/it] 98%|█████████▊| 7323/7500 [4:10:12<06:07, 2.08s/it] 98%|█████████▊| 7324/7500 [4:10:14<06:05, 2.07s/it] 98%|█████████▊| 7325/7500 [4:10:16<05:57, 2.04s/it] 98%|█████████▊| 7326/7500 [4:10:18<05:55, 2.05s/it] 98%|█████████▊| 7327/7500 [4:10:20<05:54, 2.05s/it] 98%|█████████▊| 7328/7500 [4:10:22<05:52, 2.05s/it] 98%|█████████▊| 7329/7500 [4:10:25<05:54, 2.07s/it] 98%|█████████▊| 7330/7500 [4:10:27<05:47, 2.05s/it] {'loss': 0.364, 'grad_norm': 3.0098934173583984, 'learning_rate': 7.821201565316182e-08, 'logqs/rejected': -2.853797435760498, 'logqs/chosen': -0.8524587750434875, 'logqs_logps/rejected': -0.023991364985704422, 'logps_logqs/chosen': 0.20508842170238495, 'epoch': 1.95} 98%|█████████▊| 7330/7500 [4:10:27<05:47, 2.05s/it] 98%|█████████▊| 7331/7500 [4:10:29<05:45, 2.04s/it] 98%|█████████▊| 7332/7500 [4:10:31<05:39, 2.02s/it] 98%|█████████▊| 7333/7500 [4:10:33<05:36, 2.02s/it] 98%|█████████▊| 7334/7500 [4:10:35<05:40, 2.05s/it] 98%|█████████▊| 7335/7500 [4:10:37<05:37, 2.05s/it] 98%|█████████▊| 7336/7500 [4:10:39<05:44, 2.10s/it] 98%|█████████▊| 7337/7500 [4:10:41<05:38, 2.08s/it] 98%|█████████▊| 7338/7500 [4:10:43<05:35, 2.07s/it] 98%|█████████▊| 7339/7500 [4:10:45<05:33, 2.07s/it] 98%|█████████▊| 7340/7500 [4:10:47<05:34, 2.09s/it] {'loss': 0.358, 'grad_norm': 3.5447309017181396, 'learning_rate': 6.928535944514536e-08, 'logqs/rejected': -2.6725151538848877, 'logqs/chosen': -0.8317620158195496, 'logqs_logps/rejected': -0.01607072353363037, 'logps_logqs/chosen': 0.20355287194252014, 'epoch': 1.96} 98%|█████████▊| 7340/7500 [4:10:47<05:34, 2.09s/it] 98%|█████████▊| 7341/7500 [4:10:49<05:35, 2.11s/it] 98%|█████████▊| 7342/7500 [4:10:52<05:39, 2.15s/it] 98%|█████████▊| 7343/7500 [4:10:54<05:37, 2.15s/it] 98%|█████████▊| 7344/7500 [4:10:56<05:31, 2.13s/it] 98%|█████████▊| 7345/7500 [4:10:58<05:23, 2.09s/it] 98%|█████████▊| 7346/7500 [4:11:00<05:22, 2.09s/it] 98%|█████████▊| 7347/7500 [4:11:02<05:17, 2.08s/it] 98%|█████████▊| 7348/7500 [4:11:04<05:17, 2.09s/it] 98%|█████████▊| 7349/7500 [4:11:06<05:13, 2.08s/it] 98%|█████████▊| 7350/7500 [4:11:08<05:13, 2.09s/it] {'loss': 0.3533, 'grad_norm': 2.620029926300049, 'learning_rate': 6.089874350439506e-08, 'logqs/rejected': -2.5540919303894043, 'logqs/chosen': -0.869467556476593, 'logqs_logps/rejected': -0.029297953471541405, 'logps_logqs/chosen': 0.20843103528022766, 'epoch': 1.96} 98%|█████████▊| 7350/7500 [4:11:08<05:13, 2.09s/it] 98%|█████████▊| 7351/7500 [4:11:10<05:09, 2.08s/it] 98%|█████████▊| 7352/7500 [4:11:12<05:02, 2.04s/it] 98%|█████████▊| 7353/7500 [4:11:14<05:00, 2.04s/it] 98%|█████████▊| 7354/7500 [4:11:16<05:01, 2.06s/it] 98%|█████████▊| 7355/7500 [4:11:19<05:01, 2.08s/it] 98%|█████████▊| 7356/7500 [4:11:21<05:02, 2.10s/it] 98%|█████████▊| 7357/7500 [4:11:23<05:06, 2.14s/it] 98%|█████████▊| 7358/7500 [4:11:25<05:01, 2.12s/it] 98%|█████████▊| 7359/7500 [4:11:27<04:55, 2.10s/it] 98%|█████████▊| 7360/7500 [4:11:29<04:54, 2.11s/it] {'loss': 0.3637, 'grad_norm': 3.0414681434631348, 'learning_rate': 5.30523494988e-08, 'logqs/rejected': -2.545006275177002, 'logqs/chosen': -0.8295138478279114, 'logqs_logps/rejected': -0.020073426887392998, 'logps_logqs/chosen': 0.20030391216278076, 'epoch': 1.96} 98%|█████████▊| 7360/7500 [4:11:29<04:54, 2.11s/it] 98%|█████████▊| 7361/7500 [4:11:31<04:50, 2.09s/it] 98%|█████████▊| 7362/7500 [4:11:33<04:51, 2.11s/it] 98%|█████████▊| 7363/7500 [4:11:36<04:47, 2.10s/it] 98%|█████████▊| 7364/7500 [4:11:38<04:47, 2.11s/it] 98%|█████████▊| 7365/7500 [4:11:40<04:42, 2.09s/it] 98%|█████████▊| 7366/7500 [4:11:42<04:39, 2.09s/it] 98%|█████████▊| 7367/7500 [4:11:44<04:36, 2.08s/it] 98%|█████████▊| 7368/7500 [4:11:46<04:36, 2.09s/it] 98%|█████████▊| 7369/7500 [4:11:48<04:36, 2.11s/it] 98%|█████████▊| 7370/7500 [4:11:50<04:31, 2.09s/it] {'loss': 0.3392, 'grad_norm': 2.5601184368133545, 'learning_rate': 4.57463473941544e-08, 'logqs/rejected': -2.4019713401794434, 'logqs/chosen': -0.844192385673523, 'logqs_logps/rejected': -0.03015809692442417, 'logps_logqs/chosen': 0.2046830654144287, 'epoch': 1.97} 98%|█████████▊| 7370/7500 [4:11:50<04:31, 2.09s/it] 98%|█████████▊| 7371/7500 [4:11:52<04:31, 2.10s/it] 98%|█████████▊| 7372/7500 [4:11:54<04:25, 2.07s/it] 98%|█████████▊| 7373/7500 [4:11:56<04:24, 2.08s/it] 98%|█████████▊| 7374/7500 [4:11:58<04:22, 2.08s/it] 98%|█████████▊| 7375/7500 [4:12:01<04:19, 2.07s/it] 98%|█████████▊| 7376/7500 [4:12:03<04:14, 2.06s/it] 98%|█████████▊| 7377/7500 [4:12:05<04:15, 2.08s/it] 98%|█████████▊| 7378/7500 [4:12:07<04:10, 2.05s/it] 98%|█████████▊| 7379/7500 [4:12:09<04:07, 2.04s/it] 98%|█████████▊| 7380/7500 [4:12:11<04:05, 2.05s/it] {'loss': 0.3672, 'grad_norm': 4.088193416595459, 'learning_rate': 3.8980895450474455e-08, 'logqs/rejected': -2.518828868865967, 'logqs/chosen': -0.8752312660217285, 'logqs_logps/rejected': -0.0181075818836689, 'logps_logqs/chosen': 0.20669560134410858, 'epoch': 1.97} 98%|█████████▊| 7380/7500 [4:12:11<04:05, 2.05s/it] 98%|█████████▊| 7381/7500 [4:12:13<04:03, 2.05s/it] 98%|█████████▊| 7382/7500 [4:12:15<03:59, 2.03s/it] 98%|█████████▊| 7383/7500 [4:12:17<04:04, 2.09s/it] 98%|█████████▊| 7384/7500 [4:12:19<04:05, 2.12s/it] 98%|█████████▊| 7385/7500 [4:12:21<04:02, 2.11s/it] 98%|█████████▊| 7386/7500 [4:12:23<03:58, 2.10s/it] 98%|█████████▊| 7387/7500 [4:12:25<03:56, 2.09s/it] 99%|█████████▊| 7388/7500 [4:12:27<03:53, 2.08s/it] 99%|█████████▊| 7389/7500 [4:12:30<03:49, 2.07s/it] 99%|█████████▊| 7390/7500 [4:12:32<03:49, 2.09s/it] {'loss': 0.3522, 'grad_norm': 4.071069240570068, 'learning_rate': 3.2756140218576095e-08, 'logqs/rejected': -2.518167495727539, 'logqs/chosen': -0.858319878578186, 'logqs_logps/rejected': -0.026683662086725235, 'logps_logqs/chosen': 0.20793524384498596, 'epoch': 1.97} 99%|█████████▊| 7390/7500 [4:12:32<03:49, 2.09s/it] 99%|█████████▊| 7391/7500 [4:12:34<03:47, 2.08s/it] 99%|█████████▊| 7392/7500 [4:12:36<03:42, 2.06s/it] 99%|█████████▊| 7393/7500 [4:12:38<03:42, 2.08s/it] 99%|█████████▊| 7394/7500 [4:12:40<03:38, 2.06s/it] 99%|█████████▊| 7395/7500 [4:12:42<03:37, 2.07s/it] 99%|█████████▊| 7396/7500 [4:12:44<03:34, 2.06s/it] 99%|█████████▊| 7397/7500 [4:12:46<03:30, 2.05s/it] 99%|█████████▊| 7398/7500 [4:12:48<03:30, 2.06s/it] 99%|█████████▊| 7399/7500 [4:12:50<03:25, 2.04s/it] 99%|█████████▊| 7400/7500 [4:12:52<03:25, 2.05s/it] {'loss': 0.3568, 'grad_norm': 2.76471209526062, 'learning_rate': 2.7072216536885853e-08, 'logqs/rejected': -2.6048502922058105, 'logqs/chosen': -0.8243298530578613, 'logqs_logps/rejected': -0.020602483302354813, 'logps_logqs/chosen': 0.19735369086265564, 'epoch': 1.97} 99%|█████████▊| 7400/7500 [4:12:52<03:25, 2.05s/it] 99%|█████████▊| 7401/7500 [4:12:54<03:26, 2.08s/it] 99%|█████████▊| 7402/7500 [4:12:56<03:23, 2.08s/it] 99%|█████████▊| 7403/7500 [4:12:59<03:22, 2.09s/it] 99%|█████████▊| 7404/7500 [4:13:01<03:21, 2.09s/it] 99%|█████████▊| 7405/7500 [4:13:03<03:18, 2.09s/it] 99%|█████████▊| 7406/7500 [4:13:05<03:15, 2.08s/it] 99%|█████████▉| 7407/7500 [4:13:07<03:14, 2.09s/it] 99%|█████████▉| 7408/7500 [4:13:09<03:11, 2.08s/it] 99%|█████████▉| 7409/7500 [4:13:11<03:08, 2.07s/it] 99%|█████████▉| 7410/7500 [4:13:13<03:09, 2.10s/it] {'loss': 0.351, 'grad_norm': 2.2708535194396973, 'learning_rate': 2.192924752854042e-08, 'logqs/rejected': -2.272839069366455, 'logqs/chosen': -0.8689607381820679, 'logqs_logps/rejected': -0.025569159537553787, 'logps_logqs/chosen': 0.207604318857193, 'epoch': 1.98} 99%|█████████▉| 7410/7500 [4:13:13<03:09, 2.10s/it] 99%|█████████▉| 7411/7500 [4:13:15<03:06, 2.10s/it] 99%|█████████▉| 7412/7500 [4:13:17<03:05, 2.10s/it] 99%|█████████▉| 7413/7500 [4:13:19<03:03, 2.11s/it] 99%|█████████▉| 7414/7500 [4:13:22<03:01, 2.11s/it] 99%|█████████▉| 7415/7500 [4:13:24<02:58, 2.10s/it] 99%|█████████▉| 7416/7500 [4:13:26<02:57, 2.12s/it] 99%|█████████▉| 7417/7500 [4:13:28<02:53, 2.09s/it] 99%|█████████▉| 7418/7500 [4:13:30<02:51, 2.09s/it] 99%|█████████▉| 7419/7500 [4:13:32<02:49, 2.10s/it] 99%|█████████▉| 7420/7500 [4:13:34<02:45, 2.07s/it] {'loss': 0.3493, 'grad_norm': 3.525524616241455, 'learning_rate': 1.7327344598702667e-08, 'logqs/rejected': -2.5241951942443848, 'logqs/chosen': -0.8367463946342468, 'logqs_logps/rejected': -0.025621455162763596, 'logps_logqs/chosen': 0.2067752331495285, 'epoch': 1.98} 99%|█████████▉| 7420/7500 [4:13:34<02:45, 2.07s/it] 99%|█████████▉| 7421/7500 [4:13:36<02:43, 2.06s/it] 99%|█████████▉| 7422/7500 [4:13:38<02:44, 2.11s/it] 99%|█████████▉| 7423/7500 [4:13:41<02:44, 2.14s/it] 99%|█████████▉| 7424/7500 [4:13:43<02:40, 2.11s/it] 99%|█████████▉| 7425/7500 [4:13:45<02:37, 2.10s/it] 99%|█████████▉| 7426/7500 [4:13:47<02:35, 2.10s/it] 99%|█████████▉| 7427/7500 [4:13:49<02:34, 2.12s/it] 99%|█████████▉| 7428/7500 [4:13:51<02:33, 2.13s/it] 99%|█████████▉| 7429/7500 [4:13:53<02:31, 2.13s/it] 99%|█████████▉| 7430/7500 [4:13:55<02:28, 2.13s/it] {'loss': 0.35, 'grad_norm': 2.8617947101593018, 'learning_rate': 1.3266607432155243e-08, 'logqs/rejected': -2.2723476886749268, 'logqs/chosen': -0.8759431838989258, 'logqs_logps/rejected': -0.02565162442624569, 'logps_logqs/chosen': 0.20408952236175537, 'epoch': 1.98} 99%|█████████▉| 7430/7500 [4:13:55<02:28, 2.13s/it] 99%|█████████▉| 7431/7500 [4:13:57<02:24, 2.10s/it] 99%|█████████▉| 7432/7500 [4:13:59<02:22, 2.09s/it] 99%|█████████▉| 7433/7500 [4:14:02<02:20, 2.10s/it] 99%|█████████▉| 7434/7500 [4:14:04<02:18, 2.10s/it] 99%|█████████▉| 7435/7500 [4:14:06<02:17, 2.12s/it] 99%|█████████▉| 7436/7500 [4:14:08<02:15, 2.12s/it] 99%|█████████▉| 7437/7500 [4:14:10<02:11, 2.09s/it] 99%|█████████▉| 7438/7500 [4:14:12<02:07, 2.06s/it] 99%|█████████▉| 7439/7500 [4:14:14<02:05, 2.05s/it] 99%|█████████▉| 7440/7500 [4:14:16<02:03, 2.05s/it] {'loss': 0.3499, 'grad_norm': 2.8997457027435303, 'learning_rate': 9.747123991141194e-09, 'logqs/rejected': -2.529975652694702, 'logqs/chosen': -0.8333168029785156, 'logqs_logps/rejected': -0.02364589460194111, 'logps_logqs/chosen': 0.20274953544139862, 'epoch': 1.98} 99%|█████████▉| 7440/7500 [4:14:16<02:03, 2.05s/it] 99%|█████████▉| 7441/7500 [4:14:18<02:01, 2.06s/it] 99%|█████████▉| 7442/7500 [4:14:20<01:59, 2.05s/it] 99%|█████████▉| 7443/7500 [4:14:22<01:57, 2.05s/it] 99%|█████████▉| 7444/7500 [4:14:24<01:55, 2.07s/it] 99%|█████████▉| 7445/7500 [4:14:26<01:54, 2.08s/it] 99%|█████████▉| 7446/7500 [4:14:29<01:52, 2.08s/it] 99%|█████████▉| 7447/7500 [4:14:31<01:49, 2.06s/it] 99%|█████████▉| 7448/7500 [4:14:33<01:47, 2.06s/it] 99%|█████████▉| 7449/7500 [4:14:35<01:45, 2.06s/it] 99%|█████████▉| 7450/7500 [4:14:37<01:43, 2.06s/it] {'loss': 0.3559, 'grad_norm': 2.8883795738220215, 'learning_rate': 6.768970513457151e-09, 'logqs/rejected': -2.6389100551605225, 'logqs/chosen': -0.8540390133857727, 'logqs_logps/rejected': -0.021870864555239677, 'logps_logqs/chosen': 0.19650888442993164, 'epoch': 1.99} 99%|█████████▉| 7450/7500 [4:14:37<01:43, 2.06s/it] 99%|█████████▉| 7451/7500 [4:14:39<01:41, 2.07s/it] 99%|█████████▉| 7452/7500 [4:14:41<01:40, 2.09s/it] 99%|█████████▉| 7453/7500 [4:14:43<01:38, 2.09s/it] 99%|█████████▉| 7454/7500 [4:14:45<01:35, 2.08s/it] 99%|█████████▉| 7455/7500 [4:14:47<01:31, 2.03s/it] 99%|█████████▉| 7456/7500 [4:14:49<01:29, 2.03s/it] 99%|█████████▉| 7457/7500 [4:14:51<01:28, 2.06s/it] 99%|█████████▉| 7458/7500 [4:14:53<01:27, 2.09s/it] 99%|█████████▉| 7459/7500 [4:14:55<01:25, 2.09s/it] 99%|█████████▉| 7460/7500 [4:14:57<01:23, 2.09s/it] {'loss': 0.3722, 'grad_norm': 2.9061543941497803, 'learning_rate': 4.3322115108074265e-09, 'logqs/rejected': -2.6592540740966797, 'logqs/chosen': -0.8481138944625854, 'logqs_logps/rejected': -0.017236636951565742, 'logps_logqs/chosen': 0.20505103468894958, 'epoch': 1.99} 99%|█████████▉| 7460/7500 [4:14:58<01:23, 2.09s/it] 99%|█████████▉| 7461/7500 [4:15:00<01:21, 2.08s/it] 99%|█████████▉| 7462/7500 [4:15:02<01:19, 2.10s/it] 100%|█████████▉| 7463/7500 [4:15:04<01:17, 2.10s/it] 100%|█████████▉| 7464/7500 [4:15:06<01:14, 2.07s/it] 100%|█████████▉| 7465/7500 [4:15:08<01:12, 2.07s/it] 100%|█████████▉| 7466/7500 [4:15:10<01:10, 2.08s/it] 100%|█████████▉| 7467/7500 [4:15:12<01:08, 2.07s/it] 100%|█████████▉| 7468/7500 [4:15:14<01:05, 2.06s/it] 100%|█████████▉| 7469/7500 [4:15:16<01:04, 2.07s/it] 100%|█████████▉| 7470/7500 [4:15:18<01:01, 2.06s/it] {'loss': 0.3483, 'grad_norm': 3.1801071166992188, 'learning_rate': 2.4368997673940297e-09, 'logqs/rejected': -2.5059263706207275, 'logqs/chosen': -0.8699806332588196, 'logqs_logps/rejected': -0.02710362710058689, 'logps_logqs/chosen': 0.20756259560585022, 'epoch': 1.99} 100%|█████████▉| 7470/7500 [4:15:18<01:01, 2.06s/it] 100%|█████████▉| 7471/7500 [4:15:20<01:00, 2.08s/it] 100%|█████████▉| 7472/7500 [4:15:22<00:57, 2.07s/it] 100%|█████████▉| 7473/7500 [4:15:25<00:56, 2.09s/it] 100%|█████████▉| 7474/7500 [4:15:27<00:53, 2.07s/it] 100%|█████████▉| 7475/7500 [4:15:29<00:52, 2.10s/it] 100%|█████████▉| 7476/7500 [4:15:31<00:50, 2.08s/it] 100%|█████████▉| 7477/7500 [4:15:33<00:48, 2.09s/it] 100%|█████████▉| 7478/7500 [4:15:35<00:46, 2.10s/it] 100%|█████████▉| 7479/7500 [4:15:37<00:44, 2.14s/it] 100%|█████████▉| 7480/7500 [4:15:39<00:42, 2.11s/it] {'loss': 0.3354, 'grad_norm': 2.560873508453369, 'learning_rate': 1.0830763387897903e-09, 'logqs/rejected': -2.538719892501831, 'logqs/chosen': -0.843474268913269, 'logqs_logps/rejected': -0.02599402144551277, 'logps_logqs/chosen': 0.2066444605588913, 'epoch': 1.99} 100%|█████████▉| 7480/7500 [4:15:39<00:42, 2.11s/it] 100%|█████████▉| 7481/7500 [4:15:41<00:39, 2.10s/it] 100%|█████████▉| 7482/7500 [4:15:43<00:37, 2.10s/it] 100%|█████████▉| 7483/7500 [4:15:45<00:35, 2.09s/it] 100%|█████████▉| 7484/7500 [4:15:48<00:33, 2.08s/it] 100%|█████████▉| 7485/7500 [4:15:50<00:31, 2.07s/it] 100%|█████████▉| 7486/7500 [4:15:52<00:28, 2.03s/it] 100%|█████████▉| 7487/7500 [4:15:54<00:26, 2.04s/it] 100%|█████████▉| 7488/7500 [4:15:56<00:24, 2.06s/it] 100%|█████████▉| 7489/7500 [4:15:58<00:22, 2.03s/it] 100%|█████████▉| 7490/7500 [4:16:00<00:20, 2.06s/it] {'loss': 0.3512, 'grad_norm': 2.595893621444702, 'learning_rate': 2.7077055103075235e-10, 'logqs/rejected': -2.5270161628723145, 'logqs/chosen': -0.8422940969467163, 'logqs_logps/rejected': -0.023508399724960327, 'logps_logqs/chosen': 0.20261681079864502, 'epoch': 2.0} 100%|█████████▉| 7490/7500 [4:16:00<00:20, 2.06s/it] 100%|█████████▉| 7491/7500 [4:16:02<00:18, 2.07s/it] 100%|█████████▉| 7492/7500 [4:16:04<00:16, 2.07s/it] 100%|█████████▉| 7493/7500 [4:16:06<00:14, 2.05s/it] 100%|█████████▉| 7494/7500 [4:16:08<00:12, 2.07s/it] 100%|█████████▉| 7495/7500 [4:16:10<00:10, 2.06s/it] 100%|█████████▉| 7496/7500 [4:16:12<00:08, 2.07s/it] 100%|█████████▉| 7497/7500 [4:16:14<00:06, 2.09s/it] 100%|█████████▉| 7498/7500 [4:16:16<00:04, 2.06s/it] 100%|█████████▉| 7499/7500 [4:16:18<00:02, 2.06s/it] 100%|██████████| 7500/7500 [4:16:21<00:00, 2.09s/it] {'loss': 0.3491, 'grad_norm': 2.81526255607605, 'learning_rate': 0.0, 'logqs/rejected': -2.4051358699798584, 'logqs/chosen': -0.8657213449478149, 'logqs_logps/rejected': -0.023931410163640976, 'logps_logqs/chosen': 0.1996544897556305, 'epoch': 2.0} 100%|██████████| 7500/7500 [4:16:21<00:00, 2.09s/it][INFO|trainer.py:3503] 2025-12-21 17:25:09,384 >> Saving model checkpoint to ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500 [INFO|configuration_utils.py:472] 2025-12-21 17:25:09,388 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/config.json [INFO|configuration_utils.py:807] 2025-12-21 17:25:09,388 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/generation_config.json [INFO|modeling_utils.py:2755] 2025-12-21 17:25:11,287 >> Model weights saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/model.safetensors [INFO|tokenization_utils_base.py:2702] 2025-12-21 17:25:11,288 >> tokenizer config file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/tokenizer_config.json [INFO|tokenization_utils_base.py:2711] 2025-12-21 17:25:11,288 >> Special tokens file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/special_tokens_map.json [INFO|trainer.py:3503] 2025-12-21 17:25:11,517 >> Saving model checkpoint to ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500 [INFO|configuration_utils.py:472] 2025-12-21 17:25:11,520 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/config.json [INFO|configuration_utils.py:807] 2025-12-21 17:25:11,520 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/generation_config.json [INFO|modeling_utils.py:2755] 2025-12-21 17:25:13,775 >> Model weights saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/model.safetensors [INFO|tokenization_utils_base.py:2702] 2025-12-21 17:25:13,777 >> tokenizer config file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/tokenizer_config.json [INFO|tokenization_utils_base.py:2711] 2025-12-21 17:25:13,777 >> Special tokens file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/checkpoint-7500/special_tokens_map.json [INFO|trainer.py:2394] 2025-12-21 17:25:14,001 >> Training completed. Do not forget to share your model on huggingface.co/models =) {'train_runtime': 15385.6816, 'train_samples_per_second': 3.9, 'train_steps_per_second': 0.487, 'train_loss': 0.41974376156330107, 'epoch': 2.0} 100%|██████████| 7500/7500 [4:16:25<00:00, 2.09s/it] 100%|██████████| 7500/7500 [4:16:25<00:00, 2.05s/it] ***** train metrics ***** epoch = 2.0 total_flos = 0GF train_loss = 0.4197 train_runtime = 4:16:25.68 train_samples = 30000 train_samples_per_second = 3.9 train_steps_per_second = 0.487 2025-12-21 17:25:14 - INFO - __main__ - *** Training complete *** 2025-12-21 17:25:14 - INFO - __main__ - *** Save model *** [INFO|trainer.py:3503] 2025-12-21 17:25:14,039 >> Saving model checkpoint to ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2 [INFO|configuration_utils.py:472] 2025-12-21 17:25:14,043 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/config.json [INFO|configuration_utils.py:807] 2025-12-21 17:25:14,043 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/generation_config.json [INFO|modeling_utils.py:2755] 2025-12-21 17:25:15,959 >> Model weights saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/model.safetensors [INFO|tokenization_utils_base.py:2702] 2025-12-21 17:25:15,961 >> tokenizer config file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/tokenizer_config.json [INFO|tokenization_utils_base.py:2711] 2025-12-21 17:25:15,961 >> Special tokens file saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/special_tokens_map.json 2025-12-21 17:25:16 - INFO - __main__ - Model saved to ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2 [INFO|modelcard.py:449] 2025-12-21 17:25:16,419 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'distillm-2-master/data/reformatted/qwen_math', 'type': 'distillm-2-master/data/reformatted/qwen_math'}} [INFO|configuration_utils.py:472] 2025-12-21 17:25:16,421 >> Configuration saved in ./distillm-2-master/outputs/qwen2.5-math-span-distillm2_2/config.json 2025-12-21 17:25:16 - INFO - __main__ - *** Evaluate *** [INFO|trainer.py:3819] 2025-12-21 17:25:16,424 >> ***** Running Evaluation ***** [INFO|trainer.py:3821] 2025-12-21 17:25:16,424 >> Num examples = 10 [INFO|trainer.py:3824] 2025-12-21 17:25:16,424 >> Batch size = 2 0%| | 0/5 [00:00